Model Overview
Astromer-2 is an enhanced version of the transformer-based model specifically designed for astronomical time series analysis. It is pretrained in a self-supervised manner with novel Uncertainty-weighted loss. Instead of generating embeddings using the final encoder layer, it uses weighted aggregation of intermediate attention block outputs. Astromer-2 shows superior performance over Astromer-1 in downstream tasks.
Key Improvements
- Novel Pretraining: Pretrained on time series reconstruction task with uncertainty-weighted loss
- Irregular Sample Modeling: Uses adapted positional encoder to capture the observation time information on irregular sampling
- Layer-mixing Embeddings: Helps the model flexibly incorporate features at different abstraction levels
