Back to Star-Embed

MOIRAI-Small

ICML 2024
View Paper GitHub Repository Hugging Face Model

Model Overview

MOIRAI is a foundation model for time series forecasting that leverages transformer architecture. It can handle arbitrary number of covariates via its unique any-variate encoding and any-variate attention mechanism. MOIRAI is trained on probabilistic forecasting objective, with the target distribution being a mixture of Gaussian, T-student, log-normal distributions.

Key Features

  • Any-variate Encoding: Concatenate all covariates into a single univariate time series
  • Multi-variate Foundation Model: Pre-trained on large-scale time series data with various number of covariates
  • Zero-Shot Performance: Outperforms domain-specific baselines even in zero-shot settings

Embedding Clustering Visualization

UMAP visualization showing how Moirai Small embeddings cluster different types of astronomical objects

Moirai Small UMAP Visualization UMAP Legend

Each point represents a light curve embedding, with colors indicating different astronomical object types