Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
read the original abstract
Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data.
This paper has not been read by Pith yet.
Forward citations
Cited by 19 Pith papers
-
FactoryNet: A Large-Scale Dataset toward Industrial Time-Series Foundation Models
FactoryNet is the first universal pretraining corpus for industrial time-series data with a shared S-E-F-C schema that supports cross-embodiment transfer and competitive anomaly detection.
-
UC-Search: Risk-Aware Test-Time Search for Delayed Constrained Time-Series Control
UC-Search is a model-agnostic test-time wrapper that adds feasibility-automaton search and uncertainty-based risk adjustment to produce better delayed constrained control than CEM, MPPI, and risk-random baselines on p...
-
GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoring
GlucoFM decomposes CGM traces into dual state-event streams, pretrains on 109k hours of unlabeled data, and reports superior subject-disjoint performance on seven clinical tasks across four cohorts.
-
SurF: A Generative Model for Multivariate Irregular Time Series Forecasting
SurF applies the Time Rescaling Theorem as a learnable bijection to create a single generative model for forecasting irregular multivariate event streams that outperforms or matches baselines on six benchmarks.
-
TimeClaw: A Time-Series AI Agent with Exploratory Execution Learning
TimeClaw is an exploratory execution learning system that turns multiple valid tool-use paths into hierarchical distilled experience for improved time-series reasoning without test-time adaptation.
-
FactoryNet: A Large-Scale Dataset toward Industrial Time-Series Foundation Models
FactoryNet is a 51M-point industrial time-series dataset with an S-E-F-C schema that supports zero-shot cross-embodiment transfer and competitive anomaly detection across robotic and machining tasks.
-
Explainable Load Forecasting with Covariate-Informed Time Series Foundation Models
Time series foundation models match the performance of specialized models for day-ahead load forecasting while providing explanations that match domain knowledge on weather and calendar effects.
-
TempusBench: An Evaluation Framework for Time-Series Forecasting
TempusBench is a new evaluation framework for time-series forecasting models that supplies fresh non-overlapping datasets, tasks beyond horizon and domain, consistent tuning across models, and visualization tools.
-
Sundial: A Family of Highly Capable Time Series Foundation Models
Sundial uses TimeFlow Loss for native pre-training of Transformers on continuous time series from TimeBench, achieving SOTA point and probabilistic forecasting with millisecond inference.
-
Deep Time Series Models: A Comprehensive Survey and Benchmark
This survey and benchmark of deep time series models using the released TSLib library finds that models with specific structures perform well only on distinct analysis tasks.
-
Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answering
CADE framework uses direct timestep embedding and supervised contrastive alignment to improve time-series question answering, reporting gains on six tasks in the Time-MQA benchmark over LLM baselines.
-
TimeRouter: Efficient and Adaptive Routing of Time-Series Foundation Models
TimeRouter routes among time-series foundation models via discriminative routing, selective gating and ensemble fallback, reporting SOTA LB MASE 0.6765 on GIFT-EVAL.
-
Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning
CondI applies conditional diffusion models in a two-phase federated pipeline to impute within-modality missing data, then trains extractors on the completed inputs for downstream tasks on clinical datasets.
-
Zeus: Towards Tuning-Free Foundation Model for Time Series Analysis
Zeus proposes a multi-scale Transformer with point-wise tokenization and Multi-Objective Temporal Masking to enable tuning-free performance on forecasting, interpolation, and other time series tasks.
-
Benchmarking Deep Time Series Models for Equity Portfolios
Benchmark of 15 time-series architectures on equity portfolios finds no model dominates, with TransEnc-8 at 0.352 rank-1 acceptability and all promoted models showing negative net Sharpe at 20 bps costs under constraints.
-
PaP-NF: Probabilistic Long-Term Time Series Forecasting via Prefix-as-Prompt Reprogramming and Normalizing Flows
PaP-NF uses prefix-as-prompt reprogramming of a frozen LLM to extract global context that conditions a normalizing flow decoder, producing probabilistic long-term time series forecasts evaluated by CRPS.
-
Heterogeneous Scientific Foundation Model Collaboration
Eywa enables language-based agentic AI systems to collaborate with specialized scientific foundation models for improved performance on structured data tasks.
-
Thermal-GEMs: Generalized Models for Building Thermal Dynamics
Multi-source transfer learning for building thermal dynamics yields up to 63% lower forecasting errors than single-source models and outperforms time series foundation models when pretrained on 16-32 buildings over one year.
-
Time Series Analysis in Machine Learning
A review chapter covering basic time series concepts, classical models like ARIMA, and ML approaches including neural networks and transformers.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.