Pith. sign in

REVIEW 23 cited by

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2409.16040 v4 pith:XGRNUMWN submitted 2024-09-24 cs.LG cs.AI

Xiaoming Shi , Shiyu Wang , Yuqi Nie , Dianqi Li , Zhou Ye , Qingsong Wen , Ming Jin This is my paper · ORCID
classification cs.LGcs.AI
keywords modelstimeforecastingseriestime-moefoundationmodeladvancements
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Deep learning for time series forecasting has seen significant advancements over the past decades. However, despite the success of large-scale pre-training in language and vision domains, pre-trained time series models remain limited in scale and operate at a high cost, hindering the development of larger capable forecasting models in real-world applications. In response, we introduce Time-MoE, a scalable and unified architecture designed to pre-train larger, more capable forecasting foundation models while reducing inference costs. By leveraging a sparse mixture-of-experts (MoE) design, Time-MoE enhances computational efficiency by activating only a subset of networks for each prediction, reducing computational load while maintaining high model capacity. This allows Time-MoE to scale effectively without a corresponding increase in inference costs. Time-MoE comprises a family of decoder-only transformer models that operate in an auto-regressive manner and support flexible forecasting horizons with varying input context lengths. We pre-trained these models on our newly introduced large-scale data Time-300B, which spans over 9 domains and encompassing over 300 billion time points. For the first time, we scaled a time series foundation model up to 2.4 billion parameters, achieving significantly improved forecasting precision. Our results validate the applicability of scaling laws for training tokens and model size in the context of time series forecasting. Compared to dense models with the same number of activated parameters or equivalent computation budgets, our models consistently outperform them by large margin. These advancements position Time-MoE as a state-of-the-art solution for tackling real-world time series forecasting challenges with superior capability, efficiency, and flexibility.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 23 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Crossing-Free Probabilistic K-Line Forecasts Without Retraining

    stat.ML 2026-07 conditional novelty 6.0 of 10

    KQSP eliminates quantile and K-line crossings in probabilistic OHLC forecasts via sequential minimum-distance projections, without retraining and with smaller corrections than standard alternatives.

  2. Learning Spatio-Temporal Foundation Models from Pure Synthetic Data

    cs.LG 2026-06 conditional novelty 6.0 of 10

    A spatio-temporal foundation model pre-trained exclusively on synthetic stochastic graph dynamics outperforms real-data-pretrained STFMs in zero-shot traffic forecasting, according to the paper's benchmarks.

  3. Uncertainty-Guided Label Rebalancing for CPS Safety Monitoring

    cs.LG 2026-03 conditional novelty 6.0 of 10

    Uncertainty-guided label rebalancing (uLNR) lifts UAV safety-prediction F1 to 0.806 under 46:1 imbalance by probabilistically flipping high-uncertainty safe windows to unsafe.

  4. Dual-Prototype Disentanglement: A Context-Aware Enhancement Framework for Time Series Forecasting

    cs.LG 2026-01 conditional novelty 6.0 of 10

    A model-agnostic module that retrieves common and rare prototype patterns improves forecasting error on many standard benchmarks, but not on all reported cases.

  5. WindFM: An Open-Source Foundation Model for Zero-Shot Wind Power Forecasting

    cs.LG 2025-09 conditional novelty 6.0 of 10

    A wind-specific foundation model, WindFM, uses hierarchical tokenization and autoregressive pre-training on the NREL WIND Toolkit to achieve state-of-the-art zero-shot wind power forecasts.

  6. LightGTS: A Lightweight General Time Series Forecasting Model

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A lightweight time series foundation model using period-aligned patches and parallel decoding reports zero-shot and full-shot accuracy on nine benchmarks comparable to much larger models.

  7. Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction

    cs.AI 2025-05 conditional novelty 6.0 of 10

    NextLocMoE combines a location-semantics expert mixer and a persona-based expert mixer inside a trimmed LLaMA backbone to set new accuracy marks on next-location prediction benchmarks.

  8. BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A balanced sampling strategy over statistically characterized time series patterns lets universal forecasting models train on 78 billion tokens instead of 419 billion, with equal or better zero-shot accuracy.

  9. Harnessing Vision Models for Time Series Analysis: A Survey

    cs.LG 2025-02 conditional novelty 6.0 of 10

    A survey organizing existing methods that encode time series as images and apply vision models, with a dual-view taxonomy of imaging and modeling approaches.

  10. TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning

    cs.CL 2025-10 conditional novelty 5.0 of 10

    A frozen time-series encoder aligned to an LLM through a small adapter and two-stage training beats same-scale open-source models on time-series reasoning benchmarks.

  11. FinCast: A Foundation Model for Financial Time-Series Forecasting

    cs.LG 2025-08 conditional novelty 5.0 of 10

    FinCast, a 1B-parameter sparse-MoE transformer pretrained on 20B+ financial time points, reports 20% and 23% average MSE reductions over SOTA in zero-shot and supervised financial forecasting.

  12. Benchmarking Pre-Trained Time Series Models for Electricity Price Forecasting

    cs.LG 2025-06 conditional novelty 5.0 of 10

    No time series foundation model statistically outperforms the biseasonal MSTL model in most European day-ahead electricity price markets in 2024, though Chronos-Bolt and Time-MoE match traditional methods.

  13. dots.llm1 Technical Report

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A 14B-active MoE model roughly matches Qwen2.5-72B on a broad benchmark suite while reporting about a 4x reduction in training GPU-hours.

  14. EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Media

    cs.CV 2025-05 conditional novelty 5.0 of 10

    EvidenceMoE estimates depth and fluorescence lifetime from simulated fluorescence LiDAR signals using physics-segmented experts, evidential critics, and a learned fusion gate, reporting NRMSE 0.030 and 0.074.

  15. N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting

    cs.LG 2025-08 conditional novelty 4.0 of 10

    Adding a gating network on top of N-BEATS block outputs gives modest SMAPE improvements on some heterogeneous benchmark series, but the gains are small and not statistically validated.

  16. Sequence Aware SAC Control for Engine Fuel Consumption Optimization in Electrified Powertrain

    eess.SY 2025-08 unverdicted novelty 4.0 of 10

    The abstract promises RL-based hybrid-engine fuel optimization, but the manuscript body describes PriceFM, an electricity price forecasting model, and the two do not match.

  17. Foundation Models for Demand Forecasting via Dual-Strategy Ensembling

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A dual ensemble of hierarchical partitions and diverse backbones improves foundation-model sales forecasts on M5 and three external datasets, though the zero-shot protocol is under-specified.

  18. MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models

    cs.LG 2025-07 conditional novelty 4.0 of 10

    MoFE-Time reports average MSE 0.2755 and MAE 0.3226 across six public benchmarks, about 7% lower than Time-MoE, by adding frequency-domain experts to a Mixture of Experts transformer.

  19. Towards Accurate and Efficient 3D Object Detection for Autonomous Driving: A Mixture of Experts Computing System on Edge

    cs.CV 2025-07 reject novelty 4.0 of 10

    EMC2 routes each driving scene to one of three expert 3D detectors using distance and confidence thresholds, reporting higher accuracy and 2.6x faster inference on Jetson, though thresholds are tuned on the validation set.

  20. Foundation Model Empowered Synesthesia of Machines (SoM): AI-native Intelligent Multi-Modal Sensing-Communication Integration

    eess.SP 2025-06 conditional novelty 4.0 of 10

    The paper proposes a systematic classification and two roadmaps for using foundation models (LLMs and wireless foundation models) to design Synesthesia of Machines systems for 6G, with preliminary case-study evidence ...

  21. Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities

    cs.CL 2025-05 conditional novelty 4.0 of 10

    Activation-frequency analysis identifies sparse units in LLMs that respond to instructions; same-category instructions share more of these units than different-category ones, and fine-tuning measurably changes the sets.

  22. Foundation Models for Clean Energy Forecasting: A Comprehensive Review

    eess.SY 2025-07 conditional novelty 3.0 of 10

    A survey of foundation model methods, data, and open problems for renewable energy forecasting, built from roughly 218 cited works.

  23. Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?

    cs.LG 2025-06 reject novelty 3.0 of 10

    LLM4TS_FS achieves the best MSE on four of seven long-term datasets, but the claimed broad advantage of pre-trained large models over small transformers is not consistent across all benchmarks.

Pith tools