Pith. sign in

REVIEW 1 major objections 1 minor 53 references

Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection

T0 review · 1 major / 1 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-stage modular pipeline matches top forecasters with fewer parameters.

desk verdict A competent modular forecasting architecture with a genuinely useful component ablation, but the SOTA claim is undercut by an asymmetric benchmark protocol. read the letter →

arxiv 2507.05891 v1 pith:RGEAKUOK submitted 2025-07-08 cs.AI

Robert Leppich , Michael Stenger , André Bauer , Samuel Kounev This is my paper
classification cs.AI
keywords timeseriesforecastingmodulararchitecturepatch-basedrepresentationmemorymoduleLSTMprojectionself-attentionablationlong-termbenchmarkevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that time series forecasting does not need a single clever architecture: it can be decomposed into three interchangeable stages—representing the input sequence, extracting and storing information in a memory, and projecting that memory onto the forecast horizon—and that a model built this way, with each stage configured per task, matches or beats published state-of-the-art results on seven long-term forecasting benchmarks while using fewer parameters and less compute. The proposed architecture, REP-Net, combines multiple patch extractors operating at different abstraction levels, time-informed patches that carry calendar context, an MLP-style memory module with optional sparse self-attention and gated linear units, and an LSTM-plus-linear projection. The reason to care is that the paper frames architecture selection as task-specific engineering rather than a winner-take-all model family, and its ablations quantify which components earn their cost on which datasets and horizons.

What carries the argument

The carrying mechanism is the time-informed multi-patch representation feeding a memory-enrichment module and a per-patch projection. The input is cut into K overlapping subsequences by patch extractors with different cover sizes, dilations, and strides, so fine-grained local patterns and coarse trends are available simultaneously; each patch is embedded (with linear, GLU-gated linear, or CNN-based options) and concatenated with an embedding of its temporal context such as hour or day of week. Those time-informed patches are concatenated and passed through N stacked memory blocks built on linear time/feature mixing, optionally with sparse self-attention and a GLU gate, and then split back into K streams, each processed by R LSTM layers followed by a linear layer, whose outputs are summed into the forecast. The argument works by treating each dataset-horizon pair as a distinct task and searching the configuration space per task.

What would settle it

Re-running the compared baselines under the identical protocol used for REP-Net—same lookback of 96, same random-search configuration space, same early-stopping rule, and the same Traffic feature-840 exclusion—and finding that any baseline achieves lower average MSE/MAE than reported for REP-Net would refute the paper's state-of-the-art claim; equally, reporting REP-Net's Traffic scores with feature 840 included (average MSE 0.437 per Table 1) as the primary result would already change the comparison.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a deliberately modular network—REP-Net—can match or beat the compared state of the art on seven long-term multivariate forecasting benchmarks (ECL, four ETT subsets, Weather, Traffic) when each of its three stages is tuned to the dataset and prediction horizon, and that it does so with lower inference time and memory footprint than the efficiency-oriented baselines and a parameter count in the middle of the pack. The accuracy advantage is clearest at the longest horizon H=720, while at shorter horizons the paper says predictive capacity is approaching saturation and scores converge. On Traffic, the paper reports that REP-Net did not beat the state of the art on the full feature set, but that after excluding feature 840—which contains significant test-set outliers—its average MSE drops to 0.372 and MAE to 0.279. The component analysis across 28 tasks concludes that gated linear units and time-informed patches help most tasks, multiple patch extractors usually beat a single one, self-attention rarely pays for its cost, and LSTM layers help only on a few datasets.

Load-bearing premise

The claim that REP-Net meets or beats state-of-the-art accuracy rests on the comparability of its numbers—obtained with per-task random hyperparameter search and, on Traffic, with one test feature removed because of outliers—to published baseline numbers that did not receive the same tuning or exclusion treatment.

Editorial extensions

If this is right

  • Per-task configuration matters more than a single architecture: no configuration wins all 28 dataset-horizon tasks, consistent with the paper's invocation of a no-free-lunch view.
  • Self-attention is dispensable in this pipeline: in 21 of 28 tasks, attention-free models performed at least as well, with six significant wins for no-attention and only one significant win for attention, so removing it improves efficiency without hurting accuracy.
  • Time-informed patches and GLU gating are broadly useful: time-informed patches beat non-informed ones in 22 of 28 tasks (nine significant), and GLU improved 23 of 28 tasks (19 significant).
  • Multiple patch extractors at different abstraction levels usually help, beating single-patch representations in 23 of 28 tasks.
  • A memory module earns its place: a single memory block beats none in 19 of 28 tasks (13 significant), while stacking multiple blocks gives mixed, mostly insignificant results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication the paper leaves implicit: if the per-task configuration search is the real source of the gains, then a strong baseline run through the same search procedure and the same fixed lookback could close most of the gap; the paper does not provide that experiment.
  • The Traffic feature-840 exclusion suggests a concrete robustness check: re-evaluate REP-Net and the compared baselines with that feature included and with a pre-registered outlier rule, since the paper's headline average changes materially depending on the exclusion.
  • The patterns found here could transfer to short-horizon forecasting or to forecasting with variable lookback windows, but the paper only studies T=96 with horizons 96–720, so extending the modular comparison in that direction is open.
  • Because time-informed patches help most on Traffic, a reasonable hypothesis is that calendar context matters more on datasets with strong weekly seasonality; comparing per-dataset gains against measured seasonal strength would test that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper proposes REP-Net, a modular time series forecasting architecture that decomposes the pipeline into Representation, Memory, and Projection stages. Each stage can take multiple configurations (patch extractors, embeddings, attention, GLU, LSTM layers, etc.), and the authors perform a per-task random search over these configurations on seven benchmarks (ECL, ETTm1/2, ETTh1/2, Weather, Traffic) with horizons 96-720. The paper claims state-of-the-art forecasting accuracy and lower computational cost, and it reports a detailed ablation analysis with paired significance tests. The main evidence is Tables 1 and 5 (accuracy) and Table 2 (efficiency). The manuscript also includes a Limitations section that acknowledges some of these issues.

Significance. If the claims were fully supported, the work would be a useful contribution to the empirical time series forecasting literature: it provides a flexible, modular architecture, a systematic per-component analysis, and public code. The architecture is not radically novel but the systematic comparison of components (time-informed patches, multi-scale patch extraction, GLU, LSTM projection) across tasks is informative. The honest reporting of ablations with statistical significance tests (Appendix E, Figures 2-9) and the public codebase are strengths. However, the headline 'state-of-the-art' claim is currently not robustly established because the comparison protocol is asymmetric: REP-Net benefits from per-task tuning and a test-set-driven feature exclusion, while baseline numbers are imported from prior publications without the same treatment. The significance of the paper therefore hinges on whether the evaluation can be made fair and transparent.

major comments (1)
  1. [Tables 1 and 5] Table 1 (averaged over horizons) and Table 5 (per horizon) are not entirely consistent in presentation. For example, in Table 5, the ETTm2 720 row shows TSRM with MSE 0.276, which is better than REP-Net's 0.377, yet Table 1's average for ETTm2 lists TSRM at 0.245 and REP-Net at 0.264. The averaging is correct, but the per-horizon table reveals that REP-Net is not uniformly better; this is fine, but the reader should be told that the 'best' markers in Table 1 are based on averages with a 1% margin, not on per-horizon wins, to avoid overinterpretation.
minor comments (1)
  1. [Appendix B] The Traffic dataset description says 'culminating in 17,544 hourly records per sensor' but likely means the total number of time steps per sensor is 17,544; the phrasing is ambiguous. Minor rewording would help.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the SOTA claim is an empirical benchmark result against external baselines; the single self-citation (TSRM) is not load-bearing, and the disclosed Traffic feature-exclusion is a protocol caveat, not a circular derivation.

full rationale

This paper is an empirical study, so there is no derivation chain in which an output reduces to an input by construction; the central claim of SOTA accuracy (abstract; Section 3.3) is a held-out test-set comparison in Tables 1 and 5 against externally published baselines (CycleNet, TimeMixer, TimeMachine, iTransformer, PatchTST, PathFormer, TimeCNN, RLinear, DLinear), which gives the claim independent empirical content. The only self-citation is [34] (TSRM, same authors), used in Section 2.2 as a design precedent ('following a strategy similar to that proposed by Leppich et al.') and as one of nine compared baselines; the SOTA claim does not depend on the cited work's validity, so the self-citation is minor and non-load-bearing. The potentially suspicious step is the Traffic analysis in Section 3.3, where the paper first concedes 'our approach did not outperform SOTA methods on the Traffic dataset' and then reports an improved MSE of 0.372/MAE 0.279 after excluding feature 840, which 'contained significant outliers in the test set'; this truncated number is not placed in the comparison table, and Section 5 ('Impact of outliers') restates the caveat. Excluding a test-set feature post hoc without recomputing the baselines on the reduced feature set, and selecting REP-Net configurations by per-task random search while importing baseline numbers from their publications, are protocol-symmetry and selection-bias risks that belong in a correctness review, not circularity: the reported errors are measured on held-out test windows rather than being recovered from the fitted configuration by construction. The paper also honestly discloses internal inconsistencies (Section 3.3: 'In terms of trainable parameters, REP-Net ranks third among the evaluated models,' against the abstract's 'lower parameter count'; Section 5 'Complexity and runtime'), which further weighs against any attempt to hide a forced result. No circular step meeting the evidentiary bar was found.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The results depend on a large set of hand-tuned hyperparameters per task that are not fully disclosed, on the standard benchmark protocol, and on the implicit reliability of the PyTorch building blocks. No new physical or conceptual entities are introduced.

free parameters (7)
  • Huber loss threshold delta = not reported
    Section 3.2 states delta depends on the dataset, but specific values are never given; this affects training for all reported results.
  • Number of patch extractors K = task-specific, 1 to 5
    K is selected per task via random search; Table 3 shows multi-patch configurations generally help.
  • Patch extractor configurations (cover size, dilation, stride) = not reported per task
    Each extractor uses a unique combination chosen from listed ranges; the exact final values are absent from the paper.
  • Feature and temporal embedding sizes = ranges: ef 4..128, et 8..16
    Embedding sizes are free choices tuned per task and are not reported as final settings.
  • Number of memory modules N = task-specific, 0 to 10
    Stack depth is tuned per task; ablations show inconsistent effects across tasks.
  • Number of LSTM projection layers R = task-specific, 0 to 3
    R is tuned per task; only two datasets clearly benefit from LSTMs.
  • Dropout rate = ranges from 0.25 to 0.9
    Dropout is tuned per task; final values are not reported.
assumptions (4)
  • domain assumption The standard train/validation/test splits of Autoformer are used for all datasets.
    Section 3.1 and Appendix B state the splits follow [13]; comparability across models rests on this assumption.
  • domain assumption Minimizing Huber/MSE loss on benchmark test sets is a valid measure of forecasting quality.
    All comparisons use MSE/MAE; no uncertainty quantification or application-specific evaluation is provided.
  • standard math The deep learning components (convolution, attention, LSTM, GLU) are implemented correctly and behave as standard.
    The paper relies on standard PyTorch implementations without formal verification.
  • domain assumption Temporal features such as hour and day-of-week are available and informative for all datasets.
    Time-informed patching is a core contribution; its value presupposes these covariates exist and help forecasting, which is plausible for ETT and Weather but less clear for Traffic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection." pith.science (2026). https://pith.science/paper/RGEAKUOK

@misc{pith2026250705891,
  author       = {Pith},
  title        = {Pith review of: Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGEAKUOK}},
  note         = {Machine review of arXiv:2507.05891}
}
read the original abstract

With the advent of Transformers, time series forecasting has seen significant advances, yet it remains challenging due to the need for effective sequence representation, memory construction, and accurate target projection. Time series forecasting remains a challenging task, demanding effective sequence representation, meaningful information extraction, and precise future projection. Each dataset and forecasting configuration constitutes a distinct task, each posing unique challenges the model must overcome to produce accurate predictions. To systematically address these task-specific difficulties, this work decomposes the time series forecasting pipeline into three core stages: input sequence representation, information extraction and memory construction, and final target projection. Within each stage, we investigate a range of architectural configurations to assess the effectiveness of various modules, such as convolutional layers for feature extraction and self-attention mechanisms for information extraction, across diverse forecasting tasks, including evaluations on seven benchmark datasets. Our models achieve state-of-the-art forecasting accuracy while greatly enhancing computational efficiency, with reduced training and inference times and a lower parameter count. The source code is available at https://github.com/RobertLeppich/REP-Net.

Figures

Figures reproduced from arXiv: 2507.05891 by the authors.

Figure 1
Figure 1. Illustraton of the proposed REP-Net Architecture, consisting of three modules: Representation (gray), Memory [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Comparison of models with attention and without. Each row represents one of the four prediction lengths and [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Comparison of models with time-informed patches and non-informed patches. Each row represents one of the [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison of models with LSTM and without LSTM. Each row represents one of the four prediction [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Comparison of models with GLU and without GLU. Each row represents one of the four prediction [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Comparison of models with a single patch extractor and multiple patch extractors. Each row represents [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Comparison of models with a single memory module and no memory module. Each row represents one of [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Comparison of models with a single memory module and multiple memory modules. Each row represents [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Comparison of models with CNN and without CNN. Each row represents one of the four prediction lengths [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 31 canonical work pages

  1. [1]

    Deep time series forecasting models: A comprehensive survey

    Xinhe Liu and Wenmin Wang. Deep time series forecasting models: A comprehensive survey. Mathematics, 12 (10):1504, 2024

    work page 2024

  2. [2]

    Deep learning for time series classification and extrinsic regression: A current survey

    Navid Mohammadi Foumani, Lynn Miller, Chang Wei Tan, Geoffrey I Webb, Germain Forestier, and Mahsa Salehi. Deep learning for time series classification and extrinsic regression: A current survey. ACM Computing Surveys, 56(9):1–45, 2024

    work page 2024

  3. [3]

    Review of data imputation techniques in time series data: Comparative analysis

    Yousef Jaradat, Mohammad Masoud, Ahmad Manasrah, Mohammad Alia, and Ismael Jannoud. Review of data imputation techniques in time series data: Comparative analysis. The Eurasia Proceedings of Science Technology Engineering and Mathematics, 27:122–129, 2024

    work page 2024

  4. [4]

    Optimal multi-scale patterns in time series streams

    Spiros Papadimitriou and Philip Yu. Optimal multi-scale patterns in time series streams. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 647–658, 2006

    work page 2006

  5. [5]

    Statstream: Statistical monitoring of thousands of data streams in real time

    Yunyue Zhu and Dennis Shasha. Statstream: Statistical monitoring of thousands of data streams in real time. In VLDB’02: Proceedings of the 28th International Conference on Very Large Databases, pages 358–369. Elsevier, 2002

    work page 2002

  6. [6]

    Transformer-based models to deal with heterogeneous environments in human activity recognition

    Sannara Ek, François Portet, and Philippe Lalanda. Transformer-based models to deal with heterogeneous environments in human activity recognition. Personal and Ubiquitous Computing, 27(6):2267–2280, 2023

    work page 2023

  7. [7]

    Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, 2001

    Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, 2001

    work page 2001

  8. [8]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017

    2017

Show all 53 references
  1. [9]

    Deep transformer models for time series forecasting: The influenza prevalence case

    Neo Wu, Bradley Green, Xue Ben, and Shawn O’Banion. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317, 2020

    2001 arXiv

  2. [10]

    An improved relative self-attention mechanism for transformer with application to music generation

    Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Curtis Hawthorne, Andrew M Dai, Matthew D Hoffman, and Douglas Eck. An improved relative self-attention mechanism for transformer with application to music generation. arXiv preprint arXiv:1809.04281, 2, 2018

    2018 arXiv

  3. [11]

    A time-restricted self-attention layer for asr

    Daniel Povey, Hossein Hadian, Pegah Ghahremani, Ke Li, and Sanjeev Khudanpur. A time-restricted self-attention layer for asr. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 5874–5878. IEEE, 2018

    2018

  4. [12]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021

    2021

  5. [13]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems , 34: 22419–22430, 2021

    2021

  6. [14]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pages 27268–27286. PMLR, 2022

    2022

  7. [15]

    Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting

    Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations, 2023

    2023

  8. [16]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  9. [17]

    itransformer: Inverted transformers are effective for time series forecasting

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2024

    2024 arXiv

  10. [18]

    Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting

    Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. In International Conference on Learning Representations, 2024

    2024

  11. [19]

    Timesnet: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022

    2022 arXiv

  12. [20]

    Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023. 10 RepNet PREPRINT

    2023

  13. [21]

    Revisiting long-term time series forecasting: An investigation on linear mapping

    Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721, 2023

    2023 arXiv

  14. [22]

    TSMixer: An all-MLP architecture for time series forecast-ing

    Si-An Chen, Chun-Liang Li, Sercan O Arik, Nathanael Christian Yoder, and Tomas Pfister. TSMixer: An all-MLP architecture for time series forecast-ing. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=wbpxTuXgm0

    2023

  15. [23]

    Cyclenet: Enhancing time series forecasting through modeling periodic patterns

    Shengsheng Lin, Weiwei Lin, Xinyi Hu, Wentai Wu, Ruichao Mo, and Haocheng Zhong. Cyclenet: Enhancing time series forecasting through modeling periodic patterns. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural In...

    2024

  16. [24]

    Timecnn: Refining cross-variable interaction on time point for time series forecasting

    Ao Hu, Dongkai Wang, Yong Dai, Shiyi Qi, Liangjian Wen, Jun Wang, Zhi Chen, Xun Zhou, Zenglin Xu, and Jiang Duan. Timecnn: Refining cross-variable interaction on time point for time series forecasting. arXiv preprint,

  17. [25]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021

    2021 arXiv

  18. [26]

    Timemachine: A time series is worth 4 mambas for long-term forecasting

    Md Atik Ahamed and Qiang Cheng. Timemachine: A time series is worth 4 mambas for long-term forecasting. In 27th European Conference on Artificial Intelligence, volume 392, pages 1688–1695, 2024

    2024

  19. [27]

    Self-supervised contrastive pre-training for time series via time-frequency consistency.Advances in Neural Information Processing Systems, 35:3988–4003, 2022

    Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised contrastive pre-training for time series via time-frequency consistency.Advances in Neural Information Processing Systems, 35:3988–4003, 2022

    2022

  20. [28]

    One fits all: Power general time series analysis by pretrained lm

    Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems, 36:43322–43355, 2023

    2023

  21. [29]

    Lag-llama: Towards foundation models for time series forecasting

    Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhag- watkar, Marin Biloš, Hena Ghonia, Nadhir Vincent Hassen, Anderson Schneider, et al. Lag-llama: Towards foundation models for time series forecasting. arXiv preprint arXiv:2310...

    arXiv

  22. [30]

    MOMENT: A family of open time-series foundation models

    Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MOMENT: A family of open time-series foundation models. In Forty-first International Conference on Machine Learning, 2024

    2024

  23. [31]

    Unified training of universal time series forecasting transformers

    Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Forty-first International Conference on Machine Learning, 2024

    2024

  24. [32]

    A decoder-only foundation model for time-series forecasting

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310.10688, 2024

    2024 arXiv

  25. [33]

    Addressing prediction delays in time series forecasting: A continuous gru approach with derivative regularization

    Sheo Yon Jhin, Seojin Kim, and Noseong Park. Addressing prediction delays in time series forecasting: A continuous gru approach with derivative regularization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1234–1245, 2024

    2024

  26. [34]

    Tsrm: A lightweight temporal feature encoding architecture for time series forecasting and imputation.arXiv preprint arXiv:2504.18878, 2025

    Robert Leppich, Michael Stenger, Daniel Grillmeyer, Vanessa Borst, and Samuel Kounev. Tsrm: A lightweight temporal feature encoding architecture for time series forecasting and imputation.arXiv preprint arXiv:2504.18878, 2025

    2025 arXiv

  27. [35]

    Mlp-mixer: An all-mlp architecture for vision

    Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:242...

    2021

  28. [36]

    Adversarial sparse transformer for time series forecasting

    Sifan Wu, Xi Xiao, Qianggang Ding, Peilin Zhao, Ying Wei, and Junzhou Huang. Adversarial sparse transformer for time series forecasting. Advances in Neural Information Processing Systems, 33, 2020

    2020

  29. [37]

    Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504, 2022

    Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504, 2022

    2022 arXiv

  30. [38]

    Uci machine learning repository

    Dheeru Dua, Casey Graff, et al. Uci machine learning repository. 2017

    2017

  31. [39]

    http://pems.dot.ca.gov

    Calefornia department of transportation. http://pems.dot.ca.gov. [Online; accessed 1. Feb 2024]

    2024

  32. [40]

    Zhang, and JUN ZHOU

    Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y . Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/for...

    2024

  33. [41]

    Timemachine: A time series is worth 4 mambas for long-term forecasting

    Md Atik Ahamed and Qiang Cheng. Timemachine: A time series is worth 4 mambas for long-term forecasting. arXiv preprint arXiv:2403.09898, 2024. 11 RepNet PREPRINT

    2024 arXiv

  34. [42]

    No free lunch theorems for optimization

    David H Wolpert and William G Macready. No free lunch theorems for optimization. IEEE transactions on evolutionary computation, 1(1):67–82, 1997

    1997

  35. [43]

    https://cbergmeir.com/talks/neurips2024/

    Fundamental limitations of foundational forecasting models: The need for multimodality and rigorous evaluation. https://cbergmeir.com/talks/neurips2024/. [Online; accessed 01. Feb 2025]

    2025

  36. [44]

    Transferability in deep learning: A survey

    Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05867, 2022

    2022 arXiv

  37. [45]

    Simmtm: A simple pre- training framework for masked time-series modeling

    Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre- training framework for masked time-series modeling. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processi...

    2023

  38. [46]

    Himtm: Hierarchical multi-scale masked time series modeling for long-term forecasting

    Shubao Zhao, Ming Jin, Zhaoxiang Hou, Chengyi Yang, Zengxiang Li, Qingsong Wen, and Yi Wang. Himtm: Hierarchical multi-scale masked time series modeling for long-term forecasting. arXiv preprint arXiv:2401.05012, 2024

    2024 arXiv

  39. [47]

    CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting

    Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. In International Conference on Learning Representations, 2022

    2022

  40. [48]

    Learning to embed time series patches independently

    Seunghan Lee, Taeyoung Park, and Kibok Lee. Learning to embed time series patches independently. In The International Conference on Learning Representations (ICLR), 2024

    2024

  41. [49]

    On the opportunities and risks of foundation models

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  42. [50]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  43. [51]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In The International Conference on Learning Representations (ICLR), 2022

    2022

  44. [52]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=tEYskw1VY2. 12 RepNet PREPRINT A Related Work Transformer-based models. Since its inception in 2017, Tra...

    2024

  45. [2024]

    URL https://arxiv.org/abs/2410.04853

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.