Pith. sign in

REVIEW 5 major objections 5 minor 48 references

Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that pretrained large-scale time series models, particularly LLM4TS_FS and Timer_XL, achieve lower forecast error than small task-specific transformers across the tested long- and short-term benchmarks.

desk verdict Useful benchmark tables, but the central claim that pretrained large models significantly outperform small transformers is not supported by the paper's own numbers, and the five 'Theorems' are empirical restatements in disguise. read the letter →

arxiv 2507.02907 v1 pith:7XX64X4M submitted 2025-06-24 cs.LG

Sanjay Chakraborty , Ibrahim Delibasoglu , Fredrik Heintz This is my paper
classification cs.LG
keywords timeseriesforecastingpretrainedlarge-scalemodelssmall-scaletransformerslong-termshort-termMSEandMAEcomparisonfoundationtransferlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to settle a practical question: when a forecasting team has a dataset and a compute budget, should it reach for a pretrained large-scale time series model or train a small transformer from scratch? On long-term multivariate benchmarks with lookback 672 and horizons 96 to 720, the full-shot fine-tuned model LLM4TS posts the lowest average MSE on four of seven datasets, and on short-term traffic benchmarks the large long-context pretrained transformer Timer_XL leads across PEMS03, PEMS04, and PEMS08. The authors read these results as evidence that pretrained large models minimize expected loss—MSE and MAE—more effectively than small task-specific transformers, and they formalize that ordering as a set of theorems. They also report that large architectures trained from scratch remain competitive, and they explicitly leave interpretability and real-time inference efficiency out of scope. The practical stake is whether the default choice for forecasting should shift toward pretrained foundation models.

What carries the argument

The argument is carried by a fixed benchmark protocol: long-term models use lookback 672 and average MSE/MAE over horizons 96, 192, 336, and 720 on eight public datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather, Electricity, Traffic, Exchange), while short-term models use lookback 96 and horizons 12, 24, and 48 on PEMS03, PEMS04, and PEMS08. Within that protocol, the load-bearing comparison is Table 2 (large models trained from scratch versus non-pretrained small transformers) against Table 3 (pretrained and fine-tuned large models versus the same small transformers). A secondary mechanism is the alpha spectral metric computed from weight matrices, used to argue that pretrained layers sit in the 2-6 range characteristic of generalizable training. The five theorems restate the empirical ordering as expected-loss inequalities; their work is to formalize the comparison rather than to introduce a new architectural mechanism.

What would settle it

Re-run the Table 2 and Table 3 protocols with the small baselines (iTransformer, PatchTST, Crossformer, FEDformer) using their published hyperparameters, multiple seeds, and the same lookback of 672; if any small model's average MSE on ETTh1 falls below LLM4TS_FS's 0.404 under the same evaluation, the paper's central ordering is falsified for that setting.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that pretraining at scale transfers to forecasting accuracy. Across long-term multivariate tasks, LLM4TS_FS achieves the best average MSE on ETTh1 (0.404), ETTh2 (0.333), ETTm1 (0.343), and ETTm2 (0.251), with the best or near-best MAE on the same datasets, while Timer_XL dominates the short-term PEMS benchmarks at horizons 12, 24, and 48. Models pretrained on the curated ERA5-Large corpus (40 years from 4,920 stations) show alpha values from spectral weight-matrix analysis inside the 2-6 band that the authors associate with well-generalized layers. The paper states five theorems asserting that pretrained large models achieve lower expected MSE/MAE than small-scale transformers and that even large models trained from scratch stay competitive, given sufficient data and capacity. The study explicitly limits its scope to forecasting accuracy under its chosen protocol, leaving interpretability and inference-time efficiency to future work.

Load-bearing premise

The comparison assumes the small-scale transformer baselines were tuned to the same standard as the large pretrained models, but the paper does not report hyperparameters, seeds, or repeated runs for the long-term experiments, and some baseline numbers in Table 2 run higher than published values for the same models and datasets.

Editorial extensions

If this is right

  • If the ordering holds, a full-shot fine-tuned pretrained forecasting transformer such as LLM4TS is the better default than a small transformer trained from scratch for long-horizon multivariate tasks.
  • For short-horizon traffic data, Timer_XL is the safer default across horizons 12, 24, and 48, with Crossformer and Moirai as closest alternatives.
  • Scale alone buys part of the advantage: large architectures trained from scratch (Timer, Moment) beat the small transformers even without pretraining.
  • Pretraining on a large climate-era corpus (ERA5-Large) yields layer-quality alpha measures in the 2-6 band, which the authors link to better generalization.
  • Small transformers remain useful as lightweight baselines but are not the top-accuracy choice in the settings tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would apply the same protocol under equal tuning budgets—identical seed counts and a shared hyperparameter search per model—to quantify how much of the gap comes from pretraining versus model scale alone.
  • The alpha spectral measurement could be turned into a cheap pre-deployment check: compute alpha on a candidate model's weight matrices, flag layers outside the 2-6 band, and only then run the full evaluation.
  • The same comparison could be repeated on non-meteorological and non-traffic domains such as finance, health, and energy demand to see whether the LLM4TS and Timer_XL ordering persists where seasonal and cross-series structure differ.
  • Interpretability and inference-latency metrics could be added to the protocol, since the paper itself identifies those as missing, giving practitioners a fuller cost-accuracy picture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper asks whether pretrained large-scale time series models (LSTSMs) such as Moirai, Moment, Timer, Timer_XL, GPT4TS, LLM4TS, TimeMachine, and TimeLLM outperform smaller non-pretrained transformers (Crossformer, ETSformer, iTransformer, PatchTST, FEDformer) on multivariate long-term and short-term forecasting. The empirical study reports average MSE and MAE over fixed horizons on ETT, Weather, Electricity, Traffic, Exchange, and PEMS datasets, comparing models trained from scratch, fine-tuned, and in few-shot modes. The paper also presents five statements labeled as 'Theorems' in Section 6 and concludes in Section 7 that pretrained large-scale models 'significantly outperform' smaller task-specific models.

Significance. A careful controlled comparison of large pretrained time series models against well-tuned small transformers would be a valuable contribution to a currently active debate. The paper has a broad empirical sweep and uses public benchmark datasets through TSLib, which is a strength. However, the central claim is not established: the tables contain no repeated trials or uncertainty estimates, the small-model baselines appear not to be tuned to a comparable standard, several of the paper's own entries contradict the uniform-superiority conclusion, and the 'Theorems' in Section 6 are restatements of empirical observations rather than proofs. As a result, the manuscript does not answer the title question reliably in its current form.

major comments (5)
  1. [Sections 5.2, 5.3, and 7 (Tables 2, 3, 4)] Every experimental result is a single point estimate. No seeds, number of runs, confidence intervals, or standard deviations are reported for any table. The word 'significantly' in Section 7 therefore cannot be justified; differences such as LLM4TS_FS 0.404 vs TimeMachine_FS 0.417 on ETTh1 in Table 3, or Timer_XL 0.074 vs Crossformer 0.072 on PEMS03 in Table 4, cannot be distinguished from run-to-run variation without repeated trials.
  2. [Section 5.2 (Table 2)] The comparison is not demonstrably apples-to-apples because no hyperparameter table, tuning procedure, or per-dataset configuration is provided for the small-scale transformers. The fixed context length L=672 is used for all models, which is not the standard setting for several baselines, and values for models such as iTransformer on ETTh1 appear higher than published results for these benchmarks. Without evidence that the scratch-trained baselines were tuned to a comparable standard, the claimed advantage of pretrained LSTSMs on ETT datasets is not established.
  3. [Section 6 (Theorems 1-5)] The statements labeled 'Theorem' are not theorems: Theorem 1 cites 'Empirical evidence' from the experimental tables as its support, Theorem 4 assumes 'sufficient training data and model capacity' and then asserts exactly the inequality it is supposed to prove, and Theorem 5 is an assumption-based assertion with no derivation. There is no formal proof or statement of conditions. Section 7 cites 'theoretical analyses' as corroboration, but this support is circular because the only evidence is the empirical tables themselves, which are the object under dispute.
  4. [Section 7 and Tables 3-4] The conclusion that pretrained large-scale models 'consistently demonstrate' superiority is contradicted by the paper's own data. In Table 3, LLM4TS_FS has the best average MSE on only four of seven datasets; GPT4TS_FS beats it on Electricity (0.166 vs 0.169), and Timer_FS (0.381) and GPT4TS_FWS (0.377) beat it on Exchange (0.415). In Table 4, Crossformer beats Timer_XL on PEMS03 at horizon 12 (0.072 vs 0.074), and Moirai is better or comparable in several PEMS settings. A defensible conclusion would be that large pretrained models are competitive on some datasets and horizons, not uniformly superior.
  5. [Section 5.3] The text states that short-term forecasting uses 'two benchmark datasets: the PEMS dataset and the M4 dataset,' but no M4 results appear anywhere in the paper. Table 4 reports only PEMS03, PEMS04, and PEMS08. Either the M4 experiments were omitted from the analysis or the statement is unsupported; as written, the short-term evaluation covers only the PEMS datasets.
minor comments (5)
  1. [Abstract and Section 4.2] The abstract mentions TimeGPT as a pretrained model analyzed in the paper, but TimeGPT does not appear in the experiments, tables, or comparisons. Either include it in the evaluation or remove the mention.
  2. [Table 5] The row for GPT4TS reports 'Decoder 1.7T 175B,' which is numerically inconsistent and appears to confuse parameter count with pretraining token scale; please verify all entries in this table for correctness and consistency.
  3. [Figure 5 caption] The caption lists '(d) Moirai' twice; the second occurrence should label a different model, likely GPT4TS or another LSTSM.
  4. [Section 6] The label 'Theorem' is misleading for statements that are empirical observations or assumptions; renaming them 'Observation' or 'Claim' would better reflect their content and avoid giving an impression of formal proof.
  5. [Section 5.1] The sentence beginning 'Features 60 compute nodes...' is a fragment; it should be rephrased as a complete sentence describing the Berzelius cluster configuration.

Circularity Check

2 steps flagged · score 6.0 of 10

The Section 6 'theorems' are empirical restatements: their conclusions are assumed or read off the same tables they claim to prove, so the paper's 'theoretical demonstration' adds no independent support.

  1. self definitional [Section 6, Theorem 2]
    "The generalization performance of LSTSMs is superior to that of small-scale transformers, as E[L(f p(X))]≤E[L(f t(X))], where f p ∈ Mp and f t ∈ Mt, assuming the benefit of large-scale pretraining for LSTSMs."

    The claimed inequality is introduced as the thing to be shown, and its only support is 'assuming the benefit of large-scale pretraining for LSTSMs'—which is the same superiority claim in other words. No bound, derivation, or separation argument connects the architecture definitions to the inequality. Section 7 then invokes this 'theoretical analysis' as part of the demonstration that pretrained large models significantly outperform small ones, so the theorem supplies no independent evidence; it is circular by construction.

  2. renaming known result [Section 6, Theorems 1, 4, and 5; relied on in Section 7]
    "Empirical evidence shows that LLM4TS_FS achieves the lowest average MSE and MAE across multiple datasets, demonstrating superior generalization. ... Then, under sufficient training data and model capacity, the following inequality holds:min θ∈Mlarge E[L]< minθ∈Msmall E[L]."

    Theorems 1, 4, and 5 do not derive their inequalities from stated assumptions; they wrap the observed rankings in Tables 2-4 in mathematical notation. Theorem 1 cites the experimental result as its evidence; Theorem 4 asserts the Table 3 ranking as the consequence of vague conditions ('sufficient training data and model capacity'). The phrase 'Empirical evidence shows' makes explicit that the theorem is a restatement of the measurements, not a first-principles result. Because Section 7's central claim is said to follow from 'empirical and theoretical analyses,' this theoretical component reduces to the empirical tables it concludes.

full rationale

The empirical comparison itself is not circular: the MSE/MAE numbers are external measurements, and no fitted parameter is relabeled as a prediction. No self-citation chain is load-bearing in this paper; the circularity is internal to Section 6. There, results are rebranded as theorems. Theorem 2 assumes the benefit of large-scale pretraining in order to prove that LSTSMs generalize better, which is circular by construction. Theorems 1, 3, 4, and 5 are restatements of the tables or tautologies about 'the optimal model,' with no derivation from assumptions. Theorem 3, in particular, defines the optimal model as the minimizer of expected loss and then reads Timer_XL's ranking off Table 4, so it contributes nothing beyond the observations. Since Section 7's headline claim is said to follow from 'empirical and theoretical analyses,' the theoretical component provides no independent support. The underlying empirical claim about model rankings remains an independent, falsifiable observation; concerns about missing hyperparameters, seeds, and error bars, and about counterexamples inside the paper's own tables, weaken its evidentiary value but are not circularity. Score 6 therefore reflects partial circularity in the claimed derivation chain, not a fully circular empirical study.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities or fitted parameters. Its central claim rests on the integrity of the empirical runs and on vague theorem assumptions about 'sufficient training data and model capacity'.

assumptions (3)
  • domain assumption The reported MSE/MAE values are accurate and representative of each model's true performance without multiple seeds or error bars.
    Tables 2-4 report single-run metrics; the paper treats these as definitive comparisons.
  • domain assumption The optimal alpha range 2-6 from WeightWatcher literature indicates well-trained layers.
    Section 5.5 uses this range to conclude pretraining improves layer quality; it cites [24,25] but does not validate the range on these models.
  • ad hoc to paper Sufficient training data and model capacity guarantee that large models achieve lower expected loss than small models (Theorem 4).
    The theorem's assumption is vague and not verified; it is introduced to justify the claimed inequality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?." pith.science (2026). https://pith.science/paper/7XX64X4M

@misc{pith2026250702907,
  author       = {Pith},
  title        = {Pith review of: Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7XX64X4M}},
  note         = {Machine review of arXiv:2507.02907}
}
read the original abstract

Large pre-trained models have demonstrated remarkable capabilities across domains, but their effectiveness in time series forecasting remains understudied. This work empirically examines whether pre-trained large-scale time series models (LSTSMs) trained on diverse datasets can outperform traditional non-pretrained small-scale transformers in forecasting tasks. We analyze state-of-the-art (SOTA) pre-trained universal time series models (e.g., Moirai, TimeGPT) alongside conventional transformers, evaluating accuracy, computational efficiency, and interpretability across multiple benchmarks. Our findings reveal the strengths and limitations of pre-trained LSTSMs, providing insights into their suitability for time series tasks compared to task-specific small-scale architectures. The results highlight scenarios where pretraining offers advantages and where simpler models remain competitive.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 34 canonical work pages

  1. [1]

    Abdel-Sater and A

    R. Abdel-Sater and A. Ben Hamza. A federated large language model for long-term time series forecasting. InECAI 2024, pages 2452–2459. IOS Press, 2024

    work page 2024

  2. [2]

    M. A. Ahamed and Q. Cheng. Timemachine: A time series is worth 4 mambas for long-term forecasting. InECAI 2024, volume 392, page 1688, 2024

    work page 2024

  3. [3]

    Ahmed, I

    S. Ahmed, I. E. Nielsen, A. Tripathi, S. Siddiqui, R. P. Ramachandran, and G. Rasool. Transformers in time-series analysis: A tutorial.Cir- cuits, Systems, and Signal Processing, 42(12):7433–7466, 2023

    work page 2023

  4. [4]

    A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. Rangapuram, S. P. Arango, S. Kapoor, et al. Chronos: Learning the language of time series. 2024

    work page 2024

  5. [5]

    Campagner, M

    A. Campagner, M. Barandas, D. Folgado, H. Gamboa, and F. Cabitza. Ensemble predictors: Possibilistic combination of conformal predictors for multivariate time series classification.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    work page 2024

  6. [6]

    Chang, W.-Y

    C. Chang, W.-Y . Wang, W.-C. Peng, and T.-F. Chen. Llm4ts: Aligning pre-trained llms as data-efficient time-series forecasters.arXiv preprint arXiv:2308.08469, 2023

    arXiv 2023

  7. [7]

    A. Das, W. Kong, R. Sen, and Y . Zhou. A decoder-only foundation model for time-series forecasting. InForty-first International Confer- ence on Machine Learning, 2024

    work page 2024

  8. [8]

    E. A. Engel and N. E. Engel. A transformer with a fuzzy attention mech- anism for weather time series forecasting. InInternational Conference on Neuroinformatics, pages 418–425. Springer, 2024

    work page 2024

Show all 48 references
  1. [9]

    H. Face. Hugging face’s model hub, 2023. URL https://huggingface.co

    2023

  2. [10]

    N. M. Foumani, C. W. Tan, G. I. Webb, and M. Salehi. Improving posi- tion encoding of transformers for multivariate time series classification. Data mining and knowledge discovery, 38(1):22–48, 2024

    2024

  3. [11]

    Goswami, K

    M. Goswami, K. Szafer, A. Choudhry, Y . Cai, S. Li, and A. Dubrawski. Moment: A family of open time-series foundation models.arXiv preprint arXiv:2402.03885, 2024

    2024 arXiv

  4. [12]

    M. Jin, S. Wang, L. Ma, Z. Chu, J. Y . Zhang, X. Shi, P.-Y . Chen, Y . Liang, Y .-F. Li, S. Pan, et al. Time-llm: Time series forecasting by re- programming large language models.arXiv preprint arXiv:2310.01728, 2023

    2023 arXiv

  5. [13]

    Kitaev, Ł

    N. Kitaev, Ł. Kaiser, and A. Levskaya. Reformer: The efficient trans- former.arXiv preprint arXiv:2001.04451, 2020

    2001 arXiv

  6. [14]

    T. Li, B. Yu, J. Li, and Z. Zhu. Functional relation field: A model- agnostic framework for multivariate time series forecasting.Artificial Intelligence, 334:104158, 2024

    2024

  7. [15]

    Li and K

    W. Li and K. E. Law. Deep learning models for time series forecasting: a review.IEEE Access, 2024

    2024

  8. [16]

    Liang, H

    Y . Liang, H. Wen, Y . Nie, Y . Jiang, M. Jin, D. Song, S. Pan, and Q. Wen. Foundation models for time series analysis: A tutorial and survey. In Proceedings of the 30th ACM SIGKDD conference on knowledge dis- covery and data mining, pages 6555–6565, 2024

    2024

  9. [17]

    P. Liu, H. Guo, T. Dai, N. Li, J. Bao, X. Ren, Y . Jiang, and S.-T. Xia. Calf: Aligning llms for time series forecasting via cross-modal fine- tuning.arXiv preprint arXiv:2403.07300, 2024

    2024 arXiv

  10. [18]

    Y . Liu, H. Wu, J. Wang, and M. Long. Non-stationary transformers: Exploring the stationarity in time series forecasting.Advances in Neural Information Processing Systems, 35:9881–9893, 2022

    2022

  11. [19]

    Y . Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long. itrans- former: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023

    2023 arXiv

  12. [20]

    Y . Liu, G. Qin, X. Huang, J. Wang, and M. Long. Timer-xl: Long- context transformers for unified time series forecasting.arXiv preprint arXiv:2410.04803, 2024

    2024 arXiv

  13. [21]

    Y . Liu, H. Zhang, C. Li, X. Huang, J. Wang, and M. Long. Timer: Generative pre-trained transformers are large time series models.arXiv preprint arXiv:2402.02368, 2024

    2024 arXiv

  14. [22]

    Y . Liu, G. Qin, X. Huang, J. Wang, and M. Long. Autotimes: Autore- gressive time series forecasters via large language models.Advances in Neural Information Processing Systems, 37:122154–122184, 2025

    2025

  15. [23]

    Z. Liu, Z. Zhu, J. Gao, and C. Xu. Forecast methods for time series data: A survey.Ieee Access, 9:91896–91912, 2021

    2021

  16. [24]

    C. H. Martin and M. W. Mahoney. Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning.Journal of Machine Learning Research, 22(165):1–73, 2021

    2021

  17. [25]

    C. H. Martin, T. Peng, and M. W. Mahoney. Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data.Nature Communications, 12(1):4122, 2021

    2021

  18. [26]

    D. C. Montgomery, C. L. Jennings, and M. Kulahci.Introduction to time series analysis and forecasting. John Wiley & Sons, 2015

    2015

  19. [27]

    Berzelius supercomputer, 2025

    National Supercomputer Centre (NSC). Berzelius supercomputer, 2025. URL https://www.nsc.liu.se/systems/berzelius/. Accessed: 2025-03-04

    2025

  20. [28]

    Y . Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. A time se- ries is worth 64 words: Long-term forecasting with transformers.arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  21. [29]

    S. Shi, N. Wang, S. Chen, B. Hu, J. Peng, and Z. Shi. Digital mapping of soil salinity with time-windows features optimization and ensemble learning model.Ecological Informatics, 85:102982, 2025

    2025

  22. [30]

    X. Shi, S. Wang, Y . Nie, D. Li, Z. Ye, Q. Wen, and M. Jin. Time- moe: Billion-scale time series foundation models with mixture of ex- perts.arXiv preprint arXiv:2409.16040, 2024

    2024 arXiv

  23. [31]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  24. [32]

    Sa-timeseries: Self-attention time series models, 2024

    UV A-MLSys. Sa-timeseries: Self-attention time series models, 2024. URL https://github.com/UV A-MLSys/SA-Timeseries. Last accessed: 15 Nov 2024

    2024

  25. [33]

    A. Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  26. [34]

    Q. Wen, T. Zhou, C. Zhang, W. Chen, Z. Ma, J. Yan, and L. Sun. Trans- formers in time series: A survey.arXiv preprint arXiv:2202.07125, 2022

    2022 arXiv

  27. [35]

    G. Woo, C. Liu, D. Sahoo, A. Kumar, and S. Hoi. Etsformer: Exponen- tial smoothing transformers for time-series forecasting.arXiv preprint arXiv:2202.01381, 2022

    2022 arXiv

  28. [36]

    G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo. Uni- fied training of universal time series forecasting transformers. InPMLR, 2024

    2024

  29. [37]

    Y . Wu, C. Lian, Z. Zeng, B. Xu, and Y . Su. An aggregated convolutional transformer based on slices and channels for multivariate time series classification.IEEE Transactions on Emerging Topics in Computational Intelligence, 7(3):768–779, 2022

    2022

  30. [38]

    G. Yang, E. J. Hu, I. Babuschkin, S. Sidor, X. Liu, D. Farhi, N. Ryder, J. Pachocki, W. Chen, and J. Gao. Tensor programs v: Tuning large neural networks via zero-shot hyperparameter transfer.arXiv preprint arXiv:2203.03466, 2022

    2022 arXiv

  31. [39]

    Z. Yang, B. Zhou, X. Tang, R. Li, and S. Hu. Breaking the weak se- mantics bottleneck of transformers in time series forecasting. InECAI 2024, pages 1430–1437, 2024

    2024

  32. [40]

    Yaprakdal and M

    F. Yaprakdal and M. Varol Arısoy. A multivariate time series analysis of electrical load forecasting based on a hybrid feature selection approach and explainable deep learning.Applied Sciences, 13(23):12946, 2023

    2023

  33. [41]

    A. Zeng, M. Chen, L. Zhang, and Q. Xu. Are transformers effective for time series forecasting? InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023

    2023

  34. [42]

    Zerveas, S

    G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff. A transformer-based framework for multivariate time series representa- tion learning. InProceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2114–2124, 2021

    2021

  35. [43]

    Zhang, H

    J. Zhang, H. Liu, W. Bai, and X. Li. A hybrid approach of wavelet transform, arima and lstm model for the share price index futures fore- casting.The North American Journal of Economics and Finance, 69: 102022, 2024

    2024

  36. [44]

    Zhang, Q

    K. Zhang, Q. Wen, C. Zhang, R. Cai, M. Jin, Y . Liu, J. Y . Zhang, Y . Liang, G. Pang, D. Song, et al. Self-supervised learning for time series analysis: Taxonomy, progress, and prospects.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  37. [45]

    Zhang and J

    Y . Zhang and J. Yan. Crossformer: Transformer utilizing cross- dimension dependency for multivariate time series forecasting. InThe eleventh international conference on learning representations, 2023

    2023

  38. [46]

    Zhang, W

    Z. Zhang, W. Li, and H. Liu. Multivariate time series forecasting by graph attention networks with theoretical guarantees. InInternational Conference on Artificial Intelligence and Statistics, pages 2845–2853. PMLR, 2024

    2024

  39. [47]

    T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series fore- casting. InInternational conference on machine learning, pages 27268– 27286. PMLR, 2022

    2022

  40. [48]

    T. Zhou, P. Niu, L. Sun, R. Jin, et al. One fits all: Power general time series analysis by pretrained lm.Advances in neural information pro- cessing systems, 36:43322–43355, 2023

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.