Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Large Language Models are Few-shot Multivariate Time Series Classifiers

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a pre-trained LLM, adapted with a temporal-convolution encoder and LoRA fine-tuning, beats all 11 baselines on 10 multivariate time series datasets in the 1-shot setting, with accuracy improvements up to 125.2%.

desk verdict A useful empirical study of LLM fine-tuning for few-shot multivariate time series classification, but the headline claims about pre-trained knowledge are not fully supported by the ablations. read the letter →

arxiv 2502.00059 v1 pith:2RKG7MZT submitted 2025-01-30 cs.LG cs.AI

Yakun Chen , Zihao Li , Chao Yang , Xianzhi Wang , Guandong Xu This is my paper · ORCID
classification cs.LGcs.AI
keywords multivariatetimeseriesclassificationfew-shotlearninglargelanguagemodelsLoRAfine-tuningtemporalconvolutionalencodertransfersequencemodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that knowledge embedded in large language models during text pre-training can be transferred to few-shot multivariate time series classification, where labeled examples are extremely scarce and conventional deep models tend to overfit. The proposed LLMFew pipeline splits each series into patches, encodes them with a causal dilated-convolution module (PTCEnc), runs the result through a pre-trained LLM decoder fine-tuned only on attention projections via LoRA, and fuses the encoder and LLM representations for classification. Experiments on the 10 UEA datasets show that with one example per class, LLMFew outperforms all 11 baselines on every dataset, with gains of 125.2% on Handwriting and 50.2% on EthanolConcentration. The paper interprets this as evidence that pre-trained LLM knowledge, rather than the transformer architecture alone, is what enables strong few-shot performance.

What carries the argument

The load-bearing mechanism is the pairing of a Patch-wise Temporal Convolution Encoder (PTCEnc) with a pre-trained LLM decoder fine-tuned by Low-Rank Adaptation (LoRA). PTCEnc stacks causal convolutional layers with exponentially increasing dilation and residual connections, turning multivariate time series patches into vectors that match the LLM's embedding dimension. LoRA injects trainable low-rank matrices into the query, key, and value projections so that only a small fraction of parameters is updated during fine-tuning. The final representation is the element-wise sum of the encoder output and the LLM output, passed through an MLP classification head.

What would settle it

Train a transformer with identical architecture and parameter count, random initialization, and the same PTCEnc and LoRA fine-tuning on the same 1-shot tasks; if its average accuracy matches or exceeds LLMFew, the claim that pre-trained knowledge is the key driver is falsified.

Watch

Extended reading notes

Core claim

The central claim is that a pre-trained LLM, lightly adapted, is a state-of-the-art few-shot multivariate time series classifier. The input is patched into $X_P$, the PTCEnc produces embeddings $H_e$, the LLM decoder maps these to $H_d$, and the classifier uses $\mathrm{ReLU}(H_e + H_d)$ before a softmax head. With one labeled example per class, LLMFew improves over the second-best baseline on every one of the ten UEA datasets, with the largest gains on the hardest ones: 125.2% on Handwriting (26 classes) and 50.2% on EthanolConcentration (series of length 1751). Ablations show that freezing the LLM loses part of the gain, removing the LLM entirely loses more, and replacing PTCEnc with a plain 1-D convolution hurts accuracy, all of which the paper takes as support for each component's contribution.

Load-bearing premise

The central claim rests on the assumption that the accuracy gains come from the LLM's pre-trained weights, but the ablation compares only against a frozen LLM and a model with the LLM removed, never against a randomly initialized transformer trained with the same procedure, so the possibility that the architecture and fine-tuning recipe alone drive the results remains open.

Editorial extensions

If this is right

  • If the claim holds, few-shot MTSC no longer requires source-domain time series data; a text-pretrained model suffices.
  • The largest gains appear on datasets with many classes (Handwriting, 26) and very long series (EthanolConcentration, length 1751), suggesting that LLM priors help most where local and global temporal structure is hard to learn from a few samples.
  • With more shots, accuracy rises but with diminishing returns, and full-data training still keeps LLMFew competitive, so the approach is not limited to few-shot regimes.
  • The frozen-LLM ablation shows that part of the value survives without any fine-tuning, but LoRA fine-tuning is needed for the largest gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A decisive control the paper leaves implicit: training a randomly initialized transformer of the same size and architecture with the same PTCEnc and LoRA recipe would directly test whether the gains come from pre-trained knowledge or merely from the fine-tuning setup; if that control matched LLMFew, the paper's central attribution would be in doubt.
  • The same alignment-plus-LoRA recipe may transfer to other label-scarce sequential modalities, such as sensor streams, financial tick data, or biomedical signals, because the encoder only needs to map the input into the LLM's embedding space.
  • Larger LLMs did not consistently beat smaller ones across datasets, which suggests that backbone selection should be made per dataset rather than by parameter count, and that a small model can suffice when the dataset's structure aligns with what the LLM already knows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LLMFew, a framework for few-shot multivariate time series classification (MTSC) that combines a Patch-wise Temporal Convolution Encoder (PTCEnc) with a pre-trained decoder-only LLM fine-tuned via LoRA. The method is evaluated on 10 UEA datasets under 1-shot, 3-shot, 5-shot, and full-training settings, comparing against 11 baselines. The paper reports large improvements in 1-shot accuracy, e.g., 125.2% on Handwriting and 50.2% on EthanolConcentration, and interprets these gains as evidence that pre-trained LLM knowledge is essential for few-shot MTSC. Ablations evaluate the contributions of PTCEnc, the frozen LLM, and the LLM itself, along with analyses of LLM size, hyperparameters, and efficiency.

Significance. If the empirical claims hold, the paper makes a timely and potentially useful contribution by adapting LLMs for few-shot MTSC, a relatively underexplored problem. The proposed architecture is simple and the authors provide code, which aids reproducibility. The extensive benchmark across 10 datasets and multiple LLM sizes is a strength. However, the central scientific claim—that pre-trained knowledge, rather than the transformer architecture or the training scheme, drives the gains—is not adequately supported by the current experiments. The reported numerical results also contain an inconsistency that affects the headline improvement. With additional controls and corrected reporting, the paper could be a solid empirical study.

major comments (4)
  1. [Section 4.2.5 (Table 4)] The ablation study compares LLMFew against Frozen, w/o LLM, and w/o PTCEnc, but it does not include a control with a randomly initialized transformer of the same architecture trained with the same PTCEnc and LoRA protocol. The claim in Section 4.2.1 that 'pre-training knowledge in LLMs is essential' is therefore not directly tested. The Vanilla Transformer baseline differs in architecture, input encoding, and training scheme, so its inferior performance cannot be attributed solely to the lack of pre-training. A random-init variant of the LLM decoder would isolate the contribution of pre-trained weights and is necessary to substantiate the paper's main conclusion.
  2. [Table 2, Abstract] The improvement reported for EthanolConcentration is inconsistent with the stated definition. Table 2 defines 'Improv. %' as the performance improvement over the second-best baseline, and the second-best for EC is TF-C at 32.5%; (42.6−32.5)/32.5 ≈ 31.1%, not the 50.2% stated in the abstract and Section 4.2.1. The value 50.2% appears to be computed against a different baseline (e.g., Reformer at 28.3%), which violates the table's definition and misrepresents the result. This discrepancy affects a headline number and must be corrected.
  3. [Section 4.2.3 (Figure 4)] The full-training comparison in Figure 4 relies on results reported from OneFitsAll for most baselines, while only LLMFew, Crossformer, and PatchTST are run by the authors under their own protocol. This mixes experimental setups (different training details, seeds, hardware) and does not support the claim that LLMFew 'consistently outperforms all state-of-the-art baselines,' especially since only an average across datasets is shown. A per-dataset table with all baselines evaluated under the same protocol is needed to justify this claim.
  4. [Table 2, footnote] The footnote states that '*' denotes a significant improvement over the best baseline (P < 0.05), but no details are given about the statistical test used, the number of runs, or whether multiple comparisons were corrected. For example, on SCP1 the improvement over Reformer is 0.7 percentage points (88.5 vs. 87.9) with overlapping standard deviations, yet the paper still claims LLMFew outperforms all baselines. The paper should either report the test details or soften the significance claims.
minor comments (6)
  1. [Abstract] The phrase 'LLMF ew' appears with a stray space; please fix the formatting.
  2. [Table 1] The caption reads 'Datasets Satistics' instead of 'Datasets Statistics'.
  3. [Section 4.1.2] The paper does not describe how forecasting-based baselines (e.g., Autoformer, FEDformer, Crossformer) are adapted for classification; this detail is needed for reproducibility.
  4. [Figure 3] The caption does not specify what the error bars represent (e.g., standard deviation over the 5 runs).
  5. [Section 4.2.4] The statement that Phi3's good performance on PEMS-SF is due to its inter-GPS ability is speculative and not supported by the data.
  6. [Implementation Details] The text refers to 'Section 3 in the Appendix' for implementation details, but no appendix is present in the submitted manuscript.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are measured against held-out UEA test data and do not reduce by construction to the model's inputs or fitted parameters.

full rationale

The paper is an empirical benchmarking study rather than a derivation, and I could not find any step in which a claimed prediction reduces to an input by construction. LLMFew's reported accuracies (Table 2, Section 4.1.4) are measured on held-out UEA test sets after training on K-shot subsets, so the headline improvements are external measurements, not identities with fitted values. The attribution of gains to pre-trained LLM knowledge (Section 4.2.1) is supported by the Frozen / w/o LLM ablations in Table 4, and although the absence of a randomly initialized transformer of matched size leaves a control gap, that is an experimental-design limitation, not a circularity. The one self-citation with author overlap, Dyformer [14], appears only in related-work description and is not used as a baseline or as load-bearing evidence for the paper's claims. Patching and LoRA are adopted via external citations [26, 28], and no uniqueness theorem or author-imported ansatz is invoked to force the framework. Accordingly the circularity score is 0.

Assumptions & free parameters 7 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities. It relies on several domain assumptions about dataset representativeness and evaluation fairness, and on a set of freely chosen hyperparameters that are tuned per dataset. The absence of a randomly initialized transformer control means the contribution of pre-trained knowledge is not isolated cleanly.

free parameters (7)
  • patch length and stride = (16,8) to (128,64)
    Chosen per dataset in sensitivity analysis; affects number of tokens and performance.
  • PTCEnc depth = 1 to 5 (best 3 for most)
    Determines receptive field; tuned in Figure 6.
  • convolution kernel size = 3 (best for most)
    Affects local feature extraction.
  • convolution hidden size = 128 to 1024
    Tuned per dataset; larger sizes can overfit.
  • LoRA rank r = not reported in text
    Controls capacity of fine-tuning; not specified in the available text.
  • learning rate = 0.0002 with decay 80% every 50 epochs
    Training hyperparameter reported in Section 4.1.4.
  • training epochs = 200
    Training hyperparameter reported in Section 4.1.4.
assumptions (3)
  • domain assumption The UEA Archive datasets are representative of multivariate time series classification tasks.
    The paper generalizes its findings from 10 datasets to the broader MTSC problem.
  • domain assumption Random selection of K samples per class from the training split without controlling for seed or using cross-validation yields a fair few-shot evaluation.
    Section 4.1.3 describes the few-shot evaluation; randomness could bias results if not carefully handled.
  • domain assumption The pre-trained knowledge of LLMs can transfer to numeric time series modalities through fine-tuning.
    This is the premise of the method, partially tested by the Frozen ablation but not fully controlled.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Models are Few-shot Multivariate Time Series Classifiers." pith.science (2026). https://pith.science/paper/2RKG7MZT

@misc{pith2026250200059,
  author       = {Pith},
  title        = {Pith review of: Large Language Models are Few-shot Multivariate Time Series Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2RKG7MZT}},
  note         = {Machine review of arXiv:2502.00059}
}
read the original abstract

Large Language Models (LLMs) have been extensively applied in time series analysis. Yet, their utility in the few-shot classification (i.e., a crucial training scenario due to the limited training data available in industrial applications) concerning multivariate time series data remains underexplored. We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem within multivariate time series. Specifically, we propose LLMFew, an LLM-enhanced framework to investigate the feasibility and capacity of LLMs for few-shot multivariate time series classification. This model introduces a Patch-wise Temporal Convolution Encoder (PTCEnc) to align time series data with the textual embedding input of LLMs. We further fine-tune the pre-trained LLM decoder with Low-rank Adaptations (LoRA) to enhance its feature representation learning ability in time series data. Experimental results show that our model outperformed state-of-the-art baselines by a large margin, achieving 125.2% and 50.2% improvement in classification accuracy on Handwriting and EthanolConcentration datasets, respectively. Moreover, our experimental results demonstrate that LLM-based methods perform well across a variety of datasets in few-shot MTSC, delivering reliable results compared to traditional models. This success paves the way for their deployment in industrial environments where data are limited.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 26 canonical work pages

  1. [1]

    Data Mining and Knowledge Discovery 33(4), 917–963 (2019)

    Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Deep learning for time series classification: a review. Data Mining and Knowledge Discovery 33(4), 917–963 (2019)

    work page 2019

  2. [2]

    NPJ digital medicine 1(1), 1–10 (2018)

    Rajkomar, A., Oren, E., Chen, K., Dai, A.M., Hajaj, N., Hardt, M., Liu, P.J., Liu, X., Marcus, J., Sun, M., et al.: Scalable and accurate deep learning with electronic health records. NPJ digital medicine 1(1), 1–10 (2018)

    work page 2018

  3. [3]

    World Wide Web 23(5), 2653–2669 (2020)

    Jiang, J., Ji, S., Long, G.: Decentralized knowledge acquisition for mobile internet applications. World Wide Web 23(5), 2653–2669 (2020)

    work page 2020

  4. [4]

    In: 2018 IEEE International Conference on Big Data (Big Data), pp

    Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Transfer learn- ing for time series classification. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 1367–1376 (2018). IEEE

    work page 2018

  5. [5]

    Advances in Neural Information Processing Systems 35, 3988–4003 (2022)

    Zhang, X., Zhao, Z., Tsiligkaridis, T., Zitnik, M.: Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems 35, 3988–4003 (2022)

    work page 2022

  6. [6]

    In: 2020 International Joint Conference on Neural Networks (IJCNN), pp

    Tang, W., Liu, L., Long, G.: Interpretable time-series classification on few-shot samples. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2020). IEEE

    work page 2020

  7. [7]

    In: Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pp

    Narwariya, J., Malhotra, P., Vig, L., Shroff, G., Vishnu, T.: Meta-learning for few-shot time series classification. In: Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pp. 28–36 (2020)

    work page 2020

  8. [8]

    ACM computing surveys (csur) 53(3), 1–34 (2020)

    Wang, Y., Yao, Q., Kwok, J.T., Ni, L.M.: Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur) 53(3), 1–34 (2020)

    2020

Show all 43 references
  1. [9]

    In: Forty-first International Conference on Machine Learning (2024)

    Jin, M., Zhang, Y., Chen, W., Zhang, K., Liang, Y., Yang, B., Wang, J., Pan, S., Wen, Q.: Position: What can large language models tell us about time series analysis. In: Forty-first International Conference on Machine Learning (2024)

    2024

  2. [10]

    In: 2008 IEEE International Joint Confer- ence on Neural Networks (IEEE World Congress on Computational Intelligence), pp

    Gudmundsson, S., Runarsson, T.P., Sigurdsson, S.: Support vector machines and dynamic time warping for time series. In: 2008 IEEE International Joint Confer- ence on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 2772–2776 (2008). IEEE

    2008

  3. [11]

    Data Mining and 17 Knowledge Discovery 34(5), 1454–1495 (2020)

    Dempster, A., Petitjean, F., Webb, G.I.: Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and 17 Knowledge Discovery 34(5), 1454–1495 (2020)

    2020

  4. [12]

    IEEE access 6, 1662–1669 (2017)

    Karim, F., Majumdar, S., Darabi, H., Chen, S.: Lstm fully convolutional networks for time series classification. IEEE access 6, 1662–1669 (2017)

    2017

  5. [13]

    arXiv preprint arXiv:2002.10061, 1–7 (2020)

    Tang, W., Long, G., Liu, L., Zhou, T., Jiang, J., Blumenstein, M.: Rethink- ing 1d-cnn for time series classification: A stronger baseline. arXiv preprint arXiv:2002.10061, 1–7 (2020)

    2020 arXiv

  6. [14]

    Information Sciences 656, 119881 (2024)

    Yang, C., Wang, X., Yao, L., Long, G., Xu, G.: Dyformer: A dynamic transformer- based architecture for multivariate time series classification. Information Sciences 656, 119881 (2024)

    2024

  7. [15]

    arXiv preprint arXiv:2405.14608 (2024)

    Le, X.-M., Luo, L., Aickelin, U., Tran, M.-T.: Shapeformer: Shapelet transformer for multivariate time series classification. arXiv preprint arXiv:2405.14608 (2024)

    2024 arXiv

  8. [16]

    In: 2021 Digital Image Computing: Techniques and Applications (DICTA), pp

    Gupta, P., Bhaskarpandit, S., Gupta, M.: Similarity learning based few shot learning for ecg time series classification. In: 2021 Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8 (2021). IEEE

    2021

  9. [17]

    In: 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI), pp

    Zhang, H., Pang, Z., Wang, J., Li, T.: Few-shot learning using data augmen- tation and time-frequency transformation for time series classification. In: 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI), pp. 733–738 (2023). IEEE

    2023

  10. [18]

    IEEE Access 11, 41400–41414 (2023)

    Park, S.-H., Syazwany, N.S., Lee, S.-C.: Meta-feature fusion for few-shot time series classification. IEEE Access 11, 41400–41414 (2023)

    2023

  11. [19]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Chen, X., Ge, C., Wang, M., Wang, J.: Supervised contrastive few-shot learn- ing for high-frequency time series. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 7069–7077 (2023)

    2023

  12. [20]

    Advances in Neural Information Processing Systems 36 (2024)

    Gruver, N., Finzi, M., Qiu, S., Wilson, A.G.: Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems 36 (2024)

    2024

  13. [21]

    IEEE Transactions on Knowledge and Data Engineering (2023)

    Xue, H., Salim, F.D.: Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering (2023)

    2023

  14. [22]

    arXiv preprint arXiv:2310.01728 (2023)

    Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J.Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., et al.: Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728 (2023)

    2023 arXiv

  15. [23]

    Advances in neural information processing systems 36, 43322–43355 (2023) 18

    Zhou, T., Niu, P., Sun, L., Jin, R., et al.: One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems 36, 43322–43355 (2023) 18

    2023

  16. [24]

    arXiv preprint arXiv:2310.04948 (2023)

    Cao, D., Jia, F., Arik, S.O., Pfister, T., Zheng, Y., Ye, W., Liu, Y.: Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:2310.04948 (2023)

    2023 arXiv

  17. [25]

    In: Forty-first International Conference on Machine Learning (2024)

    Pan, Z., Jiang, Y., Garg, S., Schneider, A., Nevmyvaka, Y., Song, D.: S2ip-llm: Semantic space informed prompt learning with llm for time series forecasting. In: Forty-first International Conference on Machine Learning (2024)

    2024

  18. [26]

    arXiv preprint arXiv:2211.14730 (2022)

    Nie, Y., Nguyen, N.H., Sinthong, P., Kalagnanam, J.: A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730 (2022)

    2022 arXiv

  19. [27]

    arXiv preprint arXiv:1803.01271 (2018)

    Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)

    2018 arXiv

  20. [28]

    arXiv preprint arXiv:2106.09685 (2021)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  21. [29]

    arXiv preprint arXiv:1811.00075 (2018)

    Bagnall, A., Dau, H.A., Lines, J., Flynn, M., Large, J., Bostrom, A., Southam, P., Keogh, E.: The uea multivariate time series classification archive, 2018. arXiv preprint arXiv:1811.00075 (2018)

    2018 arXiv

  22. [30]

    Zeng, A., Chen, M., Zhang, L., Xu, Q.: Are transformers effective for time series forecasting? In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 11121–11128 (2023)

    2023

  23. [31]

    arXiv preprint arXiv:2210.02186 (2022)

    Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., Long, M.: Timesnet: Tem- poral 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186 (2022)

    2022 arXiv

  24. [32]

    Advances in neural information processing systems 34, 22419–22430 (2021)

    Wu, H., Xu, J., Wang, J., Long, M.: Autoformer: Decomposition transform- ers with auto-correlation for long-term series forecasting. Advances in neural information processing systems 34, 22419–22430 (2021)

    2021

  25. [33]

    In: The Eleventh International Conference on Learning Representations (2023)

    Zhang, Y., Yan, J.: Crossformer: Transformer utilizing cross-dimension depen- dency for multivariate time series forecasting. In: The Eleventh International Conference on Learning Representations (2023)

    2023

  26. [34]

    In: International Conference on Machine Learning, pp

    Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., Jin, R.: Fedformer: Fre- quency enhanced decomposed transformer for long-term series forecasting. In: International Conference on Machine Learning, pp. 27268–27286 (2022). PMLR

    2022

  27. [35]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 19 11106–11115 (2021)

    2021

  28. [36]

    arXiv preprint arXiv:2001.04451 (2020)

    Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451 (2020)

    2020 arXiv

  29. [37]

    Advances in neural information processing systems 30 (2017)

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  30. [38]

    OpenAI blog 1(8), 9 (2019)

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)

    2019

  31. [39]

    arXiv preprint arXiv:2309.16609 (2023)

    Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al.: Qwen technical report. arXiv preprint arXiv:2309.16609 (2023)

    2023 arXiv

  32. [40]

    arXiv preprint arXiv:2404.14219 (2024)

    Abdin, M., Jacobs, S.A., Awan, A.A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., et al.: Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 (2024)

    2024 arXiv

  33. [41]

    arXiv preprint arXiv:2407.21783 (2024)

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  34. [42]

    In: International Conference on Machine Learning, pp

    Biderman, S., Schoelkopf, H., Anthony, Q.G., Bradley, H., O’Brien, K., Hallahan, E., Khan, M.A., Purohit, S., Prashanth, U.S., Raff, E., et al.: Pythia: A suite for analyzing large language models across training and scaling. In: International Conference on Machine Learning, p...

    2023

  35. [43]

    arXiv preprint arXiv:2105.04165 (2021) 20

    Lu, P., Gong, R., Jiang, S., Qiu, L., Huang, S., Liang, X., Zhu, S.-C.: Inter- gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint arXiv:2105.04165 (2021) 20

    2021 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.