Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Sequence Complementor: Complementing Transformers For Time Series Forecasting with Learnable Sequences

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Concatenating a few learnable tokens to the input patches makes transformer time-series forecasters more accurate on most long-term settings and 14 of 15 short-term settings.

desk verdict A useful empirical trick with a vacuous theoretical wrapper; the entropy 'guarantee' doesn't apply to the actual network, but the gains and diagnostic are worth a serious look. read the letter →

arxiv 2501.02735 v1 pith:NLLSEMYF submitted 2025-01-06 cs.LG

Xiwen Chen , Peijie Qiu , Wenhui Zhu , Huayu Li , Hao Wang , Aristeidis Sotiras , Yalin Wang , Abolfazl Razi This is my paper · ORCID
classification cs.LG
keywords timeseriesforecastingtransformerlearnablesequencerepresentationdiversityGaussianentropydiversificationlossinformationtheorypatchtokenization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that transformer time-series forecasters are held back less by architecture than by the poverty of their learned latent representations: across four representative transformer variants, the Gaussian entropy of the encoder output tracks mean squared error almost linearly, with richer representations predicting lower error. To exploit that, it appends K learnable sequences, called Sequence Complementors, to the patched input before self-attention, lets attention mix them with the real tokens, and then discards them before decoding. A volume-maximization loss keeps the complementors mutually orthogonal so they keep injecting distinct information. The paper reports that this plug-in outperforms recent baselines on most long-term settings and on 14 of 15 short-term settings, including a case study where it improves iTransformer by roughly 2.4% in mean squared error.

What carries the argument

The central object is the learnable Sequence Complementor $S$: K small trainable vectors, set to three in all experiments, concatenated to the patched input tokens (Eq. 4) and carried through the transformer encoder as extra keys and values while only the original patches are fed to the decoder (Eqs. 7–8). The argument is carried by two pieces: the entropy chain $H(Z_{\mathrm{enc}}) \le H(Z_{\mathrm{enc}},S)$ and $H(Y|Z_{\mathrm{enc}},S) \le H(Y|Z_{\mathrm{enc}})$ together with the Gaussian MMSE bound from Lemma 1, and the differentiable diversification loss $L_{\mathrm{dcs}} = -\sum_i 2\log((\sigma_S)_i + \varepsilon)$ (Eq. 14), whose minimization, under unit-norm rows, drives the singular values to one and makes the complementor rows orthogonal.

What would settle it

Train the same model twice on the same splits, once with learned Sequence Complementors and once with randomly initialized, frozen complementors of the same shape and count; if the learned version does not consistently lower validation MSE, the claim that learned complementary information drives the gains fails. Alternatively, estimate $H(Y|Z_{\mathrm{enc}},S)$ and $H(Y|Z_{\mathrm{enc}})$ on held-out data; if conditioning on $S$ does not reduce conditional entropy, the stated information-theoretic mechanism is not operating in the trained model.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that conditioning transformer attention on a few extra learnable tokens can lower the mean-squared-error bound for time-series forecasting. The mechanism is Eq. 4: concatenate $S \in \mathbb{R}^{K \times P}$ to the patch embedding $Z_0$, run the usual encoder, and use only the original patches $Z_{\mathrm{enc}}[:L_c]$ for prediction; self-attention lets each original token query keys and values that include the complementors, so the decoder-side representations see information that is not literally present in the input. The theoretical part argues that $H(Z_{\mathrm{enc}}) \le H(Z_{\mathrm{enc}},S)$ and $H(Y|Z_{\mathrm{enc}},S) \le H(Y|Z_{\mathrm{enc}})$, so appending $S$ cannot increase conditional uncertainty about the target and, under a Gaussian MMSE bound, can only lower the forecast error. The empirical part reports consistent MSE reductions across eight long-term and six short-term datasets, with the largest gains on ETTm1 and on the M4 short-term tasks.

Load-bearing premise

The theoretical guarantee assumes that the entropy inequality for a fixed encoder output and an added random variable still applies to the network actually trained, where the encoder output itself changes because it has seen the complementors; the paper never justifies that identification.

Editorial extensions

If this is right

  • Any transformer-based forecaster that tokenizes time series can absorb the plug-in with negligible extra parameters (K×P per channel) and about $O(K^2)$ added attention cost, so the reported gains transfer without redesigning attention.
  • If the entropy–error relationship is as tight as reported, forecasting error on a new dataset could be predicted partly from the Gaussian entropy of encoder features, giving a cheap diagnostic for representation quality.
  • On the short-term M4 benchmark, the method wins 14 of 15 settings, suggesting the benefit grows when training data are scarce and series are heterogeneous rather than when data are plentiful.
  • The iTransformer case study, about 2.4% average MSE reduction, supports the claim that the mechanism is model-agnostic and orthogonal to existing architectural improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of whether attention is necessary: apply the same concatenated learnable tokens to a linear forecaster or MLP; if the gain persists, the effect may be added capacity rather than attention-specific complementation.
  • The diversification loss is essentially a differentiable determinantal-point-process repulsion; the same loss could regularize other learned-token schemes, such as prompts in language models or class tokens in vision transformers, wherever tokens must stay mutually informative.
  • The paper's entropy measurements compare models trained under a fixed budget; a cross-check with differently sized or differently trained transformers would clarify whether the entropy–MSE correlation is about representation richness per se or about model capacity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a model-agnostic plug-in for transformer-based time-series forecasting: K learnable sequences S are concatenated to the input tokens before the encoder, and the encoder's outputs for the original tokens are used for forecasting. A diversification loss based on the volume of the sub-matrix formed by S is added to the MSE objective. The authors report strong empirical gains on long-term benchmarks (eight datasets) and short-term M4 benchmarks (six subsets), plus ablations and a case study on iTransformer. They also claim an information-theoretic justification: adding S should increase representation entropy and reduce conditional entropy H(Y|Z_enc,S), thereby lowering a lower bound on the MMSE.

Significance. If the empirical results hold up, the contribution is practically useful: a simple, cheap, model-agnostic mechanism that consistently improves several transformer baselines, including PatchTST and iTransformer, across many forecasting settings. The paper contains several genuine strengths: extensive comparisons, ablations on the number of complementors and the diversification loss, statistical significance testing (Wilcoxon signed-rank test), a case study on iTransformer, and analysis of representation entropy versus MSE. However, the advertised 'theoretical guarantee' is the central load-bearing claim of the paper, and that part is not sound as written. The empirical evaluation is extensive and could support a revised manuscript, but the current theoretical framing needs substantial correction or removal.

major comments (3)
  1. [Theoretical Justification, Eqs. (10)-(12)] The entropy inequalities stated in Eqs. (10) and (12) do not establish the claimed guarantee for the actual augmented network. Eq. (10) compares H(Z_enc) of the baseline encoder with the joint entropy H(Z_enc,S); since joint entropy is always at least as large as the marginal entropy, this inequality is true for any S and says nothing about whether the augmented representation Z_enc^{(aug)} = f_enc(concat(Z0,S)) has higher entropy than the baseline representation. Eq. (12), H(Y|Z_enc,S) ≤ H(Y|Z_enc), is also not the relevant comparison: in the model of Eq. (7), S is a deterministic learnable parameter, not a random variable, and Z_enc is redefined as the encoder output on the concatenated input. Conditioning on a constant does not reduce entropy, so H(Y|Z_enc^{(aug)},S) = H(Y|Z_enc^{(aug)}). The inequality that would be needed, H(Y|Z_enc^{(aug)}) ≤ H(Y|Z_enc^{(base)}), is not derived anywhere, and the proof in Appendix A of Theorem 1 is simply the monotonicity of conditional entropy for the symbols used, not a statement about the augmented network. Therefore the assertion that the method is 'theoretically sound' is not supported; only the empirical results remain.
  2. [Diversified Complementary Sequence, Eq. (14) and Theorem 2] Theorem 2 assumes the constraint ||S_i|| = 1 for every row of S, but the manuscript does not specify how this constraint is enforced during optimization. Without explicit normalization, the diversification loss L_dcs(S) can be decreased simply by scaling up the rows of S, without making them orthogonal; the reported near-orthogonality in Fig. S7 suggests the constraint is enforced in the implementation, but this must be described. In addition, the theorem implicitly requires K ≤ P (the number of complementors cannot exceed the patch length) for S to have K nonzero singular values; this condition is not stated. These omissions matter because the theoretical claim about diversification and the interpretation of the ablation depend on the constraint being actually imposed.
  3. [Relation between entropy and performance, Fig. 2 and Appendix D] The empirical analysis that motivates the method uses only four to six model configurations per dataset and reports strong correlations (|r| > 0.8) between representation entropy and MSE. This is a useful observation, but it is correlational and based on a small number of points; it does not by itself establish that increasing entropy causes lower forecasting error. Since the theoretical argument in Eqs. (10)-(12) is not valid as stated, the paper's central narrative currently rests on this correlation plus the empirical tables. The authors should either supply a correct formal argument for why the proposed training objective moves the augmented network into a lower-MMSE regime, or explicitly reposition the contribution as an empirical one.
minor comments (5)
  1. [Table 2] The header of Table 2 lists 'FEDformer' twice, which makes the reported columns ambiguous and should be corrected.
  2. [Appendix D, Table S8] The ILI results in the appendix show that iTransformer has a lower average MSE (2.039 vs. 2.152) than the proposed method, although the proposed method has a lower MAE. This is a relevant counterexample to any unqualified claim of superiority and should be acknowledged in the main text.
  3. [Notation in Eq. (4)] The notation for Z_0 is inconsistent: Eq. (1) defines Z_0 ∈ R^{L×D0}, while Eq. (4) reuses the same symbol for a per-channel patchified tensor of size (Lc+K)×P. The distinction between L, Lc, and P should be clarified.
  4. [Case Study on iTransformer] The sentence 'The larger reduction rate of MSE over MAE matches our theoretical analysis' is not a meaningful test of the theory, because MSE and MAE are on different scales and the theoretical bound concerns MMSE, not a comparison of relative reductions in two different error metrics.
  5. [Throughout] There are several typos and small errors, including 'ration' for 'ratio', 'Transofmer' for 'Transformer', and 'iTransoformer' for 'iTransformer'. These do not affect the technical content but should be fixed in a revision.

Circularity Check

2 steps flagged · score 7.0 of 10

The paper's information-theoretic 'guarantee' is a definitional tautology: Eq. (10) and Theorem 1 restate H(Z,S) ≥ H(Z) and conditional-entropy monotonicity, and never compare the augmented encoder with the baseline. The central theoretical claim is vacuous; only the empirical tables carry the method.

  1. self definitional [Theoretical Justification, Justification 1, Eq. (10); Appendix A Proof of Justification 1]
    "Justification 1. We first note that including the sequence complementors has the potential to result in a richer representation. This is because it can increase the entropy H(Z enc): H(Z enc) ≤ H(Z enc, S). (10) Proof. We can decompose the joint entropy H(Z enc, S) to, H(Z enc, S) =H(Z enc) +H(S|Z enc). (17) Since the conditional entropy H(S|Z enc) is always non-negative (Cover 1999), the proof is completed."

    The inequality is the definition of joint entropy: H(Z,S) = H(Z) + H(S|Z) ≥ H(Z), so it holds for any S whatsoever and imposes no constraint on the learned model. In the actual forward pass (Eqs. 4-7), Z_enc is the encoder output computed from the concatenated input, so the left-hand H(Z_enc) and the right-hand H(Z_enc,S) refer to different objects if the left side is meant to be the baseline representation. The conclusion that complementors make the representation richer is just the tautology that a joint distribution has at least as much entropy as one of its marginals; it does not show H(Z_enc_aug) > H(Z_enc_base), which is the comparison the method requires.

  2. self definitional [Theoretical Justification, Theorem 1, Eq. (12); Appendix A Proof of Theorem 1]
    "Theorem 1. The integration of the proposed complementary sequence lowers the bound of MMSE: H(Y |Z enc, S) ≤ H(Y |Z enc). (12) Proof. This is trivial as conditional information is monotonocity, meaning conditioning on additional information (in this case, the complementary sequences S) is impossible to increase the uncertainty about Y ."

    The claimed prediction—that adding S lowers the MMSE bound—is exactly the general identity H(Y|X,Z) ≤ H(Y|X), which is true for arbitrary Z and does not depend on S being learnable, diversified, or even informative. In the implemented model, S is a deterministic learnable parameter, and Eq. (7) already puts S inside Z_enc, while the decoder uses only Z_enc[:Lc]. The inequality that would support the paper's claim is H(Y|Z_enc^(aug)) ≤ H(Y|Z_enc^(base)), and no version of Eq. (12) establishes it. Thus the 'theoretically sound' assertion reduces to the definitional monotonicity of conditional entropy; the actual evidence for the method is empirical, not theoretical.

full rationale

The circularity is confined to the paper's central theoretical framing. Justification 1 and Theorem 1 are presented as the reason the Sequence Complementor 'is theoretically sound' and 'prove feasible from an information theory perspective,' but both reduce to definitional identities: joint entropy is never smaller than marginal entropy, and conditioning cannot increase conditional entropy. Neither identity compares the baseline encoder with the augmented encoder actually used in Eqs. (4)-(9). The paper's own proof of Theorem 1 says only 'This is trivial as conditional information is monotonocity,' which is a restatement of the conclusion, not a derivation from the architecture. The empirical evaluation, by contrast, is substantial and self-contained: it benchmarks against external methods on standard datasets, reports ablations, statistical tests, and an iTransformer case study, and the diversification-loss theorem (Theorem 2) is an independent AM-GM argument. Those parts are not circular and would remain evidence even if the theory were removed. The self-citations to the authors' prior DPP/entropy-estimation work are not load-bearing for the main claim. Because the paper's headline theoretical guarantee is a tautology rather than a substantive result, while the empirical method is independently testable, the appropriate score is 7: partial circularity in the central claim, not a fully fabricated derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The paper's central claim rests on two classes of premises: (a) standard information-theoretic identities (conditional entropy monotonicity, MMSE bounds), which are true but are not shown to refer to the trained augmented model; and (b) empirical choices (K=3, lambda=0.1, Gaussian entropy proxy) that are tuned or assumed for the benchmarks. The theoretical contribution therefore supplies no non-tautological constraint on the model's actual MSE, leaving the empirical tables as the only support.

free parameters (4)
  • Number of Sequence Complementors K = 3
    Selected from an ablation (Fig. 4) on four test datasets; fixed to 3 for all experiments. Tuning on evaluation benchmarks is a mild form of data-dependent selection.
  • Diversification weight lambda_dcs = 0.1
    Set by the authors for all experiments; no sensitivity analysis is provided.
  • Encoder depth and embedding dimension = 2 blocks, 512 dims (long-term)
    The paper says these are searched over {1,2,3} and {128,512}, then fixed globally; the selection criterion is not stated.
  • Entropy estimate regularizer epsilon = Not specified
    Used in Eq. (3) and Eq. (14) to avoid non-positive log arguments; value not reported.
assumptions (5)
  • domain assumption The learned representation Z_enc and the target Y follow Gaussian distributions (used in Lemma 1).
    Needed for the MMSE bound in Eq. (11); no evidence is provided that features or targets are Gaussian.
  • ad hoc to paper The conditional entropy inequalities H(Y|Z_enc,S) <= H(Y|Z_enc) apply to the trained augmented network, treating S as a random variable.
    The paper's own model (Eq. 7) defines Z_enc as the encoder output on the concatenated input, not the old representation plus S. The theory never establishes this link.
  • ad hoc to paper H(Z_enc) <= H(Z_enc,S) implies the augmented model's representation has higher entropy.
    The joint entropy of the old representation and S does not upper-bound the entropy of the new representation produced by the augmented network.
  • ad hoc to paper The volume-maximization loss promotes orthogonality under the constraint ||S_i||=1, which is enforced in training.
    Theorem 2 requires the unit-norm constraint, but Algorithm 1 does not show normalization or projection.
  • domain assumption The Gaussian entropy estimate (Eq. 3) with a finite sample and identity-regularized covariance is a faithful proxy for representation diversity.
    Used to measure feature richness; the paper acknowledges in the Limitation section that comparing features across dimensions and samples is challenging.
invented entities (1)
  • Sequence Complementor S (learnable sequences)
    purpose: Concatenated to input patches so self-attention can use them as extra context, intended to enrich the latent representation.
    The only evidence for its usefulness is the paper's own benchmarks; no external falsifiable prediction outside the reported experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sequence Complementor: Complementing Transformers For Time Series Forecasting with Learnable Sequences." pith.science (2026). https://pith.science/paper/NLLSEMYF

@misc{pith2026250102735,
  author       = {Pith},
  title        = {Pith review of: Sequence Complementor: Complementing Transformers For Time Series Forecasting with Learnable Sequences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NLLSEMYF}},
  note         = {Machine review of arXiv:2501.02735}
}
read the original abstract

Since its introduction, the transformer has shifted the development trajectory away from traditional models (e.g., RNN, MLP) in time series forecasting, which is attributed to its ability to capture global dependencies within temporal tokens. Follow-up studies have largely involved altering the tokenization and self-attention modules to better adapt Transformers for addressing special challenges like non-stationarity, channel-wise dependency, and variable correlation in time series. However, we found that the expressive capability of sequence representation is a key factor influencing Transformer performance in time forecasting after investigating several representative methods, where there is an almost linear relationship between sequence representation entropy and mean square error, with more diverse representations performing better. In this paper, we propose a novel attention mechanism with Sequence Complementors and prove feasible from an information theory perspective, where these learnable sequences are able to provide complementary information beyond current input to feed attention. We further enhance the Sequence Complementors via a diversification loss that is theoretically covered. The empirical evaluation of both long-term and short-term forecasting has confirmed its superiority over the recent state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2501.02735 by the authors.

Figure 1
Figure 1. The vanilla self-attention mechanism v.s. the self [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The analysis of transformers for time series forecasting: (a) the correlation of the learned latent representations from [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Ablation studies on the number of learnable Sequence Complementors and the diversified Sequence Complementors on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: The qualitative results on ETTh2 Dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 5
Figure 5. Figure 5: The comparison of training dynamics with and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 19 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bengio, Y.; Courville, A.; and Vincent, P. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8): 1798--1828

    2013

  4. [4]

    Bhojanapalli, S.; Chakrabarti, A.; Veit, A.; Lukasik, M.; Jain, H.; Liu, F.; Chang, Y.-W.; and Kumar, S. 2021. Leveraging redundancy in attention with reuse transformers. arXiv preprint arXiv:2110.06821

    arXiv 2021

  5. [5]

    R.; Chen, M.; Rodrigues, M

    Carson, W. R.; Chen, M.; Rodrigues, M. R.; Calderbank, R.; and Carin, L. 2012. Communications-inspired projection design with application to compressive sensing. SIAM Journal on Imaging Sciences, 5(4): 1185--1212

    work page 2012

  6. [6]

    O.; Yoder, N

    Chen, S.-A.; Li, C.-L.; Arik, S. O.; Yoder, N. C.; and Pfister, T. 2023. TSM ixer: An All- MLP Architecture for Time Series Forecast-ing. Transactions on Machine Learning Research

    work page 2023

  7. [7]

    Chen, X.; Li, H.; Amin, R.; and Razi, A. 2024. Learning on Bandwidth Constrained Multi-Source Data with MIMO-inspired DPP MAP Inference. IEEE Transactions on Machine Learning in Communications and Networking

    work page 2024

  8. [8]

    Chen, X.; Li, H.; Qiu, P.; Zhu, W.; Amin, R.; and Razi, A. 2025. Rd-dpp: Rate-distortion theory meets determinantal point process to diversify learning data samples. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    work page 2025

Show all 46 references
  1. [9]

    Cover, T. M. 1999. Elements of information theory. John Wiley & Sons

    1999

  2. [10]

    Dai, Z.; Lai, G.; Yang, Y.; and Le, Q. 2020. Funnel-transformer: Filtering out sequential redundancy for efficient language processing. Advances in neural information processing systems, 33: 4271--4282

    2020

  3. [11]

    Dalvi, F.; Sajjad, H.; Durrani, N.; and Belinkov, Y. 2020. Analyzing redundancy in pretrained transformer models. arXiv preprint arXiv:2004.04010

    2020 arXiv

  4. [12]

    Dem s ar, J. 2006. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 7: 1--30

    2006

  5. [13]

    donghao, L.; and wang xue. 2024. Modern TCN : A Modern Pure Convolution Structure for General Time Series Analysis. In The Twelfth International Conference on Learning Representations

    2024

  6. [14]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  7. [15]

    Kim, T.; Kim, J.; Tae, Y.; Park, C.; Choi, J.-H.; and Choo, J. 2022. Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift. In International Conference on Learning Representations

    2022

  8. [16]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  9. [17]

    Kulesza, A.; Taskar, B.; et al. 2012. Determinantal point processes for machine learning. Foundations and Trends in Machine Learning , 5(2--3): 123--286

    2012

  10. [18]

    Lai, G.; Chang, W.-C.; Yang, Y.; and Liu, H. 2018. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, 95--104

    2018

  11. [19]

    LeCun, Y.; Bengio, Y.; and Hinton, G. 2015. Deep learning. nature, 521(7553): 436--444

    2015

  12. [20]

    Li, Z.; Qi, S.; Li, Y.; and Xu, Z. 2023. Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721

    2023 arXiv

  13. [21]

    Lin, S.; Lin, W.; Wu, W.; Chen, H.; and Yang, J. 2024. SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters. arXiv preprint arXiv:2405.00946

    2024 arXiv

  14. [22]

    Liu, M.; Zeng, A.; Chen, M.; Xu, Z.; Lai, Q.; Ma, L.; and Xu, Q. 2022 a . Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35: 5816--5828

    2022

  15. [23]

    X.; and Dustdar, S

    Liu, S.; Yu, H.; Liao, C.; Li, J.; Lin, W.; Liu, A. X.; and Dustdar, S. 2021. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations

    2021

  16. [24]

    Liu, Y.; Hu, T.; Zhang, H.; Wu, H.; Wang, S.; Ma, L.; and Long, M. 2024. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. In The Twelfth International Conference on Learning Representations

    2024

  17. [25]

    Liu, Y.; Wu, H.; Wang, J.; and Long, M. 2022 b . Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35: 9881--9893

    2022

  18. [26]

    Liu, Z.; Li, J.; Shen, Z.; Huang, G.; Yan, S.; and Zhang, C. 2017. Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE international conference on computer vision, 2736--2744

    2017

  19. [27]

    Ma, Y.; Derksen, H.; Hong, W.; and Wright, J. 2007. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE transactions on pattern analysis and machine intelligence, 29(9): 1546--1562

    2007

  20. [28]

    H.; Sinthong, P.; and Kalagnanam, J

    Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam, J. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In The Eleventh International Conference on Learning Representations

    2023

  21. [29]

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32

    2019

  22. [30]

    N.; and Agneeswaran, V

    Patro, B. N.; and Agneeswaran, V. S. 2024. Simba: Simplified mamba-based architecture for vision and multivariate time series. arXiv preprint arXiv:2403.15360

    2024 arXiv

  23. [31]

    Petit, C.; Roumy, A.; and Maugey, T. 2023. A Water-filling Algorithm Maximizing the Volume of Submatrices Above the Rank. In 2023 31st European Signal Processing Conference (EUSIPCO), 1295--1299. IEEE

    2023

  24. [32]

    Prasad, S. 2010. Certain relations between mutual information and fidelity of statistical estimation. arXiv preprint arXiv:1010.1508

    2010 arXiv

  25. [33]

    Steele, J. M. 2004. The Cauchy-Schwarz master class: an introduction to the art of mathematical inequalities. Cambridge University Press

    2004

  26. [34]

    Sun, C.; Shrivastava, A.; Singh, S.; and Gupta, A. 2017. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE international conference on computer vision, 843--852

    2017

  27. [35]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  28. [36]

    Y.; and ZHOU, J

    Wang, S.; Wu, H.; Shi, X.; Hu, T.; Luo, H.; Ma, L.; Zhang, J. Y.; and ZHOU, J. 2024 a . TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. In The Twelfth International Conference on Learning Representations

    2024

  29. [37]

    Wang, Y.; Wu, H.; Dong, J.; Liu, Y.; Long, M.; and Wang, J. 2024 b . Deep Time Series Models: A Comprehensive Survey and Benchmark. arXiv preprint arXiv:2407.13278

    2024 arXiv

  30. [38]

    Wen, W.; Wu, C.; Wang, Y.; Chen, Y.; and Li, H. 2016. Learning structured sparsity in deep neural networks. Advances in neural information processing systems, 29

    2016

  31. [39]

    Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; and Long, M. 2023. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In The Eleventh International Conference on Learning Representations

    2023

  32. [40]

    Wu, H.; Xu, J.; Wang, J.; and Long, M. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34: 22419--22430

    2021

  33. [41]

    Yu, Y.; Chan, K. H. R.; You, C.; Song, C.; and Ma, Y. 2020. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in neural information processing systems, 33: 9422--9434

    2020

  34. [42]

    Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2023. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, 11121--11128

    2023

  35. [43]

    Zhang, Y.; and Yan, J. 2023. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations

    2023

  36. [44]

    Zhao, C.; Ni, B.; Zhang, J.; Zhao, Q.; Zhang, W.; and Tian, Q. 2019. Variational convolutional neural network pruning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2780--2789

    2019

  37. [45]

    Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; and Zhang, W. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 11106--11115

    2021

  38. [46]

    Zhou, T.; Ma, Z.; Wen, Q.; Wang, X.; Sun, L.; and Jin, R. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, 27268--27286. PMLR

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.