Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that quantizing wavelet coefficients into a 1,024-token vocabulary gives pretrained time-series forecasters better accuracy and zero-shot generalization than scalar quantization or patching, with the best average rank…

desk verdict A genuinely new wavelet tokenizer for time series foundation models with a strong zero-shot benchmark, but the DWT boundary handling is unspecified and the in-domain superiority claim is not supported by the paper's own tables. read the letter →

arxiv 2412.05244 v1 pith:S56DRMWI submitted 2024-12-06 cs.LG cs.AI

Luca Masserano , Abdul Fatir Ansari , Boran Han , Xiyuan Zhang , Christos Faloutsos , Michael W. Mahoney , Andrew Gordon Wilson , Youngsuk Park
show 3 more authors
Syama Rangapuram Danielle C. Maddix Yuyang Wang
This is my paper · ORCID
classification cs.LGcs.AI MSC 42C4062M1068T07
keywords wavelettokenizationtimeseriesforecastingfoundationmodelsdiscretetransformquantizationzero-shotgeneralizationautoregressivetransformerT5encoder-decoder
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to answer what the right discrete "language" is for feeding real-valued time series to a pretrained transformer. It argues that quantized wavelet coefficients form that language: a 1,024-token vocabulary that separates coarse trends from fine detail, so an autoregressive T5 model can learn to forecast coefficients from coarse to fine. If the claim holds, wavelet-based tokenization beats scalar quantization (Chronos) and patching (TimesFM, Moirai) in both accuracy and zero-shot generalization, with the best average rank on 42 datasets across three metrics. A sympathetic reading is that the paper aims to establish wavelets as a more information-efficient tokenizer for general-purpose forecasting models.

What carries the argument

The load-bearing object is the maximally decimated discrete wavelet transform (DWT), a filter-bank decomposition that maps a length-$N$ signal to $N$ coefficients split into one approximation band $\{a_k\}_J$ and detail bands $\{d_k\}_j$, $j=1,\dots,J$, concentrating signal energy in few coefficients. Around it sits a pipeline: z-score scaling (chosen because the DWT is not translation-invariant), optional coefficient thresholding (VisuShrink, CDF, FDRC, or none—none wins in the final 8-GPU runs), quantization to bins sized by the Freedman–Diaconis rule, and concatenation in coarse-to-fine order $[a_J,d_J,\dots,d_1]$ so the model predicts from coarser to finer resolutions. This machinery turns a continuous series into a token stream that preserves autoregressive structure within each band and gives the cross-attention a natural quadrant structure.

What would settle it

On a test set of high-dynamic-range series dominated by sparse sharp spikes, reconstruct the context series by quantizing its DWT coefficients to the 1,024-bin codebook, mapping to bin centers, and applying the inverse DWT; measure the relative reconstruction error. If the error is large for the very patterns the paper highlights, the fidelity assumption is broken. A second check: vary vocabulary size upward; the paper predicts accuracy flattens beyond 1,024, so a benchmark where larger codebooks keep improving would contradict the compression claim.

Watch

Extended reading notes

Core claim

WaveToken's central claim is that tokenizing time series in the space of time-localized frequencies—via a maximally decimated discrete wavelet transform followed by thresholding, Freedman–Diaconis binning, and coarse-to-fine concatenation—yields a compact yet expressive codebook that simplifies next-token prediction. The paper reports that WaveToken outperforms existing pretrained forecasters (including Chronos, TimesFM, Moirai, and Lag-LLama) and performs on par or better than task-specific deep learning models, while using only 1,024 tokens, one quarter of Chronos's vocabulary. It also claims best average rank across all datasets for weighted quantile loss, mean absolute scaled error, and visual relative squared error, and shows qualitative mastery of exponential trends, sparse spikes, and non-stationary multi-frequency signals that competing models miss.

Load-bearing premise

The pipeline assumes that mapping predicted coefficient tokens to bin centers and running the inverse discrete wavelet transform reconstructs the future time series faithfully enough for accurate forecasts; the paper gives no bound on how quantization error or autoregressive coefficient errors amplify through the inverse transform, and it does not explicitly control boundary effects where the context coefficients meet the horizon coefficients.

Editorial extensions

If this is right

  • Pretrained forecasting can run with a 1,024-token codebook instead of 4,096, shrinking embedding and softmax costs without sacrificing accuracy.
  • Zero-shot generalization improves: WaveToken achieves the best average rank across all 42 datasets on WQL, MASE, and VRSE, including on datasets never seen in training.
  • The model captures exponential trends, sparse spikes, and time-varying frequencies that scalar-quantized and patch-based foundation models systematically miss.
  • Coarse-to-fine coefficient ordering induces interpretable attention structure: the decoder attends to approximation coefficients when predicting approximations and to detail coefficients when predicting details.
  • The tokenizer is architecture-agnostic enough to pair with any autoregressive transformer trained by next-token prediction, not just the T5 variant used here.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because dequantization to bin centers is a form of denoising, the model may be implicitly learning a smoothed forecast; an explicit reconstruction-error-aware training loss could push accuracy further than cross-entropy alone.
  • The DWT's O(N) cost means token counts need not grow with context length, so wavelet tokenization is a natural route to long-context forecasting without increasing sequence length—a direction the paper leaves open.
  • Continuous signals with naturally sparse high-frequency content—audio, ECG, sensor streams—are plausible next targets; the compression argument transfers whenever a wavelet basis concentrates energy.
  • The reported VRSE advantage may be partly baked into the design, since VRSE rewards matching the frequency-domain shape that wavelet coefficients encode; re-ranking on pointwise metrics alone would test how much of the gain is frequency-specific.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper proposes WaveToken, a wavelet-based tokenizer for time-series foundation models. The pipeline rescales each input series, applies a maximally decimated discrete wavelet transform (DWT), optionally thresholds the detail coefficients, quantizes all coefficients into a shared 1024-token vocabulary via Freedman-Diaconis binning, and concatenates approximation and detail coefficient tokens for autoregressive training of a T5 encoder-decoder. At inference, autoregressively sampled tokens are dequantized to bin centers and mapped back to the time domain by the inverse DWT. The method is evaluated on the Chronos benchmark collection: 15 in-domain and 27 zero-shot datasets, against statistical, task-specific deep-learning, and pretrained time-series foundation baselines, using WQL, MASE, and VRSE. The paper reports the best average rank for WaveToken across both benchmarks and all three metrics, along with ablations of vocabulary size, wavelet family, decomposition level, and thresholding, and a qualitative cross-attention analysis on trends, spikes, and non-stationary signals.

Significance. If the reported results hold, WaveToken would be a valuable demonstration that a compact wavelet-coefficient vocabulary can match or beat much larger scalar-quantization vocabularies and patch-based tokenizations for time-series forecasting, with particularly attractive zero-shot behavior. The evaluation is unusually thorough: raw per-dataset tables, three complementary metrics, ablations, a long-horizon extension, and attention analyses are included. The central modeling assumption, however, is the faithful invertibility of the coefficient language under the context/horizon split used at inference, and that assumption is not tested; the paper also omits the boundary conditions needed to make the reconstruction well-defined. The in-domain aggregate scores do not support the unqualified 'better accuracy' phrasing in the abstract. With these two issues addressed, the contribution would be a solid empirical advance in time-series tokenization.

major comments (2)
  1. [Section 3.3] Section 3.3 defines the training target as horizon coefficient tokens z_{C+1:C+H} and reconstructs forecasts by applying the IDWT to dequantized predicted coefficients, but the manuscript never specifies how the horizon coefficients are computed for training, which boundary conditions are used in the DWT/IDWT, or why the IDWT of context coefficients from x_{1:C} concatenated with independently predicted horizon coefficients is a valid wavelet representation of a length-(C+H) series. Since the maximally decimated DWT is a global filter-bank transform, coefficients at the context/horizon boundary depend on samples on both sides of the split, and an arbitrary concatenation of coefficient groups need not correspond to any real-valued series. This is load-bearing: if the IDWT introduces boundary artifacts, the benchmark numbers could reflect reconstruction bias rather than learned forecasting. A minimal fix is to add an oracle experiment that feeds ground-truth horizon coefficients through the proposed dequantization/IDWT pipeline and reports reconstruction error, and to specify the boundary mode (e.g., periodic, symmetric, zero) and the coefficient-count mapping from horizon length H to tokens at the chosen decomposition level.
  2. [Benchmark I, Tables 1-2 and Figure 3] On the in-domain benchmark, WaveToken-Large's aggregate relative scores are 0.569 for WQL and 0.698 for MASE, while Chronos-Large achieves 0.564 and 0.695, respectively. The text in Section 4.2 acknowledges this exception, but the abstract and introduction state that WaveToken 'provides better accuracy than recently proposed foundation models' and 'always performs on-par or better than all other baselines.' That claim is too strong as written. I recommend restating the headline as 'competitive on WQL/MASE and better on VRSE and best average rank,' or reporting statistical significance of the aggregate differences.
minor comments (5)
  1. [Section 3.2, quantization formula] The notation q(w) = i 1{e_{i-1} ≤ w < e_i} is unusual: since q(w) should equal the bin index i when the condition holds, the indicator should be part of the definition rather than a multiplier. Please clarify.
  2. [Section 3.3, notation] The notation z1:C is used both for the coefficient-token vector and for a temporal indexing scheme, and z_{C+1:C+H} conflates coefficient positions with original time indices; this makes the loss equation hard to parse. Please use distinct notation for coefficient indices.
  3. [Section 4.4 and Figure 8] The thresholding ablation changes ranking between the 1-GPU and 8-GPU settings, and the final choice (no thresholding) is made on the same evaluation benchmarks used for the main comparisons; please report these hyperparameter selections as part of a validation procedure rather than on the test benchmarks.
  4. [Figures 3 and 4] The aggregate relative scores are reported without confidence intervals or standard errors, although the underlying results are averaged over three seeds; adding variability estimates would strengthen the comparison.
  5. [Section 4.3] The qualitative cross-attention analysis is suggestive but not evaluated quantitatively; consider adding a simple statistic, such as attention concentration at spike positions, so the claim that the model 'attends to the right coefficients' is testable.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central wavelet-tokenization claim is empirically benchmarked and does not reduce to its inputs; one minor self-citation to the Chronos framework is not load-bearing.

full rationale

WaveToken's claimed derivation is an empirical pipeline: scale, DWT, threshold, quantize, autoregressive T5 next-token prediction, dequantize, and IDWT. Each stage is defined independently of the forecasting targets: bin centers and edges come from the training-set coefficient distribution via the Freedman-Diaconis rule; the model predicts quantized horizon coefficients and those are mapped to bin centers before IDWT. No fitted parameter is renamed as a prediction. The paper's ablations select vocabulary size, wavelet family, decomposition level, and thresholding on the same evaluation collections, which is a selection-dependence caveat but not a circular reduction, because the zero-shot Benchmark II (27 datasets) provides external validation outside the training corpus. The only notable self-citation is the adoption of the T5 encoder-decoder and the Chronos pretraining/evaluation setup from Ansari et al. (2024), a prior work with overlapping AWS authors; that citation supports the architecture and data collection but is not the basis of the wavelet-tokenization claim, which is evaluated against independent baselines. The boundary-coherence issue for horizon DWT coefficients (context coefficients from x_{1:C} versus full-series coefficients) is a correctness risk in the inverse reconstruction, not a circularity, because the paper does not define the target tokens in terms of the predicted output.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on several design choices tuned on the evaluation benchmarks themselves, including vocabulary size, wavelet family, decomposition level, and bin range. These are free parameters rather than derived constants. No new physical or categorical entities are introduced; PAD and EOS are standard special tokens.

free parameters (5)
  • Quantization bin range = [-30, 30]
    Chosen empirically by scanning the training corpus (Section 4.4); the range sets the support of the shared vocabulary and affects the precision of coefficient tokens.
  • Vocabulary size = 1024
    Selected by ablation as the optimal value on in-domain and zero-shot benchmarks (Section 4.4); larger vocabularies were flat or worse.
  • Wavelet family = Biorthogonal-2.2
    Selected by ablation among tested families; claimed to reduce edge distortions due to its symmetric dual filters.
  • Decomposition level J = 1
    Selected by ablation; deeper levels gave more coefficient groups but made it harder for attention to identify the relevant ones.
  • Thresholding method = No-thresholding
    Ablation showed no-thresholding was best in the 8-GPU setting, although VisuShrink was best in the single-GPU setting; thresholding is implicitly handled by the zero bin in quantization.
assumptions (4)
  • standard math The maximally decimated DWT with the chosen boundary conditions is invertible and preserves signal energy (Parseval's theorem).
    Required to reconstruct the time domain from coefficients; stated in Section 3.1 and Appendix A.2.
  • domain assumption Quantizing coefficients to bin centers introduces reconstruction error small enough for forecasting quality.
    The paper does not bound this error; it relies on empirical validation (Section 3.2).
  • domain assumption Autoregressive prediction on concatenated coefficient groups is a learnable and stable objective.
    The paper argues the model learns to attend to relevant groups, shown via attention maps (Section 4.3), but provides no convergence or stability analysis.
  • domain assumption The pretraining corpus collected by Ansari et al. (2024) is adequate for learning a general purpose time series vocabulary.
    The paper reuses the Chronos dataset collection and augmentation scheme (Section 4.1), and this corpus is not independently scrutinized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization." pith.science (2026). https://pith.science/paper/S56DRMWI

@misc{pith2026241205244,
  author       = {Pith},
  title        = {Pith review of: Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S56DRMWI}},
  note         = {Machine review of arXiv:2412.05244}
}
read the original abstract

How to best develop foundational models for time series forecasting remains an important open question. Tokenization is a crucial consideration in this effort: what is an effective discrete vocabulary for a real-valued sequential input? To address this question, we develop WaveToken, a wavelet-based tokenizer that allows models to learn complex representations directly in the space of time-localized frequencies. Our method first scales and decomposes the input time series, then thresholds and quantizes the wavelet coefficients, and finally pre-trains an autoregressive model to forecast coefficients for the forecast horizon. By decomposing coarse and fine structures in the inputs, wavelets provide an eloquent and compact language for time series forecasting that simplifies learning. Empirical results on a comprehensive benchmark, including 42 datasets for both in-domain and zero-shot settings, show that WaveToken: i) provides better accuracy than recently proposed foundation models for forecasting while using a much smaller vocabulary (1024 tokens), and performs on par or better than modern deep learning models trained specifically on each dataset; and ii) exhibits superior generalization capabilities, achieving the best average rank across all datasets for three complementary metrics. In addition, we show that our method can easily capture complex temporal patterns of practical relevance that are challenging for other recent pre-trained models, including trends, sparse spikes, and non-stationary time series with varying frequencies evolving over time.

Figures

Figures reproduced from arXiv: 2412.05244 by the authors.

Figure 1
Figure 1. WaveToken-Base (199M parameters) provides excellent forecasts with very low uncer￾tainty. Performance of different foundation models for time series forecasting on complex patterns of practical relevance: Chronos-Base (201M), TimesFM (200M) and Moirai-Large (311M) struggle to capture exponential trends (top row), sparse spikes (second row), and non-stationary signals with 2 and 5 frequencies evolving over time (bott… view at source ↗
Figure 2
Figure 2. High-level depiction of our method. (Left) WaveToken first re-scales the input time series by computing x˜t = (xt −µ1:C )/σ1:C , then it applies the DWT and possibly thresholds the resulting detail coefficients to zero (red crosses). The wavelet coefficients are finally quantized to bins of optimal size given their empirical distribution, and then concatenated together (excluding the first J − 1 approximations). (Ri… view at source ↗
Figure 3
Figure 3. WaveToken performs on par or better than other baselines on in-domain datasets. Forecasting accuracy on Benchmark I in terms of WQL, MASE and VRSE. namely DeepAR (Salinas et al., 2020), PatchTST (Nie et al., 2022) and TFT (Lim et al., 2021); and ii) recently proposed foundation models for time series forecasting — namely, TimesFM (Das et al., 2023), Chronos Mini-Large (Ansari et al., 2024), Moirai Base & Large (Woo … view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: WaveToken performs on par or better even relative to task-specific models on zero￾shot datasets. Forecasting accuracy on Benchmark II in terms of WQL, MASE and VRSE. size of 1024, while Chronos, for example, uses four times as much tokens with |V| = 4096. This empirica…
Figure 5
Figure 5. Figure 5: Wavelet-based tokenization induces structured patterns in the cross-attention maps. Cross-attention weights for the eighth decoder layer when forecasting the spiky data of [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Effect of different vocabulary sizes, wavelet families and decomposition levels on downstream forecasting accuracy of WaveToken-Small. The optimal hyper-parameters were a single-level wavelet decomposition with a Biorthogonal-2.2 family and a vocabulary size of 1024. S…
Figure 7
Figure 7. Figure 7: (Left) Mother and father wavelets for the Haar and Biorthogonal-2.2 families. Note the dual structure of the latter, which uses two filters: one for decomposition and one for reconstruc￾tion. (Right) Example of discrete wavelet transform (DWT) applied to a time series …
Figure 8
Figure 8. Figure 8: Effect of the different thresholding techniques of Section [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Pitfall of standard evaluation metrics. RMSE and MASE fail to distinguish a poor constant forecast from a visually much more accurate shifted forecast. By comparing the amplitudes at all frequencies, VRSE captures the difference and assigns a lower score to the shifted…
Figure 10
Figure 10. Figure 10: WaveToken achieves best average ranks on in-domain datasets. Average rank of models on Benchmark I (in-domain) in terms of WQL, MASE and VRSE. 0 5 10 15 Avg. Rank (WQL) WaveToken (Large) TFT PatchTST TimesFM Chronos (Large) DeepAR WaveToken (Base) Chronos (Base) WaveT…
Figure 11
Figure 11. Figure 11: WaveToken achieves best average ranks on zero-shot datasets. Average rank of models on Benchmark II (zero-shot) in terms of WQL, MASE and VRSE. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Long-horizon benchmark constructed by increasing the forecast length of each dataset in Benchmark II (Zero-Shot) by a factor of 2 and 3. (Top row) Results with horizon multiplied by 2: WaveToken-Base outperforms other foundation models across all three metrics. (Botto…
Figure 13
Figure 13. Figure 13: Cross-attention maps for the first to fourth (from top) decoder layers of Chronos-Base [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Cross-attention maps for the fifth to eighth (from top) decoder layers of Chronos-Base [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Cross-attention maps for the ninth to twelfth (from top) decoder layers of Chronos-Base [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures

    astro-ph.IM 2026-06 unverdicted novelty 6.0 of 10

    SeismoGPT is a transformer autoregressive model achieving median normalized cross-correlation above 0.93 when forecasting synthetic three-component seismograms up to 240 s ahead from P- and S-wave context.

  2. JEPA for AI-Native 6G: Predictive Representations and Open Challenges

    cs.LG 2026-07 conditional novelty 5.5 of 10

    Wireless-aware JEPA pretraining with an auxiliary future beam-energy target improves label-efficient beam ranking and OOD robustness on synthetic mmWave data relative to generic JEPA, MAE, SimCLR and supervised scratch.

Reference graph

Works this paper leans on

65 extracted references · 28 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    L օx1?7 VI<5F mL G,+@ T jAF\

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv 1999

  5. [5]

    Adaptive thresholding of wavelet coefficients

    Felix Abramovich and Yoav Benjamini. Adaptive thresholding of wavelet coefficients. Computational Statistics & Data Analysis, 22 0 (4): 0 351--361, 1996

    work page 1996

  6. [6]

    Gluonts: Probabilistic and neural time series modeling in python

    Alexander Alexandrov, Konstantinos Benidis, Michael Bohlke-Schneider, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle C Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, et al. Gluonts: Probabilistic and neural time series modeling in python. Journal of Machine Learning Research, 21 0 (116): 0 1--6, 2020

    2020

  7. [7]

    Evaluating various tokenizers for arabic text classification

    Zaid Alyafeai, Maged S Al-shaibani, Mustafa Ghaleb, and Irfan Ahmad. Evaluating various tokenizers for arabic text classification. Neural Processing Letters, 55 0 (3): 0 2911--2933, 2023

    work page 2023

  8. [8]

    Chronos: Learning the language of time series

    Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.07815, 2024

    arXiv 2024

Show all 65 references
  1. [9]

    Deep learning for time series forecasting: Tutorial and literature survey

    Konstantinos Benidis, Syama Sundar Rangapuram, Valentin Flunkert, Yuyang Wang, Danielle Maddix, Caner Turkmen, Jan Gasthaus, Michael Bohlke-Schneider, David Salinas, Lorenzo Stella, et al. Deep learning for time series forecasting: Tutorial and literature survey. ACM Computing...

    2022

  2. [10]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020

    2005 arXiv

  3. [11]

    Adaptive wavelet thresholding for image denoising and compression

    S Grace Chang, Bin Yu, and Martin Vetterli. Adaptive wavelet thresholding for image denoising and compression. IEEE transactions on image processing, 9 0 (9): 0 1532--1546, 2000

    2000

  4. [12]

    Ecgbert: Understanding hidden language of ecgs with self-supervised representation learning

    Seokmin Choi, Sajad Mousavi, Phillip Si, Haben G Yhdego, Fatemeh Khadem, and Fatemeh Afghah. Ecgbert: Understanding hidden language of ecgs with self-supervised representation learning. arXiv preprint arXiv:2306.06340, 2023

    2023 arXiv

  5. [13]

    The jpeg2000 still image coding system: an overview

    Charilaos Christopoulos, Athanassios Skodras, and Touradj Ebrahimi. The jpeg2000 still image coding system: an overview. IEEE transactions on consumer electronics, 46 0 (4): 0 1103--1127, 2000

    2000

  6. [14]

    Getting the most out of your tokenizer for pre-training and domain adaptation

    Gautier Dagan, Gabriel Synnaeve, and Baptiste Roziere. Getting the most out of your tokenizer for pre-training and domain adaptation. In Proceedings of the 41st International Conference on Machine Learning, volume 235, pp.\ 9784--9805. PMLR, 2024

    2024

  7. [15]

    A decoder-only foundation model for time-series forecasting

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310.10688, 2023

    2023 arXiv

  8. [16]

    Ten lectures on wavelets

    Ingrid Daubechies. Ten lectures on wavelets. SIAM, 1992

    1992

  9. [17]

    De-noising by soft-thresholding

    David L Donoho. De-noising by soft-thresholding. IEEE transactions on information theory, 41 0 (3): 0 613--627, 1995

    1995

  10. [18]

    On the histogram as a density estimator: L 2 theory

    David Freedman and Persi Diaconis. On the histogram as a density estimator: L 2 theory. Zeitschrift f \"u r Wahrscheinlichkeitstheorie und verwandte Gebiete , 57 0 (4): 0 453--476, 1981

    1981

  11. [19]

    A new algorithm for data compression

    Philip Gage. A new algorithm for data compression. The C Users Journal, 12 0 (2): 0 23--38, 1994

    1994

  12. [20]

    Statsforecast: Lightning fast forecasting with statistical and econometric models

    Federico Garza, Max Mergenthaler Canseco, Cristian Chall \'u , and Kin G Olivares. Statsforecast: Lightning fast forecasting with statistical and econometric models. PyCon: Salt Lake City, UT, USA, 2022

    2022

  13. [21]

    xval: A continuous number encoding for large language models

    Siavash Golkar, Mariel Pettee, Michael Eickenberg, Alberto Bietti, Miles Cranmer, Geraud Krawezik, Francois Lanusse, Michael McCabe, Ruben Ohana, Liam Parker, et al. xval: A continuous number encoding for large language models. arXiv preprint arXiv:2310.02989, 2023

    arXiv

  14. [22]

    Masked particle modeling on sets: Towards self-supervised high energy physics foundation models

    Tobias Golling, Lukas Heinrich, Michael Kagan, Samuel Klein, Matthew Leigh, Margarita Osadchy, and John Andrew Raine. Masked particle modeling on sets: Towards self-supervised high energy physics foundation models. Machine Learning: Science and Technology, 5 0 (3): 0 035074, 2024

    2024

  15. [23]

    Moment: A family of open time-series foundation models

    Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: A family of open time-series foundation models. arXiv preprint arXiv:2402.03885, 2024

    2024 arXiv

  16. [24]

    Large language models are zero-shot time series forecasters

    Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024

    2024

  17. [25]

    Wavelet score-based generative modeling

    Florentin Guth, Simon Coste, Valentin De Bortoli, and Stephane Mallat. Wavelet score-based generative modeling. Advances in neural information processing systems, 35: 0 478--491, 2022

    2022

  18. [26]

    Zur theorie der orthogonalen funktionensysteme

    Alfred Haar. Zur theorie der orthogonalen funktionensysteme. Mathematische Annalen, 71 0 (1): 0 38--53, 1911

    1911

  19. [27]

    Another look at measures of forecast accuracy

    Rob J Hyndman and Anne B Koehler. Another look at measures of forecast accuracy. International journal of forecasting, 22 0 (4): 0 679--688, 2006

    2006

  20. [28]

    Time-llm: Time series forecasting by reprogramming large language models

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728, 2023

    arXiv

  21. [29]

    Quantile regression

    Roger Koenker and Kevin F Hallock. Quantile regression. Journal of economic perspectives, 15 0 (4): 0 143--156, 2001

    2001

  22. [30]

    An experimental review on deep learning architectures for time series forecasting

    Pedro Lara-Ben \' tez, Manuel Carranza-Garc \' a, and Jos \'e C Riquelme. An experimental review on deep learning architectures for time series forecasting. International journal of neural systems, 31 0 (03): 0 2130001, 2021

    2021

  23. [31]

    Shape and time distortion loss for training deep time series forecasting models

    Vincent Le Guen and Nicolas Thome. Shape and time distortion loss for training deep time series forecasting models. Advances in neural information processing systems, 32, 2019

    2019

  24. [32]

    Temporal fusion transformers for interpretable multi-horizon time series forecasting

    Bryan Lim, Sercan \"O Ar k, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37 0 (4): 0 1748--1764, 2021

    2021

  25. [33]

    A Wavelet Tour of Signal Processing: The Sparse Way

    St \'e phane Mallat. A Wavelet Tour of Signal Processing: The Sparse Way. Academic Press, 3rd edition, 2009

    2009

  26. [34]

    Multiresolution approximations and wavelet orthonormal bases of L ^2( R )

    Stephane G Mallat. Multiresolution approximations and wavelet orthonormal bases of L ^2( R ) . Transactions of the American mathematical society, 315 0 (1): 0 69--87, 1989

    1989

  27. [35]

    Wavelets are all you need for autoregressive image generation

    Wael Mattar, Idan Levy, Nir Sharon, and Shai Dekel. Wavelets are all you need for autoregressive image generation. arXiv preprint arXiv:2406.19997, 2024

    2024 arXiv

  28. [36]

    Wavelets and operators: volume 1

    Yves Meyer. Wavelets and operators: volume 1. Cambridge university press, 1992

    1992

  29. [37]

    Large language models as general pattern machines

    Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, and Andy Zeng. Large language models as general pattern machines. arXiv preprint arXiv:2307.04721, 2023

    2023 arXiv

  30. [38]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  31. [39]

    Oppenheim and Ronald W

    Alan V. Oppenheim and Ronald W. Schafer. Discrete-Time Signal Processing. Pearson, 2010

    2010

  32. [40]

    Adaptive, hands-off stream mining

    Spiros Papadimitriou, Anthony Brockwell, and Christos Faloutsos. Adaptive, hands-off stream mining. In Proceedings 2003 VLDB Conference, pp.\ 560--571. Elsevier, 2003

    2003

  33. [41]

    Difffind: Discovering differential equations from time series

    Lalithsai Posam, Shubhranshu Shekhar, Meng-Chieh Lee, and Christos Faloutsos. Difffind: Discovering differential equations from time series. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp.\ 175--187. Springer, 2024

    2024

  34. [42]

    Deep learning for audio signal processing

    Hendrik Purwins, Bo Li, Tuomas Virtanen, Jan Schl \"u ter, Shuo-Yiin Chang, and Tara Sainath. Deep learning for audio signal processing. IEEE Journal of Selected Topics in Signal Processing, 13 0 (2): 0 206--219, 2019

    2019

  35. [43]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140): 0 1--67, 2020

    2020

  36. [44]

    Lag-llama: Towards foundation models for time series forecasting

    Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Bilo s , Hena Ghonia, Nadhir Vincent Hassen, Anderson Schneider, et al. Lag-llama: Towards foundation models for time series forecasting. arXiv preprint arXiv:2310...

    arXiv

  37. [45]

    Deepar: Probabilistic forecasting with autoregressive recurrent networks

    David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting, 36 0 (3): 0 1181--1191, 2020

    2020

  38. [46]

    W-transformers: a wavelet-based transformer framework for univariate time series forecasting

    Lena Sasal, Tanujit Chakraborty, and Abdenour Hadid. W-transformers: a wavelet-based transformer framework for univariate time series forecasting. In 2022 21st IEEE international conference on machine learning and applications (ICMLA), pp.\ 671--676. IEEE, 2022

    2022

  39. [47]

    Tokenization counts: the impact of tokenization on arithmetic in frontier llms

    Aaditya K Singh and DJ Strouse. Tokenization counts: the impact of tokenization on arithmetic in frontier llms. arXiv preprint arXiv:2402.14903, 2024

    2024 arXiv

  40. [48]

    Wavelets and filter banks

    Gilbert Strang and Truong Nguyen. Wavelets and filter banks. SIAM, 1996

    1996

  41. [49]

    Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior

    Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior. Advances in Neural Information Processing Systems,...

    2024

  42. [50]

    Totem: Tokenized time series embeddings for general time series analysis

    Sabera Talukder, Yisong Yue, and Georgia Gkioxari. Totem: Tokenized time series embeddings for general time series analysis. arXiv preprint arXiv:2402.16412, 2024

    2024 arXiv

  43. [51]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905, 2024

    2024 arXiv

  44. [52]

    Juman++: A morphological analysis toolkit for scriptio continua

    Arseny Tolmachev, Daisuke Kawahara, and Sadao Kurohashi. Juman++: A morphological analysis toolkit for scriptio continua. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.\ 54--59, 2018

    2018

  45. [53]

    A practical guide to wavelet analysis

    Christopher Torrence and Gilbert P Compo. A practical guide to wavelet analysis. Bulletin of the American Meteorological society, 79 0 (1): 0 61--78, 1998

    1998

  46. [54]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  47. [55]

    A tutorial on modern lossy wavelet image compression: foundations of jpeg 2000

    Bryan E Usevitch. A tutorial on modern lossy wavelet image compression: foundations of jpeg 2000. IEEE signal processing magazine, 18 0 (5): 0 22--35, 2001

    2000

  48. [56]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

    2017

  49. [57]

    Wavelets and subband coding

    Martin Vetterli and Jelena Kovacevic. Wavelets and subband coding. Prentice-hall, 1995

    1995

  50. [58]

    Unified training of universal time series forecasting transformers

    Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. arXiv preprint arXiv:2402.02592, 2024

    2024 arXiv

  51. [59]

    Promptcast: A new prompt-based learning paradigm for time series forecasting

    Hao Xue and Flora D Salim. Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 2023

    2023

  52. [60]

    Soundstream: An end-to-end neural audio codec

    Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30: 0 495--507, 2021

    2021

  53. [61]

    First de-trend then attend: Rethinking attention for time-series forecasting

    Xiyuan Zhang, Xiaoyong Jin, Karthick Gopalswamy, Gaurav Gupta, Youngsuk Park, Xingjian Shi, Hao Wang, Danielle C Maddix, and Yuyang Wang. First de-trend then attend: Rethinking attention for time-series forecasting. arXiv preprint arXiv:2212.08151, 2022

    2022 arXiv

  54. [62]

    Large language models for time series: A survey

    Xiyuan Zhang, Ranak Roy Chowdhury, Rajesh K Gupta, and Jingbo Shang. Large language models for time series: A survey. arXiv preprint arXiv:2402.01801, 2024

    2024 arXiv

  55. [63]

    Outline, then details: Syntactically guided coarse-to-fine code generation

    Wenqing Zheng, SP Sharan, Ajay Kumar Jaiswal, Kevin Wang, Yihan Xi, Dejia Xu, and Zhangyang Wang. Outline, then details: Syntactically guided coarse-to-fine code generation. In International Conference on Machine Learning, pp.\ 42403--42419. PMLR, 2023

    2023

  56. [64]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pp.\ 27268--27286. PMLR, 2022

    2022

  57. [65]

    Wavelet-based image tokenizer for vision transformers

    Zhenhai Zhu and Radu Soricut. Wavelet-based image tokenizer for vision transformers. arXiv preprint arXiv:2405.18616, 2024

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.