EasyTime: Time Series Forecasting Made Easy

Aoying Zhou; Bin Yang; Chengcheng Yang; Chenjuan Guo; Christian S. Jensen; Jilin Hu; Liu Yang; Ruiyang Pang; Xiangfei Qiu; Xingjian Wu

REVIEW 3 major objections 6 minor 12 references

EasyTime: Time Series Forecasting Made Easy

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EasyTime claims that an automated ensemble of top-ranked forecasting methods beats every individual method on a new dataset.

desk verdict A clean demo system paper that integrates TFB, TS2Vec, and SimpleTS-style model selection with an LLM Q&A module, but its headline claim of superior ensemble accuracy is not supported by any experiment in the text. read the letter →

arxiv 2412.17603 v1 pith:4ZURZTQM submitted 2024-12-23 cs.LG stat.ML

Xiangfei Qiu , Xiuwen Li , Ruiyang Pang , Zhicheng Pan , Xingjian Wu , Liu Yang , Jilin Hu , Yang Shu

show 6 more authors

Xuesong Lu Chengcheng Yang Chenjuan Guo Aoying Zhou Christian S. Jensen Bin Yang

This is my paper · ORCID

classification cs.LGstat.ML

keywords timeseriesforecastingautomatedensemblemethodselectionbenchmarkingone-clickevaluationnaturallanguageQ&ArepresentationlearningTS2Vec

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EasyTime is a demonstration system that wraps an existing time-series benchmark into three services: one-click evaluation of a new forecasting method across thousands of benchmark series, an automated ensemble that selects and combines promising methods for a user-supplied dataset, and natural-language Q&A that turns questions about method performance into SQL queries and charts. The paper's central claim is that the Automated Ensemble module, which ranks methods with a pretrained classifier and learns combination weights on the user's own data, yields more accurate forecasts than any individual method. If true, practitioners would no longer need to hand-pick a forecasting method for each new dataset, and researchers could evaluate new methods under consistent, well-rounded conditions with minimal effort.

What carries the argument

The load-bearing mechanism is a two-stage offline/online pipeline. Offline, the system trains an unsupervised representation encoder (TS2Vec, a self-supervised model that maps a time series to a fixed vector) and then trains a classifier on the benchmark's 8,000-plus series using a soft-label loss, so the classifier outputs a probability ranking over candidate forecasting methods rather than a single label. Online, a new dataset is embedded by the same encoder, the classifier selects the top-k methods, those methods are trained on the training portion of the dataset, and ensemble weights are fitted on the validation portion. The pretrained ranking is what lets the system recommend methods for a dataset it has never seen.

What would settle it

Run the automated ensemble on several held-out datasets that are not part of the benchmark, under identical train/validation/test splits, and compare it with each individual method; if a single method wins on most datasets, the claim that the ensemble is superior to every individual method is false.

Watch

Extended reading notes

Core claim

The authors aim to show that a large body of offline benchmark results can be reused as knowledge for new forecasting tasks. When a new time series arrives, an unsupervised representation model extracts its features; a classifier trained with soft-label supervision on 8,000-plus benchmark series converts those features into a probability ranking of 30+ methods; the top-k methods are then trained on the new dataset and combined with weights learned on a validation split. The paper asserts that this automated ensemble achieves superior forecasting accuracy compared to any individual method, and that the same benchmark knowledge, exposed through one-click evaluation and natural-language queries, makes high-quality forecasting accessible to non-specialists.

Load-bearing premise

The entire approach depends on a classifier trained on the benchmark's 8,000-plus series being able to name, for a brand-new series it has never seen, which forecasting methods will do well; the paper reports no test of that ability.

Editorial extensions

If this is right

A practitioner with a new dataset can upload it and receive a forecast from an ensemble tailored to that dataset instead of relying on a single default method.
A researcher can add a new method to the pipeline once and have it evaluated across thousands of series under fixed and rolling forecasting strategies without writing evaluation code.
The accumulated benchmark results become queryable in plain language, so non-specialists can ask which methods work best for long-horizon or seasonal series and get chart-backed answers.
As more methods and datasets are added to the benchmark, the pretrained ranking and the ensemble both improve without any change to the pipeline itself.
If the ensemble claim holds, EasyTime offers an AutoML-style baseline for forecasting: automatic method selection plus ensembling replaces manual model search.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the pretrained ranking could support zero-shot method recommendation, skipping the training step on the new dataset and trading a little accuracy for much faster inference.
Beyond the paper, an explicit test on underrepresented dataset types would show whether the ensemble's gain comes from the ranking or from the validation-learned weights.
Beyond the paper, the Q&A module could become an interactive agent that asks about horizon and metric before querying, which the architecture already permits.
Beyond the paper, the pretrained classifier's ranking is tied to the methods inside the benchmark, so the system would need retraining to stay current as new forecasting methods appear.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

The takeaway in one line: EasyTime is a clean integration of existing pieces — TFB, TS2Vec, SimpleTS-style soft-label selection, and an LLM NL2SQL module — and it is a reasonable demo to show how these fit together. What it is not: a paper that demonstrates the automated ensemble's superiority claim anywhere in the text. The abstract says the ensemble "yields superior forecasting accuracy compared to individual methods," and Section II-C describes the mechanism, but there is no table, no baseline comparison, no error bar, no held-out evaluation. That is the load-bearing claim, and it is entirely unbacked. The stress-test note gets this right, and it holds up on reading.

What the paper does well: the architecture is clear and modular; the Q&A module has a sensible design with SQL verification and structured output; the demonstration walkthrough (S1–S3) is concrete enough that a user could reproduce the intended flow. The one-click evaluation over TFB is useful if you already trust TFB's pipeline. Credit where due: the integration work is non-trivial and the paper is honest about which components are reused — TFB [5], TS2Vec [11], SimpleTS [10] are all cited. No invented entities, no parameter fudging; the only free parameter is top-k, which is reasonable.

The soft spots: First, the ensemble claim. No evidence that a classifier trained on TFB's 8,000+ univariate series transfers to a new dataset, let alone that the resulting ensemble beats the best individual method. The paper also advertises multivariate support in TFB (25 multivariate datasets), but the pretrained classifier is described as trained on UTSF methods on univariate series; the demo screenshot shows a univariate example. If a user uploads multivariate data, the feature/ranking pipeline may not apply. That scope ambiguity should be stated and either evaluated or explicitly excluded. Second, the Q&A module has no accuracy evaluation — how often does the NL2SQL generation produce correct SQL? That is a minor omission for a demo, but the authors claim "ensures accuracy and reliability," which is stronger than what a screenshot supports.

Overall verdict: for a demonstration-track paper, this is probably acceptable if the abstract is toned down to describe the system rather than assert unmeasured superiority. As a research contribution to forecasting science, it adds no new algorithm or empirical finding, and I would not cite it for results. But it is a well-constructed system demo that a serious referee could reasonably send to revision with the request to either add a small evaluation of the ensemble transfer or soften the claim. So yes, I would send it to peer review, with the evaluation requirement as the main condition.

The serious_thinker flag: yes — the work is coherent, honest about its components, and free of internal contradictions. It just does not back its main headline.

For your reading group: maybe, as a case study in how demo papers should handle claimed performance. Not essential.

Referee Report

3 major / 6 minor

Summary. The paper presents EasyTime, a demonstration system built on top of the TFB benchmark, with three advertised capabilities: one-click evaluation of forecasting methods, an Automated Ensemble module that recommends and combines methods for a new dataset, and a natural-language Q&A module that answers time-series forecasting questions by converting queries into SQL over benchmark results. The Automated Ensemble module uses a pretrained TS2Vec encoder and a soft-label classifier, trained on TFB's evaluations of 30+ methods on 8,000+ series, to rank methods for a new dataset, then trains the top-k candidates and learns ensemble weights on the validation split. The abstract claims that the Automated Ensemble yields superior forecasting accuracy compared to individual methods, but the manuscript reports no experimental evaluation of this claim. The Q&A module is described with a workflow and a single illustrative SQL query example.

Significance. If the accuracy claim were validated, EasyTime would provide a practical tool for practitioners to select and combine forecasting methods without deep expertise. The paper's strengths are its integration of a mature benchmark (TFB) with a consistent evaluation pipeline, the adoption of an unsupervised representation model (TS2Vec), the use of a soft-label classifier for method ranking, and a Q&A module that verifies SQL statements before execution. However, the central claim of the paper is entirely unvalidated: no experiment, table, or baseline comparison appears anywhere in the manuscript. The system's value proposition, as stated, depends on the empirical transfer of the pretrained classifier to new datasets, and that transfer is not demonstrated. The manuscript would be acceptable as a demonstration if the accuracy claim were softened or if quantitative evidence were added; as written, the claim is a factual assertion that requires support. The paper does not provide machine-checked proofs or reproducible evaluation code, so the credibility of the main claim rests solely on the missing experiments.

major comments (3)

[Abstract; Section II-C; Section III-S2] The abstract's central claim that the Automated Ensemble module 'yields superior forecasting accuracy compared to individual methods' is not supported by any experiment, table, or baseline comparison in the manuscript. Section II-C describes the offline pretraining and online inference procedure but reports no accuracy numbers, no held-out evaluation, no comparison against the best individual method, and no error analysis. Section III-S2 and Figure 4 show only a single illustrative screenshot of the UI and do not provide aggregate statistics. Because the system's stated value rests entirely on this empirical transfer, the authors must add an evaluation: at minimum, compare the ensemble against each individual method on held-out TFB series or external datasets, with error bars and, ideally, statistical significance tests.
[Section II-C; Section I; Figure 5] The classifier is described as trained on evaluations of 30+ UTSF methods on the 8,000+ univariate series in TFB, but the paper advertises EasyTime for datasets generally and the Q&A example in Figure 5 queries multivariate datasets (if_univariate = 0). It is unclear whether the Automated Ensemble module is restricted to univariate series. If it is restricted, the claim should be explicitly scoped to univariate forecasting; if it is not restricted, the transfer of the pretrained feature space and soft-label classifier to multivariate series must be justified and evaluated. Please clarify the intended input regime and provide evidence for transfer within that regime.
[Section II-C; Section I] The knowledge base used for both classifier training and the Q&A module consists of the TFB benchmark results of reference [5], which come from the same research group; the paper presents no independent evaluation against external benchmarks. This is not by itself an error, but it means that the 'superior accuracy' claim is currently supported only by in-group benchmark results. An evaluation on external datasets, or at least on a held-out split of TFB with hyperparameter tuning separated from the training data, would mitigate the circularity concern.

minor comments (6)

[Section I; Figure 3] The text contains typos: 'langauge' (Section I) and 'infomations' (Figure 3) should be 'language' and 'information'.
[Section I] The phrase 'active field of research field that has produced' appears to be a duplication; it should be 'active research field that has produced'.
[Section III-S3] 'TFS methods' should be 'TSF methods'.
[Section III (closing paragraph)] 'shed lights' should be 'shed light'.
[Figure 5] The model name 'Nostationary' is inconsistent with the usual spelling 'Nonstationary'; please unify the spelling across the figure and text.
[Section II-A; Section II-C] The paper states that TFB includes 25 multivariate datasets and 8,068 univariate datasets, but the Automated Ensemble is trained only on univariate evaluations; the reader would benefit from an explicit statement about which data types the one-click evaluation and the automated ensemble support.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: the Automated Ensemble's transfer claim is untested but not equivalent to its inputs.

full rationale

The paper contains no derivation chain that reduces to its own inputs by construction. The Automated Ensemble module trains a TS2Vec representation and a soft-label classifier on benchmark evaluations from TFB, then uses the pretrained classifier to recommend top-k methods for a new dataset and learns ensemble weights on the validation part. This is an empirical supervised-learning pipeline: the trained classifier is a fitted model, and the paper makes no equation-level claim that the ensemble must beat individual methods by construction. The abstract's claim that the ensemble 'yields superior forecasting accuracy compared to individual methods' is not supported by any reported held-out evaluation, but absence of evidence is a correctness or completeness concern, not a circularity concern under the specified criteria. The Q&A module translates natural language to SQL over the stored TFB benchmark results; presenting stored results is retrieval, not a prediction derived from itself. The reliance on TFB [5] is a normal citation to prior work by overlapping authors; TFB is a public benchmark with code, and the paper does not invoke a self-cited uniqueness theorem or an unverified claim to forbid alternatives. No fitted parameter is renamed as a prediction, and no known result is repackaged as organization or unification. Accordingly, no specific circular step can be quoted and exhibited, and the honest finding is no significant circularity with score 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper reports no fitted numerical constants. The central claim depends on the transferability of a classifier trained on TFB results to new datasets, on the consistency of TFB's benchmark results, and on the reliability of validation-based ensemble weighting. No new entities are postulated.

free parameters (1)

top-k (number of recommended methods)
The automated ensemble selects the top-k methods from the classifier's ranking (Section II-C). The value of k is not specified in the paper and affects ensemble composition and accuracy.

assumptions (3)

domain assumption Time series features extracted by TS2Vec are predictive of forecasting method performance.
The automated ensemble in Section II-C assumes that a classifier over TS2Vec embeddings, trained on TFB benchmark results, can rank methods for a new dataset.
domain assumption Benchmark results from TFB are consistent and fair enough to serve as ground truth for method ranking.
The system builds its knowledge base on TFB's evaluation results (Section II-A), assuming these are correct and unbiased across datasets and methods.
domain assumption Validation performance on dataset X is a reliable guide for final ensemble weights on the test portion.
In Section II-C, ensemble weights are learned on the validation part of X and then used for forecasting, assuming this generalizes.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EasyTime: Time Series Forecasting Made Easy." pith.science (2026). https://pith.science/paper/4ZURZTQM

@misc{pith2026241217603,
  author       = {Pith},
  title        = {Pith review of: EasyTime: Time Series Forecasting Made Easy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ZURZTQM}},
  note         = {Machine review of arXiv:2412.17603}
}

read the original abstract

Time series forecasting has important applications across diverse domains. EasyTime, the system we demonstrate, facilitates easy use of time-series forecasting methods by researchers and practitioners alike. First, EasyTime enables one-click evaluation, enabling researchers to evaluate new forecasting methods using the suite of diverse time series datasets collected in the preexisting time series forecasting benchmark (TFB). This is achieved by leveraging TFB's flexible and consistent evaluation pipeline. Second, when practitioners must perform forecasting on a new dataset, a nontrivial first step is often to find an appropriate forecasting method. EasyTime provides an Automated Ensemble module that combines the promising forecasting methods to yield superior forecasting accuracy compared to individual methods. Third, EasyTime offers a natural language Q&A module leveraging large language models. Given a question like "Which method is best for long term forecasting on time series with strong seasonality?", EasyTime converts the question into SQL queries on the database of results obtained by TFB and then returns an answer in natural language and charts. By demonstrating EasyTime, we intend to show how it is possible to simplify the use of time series forecasting and to offer better support for the development of new generations of time series forecasting methods.

Figures

Figures reproduced from arXiv: 2412.17603 by the authors.

**Figure 1.** EasyTime Overview. mark [5], which incorporates a suite of datasets spanning dozens of application domains to represent dataset characteristics comprehensively; and it encompasses a diverse range of state-of-the-art methods that include statistical learning, machine learning, and deep learning methods. It also provides support for different evaluation strategies and metrics thus enabling comprehensive evaluation of… view at source ↗

**Figure 2.** Automated Ensemble Overview. forecasts on univariate datasets, they just need to modify the system’s configuration file information. This includes changing the evaluation strategy to rolling forecasting, adjusting the forecasting horizons, etc. Then, they can utilize the one-click evaluation. C. Automated Ensemble EasyTime employs the Automated Ensemble Module to help practitioners, e.g., data scientists, build an a… view at source ↗

**Figure 4.** An Example of Method Recommendation, Automated Ensembles, and Forecasts Visualizations. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: An Example of EasyTime Q&A. can fit the selected time series the best automatically. Finally, the visualizations and evaluation metrics of forecasts on both ensemble and chosen methods are displayed (see labels ❾ and ❿), respectively. S3. Natural Language Q&A. We demon…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

[5]

X. Qiu, J. Hu, L. Zhou, X. Wu, J. Du, B. Zhang, C. Guo, A. Zhou, C. S. Jensen, Z. Sheng, and B. Yang, ``Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods,'' Proc. VLDB Endow. , vol. 17, no. 9, pp. 2363--2377, 2024

work page 2024
[11]

Z. Yue, Y. Wang, J. Duan, T. Yang, C. Huang, Y. Tong, and B. Xu, `` TS2Vec : Towards universal representation of time series,'' in AAAI, vol. 36, no. 8, 2022, pp. 8980--8987

work page 2022
[10]

Y. Yao, D. Li, H. Jie, L. Chen, T. Li, J. Chen, J. Wang, F. Li, and Y. Gao, ``Simplets: An efficient and universal model selection framework for time series forecasting,'' Proceedings of the VLDB Endowment, vol. 16, no. 12, pp. 3741--3753, 2023

work page 2023
[1]

H. Gao, W. Shen, X. Qiu, R. Xu, J. Hu, and B. Yang, ``Diffimp: Efficient diffusion model for probabilistic time series imputation with bidirectional mamba backbone,'' arXiv preprint arXiv:2410.13338, 2024

arXiv 2024
[2]

Herzen, F

J. Herzen, F. L \"a ssig, S. G. Piazzetta, T. Neuer, L. Tafti, G. Raille, T. Van Pottelbergh, M. Pasieka, A. Skrodzki, N. Huguenin et al., ``Darts: User-friendly modern machine learning for time series,'' The Journal of Machine Learning Research, vol. 23, no. 1, pp. 5442--5447, 2022

work page 2022
[3]

S. Hu, K. Zhao, X. Qiu, Y. Shu, J. Hu, B. Yang, and C. Guo, ``Multirc: Joint learning for time series anomaly prediction and detection with multi-scale reconstructive contrast,'' arXiv preprint arXiv:2410.15997, 2024

arXiv 2024
[4]

Z. Li, X. Qiu, P. Chen, Y. Wang, H. Cheng, Y. Shu, J. Hu, C. Guo, A. Zhou, Q. Wen et al., ``Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting,'' arXiv preprint arXiv:2410.11802, 2024

arXiv
[6]

X. Qiu, X. Wu, Y. Lin, C. Guo, J. Hu, and B. Yang, ``Duet: Dual clustering enhanced multivariate time series forecasting,'' in SIGKDD, 2025

work page 2025

Show all 12 references

[7]

H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long, ``Timesnet: Temporal 2d-variation modeling for general time series analysis,'' in ICLR, 2023

2023
[8]

X. Wu, X. Qiu, Z. Li, Y. Wang, J. Hu, C. Guo, H. Xiong, and B. Yang, ``Catch: Channel-aware multivariate time series anomaly detection via frequency patching,'' arXiv preprint arXiv:2410.12261, 2024

2024 arXiv
[9]

X. Wu, X. Wu, B. Yang, L. Zhou, C. Guo, X. Qiu, J. Hu, Z. Sheng, and C. S. Jensen, ``Autocts++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting,'' The VLDB Journal, vol. 33, no. 5, pp. 1743--1770, 2024

2024
[12]

Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [5]

X. Qiu, J. Hu, L. Zhou, X. Wu, J. Du, B. Zhang, C. Guo, A. Zhou, C. S. Jensen, Z. Sheng, and B. Yang, ``Tfb: Towards comprehensive and fair benchmarking of time series forecasting methods,'' Proc. VLDB Endow. , vol. 17, no. 9, pp. 2363--2377, 2024

work page 2024

[2] [11]

Z. Yue, Y. Wang, J. Duan, T. Yang, C. Huang, Y. Tong, and B. Xu, `` TS2Vec : Towards universal representation of time series,'' in AAAI, vol. 36, no. 8, 2022, pp. 8980--8987

work page 2022

[3] [10]

Y. Yao, D. Li, H. Jie, L. Chen, T. Li, J. Chen, J. Wang, F. Li, and Y. Gao, ``Simplets: An efficient and universal model selection framework for time series forecasting,'' Proceedings of the VLDB Endowment, vol. 16, no. 12, pp. 3741--3753, 2023

work page 2023

[4] [1]

H. Gao, W. Shen, X. Qiu, R. Xu, J. Hu, and B. Yang, ``Diffimp: Efficient diffusion model for probabilistic time series imputation with bidirectional mamba backbone,'' arXiv preprint arXiv:2410.13338, 2024

arXiv 2024

[5] [2]

Herzen, F

J. Herzen, F. L \"a ssig, S. G. Piazzetta, T. Neuer, L. Tafti, G. Raille, T. Van Pottelbergh, M. Pasieka, A. Skrodzki, N. Huguenin et al., ``Darts: User-friendly modern machine learning for time series,'' The Journal of Machine Learning Research, vol. 23, no. 1, pp. 5442--5447, 2022

work page 2022

[6] [3]

S. Hu, K. Zhao, X. Qiu, Y. Shu, J. Hu, B. Yang, and C. Guo, ``Multirc: Joint learning for time series anomaly prediction and detection with multi-scale reconstructive contrast,'' arXiv preprint arXiv:2410.15997, 2024

arXiv 2024

[7] [4]

Z. Li, X. Qiu, P. Chen, Y. Wang, H. Cheng, Y. Shu, J. Hu, C. Guo, A. Zhou, Q. Wen et al., ``Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting,'' arXiv preprint arXiv:2410.11802, 2024

arXiv

[8] [6]

X. Qiu, X. Wu, Y. Lin, C. Guo, J. Hu, and B. Yang, ``Duet: Dual clustering enhanced multivariate time series forecasting,'' in SIGKDD, 2025

work page 2025