Federated Foundation Models on Heterogeneous Time Series

Chengqi Zhang; Guodong Long; Jing Jiang; Shengchao Chen

REVIEW 4 major objections 6 minor 50 references

Federated Foundation Models on Heterogeneous Time Series

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training time series foundation models under federated learning—each dataset kept at its own client—generalizes better than centralized pretraining on the same data.

desk verdict First federated from-scratch TSFM training, with a sensible method but a checkable leakage gap and an inflated headline number. read the letter →

arxiv 2412.08906 v1 pith:CVNG3HJT submitted 2024-12-12 cs.LG

Shengchao Chen , Guodong Long , Jing Jiang , Chengqi Zhang This is my paper · ORCID

classification cs.LG

keywords federatedlearningtimeseriesfoundationmodelcross-domainheterogeneityadaptivetrend-awarenessmodulemixtureofexpertsmaskedreconstructionzero-shotforecastinganomalydetection

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that fusing heterogeneous time series into a single centralized corpus hurts the training of time series foundation models, because the same numerical pattern can mean different things in different domains. It proposes FFTS, a federated learning scheme in which each dataset stays on its own client, trains a local encoder-only Transformer, and only shares model parameters with a server. The paper claims that the resulting global model transfers better to unseen forecasting, imputation, and anomaly detection tasks than centralized training on the same data, and even beats task-specific state-of-the-art baselines. This matters because it suggests organizations can collaboratively pretrain strong time series models without pooling raw data, which is both a privacy advantage and a practical path to larger training corpora.

What carries the argument

The load-bearing component is the Adaptive Trend-awareness Module (ATM), a mixture-of-experts-style block appended to each Transformer layer with four fixed timescale experts for second, minute, hour, and day patterns. A gating network decomposes each representation into trend and seasonal components, computes softmax timescale weights, activates the top-k experts, and fuses their outputs through a feed-forward layer. The second mechanism is heterogeneous knowledge alignment: an ATM-specific regularization term $\lambda\|\Theta_T - \hat{\Theta}_T\|^2$ is applied both locally and on the server to pull each client's ATM parameters toward the global ATM parameters. A unified masking strategy with geometrically distributed mask segments forces local models to reconstruct masked time points rather than memorize domain-specific patterns. Together these pieces let each client preserve its unique data characteristics while the regularization aligns shared cross-timescale trends in the global model.

What would settle it

Run the same federated pretraining with the ATM disabled but the alignment regularization kept intact: if the resulting model matches FFTS on forecasting, imputation, and anomaly detection, the timescale experts are not carrying the reported gain. Conversely, removing the regularization while keeping the ATM would show whether the alignment term is the essential component.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a time series foundation model pretrained with federated learning, where each domain is an independent client, achieves superior zero- and few-shot generalization compared with centralized pretraining that fuses the same datasets. FFTS reports the best average long-term forecasting error across ETT, Weather, and ILI benchmarks, surpassing the LLM-based Time-LLM by 9.52% on average in the fine-tuned setting, by 4.1% in few-shot settings, and by 3.87% in zero-shot transfer. For imputation, the FFTS-pretrained model reduces MSE by 14.7% relative to GPT4TS, and for anomaly detection it reaches an average F1 of 88.10%, beating GPT4TS by 1.02 percentage points and the centralized FFTS-Cen baseline on most tasks. The paper interprets these results as evidence that federated pretraining can replace centralized pretraining for time series foundation models while preserving data privacy.

Load-bearing premise

The load-bearing premise is that the Adaptive Trend-awareness Module's fixed set of four timescale experts and its gating network extract cross-domain trend patterns that improve the global model; if that fails, FFTS collapses to FedAvg with a proximal penalty.

Editorial extensions

If this is right

Federated pretraining can replace centralized pretraining for time series foundation models when data holders cannot share raw data, without giving up accuracy.
A single FFTS-pretrained model transfers across forecasting, imputation, and anomaly detection using one lightweight adaptation head, so the same pretrained weights serve multiple downstream tasks.
The federated model beats the same-architecture centralized baseline on most benchmarks, implying that statistical heterogeneity across domains is not only an obstacle but can act as a useful regularizer.
Standard federated algorithms designed for image-style heterogeneity, such as FedProx and pFedMe, underperform on heterogeneous time series, so time series heterogeneity needs domain-specific alignment mechanisms.
Even vanilla FedAvg produces a competitive time series foundation model, supporting federated learning as a general pretraining strategy for this modality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The fixed set of four timescale experts may not be the actual source of gain: a variant that keeps the alignment regularization but replaces the ATM with a simpler linear projection would isolate whether trend decomposition matters, and the paper's current evidence for the ATM is visual only.
The approach should transfer to other settings where token meaning is context-dependent, such as multisensor IoT streams, where local training with an alignment penalty may beat cross-source fusion whenever identical numeric patterns mean different things in different contexts.
A data-driven alternative would learn timescale partitions per client or per layer; if that matches or beats the fixed second/minute/hour/day split, then the specific choice of four timescales is a convenience rather than a discovered property of time series.
The strong FedAvg baseline results suggest that the largest benefit may come from not fusing heterogeneous corpora at all; testing on more datasets and more clients could clarify how much the ATM adds beyond that effect.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's the short version: this is the first paper I know of that trains a time series foundation model from scratch in a federated way, and the idea is sensible. The components—patch embedding, encoder-only Transformer, Mixture-of-Experts over timescales, masking, proximal alignment—are all known, but the combination, and specifically the choice to treat each dataset as a client rather than concatenating them, is a genuine new configuration. If the empirical claims hold, it matters for privacy-sensitive domains.

The paper does a fair amount of work well: experiments across forecasting, imputation, and anomaly detection, with ablations and hyperparameter sensitivity. The framework ablation (FFTS-A and FFTS-B) gives some evidence that both the ATM and the alignment term contribute. The writing is clear enough to reproduce the method.

Now the soft spots, in proportion.

First, the headline number is wrong. The text says FFTS surpasses Time-LLM by 9.52% on long-term forecasting; Table 4 gives Time-LLM 0.497 and FFTS 0.473, a relative improvement of about 4.8%. Discrepancies like that make a reader wonder what else in the tables was computed the same way.

Second, no error bars or significance tests anywhere. The differences between FFTS and the centralized FFTS-Cen are small—roughly 0.4% to 2.9% relative across tasks—so the word "superior" is stronger than the evidence. The pattern is consistent, so I don't think it's noise, but I can't rule it out.

Third, and most important: the no-leakage claim cannot be checked. The paper states that all downstream datasets were excluded from pretraining, but the list of 18 pretraining datasets is deferred to an omitted Appendix B. The downstream sets are common public benchmarks, so this is not a hypothetical concern. If any of them appear in pretraining, the zero-shot and few-shot results are not generalization. The stress-test note is right: this is load-bearing. The code is public, so a referee can inspect configs, but the manuscript itself is missing the evidence.

Fourth, the comparison set is thin for a foundation model paper. Time-LLM and GPT4TS are LLM adapters; there are no numbers against Moirai, TimesFM, or Chronos. That weakens the "state of the art" claim, though it doesn't undermine the core federated-pretraining idea.

Bottom line: this is a solid new combination, honestly presented in most places, with one concrete arithmetic inconsistency and one unverifiable leakage assertion. It deserves a serious referee, but the revision should fix the 9.52% claim, add error bars, and disclose the pretraining corpus.

For a reading group: yes. I'd bring it to prompt a discussion about what counts as evidence in TSFM papers.

Referee Report

4 major / 6 minor

Summary. The paper proposes FFTS, a federated learning framework for pretraining time series foundation models from heterogeneous datasets. Each dataset is treated as an independent client that trains a local encoder-only Transformer with patch embedding, an Adaptive Trend-awareness Module with four fixed timescale experts, and a masking-based reconstruction objective. A heterogeneous knowledge alignment regularization term is added to both local and global objectives, and a unified adaptation head is used for downstream forecasting, imputation, and anomaly detection. The authors evaluate FFTS on standard benchmarks, comparing it with FedAvg, FedProx, pFedMe, a centralized variant FFTS-Cen, and several task-specific and LLM-based baselines.

Significance. If the claims hold, the paper makes a useful contribution: it provides an alternative to centralized pretraining for time series foundation models, preserves data locality, and includes a code release. The evaluation covers three major tasks with many baselines, and the ablation study attempts to isolate the effect of the proposed module and regularizer. The main limitations are evidentiary: the pretraining dataset list is deferred to an omitted appendix, making the no-leakage guarantee unverifiable; the headline improvement number is inconsistent with the reported table; and the differences from the centralized baseline are small and reported without error bars. These issues are load-bearing for the central claim that federated pretraining can replace centralized pretraining for TSFMs, but they are fixable with additional disclosure and analysis.

major comments (4)

[Time Series Forecasting, Main Results (Table 4)] The text states that FFTS surpasses Time-LLM by an averaged 9.52%, but the table averages are 0.497 for Time-LLM and 0.473 for FFTS, which is a relative improvement of about 4.8% and an absolute difference of 0.024. Please correct this number and state explicitly whether reported percentages are relative or absolute improvements.
[Experiments, Federated Pretraining Setups and Downstream Setups] The guarantee that all downstream datasets were excluded from pretraining cannot be checked: the list of 18 pretraining datasets is deferred to an omitted Appendix B. This is not a cosmetic issue because Table 4 excludes Traffic and Electricity specifically due to their presence in pretraining, so overlap with other common public downstream sets (ETT variants, Weather, ILI, M4, SMD, MSL, SMAP, SWaT, PSM) is a live risk. Please provide the full pretraining corpus and an explicit overlap check with all downstream benchmarks.
[Tables 4-9 (FFTS versus FFTS-Cen)] The reported differences between FFTS and FFTS-Cen are small (Table 4: 0.473 vs 0.475; Table 5: 0.366 vs 0.369; Table 8: 0.034 vs 0.035; Table 9: 88.10 vs 87.69), and no error bars, number of seeds, or significance tests are reported. The claims that FFTS outperforms centralized pretraining in most cases and is superior to FL baselines need uncertainty quantification before they can be accepted.
[Heterogeneous Knowledge Alignment, Eq. (5)] The global objective in Eq. (5) is written as a weighted sum of local objectives plus a regularization term involving local and global ATM parameters, but the server does not train model parameters in the described protocol. It is unclear how the server-side ATM regularization is computed and optimized. Please specify the exact server update rule or revise the formulation to match the actual algorithm.

minor comments (6)

[Throughout] There are several typographical errors: 'FedAvd' should be 'FedAvg', 'shwon' should be 'shown', 'resuls' should be 'results', 'Knowledege' should be 'Knowledge', and 'statics' should be 'statistics'.
[Preliminary, Eq. (1) and Eq. (5)] The client index is introduced as k in the text but the sums use i; please align the notation consistently.
[Discussion, Computational/Communication Efficiency] The claim that regularization 'mitigates frequent global updates, thus lowering bandwidth requirements' is not supported by any experiment or quantitative analysis; the communication cost per round is the same as in standard FedAvg. Please remove or substantiate this claim.
[Abstract and Title] The model is described as a 'foundation model', but no parameter count or model scale is reported. Since scale is often part of what distinguishes foundation models, please report model size and training cost.
[Tables 5, 6, 8, 9 and accompanying text] Several percentage improvements (4.1%, 3.87%, 14.7%, 1.02%) are reported without specifying the baseline average or the formula used; please state whether these are relative or absolute and provide the corresponding standard deviations.
[Discussion, Privacy Guarantee] The privacy discussion claims that regularization reduces overfitting risk and that ATM reduces the data footprint in updates, but these are not formal privacy guarantees and are not quantified. Consider softening the wording or adding measurements such as gradient inversion attack evaluations.

Circularity Check

0 steps flagged · score 2.0 of 10

No construction-level circularity: the federated pretraining objective (Eqs. 4-5) does not encode any downstream target, and the generalization claims are benchmarked against external baselines. The main flagged gap is the unverifiable no-leakage guarantee resting on the omitted Appendix B pretraining list, which is a verification risk, not a circular reduction.

full rationale

The derivation chain is not circular at the equation level. The pretraining objective (Eqs. 4-5) is an unsupervised masked-point reconstruction MSE plus a proximal ATM-alignment term lambda*||Theta_T - Theta_hat_T||^2; its hyperparameters (Lm, rm, k, lambda) are selected on the pretraining validation loss reported in Table 1, and no downstream target (forecast MSE, imputation MSE, anomaly F1) appears in that objective or in the hyperparameter selection. Downstream evaluation fine-tunes a unified MLP+LayerNorm adaptation head on external benchmarks (ETT variants, Weather, ILI, M4, SMD, MSL, SMAP, SWaT, PSM) and compares against externally sourced baselines (Time-LLM, GPT4TS, PatchTST, etc.), so the generalization numbers are not defined in terms of any fitted quantity. The ATM claim is supported by ablations that partially isolate it (FFTS-B keeps ATM but drops the heterogeneous-knowledge-alignment regularizer; Table 1e removes single timescale experts), so the Fig. 6 weight visualizations are supplementary rather than load-bearing. The paper cites the authors' prior work (Chen et al. 2023a-f, 2024a-b, 2022) for motivation in related work and discussion, but no load-bearing premise is justified only by those self-citations, and no uniqueness theorem is imported from the authors' prior work. The genuine gap is an unverifiable no-leakage assertion at Experiments, Federated Pretraining Setups: 'All downstream datasets were excluded from the pre-training phase to prevent data leakage.' The full 18-dataset pretraining list is deferred to an omitted 'Appendix B' ('More about pretraining can be found at Appendix B'), so the exclusion cannot be checked from the manuscript. If any downstream set were among the 18 clients, the zero-/few-shot results in Tables 5-6 would be memorization rather than generalization. The Forecasting Setups sentence 'excluding Traffic and Electricity due to their presence in pretraining' shows leakage control was attempted for at least those two sets, but the complete list is unavailable, capping the evidentiary strength of the headline claim. This is a missing-evidence correctness risk, not a by-construction circularity, and therefore does not raise the circularity score above 2.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hyperparameters and domain assumptions; no new entities are posited. The ATM is an architectural module, not an invented physical entity.

free parameters (5)

regularization weight lambda = 0.1
Tuned in Table 1(d); default lambda = 1e-1 controls ATM parameter alignment between local and global models.
top-k in ATM = 3
Tuned in Table 1(b); number of timescale experts activated per sample.
mask length Lm = 16 for evaluation (searched over {8,16,24})
Controls length of masked segments in unified masking; used in downstream adaptation.
mask ratio rm = 35% for evaluation (searched over {15%,25%,50%})
Controls masking probability in unified masking; used in downstream adaptation.
input length L = 512
Fixed pretraining sequence length for all clients.

assumptions (4)

domain assumption Time series tokens from different domains have less transferable semantics than text/image tokens
Motivates federated training; asserted in Introduction with Fig. 1 but not formally proven.
domain assumption Statistical heterogeneity across domains degrades centralized cross-domain pretraining
Empirical motivation; supported by Fig. 1 examples but not a rigorous result.
ad hoc to paper Four fixed timescales (second, minute, hour, day) are sufficient to capture cross-domain trend similarities
ATM uses exactly four experts; no analysis justifies this choice.
domain assumption Federated averaging with ATM parameter regularization converges to a useful global model under heterogeneous clients
Standard FL assumption; no convergence theorem is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Federated Foundation Models on Heterogeneous Time Series." pith.science (2026). https://pith.science/paper/CVNG3HJT

@misc{pith2026241208906,
  author       = {Pith},
  title        = {Pith review of: Federated Foundation Models on Heterogeneous Time Series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CVNG3HJT}},
  note         = {Machine review of arXiv:2412.08906}
}

read the original abstract

Training a general-purpose time series foundation models with robust generalization capabilities across diverse applications from scratch is still an open challenge. Efforts are primarily focused on fusing cross-domain time series datasets to extract shared subsequences as tokens for training models on Transformer architecture. However, due to significant statistical heterogeneity across domains, this cross-domain fusing approach doesn't work effectively as the same as fusing texts and images. To tackle this challenge, this paper proposes a novel federated learning approach to address the heterogeneity in time series foundation models training, namely FFTS. Specifically, each data-holding organization is treated as an independent client in a collaborative learning framework with federated settings, and then many client-specific local models will be trained to preserve the unique characteristics per dataset. Moreover, a new regularization mechanism will be applied to both client-side and server-side, thus to align the shared knowledge across heterogeneous datasets from different domains. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed federated learning approach. The newly learned time series foundation models achieve superior generalization capabilities on cross-domain time series analysis tasks, including forecasting, imputation, and anomaly detection.

Figures

Figures reproduced from arXiv: 2412.08906 by the authors.

**Figure 1.** Examples of statistical heterogeneity across time series datasets. HeartRate: healthcare data, Precipitation: weather data. Heterogeneity arises as cross-domain time series often exhibit significant variations in temporal patterns, including trends and timescales. This leads to two main issues: (1) inconsistent convergence rates across different domains (refer to [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** Overview of Time Series Foundation Models (TSFMs) that training from scratch. (a) STS Pretraing: Training from scratch using a single time series (STS) obtained by fusing data from different domains (Goswami et al. 2024; Liu et al. 2024c; Das et al. 2023). (b) MTS pretraing: Training from scratch using multiple time series (MTS) from different domains (Woo et al. 2024). (c) Federated Pretraining (Ours, this paper): … view at source ↗

**Figure 3.** Architecture of model within FFTS. a Structure of local model in each client. b Architecture of the proposed Adaptive Trend-awareness Module (ATM), which consists of four independent experts for extracting trends at different timescales based on the representation from Attention. Structurally inspired by the Mixture of Experts (MoE) (Fedus, Zoph, and Shazeer 2022). c Architecture of the Gating Network [PITH_FUL… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Visualization of cross-domain trend similarity within historical observations. Upper: Weather (1-hour resolution), Energy (5-minute resolution), Network (30-second resolution), Natural (1-day resolution), Bottom: Corresponding trend. Adaptive Trend-awareness Module H…

**Figure 5.** Figure 5: Schematic diagram of FFTS for downstream adaption. A unified adaptation head facilitates knowledge transfer across different downstream tasks. (a) Predicting future trends from past data. (b) Filling gaps in data using related time series and context. (c) Identifying …

**Figure 6.** Figure 6: Visualization of our proposed ATM. (a): variations in timescale weights for selected clients across various communication rounds. (b): featuring regular sinusoidal patterns for the rightmost input features. Subsequent depict the Gating network representation in the …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 11 canonical work pages

[1]

O.; Pfister, T.; Zheng, Y.; Ye, W.; and Liu, Y

Cao, D.; Jia, F.; Arik, S. O.; Pfister, T.; Zheng, Y.; Ye, W.; and Liu, Y. 2023. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:2310.04948

arXiv 2023
[2]

G.; Oreshkin, B

Challu, C.; Olivares, K. G.; Oreshkin, B. N.; Garza, F.; Mergenthaler-Canseco, M.; and Dubrawski, A. 2022. N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting. arXiv:2201.12886

arXiv 2022
[3]

Chang, C.; Peng, W.-C.; and Chen, T.-F. 2023. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. arXiv preprint arXiv:2308.08469

arXiv 2023
[4]

Chen, S.; Long, G.; Jiang, J.; Liu, D.; and Zhang, C. 2023 a . Foundation models for weather and climate data understanding: A comprehensive survey. arXiv preprint arXiv:2312.03014

arXiv 2023
[5]

Chen, S.; Long, G.; Jiang, J.; and Zhang, C. 2024 a . Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Models. arXiv preprint arXiv:2405.20348

arXiv 2024
[6]

Chen, S.; Long, G.; Shen, T.; and Jiang, J. 2023 b . Prompt federated learning for weather forecasting: Toward foundation models on meteorological data. arXiv preprint arXiv:2301.09152

arXiv 2023
[7]

Chen, S.; Long, G.; Shen, T.; Jiang, J.; and Zhang, C. 2023 c . Federated Prompt Learning for Weather Foundation Models on Devices. arXiv preprint arXiv:2305.14244

arXiv 2023
[8]

Chen, S.; Ren, S.; Wang, G.; Huang, M.; and Xue, C. 2023 d . Interpretable CNN-multilevel attention transformer for rapid recognition of pneumonia from chest X-ray images. IEEE Journal of Biomedical and Health Informatics, 28(2): 753--764

work page 2023

Show all 50 references

[9]

Chen, S.; Shu, T.; Zhao, H.; Wan, Q.; Huang, J.; and Li, C. 2022. Dynamic multiscale fusion generative adversarial network for radar image extrapolation. IEEE Transactions on Geoscience and Remote Sensing, 60: 1--11

2022
[10]

Chen, S.; Shu, T.; Zhao, H.; Wang, J.; Ren, S.; and Yang, L. 2024 b . Free lunch for federated remote sensing target fine-grained classification: A parameter-efficient framework. Knowledge-Based Systems, 294: 111694

2024
[11]

Chen, S.; Shu, T.; Zhao, H.; Zhong, G.; and Chen, X. 2023 e . Tempee: Temporal-spatial parallel transformer for radar echo extrapolation beyond auto-regression. IEEE Transactions on Geoscience and Remote Sensing

2023
[12]

Chen, S.; Wang, X.; Ren, S.; Yang, J.; Zhang, Y.; and Wang, G. 2023 f . Collaborative photonic crystal fiber property optimization: A new paradigm for reverse design. IEEE Photonics Technology Letters

2023
[13]

Das, A.; Kong, W.; Sen, R.; and Zhou, Y. 2023. A decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310.10688

2023 arXiv
[14]

S.; Mohapatra, C.; Naidu, S

Dooley, S.; Khurana, G. S.; Mohapatra, C.; Naidu, S. V.; and White, C. 2024. Forecastpfn: Synthetically-trained zero-shot forecasting. Advances in Neural Information Processing Systems, 36

2024
[15]

Fedus, W.; Zoph, B.; and Shazeer, N. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120): 1--39

2022
[16]

Garza, A.; and Mergenthaler-Canseco, M. 2023. TimeGPT-1. arXiv preprint arXiv:2310.03589

2023 arXiv
[17]

Goswami, M.; Szafer, K.; Choudhry, A.; Cai, Y.; Li, S.; and Dubrawski, A. 2024. Moment: A family of open time-series foundation models. arXiv preprint arXiv:2402.03885

2024 arXiv
[18]

Gruver, N.; Finzi, M.; Qiu, S.; and Wilson, A. G. 2024. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36

2024
[19]

Y.; Shi, X.; Chen, P.-Y.; Liang, Y.; Li, Y.-F.; Pan, S.; et al

Jin, M.; Wang, S.; Ma, L.; Chu, Z.; Zhang, J. Y.; Shi, X.; Chen, P.-Y.; Liang, Y.; Li, Y.-F.; Pan, S.; et al. 2023. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728

2023 arXiv
[20]

Kim, T.; Kim, J.; Tae, Y.; Park, C.; Choi, J.-H.; and Choo, J. 2021. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference on Learning Representations

2021
[21]

Kitaev, N.; Kaiser, L.; and Levskaya, A. 2020. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451

2020 arXiv
[22]

Li, S.; Jin, X.; Xuan, Y.; Zhou, X.; Chen, W.; Wang, Y.-X.; and Yan, X. 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in neural information processing systems, 32

2019
[23]

K.; Zaheer, M.; Sanjabi, M.; Talwalkar, A.; and Smith, V

Li, T.; Sahu, A. K.; Zaheer, M.; Sanjabi, M.; Talwalkar, A.; and Smith, V. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2: 429--450

2020
[24]

Liu, Q.; Liu, X.; Liu, C.; Wen, Q.; and Liang, Y. 2024 a . Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting. arXiv preprint arXiv:2405.14252

2024 arXiv
[25]

X.; and Dustdar, S

Liu, S.; Yu, H.; Liao, C.; Li, J.; Lin, W.; Liu, A. X.; and Dustdar, S. 2021. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations

2021
[26]

Liu, X.; Hu, J.; Li, Y.; Diao, S.; Liang, Y.; Hooi, B.; and Zimmermann, R. 2024 b . Unitime: A language-empowered unified model for cross-domain time series forecasting. In Proceedings of the ACM on Web Conference 2024, 4095--4106

2024
[27]

Liu, Y.; Wu, H.; Wang, J.; and Long, M. 2022. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35: 9881--9893

2022
[28]

Liu, Y.; Zhang, H.; Li, C.; Huang, X.; Wang, J.; and Long, M. 2024 c . Timer: Transformers for time series analysis at scale. arXiv preprint arXiv:2402.02368

2024 arXiv
[29]

McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273--1282. PMLR

2017
[30]

Nevrataki, T.; Iliadou, A.; Ntolkeras, G.; Sfakianakis, I.; Lazaridis, L.; Maraslidis, G.; Asimopoulos, N.; and Fragulis, G. F. 2023. A survey on federated learning applications in healthcare, finance, and data privacy/data security. In AIP Conference Proceedings, volume 2909....

2023
[31]

H.; Sinthong, P.; and Kalagnanam, J

Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam, J. 2022. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730

2022 arXiv
[32]

N.; Carpov, D.; Chapados, N.; and Bengio, Y

Oreshkin, B. N.; Carpov, D.; Chapados, N.; and Bengio, Y. 2020. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting. arXiv:1905.10437

2020 arXiv
[33]

Ren, S.; Chen, S.; Wang, J.; Xu, H.; Hou, X.; Huang, M.; Liu, J.; and Wang, G. 2024 a . A distributed photonic crystal fiber reverse design framework based on multi-source knowledge fusion. Optical Fiber Technology, 84: 103718

2024
[34]

Ren, S.; Hu, Y.; Chen, S.; and Wang, G. 2024 b . Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis. arXiv preprint arXiv:2407.02261

2024 arXiv
[35]

Sun, H.; Tang, X.; Yang, C.; Yu, Z.; Wang, X.; Ding, Q.; Li, Z.; and Yu, H. 2024. HiFi-Gas: Hierarchical Federated Learning Incentive Mechanism Enhanced Gas Usage Estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 22824--22832

2024
[36]

T Dinh, C.; Tran, N.; and Nguyen, J. 2020. Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, 33: 21394--21405

2020
[37]

Woo, G.; Liu, C.; Kumar, A.; Xiong, C.; Savarese, S.; and Sahoo, D. 2024. Unified Training of Universal Time Series Forecasting Transformers. arXiv:2402.02592

2024 arXiv
[38]

Woo, G.; Liu, C.; Sahoo, D.; Kumar, A.; and Hoi, S. 2022. Etsformer: Exponential smoothing transformers for time-series forecasting. arXiv preprint arXiv:2202.01381

2022 arXiv
[39]

Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; and Long, M. 2022. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186

2022 arXiv
[40]

Wu, H.; Xu, J.; Wang, J.; and Long, M. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34: 22419--22430

2021
[41]

Xu, J.; Wu, H.; Wang, J.; and Long, M. 2022. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. arXiv:2110.02642

2022 arXiv
[42]

Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2023. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, 11121--11128

2023
[43]

Zerveas, G.; Jayaraman, S.; Patel, D.; Bhamidipaty, A.; and Eickhoff, C. 2021. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 2114--2124

2021
[44]

Zhang, T.; Zhang, Y.; Cao, W.; Bian, J.; Yi, X.; Zheng, S.; and Li, J. 2022. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprint arXiv:2207.01186

2022 arXiv
[45]

Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; and Zhang, W. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 11106--11115

2021
[46]

Zhou, T.; Ma, Z.; Wen, Q.; Wang, X.; Sun, L.; and Jin, R. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, 27268--27286. PMLR

2022
[47]

Zhou, T.; Niu, P.; Sun, L.; Jin, R.; et al. 2023. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems, 36: 43322--43355

2023
[48]

Zhuang, W.; Chen, C.; and Lyu, L. 2023. When foundation model meets federated learning: Motivations, challenges, and future directions. arXiv preprint arXiv:2306.15546

2023 arXiv
[49]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[50]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

O.; Pfister, T.; Zheng, Y.; Ye, W.; and Liu, Y

Cao, D.; Jia, F.; Arik, S. O.; Pfister, T.; Zheng, Y.; Ye, W.; and Liu, Y. 2023. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint arXiv:2310.04948

arXiv 2023

[2] [2]

G.; Oreshkin, B

Challu, C.; Olivares, K. G.; Oreshkin, B. N.; Garza, F.; Mergenthaler-Canseco, M.; and Dubrawski, A. 2022. N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting. arXiv:2201.12886

arXiv 2022

[3] [3]

Chang, C.; Peng, W.-C.; and Chen, T.-F. 2023. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. arXiv preprint arXiv:2308.08469

arXiv 2023

[4] [4]

Chen, S.; Long, G.; Jiang, J.; Liu, D.; and Zhang, C. 2023 a . Foundation models for weather and climate data understanding: A comprehensive survey. arXiv preprint arXiv:2312.03014

arXiv 2023

[5] [5]

Chen, S.; Long, G.; Jiang, J.; and Zhang, C. 2024 a . Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Models. arXiv preprint arXiv:2405.20348

arXiv 2024

[6] [6]

Chen, S.; Long, G.; Shen, T.; and Jiang, J. 2023 b . Prompt federated learning for weather forecasting: Toward foundation models on meteorological data. arXiv preprint arXiv:2301.09152

arXiv 2023

[7] [7]

Chen, S.; Long, G.; Shen, T.; Jiang, J.; and Zhang, C. 2023 c . Federated Prompt Learning for Weather Foundation Models on Devices. arXiv preprint arXiv:2305.14244

arXiv 2023

[8] [8]

Chen, S.; Ren, S.; Wang, G.; Huang, M.; and Xue, C. 2023 d . Interpretable CNN-multilevel attention transformer for rapid recognition of pneumonia from chest X-ray images. IEEE Journal of Biomedical and Health Informatics, 28(2): 753--764

work page 2023