Pith. sign in

REVIEW 3 major objections 6 minor 161 references

Modular Foundation Models for Time-Series Perception in Digital Twins

T0 review · 3 major / 6 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A bank of frozen self-supervised time-series encoders, gated by similarity and structure then Transformer-aggregated, can power multiple digital-twin perception tasks without retraining the encoders.

desk verdict Solid industrial systems paper: a reusable frozen-encoder bank with offline gating and Transformer aggregation that works competitively on ETT and a real hydro-generator virtual sensor, with an honest caveat that the gate only helps if the bank already holds the right structure. read the letter →

arxiv 2607.03585 v1 pith:R6BA5KBP submitted 2026-07-03 cs.LG

Quang Hung Pham , Ryad Zemouri , Martin Gagnon , Luc Vouligny This is my paper
classification cs.LG
keywords time-seriesanalysisfoundationmodelsmixtureofexpertsself-supervisedlearningrepresentationdigitaltwinsPHMvirtualsensing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Engineering digital twins need one perception layer that can impute missing sensor values, forecast far ahead, work with few labels, and estimate hard-to-measure quantities such as rotor temperature, yet most models are rebuilt for each task and each machine. This paper claims that a modular foundation model can meet that need: many encoders are first pretrained once, with reconstruction-plus-contrastive self-supervision, on heterogeneous unlabeled series; for any new target a two-stage gate keeps only the most relevant encoders; their outputs are projected into a shared space, aligned, and fused by Transformer self-attention; lightweight heads then solve the actual tasks while the encoders stay frozen. Ablations show that informed gating and adaptive aggregation give the largest gains; on the public ETT suite the system is competitive with strong baselines for imputation and few-shot forecasting and remains usable for long-horizon prediction; a hydro-generator virtual-sensor case study further shows calibrated temperature estimates that transfer across units. The practical claim is that reusable modular representations, rather than end-to-end task-specific networks, can serve as the scalable perception layer for industrial digital twins and hybrid PHM systems.

What carries the argument

The FM-TSP triple (encoder bank E, gating-plus-aggregation module G, task heads T): two-stage encoder selection by statistical/spectral distance followed by Pearson correlation of input-versus-latent cosine-similarity matrices, linear projection with optional early MMD alignment, and Transformer self-attention over the selected encoder tokens.

What would settle it

On a held-out industrial or ETT-style target whose long-range past–future structure is absent from every encoder in the bank, measure whether the two-stage gate still ranks the truly best encoders first and whether the frozen-plus-light-head pipeline remains competitive with a model trained end-to-end on that target; if ranking fails or accuracy collapses, the modular-transfer claim fails.

Watch

Extended reading notes

Core claim

A collection of heterogeneous encoders pretrained once with self-supervised reconstruction and contrastive losses can be frozen, dynamically selected for a new target by data-similarity and structural-correlation gating, projected and Transformer-aggregated into a shared representation, and then adapted with only lightweight heads to support imputation, long-term forecasting, few-shot forecasting, and industrial virtual sensing at competitive accuracy.

Load-bearing premise

The paper assumes that simple statistical and spectral features plus correlation of neighborhood structures are enough to pick which frozen encoders will actually transfer useful representations to an unseen target domain.

Editorial extensions

If this is right

  • A single frozen encoder bank can be shared across many digital-twin assets and tasks, cutting the cost of building and maintaining separate perception models.
  • New sensors or operating regimes can be handled by adding or selecting pretrained encoders rather than retraining the whole stack.
  • Imputation, forecasting, few-shot adaptation, and virtual sensing become interchangeable heads on the same modular backbone.
  • Hybrid PHM pipelines can plug the same perception layer into physics-based models without task-specific redesign.
  • Conditional computation at the encoder level keeps inference cost low even as the bank grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gate’s proxy ranking is weak for forecasting structure, the practical fix is to enlarge the encoder bank with models deliberately pretrained on long-horizon prediction objectives rather than redesigning the aggregation head.
  • The same modular pattern could be applied to anomaly detection or remaining-useful-life heads without changing the frozen representation layer.
  • Because encoders stay frozen, the architecture is naturally suited to continual addition of new assets or sites without catastrophic forgetting of earlier regimes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FM-TSP, a modular foundation model for time-series perception in digital twins and PHM. A bank of heterogeneous encoders (MLP/CNN/Transformer) is pretrained with a reconstruction–contrastive objective on diverse unlabeled datasets and then frozen. For a target dataset, a two-stage offline gate ranks encoders by statistical/spectral data similarity and by Pearson correlation of input vs. latent cosine-similarity matrices, selects a top-k subset, projects the latent vectors into a shared space (optionally with early MMD alignment), and aggregates them with a Transformer self-attention module. Lightweight task heads support imputation, long-term forecasting, few-shot forecasting, and probabilistic virtual sensing while only projectors, aggregator, and heads are fine-tuned. Ablations isolate pretraining, gating, projection/alignment, and aggregation; ETT results are competitive with DLinear, LightTS, PatchTST, and iTransformer; a hydro-generator rotor-temperature virtual-sensor case shows practical transfer across units.

Significance. If the modular transfer story holds, the work offers a practical perception layer for industrial digital twins: reusable frozen encoders, conditional computation, multi-task heads, and limited labeled adaptation. Strengths include systematic ablations with multi-metric box plots (MSE, DTW, Jacobian), shared-protocol ETT comparisons, and a real multi-unit industrial virtual-sensor study with calibrated uncertainty. The design is well aligned with PHM constraints (heterogeneous sensors, scarce labels, non-stationarity) and is more modular than typical end-to-end time-series foundation models. The main significance is architectural and empirical rather than theoretical; the contribution is a coherent, deployable recipe rather than a new universal approximator or scaling law.

major comments (3)
  1. §4.3 (H1–H3) and §6.1.5: The central claim that the offline gate reliably activates task-useful frozen encoders is only weakly supported. Ablation Fig. 4 shows top-k beats random/worst-k under the same proxy, but does not test whether high structural-correlation scores predict downstream utility when the library lacks the needed structure. The authors themselves note weaker long-term forecasting because encoders were not pretrained on ETT and may miss past–future dependencies. A load-bearing experiment is missing: correlation of gate scores with task metrics under controlled library incompleteness, or an oracle/task-aware selection baseline. Without this, multi-task competitiveness and the industrial case remain contingent on a sufficiently rich bank rather than on the selection mechanism itself.
  2. §6.1.4–6.1.5 and Figs. 7–9: ETT results are reported only as overall averages across subsets and horizons, without per-dataset/per-horizon tables or standard deviations. Given that long-term forecasting is already acknowledged as weaker than imputation/few-shot, aggregate plots make it hard to judge whether competitiveness is uniform or driven by easier settings. Full numerical tables (MSE/MAE/DTW per ETTh1/h2/m1/m2 and horizon) against the reimplemented baselines are needed for the benchmark claim to be verifiable.
  3. §6.2.3 and Fig. 11: The industrial transfer result is important but incomplete. Coverage drops from ~93.5% on the training TGU to ~65% on the held-out TGU, with systematic overestimation at low peaks. The paper does not quantify how much of this gap is closed by the modular gate versus a single encoder or a non-gated concatenation of the same bank, nor does it report error metrics (e.g., MAE/RMSE) alongside coverage. A controlled comparison on the same multi-unit split is required to substantiate that modular selection, rather than the probabilistic head alone, drives practical virtual-sensing performance.
minor comments (6)
  1. §1 and §2: The Jacobs et al. (1991) and Vats et al. (2024) citations are duplicated in consecutive sentences; clean the redundancy.
  2. Definition 3 and §4.1: Typographical issues (“agregartion”, “τT” indexing, “Noted that”) and inconsistent notation for the aggregation map G should be corrected.
  3. Eq. (17) vs. Table 1: The contrastive weight α is described inconsistently (trade-off with reconstruction vs. “contrastive weight”); clarify the parameterization used in experiments.
  4. §5: Jacobian-norm interpretation is useful but under-specified (which input perturbations, layer, and normalization). A short formal definition would help readers interpret the box plots.
  5. Figures 7–11: Axis labels, metric units, and exact masking/horizon settings are hard to read from the text alone; ensure captions are self-contained.
  6. Related work: Time-MoE, MOIRAI-MoE, and recent time-series foundation models are cited; a clearer positioning of offline encoder-level gating versus token-level sparse MoE would strengthen novelty claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: standard empirical ML pipeline with frozen SSL encoders, external ETT splits, and held-out industrial transfer; self-citations supply motivation only.

full rationale

The paper's load-bearing claims are empirical performance numbers (imputation / long-term / few-shot forecasting on ETT; virtual rotor-temperature sensing on a held-out hydro-generator unit). Pretraining uses reconstruction + NT-Xent on heterogeneous non-ETT corpora with encoders then frozen; fine-tuning updates only projectors, Transformer aggregation, and task heads on train splits and reports held-out test metrics against reimplemented external baselines (DLinear, LightTS, PatchTST, iTransformer). Gating (data-similarity via statistical/spectral ϕ plus structural Pearson of cosine matrices under H1–H3) is a design heuristic validated by top-k vs random/worst-k ablations, not a quantity defined from the reported test losses. Self-citations (Jose et al. 2026; Zemouri et al. 2025) motivate modular MoE for hydrogenerator diagnostics but do not supply the ETT or virtual-sensor numbers. No uniqueness theorem, fitted identity renamed as prediction, or self-definitional reduction appears. The authors' own admission that long-term forecasting weakens when the encoder bank lacks past–future structure is an honesty about transfer limits, not circularity. Score 0 is the correct outcome.

Assumptions & free parameters 6 free parameters · 4 assumptions · 2 invented entities

The work is empirical systems ML. Load-bearing content is architectural choices and selection heuristics rather than physical laws. Free parameters control pretraining trade-offs, gate width, alignment strength, and task heads; axioms are standard SSL/MoE assumptions plus the paper’s three transfer hypotheses for gating; invented entities are the FM-TSP composite and its offline two-stage gate, not new physical objects.

free parameters (6)
  • contrastive–reconstruction weight α
    Traded off in Eq. (17) and Table 1 over {0,0.25,0.5,0.75,1}; controls encoder representation quality used by all downstream claims.
  • number of selected encoders NE / top-k
    Gate retains NE encoders; performance claims depend on this hand-chosen width (ablation contrasts top-k vs random/worst-k).
  • MMD alignment coefficient β and warm-up epochs ne
    Eq. (29) schedule; authors note no principled β selection beyond empirical tuning, yet alignment is part of the claimed pipeline.
  • masking ratio and latent dimensions
    Pretraining diversity factors in Table 1 (masking 0/0.25/0.5; dims 8–64) that shape the frozen bank.
  • shared projection dimension d and Transformer aggregation hyperparameters
    Define the fused space DG; not uniquely determined by theory.
  • virtual-sensor Beta-loss / mean–variance head parameters
    Industrial case head predicts distribution parameters; calibration coverage depends on this likelihood choice and training.
assumptions (4)
  • domain assumption Self-supervised reconstruction + contrastive pretraining on heterogeneous unlabeled series yields transferable task-agnostic representations usable when encoders are frozen.
    Core premise of §4.2 and the foundation-model framing; supported by SSL literature but not proved for the industrial regimes studied.
  • ad hoc to paper (H1) Encoders pretrained on distributions closer to the target extract more informative representations; (H2) statistical/spectral embeddings suffice to represent the process; (H3) a good encoder preserves local neighborhood structure across input and latent spaces.
    Explicitly stated in §4.3 as the basis of the two-stage gate; load-bearing for conditional computation claims.
  • domain assumption Mixture-of-experts / conditional computation improves scalability and regime specialization for non-stationary industrial series.
    Motivation in §§1–2 citing Jacobs et al., Time-MoE, industrial MoE diagnostics.
  • domain assumption Standard supervised losses on ETT splits and industrial train/val/test splits are valid proxies for perception quality in digital twins.
    Evaluation design in §6; usual ML benchmark assumption.
invented entities (2)
  • FM-TSP modular foundation model (E, G, T) with frozen multi-architecture encoder bank
    purpose: Unify multi-task time-series perception for digital twins under limited labels.
    Definition 3 and §4; composite architecture rather than a new physical entity. independent_evidence false beyond this paper’s experiments.
  • Two-stage offline encoder gate (data similarity d(Pi,Y) + structural correlation r(Ei,Y))
    purpose: Select NE relevant frozen encoders without joint retraining.
    §4.3 Eqs. (21)–(24); heuristic selection mechanism introduced for this framework.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Modular Foundation Models for Time-Series Perception in Digital Twins." pith.science (2026). https://pith.science/paper/R6BA5KBP

@misc{pith2026260703585,
  author       = {Pith},
  title        = {Pith review of: Modular Foundation Models for Time-Series Perception in Digital Twins},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R6BA5KBP}},
  note         = {Machine review of arXiv:2607.03585}
}
read the original abstract

Engineering Digital Twins and Prognostics and Health Management (PHM) systems rely on robust perception modules to extract actionable information from heterogeneous and non-stationary time-series data. However, most existing approaches remain task-specific, data-hungry, and difficult to integrate into scalable monitoring and decision-making pipelines. Moreover, purely data-driven models often lack robustness and transferability across varying operating conditions. To address these challenges, this paper proposes a modular foundation model for time-series perception based on a collection of pretrained representation encoders. The framework leverages self-supervised learning on heterogeneous datasets to learn transferable and task-agnostic representations, which can be reused across multiple PHM tasks. A gating mechanism is introduced to dynamically select relevant encoders for a given target dataset, enabling conditional computation and adaptive model composition. The selected representations are projected into a shared latent space and aggregated using a Transformer-based self-attention module that explicitly models cross-encoder interactions. The resulting architecture supports multiple downstream tasks, including imputation, long-term forecasting, and few-shot learning, through lightweight task-specific heads, while keeping pretrained encoders frozen during adaptation. Extensive ablation studies demonstrate the complementary roles of self-supervised pretraining, encoder selection, representation alignment, and adaptive aggregation. Experimental results on the ETT benchmark show competitive performance across tasks, while a real-world industrial case study on virtual sensing for hydro-generator rotor temperature highlights the practical relevance of the approach.

Figures

Figures reproduced from arXiv: 2607.03585 by the authors.

Figure 1
Figure 1. Conceptual framework of the proposed foundation model for time-series processing. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Encoder pretraining with a lightweight decoder to focus learning on the encoder. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Effect of Self-Supervised Pretraining 5.3 Impact of Encoder Gating and Selection The second ablation examines the role of the encoder gating and selection mechanism described in Section IV-C. We evaluate three configurations: 1. Random selection: A subset of encoders is randomly sampled, and this selection process is repeated 10 times to ensure diversity; 11 [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: shows that random selection yields high variance and inconsistent performance, whereas worst-k selection leads to the poorest results. The near-zero Jacobian-based values in the latter case indicate that the selected encoders do not learn meaningful representations. In…
Figure 5
Figure 5. Figure 5: Role of Projection and Representation Alignment [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Impact of Downstream aggregation All configurations share the same encoder selection, projection, and fine-tuning settings, and differ only in the aggregation strategy. Experimental results, summarized in [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison for imputation downstream task [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Performance comparison for long-term forecasting task [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Performance comparison for few-shot learning forecasting task [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Temperature estimated by virtual sensor at training TGU. The zoomed-in section is taken from the split test [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Temperature estimated by virtual sensor at test TGU [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

161 extracted references · 23 canonical work pages

  1. [1]

    Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages=

    Foundation models for time series analysis: A tutorial and survey , author=. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages=

  2. [2]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Self-supervised learning for time series analysis: Taxonomy, progress, and prospects , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  3. [3]

    arXiv preprint arXiv:2310.10688 , year=

    A decoder-only foundation model for time-series forecasting , author=. arXiv preprint arXiv:2310.10688 , year=

    arXiv

  4. [4]

    ACM Computing Surveys (CSUR) , volume=

    How complex is your classification problem? a survey on measuring classification complexity , author=. ACM Computing Surveys (CSUR) , volume=. 2019 , publisher=

    2019

  5. [5]

    Sustainability , volume=

    Few-shot learning approaches for fault diagnosis using vibration data: a comprehensive review , author=. Sustainability , volume=. 2023 , publisher=

    2023

  6. [6]

    PHM Soc Eur Conf

    Diagnosis of tidal turbine vibration data through deep neural networks. PHM Soc Eur Conf. 2016 , author=

    2016

  7. [7]

    Journal of Computational Design and Engineering , volume=

    Vibration data feature extraction and deep learning-based preprocessing method for highly accurate motor fault diagnosis , author=. Journal of Computational Design and Engineering , volume=. 2023 , publisher=

    2023

  8. [8]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Multimodal learning with transformers: A survey , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2023 , publisher=

    2023

Show all 161 references
  1. [9]

    arXiv preprint arXiv:2406.12034 , year=

    Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts , author=. arXiv preprint arXiv:2406.12034 , year=

    arXiv

  2. [10]

    arXiv preprint arXiv:2402.05859 , year=

    Learning to route among specialized experts for zero-shot generalization , author=. arXiv preprint arXiv:2402.05859 , year=

    arXiv

  3. [11]

    arXiv preprint arXiv:2403.09809 , year=

    Self-Supervised Learning for Time Series: Contrastive or Generative? , author=. arXiv preprint arXiv:2403.09809 , year=

    arXiv

  4. [12]

    Information Sciences , volume=

    Multi-task self-supervised time-series representation learning , author=. Information Sciences , volume=. 2024 , publisher=

    2024

  5. [13]

    Proceedings of Neural Information Processing Systems, NeurIPS , year =

    Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency , author =. Proceedings of Neural Information Processing Systems, NeurIPS , year =

  6. [14]

    arXiv preprint arXiv:2402.02592 , year=

    Unified training of universal time series forecasting transformers , author=. arXiv preprint arXiv:2402.02592 , year=

    arXiv

  7. [15]

    Preprint , year=

    Lag-llama: Towards foundation models for probabilistic time series forecasting , author=. Preprint , year=

  8. [16]

    arXiv preprint arXiv:2403.00131 , year=

    Units: Building a unified time series model , author=. arXiv preprint arXiv:2403.00131 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2310.03589 , year=

    TimeGPT-1 , author=. arXiv preprint arXiv:2310.03589 , year=

    arXiv

  10. [18]

    Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining , pages=

    Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting , author=. Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining , pages=. 2022 , note=

    2022

  11. [19]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Ts2vec: Towards universal representation of time series , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=. 2022 , note=

    2022

  12. [20]

    arXiv preprint arXiv:2303.00320 , year=

    Timemae: Self-supervised representations of time series with decoupled masked autoencoders , author=. arXiv preprint arXiv:2303.00320 , year=

  13. [21]

    Advances in Neural Information Processing Systems , volume=

    Self-supervised contrastive pre-training for time series via time-frequency consistency , author=. Advances in Neural Information Processing Systems , volume=. 2022 , note=

    2022

  14. [22]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    A review of generalized zero-shot learning methods , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2022 , publisher=

    2022

  15. [23]

    IEEE transactions on knowledge and data engineering , volume=

    Self-supervised learning: Generative or contrastive , author=. IEEE transactions on knowledge and data engineering , volume=. 2021 , publisher=

    2021

  16. [24]

    arXiv preprint arXiv:2403.07815 , year=

    Chronos: Learning the language of time series , author=. arXiv preprint arXiv:2403.07815 , year=

    arXiv

  17. [25]

    2024 , note =

    Nixtla Team , title =. 2024 , note =

    2024

  18. [26]

    Advances in Neural Information Processing Systems , volume=

    Simmtm: A simple pre-training framework for masked time-series modeling , author=. Advances in Neural Information Processing Systems , volume=. 2024 , note=

    2024

  19. [27]

    Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages=

    A transformer-based framework for multivariate time series representation learning , author=. Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining , pages=. 2021 , note=

    2021

  20. [28]

    arXiv preprint arXiv:2106.14112 , year=

    Time-series representation learning via temporal and contextual contrasting , author=. arXiv preprint arXiv:2106.14112 , year=

    arXiv

  21. [29]

    arXiv preprint arXiv:1706.08838 , year=

    TimeNet: Pre-trained deep recurrent neural network for time series classification , author=. arXiv preprint arXiv:1706.08838 , year=

    arXiv

  22. [30]

    arXiv preprint arXiv:1610.07258 , year=

    Representation learning with deconvolution for multivariate time series classification and visualization , author=. arXiv preprint arXiv:1610.07258 , year=

    arXiv

  23. [31]

    ACM Transactions on Intelligent Systems and Technology (TIST) , volume=

    Representation learning on variable length and incomplete wearable-sensory time series , author=. ACM Transactions on Intelligent Systems and Technology (TIST) , volume=. 2022 , publisher=

    2022

  24. [32]

    Representation learning using a multi-branch transformer for industrial time series anomaly detection , author=

  25. [33]

    IEEE Transactions on Neural Networks and Learning Systems , year=

    Self-supervised time series representation learning via cross reconstruction transformer , author=. IEEE Transactions on Neural Networks and Learning Systems , year=

  26. [34]

    arXiv preprint arXiv:2308.01578 , year=

    Unsupervised representation learning for time series: A review , author=. arXiv preprint arXiv:2308.01578 , year=

    arXiv

  27. [35]

    7th international Mardin Artuklu scientific research conference , pages=

    Ideal dataset splitting ratios in machine learning algorithms: general concerns for data scientists and data analysts , author=. 7th international Mardin Artuklu scientific research conference , pages=

  28. [36]

    Artificial Intelligence Review , volume=

    Transformer models for text-based emotion detection: a review of BERT-based approaches , author=. Artificial Intelligence Review , volume=. 2021 , publisher=

    2021

  29. [37]

    Exploring the power of ChatGPT: Applications, techniques, and implications , pages=

    What Is ChatGPT? , author=. Exploring the power of ChatGPT: Applications, techniques, and implications , pages=. 2023 , publisher=

    2023

  30. [38]

    2025 , url =

    IBM , title =. 2025 , url =

    2025

  31. [39]

    Proceedings of the 35th AAAI Conference on Artificial Intelligence , year =

    Haoyi Zhou and Shanghang Zhang and Jieqi Peng and Shuai Zhang and Jianxin Li and Hui Xiong and Wancai Zhang , title =. Proceedings of the 35th AAAI Conference on Artificial Intelligence , year =

  32. [40]

    , title =

    Trindade, A. , title =. 2015 , howpublished =

    2015

  33. [41]

    2020 , howpublished =

    Kolle, Olaf , title =. 2020 , howpublished =

    2020

  34. [42]

    California Performance Measurement System (PeMS) , year =

  35. [43]

    Barrage Hydraulique , year =

  36. [44]

    2020 , url =

    Towards Data Science , title =. 2020 , url =

    2020

  37. [46]

    arXiv preprint arXiv:1807.03748 , year=

    Representation learning with contrastive predictive coding , author=. arXiv preprint arXiv:1807.03748 , year=

    arXiv

  38. [47]

    2021 , isbn =

    Zerveas, George and Jayaraman, Srideepika and Patel, Dhaval and Bhamidipaty, Anuradha and Eickhoff, Carsten , title =. 2021 , isbn =. doi:10.1145/3447548.3467401 , booktitle =

    2021 doi

  39. [48]

    Siddharth and Brooks Paige and Jan-Willem van de Meent and Alban Desmaison and Noah D

    N. Siddharth and Brooks Paige and Jan-Willem van de Meent and Alban Desmaison and Noah D. Goodman and Pushmeet Kohli and Frank Wood and Philip H. S. Torr , title =. 2017 , archiveprefix =. 1706.00400 , file =

    2017 arXiv

  40. [49]

    2023 , archiveprefix =

    Jonas Pfeiffer and Sebastian Ruder and Ivan Vulić and Edoardo Maria Ponti , title =. 2023 , archiveprefix =. 2302.11529 , file =

    2023 arXiv

  41. [50]

    Hudson and Ehsan Adeli and Russ B

    Rishi Bommasani and Drew A. Hudson and Ehsan Adeli and Russ B. Altman and Simran Arora and Sydney von Arx and Michael S. Bernstein and Jeannette Bohg and Antoine Bosselut and Emma Brunskill and Erik Brynjolfsson and Shyamal Buch and Dallas Card and Rodrigo Castellon and Niladr...

    2021 arXiv

  42. [51]

    2022 , archiveprefix =

    Nasim Rahaman and Martin Weiss and Frederik Träuble and Francesco Locatello and Alexandre Lacoste and Yoshua Bengio and Chris Pal and Li Erran Li and Bernhard Schölkopf , title =. 2022 , archiveprefix =. 2211.02348 , file =

    2022 arXiv

  43. [52]

    2022 , archiveprefix =

    Nasim Rahaman and Martin Weiss and Francesco Locatello and Chris Pal and Yoshua Bengio and Bernhard Schölkopf and Li Erran Li and Nicolas Ballas , title =. 2022 , archiveprefix =. 2210.08031 , file =

    2022 arXiv

  44. [53]

    CoRR , title =

    Andrew Jaegle and Sebastian Borgeaud and Jean. CoRR , title =. 2021 , volume =. 2107.14795 , eprinttype =

    2021 arXiv

  45. [54]

    Towards artificial general intelligence via a multimodal foundation model , year =

    Fei, Nanyi and Lu, Zhiwu and Gao, Yizhao and Yang, Guoxing and Huo, Yuqi and Wen, Jingyuan and Lu, Haoyu and Song, Ruihua and Gao, Xin and Xiang, Tao and Sun, Hao and Wen, Ji-Rong , journal =. Towards artificial general intelligence via a multimodal foundation model , year =. ...

    doi

  46. [55]

    Dusenberry and Du Phan and Mark Collier and Jie Ren and Kehang Han and Zi Wang and Zelda Mariet and Huiyi Hu and Neil Band and Tim G

    Dustin Tran and Jeremiah Liu and Michael W. Dusenberry and Du Phan and Mark Collier and Jie Ren and Kehang Han and Zi Wang and Zelda Mariet and Huiyi Hu and Neil Band and Tim G. J. Rudner and Karan Singhal and Zachary Nado and Joost van Amersfoort and Andreas Kirsch and Rodolp...

    2022 arXiv

  47. [56]

    2020 , archiveprefix =

    Michael Crawshaw , title =. 2020 , archiveprefix =. 2009.09796 , file =

    2020 arXiv

  48. [57]

    Multi-Task Learning for Dense Prediction Tasks: A Survey , year =

    Vandenhende, Simon and Georgoulis, Stamatios and Van Gansbeke, Wouter and Proesmans, Marc and Dai, Dengxin and Van Gool, Luc , journal =. Multi-Task Learning for Dense Prediction Tasks: A Survey , year =. doi:10.1109/TPAMI.2021.3054719 , file =

    2021 doi

  49. [58]

    and Mann, Matthias , journal =

    Zeng, Wen-Feng and Zhou, Xie-Xuan and Willems, Sander and Ammar, Constantin and Wahle, Maria and Bludau, Isabell and Voytik, Eugenia and Strauss, Maximillian T. and Mann, Matthias , journal =. 2022 , issn =. doi:10.1038/s41467-022-34904-3 , file =

    2022 doi

  50. [59]

    Advances in Neural Information Processing Systems , title =

    Swamy, Vinitra and Satayeva, Malika and Frej, Jibril and Bossy, Thierry and Vogels, Thijs and Jaggi, Martin and K\". Advances in Neural Information Processing Systems , title =. 2023 , editor =

    2023

  51. [60]

    2024 , archiveprefix =

    Byeongjun Park and Sangmin Woo and Hyojun Go and Jin-Young Kim and Changick Kim , title =. 2024 , archiveprefix =. 2310.07138 , file =

    2024 arXiv

  52. [61]

    2023 , archiveprefix =

    Mohammed Muqeeth and Haokun Liu and Colin Raffel , title =. 2023 , archiveprefix =. 2306.03745 , file =

    2023 arXiv

  53. [62]

    Salman Asif , title =

    Yash Garg and Nebiyou Yismaw and Rakib Hyder and Ashley Prater-Bennette and M. Salman Asif , title =. 2023 , archiveprefix =. 2310.06124 , file =

    2023

  54. [63]

    2023 , month = Dec, type =

    Sun, Haozhe , school =. 2023 , month = Dec, type =

    2023

  55. [64]

    CoRR , title =

    Bernhard Sch. CoRR , title =. 2021 , volume =. 2102.11107 , eprinttype =

    2021 arXiv

  56. [65]

    Inductive Biases for Deep Learning of Higher-Level Cognition , year =

    Anirudh Goyal and Yoshua Bengio , journal =. Inductive Biases for Deep Learning of Higher-Level Cognition , year =. 2011.15091 , eprinttype =

    2011 arXiv

  57. [66]

    2023 , archiveprefix =

    Clifton Poth and Hannah Sterz and Indraneil Paul and Sukannya Purkayastha and Leon Engländer and Timo Imhof and Ivan Vulić and Sebastian Ruder and Iryna Gurevych and Jonas Pfeiffer , title =. 2023 , archiveprefix =. 2311.11077 , file =

    2023 arXiv

  58. [67]

    Yu and Lichao Sun , title =

    Ce Zhou and Qian Li and Chen Li and Jun Yu and Yixin Liu and Guangjing Wang and Kai Zhang and Cheng Ji and Qiben Yan and Lifang He and Hao Peng and Jianxin Li and Jia Wu and Ziwei Liu and Pengtao Xie and Caiming Xiong and Jian Pei and Philip S. Yu and Lichao Sun , title =. 202...

    2023 arXiv

  59. [68]

    CoRR , title =

    Junyi Li and Tianyi Tang and Wayne Xin Zhao and Ji. CoRR , title =. 2021 , volume =. 2105.10311 , eprinttype =

    2021 arXiv

  60. [69]

    Data Efficient Masked Language Modeling for Vision and Language , year =

    Yonatan Bitton and Gabriel Stanovsky and Michael Elhadad and Roy Schwartz , journal =. Data Efficient Masked Language Modeling for Vision and Language , year =. 2109.02040 , eprinttype =

    arXiv

  61. [70]

    and Healy, Graham and Smeaton, Alan F

    Le-Khac, Phuc H. and Healy, Graham and Smeaton, Alan F. , journal =. Contrastive Representation Learning: A Framework and Review , year =. doi:10.1109/ACCESS.2020.3031549 , file =

    2020 doi

  62. [71]

    A Theoretical Analysis of Contrastive Unsupervised Representation Learning , year =

    Saunshi, Nikunj and Plevrakis, Orestis and Arora, Sanjeev and Khodak, Mikhail and Khandeparkar, Hrishikesh , booktitle =. A Theoretical Analysis of Contrastive Unsupervised Representation Learning , year =

  63. [72]

    Martinez and Juan Manuel Gorriz , journal =

    Yu-Dong Zhang and Zhengchao Dong and Shui-Hua Wang and Xiang Yu and Xujing Yao and Qinghua Zhou and Hua Hu and Min Li and Carmen Jiménez-Mesa and Javier Ramirez and Francisco J. Martinez and Juan Manuel Gorriz , journal =. Advances in multimodal data fusion in neuroimaging: Ov...

    2020 doi

  64. [73]

    Deep multimodal fusion for semantic image segmentation: A survey , year =

    Yifei Zhang and Désiré Sidibé and Olivier Morel and Fabrice Mériaudeau , journal =. Deep multimodal fusion for semantic image segmentation: A survey , year =. doi:https://doi.org/10.1016/j.imavis.2020.104042 , keywords =

    2020 doi

  65. [74]

    Attention Bottlenecks for Multimodal Fusion , year =

    Nagrani, Arsha and Yang, Shan and Arnab, Anurag and Jansen, Aren and Schmid, Cordelia and Sun, Chen , booktitle =. Attention Bottlenecks for Multimodal Fusion , year =

  66. [75]

    and Marks, Tim K

    Hori, Chiori and Hori, Takaaki and Lee, Teng-Yok and Zhang, Ziming and Harsham, Bret and Hershey, John R. and Marks, Tim K. and Sumi, Kazuhiko , booktitle =. Attention-Based Multimodal Fusion for Video Description , year =

  67. [76]

    Multi-source heterogeneous data fusion , year =

    Zhang, Lili and Xie, Yuxiang and Xidao, Luan and Zhang, Xin , booktitle =. Multi-source heterogeneous data fusion , year =. doi:10.1109/ICAIBD.2018.8396165 , keywords =

    2018 doi

  68. [77]

    Lifting the Curse of Multilinguality by Pre-training Modular Transformers , year =

    Pfeiffer, Jonas and Goyal, Naman and Lin, Xi and Li, Xian and Cross, James and Riedel, Sebastian and Artetxe, Mikel , booktitle =. Lifting the Curse of Multilinguality by Pre-training Modular Transformers , year =. doi:10.18653/v1/2022.naacl-main.255 , url =

    2022 doi

  69. [78]

    Beyond English-Centric Multilingual Machine Translation , year =

    Angela Fan and Shruti Bhosale and Holger Schwenk and Zhiyi Ma and Ahmed El-Kishky and Siddharth Goyal and Mandeep Baines and Onur Celebi and Guillaume Wenzek and Vishrav Chaudhary and Naman Goyal and Tom Birch and Vitaliy Liptchinsky and Sergey Edunov and Michael Auli and Arma...

  70. [79]

    and Zettlemoyer, Luke , booktitle =

    Gururangan, Suchin and Lewis, Mike and Holtzman, Ari and Smith, Noah A. and Zettlemoyer, Luke , booktitle =. 2022 , address =. doi:10.18653/v1/2022.naacl-main.407 , url =

    2022 doi

  71. [80]

    Smith and Luke Zettlemoyer , title =

    Margaret Li and Suchin Gururangan and Tim Dettmers and Mike Lewis and Tim Althoff and Noah A. Smith and Luke Zettlemoyer , title =. 2022 , archiveprefix =. 2208.03306 , primaryclass =

    2022 arXiv

  72. [81]

    Modular Networks: Learning to Decompose Neural Computation , year =

    Kirsch, Louis and Kunze, Julius and Barber, David , booktitle =. Modular Networks: Learning to Decompose Neural Computation , year =

  73. [82]

    2022 , archiveprefix =

    Andrea Gesmundo and Jeff Dean , title =. 2022 , archiveprefix =. 2205.12755 , primaryclass =

    2022 arXiv

  74. [83]

    Ponti and Alessandro Sordoni and Yoshua Bengio and Siva Reddy , title =

    Edoardo M. Ponti and Alessandro Sordoni and Yoshua Bengio and Siva Reddy , title =. 2022 , archiveprefix =. 2202.13914 , primaryclass =

    2022 arXiv

  75. [84]

    2023 , archiveprefix =

    Lucas Caccia and Edoardo Ponti and Zhan Su and Matheus Pereira and Nicolas Le Roux and Alessandro Sordoni , title =. 2023 , archiveprefix =. 2211.03831 , primaryclass =

    2023 arXiv

  76. [85]

    Supermasks in Superposition , year =

    Wortsman, Mitchell and Ramanujan, Vivek and Liu, Rosanne and Kembhavi, Aniruddha and Rastegari, Mohammad and Yosinski, Jason and Farhadi, Ali , booktitle =. Supermasks in Superposition , year =

  77. [86]

    2021 , url =

    Dmitry Lepikhin and HyoukJoong Lee and Yuanzhong Xu and Dehao Chen and Orhan Firat and Yanping Huang and Maxim Krikun and Noam Shazeer and Zhifeng Chen , booktitle =. 2021 , url =

    2021

  78. [87]

    Davison , journal =

    Shikun Liu and Edward Johns and Andrew J. Davison , journal =. End-to-End Multi-Task Learning with Attention , year =. 1803.10704 , eprinttype =

    arXiv

  79. [88]

    2018 , archiveprefix =

    Alex Kendall and Yarin Gal and Roberto Cipolla , title =. 2018 , archiveprefix =. 1705.07115 , file =

    2018 arXiv

  80. [89]

    Self-Supervised Learning of Graph Neural Networks: A Unified Review , year =

    Xie, Yaochen and Xu, Zhao and Zhang, Jingtun and Wang, Zhengyang and Ji, Shuiwang , journal =. Self-Supervised Learning of Graph Neural Networks: A Unified Review , year =. doi:10.1109/TPAMI.2022.3170559 , file =

    2022 doi

  81. [90]

    Self-Supervised Learning for Recommender Systems: A Survey , year =

    Yu, Junliang and Yin, Hongzhi and Xia, Xin and Chen, Tong and Li, Jundong and Huang, Zi , journal =. Self-Supervised Learning for Recommender Systems: A Survey , year =. doi:10.1109/TKDE.2023.3282907 , file =

    2023 doi

  82. [91]

    Self-Supervised Hypergraph Learning for Enhanced Multimodal Representation , year =

    Shu, Hongji and Meng, Chaojun and De Meo, Pasquale and Wang, Qing and Zhu, Jia , journal =. Self-Supervised Hypergraph Learning for Enhanced Multimodal Representation , year =. doi:10.1109/ACCESS.2024.3361680 , file =

    2024 doi

  83. [92]

    , journal =

    Liu, Yixin and Jin, Ming and Pan, Shirui and Zhou, Chuan and Zheng, Yu and Xia, Feng and Yu, Philip S. , journal =. Graph Self-Supervised Learning: A Survey , year =. doi:10.1109/TKDE.2022.3172903 , file =

    2022 doi

  84. [93]

    Graph-Based Contrastive Learning for Description and Detection of Local Features , year =

    Wang, Zihao and Li, Zhen and Li, Xueyi and Chen, Wenjie and Liu, Xiangdong , journal =. Graph-Based Contrastive Learning for Description and Detection of Local Features , year =. doi:10.1109/TNNLS.2022.3208837 , file =

    2022 doi

  85. [94]

    , journal =

    Wu, Lirong and Lin, Haitao and Tan, Cheng and Gao, Zhangyang and Li, Stan Z. , journal =. Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive , year =. doi:10.1109/TKDE.2021.3131584 , file =

    2021 doi

  86. [95]

    Self-supervised graph representations with generative adversarial learning , year =

    Xuecheng Sun and Zonghui Wang and Zheming Lu and Ziqian Lu , journal =. Self-supervised graph representations with generative adversarial learning , year =. doi:https://doi.org/10.1016/j.neucom.2024.127786 , file =

    2024 doi

  87. [96]

    and Braham, Nassim Ait Ali and Mou, Lichao and Zhu, Xiao Xiang , journal =

    Wang, Yi and Albrecht, Conrad M. and Braham, Nassim Ait Ali and Mou, Lichao and Zhu, Xiao Xiang , journal =. Self-Supervised Learning in Remote Sensing: A review , year =. doi:10.1109/MGRS.2022.3198244 , file =

    2022 doi

  88. [97]

    Self-Supervised Learning: Generative or Contrastive , year =

    Liu, Xiao and Zhang, Fanjin and Hou, Zhenyu and Mian, Li and Wang, Zhaoyu and Zhang, Jing and Tang, Jie , journal =. Self-Supervised Learning: Generative or Contrastive , year =. doi:10.1109/TKDE.2021.3090866 , file =

    2021 doi

  89. [98]

    A review of graph neural networks: concepts, architectures, techniques, challenges, datasets, applications, and future directions , year =

    Khemani, Bharti and Patil, Shruti and Kotecha, Ketan and Tanwar, Sudeep , journal =. A review of graph neural networks: concepts, architectures, techniques, challenges, datasets, applications, and future directions , year =. doi:10.1186/s40537-023-00876-4 , file =

    doi

  90. [99]

    Simple and Deep Graph Convolutional Networks , year =

    Chen, Ming and Wei, Zhewei and Huang, Zengfeng and Ding, Bolin and Li, Yaliang , booktitle =. Simple and Deep Graph Convolutional Networks , year =

  91. [100]

    Ben , journal =

    Mesgaran, Mahsa and Hamza, A. Ben , journal =. A graph encoder–decoder network for unsupervised anomaly detection , year =. doi:10.1007/s00521-023-08964-5 , file =

    doi

  92. [101]

    2022 , archiveprefix =

    Yuecai Zhu and Fuyuan Lyu and Chengming Hu and Xi Chen and Xue Liu , title =. 2022 , archiveprefix =. 2203.10480 , file =

    2022 arXiv

  93. [102]

    , journal =

    Hamilton, William L. , journal =. Graph Representation Learning , year =. doi:https://doi.org/10.1007/978-3-031-01588-5 , file =

    doi

  94. [103]

    2024 , archiveprefix =

    OpenAI and al , title =. 2024 , archiveprefix =. 2303.08774 , primaryclass =

    2024 arXiv

  95. [104]

    2024 , eprint=

    TSFormer: A Robust Framework for Efficient UHD Image Restoration , author=. 2024 , eprint=

    2024

  96. [105]

    Understanding open-set recognition by Jacobian norm and inter-class separation , year =

    Jaewoo Park and Hojin Park and Eunju Jeong and Andrew Beng Jin Teoh , journal =. Understanding open-set recognition by Jacobian norm and inter-class separation , year =. doi:https://doi.org/10.1016/j.patcog.2023.109942 , file =

    2023 doi

  97. [106]

    Roberts and Sho Yaida , title =

    Judy Hoffman and Daniel A. Roberts and Sho Yaida , title =. 2019 , archiveprefix =. 1908.02729 , file =

    2019 arXiv

  98. [107]

    Jacobian norm with Selective Input Gradient Regularization for interpretable adversarial defense , year =

    Deyin Liu and Lin Yuanbo Wu and Bo Li and Farid Boussaid and Mohammed Bennamoun and Xianghua Xie and Chengwu Liang , journal =. Jacobian norm with Selective Input Gradient Regularization for interpretable adversarial defense , year =. doi:https://doi.org/10.1016/j.patcog.2023....

    2023 doi

  99. [108]

    Improving DNN Robustness to Adversarial Attacks Using Jacobian Regularization , year =

    Jakubovitz, Daniel and Giryes, Raja , booktitle =. Improving DNN Robustness to Adversarial Attacks Using Jacobian Regularization , year =

  100. [109]

    , title =

    Huber, Peter J. , title =. The Annals of Mathematical Statistics , volume =. 1964 , publisher =

    1964

  101. [110]

    Proceedings of the 37th International Conference on Machine Learning (ICML) , pages =

    Chen, Ting and Kornblith, Simon and Norouzi, Mohammad and Hinton, Geoffrey , title =. Proceedings of the 37th International Conference on Machine Learning (ICML) , pages =. 2020 , publisher =

    2020

  102. [111]

    and Rasch, Malte J

    Gretton, Arthur and Borgwardt, Karsten M. and Rasch, Malte J. and Sch. A Kernel Two-Sample Test , booktitle =

  103. [112]

    Advances in Neural Information Processing Systems (NeurIPS) , volume =

    Ben-David, Shai and Blitzer, John and Crammer, Koby and Pereira, Fernando , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  104. [113]

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Tzeng, Eric and Hoffman, Judy and Saenko, Kate and Darrell, Trevor , title =. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  105. [114]

    arXiv preprint arXiv:2407.13278 , year =

    Deep Time Series Models: A Comprehensive Survey and Benchmark , author =. arXiv preprint arXiv:2407.13278 , year =

    arXiv

  106. [115]

    Artificial Intelligence Review , volume =

    A Comprehensive Survey of Deep Learning for Time Series Forecasting: Architectural Diversity and Open Challenges , author =. Artificial Intelligence Review , volume =. 2025 , url =

    2025

  107. [116]

    Mathematics , volume =

    Deep Time Series Forecasting Models: A Comprehensive Survey , author =. Mathematics , volume =. 2024 , url =

    2024

  108. [117]

    IEEE Xplore , year =

    Deep Learning Models for Time Series Forecasting: A Review , author =. IEEE Xplore , year =

  109. [118]

    arXiv preprint arXiv:2504.04011 , year =

    Foundation Models for Time Series: A Survey , author =. arXiv preprint arXiv:2504.04011 , year =

    arXiv

  110. [119]

    Journal of The Royal Society Interface , author =

    Highly comparative time-series analysis: the empirical structure of time series and their methods , volume =. Journal of The Royal Society Interface , author =. 2013 , pages =. doi:10.1098/rsif.2013.0048 , number =

    2013 doi

  111. [120]

    Data Mining and Knowledge Discovery , author =

    catch22:. Data Mining and Knowledge Discovery , author =. 2019 , keywords =. doi:10.1007/s10618-019-00647-x , language =

    2019 doi

  112. [121]

    IEEE Transactions on Speech and Audio Processing , author =

    Musical genre classification of audio signals , volume =. IEEE Transactions on Speech and Audio Processing , author =. 2002 , pages =. doi:10.1109/TSA.2002.800560 , number =

    2002 doi

  113. [122]

    Sinkhorn

    Cuturi, Marco , year =. Sinkhorn. Advances in

  114. [123]

    Villani, Cédric , year =. Optimal

  115. [124]

    Journal of Machine Learning Research , author =

    A. Journal of Machine Learning Research , author =. 2012 , pages =

    2012

  116. [125]

    Zhang, Kexin and Wen, Qingsong and Zhang, Chaoli and Cai, Rongyao and Jin, Ming and Liu, Yong and Zhang, James and Liang, Yuxuan and Pang, Guansong and Song, Dongjin and Pan, Shirui , month = apr, year =. Self-. doi:10.48550/arXiv.2306.10125 , urldate =

    doi arXiv

  117. [126]

    Zhou, Tian and Niu, PeiSong and Wang, Xue and Sun, Liang and Jin, Rong , month = oct, year =. One. doi:10.48550/arXiv.2302.11939 , urldate =

    doi arXiv

  118. [127]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  119. [128]

    , journal =

    Chitty-Venkata, Krishna Teja and Emani, Murali and Vishwanath, Venkatram and Somani, Arun K. , journal =. Neural Architecture Search for Transformers: A Survey , year =. doi:10.1109/ACCESS.2022.3212767 , file =

    2022 doi

  120. [129]

    2024 , archiveprefix =

    Kai Zhang and Jun Yu and Eashan Adhikarla and Rong Zhou and Zhiling Yan and Yixin Liu and Zhengliang Liu and Lifang He and Brian Davison and Xiang Li and Hui Ren and Sunyang Fu and James Zou and Wei Liu and Jing Huang and Chen Chen and Yuyin Zhou and Tianming Liu and Xun Chen ...

    2024 arXiv

  121. [130]

    CrystalGPT: Enhancing system-to-system transferability in crystallization prediction and control using time-series-transformers , year =

    Niranjan Sitapure and Joseph Sang-Il Kwon , journal =. CrystalGPT: Enhancing system-to-system transferability in crystallization prediction and control using time-series-transformers , year =. doi:https://doi.org/10.1016/j.compchemeng.2023.108339 , file =

    2023 doi

  122. [131]

    ChatGPT-like large-scale foundation models for prognostics and health management: A survey and roadmaps , year =

    Yan-Fu Li and Huan Wang and Muxia Sun , journal =. ChatGPT-like large-scale foundation models for prognostics and health management: A survey and roadmaps , year =. doi:https://doi.org/10.1016/j.ress.2023.109850 , file =

    2023 doi

  123. [132]

    2023 , archiveprefix =

    Qingsong Wen and Tian Zhou and Chaoli Zhang and Weiqi Chen and Ziqing Ma and Junchi Yan and Liang Sun , title =. 2023 , archiveprefix =. 2202.07125 , file =

    2023 arXiv

  124. [133]

    Forschung im Ingenieurwesen , volume=

    Practical Insights into the Perception of Digital Twins: An Analysis of Surveys , author=. Forschung im Ingenieurwesen , volume=

  125. [134]

    A Survey on Mixture of Experts in Large Language Models , year =

    Cai, Weilin and Jiang, Juyong and Wang, Fan and Tang, Jing and Kim, Sunghun and Huang, Jiayi , journal =. A Survey on Mixture of Experts in Large Language Models , year =. doi:10.1109/tkde.2025.3554028 , publisher =

    2025 doi

  126. [135]

    Neural Computation , volume=

    Adaptive Mixtures of Local Experts , author=. Neural Computation , volume=

  127. [136]

    ACM Journal , year=

    The Evolution of Mixture of Experts: A Survey from Basics to Breakthroughs , author=. ACM Journal , year=

  128. [137]

    A Scalable Machine Learning based Device Model with Mixture-of-Expert Neural Networks for Enhanced Accuracy and Efficiency , year=

    Zhou, Yuxiang and Peng, Baokang and Wang, Zian and Li, Yu and Wang, Runsheng and Zhang, Lining , booktitle=. A Scalable Machine Learning based Device Model with Mixture-of-Expert Neural Networks for Enhanced Accuracy and Efficiency , year=

  129. [138]

    Discover Applied Sciences , volume=

    Towards the Application of Machine Learning in Digital Twin Technology: A Multi-Scale Review , author=. Discover Applied Sciences , volume=

  130. [139]

    A modular deep learning methodology for multi-fault machine health diagnostics from sparse and imbalanced multimodal data , journal =

    Sagar Jose and Khanh T.P Nguyen and Kamal Medjaher and Ryad Zemouri and Mélanie Lévesque and Antoine Tahan , keywords =. A modular deep learning methodology for multi-fault machine health diagnostics from sparse and imbalanced multimodal data , journal =. 2026 , issn =. doi:ht...

    2026 doi

  131. [140]

    A decoder-only foundation model for time-series forecasting , url =

    Das, Abhimanyu and Kong, Weihao and Sen, Rajat and Zhou, Yichen , month = apr, year =. A decoder-only foundation model for time-series forecasting , url =. doi:10.48550/arXiv.2310.10688 , urldate =

    doi arXiv

  132. [141]

    Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey , year =

    Jing, Longlong and Tian, Yingli , journal =. Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey , year =. doi:10.1109/TPAMI.2020.2992393 , file =

    2020 doi

  133. [142]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Time series prediction using mixtures of experts , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  134. [143]

    Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , year =

    Mixture-of-Linear-Experts for Long-Term Time Series Forecasting , author =. Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , year =

  135. [144]

    Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification , year =

    Gu, Bin and Zhao, Haitao and Wei, Jibo , journal =. Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification , year =. doi:10.1109/LSP.2026.3673617 , keywords =

    2026 doi

  136. [145]

    A Heterogeneous Multiple-Experts Approach to Low-Frequency Nonintrusive Load Monitoring , year =

    Liang, Zipeng and Yung Chung, Chi and Yang, Haosen and Liang, Jian and Zhang, Wenjie and Dong, Hanjiang and Zhu, Jizhong , journal =. A Heterogeneous Multiple-Experts Approach to Low-Frequency Nonintrusive Load Monitoring , year =. doi:10.1109/TSG.2025.3610482 , keywords =

    2025 doi

  137. [146]

    International Journal of Forecasting , volume =

    Temporal Fusion Transformers for Interpretable Multi-Horizon Time Series Forecasting , author =. International Journal of Forecasting , volume =

  138. [147]

    2020 , eprint=

    GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding , author=. 2020 , eprint=

    2020

  139. [148]

    2022 , eprint=

    Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity , author=. 2022 , eprint=

    2022

  140. [149]

    International Conference on Learning Representations (ICLR) , year =

    Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts , author =. International Conference on Learning Representations (ICLR) , year =

  141. [150]

    NeurIPS Workshop on Time Series in the Age of Large Models , year=

    Mixture of Experts for Time Series Foundation Models , author=. NeurIPS Workshop on Time Series in the Age of Large Models , year=

  142. [151]

    2026 , eprint=

    Time Tracker: Mixture-of-Experts-Enhanced Foundation Time Series Forecasting Model with Decoupled Training Pipelines , author=. 2026 , eprint=

    2026

  143. [152]

    arXiv preprint arXiv:2510.07459 , year =

    MoGU: Mixture-of-Gaussians with Uncertainty-Based Gating for Time Series Forecasting , author =. arXiv preprint arXiv:2510.07459 , year =

  144. [153]

    Zhang, Tianping and Zhang, Yizhuo and Cao, Wei and Bian, Jiang and Yi, Xiaohan and Zheng, Shun and Li, Jian , month = jul, year =. Less. doi:10.48550/arXiv.2207.01186 , urldate =

    doi arXiv

  145. [154]

    Proceedings of the AAAI Conference on Artificial Intelligence , author =

    Are. Proceedings of the AAAI Conference on Artificial Intelligence , author =. 2023 , note =. doi:10.1609/aaai.v37i9.26317 , language =

    2023 doi

  146. [155]

    and Sinthong, Phanwadee and Kalagnanam, Jayant , month = mar, year =

    Nie, Yuqi and Nguyen, Nam H. and Sinthong, Phanwadee and Kalagnanam, Jayant , month = mar, year =. A. doi:10.48550/arXiv.2211.14730 , urldate =

    doi arXiv

  147. [156]

    doi:10.48550/arXiv.2310.06625 , urldate =

    Liu, Yong and Hu, Tengge and Zhang, Haoran and Wu, Haixu and Wang, Shiyu and Ma, Lintao and Long, Mingsheng , month = mar, year =. doi:10.48550/arXiv.2310.06625 , urldate =

    doi arXiv

  148. [157]

    IET Conference Proceedings , author =

    Building a hydro-generator rotor temperature virtual sensor using machine-learning , volume =. IET Conference Proceedings , author =. 2025 , pages =. doi:10.1049/icp.2025.2334 , language =

    2025 doi

  149. [158]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =

  150. [159]

    Seitzer, Maximilian and Tavakoli, Arash and Antic, Dimitrije and Martius, Georg , month = apr, year =. On the

  151. [160]

    IOP Conference Series: Earth and Environmental Science , author =

    Virtual sensors to generate turbine runner blade strains from indirect measurements , volume =. IOP Conference Series: Earth and Environmental Science , author =. 2022 , note =. doi:10.1088/1755-1315/1079/1/012067 , language =

    2022 doi

  152. [161]

    IOP Conference Series: Earth and Environmental Science , author =

    Virtual sensors for indirect strain measurement during. IOP Conference Series: Earth and Environmental Science , author =. 2025 , note =. doi:10.1088/1755-1315/1483/1/012013 , language =

    2025 doi

  153. [162]

    Prognostics and Health Management in Energy and Power Systems: Integrating Situation Awareness into Large-Scale Foundation Models , year =

    Zemouri, Ryad and Raymond, Jean and Komljenovic, Dragan , editor =. Prognostics and Health Management in Energy and Power Systems: Integrating Situation Awareness into Large-Scale Foundation Models , year =

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.