REVIEW 26 cited by
Transformers in Time Series: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.
Forward citations
Cited by 26 Pith papers
-
MBDiff: Multi-view Behavior-aware Diffusion Model for Probabilistic Utility Data Imputation
A multi-view behavior-aware conditional diffusion model for imputing missing utility-meter data is claimed to beat ten baselines on a Florida utility dataset, though the paper's own tables conflict with parts of the claim.
-
HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data
A transformer with AttentiveCAT yields class-specific, time-step importance scores for wearable health data, beats deep-learning baselines, and beats random time-step selection in masking tests.
-
Emergent Latent-State Computation under Stochastic Volatility
Volatility forecasters develop linearly decodable representations of the next hidden log-volatility state; in long cycles this appears immediately after the input projection and ℓ2 normalization.
-
Transformers with Physics-Informed Encodings and Simulation-Based Inference for Robust Detection of Eccentric Binary Black Holes in Pulsar Timing Array Data
Physics-informed Transformer encodings plus conditional normalizing flows yield sharper, better-calibrated posteriors for eccentric BBHs in white-noise PTA data than physics-agnostic SBI baselines.
-
Frequency-Guided Deformable Networks for Continuous Phase Alignment
RFFT-derived periods guide deformable convolutions with Gaussian RBF interpolation and asymmetric routing to improve multi-task time-series modeling over rigid grids and bilinear sampling.
-
Predicting Large-scale Urban Network Dynamics with Energy-informed Graph Neural Diffusion
A scalable spatiotemporal Transformer, ScaleSTF, matches the accuracy of much larger models on city-scale forecasting tasks at a fraction of the compute and memory cost.
-
The Power of Architecture: Deep Dive into Transformer Architectures for Long-Term Time Series Forecasting
Bidirectional joint-attention, complete forecasting aggregation, and direct mapping form the most effective Transformer design for long-term time series forecasting.
-
A foundation model with multi-variate parallel attention to generate neuronal activity
MVPFormer, a transformer with disentangled content, time, and channel attention, achieves expert-level zero-shot seizure detection on 50 unseen patients and near-SOTA results on speech decoding, alongside the largest ...
-
Neural Functions for Learning Periodic Signal
NeRT factorizes periodic signals into a sine-based periodic factor and an unbounded scale factor, enabling extrapolation beyond the training range on several periodic benchmarks.
-
Estimating Perceptual Attributes of Haptic Textures Using Visuo-Tactile Data
A visuo-tactile deep network using a CNN autoencoder and ConvLSTM predicts four haptic attribute ratings from images and tool vibrations, beating single-modality baselines in leave-one-out tests.
-
Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
A fused exponential-multiplication hardware unit using logarithmic quantization and exponent adjustment reduces FlashAttention accelerator area by about 29% and power by about 18% without visible accuracy loss on GLUE.
-
Day-Ahead Forecasting of Largest Single Infeed/Outfeed on the Irish Power Grid: A Generative Artificial Intelligence Approach
A transformer-based day-ahead forecaster for Ireland's largest single infeed matches an 8-hour operational model within 1.1% MAPE for infeed, but outfeed errors and the claimed 15% cost savings are not supported.
-
Modular Foundation Models for Time-Series Perception in Digital Twins
A gated bank of frozen self-supervised time-series encoders, aligned and aggregated by a Transformer, supports competitive multi-task perception for digital twins and hydro-generator virtual sensing.
-
TinyD\'ej\`aVu: Smaller RAM and Faster Inference with Neural Networks on MCUs for Sensor Data Streams
TinyDéjàVu turns time-series neural-network layers into streaming buffers (SSMs), cutting peak RAM by up to 99% and redundant compute on overlapping windows for microcontroller inference.
-
TwinTac: A Wide-Range, Highly Sensitive Tactile Sensor with Real-to-Sim Digital Twin Sensor Model
A tactile sensor made from eight barometer chips reads forces from 0.01 N to over 200 N, and a learned FEM-to-signal model generates simulated tactile data that lifts shape classification accuracy from 33.6% to 95%.
-
Multimodal Behavioral Patterns Analysis with Eye-Tracking and LLM-Based Reasoning
A human-AI framework uses horizontal and vertical segmentation with LLMs, expert co-scoring, and LSTM anomaly detection to extract behavioral patterns from eye-tracking data.
-
Scalable Unsupervised Segmentation via Random Fourier Feature-based Gaussian Process
RFF-GP-HSMM speeds up unsupervised time-series segmentation by approximating Gaussian processes with random Fourier features, cutting computation time by up to 278 times on motion capture data with similar accuracy.
-
Evaluation of a Foundational Model and Stochastic Models for Forecasting Sporadic or Spiky Production Outages of High-Performance Machine Learning Services
On seven years of monthly production outage counts from a large ML service, a fine-tuned TimesFM foundation model beats moving-average and autoregressive baselines for total outages, but per root cause the best model varies.
-
Benchmarking Unsupervised Strategies for Anomaly Detection in Multivariate Time Series
Across ten public datasets, a reconstruction-based inverted transformer with per-variate anomaly labelling achieves the best or tied best MCC on most datasets, but the comparison is weakened by test-set-based configur...
-
FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
FMMVCC combines Mamba-based encoders with multi-view contrastive learning and fuzzy clustering to achieve state-of-the-art univariate time series clustering with linear computational complexity.
-
TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization
TinyMusician distills MusicGen and applies hand-picked mixed-precision quantization to make a 1.04 GB on-device music generator, but the headline '93% quality, 55% smaller' claims conflict with the paper's own tables.
-
Transformer with Koopman-Enhanced Graph Convolutional Network for Spatiotemporal Dynamics Forecasting
TK-GCN, a Koopman-enhanced graph convolution plus Transformer model, is proposed for spatiotemporal forecasting; its claimed consistent superiority is contradicted by its own ablation results.
-
Time Series Transformer-Based Modeling of Pavement Skid and Texture Deterioration
A time series transformer is reported to predict post-milling skid number with R2=0.981, but the evaluation may leak temporal information and the underlying data counts are inconsistent.
-
Synthetic Time Series Forecasting with Transformer Architectures: Extensive Simulation Benchmarks
Autoformer and PatchTST outperform Informer across synthetic forecasting benchmarks, while a proposed Koopman-Transformer hybrid is illustrated on Van der Pol and Lorenz systems.
-
Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?
LLM4TS_FS achieves the best MSE on four of seven long-term datasets, but the claimed broad advantage of pre-trained large models over small transformers is not consistent across all benchmarks.
-
W4S4: WaLRUS Meets S4 for Long-Range Sequence Modeling
W4S4 initializes S4 state space models with WaLRUS wavelet frames and reports better delay reconstruction and classification accuracy than HiPPO-based S4, with frozen (A,B).
Discussion (0). Sign in to comment.