REVIEW 12 cited by
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the prediction capacity. However, there are several severe issues with Transformer that prevent it from being directly applicable to LSTF, including quadratic time complexity, high memory usage, and inherent limitation of the encoder-decoder architecture. To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a $ProbSparse$ self-attention mechanism, which achieves $O(L \log L)$ in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences. (iii) the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference speed of long-sequence predictions. Extensive experiments on four large-scale datasets demonstrate that Informer significantly outperforms existing methods and provides a new solution to the LSTF problem.
Forward citations
Cited by 12 Pith papers
-
TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis
TelecomTS is a new observability dataset from 5G networks that preserves absolute scale and supports multi-modal tasks, showing that current time series and language models struggle with abrupt noisy dynamics.
-
Customizing the Inductive Biases of Softmax Attention using Structured Matrices
Structured-matrix scoring functions, BTT and MLR, let attention escape the low-rank bottleneck and add a distance-dependent compute bias, improving accuracy for fixed compute on regression, language modeling, and forecasting.
-
Time Series Foundation Models for Multivariate Financial Time Series Forecasting
Pretrained TTM shows large transfer and sample-efficiency gains in three financial forecasting tasks relative to training from scratch, but methodological flaws including possible look-ahead bias weaken the quantitati...
-
Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions
CHARM is a 7M-parameter self-supervised embedding model for multivariate time series that uses channel descriptions to beat specialized baselines on forecasting, classification, and anomaly detection.
-
Forking-Sequences: Statistically and Computationally Efficient Multi-Horizon Forecasting with Reduced Volatility
Forking-sequences trains forecasting models on all forecast-creation dates jointly, reducing gradient and forecast variance relative to per-date window sampling.
-
From Noise to Precision: A Diffusion-Driven Approach to Zero-Inflated Precipitation Prediction
A Gaussian-smoothing, Transformer, and DDPM denoising stack is claimed to cut MSE by up to 56.7% on zero-inflated precipitation forecasting, with a theoretical guarantee that does not hold.
-
FinCast: A Foundation Model for Financial Time-Series Forecasting
FinCast, a 1B-parameter sparse-MoE transformer pretrained on 20B+ financial time points, reports 20% and 23% average MSE reductions over SOTA in zero-shot and supervised financial forecasting.
-
Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting
Fine-tuning time series foundation models helps on some datasets and sizes, but the paper's claim that it consistently beats zero-shot forecasting is not supported by the reported experiments.
-
STGAtt: A Spatial-Temporal Unified Graph Attention Network for Traffic Flow Forecasting
STGAtt claims a unified spatial-temporal graph attention model with a neighborhood signal-exchanging mechanism beats state-of-the-art traffic forecasters on PEMS-BAY and SHMetro.
-
Applying multimodal learning to Classify transient Detections Early (AppleCiDEr) I: Data set, methods, and infrastructure
AppleCiDEr combines photometry, images, metadata, and spectra in one deep learning pipeline to classify ZTF transients and variable stars, with high accuracy on common classes but poor performance on tidal disruption events.
-
Contrast & Compress: Learning Lightweight Embeddings for Short Trajectories
A small Transformer trained with a cosine-based triplet loss learns 16-dimensional embeddings that retrieve similar short driving trajectories from Argoverse 2 substantially better than FFT-based triplet training.
-
Generative Modeling of Networked Time-Series via Transformer Architectures
A GPT-style encoder-only transformer generates synthetic time-series data that slightly improves downstream ML performance on two network datasets.
Discussion (0). Sign in to comment.