Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Embedding each satellite patch's true time, latitude, and longitude into a model's position IDs beats prior AI typhoon forecasters, cutting intensity error by 12%.

desk verdict A plausible, well-ablated method for injecting physical coordinates into VLM position encodings, but the unspecified pixel-to-geography conversion is a load-bearing gap that blocks reproduction. read the letter →

arxiv 2506.14786 v1 pith:DUIM3GXY submitted 2025-05-27 cs.LG cs.AIcs.CV

Haobo Li , Eunseo Jung , Zixin Chen , Zhaowei Wang , Yueya Wang , Huamin Qu , Alexis Kai Hon Lau This is my paper
classification cs.LGcs.AIcs.CV
keywords physics-informedpositionencodingmultimodaltimeseriesforecastingsatelliteimagerytyphoonvisionlanguagemodelpositionalembeddinggeospatialalignmentvariant-frequency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PIPE is the claim that the physical context of an image — when and where it was taken — should be built into how a vision-language model indexes and embeds its tokens, not treated as an afterthought. The paper argues that satellite imagery carries forecast information that text-only and time-series-only multimodal methods leave unused, and that encoding each image patch's hour within the year, latitude, and longitude into position IDs plus a variable-wavelength sinusoidal embedding materially improves alignment between pixels and numerical weather series. On the Digital Typhoon benchmark, the method beats both the base vision-language model and all prior AI forecasters, cutting typhoon intensity mean absolute error by about 12 percent at a six-hour lead time. A reader should care because the change is a cheap plug-in — no new architecture, only redefined position encoding — with gains that hold across lead times, variables, and model sizes.

What carries the argument

The argument is carried by two coupled encoding changes applied to a base vision-language model. The first is physics-informed positional indexing: instead of indexing image patches by their order in the sequence, each patch's position ID is the triple $(t, \mathrm{lat}, \mathrm{lng})$ computed from the image's timestamp and the patch center's true coordinates, with the three ranges mapped to negative integers so they do not overlap the text tokens' sequential IDs; these IDs then feed the model's rotary position embedding, so attention is computed over physically meaningful distances. The second is a variant-frequency sinusoidal function: the standard Transformer sine-cosine encoding is multiplied by $2\pi/p$ with per-variable wavelengths ($p=366$ for day-of-year, 24 for hour, 180 for latitude, 360 for longitude), and the model dimension is split into temporal and spatial halves, producing the position embedding that is added to the input embeddings. The negative-range mapping is load-bearing: sharing the ID range between vision and text degrades intensity mean absolute error from 1.515 to 1.961 hPa, the largest single ablation swing in the paper.

What would settle it

A direct test: rerun the physics-informed indexing ablation with each patch's latitude and longitude replaced by random values drawn uniformly from the same ranges; if the intensity mean absolute error gain (1.617 to 1.515 hPa) persists, the improvement is not coming from true geography. A complementary check is to reconstruct the pixel-to-geographic conversion from the dataset's stated geometry — 512-by-512 pixel images covering 1250 km from the typhoon center under azimuthal equal-area projection — and verify that get_center's patch coordinates match true positions to within a patch width.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that physics-informed positional encoding turns satellite images from raw pixels into physically grounded tokens and thereby improves cross-modal forecasting. Each image patch receives a three-dimensional position ID built from real physics — time as day-of-year times 24 plus hour, latitude on a 0-to-180 scale, longitude on a 0-to-360 scale — pushed into negative values so the vision tokens never collide with text-token positions, and a second channel adds a variant-frequency sinusoidal embedding whose wavelengths match each physical variable (366 days, 24 hours, 180 degrees of latitude, 360 degrees of longitude). With this in place, PIPE-3B reaches a 1.515 hPa intensity mean absolute error at a six-hour lead time, against 1.617 for the unmodified Qwen-2.5-VL-3B and 1.724 for TiDE, the best vision-free forecaster; the ablations attribute roughly an eight percent gain to adding vision and a further six percent to the physics-informed indexing. The paper reads this as evidence that explicit global physical knowledge, shared across all typhoon instances, is what current pixel-level alignment misses.

Load-bearing premise

The load-bearing premise is that every image patch can be assigned a correct latitude and longitude from the typhoon center and the image geometry; the extraction algorithm calls an unspecified function get_center(T_image, lat_image, lng_image), and the paper never gives the pixel-to-geographic conversion for the azimuthal equal-area projection used by the dataset, so if that mapping is wrong, the physics-informed position IDs encode false geography and the reported gains would not transfer or reproduce.

Editorial extensions

If this is right

  • Adding satellite vision to the same LLM backbone improves typhoon intensity mean absolute error by about 8 percent; adding physics-informed indexing on top contributes another 6 percent.
  • The negative-value mapping of physics position IDs is essential: letting vision and text tokens share the positional ID range collapses the gain, moving intensity mean absolute error from 1.515 to 1.961 hPa.
  • The gains are not limited to intensity: latitude, longitude, and track-distance errors all improve, and the advantage persists across lead times from 1 to 12 hours and across the 3B, 7B, and 32B (LoRA) model scales.
  • PIPE is a plug-in rather than a new architecture — only position IDs and the sinusoidal embedding change — so it can be dropped into existing VLMs at low training cost (about two hours on four H800 GPUs for the 3B variant).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Nothing in the encoding is typhoon-specific, so the natural next experiment is transferring PIPE to other geophysical forecasting tasks with geographic imagery, such as wildfire spread, sea-ice drift, or flood mapping; the paper does not run these, but the mechanism would generalize unchanged.
  • The strong ablation result for the negative-ID trick suggests a broader design principle worth testing outside weather: giving different modalities disjoint position-ID ranges in a multimodal LLM may reduce cross-modal interference even without any physics content.
  • The physical wavelengths are hand-chosen constants; a learnable-wavelength variant would test whether 366, 24, 180, and 360 are near-optimal or merely adequate, and could squeeze additional accuracy from the same scheme.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces PIPE, a physics-informed positional encoding scheme for vision-language-model-based multimodal time series forecasting. The method assigns each image patch a positional ID derived from the timestamp and the latitude/longitude of the patch center, maps these vision IDs to negative ranges to avoid conflict with text token IDs, and augments the sinusoidal positional embedding with variable-specific wavelengths (p_day=366, p_hour=24, p_lat=180, p_lng=360). The authors evaluate on Digital Typhoon for 6- and 12-hour typhoon intensity/latitude/longitude forecasting, comparing with time-series baselines, a Qwen-2.5-VL-3B vision baseline, and domain models. They report that PIPE-3B achieves 1.515 hPa intensity MAE at 6h, improves on Qwen-2.5-VL-3B (1.617) and TiDE (1.724), and that ablations removing vision, physics-informed indexing, negative ID mapping, and variant-frequency encoding each degrade performance. Code is promised in the supplementary material.

Significance. The central empirical claim — that adding physics-informed positional IDs improves forecasting over both time-series-only baselines and a VLM with standard 3D indexing — is supported by held-out evaluations and a consistent ablation pattern. The contribution is lightweight and does not change the VLM architecture, and the paper reports standard deviations over three seeds for the main model. If the position-to-geography mapping is fully specified and the SOTA claim over domain models is appropriately scoped, the method would be a useful and generalizable embedding change for satellite-image time-series forecasting. However, as submitted, the paper omits the specification of the patch-center geographic coordinate computation, which is the defining input of the method, and it overstates the comparison with domain baselines that were not run under a common protocol.

major comments (3)
  1. [Appendix B, Algorithm 2; Section 3.3.1] Algorithm 2 calls get_center(T_image, lat_image, lng_image) to compute the latitude and longitude of each image patch center, but no definition of this function is given anywhere in the manuscript. This is load-bearing: every vision-token position ID in the physics-informed indexing scheme is derived from these patch-center coordinates, and the ablation in Tables 2 and 5 attributes roughly 6% intensity MAE to this indexing. Table 3 states that Digital Typhoon images use an azimuthal equal-area projection, cover 1250 km from the center, and are resized from 512x512 to 224x224; converting patch pixel coordinates to geographic coordinates under this projection requires an explicit inverse projection formula centered on the typhoon center and a ground-sampling-distance calculation after resizing. Please provide the exact formula (or the precise code path) for get_center, and state whether the projection is centered on the typhoon center or the image center. If get_center instead uses linear pixel interpolation or assigns the storm-center coordinates to all patches, the claimed 'physics-informed' property does not hold and the gains would be unlikely to transfer to other satellite datasets.
  2. [Section 4.2; Tables 1 and 4] The domain baselines (ECMWF-HRES, PanGu, GenCast, TIFS) are copied from their original papers 'without retraining,' as stated in Section 4.2, and the reported metrics in Tables 1 and 4 are not aligned with the paper's protocol: the domain rows contain only one metric each (e.g., distance MAE for ECMWF-HRES, intensity RMSE for PanGu and TIFS), with no indication of matching lead time, target definition, or evaluation subset. Consequently, the abstract's claim of 'state-of-the-art performance in ... climate domain methods' is unsupported. Please either evaluate these or equivalent domain models under the same protocol, or explicitly restrict the SOTA claim to the AI-based baselines that were trained and evaluated in this paper.
  3. [Section 4.4; Table 1; Abstract] The widely advertised '12% improvement in typhoon intensity forecasting over prior works' is computed relative to TiDE (1.724 vs 1.515 hPa MAE), which is not a 'prior work' in multimodal or domain-specific typhoon forecasting, and Table 1 also shows that PIPE-3B has a higher intensity RMSE (2.981) than TiDE (2.819) and PatchTST (2.867). Please qualify the 12% claim as relative to the best unimodal time-series baseline under this paper's protocol, and adjust the 'superiority across benchmarks' phrasing accordingly.
minor comments (5)
  1. [Appendix B.2, Equation (9)] In the last line of Equation (9), the cosine for longitude uses p_hour in the denominator; this should presumably be p_lng = 360. Please correct.
  2. [Section 3.3.2, Equation (8)] Equation (8) introduces sin(pos/10000^{2i/d_model} x 2pi/p), but it is not clear whether pos here is tday, thour, lat, lng or the combined temporal ID from Equation (7); please clarify the relationship between Equation (7)'s t = tday x 24 + thour and the separate encodings of tday and thour in Equation (9).
  3. [Appendix B, Algorithm 2] The notation 'lat_image, lng_image <- x' is ambiguous; please specify which fields of x are used and how the typhoon-center coordinates are obtained for each image.
  4. [Section 4.4; Table 1] There are typographical artifacts in the table and text, e.g., 'TimeXer [51])' and 'One Fits All [60])' with unbalanced parentheses.
  5. [Figures 2 and 4] The schematic arrows in Figure 2 are difficult to read, and Figure 4's caption does not explain what the color scale represents; please improve the readability of both figures.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: PIPE's claimed gains come from an empirically evaluated position-encoding change on held-out data, not from a fitted input or self-citation chain.

full rationale

No circular step can be exhibited. PIPE's two components are defined independently of the forecasting targets: physics-informed positional indexing assigns image-token IDs from timestamps and patch-center latitude/longitude computed from the current image and Best Track metadata (Section 3.3.1, Appendix B, Algorithms 1-4), while variant-frequency sinusoidal encoding uses fixed physical periods (pday = 366, phour = 24, platitude = 180, plongitude = 360; Equation 8 and Algorithm 4). The forecast targets are future intensity, latitude, and longitude, evaluated on a held-out typhoon split (Section 4.1). Gains in Tables 1, 2, 4, and 5 are comparisons against independently trained baselines and ablated variants; no parameter is fitted to a subset and then reported as a prediction of a closely related quantity. The design choices of negative position-ID ranges and physical periods are motivated by ablation results, but they are not fitted values that force the reported outcome by construction. There are no load-bearing self-citations and no imported uniqueness theorem. The main substantive gap, that Algorithm 2 calls get_center(T_image, lat_image, lng_image) without specifying the inverse azimuthal equal-area projection from Digital Typhoon's 512x512 pixel geometry to geographic patch-center coordinates (Appendix D, Table 3), is a reproducibility and correctness concern about whether the position IDs are truly physics-informed; it is not a circular reduction. Even an incorrect or approximate get_center would not make the predictions equivalent to the inputs by definition; it would undermine the method's premise. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the Digital Typhoon dataset and Best Track labels, the transferability of a frozen pretrained vision encoder to typhoon imagery, and an unspecified pixel-to-geo mapping for patch centers. The only hand-chosen numbers that affect the method are the four physical periods used in the variant-frequency sinusoidal function. No new physical entities are introduced.

free parameters (1)
  • Variant-frequency periods p_day=366, p_hour=24, p_lat=180, p_lng=360 = p_day=366, p_hour=24, p_lat=180, p_lng=360
    Hand-chosen periods in Equation 9 and Section 3.3.2 that define the sinusoidal embedding for physical variables. They are physically motivated but hand-set, and the method's benefit depends on them.
assumptions (4)
  • domain assumption Best Track (IBTrACS/JMA) annotations are reliable enough to serve as ground truth for intensity, latitude, and longitude
    Section 4.1 treats Best Track as the target for training and evaluation; any biases or noise in these reanalyses are inherited by all reported numbers.
  • domain assumption The pretrained Qwen-2.5-VL vision encoder, kept frozen, extracts features from typhoon satellite images that are useful for forecasting
    Section 3.2 freezes the vision encoder; if the encoder lacks domain-specific visual features, the multimodal alignment and the claimed vision gain would not materialize.
  • domain assumption Pixel-to-geographic coordinate mapping for image patches is correct and consistent across the azimuthal equal-area projection
    Algorithm 2 calls get_center to assign latitude and longitude to patch centers, but the paper does not specify the projection math; PIPE's physical position IDs are only as good as this mapping.
  • standard math Standard sinusoidal position encoding and RoPE behave as described and remain compatible after appending negative position IDs
    Section 3.3.1 relies on RoPE over negative position IDs and Equation 1 for text tokens; no proof or numerical check is given for this combination.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series." pith.science (2026). https://pith.science/paper/DUIM3GXY

@misc{pith2026250614786,
  author       = {Pith},
  title        = {Pith review of: PIPE: Physics-Informed Position Encoding for Alignment of Satellite Images and Time Series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DUIM3GXY}},
  note         = {Machine review of arXiv:2506.14786}
}
read the original abstract

Multimodal time series forecasting is foundational in various fields, such as utilizing satellite imagery and numerical data for predicting typhoons in climate science. However, existing multimodal approaches primarily focus on utilizing text data to help time series forecasting, leaving the visual data in existing time series datasets untouched. Furthermore, it is challenging for models to effectively capture the physical information embedded in visual data, such as satellite imagery's temporal and geospatial context, which extends beyond images themselves. To address this gap, we propose physics-informed positional encoding (PIPE), a lightweight method that embeds physical information into vision language models (VLMs). PIPE introduces two key innovations: (1) a physics-informed positional indexing scheme for mapping physics to positional IDs, and (2) a variant-frequency positional encoding mechanism for encoding frequency information of physical variables and sequential order of tokens within the embedding space. By preserving both the physical information and sequential order information, PIPE significantly improves multimodal alignment and forecasting accuracy. Through the experiments on the most representative and the largest open-sourced satellite image dataset, PIPE achieves state-of-the-art performance in both deep learning forecasting and climate domain methods, demonstrating superiority across benchmarks, including a 12% improvement in typhoon intensity forecasting over prior works. Our code is provided in the supplementary material.

Figures

Figures reproduced from arXiv: 2506.14786 by the authors.

Figure 1
Figure 1. The multimodal time series forecasting task and the forecasting results for Typhoon Yutu [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of physics-informed positional encoding. It includes: (1) a physics [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The visualization for the regression results between predicted values and true values (lead [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The 64-dimensional positional encoding for the physical variables. Each row represents [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: The performance across leading times ranging from 1 to 12 hours. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 7
Figure 7. Figure 7: Visualization of attention (normalized to 0-1 in each step) from the penultimate layer of [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 6
Figure 6. Figure 6: The results of Typhoon Phanfone comparison between PIPE, removing satellite images, [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 8
Figure 8. Figure 8: Visualization of attention (normalized to 0-1 in each step) using Attention Rollout of the [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Transformers with Physics-Informed Encodings and Simulation-Based Inference for Robust Detection of Eccentric Binary Black Holes in Pulsar Timing Array Data

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Physics-informed Transformer encodings plus conditional normalizing flows yield sharper, better-calibrated posteriors for eccentric BBHs in white-noise PTA data than physics-agnostic SBI baselines.

  2. KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering

    cs.CL 2025-09 conditional novelty 6.0 of 10

    KERAG improves knowledge-graph question answering by retrieving broad entity neighborhoods instead of exact query paths and using a fine-tuned chain-of-thought summarizer.

Reference graph

Works this paper leans on

77 extracted references · 49 canonical work pages · cited by 2 Pith papers

  1. [1]

    Quantifying attention flow in transformers

    Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Dan Ju- rafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 4190–4197, Online, July

    work page

  2. [2]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jian- qiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report....

    arXiv 2025

  3. [3]

    Accurate medium-range global weather forecasting with 3d neural networks

    Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533– 538, 2023

    work page 2023

  4. [4]

    Llm4ts: Aligning pre- trained llms as data-efficient time-series forecasters

    Ching Chang, Wei-Yao Wang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Aligning pre- trained llms as data-efficient time-series forecasters. ACM Trans. Intell. Syst. Technol., 16(3), April 2025

    work page 2025

  5. [5]

    Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting

    Peng Chen, Yingying Zhang, Yunyao Cheng, Yang Shu, Yihang Wang, Qingsong Wen, Bin Yang, and Chenjuan Guo. Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting. arXiv preprint arXiv:2402.05956, 2024

    arXiv 2024

  6. [6]

    Structured information extraction from scientific text with large language models

    John Dagdelen, Alexander Dunn, Sanghoon Lee, Nicholas Walker, Andrew S Rosen, Gerbrand Ceder, Kristin A Persson, and Anubhav Jain. Structured information extraction from scientific text with large language models. Nature Communications, 15(1):1418, 2024

    2024

  7. [7]

    Transformer-XL: Attentive language models beyond a fixed-length context

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. Transformer-XL: Attentive language models beyond a fixed-length context. In Anna Korho- nen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 2978–2988, Florence, Italy, July 2019...

    work page 2019

  8. [8]

    Long- term forecasting with tide: Time-series dense encoder.arXiv preprint arXiv:2304.08424, 2023

    Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur, Rajat Sen, and Rose Yu. Long- term forecasting with tide: Time-series dense encoder.arXiv preprint arXiv:2304.08424, 2023

    arXiv 2023

Show all 77 references
  1. [9]

    Part v: Ensemble prediction system

    IFS DOCUMENTATION-Cy40r1. Part v: Ensemble prediction system. 2020

    2020

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

    2010 arXiv

  3. [11]

    The use of arima models for reliability forecasting and analysis

    Siu Lau Ho and Min Xie. The use of arima models for reliability forecasting and analysis. Computers & industrial engineering, 35(1-2):213–216, 1998

    1998

  4. [12]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. 10

    1997

  5. [13]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  6. [14]

    Combining satellite imagery and machine learning to predict poverty.Science, 353(6301):790–794, 2016

    Neal Jean, Marshall Burke, Michael Xie, W Matthew Alampay Davis, David B Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to predict poverty.Science, 353(6301):790–794, 2016

    2016

  7. [15]

    Gpt4mts: Prompt-based large language model for multimodal time-series forecasting

    Furong Jia, Kevin Wang, Yixiang Zheng, Defu Cao, and Yan Liu. Gpt4mts: Prompt-based large language model for multimodal time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23343–23351, 2024

    2024

  8. [16]

    Time-LLM: Time series fore- casting by reprogramming large language models

    Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-LLM: Time series fore- casting by reprogramming large language models. In International Conference on Learning Representations...

    2024

  9. [17]

    Algorithms for geodesics

    Charles FF Karney. Algorithms for geodesics. Journal of Geodesy, 87:43–55, 2013

    2013

  10. [18]

    Rethinking positional encoding in language pre-training

    Guolin Ke, Di He, and Tie-Yan Liu. Rethinking positional encoding in language pre-training. In International Conference on Learning Representations, 2021

    2021

  11. [19]

    Reformer: The efficient transformer

    Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference on Learning Representations, 2020

    2020

  12. [20]

    Digital typhoon: Long-term satellite image dataset for the spatio-temporal modeling of tropical cyclones

    Asanobu Kitamoto, Jared Hwang, Bastien Vuillod, Lucas Gautier, Yingtao Tian, and Tarin Clanuwat. Digital typhoon: Long-term satellite image dataset for the spatio-temporal modeling of tropical cyclones. Advances in Neural Information Processing Systems , 36:40623–40636, 2023

    2023

  13. [21]

    The international best track archive for climate stewardship (ibtracs) unifying trop- ical cyclone data

    Kenneth R Knapp, Michael C Kruk, David H Levinson, Howard J Diamond, and Charles J Neumann. The international best track archive for climate stewardship (ibtracs) unifying trop- ical cyclone data. Bulletin of the American Meteorological Society, 91(3):363–376, 2010

    2010

  14. [22]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  15. [23]

    Urbangpt: Spatio-temporal large language models

    Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. Urbangpt: Spatio-temporal large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5351–5362, 2024

    2024

  16. [24]

    ROUGE: A package for automatic evaluation of summaries

    Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summa- rization Branches Out, pages 74–81, Barcelona, Spain, July 2004. Association for Computa- tional Linguistics

    2004

  17. [25]

    Cat: Cross attention in vision transformer

    Hezheng Lin, Xing Cheng, Xiangyu Wu, and Dong Shen. Cat: Cross attention in vision transformer. In 2022 IEEE international conference on multimedia and expo (ICME) , pages 1–6. IEEE, 2022

    2022

  18. [26]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Ad- vances in neural information processing systems, 36:34892–34916, 2023

    2023

  19. [27]

    Unitime: A language-empowered unified model for cross-domain time series forecast- ing

    Xu Liu, Junfeng Hu, Yuan Li, Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmer- mann. Unitime: A language-empowered unified model for cross-domain time series forecast- ing. In Proceedings of the ACM Web Conference 2024, pages 4095–4106, 2024

    2024

  20. [28]

    Learning to encode po- sition for transformer with continuous dynamical model

    Xuanqing Liu, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh. Learning to encode po- sition for transformer with continuous dynamical model. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proce...

    2020

  21. [29]

    itransformer: Inverted transformers are effective for time series forecasting

    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023. 11

    2023 arXiv

  22. [30]

    Auto- times: Autoregressive time series forecasters via large language models

    Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Auto- times: Autoregressive time series forecasters via large language models. arXiv preprint arXiv:2402.02370, 2024

    2024 arXiv

  23. [31]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  24. [32]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730, 2022

    2022 arXiv

  25. [33]

    Operational use of the typhoon intensity forecasting scheme based on ships (tifs) and commencement of five-day tropical cyclone intensity forecasts

    Marika Ono, Shoji Notsuhara, Junya Fukuda, Yohko Igarashi, and Kotaro Bessho. Operational use of the typhoon intensity forecasting scheme based on ships (tifs) and commencement of five-day tropical cyclone intensity forecasts. ENE, 128(40):128–7, 2019

    2019

  26. [34]

    Gpt-4 technical report, 2024

    OpenAI et al. Gpt-4 technical report, 2024

    2024

  27. [35]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  28. [36]

    Pytorch: An imperative style, high- performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  29. [37]

    Smith, and Mike Lewis

    Ofir Press, Noah A. Smith, and Mike Lewis. Shortformer: Better language modeling using shorter inputs. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors,Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Int...

    2021

  30. [38]

    Proba- bilistic weather forecasting with machine learning

    Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R Andersson, Andrew El-Kadi, Do- minic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, et al. Proba- bilistic weather forecasting with machine learning. Nature, 637(8044):84–90, 2025

    2025

  31. [39]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, ...

    2021

  32. [40]

    Learning representations by back-propagating errors

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323(6088):533–536, 1986

    1986

  33. [41]

    Self-attention with relative position repre- sentations

    Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position repre- sentations. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguis- tics:...

    2018

  34. [42]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomput., 568(C), February 2024

    2024

  35. [43]

    TEST: Text prototype aligned em- bedding to activate LLM’s ability for time series

    Chenxi Sun, Hongyan Li, Yaliang Li, and Shenda Hong. TEST: Text prototype aligned em- bedding to activate LLM’s ability for time series. In The Twelfth International Conference on Learning Representations, 2024

    2024

  36. [44]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 12

    2023 arXiv

  37. [45]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  38. [46]

    Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology

    Mark Veillette, Siddharth Samsi, and Chris Mattioli. Sevir: A storm event imagery dataset for deep learning applications in radar and satellite meteorology. Advances in Neural Information Processing Systems, 33:22009–22019, 2020

    2020

  39. [47]

    Chattime: A unified multimodal time series foundation model bridging numerical and textual data

    Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, Lei Zhang, and Jianxin Liao. Chattime: A unified multimodal time series foundation model bridging numerical and textual data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, ...

    2025

  40. [48]

    Timemixer: Decomposable multiscale mixing for time series forecasting

    Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405.14616, 2024

    2024 arXiv

  41. [49]

    Crossformer++: A versatile vision transformer hinging on cross-scale attention

    Wenxiao Wang, Wei Chen, Qibo Qiu, Long Chen, Boxi Wu, Binbin Lin, Xiaofei He, and Wei Liu. Crossformer++: A versatile vision transformer hinging on cross-scale attention. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3123–3136, 2023

    2023

  42. [50]

    Contrast everything: A hierarchical contrastive framework for medical time-series

    Yihe Wang, Yu Han, Haishuai Wang, and Xiang Zhang. Contrast everything: A hierarchical contrastive framework for medical time-series. Advances in Neural Information Processing Systems, 36:55694–55717, 2023

    2023

  43. [51]

    Timexer: Empowering transformers for time series forecasting with exogenous variables

    Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. arXiv preprint arXiv:2402.19072, 2024

    2024 arXiv

  44. [52]

    DA-transformer: Distance-aware trans- former

    Chuhan Wu, Fangzhao Wu, and Yongfeng Huang. DA-transformer: Distance-aware trans- former. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, editors, Proceedings of the 20...

    2021

  45. [53]

    Timesnet: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022

    2022 arXiv

  46. [54]

    Timesnet: Temporal 2d-variation modeling for general time series analysis

    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023

    2023

  47. [55]

    Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting

    Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural infor- mation processing systems, 34:22419–22430, 2021

    2021

  48. [56]

    Self-supervised pretraining of transformers for satellite image time se- ries classification

    Yuan Yuan and Lei Lin. Self-supervised pretraining of transformers for satellite image time se- ries classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:474–487, 2020

    2020

  49. [57]

    Sageformer: Series-aware frame- work for long-term multivariate time-series forecasting

    Zhenwei Zhang, Linghang Meng, and Yuantao Gu. Sageformer: Series-aware frame- work for long-term multivariate time-series forecasting. IEEE Internet of Things Journal , 11(10):18435–18448, 2024

    2024

  50. [58]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024

  51. [59]

    Informer: Beyond efficient transformer for long sequence time-series forecasting

    Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021

    2021

  52. [60]

    One fits all: Power general time series analysis by pretrained lm

    Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems , 36:43322– 43355, 2023. A Prompts Design We show our prompt design for the multimodal time series forecasti...

    2023

  53. [63]

    Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: We state our claim and contributions in our abstract and introduction. Guidelines: • The answer NA means that the abst...

  54. [64]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discuss the limitation in Appendix G. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that the paper ...

  55. [65]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] 24 Justification: We do not include theoretical results. Guidelines: • The answer NA means that the paper do...

  56. [66]

    We also provide the code and data in supplementary materials for reproduction

    Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and da...

  57. [67]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code 25 Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We provide the code and ...

  58. [68]

    We introduce settings, training hyperparameters, optimizer, etc

    Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: Please refer to section 4 and...

  59. [69]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments? Answer: [Yes] Justification: We report standard deviations with three random ...

  60. [70]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We provide the computat...

  61. [71]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: We have checked the NeurIPS Code of Ethics and our research conforms with it. G...

  62. [72]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: We discuss the broader impacts of our work in Appendix F, including the impacts on other domains and scen...

  63. [73]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Our paper poses...

  64. [74]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We properly cite...

  65. [75]

    Guidelines: • The answer NA means that the paper does not release new assets

    New assets Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets? Answer: [Yes] Justification: We provide the readme file for our code in the supplementary materials. Guidelines: • The answer NA means that the...

  66. [76]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?...

  67. [77]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  68. [78]

    Answer: [NA] Justification: We do not use LLMs for the core method development

    Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the ...

    2025

  69. [2021]

    Association for Computational Linguistics

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.