The reviewed record of science sign in
Pith

arxiv: 2606.28546 · v1 · pith:GAZMMIWN · submitted 2026-06-26 · cs.LG

NIVA: A Multimodal Foundation Model for Actionable Earth System Intelligence

Anisha Pal , Aodhan Sweeney , Kyle Heyblom , Kalai Ramea This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-30 00:50 UTCgrok-4.3pith:GAZMMIWNrecord.jsonopen to challenge →

classification cs.LG
keywords multimodal foundation modelearth system modelingocean atmosphere interactionsclimate indicessubseasonal predictionclimate variabilitycoupled dynamics
0
0 comments X

The pith

NIVA shows a multimodal model can learn coupled ocean-atmosphere dynamics from simulations to predict major climate indices.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents NIVA as a multimodal foundation model that learns unified representations across Earth system components. It focuses on a controlled ocean-atmosphere setting trained on large-scale simulations as proof that such models can capture physically meaningful cross-modal structure. This structure is intended to support prediction of climate variability beyond the current two-week limit. A sympathetic reader would see value in extending data-driven forecasts to subseasonal-to-seasonal scales if the cross-modal learning holds. The work positions the two-modality case as an initial step toward full Earth system modeling.

Core claim

Trained on Earth system simulations, NIVA learns physically meaningful cross-modal structure in a two-modality ocean-atmosphere setting and captures key modes of climate variability through accurate prediction of major climate indices.

What carries the argument

NIVA, the multimodal foundation model that produces unified representations across ocean and atmosphere data to support coupled dynamics learning.

If this is right

  • Supports development of subseasonal-to-seasonal forecasts that account for ocean-atmosphere interactions.
  • Opens a path to include ice and land modalities in the same unified representation framework.
  • Offers a route to lower computational cost for coupled Earth system predictions compared with traditional models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the two-modality results generalize, adding land and ice data could produce a single model usable for broader Earth system forecasts.
  • Testing the same architecture on observational records instead of simulations would check whether the learned structure transfers outside controlled data.
  • Similar cross-modal training might apply to other coupled scientific domains where multiple data streams describe one physical system.

Load-bearing premise

That training on simulations in only the ocean-atmosphere pair is enough to establish that foundation models can learn the full set of Earth system couplings.

What would settle it

The model produces large errors when predicting held-out major climate indices from the simulation data.

Figures

Figures reproduced from arXiv: 2606.28546 by Anisha Pal, Aodhan Sweeney, Kalai Ramea, Kyle Heyblom.

Figure 1
Figure 1. Figure 1: Schematic of the NIVA end-to-end pipeline. The framework uses ESM output to pretrain the foundation model that learns coupled Earth system representations, which can be transferred to a range of downstream tasks. a multimodal foundation model designed to learn coupled Earth system dynamics. Drawing on methodologies from vision–language architectures (Cheng et al., 2024; He et al., 2021; Singh et al., 2022;… view at source ↗
Figure 2
Figure 2. Figure 2: Pretraining approach. Aggregated oceanic and atmospheric states are processed by separate encoders and jointly optimized with a contrastive objective to learn a shared latent representation of coupled dynamics. as a foundation for future extensions to multi-resolution and one-to-many training regimes(see Sec. B.1). 3.2.2. OCEAN & ATMOSPHERIC ENCODER We evaluated several encoder architectures for both modal… view at source ↗
Figure 3
Figure 3. Figure 3: Pretraining metrics for NIVA on training and validation data (a) Cosine similarity matrix, where blue indicates high similarity and red indicates low similarity. The strong diagonal structure reflects correct alignment between paired ocean and atmospheric states in the latent space. (b) Reciprocal Rank (RR) distribution over 2000 samples. The concentration of samples at RR = 1 indicates that the model reli… view at source ↗
Figure 4
Figure 4. Figure 4: Post-training results for the RONI and IOD climate indices. Line plots compare predicted values (orange) with ground-truth values (blue), while scatter plots show their correlations. RONI exhibits strong performance (R2 = 0.969), and IOD achieves moderate performance (R2 = 0.448). the diagonal confirms correct pairwise alignment, while the presence of coherent off-diagonal patterns indicates that the model… view at source ↗
Figure 5
Figure 5. Figure 5: Post-training results for the Real-time multivariate MJO components 1 and 2 (RMM1 and RMM2). Line plots compare predicted values (orange) with ground-truth values (blue), while scatter plots show their correlations. variability, typically operating on 30–60 day timescales. Second, MJO dynamics are not governed solely by oceanic initial conditions; they depend on a broader set of processes, including atmosp… view at source ↗
Figure 6
Figure 6. Figure 6: Post-training results for all the climate indices. Line plots compare predicted values (orange) with ground-truth values (blue). where cos(·, ·) denotes cosine similarity and τ is a tempera￾ture parameter. The prediction distribution is obtained via row-wise soft￾max: Pij = exp(Zij ) PB k=1 exp(Zik) . The target matrix Y ∈ R B×B is defined as a one-hot iden￾tity matrix: Yij = ( 1, if i = j, 0, otherwise. W… view at source ↗
Figure 7
Figure 7. Figure 7: Post-training results for all the climate indices. Scatter plots show the correlations between ground truth and predicted values. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
read the original abstract

Recent advances in AI-driven weather and climate modeling have improved forecast skill while reducing computational cost. However, existing data-driven approaches are limited in their ability to model coupled Earth system dynamics, which is required for extending predictability beyond the ~2-week horizon. To address this, we introduce NIVA, a multimodal foundation model designed to learn unified representations across Earth system components. While the full framework targets atmosphere, ocean, ice, and land interactions, we focus here on a two-modality setting (ocean and atmosphere) as a controlled proof of concept to evaluate whether foundation models can learn coupled dynamics. Trained on large-scale Earth system simulations, NIVA learns physically meaningful cross-modal structure, providing a foundation for subseasonal-to-seasonal prediction. As initial validation, we show that NIVA captures key modes of climate variability through accurate prediction of major climate indices.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript introduces NIVA, a multimodal foundation model designed to learn unified representations across Earth system components (atmosphere, ocean, ice, land), with a two-modality (ocean-atmosphere) proof-of-concept trained on large-scale simulations. It claims that NIVA learns physically meaningful cross-modal structure and, as initial validation, captures key modes of climate variability through accurate prediction of major climate indices, laying groundwork for subseasonal-to-seasonal prediction.

Significance. If the central claims were supported by quantitative evidence, the work could represent a step toward foundation models capable of learning coupled Earth-system dynamics beyond existing single-modality weather models. However, the complete absence of any results, metrics, or methods in the manuscript precludes any assessment of significance, novelty, or whether cross-modal learning actually occurs.

major comments (1)
  1. [Abstract] Abstract: The assertion that NIVA 'captures key modes of climate variability through accurate prediction of major climate indices' is made with no accompanying quantitative results, error metrics, baselines, prediction setup, evaluation protocol, or figures. Without these elements it is impossible to determine whether any accuracy exists, whether it exceeds single-modality baselines, or whether it arises from cross-modal structure rather than other factors.

Simulated Author's Rebuttal

1 responses · 1 unresolved

We thank the referee for their review. We agree that the current manuscript version does not include quantitative results, metrics, baselines, or methods to support the claims made in the abstract regarding prediction of climate indices and cross-modal learning. This prevents evaluation of the work's validity.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The assertion that NIVA 'captures key modes of climate variability through accurate prediction of major climate indices' is made with no accompanying quantitative results, error metrics, baselines, prediction setup, evaluation protocol, or figures. Without these elements it is impossible to determine whether any accuracy exists, whether it exceeds single-modality baselines, or whether it arises from cross-modal structure rather than other factors.

    Authors: We agree with this assessment. The abstract makes claims about accurate prediction and physically meaningful cross-modal structure that are not supported by any results, metrics, or methodological details in the manuscript. The provided text consists only of the abstract and a high-level description without experiments, data, or evaluation protocols. We will revise the manuscript to either include the required quantitative evidence (if available from the underlying work) or substantially tone down the abstract and claims to match what is actually demonstrated. revision: yes

standing simulated objections not resolved
  • The current manuscript contains no results, metrics, methods, or figures, so no defense or revision can be offered on the scientific validity of the cross-modal learning or prediction claims until such content is added.

Circularity Check

0 steps flagged

No derivation chain or equations presented; claim is an unsupported assertion

full rationale

The manuscript text provides only an abstract asserting that 'NIVA captures key modes of climate variability through accurate prediction of major climate indices' as validation, with no equations, methods, data splits, baselines, or evaluation details supplied. No load-bearing steps exist to inspect for self-definition, fitted-input-as-prediction, or self-citation reduction. The central claim therefore cannot be shown to reduce to its inputs by construction; the issue is an absence of evidence rather than circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Only the abstract is available; the central claim rests on the unstated premise that simulation-trained multimodal representations will generalize to real coupled Earth system dynamics and that 'physically meaningful' can be assessed without explicit physical loss terms or verification.

axioms (1)
  • domain assumption Large-scale Earth system simulations contain sufficient information to learn coupled cross-modal dynamics via standard neural network training.
    The model is trained on these simulations to produce the claimed representations and predictions.
invented entities (1)
  • NIVA multimodal foundation model no independent evidence
    purpose: To learn unified representations across Earth system components for subseasonal-to-seasonal prediction.
    The model is newly introduced in the paper as the core contribution.

pith-pipeline@v0.9.1-grok · 5681 in / 1366 out tokens · 48620 ms · 2026-06-30T00:50:10.107700+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 4 canonical work pages · 2 internal anchors

  1. [1]

    2023 , eprint=

    Segment Anything , author=. 2023 , eprint=

    2023

  2. [2]

    Fasullo, J. T. and Lamarque, Jean-Francois and Hannay, Cecile and Rosenbloom, Nan and Tilmes, Simone and DeRepentigny, Patricia and Jahn, Alexandra and Deser, Clara , title =. Geophysical Research Letters , volume =. doi:https://doi.org/10.1029/2021GL097420 , url =. https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2021GL097420 , note =

    work page doi:10.1029/2021gl097420

  3. [3]

    2024 , eprint=

    DINOv2: Learning Robust Visual Features without Supervision , author=. 2024 , eprint=

    2024

  4. [4]

    2021 , eprint=

    Learning Transferable Visual Models From Natural Language Supervision , author=. 2021 , eprint=

    2021

  5. [5]

    2024 , eprint=

    PaliGemma: A versatile 3B VLM for transfer , author=. 2024 , eprint=

    2024

  6. [6]

    2024 , eprint=

    YOLO-World: Real-Time Open-Vocabulary Object Detection , author=. 2024 , eprint=

    2024

  7. [7]

    2019 , eprint=

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , author=. 2019 , eprint=

    2019

  8. [8]

    2023 , eprint=

    LLaMA: Open and Efficient Foundation Language Models , author=. 2023 , eprint=

    2023

  9. [9]

    OpenAI blog, 2018 , author=

    Improving language understanding by generative pre-training. OpenAI blog, 2018 , author=. URL: https://cdn. openai. com/research-covers/language-unsupervised/language\_understanding\_paper. pdf , year=

    2018

  10. [10]

    2023 , eprint=

    ClimaX: A foundation model for weather and climate , author=. 2023 , eprint=

    2023

  11. [11]

    2024 , eprint=

    Prithvi WxC: Foundation Model for Weather and Climate , author=. 2024 , eprint=

    2024

  12. [12]

    2024 , eprint=

    A Foundation Model for the Earth System , author=. 2024 , eprint=

    2024

  13. [13]

    2023 , eprint=

    AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning , author=. 2023 , eprint=

    2023

  14. [14]

    Oort, Abraham H and Peixoto, JP , year=

  15. [15]

    2022 , eprint=

    Swin Transformer V2: Scaling Up Capacity and Resolution , author=. 2022 , eprint=

    2022

  16. [16]

    2021 , eprint=

    Masked Autoencoders Are Scalable Vision Learners , author=. 2021 , eprint=

    2021

  17. [17]

    2021 , eprint=

    Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision , author=. 2021 , eprint=

    2021

  18. [18]

    2022 , eprint=

    FLAVA: A Foundational Language And Vision Alignment Model , author=. 2022 , eprint=

    2022

  19. [19]

    2017 , eprint=

    Multimodal Machine Learning: A Survey and Taxonomy , author=. 2017 , eprint=

    2017

  20. [20]

    2019 , eprint=

    ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks , author=. 2019 , eprint=

    2019

  21. [21]

    , title =

    Xu, Ran and Xiong, Caiming and Chen, Wei and Corso, Jason J. , title =. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence , pages =. 2015 , isbn =

    2015

  22. [22]

    2022 , eprint=

    CLAP: Learning Audio Concepts From Natural Language Supervision , author=. 2022 , eprint=

    2022

  23. [23]

    2019 , eprint=

    Representation Learning with Contrastive Predictive Coding , author=. 2019 , eprint=

    2019

  24. [24]

    2022 , eprint=

    TS2Vec: Towards Universal Representation of Time Series , author=. 2022 , eprint=

    2022

  25. [25]

    2026 , eprint=

    Auto-Augmentation Contrastive Learning for Wearable-based Human Activity Recognition , author=. 2026 , eprint=

    2026

  26. [26]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , author=. arXiv preprint arXiv:2010.11929 , year=

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  27. [27]

    Masked Autoencoders Are Scalable Vision Learners

    Masked Autoencoders Are Scalable Vision Learners , author=. arXiv preprint arXiv:2111.06377 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  28. [28]

    Proceedings of the 40th International Conference on Machine Learning (ICML) , year=

    Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere , author=. Proceedings of the 40th International Conference on Machine Learning (ICML) , year=

  29. [29]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

    Swin Transformer V2: Scaling Up Capacity and Resolution , author=. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=. doi:10.1109/CVPR56306.2022.01177 , url=

    work page doi:10.1109/cvpr56306.2022.01177 2022

  30. [30]

    2018 , eprint=

    Focal Loss for Dense Object Detection , author=. 2018 , eprint=

    2018

  31. [31]

    2001 , publisher=

    The Elements of Statistical Learning: Data Mining, Inference, and Prediction , author=. 2001 , publisher=

    2001

  32. [32]

    2017 , eprint=

    Adam: A Method for Stochastic Optimization , author=. 2017 , eprint=

    2017

  33. [33]

    Reviews of Geophysics , volume =

    Zhang, Chidong , title =. Reviews of Geophysics , volume =. 2005 , doi =

    2005

  34. [34]

    and Holton, James R

    Andrews, David G. and Holton, James R. and Leovy, Conway B. , title =. 1987 , pages =

    1987

  35. [35]

    and Nash, Eric R

    Newman, Paul A. and Nash, Eric R. and Rosenfield, Joan E. , title =. Journal of Geophysical Research: Atmospheres , volume =. 2001 , doi =

    2001

  36. [36]

    and Hakim, Gregory J

    Holton, James R. and Hakim, Gregory J. , title =. 2013 , pages =

    2013

  37. [37]

    and Hobbs, Peter V

    Wallace, John M. and Hobbs, Peter V. , title =. 2006 , pages =

    2006

  38. [38]

    Saji, N. H. and Goswami, B. N. and Vinayachandran, P. N. and Yamagata, T. , title =. Nature , volume =. 1999 , doi =

    1999

  39. [39]

    Thompson, David W. J. and Wallace, John M. , title =. Geophysical Research Letters , volume =. 1998 , doi =

    1998

  40. [40]

    , title =

    Hurrell, James W. , title =. Science , volume =. 1995 , doi =

    1995

  41. [41]

    and Hendon, Harry H

    Wheeler, Matthew C. and Hendon, Harry H. , title =. Monthly Weather Review , volume =. 2004 , doi =

    2004

  42. [42]

    Environmental Research Letters , volume =

    van Oldenborgh, Geert Jan and Hendon, Harry and Stockdale, Timothy and L'Heureux, Michelle and Coughlan de Perez, Erin and Singh, Roop and van Aalst, Maarten , title =. Environmental Research Letters , volume =. 2021 , doi =

    2021

  43. [43]

    and Tippett, Michael K

    L'Heureux, Michelle L. and Tippett, Michael K. and Wang, Wanqiu , title =. Journal of Climate , volume =. 2024 , doi =

    2024

  44. [44]

    , title =

    Flato, Gregory M. , title =. WIREs Climate Change , volume =. 2011 , doi =

    2011

  45. [45]

    Nature , volume =

    Bauer, Peter and Thorpe, Alan and Brunet, Gilbert , title =. Nature , volume =. 2015 , doi =

    2015

  46. [46]

    and Tebaldi, Claudia and van Vuuren, Detlef P

    O'Neill, Brian C. and Tebaldi, Claudia and van Vuuren, Detlef P. and Eyring, Veronika and Friedlingstein, Pierre and Hurtt, George and Knutti, Reto and Kriegler, Elmar and Lamarque, Jean-Francois and Lowe, Jason and Meehl, Gerald A. and Moss, Richard and Riahi, Keywan and Sanderson, Benjamin M. , title =. Geoscientific Model Development , volume =. 2016 , doi =

    2016

  47. [47]

    and Gillett, Nathan P

    Stott, Peter A. and Gillett, Nathan P. and Hegerl, Gabriele C. and Karoly, David J. and Stone, D. Detection and attribution of climate change: a regional perspective , journal =. 2010 , doi =

    2010

  48. [48]

    and Senior, Catherine A

    Eyring, Veronika and Bony, Sandrine and Meehl, Gerald A. and Senior, Catherine A. and Stevens, Bjorn and Stouffer, Ronald J. and Taylor, Karl E. , title =. Geoscientific Model Development , volume =. 2016 , doi =

    2016

  49. [49]

    and Maisonnave, E

    Balaji, V. and Maisonnave, E. and Zadeh, N. and Lawrence, B. N. and Biercamp, J. and Fladrich, U. and Aloisio, G. and Benson, R. and Caubel, A. and Durachta, J. and Foujols, M.-A. and Lister, G. and Mocavero, S. and Underwood, S. and Wright, G. , title =. Geoscientific Model Development , volume =. 2017 , doi =

    2017

  50. [50]

    and Hoefler, Torsten and Quintino, Tiago and Schulthess, Thomas C

    Bauer, Peter and Dueben, Peter D. and Hoefler, Torsten and Quintino, Tiago and Schulthess, Thomas C. and Wedi, Nils P. , title =. Nature Computational Science , volume =. 2021 , doi =

    2021

  51. [51]

    Climate goals and computing the future of clouds , journal =

    Schneider, Tapio and Teixeira, Jo. Climate goals and computing the future of clouds , journal =. 2017 , doi =

    2017

  52. [52]

    and Couvreux, Fleur and Deshayes, Julie and Gautrais, Jacques and Hourdin, Fr

    Balaji, V. and Couvreux, Fleur and Deshayes, Julie and Gautrais, Jacques and Hourdin, Fr. Are general circulation models obsolete? , journal =. 2022 , doi =

    2022

  53. [53]

    and Palomas, Sergi and Paronuzzi Ticco, Stella V

    Acosta, Mario C. and Palomas, Sergi and Paronuzzi Ticco, Stella V. and Utrera, Gladys and Biercamp, Joachim and Bretonni. The computational and energy cost of simulation and storage for climate science: lessons from. Geoscientific Model Development , volume =. 2024 , doi =

    2024

  54. [54]

    and Lee, Sun-Seon and Rosenbloom, Nan and Timmermann, Axel and Danabasoglu, Gokhan and Deser, Clara and Edwards, Jim and Kim, Ji-Eun and Simpson, Isla R

    Rodgers, Keith B. and Lee, Sun-Seon and Rosenbloom, Nan and Timmermann, Axel and Danabasoglu, Gokhan and Deser, Clara and Edwards, Jim and Kim, Ji-Eun and Simpson, Isla R. and Stein, Karl and Stuecker, Malte F. and Yamaguchi, Ryohei and Bódai, Tamás and Chung, Eui-Seok and Huang, Lei and Kim, Who M. and Lamarque, Jean-Fran. Ubiquity of human-induced chang...

    2021

  55. [55]

    Hersbach, Hans and Bell, Bill and Berrisford, Paul and Hirahara, Shoji and Horányi, András and Muñoz-Sabater, Joaquín and Nicolas, Julien and Peubey, Carole and Radu, Raluca and Schepers, Dinand and Simmons, Adrian and Soci, Cornel and Abdalla, Saleh and Abellan, Xavier and Balsamo, Gianpaolo and Bechtold, Peter and Biavati, Gionata and Bidlot, Jean and B...

    2020

  56. [56]

    and Lamarque, J.-F

    Danabasoglu, G. and Lamarque, J.-F. and Bacmeister, J. and Bailey, D. A. and DuVivier, A. K. and Edwards, J. and Emmons, L. K. and Fasullo, J. and Garcia, R. and Gettelman, A. and Hannay, C. and Holland, M. M. and Large, W. G. and Lauritzen, P. H. and Lawrence, D. M. and Lenaerts, J. T. M. and Lindsay, K. and Lipscomb, W. H. and Mills, M. J. and Neale, R....

    2020