Pith. sign in

REVIEW 4 major objections 4 minor 98 references

Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Trimodal contrastive learning depends on encoder geometry, not just loss design.

desk verdict A well-run empirical study with a practical fix for multimodal contrastive learning; the causal story about Jacobian conditioning is plausible but not proven. read the letter →

arxiv 2607.17673 v1 pith:2FOIZVO2 submitted 2026-07-20 cs.LG cs.AI

Tillmann Rheude , Roland Eils , Benjamin Wild This is my paper
classification cs.LGcs.AI
keywords multimodalcontrastivelearningJacobianconditioninggeometry-preservingencodersresidualconnectionsLeakyReLUtrimodalalignmentlinearprobingretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that successful trimodal contrastive learning hinges on the conditioning of each modality encoder's Jacobian—the local sensitivity of an encoder's output to its input. The authors show that standard MLP encoders develop exploding condition numbers and collapsing minimum singular values during training, and this degeneration tracks with poor retrieval and downstream linear-probe performance. They propose geometry-preserving encoders made of residual transport paths and LeakyReLU activations, which keep Jacobians well-conditioned and improve results across several contrastive objectives and datasets, including settings with missing modalities. The paper's central message is that objective expressivity alone is insufficient; the geometric and optimization properties of the encoders themselves are a first-order bottleneck.

What carries the argument

The central object is the Jacobian condition number κ(J)=σmax(J)/σmin(J) of each modality encoder. A small σmin indicates that input directions are suppressed; a large σmax indicates excessive amplification, and both impair gradient transport and alignment. Two mechanisms preserve it: residual transport, written Ei(x)=Pi x + h_i(x), whose Jacobian Pi + Jh_i satisfies σmin(J) ≥ σmin(Pi) − ||Jh_i||₂, and LeakyReLU activations, which keep activation derivatives in {α, 1} instead of allowing exact zero gradients. Together these provide an additive identity path and prevent directional collapse.

What would settle it

Train a standard MLP encoder on a trimodal objective while explicitly constraining its true Jacobian singular values to remain near 1, and observe no retrieval or probe improvement; or find a real dataset where residual+LeakyReLU encoders fail to improve performance despite measurably better κ(J). A negative result of either kind would break the claimed causal chain.

Watch

Extended reading notes

Core claim

The paper's central claim is that encoder Jacobian conditioning is a key factor in multimodal contrastive learning: poorly conditioned encoders exhibit collapsing or amplified singular-value spectra, leading to exploding condition numbers and degraded multimodal alignment. It supports this claim with two complementary interventions—a direct Jacobian regularizer that stabilizes directional gains, and lightweight architectural modifications (residual paths plus LeakyReLU) that keep Jacobian spectra well-conditioned throughout training. Across a synthetic benchmark, UK Biobank cohorts, and MIMIC-IV, these geometry-preserving encoders improve both retrieval and linear-probe performance for CLIP,

Load-bearing premise

The paper assumes that the observed gains are caused by improved encoder Jacobian conditioning, rather than by generic side effects of residual connections and LeakyReLU on learning dynamics, since the direct Jacobian intervention uses a directional-gain proxy that is not the true condition number and is demonstrated mainly on one synthetic benchmark.

Editorial extensions

If this is right

  • Practitioners can improve trimodal retrieval and linear-probe performance by switching MLP modality encoders to residual + LeakyReLU variants, without changing the contrastive objective.
  • Objective expressivity is not the only lever: even expressive objectives underperform when encoder Jacobians degenerate.
  • GPEs reduce the dependence of downstream representation quality on objective choice, making linear-probe results more consistent across objectives.
  • Geometry-preserving mechanisms may explain why transformer encoders train more stably than plain MLPs: dense residual transport, not attention expressivity, is the main stabilizer.
  • The benefits extend to missing-modality settings, where masked or absent inputs still benefit from well-conditioned encoders.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is right, the same geometry-preserving design should transfer to contrastive learning with four or more modalities, where conditioning problems are likely to be more severe.
  • The paper's linear-encoder ablation suggests that some multimodal retrieval signal is accessible through stable low-complexity projections; a natural extension is to test whether much of the GPE benefit comes from improved optimization rather than from nonlinearity.
  • Directly regularizing the true extremal singular values, rather than the directional-gain proxy, could yield even larger gains and would more definitively isolate the causal role of conditioning.
  • One testable extension: vary the LeakyReLU slope and measure both κ(J) and retrieval on real datasets beyond Synthetic-XNOR; the paper shows this correlation on synthetic data, so a real-world confirmation would strengthen the mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies trimodal contrastive learning and argues that encoder Jacobian conditioning is a key factor alongside objective expressivity. It introduces geometry-preserving encoders (GPEs) built from residual transport paths and LeakyReLU activations, and evaluates them on a synthetic benchmark (Synthetic-XNOR), MIMIC-IV, MIMIC-Symile, and two UKB cohorts, across four contrastive objectives (CLIP, Triangle, Gram, Symile). The authors report retrieval and linear-probe results with 5-fold cross-validation and 3 seeds, and they present a Jacobian-regularization intervention based on directional gains. The central claim is that ill-conditioned encoder Jacobians degrade multimodal alignment and that GPEs improve optimization stability, retrieval, and downstream probes.

Significance. If the causal mechanism were established, the paper would make an important contribution by shifting attention from objective design to encoder geometry in multimodal contrastive learning. The empirical protocol is rigorous: multiple objectives, several real-world datasets, missing-modality settings, 5-fold CV, 3 seeds, and extensive hyperparameter tuning. The practical recipe—residual paths plus LeakyReLU—is simple and actionable, and the code is released. However, the mechanistic evidence is currently incomplete: the direct intervention uses a proxy that the authors explicitly state is not the condition number, and the GPE gains are not separated from generic optimization effects of residual connections and LeakyReLU. Moreover, the linear-encoder ablation shows that for Triangle, a linear encoder matches the GPE result, undercutting the necessity of the geometric explanation. The empirical contribution is credible and useful, but the causal interpretation needs substantial additional support.

major comments (4)
  1. [§3.3, Eq. (9)–(10), Fig. 4] The direct intervention does not actually penalize the Jacobian condition number κ = σmax/σmin. The regularizer uses r_min and r_max over K=32 random directions (Appendix G). In embedding dimensions of 256–6144, random directions concentrate near typical singular vectors, so r_min/r_max cannot track the extremal singular values. The authors acknowledge this in the text. Consequently, the Fig. 4 improvement could result from a generic smoothness or anti-collapse effect rather than from improved conditioning specifically. To support the causal claim, the intervention should be shown to reduce an actual estimate of κ (e.g., via power iteration or Lanczos on small synthetic encoders) and the performance gain should be shown to be mediated by that reduction.
  2. [§3.4 and §4.2, GPE definition] GPEs combine two modifications—residual paths and LeakyReLU—both of which are known to improve gradient flow and prevent dead ReLU units. The paper does not ablate these components separately, nor does it control for generic optimization effects such as smoother loss landscapes or improved effective learning rates. The Fig. 3 association between maintained κ and higher accuracy is correlational. Without a control (e.g., residual-only, LeakyReLU-only, or a non-geometric modification that also stabilizes optimization without improving κ), the observed gains do not uniquely implicate Jacobian conditioning as the mediator.
  3. [Table 5, Linear encoder ablation] Triangle with a linear encoder matches Triangle with GPE on UKB retrieval (0.7438 vs 0.7434), while CLIP, Gram, and Symile degrade. A linear encoder has a constant Jacobian and no conditioning pathology, yet it fully reproduces the GPE gain for Triangle. This suggests that the GPE benefit may stem from avoiding optimization problems of deep nonlinear MLPs rather than from preserving geometric information in a way that linear projections cannot. The authors' interpretation—that GPEs retain stability while preserving nonlinear flexibility—is plausible, but the Triangle result directly weakens the claim that encoder Jacobian conditioning is the operative mechanism. The paper should either reconcile this with the central claim or soften the mechanistic conclusion.
  4. [Section 5, Limitations] The paper's own scope note limits the analysis to MLP-based projections and healthcare datasets, yet the abstract and conclusion state that 'multimodal contrastive learning depends not only on objective expressivity, but also on the geometric and optimization properties of the underlying encoders' as a general finding. The Transformer experiments in Section E are only preliminary and do not quantitatively measure Jacobian conditioning. The general claim should be tempered to the MLP-encoder setting, or additional evidence is needed that the mechanism transfers beyond MLPs. This is not a fatal flaw, but it affects the paper's headline conclusion.
minor comments (4)
  1. [Figure 9 caption] Typo: 'residaul MLPs' should be 'residual MLPs'.
  2. [Section 2, Objectives] The objectives CLIP, Triangle, Gram, and Symile are cited but not briefly defined. A one-sentence description of each scoring function would improve self-containedness, especially for readers outside the multimodal contrastive subfield.
  3. [Figure 6 caption] The caption 'Baseline +GPE Best baseline ± SE' is ambiguous. Clarify what the shaded band and the markers represent.
  4. [Equation (7)] The Jacobian expression for the ReLU MLP omits the possibility of bias terms. While biases do not affect the derivative, noting this would avoid confusion.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: the central empirical claim rests on external datasets/objectives and explicit interventions; self-citations supply benchmarks and training conventions but are not load-bearing.

full rationale

The paper's derivation chain is not circular. The theoretical pieces are self-contained: the residual-transport bounds (Eqs. 12-13) are standard singular-value perturbation inequalities, and the LeakyReLU bounds (Eqs. 15-16) follow directly from the activation derivative being in {α,1}. The direct Jacobian intervention (Eqs. 8-10) is an explicit empirical test, and the authors themselves disclose that rmin and rmax are 'not estimates of the extremal singular values themselves,' so the intervention is a proxy rather than a definitional identity with the outcome. This weakens the mechanistic inference but is an evidentiary limitation, not circularity. The main results compare well-tuned standard encoders against GPEs on external datasets (UKB, MIMIC-IV, MIMIC-Symile) and across external objectives (CLIP, Triangle, Gram, Symile), so the empirical claims do not reduce to fitted values from the authors' prior work. Self-citations (Rheude et al. 2026b for Synthetic-XNOR and the pair-sampling strategy; Hegselmann et al. 2026 for EHR embeddings; Rheude et al. 2026a for tuning practice) provide benchmarks, tools, and training conventions but are not used to derive the core result, and no uniqueness theorem or ansatz is imported from those papers. The paper's own limitation statements—MLP-only scope, the linear-encoder ablation matching Triangle+GPE on UKB, and the preliminary Transformer analysis—further narrow the claim without making it circular. Overall, the central claim has independent empirical content and is not forced by self-citation or by construction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; GPEs are an architectural modification of existing encoders. The load-bearing assumptions are the proxy status of directional-gain statistics for true Jacobian conditioning, and the domain assumption that preserving modality-specific identity structure is desirable for multimodal contrastive learning.

free parameters (2)
  • LeakyReLU negative slope α = tuned from {0.0, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8} per method/dataset via Bayesian optimization
    α is a searchable GPE hyperparameter (Listing 2); Figure 5 shows retrieval accuracy and κ(J) depend strongly on α, so part of the claimed improvement is bought by tuning this parameter.
  • Jacobian regularizer band and weights = m=0.5, M=5.0, w_min=10.0, w_max=1.0, λ=1.0, K=32
    Hand-set in Section G with no reported sensitivity analysis; the direct-intervention evidence relies on these choices.
assumptions (5)
  • standard math Weyl's singular-value perturbation bounds
    Used in Section C/Eq. (18)-(20) to justify residual transport.
  • standard math LeakyReLU activation Jacobian singular-value bounds
    Eq. (16): diagonal activation Jacobian entries in {α,1} imply σ_min(D) ≥ α and σ_max(D) ≤ 1.
  • ad hoc to paper Directional gains r_min/r_max are usable proxies for Jacobian conditioning
    Section 3.3: r_min/r_max over K random directions are treated as conditioning proxies; authors state they are not estimates of extremal singular values. The central intervention evidence depends on this heuristic.
  • domain assumption Multimodal contrastive learning requires encoders to preserve modality-specific information rather than transform it
    Section 3.2: motivates near-identity residual transport; asserted without proof and may not hold for all modalities.
  • domain assumption The nonlinear branch h_i has small enough Jacobian norm to satisfy ‖J_h‖ < σ_min(P)
    The residual bound σ_min(J_E) > 0 requires this inequality; it is not verified explicitly for the trained GPEs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learning." pith.science (2026). https://pith.science/paper/2FOIZVO2

@misc{pith2026260717673,
  author       = {Pith},
  title        = {Pith review of: Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2FOIZVO2}},
  note         = {Machine review of arXiv:2607.17673}
}
read the original abstract

Contrastive learning is increasingly moving toward settings with three or more modalities instead of image-text pairs. Yet, extending models from pairwise to higher-order multimodal alignment can introduce optimization and representation challenges. We identify encoder Jacobian conditioning as a key factor in trimodal contrastive learning: poorly conditioned encoders exhibit collapsing or amplified singular-value spectra, leading to exploding Jacobian condition numbers and degraded multimodal alignment. We introduce geometry-preserving encoders (GPEs) by directly conditioning the Jacobian through regularization and demonstrating that simple modifications like LeakyReLU activations and residual paths recover these geometric benefits. Across a synthetic benchmark and four real-world datasets including missing modalities, improving Jacobian conditioning boosts retrieval and linear probe performance across multiple contrastive objectives, whereas expressive objectives yield little benefit in linear probes. More broadly, our results show that multimodal contrastive learning depends not only on objective expressivity, but also on the geometric and optimization properties of the underlying encoders.

Figures

Figures reproduced from arXiv: 2607.17673 by the authors.

Figure 1
Figure 1. Well-tuned standard encoders vs. GPEs across multiple multimodal contrastive objectives on the UKB. GPEs improve retrieval performance for all evaluated contrastive objectives. Multimodal contrastive learning has emerged as a powerful method for aligning representations across modalities. While early work focused on bimodal vision-language models (VLMs) (Rad￾ford et al., 2021), many real-world applications involve m… view at source ↗
Figure 2
Figure 2. Multimodal contrastive learning with GPEs exemplified for MIMIC-IV (Johnson et al., [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Training dynamics of well-tuned standard encoders and GPEs. The upper row focuses on [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Well-tuned training dynamics with and with [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Well-tuned analysis of increasing LeakyReLU’s negative slope α on the Synthetic￾XNOR dataset. Higher α improves accuracy by preserving non-exploding condition numbers κ(J). LeakyReLU Analysis To better under￾stand Jacobian conditioning in case of the LeakyReLU GPE, we …
Figure 6
Figure 6. Figure 6: GPEs match or exceed non-GPE linear probe base [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Training dynamics of standard encoders and GPEs with different scorers on the Synthetic [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Training dynamics of standard encoders and GPEs with different scorers on the MIMIC-IV [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 10
Figure 10. Figure 10: Validation loss compari￾son on the MIMIC-Symile dataset for the originally published en￾coders, non-GPE, and GPEs. GPEs demonstrate faster convergence and lower validation loss values. Modality Setup in MIMIC-IV For our MIMIC-IV trimodal contrastive learning setup, we…
Figure 9
Figure 9. Figure 9: Training dynamics of standard encoders and GPEs with different scorers on the MIMIC [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

98 extracted references · 6 canonical work pages

  1. [1]

    SineProject: Machine Unlearning for Stable Vision-Language Alignment , booktitle=

    Garg, Arpit and Saratchandran, Hemanth and Lucey, Simon , year=. SineProject: Machine Unlearning for Stable Vision-Language Alignment , booktitle=

  2. [2]

    The Invisible Hand of Physics: When Video Diffusion Models Know More Than They Show

    Esmati, Parsa and Nath, Somjit and Hofmann, Katja and Nowrouzezahrai, Derek and Kahou, Samira Ebrahimi and Mirmehdi, Majid , year=. The Invisible Hand of Physics: When Video Diffusion Models Know More Than They Show , url=. doi:10.48550/arXiv.2606.05328 , abstractNote=

    work page Pith review arXiv doi:10.48550/arxiv.2606.05328

  3. [3]

    doi:10.5281/zenodo.3828935 , author=

    PyTorch Lightning , url=. doi:10.5281/zenodo.3828935 , author=

    doi:10.5281/zenodo.3828935

  4. [4]

    , editor=

    Bachlechner, Thomas and Majumder, Bodhisattwa Prasad and Mao, Huanru Henry and Cottrell, Gary and McAuley, Julian J. , editor=. ReZero is all you need: fast convergence at large depth , url=. Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, UAI 2021, Virtual Event, 27-30 July 2021 , publisher=. 2021 , pages=

    2021

  5. [5]

    Rosenblatt, F. , year=. The Perceptron, a Perceiving and Recognizing Automaton: (Project Para) , url=

  6. [7]

    Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , url=

    Rifai, Salah and Vincent, Pascal and Muller, Xavier and Glorot, Xavier and Bengio, Yoshua , editor=. Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , url=. Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011 , publisher=. 2011 , pages=

    2011

  7. [8]

    and Paiva, Ana , year=

    Poklukar, Petra and Vasco, Miguel and Yin, Hang and Melo, Francisco S. and Paiva, Ana , year=. Geometric Multimodal Contrastive Representation Learning , volume=. Proceedings of the 39th International Conference on Machine Learning , publisher=

  8. [10]

    Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere , volume=

    Wang, Tongzhou and Isola, Phillip , editor=. Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere , volume=. Proceedings of the 37th International Conference on Machine Learning , publisher=. 2020 , month=jul, pages=

    2020

Show all 98 references
  1. [11]

    Advances in Neural Information Processing Systems , author=

    Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning , url=. Advances in Neural Information Processing Systems , author=

  2. [12]

    Decipher the Modality Gap in Multimodal Contrastive Learning: From Convergent Representations to Pairwise Alignment , url=

    Yi, Lingjie and Douady, Raphael and Chen, Chao , year=. Decipher the Modality Gap in Multimodal Contrastive Learning: From Convergent Representations to Pairwise Alignment , url=

  3. [13]

    The More, the Merrier: Contrastive Fusion for Higher-Order Multimodal Alignment , booktitle=

    Koutoupis, Stefanos and Zervou, Michaela Areti and Kontras, Konstantinos and De Vos, Maarten and Tsakalides, Panagiotis and Tsagkatakis, Grigorios , year=. The More, the Merrier: Contrastive Fusion for Higher-Order Multimodal Alignment , booktitle=

  4. [14]

    Dahl and Justin Gilmer and Christopher J

    Varun Godbole and George E. Dahl and Justin Gilmer and Christopher J. Shallue and Zachary Nado , title =. 2023 , note =

    2023

  5. [15]

    Rectifier nonlinearities improve neural network acoustic models , volume=

    Maas, Andrew L and Hannun, Awni Y and Ng, Andrew Y , year=. Rectifier nonlinearities improve neural network acoustic models , volume=. Proceedings of the 30th International Conference on Machine Learning , publisher=

  6. [16]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , url=

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , editor=. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , url=. 2019 , pages=. doi:10.18653/V1/N19-1423 , booktitle=

    2019 doi

  7. [18]

    The Road Less Scheduled , url=. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 , author=

    2024

  8. [20]

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models , url=

    Zhang, Yanzhao and Li, Mingxin and Long, Dingkun and Zhang, Xin and Lin, Huan and Yang, Baosong and Xie, Pengjun and Yang, An and Liu, Dayiheng and Lin, Junyang and Huang, Fei and Zhou, Jingren , year=. Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation...

    doi arXiv

  9. [21]

    Cohort-Based Active Modality Acquisition , url=

    Rheude, Tillmann and Eils, Roland and Wild, Benjamin , year=. Cohort-Based Active Modality Acquisition , url=. doi:10.48550/arXiv.2505.16791 , abstractNote=

    doi arXiv

  10. [22]

    Fusion or Confusion? Multimodal Complexity Is Not All You Need , url=

    Rheude, Tillmann and Eils, Roland and Wild, Benjamin , year=. Fusion or Confusion? Multimodal Complexity Is Not All You Need , url=. doi:10.48550/arXiv.2512.22991 , abstractNote=

    doi arXiv

  11. [23]

    Understanding Dimensional Collapse in Contrastive Self-supervised Learning , url=

    Jing, Li and Vincent, Pascal and LeCun, Yann and Tian, Yuandong , year=. Understanding Dimensional Collapse in Contrastive Self-supervised Learning , url=. The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 , publisher=

    2022

  12. [24]

    Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function , volume=

    Tarnowski, Wojciech and Warchoł, Piotr and Jastrz ebski, Stanisław and Tabor, Jacek and Nowak, Maciej , editor=. Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function , volume=. Proceedings of the Twenty-Second International Confere...

    2019

  13. [25]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , author=

    Low Rank Gradients and Where to Find Them , url=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , author=

  14. [26]

    First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022 , author=

    Investigating Why Contrastive Learning Benefits Robustness against Label Noise , url=. First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022 , author=

    2022

  15. [28]

    and Yaida, Sho , year=

    Hoffman, Judy and Roberts, Daniel A. and Yaida, Sho , year=. Robust Learning with Jacobian Regularization , url=. doi:10.48550/arXiv.1908.02729 , abstractNote=

    1908 doi arXiv

  16. [30]

    Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks , volume=

    Xiao, Lechao and Bahri, Yasaman and Sohl-Dickstein, Jascha and Schoenholz, Samuel and Pennington, Jeffrey , editor=. Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks , volume=. Proceedings of the 35th Internati...

    2018

  17. [31]

    Initialization of ReLUs for Dynamical Isometry , volume=

    Burkholz, Rebekka and Dubatovka, Alina , editor=. Initialization of ReLUs for Dynamical Isometry , volume=. Advances in Neural Information Processing Systems , publisher=

  18. [32]

    Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA , author=

    Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice , url=. Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA , author=. 2017 , pages=

    2017

  19. [33]

    2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , author=

    Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , url=. 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , author=

    2014

  20. [34]

    and Gilmer, Justin and Ganguli, Surya and Sohl-Dickstein, Jascha , year=

    Schoenholz, Samuel S. and Gilmer, Justin and Ganguli, Surya and Sohl-Dickstein, Jascha , year=. Deep Information Propagation , url=. 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings , publisher=

    2017

  21. [35]

    Always Skip Attention , booktitle=

    Ji, Yiping and Saratchandran, Hemanth and Moghadam, Peyman and Lucey, Simon , year=. Always Skip Attention , booktitle=

  22. [36]

    Improving Transformer Optimization Through Better Initialization , volume=

    Huang, Xiao Shi and Perez, Felipe and Ba, Jimmy and Volkovs, Maksims , editor=. Improving Transformer Optimization Through Better Initialization , volume=. Proceedings of the 37th International Conference on Machine Learning , publisher=. 2020 , month=jul, pages=

    2020

  23. [38]

    Attention is All you Need , volume=

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, Łukasz and Polosukhin, Illia , editor=. Attention is All you Need , volume=. Advances in Neural Information Processing Systems , publisher=

  24. [40]

    The Thirteenth International Conference on Learning Representations , author=

    What to align in multimodal contrastive learning? , url=. The Thirteenth International Conference on Learning Representations , author=

  25. [41]

    PiCME: Pipeline for Contrastive Modality Evaluation and Encoding in the MIMIC Dataset , url=

    Golovanevsky, Michal and Mahableshwarkar, Pranav and Eickhoff, Carsten and Singh, Ritambhara , year=. PiCME: Pipeline for Contrastive Modality Evaluation and Encoding in the MIMIC Dataset , url=. doi:10.48550/arXiv.2507.03165 , abstractNote=

    doi arXiv

  26. [42]

    Sigmoid Loss for Language Image Pre-Training , booktitle=

    Zhai, Xiaohua and Mustafa, Basil and Kolesnikov, Alexander and Beyer, Lucas , year=. Sigmoid Loss for Language Image Pre-Training , booktitle=

  27. [43]

    LiT: Zero-Shot Transfer With Locked-Image Text Tuning , booktitle=

    Zhai, Xiaohua and Wang, Xiao and Mustafa, Basil and Steiner, Andreas and Keysers, Daniel and Kolesnikov, Alexander and Beyer, Lucas , year=. LiT: Zero-Shot Transfer With Locked-Image Text Tuning , booktitle=

  28. [44]

    and Sung, Yun-Hsuan and Li, Zhen and Duerig, Tom , editor=

    Jia, Chao and Yang, Yinfei and Xia, Ye and Chen, Yi-Ting and Parekh, Zarana and Pham, Hieu and Le, Quoc V. and Sung, Yun-Hsuan and Li, Zhen and Duerig, Tom , editor=. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision , url=. Proceedings ...

    2021

  29. [45]

    International Conference on Learning Representations , author=

    LoRA: Low-Rank Adaptation of Large Language Models , url=. International Conference on Learning Representations , author=

  30. [47]

    Circulation , author=

    PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals , volume=. Circulation , author=. 2000 , month=jun, pages=

    2000

  31. [50]

    Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine , volume=

    Chen, Emma and Kansal, Aman and Chen, Julie and Jin, Boyang Tom and Reisler, Julia and Kim, David E and Rajpurkar, Pranav , editor=. Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine , vo...

    2023

  32. [51]

    Scientific Data , author=

    MC-MED, multimodal clinical monitoring in the emergency department , volume=. Scientific Data , author=. 2025 , month=jul, pages=. doi:10.1038/s41597-025-05419-5 , number=

    2025 doi

  33. [52]

    Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning , url=

    Rheude, Tillmann and Hegselmann, Stefan and Eils, Roland and Wild, Benjamin , year=. Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning , url=. doi:10.48550/arXiv.2604.05834 , abstractNote=

    doi arXiv

  34. [53]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , author=

    A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity , url=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , author=

  35. [54]

    The Thirteenth International Conference on Learning Representations , author=

    Gramian Multimodal Representation Learning and Alignment , url=. The Thirteenth International Conference on Learning Representations , author=

  36. [55]

    Learning Transferable Visual Models From Natural Language Supervision , url=

    Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and Krueger, Gretchen and Sutskever, Ilya , editor=. Learning Transferable Visual Models From...

    2021

  37. [56]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , author=

    Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities , url=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , author=

  38. [58]

    Acosta, Guido J

    Julián N. Acosta, Guido J. Falcone, Pranav Rajpurkar, and Eric J. Topol. Multimodal biomedical ai. Nature Medicine, 28 0 (9): 0 1773–1784, September 2022. ISSN 1078-8956, 1546-170X. doi:10.1038/s41591-022-01981-2

    2022 doi

  39. [59]

    Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Gary Cottrell, and Julian J. McAuley. Rezero is all you need: fast convergence at large depth. In Cassio P. de Campos, Marloes H. Maathuis, and Erik Quaeghebeur (eds.), Proceedings of the Thirty-Seventh Confere...

    2021

  40. [60]

    Initialization of relus for dynamical isometry

    Rebekka Burkholz and Alina Dubatovka. Initialization of relus for dynamical isometry. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL ...

    2019

  41. [61]

    The geometric mechanics of contrastive representation learning: Alignment potentials, entropic dispersion, and cross-modal divergence

    Yichao Cai, Zhen Zhang, Yuhang Liu, and Javen Qinfeng Shi. The geometric mechanics of contrastive representation learning: Alignment potentials, entropic dispersion, and cross-modal divergence. CoRR, abs/2601.19597, 2026. doi:10.48550/ARXIV.2601.19597. URL https://doi.org/10.4...

    2026 doi arXiv

  42. [62]

    A triangle enables multimodal alignment beyond cosine similarity

    Giordano Cicchetti, Eleonora Grassucci, and Danilo Comminiello. A triangle enables multimodal alignment beyond cosine similarity. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 a . URL https://openreview.net/forum?id=3Hjfzh5Eyk

    2025

  43. [63]

    Gramian multimodal representation learning and alignment

    Giordano Cicchetti, Eleonora Grassucci, Luigi Sigillo, and Danilo Comminiello. Gramian multimodal representation learning and alignment. In The Thirteenth International Conference on Learning Representations, 2025 b . URL https://openreview.net/forum?id=ftGnpZrW7P

    2025

  44. [64]

    The road less scheduled

    Aaron Defazio, Xingyu Yang, Ahmed Khaled, Konstantin Mishchenko, Harsh Mehta, and Ashok Cutkosky. The road less scheduled. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information ...

    2024

  45. [65]

    What to align in multimodal contrastive learning? In The Thirteenth International Conference on Learning Representations, 2025

    Benoit Dufumier, Javiera Castillo Navarro, Devis Tuia, and Jean-Philippe Thiran. What to align in multimodal contrastive learning? In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=Pe3AxLq6Wf

    2025

  46. [66]

    The invisible hand of physics: When video diffusion models know more than they show

    Parsa Esmati, Somjit Nath, Katja Hofmann, Derek Nowrouzezahrai, Samira Ebrahimi Kahou, and Majid Mirmehdi. The invisible hand of physics: When video diffusion models know more than they show. arXiv:2606.05328 [cs.GR], June 2026. URL http://arxiv.org/abs/2606.05328

    2026 arXiv

  47. [67]

    Pytorch lightning, March 2019

    William Falcon and The PyTorch Lightning team. Pytorch lightning, March 2019. URL https://github.com/Lightning-AI/lightning

    2019

  48. [68]

    Sineproject: Machine unlearning for stable vision-language alignment

    Arpit Garg, Hemanth Saratchandran, and Simon Lucey. Sineproject: Machine unlearning for stable vision-language alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 31735–31745, June 2026

    2026

  49. [69]

    Dahl, Justin Gilmer, Christopher J

    Varun Godbole, George E. Dahl, Justin Gilmer, Christopher J. Shallue, and Zachary Nado. Deep learning tuning playbook, 2023. URL http://github.com/google-research/tuning_playbook. Version 1.0

    2023

  50. [70]

    A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. Ch. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. Physiobank, physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals. Circulation, 101 ...

    2000

  51. [71]

    Picme: Pipeline for contrastive modality evaluation and encoding in the mimic dataset

    Michal Golovanevsky, Pranav Mahableshwarkar, Carsten Eickhoff, and Ritambhara Singh. Picme: Pipeline for contrastive modality evaluation and encoding in the mimic dataset. arXiv:2507.03165 [cs.LG], July 2025. URL http://arxiv.org/abs/2507.03165

    2025 arXiv

  52. [72]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp.\ 770–778. IEEE Computer Society, 2016. doi:10.1109/CVPR...

    2016 doi

  53. [73]

    Large language models are powerful electronic health record encoders

    Stefan Hegselmann, Georg Von Arnim, Tillmann Rheude, Noel Kronenberg, David Sontag, Gerhard Hindricks, Roland Eils, and Benjamin Wild. Large language models are powerful electronic health record encoders. npj Digital Medicine, 9 0 (1): 0 530, July 2026. ISSN 2398-6352. doi:10....

    2026 doi

  54. [74]

    Roberts, and Sho Yaida

    Judy Hoffman, Daniel A. Roberts, and Sho Yaida. Robust learning with jacobian regularization. arXiv:1908.02729 [stat.ML], August 2019. URL http://arxiv.org/abs/1908.02729

    1908 arXiv

  55. [75]

    Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  56. [76]

    Improving transformer optimization through better initialization

    Xiao Shi Huang, Felipe Perez, Jimmy Ba, and Maksims Volkovs. Improving transformer optimization through better initialization. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Le...

    2020

  57. [77]

    Always skip attention

    Yiping Ji, Hemanth Saratchandran, Peyman Moghadam, and Simon Lucey. Always skip attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 23115–23123, October 2025

    2025

  58. [78]

    Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th...

    2021

  59. [79]

    Understanding dimensional collapse in contrastive self-supervised learning

    Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse in contrastive self-supervised learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https:/...

    2022

  60. [80]

    MIMIC-IV

    Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. MIMIC-IV . PhysioNet , January 2023 a . doi:10.13026/6mm1-ek67. URL https://doi.org/10.13026/6mm1-ek67. Version 2.2

    2023 doi

  61. [81]

    Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo A. Celi, and Roger G. Mark. Mimic-iv, a freely accessible electronic health record dataset. Scientific Da...

    2023 doi

  62. [82]

    The more, the merrier: Contrastive fusion for higher-order multimodal alignment

    Stefanos Koutoupis, Michaela Areti Zervou, Konstantinos Kontras, Maarten De Vos, Panagiotis Tsakalides, and Grigorios Tsagkatakis. The more, the merrier: Contrastive fusion for higher-order multimodal alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

    2026

  63. [83]

    Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning

    Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Pr...

    2022

  64. [84]

    Rectifier nonlinearities improve neural network acoustic models

    Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectifier nonlinearities improve neural network acoustic models. In Proceedings of the 30th International Conference on Machine Learning, volume 28, Atlanta, Georgia, USA, 2013. JMLR

    2013

  65. [85]

    Schoenholz, and Surya Ganguli

    Jeffrey Pennington, Samuel S. Schoenholz, and Surya Ganguli. Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (e...

    2017

  66. [86]

    Melo, and Ana Paiva

    Petra Poklukar, Miguel Vasco, Hang Yin, Francisco S. Melo, and Ana Paiva. Geometric multimodal contrastive representation learning. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp.\ 17782–17800...

    2022

  67. [87]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and ...

    2021

  68. [88]

    Cohort-based active modality acquisition

    Tillmann Rheude, Roland Eils, and Benjamin Wild. Cohort-based active modality acquisition. arXiv:2505.16791 [cs], December 2025. URL http://arxiv.org/abs/2505.16791

    2025 arXiv

  69. [89]

    Fusion or confusion? multimodal complexity is not all you need

    Tillmann Rheude, Roland Eils, and Benjamin Wild. Fusion or confusion? multimodal complexity is not all you need. arXiv:2512.22991 [cs], January 2026 a . URL http://arxiv.org/abs/2512.22991

    2026 arXiv

  70. [90]

    Hidden in the multiplicative interaction: Uncovering fragility in multimodal contrastive learning

    Tillmann Rheude, Stefan Hegselmann, Roland Eils, and Benjamin Wild. Hidden in the multiplicative interaction: Uncovering fragility in multimodal contrastive learning. arXiv:2604.05834 [cs.LG], May 2026 b . URL http://arxiv.org/abs/2604.05834

    2026 arXiv

  71. [91]

    Contractive auto-encoders: Explicit invariance during feature extraction

    Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In Lise Getoor and Tobias Scheffer (eds.), Proceedings of the 28th International Conference on Machine Learning, ICML 2011, B...

    2011

  72. [92]

    Rosenblatt

    F. Rosenblatt. The Perceptron, a Perceiving and Recognizing Automaton: (Project Para). Report / Cornell Aeronautical Laboratory. Cornell Aeronautical Laboratory, 1957. URL https://books.google.de/books?id=P_XGPgAACAAJ

    1957

  73. [93]

    Contrasting with symile: Simple model-agnostic representation learning for unlimited modalities

    Adriel Saporta, Aahlad Manas Puli, Mark Goldstein, and Rajesh Ranganath. Contrasting with symile: Simple model-agnostic representation learning for unlimited modalities. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openrevi...

    2024

  74. [94]

    Saxe, James L

    Andrew M. Saxe, James L. McClelland, and Surya Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 1...

    2014 arXiv

  75. [95]

    Learnable latent embeddings for joint behavioural and neural analysis

    Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617 0 (7960): 0 360–368, May 2023. ISSN 0028-0836, 1476-4687. doi:10.1038/s41586-023-06031-6

    2023 doi

  76. [96]

    Jure Sokolic, Raja Giryes, Guillermo Sapiro, and Miguel R. D. Rodrigues. Robust large margin deep neural networks. IEEE Trans. Signal Process., 65 0 (16): 0 4265–4280, 2017. doi:10.1109/TSP.2017.2708039

    2017

  77. [97]

    Low rank gradients and where to find them

    Rishi Sonthalia, Michael Murray, and Guido Montufar. Low rank gradients and where to find them. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. URL https://openreview.net/forum?id=de3kwOXQ9e

    2026

  78. [98]

    Uk biobank: An open access resource for identifying the causes of a wide range of complex diseases of middle and old age

    Cathie Sudlow, John Gallacher, Naomi Allen, Valerie Beral, Paul Burton, John Danesh, Paul Downey, Paul Elliott, Jane Green, Martin Landray, Bette Liu, Paul Matthews, Giok Ong, Jill Pell, Alan Silman, Alan Young, Tim Sprosen, Tim Peakman, and Rory Collins. Uk biobank: An open a...

    2015 doi

  79. [99]

    Dynamical isometry is achieved in residual networks in a universal way for any activation function

    Wojciech Tarnowski, Piotr Warchoł, Stanisław Jastrz ebski, Jacek Tabor, and Maciej Nowak. Dynamical isometry is achieved in residual networks in a universal way for any activation function. In Kamalika Chaudhuri and Masashi Sugiyama (eds.), Proceedings of the Twenty-Second Int...

    2019

  80. [100]

    Resmlp: Feedforward networks for image classification with data-efficient training

    Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, and Hervé Jégou. Resmlp: Feedforward networks for image classification with data-efficient training. IEEE Trans. P...

    2023

  81. [101]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Inf...

    2017

  82. [102]

    Deepnet: Scaling transformers to 1,000 layers

    Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, and Furu Wei. Deepnet: Scaling transformers to 1,000 layers. IEEE Trans. Pattern Anal. Mach. Intell., 46 0 (10): 0 6761–6774, October 2024. ISSN 0162-8828. doi:10.1109/TPAMI.2024.3386927

    2024

  83. [103]

    Understanding contrastive representation learning through alignment and uniformity on the hypersphere

    Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of M...

    2020

  84. [104]

    Dynamical isometry and a mean field theory of cnns: How to train 10,000-layer vanilla convolutional neural networks

    Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel Schoenholz, and Jeffrey Pennington. Dynamical isometry and a mean field theory of cnns: How to train 10,000-layer vanilla convolutional neural networks. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th ...

    2018

  85. [105]

    Investigating why contrastive learning benefits robustness against label noise

    Yihao Xue, Kyle Whitecross, and Baharan Mirzasoleiman. Investigating why contrastive learning benefits robustness against label noise. In First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022, 2022. URL https://openreview.net/forum?id=s436PHXRzMm

    2022

  86. [106]

    Chang, Amilcare Gentili, and Chun-Nan Hsu

    An Yan, Julian McAuley, Xing Lu, Jiang Du, Eric Y. Chang, Amilcare Gentili, and Chun-Nan Hsu. Radbert: Adapting transformer-based language models to radiology. Radiology: Artificial Intelligence, 4 0 (4): 0 e210258, July 2022. ISSN 2638-6100. doi:10.1148/ryai.210258

    2022 doi

  87. [107]

    Decipher the modality gap in multimodal contrastive learning: From convergent representations to pairwise alignment, 2025

    Lingjie Yi, Raphael Douady, and Chao Chen. Decipher the modality gap in multimodal contrastive learning: From convergent representations to pairwise alignment, 2025. URL https://arxiv.org/abs/2510.03268

    2025

  88. [108]

    Lit: Zero-shot transfer with locked-image text tuning

    Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. Lit: Zero-shot transfer with locked-image text tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 18123–18133,...

    2022

  89. [109]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 11975–11986, October 2023

    2023

  90. [110]

    Qwen3 embedding: Advancing text embedding and reranking through foundation models

    Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv:2506.05176 [cs], June 2025. URL ht...

    2025 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.