Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

How Visual Representations Map to Language Feature Space in Multimodal LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Visual information enters the language model's feature space only in middle-to-late layers, not in early ones.

desk verdict A clean, useful descriptive study of where visual tokens enter the LLM's feature space, but the GemmaScope SAE checkpoint mismatch with the instruction-tuned backbone is a load-bearing caveat for the layer-18 convergence. read the letter →

arxiv 2506.11976 v2 pith:MNDJGFLD submitted 2025-06-13 cs.CV cs.LG

Constantin Venhoff , Ashkan Khakzar , Sonia Joseph , Philip Torr , Neel Nanda This is my paper · ORCID
classification cs.CVcs.LG
keywords multimodalLLMsparseautoencodercross-modalalignmentlinearadaptervisualinstructiontuningmechanisticinterpretabilityrepresentationmappinglayer-wiseanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show where and when visual information becomes part of the language model's own internal feature space in a vision-language model whose language backbone is frozen. It connects a frozen vision encoder to a frozen instruction-tuned language model through a trainable linear adapter, so the adapter alone must translate images into the model's existing linguistic representations. Then it uses pretrained sparse autoencoders (SAEs) of the language model as probes: how well visual hidden states reconstruct, how sparsely they activate, and whether the activated features describe the image. All three signals converge around layer 18 of 26, indicating that early layers represent visual tokens as out-of-distribution inputs and only mid-to-late layers carry the mapped visual semantics. If correct, this pinpoints a concrete architectural inefficiency worth attacking.

What carries the argument

Sparse autoencoders (SAEs) trained on the language model's own text activations are the load-bearing instrument. An SAE is a dictionary-learning model that reconstructs a hidden activation from a sparse weighted combination of interpretable features; because the language backbone is frozen, a pretrained SAE stays a valid snapshot of that backbone's feature space. The paper feeds visual token activations through these SAEs and reads off three quantities: reconstruction error, the fraction of dictionary features that fire, and whether the top firing features' text descriptions name something actually present in the image. The convergence of all three in the same layers is what carries the argument.

What would settle it

A concrete check: run the same SAE probes on visual tokens from a model where images are replaced by random noise or by tokens from a different modality. If reconstruction error and sparsity still converge around layer 18, the convergence would reflect generic out-of-distribution recovery rather than visual-to-language mapping. Another check: if the same convergence appears for text tokens placed in the image positions, the effect is positional rather than visual.

Watch

Extended reading notes

Core claim

The central discovery is a layer-wise convergence point: in a 26-layer model with a linear vision-language adapter, visual representations are misaligned with language features in early layers and become fully expressible as sparse combinations of language features only around layer 18. The paper demonstrates this through three parallel measurements on visual token positions: SAE reconstruction error falls sharply, feature activation sparsity drops to language-typical levels, and the descriptions of the most strongly activated SAE features match image content at high rates. The authors interpret this as visual tokens gradually being translated into the language model's own feature dictionary, with the convergence marking where the cross-modal mapping is complete.

Load-bearing premise

The whole layer-by-layer story depends on the pretrained sparse autoencoders still being accurate dictionaries of the instruction-tuned model's features even though they were fit to the base text-only model.

Editorial extensions

If this is right

  • If the mapping converges only in middle-to-late layers, then early LLM layers are doing no useful visual semantic work in this architecture, and the adapter's output has to survive many layers before it is interpreted.
  • Layer 18 becomes a testable architectural target: inserting or training an adapter or skip connection that routes visual information into that layer could improve learning efficiency.
  • The same SAE-probe procedure can be applied to end-to-end fine-tuned VLMs to check whether the convergence point shifts earlier when the language backbone is allowed to adapt.
  • The convergence metrics could serve as a training signal or early-stopping criterion for adapter training, replacing benchmark-only evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's late-convergence result suggests the linear adapter is not truly translating vision into language semantics; it may be producing hidden states that only resemble language features after substantial nonlinear processing by the LLM.
  • One testable extension: if the adapter were trained directly to minimize SAE reconstruction error at layer 18, rather than next-token loss, alignment might occur earlier and need fewer parameters.
  • The result is consistent with a late-fusion interpretation in which visual tokens are processed as a foreign modality through the first half of the stack, which would predict that layer-18 features for visual tokens may be nearly interchangeable with corresponding text features.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies how visual tokens enter the residual-stream feature space of a frozen LLM in a LiMBeR-style VLM. The authors train a linear adapter between a frozen CLIP ViT-L/14 and a frozen Gemma-2-2b-it on LLaVA-1.5 data, then use GemmaScope sparse autoencoders (originally trained on base Gemma-2-2b) to measure layer-wise SAE reconstruction error, latent sparsity, and GPT-4o-scored semantic alignment of top SAE features for image tokens. The central empirical claim, in Section 4.3, is that all three quantities converge around layer 18 of 26, indicating that visual representations are progressively mapped onto LLM feature space in middle-to-late layers and remain misaligned in early layers.

Significance. If the layer-18 convergence is real and specific to visual inputs, the paper would provide a useful mechanistic account of cross-modal integration in adapter-based VLMs and demonstrate a practical use of frozen-backbone training for SAE-based interpretability. The manuscript is transparent about training data, hyperparameters, and code/weights, and the use of public GemmaScope SAEs is methodologically appealing. However, two load-bearing assumptions—SAE faithfulness across the base/instruct checkpoint difference and the specificity of the convergence to visual (rather than generic out-of-distribution) input—are not established, and the semantic-alignment metric lacks validation. These gaps currently prevent the strong conclusion in Section 4.3.

major comments (4)
  1. [Section 3.1.2 vs. Sections 1 and 2.2] The backbone is stated in Section 3.1.2 to be Gemma-2-2b-it, while the GemmaScope SAEs used in Section 2.2 and Section 4 were 'originally trained on Gemma-2-2b' (Section 1). Because instruction tuning reshapes residual-stream activations even with frozen weights, Equations (1) and (2) measure how well a base-model dictionary reconstructs and sparsely encodes instruct-model activations. This checkpoint mismatch can inflate early-layer reconstruction error and distort sparsity, and it may create the appearance of a mid-layer convergence even if the visual tokens are actually well-aligned with the instruct model's true features much earlier (or never). The paper should validate SAE faithfulness on Gemma-2-2b-it text-only activations against Gemma-2-2b text-only activations, or train/obtain SAEs on the instruct checkpoint, before interpreting the layer-wise curves.
  2. [Section 4.2 and Figures 1-2] The text-only baseline in Section 4.2 is in-distribution for the LLM and therefore cannot control for the possibility that any out-of-distribution input--not just visual tokens--shows decreasing SAE reconstruction error and increasing sparsity through deep layers. The central claim in Section 4.3 that visual representations are 'fully mapped onto LLM features' at layer 18 requires showing that the convergence is specific to visual content. I recommend adding controls with non-semantic inputs of matched norm and position, such as Gaussian noise, shuffled or permuted image features, or features from an unrelated modality passed through the same adapter, and reporting whether these also converge near layer 18.
  3. [Section 4.3, Figures 1 and 2] The claim that 'reconstruction error, sparsity, and semantic alignment all converge around layer 18' is made from visual inspection without a quantitative convergence criterion, error bars, or statistical testing. Since the specific layer number is the paper's headline result, the authors should define an operational rule (for example, reconstruction error within a stated margin of the text-only baseline for a run of consecutive layers, or a change-point analysis) and report variability across examples, random seeds, and repeated evaluations of the GPT-4o judge.
  4. [Section 4.1 and Figure 2] The semantic alignment metric depends on GPT-4o judgments of whether any of the top three SAE feature descriptions 'strongly matches' a concept in the image, after excluding features with ad-hoc frequency thresholds (5% of images, 0.5% of the LLM dataset). No human validation, inter-annotator agreement, or ablation of these thresholds and of the top-3 selection is reported. Because Figure 2 is one of the three pillars supporting the layer-18 conclusion, the reliability of this metric should be demonstrated, for example by comparing GPT-4o judgments against human raters on a sample.
minor comments (5)
  1. [Section 1] The phrase 'the SAEs acts as an analytical probe' should be 'the SAEs act as analytical probes', and the parenthesis around '(LiMBeR framework [17])' contains an extra closing parenthesis.
  2. [Section 2.2] When introducing GemmaScope, the manuscript should state explicitly that the SAEs were trained on the base Gemma-2-2b checkpoint, not on the instruction-tuned variant, to avoid ambiguity with Section 3.1.2.
  3. [Section 4.2] The experimental setup reports 5000 and 1000 examples but does not state the number of visual tokens per image or the number of repeated runs; adding this information would help readers assess the stability of the curves in Figures 1 and 2.
  4. [Table 1] The MME rows list 'Cognition' and 'Perception' subscores but no total score; please state whether these are standard MME subtotals and how they compare to the published aggregate.
  5. [Section 4.1, Equation (2)] The sparsity definition is the fraction of nonzero SAE latents; the y-axis label 'Sparsity' in Figure 1 could mislead readers into thinking higher is more sparse, and the caption should clarify that lower values mean sparser activations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the layer-wise SAE measurements are empirical and not fitted to the claimed conclusion.

full rationale

Every load-bearing step is a direct measurement. Equation (1) feeds frozen Gemma-2-2b-it visual-token activations through a fixed GemmaScope SAE and averages MSE; Equation (2) counts nonzero SAE features; the semantic-alignment score uses GPT-4o judgments on fixed SAE descriptions. The claimed layer-18 convergence is read off these curves; no parameter of the model is fitted to make the curves converge, and the text-only prompt-position curves provide an external baseline rather than defining the result. The GemmaScope citation [13] is a publicly released pretrained artifact trained on text activations of Gemma-2-2b, not on this VLM's visual activations; even though one author is shared, the dictionary is not constructed from the quantity being predicted, so the citation is independent evidence under the review rules. The only flagged weakness is the base- versus instruct-checkpoint mismatch: Section 1 claims the SAEs 'remain perfectly aligned with the unchanged language model,' while Section 3.1.2 specifies Gemma-2-2b-it as backbone and Section 2.2 notes GemmaScope SAEs are 'pre-trained SAEs for Gemma2-2b.' This is a validity threat (distribution shift could distort early-layer MSE and sparsity, manufacturing a mid-layer convergence), not a circularity: the mismatch does not make the conclusion true by construction. Self-citation [29] is limited to related work and is not load-bearing.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities and fits no mathematical constants, but it relies on several unstated interpretive assumptions: SAE metrics are treated as measures of semantic alignment, the base-model SAEs are assumed to apply to the instruct model, the artificial text-only baseline is assumed comparable, and GPT-4o is trusted as a judge. These assumptions are the main reason soundness is limited.

free parameters (3)
  • SAE feature frequency exclusion thresholds = 5% of image dataset, 0.5% of LLM dataset
    Features activating more often than these thresholds are removed before semantic alignment evaluation; thresholds are chosen by hand and materially affect the match-rate curves (Section 4.1).
  • Number of SAE features evaluated per example = top 3
    Only the top three activating SAE features are judged for semantic match; changing this count changes the alignment metric (Section 4.1).
  • Semantic match criterion in GPT-4o prompt = not specified
    The definition of 'strongly matches' is left to the LLM judge; changing the prompt changes the match rate, and no prompt details are provided (Section 4.1).
assumptions (4)
  • domain assumption SAE reconstruction error and sparsity directly measure how well visual activations are represented by the LLM's language feature space.
    The central interpretation of Figures 1-2 rests on this mapping from an observability metric to an alignment claim; it is not independently validated in the paper (Section 4.1).
  • domain assumption GemmaScope SAEs trained on the base language model are applicable to the instruction-tuned model.
    The paper uses Gemma-2-2b-it with SAEs 'originally trained on Gemma-2-2b' without verifying feature alignment across checkpoints (Sections 1 and 4).
  • domain assumption The text-only baseline ('Consider the following information: ...') is a fair proxy for normal language activations.
    The baseline prompt differs substantially from the actual VLM prompt format, and no validation of comparability is provided (Section 4.2).
  • domain assumption GPT-4o can reliably decide whether an SAE feature description matches image content.
    The semantic alignment metric depends on a single LLM judge without inter-annotator agreement, prompt sensitivity checks, or error bars (Section 4.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Visual Representations Map to Language Feature Space in Multimodal LLMs." pith.science (2026). https://pith.science/paper/MNDJGFLD

@misc{pith2026250611976,
  author       = {Pith},
  title        = {Pith review of: How Visual Representations Map to Language Feature Space in Multimodal LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MNDJGFLD}},
  note         = {Machine review of arXiv:2506.11976}
}
read the original abstract

Effective multimodal reasoning depends on the alignment of visual and linguistic representations, yet the mechanisms by which vision-language models (VLMs) achieve this alignment remain poorly understood. Following the LiMBeR framework, we deliberately maintain a frozen large language model (LLM) and a frozen vision transformer (ViT), connected solely by training a linear adapter during visual instruction tuning. By keeping the language model frozen, we ensure it maintains its original language representations without adaptation to visual data. Consequently, the linear adapter must map visual features directly into the LLM's existing representational space rather than allowing the language model to develop specialized visual understanding through fine-tuning. Our experimental design uniquely enables the use of pre-trained sparse autoencoders (SAEs) of the LLM as analytical probes. These SAEs remain perfectly aligned with the unchanged language model and serve as a snapshot of the learned language feature-representations. Through systematic analysis of SAE reconstruction error, sparsity patterns, and feature SAE descriptions, we reveal the layer-wise progression through which visual representations gradually align with language feature representations, converging in middle-to-later layers. This suggests a fundamental misalignment between ViT outputs and early LLM layers, raising important questions about whether current adapter-based architectures optimally facilitate cross-modal representation learning.

Figures

Figures reproduced from arXiv: 2506.11976 by the authors.

Figure 1
Figure 1. Layer-wise SAE reconstruction error and sparsity patterns. The y-axis shows the MSE reconstruction error (left) and the Sparsity [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Semantic alignment of SAE feature descriptions across [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Pathways of Visual Information Flow in Vision-Language Models

    cs.CV 2026-07 accept novelty 7.0 of 10

    VLMs use two flexible, task-dependent pathways for visual information—direct image readout and text-mediated transfer—with text mediation available as a fallback under intervention.

  2. Interpretable Open-Vocabulary Referring Object Detection with Reverse Contrast Attention

    cs.CV 2025-07 reject novelty 5.0 of 10

    A training-free attention reweighting method, Reverse Contrast Attention, is claimed to improve referring object detection in 11 of 15 VLMs, but the custom FitAP metric ranks boxes by IoU with ground truth, which make...

Reference graph

Works this paper leans on

29 extracted references · 19 canonical work pages · cited by 2 Pith papers

  1. [1]

    Towards monosemanticity: Decomposing language models with dictionary learning, 2023

    Trenton Bricken. Towards monosemanticity: Decomposing language models with dictionary learning, 2023. 1, 2

    work page 2023

  2. [2]

    Interpreting and controlling vision foundation mod- els via text explanations

    Haozhe Chen, Junfeng Yang, Carl V ondrick, and Chengzhi Mao. Interpreting and controlling vision foundation mod- els via text explanations. arXiv preprint arXiv:2310.10591,

    arXiv

  3. [3]

    InstructBLIP: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In Thirty- seventh Conference on Neural Information Processing Sys- tems, 2023. 2

    work page 2023

  4. [4]

    The cognitive revolution in interpretability: From explaining behavior to interpreting representations and algorithms, 2024

    Adam Davies and Ashkan Khakzar. The cognitive revolution in interpretability: From explaining behavior to interpreting representations and algorithms, 2024. 1

    work page 2024

  5. [5]

    Arik, Tejas Nama, and Tomas Pfister

    Sayna Ebrahimi, Sercan O. Arik, Tejas Nama, and Tomas Pfister. Crome: Cross-modal adapters for efficient multi- modal llm, 2024. 2

    work page 2024

  6. [6]

    A mathemati- cal framework for transformer circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathemati- cal framework for transformer circuits. Transformer Circuits Thread, 1(1):12, 2021. 2

    work page 2021

  7. [7]

    Mme: A compre- hensive evaluation benchmark for multimodal large language models, 2024

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A compre- hensive evaluation benchmark for multimodal large language models, 2024. 2

    2024

  8. [8]

    In- terpreting clip’s image representation via text-based decom- position

    Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. In- terpreting clip’s image representation via text-based decom- position. arXiv preprint arXiv:2310.05916, 2023. 2

    arXiv 2023

Show all 29 references
  1. [9]

    Sparse autoencoders find highly interpretable features in language models

    Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representa- tions, 2023. 2

    2023

  2. [10]

    Hudson and Christopher D

    Drew A. Hudson and Christopher D. Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. 2019 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 6693– 6702, 2019. 2

    2019

  3. [11]

    Interpreting and editing vision-language rep- resentations to mitigate hallucinations

    Nick Jiang, Anish Kachinthaya, Suzie Petryk, and Yossi Gandelsman. Interpreting and editing vision-language rep- resentations to mitigate hallucinations. arXiv preprint arXiv:2410.02762, 2024. 2

    2024 arXiv

  4. [12]

    Evaluating object hallucination in large vision- language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision- language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages 292–305, Singapore, 2023. Association for ...

    2023

  5. [13]

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2

    Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, J ´anos Kram´ar, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. arXiv preprint arXiv:2408.05147, 2024. 1, 2, 3

    2024 arXiv

  6. [14]

    Vila: On pre-training for vi- sual language models

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Moham- mad Shoeybi, and Song Han. Vila: On pre-training for vi- sual language models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 26689–26699, 2024. 1

    2024

  7. [15]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. 1, 2

    2023

  8. [16]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26296–26306, 2024. 2, 3

    2024

  9. [17]

    Linearly mapping from image to text space

    Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. Linearly mapping from image to text space. In The Eleventh International Conference on Learning Representa- tions, 2023. 1, 2, 4

    2023

  10. [18]

    Towards interpreting visual infor- mation processing in vision-language models.arXiv preprint arXiv:2410.07149, 2024

    Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. Towards interpreting visual infor- mation processing in vision-language models.arXiv preprint arXiv:2410.07149, 2024. 1, 2, 3

    2024 arXiv

  11. [19]

    Interpreting GPT: The Logit Lens

    nostalgebraist. Interpreting GPT: The Logit Lens. Less- Wrong (blog), 2020. 1, 3

    2020

  12. [20]

    Towards vision-language mechanistic interpretabil- ity: A causal tracing tool for blip

    Vedant Palit, Rohan Pandey, Aryaman Arora, and Paul Pu Liang. Towards vision-language mechanistic interpretabil- ity: A causal tracing tool for blip. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2856–2861, 2023. 2

    2023

  13. [21]

    Bridg- ing vision and language spaces with assignment prediction

    Jungin Park, Jiyoung Lee, and Kwanghoon Sohn. Bridg- ing vision and language spaces with assignment prediction. In The Twelfth International Conference on Learning Repre- sentations, 2024. 2

    2024

  14. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  15. [23]

    Multimodal neurons in pre- trained text-only transformers

    Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. Multimodal neurons in pre- trained text-only transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2862–2867, 2023. 2

    2023

  16. [24]

    Open problems in mechanistic interpretability

    Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Dill, Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et al. Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496, 2025. 1

    2025 arXiv

  17. [25]

    Paligemma 2: A family of versatile vlms for transfer, 2024

    Andreas Steiner, Andr ´e Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Grit- senko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sahar Kazemzadeh, Thomas Mesnard, Ibrahim Alabdulmohsin...

    2024

  18. [26]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram´e, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118,

    arXiv

  19. [27]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram´e, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le...

    2024

  20. [28]

    Metamorph: Multimodal un- derstanding and generation via instruction tuning, 2024

    Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal un- derstanding and generation via instruction tuning, 2024. 1

    2024

  21. [29]

    Too late to recall: The two-hop prob- lem in multimodal knowledge retrieval

    Constantin Venhoff, Ashkan Khakzar, Sonia Joseph, Philip Torr, and Neel Nanda. Too late to recall: The two-hop prob- lem in multimodal knowledge retrieval. In MIV Workshop at CVPR 2025 (Non-proceedings Track), 2025. 2 6

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.