Pith. sign in

REVIEW 1 major objections 6 minor 87 references

Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Models

T0 review · 1 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The Vision-Language-Vision (VLV) auto-encoder distills a frozen text-to-image diffusion model into 77 continuous caption tokens, then fine-tunes a small open LLM to read those tokens back as captions that match GPT-4o at a total training…

desk verdict A genuinely cheap all-open-source captioner whose core contribution—diffusion decoding creates verbalizable semantics—is plausible but not yet isolated from the 6M Gemini captions that drive Stage-2. read the letter →

arxiv 2507.07104 v2 pith:C3OCQSYF submitted 2025-07-09 cs.CV

Tiezheng Zhang , Yitong Li , Yu-cheng Chou , Jieneng Chen , Alan Yuille , Chen Wei , Junfei Xiao This is my paper
classification cs.CV
keywords imagecaptioningdiffusionmodelsknowledgedistillationinformationbottleneckvision-languageself-supervisedlearningcontinuousembeddingsspatialcompositionality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a competitive image captioning model can be built without web-scale image–text data, by borrowing knowledge already stored inside a frozen text-to-image diffusion model. A first stage trains an encoder to compress each image into 77 continuous caption-like tokens from which the frozen Stable Diffusion 2.1 decoder can reconstruct the image, so the reconstruction pressure is what forces real semantic content into those tokens. A second stage fine-tunes a small open language model to read the same tokens back out as natural-language captions, supervised by 6 million Gemini-generated captions. The resulting captioner, VLV, is rated on par with GPT-4o by human judges and a strong model judge, for a total training cost under $1,000 — three orders of magnitude below the pipelines it is compared with. If the claim holds, a frozen generative decoder's reconstruction bottleneck can substitute for most of the paired data that current vision-language pipelines require.

What carries the argument

The central object is the Vision-Language-Vision auto-encoder: a visual backbone (initialized from Florence-2) feeding a multimodal transformer that outputs 77 learnable-query tokens in the CLIP text-embedding space, with a frozen Stable Diffusion 2.1 text-to-image decoder doing the reconstruction. The load-bearing device is the information bottleneck — because the denoising objective (Eq. 1) is the only Stage-1 training signal and the decoder is frozen, the encoder can only succeed by distilling the diffusion model's cross-modal knowledge into those 77 tokens. In Stage-2 the same tokens pass through the frozen CLIP text encoder and an MLP projector into a fine-tuned Qwen-2.5 causal model (Eq. 2), which autoregressively generates captions of arbitrary length. The emergent behaviors — 3D-pose preservation and cross-image composition by truncating and concatenating embeddings — are presented as evidence that the bottleneck stores structured semantics rather than pixel statistics.

What would settle it

Retrain Stage-2 on the same 6M Gemini captions with embeddings from an encoder pretrained on a non-diffusion objective (masked autoencoding or plain L2 reconstruction) at matched compute, and compare caption FID and human ratings; if the diffusion-pretrained embeddings do not clearly beat that control, the frozen decoder is not the source of the captioning ability and the knowledge-distillation claim is falsified.

Watch

Extended reading notes

Core claim

VLV treats image understanding as the inverse of image synthesis. A frozen text-to-image diffusion model (Stable Diffusion 2.1) serves as a fixed decoder, and the Stage-1 objective is the standard denoising loss: the encoder must emit 77 continuous tokens, living in the CLIP text-embedding space, from which the frozen U-Net can reconstruct the image. Because the decoder cannot change, the only way to lower the reconstruction error is to pack into those tokens everything the decoder needs — object identity, spatial layout, pose, style. After pretraining on 40M unlabeled images, a pretrained Qwen-2.5 model is fine-tuned, with a lightweight projector, to read the same tokens out as free-form captions, supervised by 6M Gemini 2.0 Flash captions; the CLIP text encoder stays frozen throughout. The paper reports captions whose text-to-image reconstruction FID matches GPT-4o's within noise, arena ratings within 0.05 points on a 0–6 scale, and 32-shot VQA accuracy within one point of the best open captioner, and it shows the embeddings preserve object pose and can be concatenated across two images to compose novel scenes.

Load-bearing premise

The load-bearing premise is that forcing a frozen image-generation model to reconstruct each image squeezes genuinely verbalizable semantic content into the 77 tokens, rather than letting the decoder exploit low-level visual cues that no caption could ever express.

Editorial extensions

If this is right

  • High-quality captioning on par with GPT-4o becomes reproducible for under $1,000 of GPU time using only open-weight components, so the ability to build a strong captioner is no longer limited to large industrial budgets.
  • Image-only pretraining scales: moving the unlabeled corpus from 6M to 40M images steadily improves reconstruction FID and spatial pose recovery, so more unlabeled pictures, not more text pairs, drive progress.
  • The 77-token continuous embedding acts as a reusable cross-modal interface, supporting captioning, text-only VQA, text-to-image reconstruction, and multi-image composition from the same representation.
  • Spatial detail survives the language bottleneck: object layout and 3D pose recovered from the embeddings track the original images, so detail-rich captions need not sacrifice spatial structure.
  • Caption quality can be evaluated without human raters, through text-to-image reconstruction FID, which lets researchers iterate on captioners at machine speed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'distillation' contribution is only partly isolated: the 6M Gemini captions provide most of the linguistic surface, so a decisive test would hold the caption supervision fixed and vary only the Stage-1 pretraining objective — the paper's pipeline still pools both effects.
  • A testable extension, which the paper itself flags, is re-running the same recipe with a stronger open diffusion decoder (such as SD 3.5 or FLUX); better decoders should yield richer tokens and, if the distillation story is right, better captions and spatial understanding.
  • Because the tokens live in CLIP text space and compose algebraically, the same mechanism could serve as an editing interface: manipulating or concatenating caption tokens might become a lightweight, text-driven image-manipulation API.
  • The 77-token budget matches CLIP's text-token limit; relaxing that fixed budget and measuring how fidelity and verbalizability trade off would reveal whether the bottleneck is the semantics or just the size.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. The paper introduces the Vision-Language-Vision (VLV) auto-encoder, a two-stage framework for image captioning. Stage 1 trains a vision encoder to map images to 77 continuous caption embeddings, with a frozen Stable Diffusion 2.1 decoder as reconstruction target; Stage 2 fine-tunes a Qwen-2.5 LLM to decode these embeddings into natural-language captions, using 6M Gemini 2.0 Flash captions for supervision. The authors claim the method achieves captioning comparable to GPT-4o and Gemini 2.0 Flash on reconstruction FID and human/VLM ratings, at a total training cost under $1,000, using only 40M unlabeled images and 6M captions. The paper also reports emergent spatial-awareness and multi-image compositionality properties, and presents ablations on query count, module unfreezing, data scale, and decoder scale.

Significance. If the central attribution claim holds, the paper is significant: it offers a path to strong captioners using mostly unlabeled images and a frozen diffusion decoder, potentially reducing the need for web-scale paired data and large GPU budgets. The idea of using a frozen text-to-image decoder as a semantic regularizer is interesting and aligns with recent work on diffusion-based representation learning. However, the significance is conditional on the evidence that Stage-1, rather than the 6M Gemini captions and the pretrained Florence-2 encoder, is responsible for the captioning gains. Currently, the attribution is supported only by indirect reconstruction-FID ablations, and the cost and VQA claims contain methodological issues that need to be resolved before the headline contributions can be accepted.

major comments (1)
  1. [Appendix C, Table 7; Section 4.3, Table 4] The caption-arena evaluation in Section 4.2.2 has a confound: the judge is Gemini 2.0 Flash, the same model used to produce the 6M training captions for Stage-2. The model may be biased toward captions that imitate Gemini's style, and the judge may reward that style. The paper should either use a different judge (e.g., GPT-4o or a purely human panel) or show that the bias is negligible. The text also states VLV matches GPT-4o 'within <0.05 points' on the 0-6 scale, but the Gemini-rating column shows a difference of 0.07 (5.18 vs. 5.25); this claim needs correction or qualification.
minor comments (6)
  1. [Table 1, Section 4.2.1] The text states the VLV caption FID is 'essentially indistinguishable from GPT-4o's (difference < 0.5)', but at guidance scales 1-4 the differences are 0.69, 0.44, 0.60, and 0.65, respectively; only at guidance 2.0 is the difference below 0.5. The statement should be revised to match the reported numbers.
  2. [Section 4.2.3, Table 3] The claim that 'by 32-shot, VLV matches the best open-source model (Qwen-2.5) and sits within 1 percentage of the overall leader' is only true for VQAv2; on OK-VQA, VLV trails Qwen-2.5-VL by 0.95 points and Gemini by 2.06 points at 32 shots. The sentence should be qualified by benchmark.
  3. [Section 3.2, Eq. (1)] The conditioning mechanism is unclear: the paper says the frozen U-Net receives z (the caption embedding) as conditioning, but Stable Diffusion 2.1 expects text embeddings from its CLIP text encoder. Please specify how z is injected into the U-Net (e.g., as cross-attention keys/values, concatenated with text embeddings, or as a replacement) and whether the CLIP text encoder is used at all in the forward pass of the diffusion decoder.
  4. [Section 4.4.2] The compositionality experiments are purely qualitative; consider adding a quantitative measure (e.g., CLIP score or user study) to support the claim that concatenated embeddings preserve content and transfer style.
  5. [Appendix C, Table 7] The table caption says 'Skipping Stage-1 (first three rows)' but the rows are labeled by decoder size; clarify how the encoder is initialized and which modules are trained in each row. The sentence 'This leads the effectively' is incomplete.
  6. [General] There are several typos and formatting issues: 'closed-sourced' should be 'closed-source'; 'V AE' and 'V AEs' should be 'VAE'; the limitation section contains 'so re-distilling' where a capital letter or semicolon is needed. Please run a proofreading pass.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the VLV derivation chain is self-contained; the teacher-judge overlap is an evaluation confound, not an equation-level reduction.

full rationale

The core derivation is self-contained. Stage-1 (Eq. 1) trains the encoder through a standard denoising loss with a frozen diffusion decoder; Stage-2 (Eq. 2) is a standard autoregressive captioning loss on paired image-text data. Neither equation defines its output in terms of the claimed result, and no fitted parameter is renamed as a prediction. The claim that Stage-1 contributes verbalizable semantics is supported by the Table 7 ablation, which compares FID with and without Stage-1; although that ablation does not fully isolate caption-quality metrics, missing attribution is a support gap, not circularity. The main evaluation confound—Gemini 2.0 Flash generated the 6M training captions and also serves as the VLM judge in Table 2—is a benchmark-design concern rather than a derivation reduction: a Gemini rating is not forced by construction from training on Gemini captions, and the paper also reports human ratings and FID from an independent SD 3.5 decoder. Appendix B's CLIP-similarity relabeling of VQA answers is a post-hoc label adjustment, not a step in the claimed derivation. No load-bearing self-citation or imported uniqueness theorem is present; De-Diffusion [69] is used as a baseline and evaluation-protocol reference, but the pipeline equations are stated in the paper. Therefore no significant circularity is found.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or architectural entities are postulated; the caption embedding is a continuous latent in an existing CLIP text space, and all components are pretrained open-source models. The hand-chosen hyperparameters and domain assumptions listed here are the main extra ingredients beyond standard diffusion and LLM training.

free parameters (4)
  • Number of learnable query tokens Nq = 77
    Sets the information bottleneck size; ablated over 16, 32, 77 in Table 4, with FID improving as Nq grows. Chosen to match the 77-token CLIP limit of SD 2.1.
  • CLIP similarity relabeling threshold = 0.8
    In Appendix B, answers with CLIP cosine similarity >= 0.8 to ground truth are relabeled as correct, recovering 94 OK-VQA answers. Adopted post hoc and applied only to VLV's misses.
  • LAION data filtering thresholds = short side > 512, aspect ratio in [0.5, 2], watermark probability < 0.5
    Hand-chosen curation rules for the 40M image pool; they shape the distribution and exclude OCR-heavy content, which the Limitations section acknowledges.
  • Training hyperparameters = LR 5e-5 (stage 1) and 1e-5 (stage 2); 200K and 100K steps; batch 512 and 64
    Hand-selected; standard values, not derived from theory, and the paper provides no sensitivity analysis.
assumptions (5)
  • domain assumption The frozen SD 2.1 diffusion decoder serves as a valid semantic teacher for the image encoder (Eq. 1).
    The whole Stage-1 distillation rests on reconstruction loss being a proxy for semantic understanding; the paper does not prove the decoder cannot reconstruct from non-semantic cues.
  • domain assumption FID between original images and images regenerated from captions by SD 3.5 measures caption quality (Table 1).
    FID captures distributional fidelity, not necessarily caption informativeness or hallucination; used as the main quantitative caption benchmark.
  • domain assumption The 6M Gemini 2.0 Flash captions are a sufficient teacher for captioning (Section 3.3, Appendix A).
    Stage-2 supervised learning inherits any stylistic biases and errors in Gemini captions; the captioner is trained to imitate them.
  • standard math The autoregressive LM loss in Eq. 2 with an MLP projector is a valid way to align CLIP-text-space embeddings to LLM token space.
    This follows standard multimodal LLM alignment practice; no derivation is required, but it is an architectural assumption.
  • standard math Standard diffusion denoising loss and U-Net conditioning (Eq. 1) from Ho et al. 2020 and Rombach et al. 2022.
    The paper relies on the established diffusion training objective without modification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Models." pith.science (2026). https://pith.science/paper/C3OCQSYF

@misc{pith2026250707104,
  author       = {Pith},
  title        = {Pith review of: Vision-Language-Vision Auto-Encoder: Scalable Knowledge Distillation from Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C3OCQSYF}},
  note         = {Machine review of arXiv:2507.07104}
}
abstract

Building state-of-the-art Vision-Language Models (VLMs) with strong captioning capabilities typically necessitates training on billions of high-quality image-text pairs, requiring millions of GPU hours. This paper introduces the Vision-Language-Vision (VLV) auto-encoder framework, which strategically leverages key pretrained components: a vision encoder, the decoder of a Text-to-Image (T2I) diffusion model, and subsequently, a Large Language Model (LLM). Specifically, we establish an information bottleneck by regularizing the language representation space, achieved through freezing the pretrained T2I diffusion decoder. Our VLV pipeline effectively distills knowledge from the text-conditioned diffusion model using continuous embeddings, demonstrating comprehensive semantic understanding via high-quality reconstructions. Furthermore, by fine-tuning a pretrained LLM to decode the intermediate language representations into detailed descriptions, we construct a state-of-the-art (SoTA) captioner comparable to leading models like GPT-4o and Gemini 2.0 Flash. Our method demonstrates exceptional cost-efficiency and significantly reduces data requirements; by primarily utilizing single-modal images for training and maximizing the utility of existing pretrained models (image encoder, T2I diffusion model, and LLM), it circumvents the need for massive paired image-text datasets, keeping the total training expenditure under $1,000 USD.

Figures

Figures reproduced from arXiv: 2507.07104 by the authors.

Figure 1
Figure 1. VLV matches GPT-4o’s descriptive fidelity at three orders of magnitude lower cost. Left: VLV captures all salient objects, matching GPT-4o in coverage without hallucinations, yet better preserving their spatial layout. Right: On the FID–cost–throughput plane, VLV reaches comparable FID, trains for orders-of-magnitude less, and delivers vastly higher captions-per-dollar at inference—proving that detail-rich descripti… view at source ↗
Figure 2
Figure 2. Method Overview. Our method has two stages: 1) vision-language-vision autoencoding for learning language semantics, 2) representation decoding into discrete language tokens through multi-modal LLM alignment. Our model has three major modules (i) VLV Encoder: a visual backbone augmented with a lightweight multi-modal adapter maps an input image into continuous caption embedding with compact semantic information; (ii)… view at source ↗
Figure 3
Figure 3. Reconstruction with language semantics. For each original input image (top), we feed its caption embedding directly to the frozen diffusion decoder and obtain a reconstruction (middle) that preserves high-level semantics and fine-grained appearance cues. The same embedding is then decoded by the LLM; prompting Midjourney with that caption yields an image of high fidelity. Guidance Scale 1.0 2.0 3.0 4.0 Original [14]… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Representation Learning Beyond Text: Spatial Preservation. The figure compares the original images (left) with those reconstructed by our embeddings. The accurate 6D poses of individual objects and the relative spatial configurations among multiple objects demonstrate …
Figure 5
Figure 5. Figure 5: Continual Spatial Represen￾tation Learning VLV enables continual 3D spatial representation learning. # Images(M) 8 18 34 40 Angle (↓) 0.1564 0.1287 0.1227 0.1016 Center (↓) 0.1625 0.1498 0.1402 0.0988 Scale (↓) 0.1775 0.1773 0.1835 0.1222 [PITH_FULL_IMAGE:figures/full…
Figure 6
Figure 6. Figure 6: Emerging compositionality with multi-image semantics. Given two input images—a Siberian cat at the left edge of the frame and either (above) a Van Gogh-style painting or (bottom) a Mount Fuji landscape—we truncate and concatenate their caption embeddings and feed the c…
Figure 7
Figure 7. Figure 7: Data Filtering Principles. We filter and collect 40M images from LAION-2B-en-aesthetic. We apply filtering based on the image resolution and aspect ratio to ensure the image quality and then prompt Gemini 2.0 Flash with image-conditioned templates to generate rich, des…
Figure 8
Figure 8. Figure 8: VLV Captions’ Length Statistics. Histogram of token counts for all captions (our ∼6M image-text paired data, used for stage-2 captioning). Most captions fall in the 170−280 token band, with mean µ= 226.82 (red dashed) and median x˜= 226 (green dashed). 16 [PITH_FULL_I…
Figure 9
Figure 9. Figure 9: OK-VQA Example. Both our caption and Gemini caption do not mention the states information. But our caption not only capture the oranges but also the number of oranges. Our answers contain the right ones highlighting in LimeGreen. B VQA Analysis: Are “Ground Truth" labe…
Figure 10
Figure 10. Figure 10: Captioner Arena Example. All captions show the correct objects without hallucinations. Both our caption and GPT-4o caption show the spatial relationship while Qwen-2.5 VL does not. D Caption Evaluation with SoTA Multi-modal LLM (Gemini) We assess caption quality by qu…
Figure 11
Figure 11. Figure 11: VLV can capture spatial layout. The caption shows bear’s layout (in the center of the frame) in this image as well as the bear’s posture (head turned towards the right side), showing VLV’s ability of capturing spatial layout. Caption: A medium, eye-level shot depicts …
Figure 12
Figure 12. Figure 12: , [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: VLV can capture complex objects. Caption enumerates almost every object and correctly describe their spatial relationships, highlighting VLV’s comprehensive scene understanding. Caption: A medium shot captures a man skiing down a snow-covered slope. He is wearing a re…
Figure 14
Figure 14. Figure 14: VLV can capture human posture. Captions show details of human as well as his posture, demonstrating VLV’s fine-grained posture awareness. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 33 canonical work pages

  1. [1]

    Gpt-4o system card, 2024

    OpenAI GPT 4o Team. Gpt-4o system card, 2024

    2024

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Nocaps: Novel object captioning at scale

    Harsh Agrawal, Karan Desai, Yufei Wang, Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, Stefan Lee, and Peter Anderson. Nocaps: Novel object captioning at scale. In ICCV, pages 8948–8957, 2019

    2019

  4. [4]

    Pixtral 12b

    Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073, 2024

    arXiv

  5. [5]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. NeurIPS, 35:23716–23736, 2022

    2022

  6. [6]

    Openflamingo: An open-source framework for training large autoregressive vision-language models

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv preprint arXiv:2308.01390, 2023

    arXiv 2023

  7. [7]

    Layer normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016

    arXiv 2016

  8. [8]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

Show all 87 references
  1. [9]

    Introducing our multimodal models, 2023

    Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sa˘gnak Ta¸ sırlar. Introducing our multimodal models, 2023

    2023

  2. [10]

    Perception encoder: The best visual embeddings are not at the output of the network

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, et al. Perception encoder: The best visual embeddings are not at the output of the network. arXiv preprint arXiv:2504.13181, 2025

    2025 arXiv

  3. [11]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. In ECCV, pages 370–387. Springer, 2024

    2024

  4. [12]

    Pali-x: On scaling up a multilingual vision and language model

    Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, et al. Pali-x: On scaling up a multilingual vision and language model. arXiv preprint arXiv:2305.18565, 2023

    2023 arXiv

  5. [13]

    Pali: A jointly-scaled multilingual language- image model

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language- image model. arXiv preprint arXiv:2209.06794, 2022

    2022 arXiv

  6. [14]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015

    2015 arXiv

  7. [15]

    Deconstructing denoising diffusion models for self-supervised learning

    Xinlei Chen, Zhuang Liu, Saining Xie, and Kaiming He. Deconstructing denoising diffusion models for self-supervised learning. arXiv preprint arXiv:2401.14404, 2024

    2024 arXiv

  8. [16]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, pages 24185–24198, 2024

    2024

  9. [17]

    Nvlm: Open frontier-class multimodal llms

    Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nvlm: Open frontier-class multimodal llms. arXiv preprint arXiv:2409.11402, 2024

    2024 arXiv

  10. [18]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2024. 10

    2024 arXiv

  11. [19]

    Bert: Pre-training of deep bidi- rectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human lan...

    2019

  12. [20]

    Scaling rectified flow transformers for high-resolution image synthesis, 2024

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis, 2024. URL https://arxiv. org/abs/2403.03206, 2, 2024

    2024 arXiv

  13. [21]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873–12883, 2021

    2021

  14. [22]

    Introducing gemini 2.0: our new ai model for the agentic era, 2024

    Google. Introducing gemini 2.0: our new ai model for the agentic era, 2024. Accessed: Dec 2024

    2024

  15. [23]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913, 2017

    2017

  16. [24]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, pages 16000–16009, 2022

    2022

  17. [25]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NeurIPS, 30, 2017

    2017

  18. [26]

    Autoencoders, minimum description length and helmholtz free energy

    Geoffrey E Hinton and Richard Zemel. Autoencoders, minimum description length and helmholtz free energy. NeurIPS, 6, 1993

    1993

  19. [27]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33:6840– 6851, 2020

    2020

  20. [28]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  21. [29]

    Language is not all you need: Aligning perception with language models

    Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, et al. Language is not all you need: Aligning perception with language models. NeurIPS, 36:72096–72109, 2023

    2023

  22. [30]

    Soda: Bottleneck diffusion models for representation learning

    Drew A Hudson, Daniel Zoran, Mateusz Malinowski, Andrew K Lampinen, Andrew Jaegle, James L McClelland, Loic Matthey, Felix Hill, and Alexander Lerchner. Soda: Bottleneck diffusion models for representation learning. In CVPR, pages 23115–23127, 2024

    2024

  23. [31]

    Categorical reparameterization with gumbel-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016

    2016 arXiv

  24. [32]

    Auto-encoding variational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013

    2013

  25. [33]

    Obelics: An open web-scale filtered dataset of interleaved image-text documents

    Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al. Obelics: An open web-scale filtered dataset of interleaved image-text documents. NeurIPS, 36:71683–71702, 2023

    2023

  26. [34]

    Your diffusion model is secretly a zero-shot classifier

    Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. In ICCV, pages 2206–2217, 2023

    2023

  27. [35]

    Dreamteacher: Pretraining image backbones with deep generative models

    Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Seung Wook Kim, Karsten Kreis, Antonio Torralba, and Sanja Fidler. Dreamteacher: Pretraining image backbones with deep generative models. In ICCV, pages 16698–16708, 2023

    2023

  28. [36]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, pages 19730–19742. PMLR, 2023

    2023

  29. [37]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, pages 12888–12900. PMLR, 2022

    2022

  30. [38]

    What if we recaption billions of web images with llama-3? arXiv preprint arXiv:2406.08478, 2024

    Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, Yuyin Zhou, and Cihang Xie. What if we recaption billions of web images with llama-3? arXiv preprint arXiv:2406.08478, 2024. 11

    2024 arXiv

  31. [39]

    Baichuan-omni technical report

    Yadong Li, Haoze Sun, Mingan Lin, Tianpeng Li, Guosheng Dong, Tao Zhang, Bowen Ding, Wei Song, Zhenglin Cheng, Yuqi Huo, Song Chen, Xu Li, Da Pan, Shusen Zhang, Xin Wu, Zheng Liang, Jun Liu, Tao Zhang, Keer Lu, Yaqi Zhao, Yanjun Shen, Fan Yang, Kaicheng Yu, Tao Lin, Jianhua Xu...

  32. [40]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  33. [41]

    Language quantized autoencoders: Towards unsupervised text-image alignment

    Hao Liu, Wilson Yan, and Pieter Abbeel. Language quantized autoencoders: Towards unsupervised text-image alignment. NeurIPS, 36:4382–4395, 2023

    2023

  34. [42]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36:34892–34916, 2023

    2023

  35. [43]

    Sgdr: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016

    2016 arXiv

  36. [44]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  37. [45]

    The concrete distribution: A continuous relaxation of discrete random variables

    Chris J Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712, 2016

    2016 arXiv

  38. [46]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In CVPR, pages 3195–3204, 2019

    2019

  39. [47]

    Kosmos-2: Grounding multimodal large language models to the world

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824, 2023

    2023 arXiv

  40. [48]

    Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision...

    2015

  41. [49]

    Diffusion autoencoders: Toward a meaningful and decodable representation

    Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwajanakorn. Diffusion autoencoders: Toward a meaningful and decodable representation. In CVPR, pages 10619–10629, 2022

    2022

  42. [50]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763. PmLR, 2021

    2021

  43. [51]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021

    2021

  44. [52]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    2022 arXiv

  45. [53]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, pages 8821–8831. Pmlr, 2021

    2021

  46. [54]

    Generating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. NeurIPS, 32, 2019

    2019

  47. [55]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, pages 10684–10695, 2022

    2022

  48. [56]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. NeurIPS, 35:25278–25294, 2022

    2022

  49. [57]

    Textcaps: a dataset for image captioning with reading comprehension

    Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. Textcaps: a dataset for image captioning with reading comprehension. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 742–758. Springer, 2020

    2020

  50. [58]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 12

    2023 arXiv

  51. [59]

    Addp: Learning general representations for image recognition and generation with alternating denoising diffusion process

    Changyao Tian, Chenxin Tao, Jifeng Dai, Hao Li, Ziheng Li, Lewei Lu, Xiaogang Wang, Hongsheng Li, Gao Huang, and Xizhou Zhu. Addp: Learning general representations for image recognition and generation with alternating denoising diffusion process. arXiv preprint arXiv:2306.05423, 2023

    2023 arXiv

  52. [60]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. NeurIPS, 30, 2017

    2017

  53. [61]

    Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion

    Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. JMLR, 11(12), 2010

    2010

  54. [62]

    Git: A generative image-to-text transformer for vision and language

    Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-text transformer for vision and language. arXiv preprint arXiv:2205.14100, 2022

    2022 arXiv

  55. [63]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Song XiXuan, et al. Cogvlm: Visual expert for pretrained language models. NeurIPS, 37:121475– 121499, 2024

    2024

  56. [64]

    Diffusion feedback helps clip see better

    Wenxuan Wang, Quan Sun, Fan Zhang, Yepeng Tang, Jing Liu, and Xinlong Wang. Diffusion feedback helps clip see better. arXiv preprint arXiv:2407.20171, 2024

    2024 arXiv

  57. [65]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024

    2024 arXiv

  58. [66]

    Visual lexicon: Rich image features in language space

    XuDong Wang, Xingyi Zhou, Alireza Fathi, Trevor Darrell, and Cordelia Schmid. Visual lexicon: Rich image features in language space. arXiv preprint arXiv:2412.06774, 2024

    2024 arXiv

  59. [67]

    Infodiffusion: Representation learning using information maximizing diffusion models

    Yingheng Wang, Yair Schiff, Aaron Gokaslan, Weishen Pan, Fei Wang, Christopher De Sa, and V olodymyr Kuleshov. Infodiffusion: Representation learning using information maximizing diffusion models. In ICML, pages 36336–36354. PMLR, 2023

    2023

  60. [68]

    Simvlm: Simple visual language model pretraining with weak supervision

    Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, and Yuan Cao. Simvlm: Simple visual language model pretraining with weak supervision. arXiv preprint arXiv:2108.10904, 2021

    2021 arXiv

  61. [69]

    De-diffusion makes text a strong cross-modal interface

    Chen Wei, Chenxi Liu, Siyuan Qiao, Zhishuai Zhang, Alan Yuille, and Jiahui Yu. De-diffusion makes text a strong cross-modal interface. In CVPR, pages 13492–13503, 2024

    2024

  62. [70]

    Diffusion models as masked autoencoders

    Chen Wei, Karttikeya Mangalam, Po-Yao Huang, Yanghao Li, Haoqi Fan, Hu Xu, Huiyu Wang, Cihang Xie, Alan Yuille, and Christoph Feichtenhofer. Diffusion models as masked autoencoders. In ICCV, pages 16284–16294, 2023

    2023

  63. [71]

    Denoising diffusion autoencoders are unified self-supervised learners

    Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. Denoising diffusion autoencoders are unified self-supervised learners. In ICCV, pages 15802–15812, 2023

    2023

  64. [72]

    Florence-2: Advancing a unified representation for a variety of vision tasks (2023)

    Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing a unified representation for a variety of vision tasks (2023). URL https://arxiv. org/abs/2311.06242, 2023

    2023 arXiv

  65. [73]

    Palm2-vadapter: progressively aligned language model makes a strong vision-language adapter

    Junfei Xiao, Zheng Xu, Alan Yuille, Shen Yan, and Boyu Wang. Palm2-vadapter: progressively aligned language model makes a strong vision-language adapter. arXiv preprint arXiv:2402.10896, 2024

    2024 arXiv

  66. [74]

    mplug-2: A modularized multi-modal foundation model across text, image and video

    Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, et al. mplug-2: A modularized multi-modal foundation model across text, image and video. In ICML, pages 38728–38748. PMLR, 2023

    2023

  67. [75]

    Open- vocabulary panoptic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, and Shalini De Mello. Open- vocabulary panoptic segmentation with text-to-image diffusion models. In CVPR, pages 2955–2966, 2023

    2023

  68. [77]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  69. [78]

    Diffusion model as representation learner

    Xingyi Yang and Xinchao Wang. Diffusion model as representation learner. In ICCV, pages 18938–18949, 2023

    2023

  70. [79]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022

    2022 arXiv

  71. [80]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    2022 arXiv

  72. [81]

    Spae: Semantic pyramid autoencoder for multimodal generation with frozen llms

    Lijun Yu, Yong Cheng, Zhiruo Wang, Vivek Kumar, Wolfgang Macherey, Yanping Huang, David Ross, Irfan Essa, Yonatan Bisk, Ming-Hsuan Yang, et al. Spae: Semantic pyramid autoencoder for multimodal generation with frozen llms. NeurIPS, 36:52692–52704, 2023

    2023

  73. [82]

    Vision as bayesian inference: analysis by synthesis? Trends in cognitive sciences, 10(7):301–308, 2006

    Alan Yuille and Daniel Kersten. Vision as bayesian inference: analysis by synthesis? Trends in cognitive sciences, 10(7):301–308, 2006

    2006

  74. [83]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, pages 11975–11986, 2023

    2023

  75. [84]

    Vinvl: Revisiting visual representations in vision-language models

    Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. Vinvl: Revisiting visual representations in vision-language models. In CVPR, pages 5579–5588, 2021

    2021

  76. [85]

    Ground Truth

    Liming Zhao, Kecheng Zheng, Yun Zheng, Deli Zhao, and Jingren Zhou. Rleg: Vision-language repre- sentation learning with diffusion-based embedding generation. In ICML, pages 42247–42258. PMLR, 2023. 14 Appendices A Data Processing 16 B VQA Analysis: Are “Ground Truth" labels r...

    2023

  77. [86]

    Coverage of image elements – how well the caption mentions the salient objects, their attributes, actions, and contextual details

  78. [87]

    Absence of hallucinations – the caption must not invent objects, attributes, counts, spatial relations, or other details not present or implied by the image

  79. [88]

    1843\"paintedonit,positionedonatraintrackinfrontofatrainstation.Thetraincarispredominantlygraywithyellowaccentsonthefrontandaredstriperunningalongtheside.Thenumber\

    Object spatial layout consistency – whether spatial relationships ( left/right, above/below, front/behind, center, background/foreground) are described accurately. • Any incorrect or invented spatial relation is a hallucination. • Omitting an obvious spatial relation reduces c...

    2014

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.