Pith. sign in

REVIEW 3 major objections 4 minor 51 references

Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Unified 'any-to-any' models do not consistently beat specialist pairs on cross-modal consistency tests, and the only positive evidence is weak latent-space alignment visible under edit-based equivariance analysis.

desk verdict Useful benchmark and a plausible negative result, but the positive equivariance claim rests on an unvalidated assumption that two off-the-shelf editors implement the same edit. read the letter →

arxiv 2505.24211 v1 pith:HWOQFDDF submitted 2025-05-30 cs.CL

Jiwan Chung , Janghan Yoon , Junhyeong Park , Sangeyl Lee , Joowon Yang , Sooyeon Park , Youngjae Yu This is my paper
classification cs.CL
keywords any-to-anymodelsmultimodalconsistencycross-modaltransfercyclicequivariancelatentspacebenchmarkdatasetvisualtokenization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether unified any-to-any generative models — single systems that both interpret and generate across modalities with one shared parameter set — actually preserve meaning across modality transfers better than pairing two separate specialist models, one for text-to-image and one for image-to-text. To find out, the authors built ACON, a dataset of 1,000 images with dense human captions, 3,000 editing instructions, and 16,000 binary question-answer pairs, and defined three consistency criteria: cyclic consistency (round-trip recovery), forward equivariance (edit before or after a transfer), and conjugated equivariance (an edit applied inside the round trip). The central result is negative for the field's premise: any-to-any models do not consistently beat arbitrary specialist pairs on pointwise cyclic consistency, with the only notable exceptions, Seed-X and VILA-U, both using semantically aligned visual tokenizers. The positive evidence is weak and indirect: edit-based equivariance tests reveal faint signs of a shared latent space, visible only when multiple edits probe the intermediate representation in aggregate. If this finding holds, the theoretical promise that unified training yields more coherent cross-modal conversions is not yet realized in any consistent way for today's models.

What carries the argument

The paper's test object is the shared-latent hypothesis: if a single any-to-any model (one parameter set serving both directions, so that the text-to-image and image-to-text maps share $\phi = \psi$) learns one latent space $z$ from both modalities' partial views, its cross-modal conversions should cohere better than two specialists with separate latent approximations. The machinery that carries the argument is a three-part consistency suite run on the new ACON dataset: cyclic consistency (round-trip recovery, Eqs. 1–2), forward equivariance (edit-then-convert vs. convert-then-edit, Eqs. 3–4), and conjugated equivariance, in which an edit is applied inside the round trip, $x \xrightarrow{f^{i\to t}} z \xrightarrow{g_t} z' \xrightarrow{f^{t\to i}} \bar{x}'$ (Eqs. 5–6). Conjugated equivariance is the probe that carries the positive finding: by editing many points in the intermediate latent space, it turns a single-point test into a distributional one. Off-the-shelf tools supply the in-modality edits — CosXL for images ($g_i$) and Qwen2.5 for text ($g_t$) — while VQA solvers (PaliGemma2, Qwen2.5) score whether outputs answer the same binary questions, using ACON's 16,000 human-written Q&A pairs and 3,000 editing prompts as the question set.

What would settle it

Take a subset of ACON's editing prompts and have annotators verify that CosXL's image edit and Qwen2.5's text edit change the same stated fact in their own modality. Re-run the conjugated-equivariance evaluation on this validated subset: if the any-to-any models' self-consistency advantage over specialist pairs disappears, the weak-consistency finding was an artifact of editor mismatch; if it survives, the finding is genuine.

Watch

Extended reading notes

Core claim

The paper's central finding, stated on its own terms, is that any-to-any models do not consistently exhibit greater cross-modal consistency than arbitrary combinations of specialized models in pointwise evaluations such as cyclic consistency, but equivariance evaluations do uncover weak, observable consistency between text-to-image and image-to-text capabilities in distributional analyses of the intermediate latent space. The two any-to-any systems that break from the negative pattern, Seed-X and VILA-U, both build their visual tokenizers with semantic alignment, whereas Chameleon and Emu3, which rely on image reconstruction objectives alone, do not; the paper reads this as evidence that semantic modeling inside visual tokenization contributes to latent-space alignment. It also concludes that single-point cyclic tests conflate per-direction transfer quality with true cross-modal consistency, which is why a single criterion cannot settle whether unified models cohere better than specialist pairs.

Load-bearing premise

The weak-consistency result assumes that the off-the-shelf image editor and the text editor execute the same semantic edit for a given instruction, an equivalence the paper does not verify.

Editorial extensions

If this is right

  • Unified any-to-any training, as currently practiced, buys no reliable pointwise consistency advantage, so claims of cross-modal coherence should be demonstrated per model rather than assumed from the architecture.
  • Visual tokenization design is a concrete lever: only the any-to-any models with semantically aligned tokenizers (Seed-X, VILA-U) show notable self-consistency, pointing to where future unified models might actually gain coherence.
  • Pointwise cyclic tests conflate per-direction transfer quality with consistency, so evaluations of multimodal models should pair them with multi-point equivariance probes before drawing conclusions.
  • ACON provides a reusable diagnostic — 1,000 images (500 never seen in training), 3,000 editing prompts, and 16,000 binary Q&A pairs — for measuring whether future unified models improve on the consistency this study finds lacking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the negative cyclic-consistency result generalizes, the case for any-to-any models must rest on flexibility, shared computation, or editing-based coherence rather than conversion fidelity, and evaluations should target those axes directly.
  • A sharper test of the shared-latent hypothesis follows from the paper's design: on a validated set of edits shown to be executed equivalently by both editors, the equivariance gap of a genuinely unified model should be invariant to which editor supplies the in-modality edit; measuring that invariance would separate model coherence from tool behavior.
  • Porting the ACON protocol to other modality pairs, such as speech-to-text or image-to-audio, would show whether the weak-consistency finding is a property of today's unified training or specific to the text-image pair.
  • The tokenizer observation suggests a controlled ablation: training the same architecture with and without semantic alignment in the visual tokenizer would isolate how much of the consistency gain comes from tokenization rather than from the generative objective.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ACON, a benchmark of 1,000 images (500 newly collected) with human-written dense captions, editing instructions, and VQA-style question-answer pairs, and uses it to test whether any-to-any multimodal models (Chameleon, Emu3, VILA-U, Seed-X) exhibit greater cross-modal consistency than pairs of specialist models (Flux/SDXL for text-to-image, LLaVA/Qwen2VL for image-to-text). Consistency is operationalized through three criteria: cyclic consistency, forward equivariance, and conjugated equivariance. The authors report that any-to-any models do not consistently outperform specialist pairs on pointwise cyclic consistency, but that conjugated equivariance reveals 'weak but observable' self-alignment for most any-to-any models, which they attribute to shared latent structure. The dataset and code are released.

Significance. The paper makes a useful empirical contribution by constructing a carefully curated, human-in-the-loop benchmark that goes beyond retrieval-based metrics and by testing a concrete hypothesis about unified models. The negative result on cyclic consistency is interesting and likely robust to small perturbations, and the dataset itself may be reusable for future consistency evaluations. The positive equivariance claim is less convincing: it depends on an unvalidated assumption that an image editor and a text editor implement the same semantic edit, and it substitutes ground-truth answers for actual in-modality outputs. If the equivariance finding does not survive a control experiment, the paper still retains value as a negative result and a benchmark, but the headline story about 'weak but observable consistency' would need to be substantially revised.

major comments (3)
  1. [§4.4, Eqs. (5)–(6)] The positive finding of 'weak but observable consistency' rests on conjugated equivariance, which assumes gi ≃ gt in Section 2. However, CosXL and Qwen2.5 are never validated to apply the same semantic edit p to a paired (image, caption). Moreover, Section 4.4 states 'We do not generate the in-modality output (g(x,p))' and instead substitutes ground-truth answers, so the RHS of Eqs. (5)–(6) is an oracle-edited output, not the actual gt(xt,p) or gi(xi,p). As a result, any systematic over- or under-execution of p by CosXL appears as an equivariance failure for every model, making the self-alignment of Seed-X and VILA-U potentially an artifact of editor behavior rather than shared latent consistency. A control experiment measuring editor faithfulness (e.g., human ratings of whether each editor realizes the intended edit) is required before these numbers can be attributed to cross-modal consistency.
  2. [§4, Tables 1, 3, 4] All headline comparisons are point estimates with no confidence intervals or significance tests. Appendix A states that stochastic diffusion/flow models (SDXL, Flux) were run a single time. Many of the differences driving the paper's conclusions are small (e.g., Table 1 image column: LLaVA+Seed-X 62.57 vs LLaVA+Flux 61.78; Table 4 image column: Emu3+Emu3 62.76 vs VILA-U+VILA-U 56.90). Without error bars, bootstrap intervals over the 1,000 images, or multiple seeds, the claims that any-to-any models 'do not consistently outperform' specialists and that Seed-X/VILA-U are 'notable exceptions' are not statistically supported.
  3. [§4.4, definition of conjugated equivariance] The metric used for Eq. (5) does not test equivariance: instead of comparing f_i->t(gi(f_t->i(xt))) with gt(xt,p), it scores the model output against oracle ground-truth answers ho(g(x,p),c,p). This measures how well the model's edited reconstruction matches an idealized edit, not whether the diagram g∘f = f∘g commutes. The term 'equivariance' is therefore misleading; the experiment is better described as edit-conditioned reconstruction accuracy. The authors should either compute the actual gt(xt,p) (and gi(xi,p)) outputs and compare both sides through the same evaluator, or rename and reinterpret the criterion.
minor comments (4)
  1. [§4.6 vs Appendix A] Section 4.6 says 'we employ deterministic sampling throughout,' but Appendix A says 'Exceptions include diffusion and flow-matching models... we run these models a single time.' This contradiction should be resolved.
  2. [Tables 1, 3, 4] The tables use color to mark results obtained with a single any-to-any model, but color is not visible in monochrome print and may be lost in some PDF renderings; add explicit markers such as asterisks or a dedicated column.
  3. [Figures 4 and 5] The figures are normalized to [0,1] per row, but the captions and main text do not specify the normalization transform or explain whether the values are ranks or affine-rescaled correlations; please state the exact procedure.
  4. [§2, Eqs. (3)–(4)] The notation gi and gt in the equivariance equations is defined only in prose; the equations themselves use g with a subscript that could be confused with the model parameters φ and ψ. Adding a table of symbols or explicitly writing gi(·,p) and gt(·,p) in the equations would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ACON benchmark and consistency comparisons are externally grounded in human annotations and off-the-shelf evaluators.

full rationale

The paper does not derive its central claims from its own definitions or fitted parameters. The consistency criteria (cyclic, forward equivariance, conjugated equivariance) are formalized in Section 2 as testable equations, and the evaluation uses human-annotated ground-truth Q&A pairs plus external VQA models (PaliGemma2, Qwen2.5) rather than model-generated targets. The dataset construction (Section 3) involves human teller/drawer/comparer roles, and the filtering and calibration procedures are described; the benchmark numbers are not produced by the models being evaluated. The only potentially load-bearing assumption is the use of off-the-shelf editors CosXL and Qwen2.5 to instantiate g_i and g_t with the stated equivalence g_i ≃ g_t (Section 2). This is an external-validity concern about editor fidelity, not a circular reduction: the paper does not define consistency in terms of the editors' outputs, nor does it fit any parameter to the equivariance results. The shared-latent hypothesis is invoked only as a post-hoc interpretation of empirical differences, not as a premise that forces the measured outcomes. No self-citation chains, uniqueness theorems, or renamed known results are used to justify the core findings. Therefore, under the stated rules, there is no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No numerical parameters were fitted and no theoretical entities were introduced. The ledger consists of methodological assumptions about edit equivalence, VQA fidelity, and dataset representativeness that the experiments inherit.

assumptions (5)
  • domain assumption A unified latent space should yield more coherent cross-modal conversions than separate specialists.
    Stated as the paper's conjecture in Section 1 and used in Section 4.4 to interpret diagonal self-consistency; not established independently of the evaluation.
  • domain assumption The image edit operation g_i and text edit operation g_t are semantically equivalent for a given prompt p.
    Section 2 introduces g_i = g_t to define forward and conjugated equivariance; no validation is provided that image and text editors implement matching semantics.
  • domain assumption Off-the-shelf editors can stand in for the in-modality operations of any-to-any models.
    Section 2 states that editors are used because any-to-any models are not optimized for editing, which injects external model behavior into the consistency measurement.
  • domain assumption The binary VQA classifier h_theta and oracle labels h_o faithfully measure factual similarity.
    Section 4.2 Eq. (7) defines sim in terms of h_theta and h_o; PaliGemma2 provides h_theta, and its errors or biases propagate directly into every metric.
  • domain assumption ACON annotations are representative of cross-modal consistency beyond natural photos.
    Section 3 and Section 7: only 500 of roughly 2,000 private submissions were retained, 43% of annotations were discarded, annotators share a cultural background, and the distribution excludes drawings, 3D renders, and OCR-heavy images.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?." pith.science (2026). https://pith.science/paper/HWOQFDDF

@misc{pith2026250524211,
  author       = {Pith},
  title        = {Pith review of: Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HWOQFDDF}},
  note         = {Machine review of arXiv:2505.24211}
}
read the original abstract

Any-to-any generative models aim to enable seamless interpretation and generation across multiple modalities within a unified framework, yet their ability to preserve relationships across modalities remains uncertain. Do unified models truly achieve cross-modal coherence, or is this coherence merely perceived? To explore this, we introduce ACON, a dataset of 1,000 images (500 newly contributed) paired with captions, editing instructions, and Q&A pairs to evaluate cross-modal transfers rigorously. Using three consistency criteria-cyclic consistency, forward equivariance, and conjugated equivariance-our experiments reveal that any-to-any models do not consistently demonstrate greater cross-modal consistency than specialized models in pointwise evaluations such as cyclic consistency. However, equivariance evaluations uncover weak but observable consistency through structured analyses of the intermediate latent space enabled by multiple editing operations. We release our code and data at https://github.com/JiwanChung/ACON.

Figures

Figures reproduced from arXiv: 2505.24211 by the authors.

Figure 1
Figure 1. We examine the consistency of any-to-any [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We evaluate whether any-to-any modality conversion models demonstrate greater consistency compared [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Data annotation process for ACON. Three human workers perform distinct roles: the teller cre￾ates a textual description emphasizing key elements for reconstruction, following a communication game frame￾work (Kim et al., 2019), the drawer recreates the image using the description via multi-turn AI interactions, and the comparer generates Q&As to capture similarities and differences between the original and reconstruc… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Accuracy of conjugated equivariance across model pairs, normalized to [0, 1] per row. Diagonal components with red borders indicate the same any-to￾any generator used for both image-to-text and text-to￾image transfers. the cyclic consistency experiment. The only dif￾fe…
Figure 6
Figure 6. Figure 6: Example inference results using VILA-U ( [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 24 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Amith Ananthram, Elias Stengel-Eskin, Carl Vondrick, Mohit Bansal, and Kathleen McKeown. 2024. See it from my perspective: Diagnosing the western cultural bias of large vision-language models in image understanding. arXiv preprint arXiv:2406.11665

    arXiv 2024

  4. [4]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 1(2):3

    arXiv 2023

  5. [5]

    Brinnae Bent. 2024. Semantic approach to quantifying the consistency of diffusion model image generation. arXiv preprint arXiv:2404.08799

    work page Pith review arXiv 2024

  6. [6]

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. 2023. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8

    2023

  7. [7]

    Romain Bielawski and Rufin VanRullen. 2023. Clip-based image captioning via unsupervised cycle-consistency in the latent space. In 8th Workshop on Representation Learning for NLP (RepL4NLP 2023), pages 266--275. Association for Computational Linguistics

    work page 2023

  8. [8]

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Doll \'a r, and C Lawrence Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325

    arXiv 2015

Show all 51 references
  1. [9]

    Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. 2024. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06135

    2024 arXiv

  2. [10]

    Jaemin Cho, Yushi Hu, Jason Michael Baldridge, Roopal Garg, Peter Anderson, Ranjay Krishna, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. 2024. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. In The Twelfth International C...

    2024

  3. [11]

    Shih-Han Chou, Shivam Chandhok, James J Little, and Leonid Sigal. 2024. MM-R ^3 : On (in-) consistency of multi-modal large language models (mllms). arXiv preprint arXiv:2410.04778

    2024 arXiv

  4. [12]

    Taco Cohen and Max Welling. 2016. Group equivariant convolutional networks. In International conference on machine learning, pages 2990--2999. PMLR

    2016

  5. [13]

    Maurice Diesendruck, Jianzhe Lin, Shima Imani, Gayathri Mahalingam, Mingyang Xu, and Jie Zhao. 2024. Learning how to ask: Cycle-consistency refines prompts in multimodal foundation models. arXiv preprint arXiv:2402.08756

    2024 arXiv

  6. [14]

    Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. 2024 a . https://openreview.net/forum?id=0Nui91LBQS Making LL a MA SEE and draw with SEED tokenizer . In The Twelfth International Conference on Learning Representations

    2024

  7. [15]

    Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. 2024 b . Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396

    2024 arXiv

  8. [16]

    Google DeepMind . 2023. https://deepmind.google/technologies/imagen-2/ Imagen 2

    2023

  9. [17]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    2021 arXiv

  10. [18]

    Yu-Guan Hsieh, Cheng-Yu Hsieh, Shih-Ying Yeh, Louis B \'e thune, Hadi Pour Ansari, Pavan Kumar Anasosalu Vasu, Chun-Liang Li, Ranjay Krishna, Oncel Tuzel, and Marco Cuturi. 2024. Graph-based captioning: Enhancing visual descriptions by interconnecting region captions. arXiv pr...

    2024 arXiv

  11. [19]

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. 2023. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, ...

    2023

  12. [20]

    Yu Huang, Chenzhuang Du, Zihui Xue, Xuanyao Chen, Hang Zhao, and Longbo Huang. 2021. What makes multi-modal learning better than single (provably). Advances in Neural Information Processing Systems, 34:10944--10956

    2021

  13. [21]

    Yang Jin, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Bin Chen, Chenyi Lei, An Liu, Chengru Song, Xiaoqiang Lei, et al. 2023. Unified language-vision pretraining with dynamic discrete visual tokenization. arXiv preprint arXiv:2309.04669

    2023

  14. [22]

    Jin-Hwa Kim, Nikita Kitaev, Xinlei Chen, Marcus Rohrbach, Byoung-Tak Zhang, Yuandong Tian, Dhruv Batra, and Devi Parikh. 2019. Codraw: Collaborative drawing as a testbed for grounded goal-driven communication. In Proceedings of the 57th Annual Meeting of the Association for Co...

    2019

  15. [23]

    Black Forest Labs. 2024. Flux.1: Advanced text-to-image generation. https://blackforestlabs.ai/flux-1/

    2024

  16. [24]

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. 2024. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, pages 366--384. Springer

    2024

  17. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  18. [26]

    Jiasen Lu, Christopher Clark, Sangho Lee, Zichen Zhang, Savya Khosla, Ryan Marten, Derek Hoiem, and Aniruddha Kembhavi. 2024. Unified-io 2: Scaling autoregressive multimodal models with vision language audio and action. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  19. [27]

    Pengqi Lu. 2024. https://github.com/erwold/qwen2vl-flux Qwen2vl-flux: Unifying image and text guidance for controllable image generation

    2024

  20. [28]

    Zhou Lu. 2023. A theory of multimodal learning. Advances in Neural Information Processing Systems, 36:57244--57255

    2023

  21. [29]

    Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, and Ranjay Krishna. 2023. Crepe: Can vision-language foundation models reason compositionally? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10910--10921

    2023

  22. [30]

    Richard E Nisbett, Kaiping Peng, Incheol Choi, and Ara Norenzayan. 2001. Culture and systems of thought: holistic versus analytic cognition. Psychological review, 108(2):291

    2001

  23. [31]

    OpenAI. 2023. Gpt-4: Openai’s large multimodal model. https://openai.com/research/gpt-4

    2023

  24. [32]

    Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, and Furu Wei. 2024. https://openreview.net/forum?id=he6mX9LTyE Kosmos-g: Generating images in context with multimodal large language models . In The Twelfth International Conference on Learning Representations

    2024

  25. [33]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M \"u ller, Joe Penna, and Robin Rombach. 2024. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations

    2024

  26. [34]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  27. [35]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. 2022. https://arxiv.org/abs/2112.10752 High-resolution image synthesis with latent diffusion models . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVP...

    2022 arXiv

  28. [36]

    Andreas Steiner, Andr \'e Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, et al. 2024. Paligemma 2: A family of versatile vlms for transfer. arXiv preprint arXiv:2412.03555

    2024 arXiv

  29. [37]

    Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. 2024 a . Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2024

  30. [38]

    Yichen Sun, Zhixuan Chu, Zhan Qin, and Kui Ren. 2024 b . Prompt-consistency image generation (pcig): A unified framework integrating llms, knowledge graphs, and controllable diffusion models. arXiv preprint arXiv:2406.16333

    2024 arXiv

  31. [39]

    Zhaorui Tan, Xi Yang, Zihan Ye, Qiufeng Wang, Yuyao Yan, Anh Nguyen, and Kaizhu Huang. 2022. Ssd: Towards better text-image consistency metric in text-to-image generation. arXiv preprint arXiv:2210.15235

    2022 arXiv

  32. [40]

    Chameleon Team. 2024. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818

    2024 arXiv

  33. [41]

    Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, and Yuval Atzmon. 2024. Training-free consistent text-to-image generation. ACM Transactions on Graphics (TOG), 43(4):1--18

    2024

  34. [42]

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. 2024. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568--9578

    2024

  35. [43]

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural discrete representation learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 30

    2017

  36. [44]

    Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International conference on machine le...

    2022

  37. [45]

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. 2024. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869

    2024 arXiv

  38. [46]

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. 2024. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429

    2024 arXiv

  39. [47]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  40. [48]

    Jiawei Zhang, Tianyu Pang, Chao Du, Yi Ren, Bo Li, and Min Lin. 2024 a . Benchmarking large multimodal models against common corruptions. arXiv preprint arXiv:2401.11943

    2024 arXiv

  41. [49]

    Junzhe Zhang, Huixuan Zhang, Xunjian Yin, Baizhou Huang, Xu Zhang, Xinyu Hu, and Xiaojun Wan. 2024 b . Mc-mke: A fine-grained multimodal knowledge editing benchmark emphasizing modality consistency. arXiv preprint arXiv:2406.13219

    2024 arXiv

  42. [50]

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. 2024. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039

    2024 arXiv

  43. [51]

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223--2232

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.