Pith. sign in

REVIEW 4 major objections 4 minor 14 cited by

Ovis-U1 Technical Report

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 3-billion-parameter model starting from a language model claims that jointly training on understanding, text-to-image generation, and image editing improves all three tasks, with benchmarks placing it ahead of comparable 2-3B…

desk verdict Ovis-U1 is a credible, open-sourced 3B unified model, but the paper's marquee claim—that unified training improves understanding—rests on a comparison that is not actually controlled. read the letter →

arxiv 2506.23044 v2 pith:ZG6OA227 submitted 2025-06-29 cs.CV cs.AI

Guo-Hua Wang , Shanshan Zhao , Xinjie Zhang , Liangfu Cao , Pengxin Zhan , Lunhao Duan , Shiyin Lu , Minghao Fu
show 4 more authors
Xiaohao Chen Jianshan Zhao Yang Li Qing-Guo Chen
This is my paper
classification cs.CVcs.AI
keywords Ovis-U1unifiedmultimodalmodeltext-to-imagegenerationimageeditingunderstandingdiffusiontransformerbidirectionaltokenrefinertraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ovis-U1 is a 3-billion-parameter model that claims to handle three jobs at once: understanding images, generating images from text, and editing images from instructions. The report's central claim is that unified training—training one model jointly on all three tasks—improves performance on each one instead of sacrificing one for another. It reports an average of 69.6 on the OpenCompass multimodal academic benchmark, ahead of comparable small understanding models, plus strong text-to-image and editing scores. The design starts from a pretrained language model rather than a frozen multimodal model, adding a diffusion-based visual decoder and a bidirectional token refiner to connect text and image embeddings. If the claims hold, a small unified model can approach or beat task-specific systems while doing all three jobs at once.

What carries the argument

The load-bearing mechanism is a diffusion-based visual decoder paired with a bidirectional token refiner. The decoder is a 1B-parameter multimodal diffusion Transformer (MMDiT) with rotary position embeddings, trained with flow matching and a frozen SDXL VAE, and it receives both a semantic condition (the LLM's embedding of the prompt, concatenated with a visual semantic embedding) and detailed visual context tokens from the VAE encoder. The refiner, two Transformer blocks with modulation, fuses features from the last two layers of the LLM and a learnable [CLS] token so the model captures global information without CLIP. The six-stage training pipeline—decoder pretraining, adapter pretraining, visual encoder alignment, understanding learning, generation learning, and generation fine-tuning—repeatedly re-aligns the visual encoder with both understanding and generation data, which is how the authors explain the mutual improvement.

What would settle it

Run Ovis-U1, Ristretto-3B, and SAIL-VL-1.5-2B through one shared evaluation harness with identical prompts and decoding settings; if the average gap shrinks to noise or reverses, the 'surpassing' claim fails. Similarly, train the same architecture without generation and editing data and compare final understanding scores: no gain would falsify the unified-training claim.

Watch

Extended reading notes

Core claim

The paper claims that a 3B model initialized from a plain language model (Qwen3-1.7B), rather than from an existing frozen MLLM, can learn multimodal understanding, text-to-image generation, and image editing in a single six-stage training run, and that the joint training helps each task. The authors report Ovis-U1 scores 69.6 average on the eight-benchmark OpenCompass academic suite, higher than Ristretto-3B and SAIL-VL-1.5-2B; 0.89 on GenEval and 83.72 on DPG-Bench for generation; and 4.00 on ImgEdit-Bench and 6.42 on GEdit-Bench-EN for editing. Their ablations assert that adding generation and editing tasks during visual-encoder alignment lifts understanding by about a point over the understanding-only baseline, and that adding editing data lifts generation scores. In short, the discovery is that understanding and generation reinforce each other when trained together in one model.

Load-bearing premise

The headline comparisons assume benchmark numbers from different papers, leaderboards, and the authors' own runs are directly comparable even though evaluation protocols, prompt templates, and codebases differ, and that ablations run on earlier models trained on far less data transfer to the final model.

Editorial extensions

If this is right

  • If unified training genuinely helps both understanding and generation, other multimodal models can expect to improve perception by adding generation-style objectives during vision-language alignment.
  • A 3B model with a 1B decoder can reach near-GPT-4o levels on several image-editing subtasks, suggesting decoder capacity is not the main bottleneck at this scale.
  • A raw-prompt GenEval score of 0.89 indicates that prompt rewriting is not necessary for strong attribute binding in a small unified model.
  • The architecture replaces CLIP with a learnable [CLS] token plus LLM features, so future text-to-image systems may drop extra frozen encoders entirely.
  • Open-sourcing the model makes the stated benchmark numbers independently reproducible under a shared evaluation harness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the unified-training gain is real, it implies that generation data acts as an auxiliary alignment signal for the visual encoder, so understanding gains may grow as generation data quality and diversity scale; the paper does not isolate this mechanism directly.
  • The refiner ablations were run on earlier model versions with roughly 10M to 50M training images, so a testable extension is to rerun them at full data scale to see whether the CLS-token clip-free design overtakes T5+CLIP, as the paper hints.
  • The benchmark comparisons draw from several sources with different protocols; a single-harness rerun could shift the exact margins, though the model's release makes such a check straightforward.
  • The classifier-free guidance sweep shows optimal settings differ by benchmark, so downstream users may need per-task CFG tuning rather than one global setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The report introduces Ovis-U1, a 3.6B-parameter unified multimodal model built on Qwen3-1.7B, an Aimv2 visual encoder, a diffusion-based visual decoder, and a bidirectional token refiner. The authors describe a six-stage training pipeline that interleaves multimodal understanding, text-to-image generation, and image editing, and they report strong benchmark numbers: 69.6 average on OpenCompass multimodal academic benchmarks, 83.72 on DPG-Bench, 0.89 on GenEval, 4.00 on ImgEdit-Bench, and 6.42 on GEdit-Bench-EN. The paper's central scientific claim is that unified training of understanding and generation collaboratively improves both capabilities, supported by ablation tables and qualitative examples.

Significance. If the reported results hold, Ovis-U1 would be a meaningful data point for compact unified multimodal models: it shows that a 3B-scale model with a diffusion decoder can approach GPT-4o on several generation and editing benchmarks while retaining competitive understanding performance. The paper's strengths include a clearly specified architecture, a detailed six-stage training schedule, explicit use of raw GenEval prompts rather than rewritten prompts, and release of model weights and code, which make the benchmark numbers independently checkable. The claimed synergy between understanding and generation is the most scientifically interesting contribution, but the evidence for it is currently incomplete: the key comparison is not a controlled ablation, and several supporting ablations were run on earlier, smaller-scale model versions. The benchmark comparisons themselves would also be stronger with a single shared evaluation harness.

major comments (4)
  1. [§5.3, Table 10] The central claim that unified training enhances understanding is not supported by a controlled comparison. The baseline is described only as 'Ovis without the unified training'; the report does not state that the baseline uses the same LLM, visual encoder, adapter, decoder, refiner, data schedule, and training stages as Ovis-U1 with only the generation/editing objectives removed. This matters because Table 4 lists Ovis2-2B at 65.2 average, which is higher than the 64.47 'Unified Training' row, so the baseline row in Table 10 cannot simply be Ovis2. Without a same-architecture, same-data ablation, the 1.14-point improvement could be caused by architecture changes, data quality, training stages, or captions rather than by task synergy. Please provide a controlled ablation on the final Ovis-U1 configuration, or explicitly state which components differ between the baseline and the unified model.
  2. [§5.2 and §5.4, Tables 9, 11, 12] The ablation results for the refiner design and the stage-wise generation improvements were obtained on earlier model versions trained with roughly 10M to 50M images, while the final model is trained through Stages 4 and 5 with about 600K decoder fine-tuning steps at batch size 256. The report asserts that these conclusions transfer to the final model, but no evidence is given that the design choices and stage-wise gains hold at the final data scale. In particular, Table 9 shows the clip-free CLS-token variant underperforming the T5+CLIP baseline at both 10M and 50M data, yet the final model adopts the clip-free design; the role of scale in reversing this ordering is stated as a possibility, not demonstrated. Please either report ablations at the final data scale or clearly mark these tables as preliminary design studies that do not directly support the final configuration.
  3. [§5.1, Table 3] The headline comparisons across models are not made under a single evaluation harness. Table 3 draws numbers from OmniGen2's paper, external leaderboards, and the authors' own tests, and the only independent check reported is a re-run of OmniGen2, whose average (58.5) differs from the published value (61.5) by 3.0 points. This protocol sensitivity is large relative to the reported differences between Ovis-U1 and the compared understanding models, so the claim of surpassing Ristretto-3B and SAIL-VL-1.5-2B does not follow from the numbers as presented. The paper should either re-evaluate all comparison models with the same prompts and codebase or restrict the SOTA claim to benchmark families for which identical protocols are verified.
  4. [§5.5, Tables 13 and 14] The image-editing scores in Table 3 are not reconciled with the CFG tuning experiments. The report states that the same CFG settings are applied across benchmarks in Table 3, but the final scores (4.00 and 6.42) do not match any row in Tables 13 and 14 (e.g., CFG_img=4, CFG_txt=7.5 gives 4.09 and 6.351 on the ablation model). Since the editing hyperparameters were selected by evaluating on ImgEdit-Bench and GEdit-Bench, and since the final model differs from the ablation model, the reader cannot tell which configuration produced the headline numbers or whether the headline numbers are the result of benchmark-specific tuning. Please report the exact CFG settings and model version used for Table 3, and provide the Table 3 editing scores on the final model.
minor comments (4)
  1. [§2] There are several typos in the architecture section: 'insparied' should be 'inspired', and 'Oivs-U1' should be 'Ovis-U1'; these should be corrected.
  2. [Table 1 and Table 3] The parameter counts are inconsistent across the paper: Table 1 sums to 3644M, Table 3 lists '2.4B + 1.2B', the abstract says 3B, and §5.1 says 3.34B; the report should state one canonical parameter count and explain the breakdown.
  3. [Table 9] The final model uses a clip-free CLS-token refiner, yet at both 10M and 50M data the clip-free variants do not beat the T5+CLIP baseline on CLIPScore and are mixed on DPG-Bench; a short explanation of what changed between these ablations and the final model would help the reader interpret the design choice.
  4. [§5.2 and §5.4] The repeated italicized disclaimer that ablations were run on earlier model versions is easy to miss; please move this caveat to the main text of each ablation subsection, not only in the table captions, so that readers do not overgeneralize the stage-wise gains to the released model.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: Ovis-U1's benchmark results are direct measurements, and the collaborative-training claim rests on under-specified ablations rather than on a definitional or self-citation chain.

full rationale

Ovis-U1 is an empirical systems report. Its headline numbers are measured benchmark scores (OpenCompass 69.6, DPG-Bench 83.72, GenEval 0.89, ImgEdit 4.00, GEdit-EN 6.42), not quantities derived from fitted constants, so there is no derivation chain that could collapse into its inputs. The central claim that unified training collaboratively enhances both understanding and generation performance is supported by ablations, but those ablations are not definitionally circular: Table 10 compares against a baseline described only as 'Ovis without the unified training,' which is under-specified and may differ in architecture and data, and Tables 11-14 are explicitly run on earlier model versions with smaller training data. These are uncontrolled-comparison and evidence-quality problems, not reductions of the claim to its own inputs by construction. The CFG settings for image editing were explored on ImgEdit and GEdit benchmarks (Tables 13-14), which is a mild selection-bias concern, but the main table uses the same CFG across benchmarks and the scores are reported as measurements, not as fitted predictions. Self-citations to the Ovis paper provide the starting architecture and training-stage details, but no load-bearing step depends on an unverified self-cited uniqueness theorem or ansatz smuggled in by citation. Accordingly, no circular step meeting the quoted-evidence bar is present; the score of 2 reflects only minor, non-load-bearing self-citation and weak ablation controls, not substantive circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central benchmark results rest on standard machine-learning background (flow matching, diffusion transformers, pretrained LLMs and VAEs) rather than on new mathematical postulates. The main assumptions are domain-level: that the adopted backbones work as intended in the unified setting, that benchmark scores are comparable across papers, and that the early-version ablations transfer to the final model. The paper tunes only a handful of hyperparameters, notably CFG guidance scales, and filters data with hand-chosen thresholds.

free parameters (3)
  • CFG_img / CFG_txt guidance scales = CFG_img=2, CFG_txt=7.5 (ImgEdit best); fixed settings used for final Table 3
    Chosen based on ImgEdit-Bench and GEdit-Bench-EN results in Tables 13-14, i.e., tuned on the evaluation benchmarks.
  • Laion aesthetic score threshold = 6
    Hand-chosen filter threshold in Section 3.1 for constructing Laion-aes6; affects generation training data quality.
  • Data mixture ratios per stage = not disclosed
    The ratio of understanding, generation, and editing data in each stage is chosen ad hoc and not reported, so the effect of each task on the final model cannot be isolated.
assumptions (4)
  • domain assumption Flow matching with an MMDiT decoder is an appropriate visual decoder for unified multimodal training.
    Adopted from FLUX and FLUX.1 Kontext (Labs, 2024a; Labs et al., 2025); the paper's only validation is on earlier model versions (Sections 5.2 and 5.4).
  • domain assumption Benchmark scores across different papers and evaluation harnesses are directly comparable.
    Section 5.1 aggregates results from OmniGen2, leaderboards, and the authors' own runs without a single shared evaluation protocol.
  • domain assumption The frozen SDXL VAE is sufficient for encoding and decoding images in the unified model.
    Section 2 freezes the 84M-parameter VAE and relies on its pretrained quality.
  • ad hoc to paper Ablations performed on early versions of the model transfer to the final released model.
    Sections 5.2, 5.4, and 5.5 state ablations used 'earlier versions' and different data scales, yet the paper uses them to justify final design choices.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ovis-U1 Technical Report." pith.science (2026). https://pith.science/paper/ZG6OA227

@misc{pith2026250623044,
  author       = {Pith},
  title        = {Pith review of: Ovis-U1 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZG6OA227}},
  note         = {Machine review of arXiv:2506.23044}
}
read the original abstract

In this report, we introduce Ovis-U1, a 3-billion-parameter unified model that integrates multimodal understanding, text-to-image generation, and image editing capabilities. Building on the foundation of the Ovis series, Ovis-U1 incorporates a diffusion-based visual decoder paired with a bidirectional token refiner, enabling image generation tasks comparable to leading models like GPT-4o. Unlike some previous models that use a frozen MLLM for generation tasks, Ovis-U1 utilizes a new unified training approach starting from a language model. Compared to training solely on understanding or generation tasks, unified training yields better performance, demonstrating the enhancement achieved by integrating these two tasks. Ovis-U1 achieves a score of 69.6 on the OpenCompass Multi-modal Academic Benchmark, surpassing recent state-of-the-art models such as Ristretto-3B and SAIL-VL-1.5-2B. In text-to-image generation, it excels with scores of 83.72 and 0.89 on the DPG-Bench and GenEval benchmarks, respectively. For image editing, it achieves 4.00 and 6.42 on the ImgEdit-Bench and GEdit-Bench-EN, respectively. As the initial version of the Ovis unified model series, Ovis-U1 pushes the boundaries of multimodal understanding, generation, and editing.

Figures

Figures reproduced from arXiv: 2506.23044 by the authors.

Figure 1
Figure 1. Comprehensive illustration of the functional capabilities of Ovis-U1. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall architecture of Ovis-U1. (a) The Ovis-U1 model integrates both textual and visual [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the proposed six-stage training pipeline. We progressively train the Ovis-U1 model [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative results of different classifier free guidance on image editing. The instruction of [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: More qualitative results from Ovis-U1 on text-to-image generation. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: More qualitative results from Ovis-U1 on image editing. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: More qualitative results from Ovis-U1 on image understanding. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 14 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UniGeo: Unifying Geometric Guidance for Camera-Controllable Image Editing via Video Models

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    UniGeo unifies geometric guidance across three levels in video models to reduce geometric drift and improve consistency in camera-controllable image editing.

  2. Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Multimodal pretraining transfers asymmetrically: language boosts vision, understanding boosts generation, generation is mostly neutral, and early unified training prevents vision laziness.

  3. SciForma: Structure-Faithful Generation of Scientific Diagrams

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A 9B open-weights model trained with axis-decomposed conjunctive preference optimization (M-DPO) and a structural inventory beats GPT-Image-1.5 on scientific-diagram structural-fidelity benchmarks.

  4. Is This Edit Correct? A Multi-Dimensional Benchmark for Reasoning-Aware Image Editing

    cs.HC 2026-04 conditional novelty 6.0 of 10

    SOTA diffusion image editors score poorly on implicit physical, environmental, cultural, causal, and referential constraints, which a lightweight reasoning-guided post-edit can partially fix.

  5. Learning Preference-Based Objectives from Clinical Narratives for Dynamic Sepsis Treatment

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    CN-PR learns reward functions from LLM-derived preferences over clinical trajectories to improve RL policies for sequential treatment decisions, showing correlation with quality scores and better recovery outcomes.

  6. TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking

    cs.CV 2026-04 conditional novelty 6.0 of 10

    TRACE anchors a video-diffusion editor to 3D meshes to perform consistent part-level edits on 3D Gaussian scenes in about 10 minutes per edit.

  7. UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A six-level capability taxonomy plus UniICL-760K and a lightweight CAPM module improve unified multimodal few-shot learning and beat larger MLLMs on most understanding ICL tasks.

  8. Reconstruction Alignment Improves Unified Multimodal Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    RECA, a self-supervised post-training objective that conditions unified multimodal models on their own visual understanding embeddings to reconstruct input images, improves text-to-image and editing benchmarks across ...

  9. X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Again

    cs.CV 2025-07 conditional novelty 6.0 of 10

    GRPO reinforcement learning applied to a discrete autoregressive image generator with a diffusion decoder improves instruction following, image quality, and long-text rendering in a unified multimodal model.

  10. Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Codec-guided sparse patch selection plus a lightweight speak/silent gate yields a 4B streaming VLM that is competitive on static tasks, stronger on video/spatial benchmarks, and much cheaper at inference.

  11. Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A compact 4B image generation/editing system with a fast one-step VAE, native-resolution packing, RL alignment, and 4-step distillation reports competitive benchmarks against 6B–80B open models.

  12. Few-Shot Synthetic Image Attribution: Identifying Unseen Generators with Limited Samples

    cs.CV 2025-09 conditional novelty 5.0 of 10

    OmniDFA performs few-shot, open-set attribution of AI-generated images, identifying the source generator from just five support samples across 45 known and unseen generators.

  13. Instruction-based Image Editing: A Survey on Data, Models, Evaluation, and Applications

    cs.CV 2026-07 conditional novelty 4.0 of 10

    A survey of instruction-based image editing plus a new 21-task benchmark, CDD-IIE, on which ten open models are scored by human experts.

  14. Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget

    cs.CV 2026-07 conditional novelty 4.0 of 10

    An open-source image-generation family shows that agentic prompt rewriting and a stronger text encoder can lift quality to near closed-source levels with only 208.62M images and about $400K of training compute.

Reference graph

Works this paper leans on

26 extracted references · 1 canonical work pages · cited by 14 Pith papers

  1. [1]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923,

    arXiv

  2. [6]

    Ming-Lite-Uni: Advancements in unified architecture for natural multimodal interaction

    Biao Gong, Cheng Zou, Dandan Zheng, Hu Yu, Jingdong Chen, Jianxin Sun, Junbo Zhao, Jun Zhou, Kaixiang Ji, Lixiang Ru, et al. Ming-Lite-Uni: Advancements in unified architecture for natural multimodal interaction. arXiv preprint arXiv:2505.02471,

    arXiv

  3. [10]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603,

    arXiv

  4. [11]

    Generating multi-image synthetic data for text-to-image customization

    Nupur Kumari, Xi Yin, Jun-Yan Zhu, Ishan Misra, and Samaneh Azadi. Generating multi-image synthetic data for text-to-image customization. arXiv preprint arXiv:2502.01720,

    arXiv

  5. [12]

    Black Forest Labs. FLUX. https://github.com/black-forest-labs/flux, 2024a. Black Forest Labs. Introducing FLUX.1 tools. https://bfl.ai/announcements/24-11-21-tools , 2024b. Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, et al. FLUX. 1 Kontext: F...

    arXiv

  6. [13]

    UniWorld: High-resolution semantic encoders for unified visual understanding and generation

    Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, et al. UniWorld: High-resolution semantic encoders for unified visual understanding and generation. arXiv preprint arXiv:2506.03147,

    arXiv

  7. [14]

    Step1X-Edit: A practical framework for general image editing

    Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Step1X-Edit: A practical framework for general image editing. arXiv preprint arXiv:2504.17761,

    arXiv

  8. [15]

    MMbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. MMbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp. 216–233. Springer, 2024a. Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Li...

    arXiv

Show all 26 references
  1. [16]

    Ovis: Structural embedding alignment for multimodal large language model

    15 Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural embedding alignment for multimodal large language model. arXiv preprint arXiv:2405.20797,

    arXiv

  2. [17]

    Exploring the role of large language models in prompt encoding for diffusion models

    Bingqi Ma, Zhuofan Zong, Guanglu Song, Hongsheng Li, and Yu Liu. Exploring the role of large language models in prompt encoding for diffusion models. arXiv preprint arXiv:2406.11831,

    arXiv

  3. [19]

    Ominicontrol: Minimal and universal control for diffusion transformer

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15098,

    arXiv

  4. [20]

    Qwen2-VL: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-VL: Enhancing vision-language model’s pe...

    arXiv

  5. [21]

    OmniGen2: Exploration to advanced multimodal generation

    Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, et al. OmniGen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506.18871, 2025a. Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fe...

    arXiv

  6. [22]

    Qwen2.5 technical report

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    arXiv

  7. [23]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388,

    arXiv

  8. [24]

    Imgedit: A unified image editing dataset and benchmark

    Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. Imgedit: A unified image editing dataset and benchmark. arXiv preprint arXiv:2505.20275,

    arXiv

  9. [25]

    Unified multimodal understanding and generation models: Advances, challenges, and opportunities

    Xinjie Zhang, Jintao Guo, Shanshan Zhao, Minghao Fu, Lunhao Duan, Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, Weihua Luo, and Kaifu Zhang. Unified multimodal understanding and generation models: Advances, challenges, and opportunities. arXiv preprint arXiv:2505.02567, 2025a. Zechuan...

  10. [26]

    InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479,

    arXiv

  11. [2016]

    Seed-data-edit technical report: A hybrid dataset for instructional image editing

    14 Yuying Ge, Sijie Zhao, Chen Li, Yixiao Ge, and Ying Shan. Seed-data-edit technical report: A hybrid dataset for instructional image editing. arXiv preprint arXiv:2405.04007,

    arXiv

  12. [2018]

    Unicontrol: A unified diffusion model for controllable visual generation in the wild

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffusion model for controllable visual generation in the wild. arXiv preprint arXiv:2305.11147,

    arXiv

  13. [2020]

    Stylebooth: Image style editing with multimodal instruction

    Zhen Han, Chaojie Mao, Zeyinzi Jiang, Yulin Pan, and Jingfeng Zhang. Stylebooth: Image style editing with multimodal instruction. arXiv preprint arXiv:2404.12154,

    arXiv

  14. [2021]

    EllA: Equip diffusion models with LLM for enhanced semantic alignment

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. EllA: Equip diffusion models with LLM for enhanced semantic alignment. arXiv preprint arXiv:2403.05135,

    arXiv

  15. [2022]

    Blip3-o: A family of fully open unified multimodal models- architecture, training and dataset

    Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models- architecture, training and dataset. arXiv preprint arXiv:2505.09568, 2025a. Lin Chen,...

    arXiv

  16. [2023]

    Emerging properties in unified multimodal pretraining

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14683,

    arXiv

  17. [2024]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pp. 235–251. Springer,

    2016

  18. [2025]

    Scalable vision language model training via high quality data curation

    Hongyuan Dong, Zijian Kang, Weijie Yin, Xiao Liang, Chao Feng, and Jiao Ran. Scalable vision language model training via high quality data curation. arXiv preprint arXiv:2501.05952,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.