Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation

Chenyang Lyu; Chris Biemann; Jingheng Pan; Kaifu Zhang; Linlong Xu; Longyue Wang; Minghao Wu; Weihua Luo; Xiaohu Zhao; Xintong Wang

REVIEW 5 major objections 6 minor 42 references

Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning vision-language models on a single high-resource language pair hurts their other languages; a balanced multilingual sample of about 1,000 examples per direction fixes the loss.

desk verdict The dataset and prompt-style findings are worth your time; the DA Score and balanced fine-tuning claim need another pass. read the letter →

arxiv 2506.11820 v1 pith:524TYMLL submitted 2025-06-13 cs.CV cs.CL

Xintong Wang , Jingheng Pan , Yixiao Liu , Xiaohu Zhao , Chenyang Lyu , Minghao Wu , Chris Biemann , Longyue Wang

show 3 more authors

Linlong Xu Weihua Luo Kaifu Zhang

This is my paper

classification cs.CVcs.CL

keywords vision-languagetranslationlargemodelsmultilingualevaluationAibTransdatasetOCRcorrectiondensity-awarefine-tuningcross-lingualgeneralization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish three things about vision-language translation (VLT), the task of reading text inside images and translating it with visual context: the usual datasets are unreliable, the usual way of averaging automatic metrics misreports model quality, and fine-tuning on one high-resource language pair damages cross-lingual ability. To fix the data problem the authors build AibTrans, a parallel human-verified dataset with corrected OCR and culturally grounded translations. To fix the metric problem they propose the Density-Aware (DA) Score, which reweights BLEU, CHRF++, BERTScore, and COMET according to how well each correlates with human ratings under low, medium, and high information density. Their central adaptation claim is that a balanced multilingual sample of about 1,000 examples per direction is enough to adapt a vision-language model to VLT without eroding its multilingual generalization. If right, small curated multilingual data and density-aware scoring would make VLT benchmarking fairer and cheaper.

What carries the argument

Three mechanisms carry the argument. First, AibTrans: a parallel corpus of human-verified, OCR-corrected translation instances (the benchmark split is reported as 100 images and 6,993 sentence-level instances, covering Chinese into English, German, Spanish, Arabic, Russian, Japanese, and Hindi), built by manually correcting PaddleOCR output and having professional translators ground each translation in the image. Second, the information-density taxonomy: samples are clustered by bounding-box count and source token length into low, medium, and high density, and metric-human Pearson correlations are computed within each tier. Third, the Density-Aware Score, $$\text{DA Score} = \$\alpha$\cdot\text{BLEU} + \$\beta$\cdot\text{CHRF++} + \$\lambda$\cdot\text{BERTScore} + \phi\cdot\text{COMET},$$ with weights set to those per-tier correlations, replacing the naive arithmetic mean. The balanced multilingual fine-tuning strategy—sampling 500–1,000 examples per direction from MIT-10M across 8 to 13 translation directions and applying LoRA—is the mechanism that preserves multilingual competence while adapting to the task.

What would settle it

Recompute the metric-human correlations from human ratings of a different model family on the same three datasets: if the per-density weights that best track human judgment differ materially from the reported values, or if ranking models by the fixed DA Score disagrees with ranking by direct human preference on a held-out language, the central evaluation claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that cross-lingual generalization in large vision-language models is fragile under naive task adaptation: supervised fine-tuning on one high-resource pair, English-to-Chinese, consistently degrades performance across high- and low-resource directions, while sampling 500–1,000 examples per direction from a balanced multilingual corpus improves task alignment and overall scores, with diminishing returns beyond 1,000 examples. The same study finds that current VLT benchmarks are themselves unreliable, because machine-generated references are semantically and culturally shallow and because OCR errors propagate into reference translations; AibTrans is introduced as a corrected, human-verified parallel testbed. On evaluation, no single automatic metric agrees with human judgment across all input densities: BLEU is best on short low-density text, CHRF++ and COMET on denser text, and BERTScore is unstable at medium density. The DA Score therefore reweights the four metrics by density tier and, the paper reports, tracks human scores better than their arithmetic mean. Under this protocol, cascaded models with strong multilingual LLMs are competitive with end-to-end LVLMs, and the open-source Qwen2.5-VL-7B approaches commercial performance.

Load-bearing premise

The load-bearing premise is that the human-rating correlations measured on GPT-4o outputs on MIT-10M, OCRMT30K, and MTIT6 transfer to every other model and language, and that the density categories derived from K-means are valid; if those weights do not generalize, the DA Score and the fine-tuning conclusions drawn with it are unsupported.

Editorial extensions

If this is right

Fine-tuning an LVLM on a single high-resource language pair (English–Chinese) is actively harmful to other directions; a balanced multilingual sample of about 1,000 examples per direction instead improves average performance.
The DA Score, not the arithmetic mean of BLEU, BERTScore, and COMET, should be used when comparing VLT models, because simple averaging underrepresents low-quality translations and hides the medium-density failure zone.
Performance reported as one averaged number across languages can mask weak low-resource performance, so language-resource-tier reporting is needed in multilingual VLT evaluation.
Strict, heavily constrained prompts can degrade reasoning-based VLT models, while collaborative instructions and anti-hallucination constraints improve them.
External OCR remains necessary for most open LVLMs; Qwen2.5-VL-7B is the exception that performs near its OCR-supplemented ceiling without auxiliary OCR input.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The DA Score weights come from human ratings of GPT-4o outputs; a natural extension is to recompute them per model family, since models with different error patterns may need different metric weightings.
AibTrans's OCR-error statistics (90% of images with at least one correction) suggest that automatic OCR post-correction could unlock much larger corrected VLT training corpora than manual annotation alone.
The reported medium-density performance dip implies VLT difficulty is non-monotonic in input length; benchmarks that report only one average will miss the failure zone where models lack both context and brevity.
The balanced-multilingual-tuning result may transfer to other multilingual multimodal tasks such as document understanding or multilingual visual question answering, where single-language fine-tuning would similarly skew learned representations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the dataset is a real contribution, but the two headline claims—DA Score and balanced fine-tuning—are not yet supported by the experiments as designed.

What's new and useful: AibTrans is a genuine attempt at a clean evaluation set: manually corrected OCR and parallel human translations from Chinese into seven languages, with attention to cultural appropriateness. The benchmark across 17 models on three existing datasets is thorough, and the findings on prompt style are concrete: strict instructions degrade reasoning models, while anti-hallucination prompts help end-to-end LVLMs. Those results will be useful to practitioners regardless of the metric problems.

Where it gets shaky: the DA Score is a weighted average of BLEU, CHRF++, BERTScore, and COMET, with weights said to be derived from metric-human correlations, but the weights are never reported. The validation is in-sample: the same human ratings of GPT-4o outputs used to set the weights are then used to show the DA Score aligns better with human scores. No held-out set, no description of how weights are computed from the correlations. Density categories also overlap (Low: ≤10 boxes / 1-30 tokens; Medium: 5-20 boxes / 30-50 tokens; High: 8-30 boxes / 50-90 tokens), so the taxonomy isn't clean.

The bigger problem is the fine-tuning claim. The paper says fine-tuning on a high-resource pair degrades cross-lingual performance and balanced multilingual fine-tuning repairs it, but the evidence is a two-cell comparison: LoRA on all ~30K examples of OCRMT30K (single Chinese→English direction) vs. 500-1000 examples per direction from MIT-10M (up to 104K examples across 104 directions). Training volume, number of directions, domain, and reference style all change at once. Nothing isolates language-pair balance. Worse, Section 5.2 calls OCRMT30K "English-to-Chinese," contradicting Section 2.1 where it is Chinese-to-English, and Table 6's LoRA rows don't say which condition they correspond to, so they can't resolve the confound. The gains are also largest on MIT-10M, whose references are GPT-4 generated—so the balanced model may simply be mimicking those machine references.

Other inconsistencies: AibTrans is described as 630 images in Section 2.3, 100 images and 6,993 instances in Section 5.1, and "700/7000" in Table 8. No code or data is released.

Bottom line: this deserves a serious referee—the dataset and benchmark are worth curating, and the flaws are fixable. But the current version should be a major revision: report the DA Score weights, validate on a held-out set, and redo the fine-tuning experiment with matched data volume and domain. I'd also require the dataset and code to be released.

Referee Report

5 major / 6 minor

Summary. The paper presents a three-part study of multilingual vision-language translation (VLT): it critiques existing datasets (MIT-10M, OCRMT30K, MTIT6) through LLM-based quality ratings and OCR error analysis; it introduces AibTrans, a human-verified parallel dataset from Chinese into seven target languages; and it benchmarks 11 commercial and 6 open-source models across end-to-end and cascaded architectures. On the evaluation side, the paper proposes Density-Aware Scoring (DA Score), a weighted combination of BLEU, CHRF++, BERTScore, and COMET with weights meant to reflect metric-human correlation within density categories defined by bounding-box count and token length. Finally, the paper claims that fine-tuning on a single high-resource pair (OCRMT30K) degrades cross-lingual performance, while a balanced multilingual fine-tuning strategy sampling 500-1000 examples per direction from MIT-10M adapts LVLMs to VLT without sacrificing generalization.

Significance. If the claims hold, the paper makes several useful contributions: a concrete quantitative account of OCR error rates in existing VLT data (Fig. 3), a parallel and culturally grounded evaluation resource in AibTrans, a broad model comparison that includes a careful OCR-dependency analysis and the counterintuitive finding that strict instructions can hurt reasoning models (Table 2), and a cost-performance analysis in Appendix B. The empirical breadth is a real strength: 17 models across three datasets with multiple metrics, including machine-checkable tables. However, the two headline methodological contributions are currently under-evidenced. The DA Score is under-specified (weights not reported, no held-out validation) and the balanced fine-tuning conclusion rests on a confounded two-cell comparison, so the paper's central claims go beyond what the present experiments establish.

major comments (5)

[Section 4.2] The DA Score weights alpha, beta, lambda, phi are never reported; the text says only that they are derived from metric-human correlations within each density category. No per-category weight values, no normalization rule, and no held-out validation are provided. Figure 8 then shows that the DA Score distribution 'aligns more closely with the human score curve' on the same GPT-4o translations and the same datasets (MIT-10M, OCRMT30K, MTIT6) used to fit the weights; this is in-sample fitting, and the transfer of the weights to other models' outputs (as used in Tables 6 and Appendix B) is assumed but never tested. The number of human ratings, annotator count, inter-annotator agreement, and per-cell sample sizes behind Table 3 are also omitted. In addition, the text in Section 4.2 claims that BLEU, CHRF++, BERTScore, and COMET 'demonstrate the highest correlations (all > 0.4)', which is contradicted by Table 3 itself (CHRF++ low: 0.2965; BERTScore medium: 0.2650). These details are load-bearing because DA Score is the measure used to support the fine-tuning conclusions in Sections 5.2-5.3.
[Section 4.2, density taxonomy] The three density categories are given with overlapping intervals: Low Density is <=10 bounding boxes and 1-30 tokens; Medium Density is 5-20 bounding boxes and 30-50 tokens; High Density is 8-30 bounding boxes and 50-90 tokens. A sample with 6 boxes and 30 tokens satisfies both Low and Medium, and one with 8 boxes and 50 tokens satisfies both Medium and High. Because the DA weights in Eq. (1) are selected per density category, the assignment rule determines the score, and as written the rule is ambiguous. The K-means description ('clustering over these two dimensions') does not state how the two dimensions were scaled, how the number of clusters was chosen, or how the reported intervals were derived, so the taxonomy is not reproducible as specified.
[Section 5.2, Tables 4-5] The claim that fine-tuning on a high-resource pair degrades cross-lingual performance while balanced multilingual fine-tuning repairs it rests on a two-cell comparison in which training set size, number of directions, domain, and reference source change simultaneously. Table 4 fine-tunes on roughly 30K OCRMT30K examples in a single direction, whereas Table 5 fine-tunes on 500 or 1000 examples per direction sampled from MIT-10M (up to ~104 directions); nothing isolates 'language-pair balance.' Additionally, Section 2.2 and Figure 4 show that MIT-10M references are GPT-4-generated and stylistically separable from human translations, yet Table 5 reports the largest gains on MIT-10M itself, so the improvements may partly reflect imitation of the machine reference style rather than better VLT ability. Table 4 also reports only dataset-level aggregates, so the assertion of degradation 'across both high- and low-resource translation directions' is not actually exhibited; in fact the in-domain OCRMT30K scores improve after fine-tuning (e.g., Qwen2.5-VL-7B BLEU 17.78 to 25.08).
[Section 5.2, OCRMT30K direction] Section 5.2 describes OCRMT30K as an 'English-to-Chinese dataset,' which contradicts Section 2.1 and Table 8, where OCRMT30K is defined as containing only Chinese-to-English translations. Since the Table 4 experiment is the entire empirical basis for the high-resource-pair degradation claim, the direction of that pair must be stated correctly and the interpretation of the in-domain gains updated accordingly.
[Section 5.3, Table 6] The 'SFT with LoRA' rows in Table 6 do not state which fine-tuning condition they correspond to (OCRMT30K as in Table 4, or balanced MIT-10M sampling with 500 or 1000 examples per direction as in Table 5). The surrounding text credits the gains to the balanced multilingual strategy with 1000 samples, but the table as printed cannot confirm that, and the LoRA row values differ substantially from Tables 4 and 5 because Table 6 is evaluated on AibTrans while Tables 4-5 report on MIT-10M, OCRMT30K, and MTIT6. The fine-tuning condition and evaluation set must be stated explicitly in the table or caption for this held-out evidence to be usable.

minor comments (6)

[Table 2] The header 'Collabrative' is a typo for 'Collaborative', and the same word is misspelled in the table's first column; the text in Section 3.4 uses the correct spelling.
[Section 2.3 vs Section 5.1 and Table 8] The dataset scale is reported inconsistently: Section 2.3 says 630 images were selected for annotation, Section 5.1 says the benchmark contains 100 images and 6993 sentence-level instances, and Table 8 lists 700/7000. The relationship between these numbers (e.g., whether the benchmark uses a 100-image subset) should be clarified.
[Figure 8] The claim that the DA Score distribution aligns more closely with the human score distribution is supported only by a visual comparison; a quantitative agreement measure (e.g., correlation, RMSE, or KL divergence between the distributions) should be reported.
[Section 5.2, sampling description] The phrase 'from 8 to 13 translation directions in MIT-10M' is ambiguous; since MIT-10M has 8 source and 13 target languages, the intended meaning is 104 directed language pairs, and this should be stated explicitly.
[Eq. (1) and Tables 1-6] The notation for BERTScore is inconsistent: it is called BS-F1 in the tables, BERTScore in the text, and BERT-F1 in Eq. (1); please unify the notation.
[Abstract and Section 3.2] The model counts in the abstract (11 commercial, 6 open-source) do not map transparently to Table 1, which includes w/ OCR and w/o OCR variants as well as language-only cascaded models; the counting convention should be stated.

Circularity Check

1 steps flagged · score 6.0 of 10

DA Score's validation is in-sample: its weights are derived from human ratings on the same GPT-4o outputs whose distribution Figure 8 then shows 'aligning more closely' with human scores.

fitted input called prediction [Section 4.2-4.3, Density-Aware Score definition and Figure 8]
"Here, the weights 𝛼, 𝛽, 𝜆, and 𝜙 represent the relative importance of each metric, derived from their correlation with human judgments within each information density category (see Table 3). ... The DA score distribution shifts rightward compared to the arithmetic mean, aligning more closely with the human score curve."

The DA Score is constructed by taking metric weights from Pearson correlations with human ratings of GPT-4o translations on MIT-10M, OCRMT30K, and MTIT6. Figure 8 then displays the DA Score distribution against those same human scores and treats the closer alignment as evidence that DA Score is a more robust measure. Because the weights were derived from the very human judgments used for validation, the improved alignment on that sample is a property of the fitting procedure, not an independent finding. No held-out human ratings or new model outputs are used to validate the weighting, so the claimed reliability of DA Score reduces by construction to its fitting target. Subsequent benchmark results reported in DA Score units in Table 6 inherit this fitted character.

full rationale

The only circular step I can exhibit by the paper's own equations is the validation of the DA Score. In Sections 4.2 and 4.3, the weights alpha, beta, lambda, and phi are derived from metric-human correlations computed on manual ratings of GPT-4o translations across the same three datasets, and Figure 8 then compares the resulting DA Score distribution with those same human scores, reporting that it aligns more closely than an arithmetic mean. That closer alignment is a consequence of deriving the weights from the human ratings, not an independent confirmation of metric quality. Table 6 subsequently reports all model results in DA Score units, so the benchmark's headline comparisons are expressed through this fitted metric. The balanced-multilingual-fine-tuning claim itself is not circular; it is confounded because the single-pair OCRMT30K fine-tuning differs from the balanced MIT-10M sampling in data volume, domain, number of directions, and reference style, but confoundedness is a correctness risk rather than circularity and is not counted here. I found no load-bearing self-citation chain or imported uniqueness theorem. The paper is therefore partially circular in its evaluation-metric contribution, while the dataset construction, architecture analysis, and adaptation experiments retain independent content.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central evaluation and fine-tuning claims rest on several assumptions: LLM-based quality judgments are treated as ground truth, the density taxonomy is defined by two features with overlapping K-means boundaries, and DA Score weights are fitted to the same human ratings used to validate the metric. No held-out validation, released artifacts, or inter-annotator agreement is provided.

free parameters (6)

DA Score weights (alpha, beta, lambda, phi) = not reported
Weights are derived from Pearson correlations between metrics and human ratings within each density tier in Sec 4.3; without numeric values the DA Score cannot be reproduced.
Density category boundaries = Low: up to 10 boxes, 1-30 tokens; Medium: 5-20 boxes, 30-50 tokens; High: 8-30 boxes, 50-90 tokens
Chosen by K-means on bounding box count and token length in Sec 4.2; intervals overlap, so sample assignment is ambiguous.
Number of density clusters = 3
K in K-means is set to 3 without justification beyond the proposed low, medium, and high taxonomy in Sec 4.2.
Quality-deficiency threshold = 4.3
Fine-grained analysis in Sec 2.2 selects cases where any dimension scored below 4.3; the threshold is post hoc.
Balanced fine-tuning sample size = 500, 1000, and 2000 per direction
Sampling counts are chosen by the authors; the paper concludes that 1000 is sufficient from these hand-picked values in Sec 5.2.
LoRA hyperparameters = not reported
LoRA rank and alpha are not stated, making the fine-tuning comparison difficult to reproduce in Sec 5.2.

assumptions (6)

domain assumption LLM judges, Qwen-Max and DeepSeek-V3, give valid 1-5 scores for semantic adequacy, grammar, fluency, and cultural appropriateness.
Used to rate existing dataset references in Sec 2.2; no human validation or calibration of these LLM judges is provided.
domain assumption Human ratings of GPT-4o translations are treated as ground truth for metric alignment.
Sec 4.2 uses these ratings to compute metric-human correlations; no inter-annotator agreement or rating counts are reported.
domain assumption Information density is adequately captured by number of bounding boxes and source token length.
Sec 4.2 defines density solely on these two dimensions and clusters with K-means, ignoring other complexity factors.
ad hoc to paper A linear weighted sum of BLEU, CHRF++, BERTScore, and COMET yields a valid holistic translation quality score.
The DA Score formula in Sec 4.3 assumes these metrics are commensurable and additively combinable with correlation-derived weights.
domain assumption The selected 630 or 100 Taobao images are representative of general VLT scenarios.
Sec 2.3 and Sec 5.1 describe manual filtering for diversity from e-commerce images; no distributional comparison to other VLT domains is given.
domain assumption Professional translators with access to images and corrected OCR produce gold references.
Sec 2.3 relies on translator quality without reporting translator qualifications, instructions, or inter-annotator agreement.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation." pith.science (2026). https://pith.science/paper/524TYMLL

@misc{pith2026250611820,
  author       = {Pith},
  title        = {Pith review of: Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/524TYMLL}},
  note         = {Machine review of arXiv:2506.11820}
}

read the original abstract

Vision-Language Translation (VLT) is a challenging task that requires accurately recognizing multilingual text embedded in images and translating it into the target language with the support of visual context. While recent Large Vision-Language Models (LVLMs) have demonstrated strong multilingual and visual understanding capabilities, there is a lack of systematic evaluation and understanding of their performance on VLT. In this work, we present a comprehensive study of VLT from three key perspectives: data quality, model architecture, and evaluation metrics. (1) We identify critical limitations in existing datasets, particularly in semantic and cultural fidelity, and introduce AibTrans -- a multilingual, parallel, human-verified dataset with OCR-corrected annotations. (2) We benchmark 11 commercial LVLMs/LLMs and 6 state-of-the-art open-source models across end-to-end and cascaded architectures, revealing their OCR dependency and contrasting generation versus reasoning behaviors. (3) We propose Density-Aware Evaluation to address metric reliability issues under varying contextual complexity, introducing the DA Score as a more robust measure of translation quality. Building upon these findings, we establish a new evaluation benchmark for VLT. Notably, we observe that fine-tuning LVLMs on high-resource language pairs degrades cross-lingual performance, and we propose a balanced multilingual fine-tuning strategy that effectively adapts LVLMs to VLT without sacrificing their generalization ability.

Figures

Figures reproduced from arXiv: 2506.11820 by the authors.

**Figure 2.** Multilingual Multi-direction Translation Quality [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** Distributions of Machine vs. Human Translations. [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗

Figures from the paper (7 more)

**Figure 3.** Figure 3: OCR Recognition Error Distribution Analysis [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]

**Figure 5.** Figure 5: Architectures of End-to-End and Cascaded Model [PITH_FULL_IMAGE:figures/full_fig_p003_5.png]

**Figure 6.** Figure 6: Behavioral Observations of Cascaded and End-to [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

**Figure 7.** Figure 7: Information Density Clustering Medium Density: 5-20 bounding boxes, 30-50 tokens. High Density: 8-30 bounding boxes, 50-90 tokens. This categorization reflects increasing complexity along both the visual and linguistic axes, as illustrated in [PITH_FULL_IMAGE:figures…

**Figure 8.** Figure 8: Score Distribution: Human vs DA-Score vs Arith [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

**Figure 10.** Figure 10: Attribute Distribution of AibTrans Dataset [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

**Figure 9.** Figure 9: Closed-Source Model Trade-offs: DA Scores vs Cost [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 22 canonical work pages

[1]

Alibaba DAMO Academy. 2024. Qwen-VL-Max: Large-scale Vision-Language Model. https://github.com/QwenLM/Qwen-VL Accessed: 2025-04-12

work page 2024
[2]

Alibaba DAMO Academy. 2025. QvQ-72B-Preview: Multimodal Reasoning Model. https://huggingface.co/Qwen/QVQ-72B-Preview Accessed: 2025-04-12

work page 2025
[3]

Jaided AI. 2020. EasyOCR: Ready-to-use OCR with 80+ Languages Supported. https://github.com/JaidedAI/EasyOCR. Accessed: 2025-04-12

work page 2020
[4]

Anthropic. 2025. Claude 3.7: Enhanced Language Model. https://docs.anthropic. com/claude/ Accessed: 2025-04-12

work page 2025
[5]

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report.arXiv preprint arXiv:2309.16609(2023)

arXiv 2023
[6]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al . 2025. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923(2025)

arXiv 2025
[7]

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al . 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271(2024)

arXiv 2024
[8]

Alibaba Cloud. 2025. Qwen-Max: A Large Language Model by Alibaba Cloud. https://huggingface.co/Qwen/Qwen-Max

work page 2025

Show all 42 references

[9]

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. arXiv preprint arXiv:2305.06500(2023). https://arxiv.org...

2023 arXiv
[10]

Google DeepMind. 2025. Gemini 2.0 Flash: Lightweight Multimodal Model. https://deepmind.google/technologies/gemini/ Accessed: 2025-04-12

2025
[11]

Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. 2024. Internlm- xcomposer2: Mastering free-form text-image composition and comprehension in vision-language large model.arXiv preprint arXiv:24...

2024 arXiv
[12]

Ling Fu, Biao Yang, Zhebin Kuang, Jiajun Song, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, Mingxin Huang, et al. 2024. OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning.arXiv preprint arXiv:2501.00321(2024)

2024 arXiv
[13]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

2025 arXiv
[14]

Jonas Hinck, Leonhard Schneider, Jonas Pfeiffer, and Sebastian Ruder. 2024. Why do LLaVA Vision-Language Models Reply to Images in English?arXiv preprint arXiv:2407.02333(2024). https://arxiv.org/abs/2407.02333

2024 arXiv
[15]

Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury
[16]

Shaharukh Khan, Ayush Tarun, Ali Faraz, Palash Kamble, Vivek Dahiya, Praveen Pokala, Ashish Kulkarni, Chandra Khatri, Abhinav Ravi, and Shubham Agarwal
[17]

Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, Min Zhang, and Jinsong Su. 2024. Translatotron-V(ision): An End-to-End Model for In-Image Machine Translation. InFindings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, Ban...

2024
[18]

Zhibin Lan, Jiawei Yu, Xiang Li, Wen Zhang, Jian Luan, Bin Wang, Degen Huang, and Jinsong Su. 2023. Exploring better text image translation with multimodal codebook.arXiv preprint arXiv:2305.17415(2023)

2023 arXiv
[19]

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. 2024. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326(2024)

2024 arXiv
[20]

Bo Li, Shaolin Zhu, and Lijie Wen. 2024. MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translation.arXiv preprint arXiv:2412.07147(2024)

2024 arXiv
[21]

Huanyu Li, Hao Wang, Ying Zhang, Li Li, and Peng Ren. 2025. Underwater image captioning: Challenges, models, and datasets.ISPRS Journal of Photogrammetry and Remote Sensing220 (2025), 440–453

2025
[22]

Yupu Liang, Yaping Zhang, Cong Ma, Zhiyang Zhang, Yang Zhao, Lu Xiang, Chengqing Zong, and Yu Zhou. 2024. Document Image Machine Translation with Dynamic Multi-pre-trained Models Assembling. InProceedings of the 2024 Conference of the North American Chapter of the Association ...

2024 doi
[23]

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437(2024)

2024 arXiv
[24]

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning.arXiv preprint arXiv:2304.08485(2023). https://arxiv.org/abs/2304. 08485

2023 arXiv
[25]

Liqiang Niu, Fandong Meng, and Jie Zhou. 2024. UMTIT: Unifying Recogni- tion, Translation, and Generation for Multimodal Text Image Translation. In Proceedings of the 2024 Joint International Conference on Computational Linguis- tics, Language Resources and Evaluation (LREC-CO...

2024
[26]

OpenAI. 2025. GPT-4o: OpenAI’s Multimodal Model. https://openai.com/blog/ gpt-4o Accessed: 2025-04-12

2025
[27]

OpenAI. 2025. GPT-4o1: Updated Multimodal Model. https://openai.com/blog/ gpt-4o1 Accessed: 2025-04-12

2025
[28]

OpenAI. 2025. OpenAI API Reference: Embeddings. https://platform.openai. com/docs/api-reference/embeddings Accessed: 2025-04-12

2025
[29]

OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al. 2023. GPT-4 Technical Report. https://arxiv.org/abs/2303.08774 Accessed: 2025-04-12

2023 arXiv
[30]

PaddlePaddle. 2025. PaddleOCR: Multi-language OCR System. https://github. com/PaddlePaddle/PaddleOCR Accessed: 2025-04-12

2025
[31]

Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek F Wong, Xiaoshuai Sun, and Rongrong Ji. 2024. Anytrans: Translate anytext in the image with large scale models.arXiv preprint arXiv:2406.11432(2024)

2024 arXiv
[32]

Elizabeth Salesky, Philipp Koehn, and Matt Post. 2024. Benchmarking Visually- Situated Translation of Text in Natural Images. InProceedings of the Ninth Con- ference on Machine Translation. 1167–1182

2024
[33]

Macro-MT Team. 2025. Macro-MT: Emphasizing Rare Types in Machine Transla- tion. https://github.com/macro-mt/macro-mt Accessed: 2025-04-12

2025
[34]

Qwen Team. 2024. QwQ: Reflect Deeply on the Boundaries of the Unknown. https://qwenlm.github.io/blog/qwq-32b-preview/ Accessed: 2025-04-12

2024
[35]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE.Journal of Machine Learning Research9 (2008), 2579–2605. https://www. jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

2008
[36]

Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. 2024. Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding. InFindings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, 1...

2024 doi
[37]

Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. 2024. Deepseek- vl2: Mixture-of-experts vision-language models for advanced multimodal under- standing.arXiv preprint arXiv:2412.10302(2024)

2024 arXiv
[38]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115(2024)

2024 arXiv
[39]

Zhibo Yang, Jun Tang, Zhaohai Li, Pengfei Wang, Jianqiang Wan, Humen Zhong, Xuejing Liu, Mingkun Yang, Peng Wang, Yuliang Liu, et al. 2024. CC-OCR: A Com- prehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy.arXiv preprint arXiv:2412.02210(2024)

2024 arXiv
[40]

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models.arXiv preprint arXiv:2304.10592(2023). https://arxiv.org/abs/ 2304.10592 A The Masking Effect of Averaging To furth...

2023 arXiv
[2020]

arXiv preprint arXiv:2004.09095(2020)

The state and fate of linguistic diversity and inclusion in the NLP world. arXiv preprint arXiv:2004.09095(2020)

2020 arXiv
[2025]

arXiv preprint arXiv:2502.20420(2025)

Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation. arXiv preprint arXiv:2502.20420(2025)

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Alibaba DAMO Academy. 2024. Qwen-VL-Max: Large-scale Vision-Language Model. https://github.com/QwenLM/Qwen-VL Accessed: 2025-04-12

work page 2024

[2] [2]

Alibaba DAMO Academy. 2025. QvQ-72B-Preview: Multimodal Reasoning Model. https://huggingface.co/Qwen/QVQ-72B-Preview Accessed: 2025-04-12

work page 2025

[3] [3]

Jaided AI. 2020. EasyOCR: Ready-to-use OCR with 80+ Languages Supported. https://github.com/JaidedAI/EasyOCR. Accessed: 2025-04-12

work page 2020

[4] [4]

Anthropic. 2025. Claude 3.7: Enhanced Language Model. https://docs.anthropic. com/claude/ Accessed: 2025-04-12

work page 2025

[5] [5]

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report.arXiv preprint arXiv:2309.16609(2023)

arXiv 2023

[6] [6]

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al . 2025. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923(2025)

arXiv 2025

[7] [7]

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al . 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271(2024)

arXiv 2024

[8] [8]

Alibaba Cloud. 2025. Qwen-Max: A Large Language Model by Alibaba Cloud. https://huggingface.co/Qwen/Qwen-Max

work page 2025