Pith. sign in

REVIEW 2 major objections 5 minor 51 references

Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning on the model's own OCR transcript improves document image translation and preserves OCR ability.

desk verdict A cheap, well-tested fine-tuning recipe for DIMT that preserves OCR, but the load-bearing assumption that the frozen self-generated OCR transcript stays on-distribution after training is never directly checked. read the letter →

arxiv 2507.08309 v1 pith:2MSP2DJN submitted 2025-07-11 cs.CL cs.AIcs.CV

Yupu Liang , Yaping Zhang , Zhiyang Zhang , Zhiyuan Chen , Yang Zhao , Lu Xiang , Chengqing Zong , Yu Zhou This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords documentimagemachinetranslationmultimodallargelanguagemodelsopticalcharacterrecognitioncatastrophicforgettingsupervisedfine-tuningself-reviewcross-lingualgeneralizationlow-resource
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MLLMs are good at OCR but bad at translating the text inside document images, and standard supervised fine-tuning on translation data makes them forget how to read. This paper proposes SSR: before fine-tuning, prompt the model to produce its own OCR transcript of each training image, then train it on that transcript followed by the ground-truth translation. The authors claim this joint objective improves translation quality over standard SFT on all four MLLMs tested, sharply reduces catastrophic forgetting of OCR and VQA abilities, and improves zero-shot translation on unseen document domains. The practical payoff is that document image translation can be improved without sacrificing the model's monolingual competence, using only the model's own OCR output plus target translations.

What carries the argument

The central object is the SSR-constrained prompt template, which appends the original OCR instruction (e.g., "Convert the content in the image to Markdown") with "then translate into Chinese," and sets the expected response to be the frozen self-generated OCR transcript $X'$, the special token $\langle\text{Translation}\rangle$, and the ground-truth target $Y$. Training minimizes the negative log-likelihood of this response given the image and instruction. Because $X'$ is sampled from the base model's own output distribution, the training signal stays close to the model's existing monolingual behavior while teaching the cross-lingual image-to-translation mapping; the authors argue this yields a smoother loss curve and preserves OCR proficiency.

What would settle it

During SSR fine-tuning, take the training images and regenerate the OCR transcript with the partially fine-tuned model at each epoch; if the regenerated transcript drifts from the frozen $X'$ (e.g., character accuracy of regenerated versus frozen text falls) and translation BLEU on a held-out set simultaneously degrades, the stale-transcript assumption is falsified. A direct test: an online variant that resamples $X'$ each epoch from the current model should beat the frozen version if drift matters.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the target for DIMT fine-tuning should not be the translation alone but the concatenated sequence $R = \mathrm{CONCAT}(X', \langle\text{Translation}\rangle, Y)$, where $X'$ is the OCR transcript the base model itself generates from the document image and $Y$ is the ground-truth translation. Training with the standard negative log-likelihood loss on this sequence lets the model condition translation on its own OCR output. Evidence includes Qwen2-VL reaching 57.23 in-domain BLEU versus 53.92 for SFT, and retaining 85.18 document OCR character accuracy versus 5.96 for SFT; similar trends hold for Vary-base, Vary-toy, and Textmonkey. The paper further shows the method transfers cross-domain, extends to other language pairs, and can use unsupervised document images plus machine-translated synthetic targets to gain additional improvements.

Load-bearing premise

The method freezes the self-generated OCR transcript $X'$ before fine-tuning and assumes this transcript still matches the distribution the model will produce after fine-tuning, so the learned image-to-transcript-to-translation mapping is trained on a representative source text.

Editorial extensions

If this is right

  • Standard SFT for DIMT can be replaced by SSR to obtain higher BLEU, BLEU-PT, and STEDS on in-domain and zero-shot cross-domain tests across four MLLMs.
  • Catastrophic forgetting of OCR is largely avoided: for example, Qwen2-VL retains 85.18 document OCR character accuracy versus 5.96 under SFT, and SSR even improves scene-text OCR over the base model.
  • The method transfers to other language pairs (English-French, English-German) and is effective in low-resource settings, with SSR at 10K training samples beating SFT trained on 100K samples by 3.01 in-domain BLEU.
  • Unsupervised document images can be turned into training data by OCR-ing them with the MLLM and machine-translating the transcript, further improving in-domain and cross-domain translation.
  • Fine-tuning with SSR also induces cross-lingual VQA generalization: the model answers Chinese questions on English document images zero-shot.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method is effectively a self-distillation of the OCR stream: it anchors the fine-tuned model to its own pre-fine-tuning image-to-text behavior, so what is preserved is not the OCR training set but the model's own output distribution.
  • The same 'strong retained skill as prefix' recipe could be applied to other multimodal fine-tuning targets, e.g., inserting a captioning or grounding prefix before a new task output to protect those abilities.
  • A testable extension the paper does not run: resample the OCR transcript online during training from the evolving model; if that improves translation further, the fixed-transcript assumption is the active bottleneck.
  • The analogy to bilingual cognitive advantage suggests a general fine-tuning principle: chain a familiar monolingual, monotask generation step before a new cross-lingual, cross-modal step to keep both skills.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes SSR, a fine-tuning strategy for document image machine translation (DIMT) with multimodal large language models. For each training image, the base MLLM is first prompted with its original OCR instruction to sample a source-text transcript X' (Eq. 1); the training target concatenates X', a special token, and the ground-truth translation (Eq. 3), and the model is fine-tuned with the negative log-likelihood over this response (Eq. 2). At inference the model is prompted to 'convert the image to Markdown, then translate into Chinese', so it generates its own OCR prefix before translating. Experiments with Vary-toy/Vary-base, Textmonkey, and Qwen2-VL, fine-tuned with LoRA on 10K DoTA samples, report that SSR improves DIMT BLEU/BLEU-PT/STEDS relative to SFT and other baselines on in-domain DoTA and zero-shot DITrans test sets, while largely preserving OCR and VQA performance; extensions cover unsupervised data, monolingual-task selection, low-resource training, and other language pairs.

Significance. If the reported effects are robust, SSR is a practical and inexpensive way to mitigate catastrophic forgetting in DIMT fine-tuning: it requires no auxiliary data beyond the model's own OCR samples, works across four architectures of different sizes, and the gains are consistent and often large. The paper also provides useful ablations (source-text type, monolingual task, data scale) and makes code available. However, the central mechanism rests on an unverified assumption that the frozen pre-training OCR transcript remains representative of the OCR prefix the fine-tuned model generates at inference; the manuscript does not measure this drift, so the key causal story is not yet fully supported.

major comments (2)
  1. [Sec. 3.1-3.2, Eqs. (1)-(3); Sec. 5.1; Fig. 4] The load-bearing assumption of the method is that the frozen self-generated transcript X' used during training remains representative of the OCR prefix that the fine-tuned model produces at inference. Equation (1) samples X' from the base model under P_ocr before fine-tuning, while at inference the model is prompted with P_combined ('... then translate into Chinese') and must autoregressively emit its own OCR prefix from the updated parameters. The paper never measures the agreement between the online OCR prefix and the frozen X'. Table 2 evaluates OCR only under the original OCR prompt, not under P_combined; Table 4 varies the source-text type but does not test the online-vs-frozen match. Consequently, the 'smooth convergence' explanation in Section 5.1 and Figure 4 is not the only possible account of the BLEU gains; the model could partially bypass X' or re-learn OCR, and the reported gains would then not validate the stated mechanism. Please add a direct measurement (e.g., edit distance or token-level agreement between the fine-tuned model's OCR prefix under P_combined and the frozen X' used for the same images) and, ideally, an ablation that conditions inference on the frozen X' versus the model-generated prefix. The Limitations section should also acknowledge this assumption.
  2. [Sec. 4.1, App. A.1; Tables 1 and 5] Cross-domain test sets are small (100 images per domain after merging Ads & News), and no error bars, multiple seeds, or significance tests are reported. Given that a central claim is large zero-shot cross-domain improvements (e.g., Qwen2-VL Ads & News +10.13 BLEU over SFT(DIMT) in Table 1), the absence of variance estimates is material: BLEU computed on 100 documents has substantial sampling noise. Please report confidence intervals, significance tests, or results over multiple seeds, or enlarge the cross-domain evaluation sets.
minor comments (5)
  1. [Table 8] The header contains a typo: 'BELU' should be 'BLEU'.
  2. [App. A.3] In the CoT (Cascade) prompt template, the line '(original image caption instruction of the MLLM)' appears to be a copy-paste error; the instruction shown is the OCR instruction, not an image caption instruction.
  3. [Footnote 1] The code repository URL is garbled by escape sequences in the rendered text; please provide a clean, clickable URL.
  4. [Abstract and Sec. 1] The method is called 'Synchronously Self-Reviewing' in the title and abstract but 'Synchronized Self-Reviewing' in the introduction; please use one name consistently.
  5. [Table 4] The claim that Vary-base achieves 'performance parity' with ground-truth versus self-generated source text is not fully supported by the table: in the Political Report (CD) column, SSR w Ground Truth Text gives 26.05 BLEU while SSR w Self-generated Text gives 21.47 BLEU, a 4.58-point difference; this discrepancy should be acknowledged or discussed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SSR is an empirical fine-tuning method evaluated on held-out benchmarks, not a derivation that reduces to its inputs.

full rationale

The paper's central claim is that concatenating the model's own OCR transcript (X') with the ground-truth translation (Y) as the supervised target (Eqs. 2-3) improves DIMT and preserves OCR ability. This is a training-objective design, not a derivation. The self-generated X' is sampled once from the base model (Eq. 1) and used as a fixed training signal; no parameter is fitted to the test benchmarks, and the OCR evaluation (Table 2) is on held-out DITrans/FST data, not the training distribution. The method is compared against SFT, CoT, replay, and commercial baselines. Citations to the authors' prior work supply the DoTA/DITrans datasets and metric definitions, which are external testbeds rather than theorem premises. The mild self-referential aspect—the model trains on its own OCR output—is the intended mechanism, and it does not make the reported BLEU/CA gains equivalent to the input by construction. The unmeasured drift between frozen X' and the online OCR prefix is a correctness/robustness concern, not a circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality of the public DIMT benchmarks, the base MLLMs' OCR skill, the stability of self-generated OCR during training, and standard metric assumptions. There are no fitted scientific constants; the listed free parameters are implementation choices for LoRA and training setup. No invented entities are introduced.

free parameters (4)
  • LoRA rank = 16
    LoRA rank and alpha set to 16 for all linear layers of the LLM part; chosen by hand.
  • Training set size = 10K samples
    Randomly selected from DoTA for computational constraints; central to the reported gains.
  • Learning rate = 1e-4
    Linear decay with warmup ratio 0.1; standard for LoRA fine-tuning.
  • Number of epochs = 3
    All MLLMs trained for 3 epochs; no early stopping or epoch search reported.
assumptions (4)
  • domain assumption DoTA and DITrans ground-truth translations are accurate and aligned with the document images.
    The fine-tuning and evaluation rely on these public benchmarks; no manual verification is reported.
  • domain assumption The base MLLMs have sufficiently good OCR on document images for self-generated transcripts to serve as usable source text.
    Table 2 shows base OCR CA of 68-85 on document images; if OCR quality were poor, SSR would fail.
  • standard math BLEU, CA, and ANLS are meaningful proxies for translation quality, OCR accuracy, and VQA accuracy.
    Standard metrics in the field; the paper uses them without modification.
  • ad hoc to paper The self-generated OCR distribution does not drift enough during fine-tuning to break the learned mapping.
    The method freezes X' from the base model; this stability is assumed but not measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency." pith.science (2026). https://pith.science/paper/2MSP2DJN

@misc{pith2026250708309,
  author       = {Pith},
  title        = {Pith review of: Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2MSP2DJN}},
  note         = {Machine review of arXiv:2507.08309}
}
read the original abstract

Multimodal Large Language Models (MLLMs) have shown strong performance in document image tasks, especially Optical Character Recognition (OCR). However, they struggle with Document Image Machine Translation (DIMT), which requires handling both cross-modal and cross-lingual challenges. Previous efforts to enhance DIMT capability through Supervised Fine-Tuning (SFT) on the DIMT dataset often result in the forgetting of the model's existing monolingual abilities, such as OCR. To address these challenges, we introduce a novel fine-tuning paradigm, named Synchronously Self-Reviewing (SSR) its OCR proficiency, inspired by the concept "Bilingual Cognitive Advantage". Specifically, SSR prompts the model to generate OCR text before producing translation text, which allows the model to leverage its strong monolingual OCR ability while learning to translate text across languages. Comprehensive experiments demonstrate the proposed SSR learning helps mitigate catastrophic forgetting, improving the generalization ability of MLLMs on both OCR and DIMT tasks.

Figures

Figures reproduced from arXiv: 2507.08309 by the authors.

Figure 1
Figure 1. Performance of Qwen2-VL across various benchmarks. Base refers to the performance of the orig￾inal MLLM, while SFT denotes the MLLM after fine￾tuning on the DIMT dataset. DIMT (ID) and DIMT (CD) denote in-domain and cross-domain test separately. The evaluation metrics for DIMT, OCR, and VQA are BLEU, Character Accuracy (CA), and Average Normal￾ized Levenshtein Similarity (ANLS), respectively. translating text in doc… view at source ↗
Figure 2
Figure 2. Bilingual individuals exhibit greater linguistic [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of our proposed fine-tuning paradigm SSR. It contains two steps: (1) [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Training loss curves of different methods in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Results of Qwen2-VL through SSR fine￾tuning using different monolingual tasks. Detailed data can be seen in Appendix C. It is better to zoom in for a clearer view. DocVQA and InfoVQA test sets, respectively. Moreover, by comparing the performance of Vary￾base, Textmonk…
Figure 6
Figure 6. Figure 6: Results of Vary-base and Qwen2-VL through [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Results of Vary-base through SSR fine-tuning [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The output samples of Qwen2-VL (after fine-tuning with SSR in the main experiment) on the DoTA [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: The output samples of Qwen2-VL (after fine-tuning with SSR in the main experiment) on the DITrans [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: The output samples of Qwen2-VL (after fine-tuning with SSR in the main experiment) on the DITrans [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: The output samples of Qwen2-VL (after fine-tuning with SSR in the main experiment) on the OCR test. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: The output samples of Qwen2-VL (after fine-tuning with SSR in the main experiment) on the VQA test. [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 20 canonical work pages

  1. [1]

    Ellen Bialystok. 1991. Language processing in bilingual children. Cambridge University Press

    work page 1991

  2. [2]

    Ellen Bialystok. 2001. Bilingualism in development: Language, literacy, and cognition. Cambridge University Press

    work page 2001

  3. [3]

    Ellen Bialystok and Fergus IM Craik. 2010. Cognitive and linguistic processing in the bilingual mind. Current directions in psychological science, 19(1):19--23

    work page 2010

  4. [4]

    Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. 2024. https://openreview.net/forum?id=fUtxNAKpdV Nougat: Neural optical understanding for academic documents . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

  5. [5]

    Boyu Guan, Yining Zhang, Yang Zhao, and Chengqing Zong. 2025. https://aclanthology.org/2025.coling-main.547/ T ri F ine: A large-scale dataset of vision-audio-subtitle for tri-modal machine translation and benchmark with fine-grained annotated tags . In Proceedings of the 31st International Conference on Computational Linguistics, pages 8215--8231, Abu Dh...

    work page 2025

  6. [6]

    Josiane F Hamers. 1998. Cognitive and language development of bilingual children. Cultural and language diversity and the deaf experience, pages 51--75

    work page 1998

  7. [7]

    Ryota Hinami, Shonosuke Ishiwatari, Kazuhiko Yasuda, and Yusuke Matsui. 2021. https://doi.org/10.1609/AAAI.V35I14.17537 Towards fully automated manga translation . In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educat...

    work page doi:10.1609/aaai.v35i14.17537 2021

  8. [8]

    Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. 2024 a . https://doi.org/10.18653/v1/2024.findings-emnlp.175 m PLUG - D oc O wl 1.5: Unified structure learning for OCR -free document understanding . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3096--3120, M...

    doi:10.18653/v1/2024.findings-emnlp.175 2024

Show all 51 references
  1. [9]

    Anwen Hu, Haiyang Xu, Liang Zhang, Jiabo Ye, Ming Yan, Ji Zhang, Qin Jin, Fei Huang, and Jingren Zhou. 2024 b . https://doi.org/10.48550/ARXIV.2409.03420 mplug-docowl2: High-resolution compressing for ocr-free multi-page document understanding . CoRR, abs/2409.03420

    2024 doi arXiv

  2. [10]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representat...

    2022

  3. [11]

    Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and et al

    Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and et al. 2024. https://doi.org/10.48550/ARXIV.2410.21276 Gpt-4o system card . CoRR, abs/2410.21276

    2024 doi arXiv

  4. [12]

    Pu Jian, Donglei Yu, and Jiajun Zhang. 2024. Large language models know what is key visual entity: An llm-assisted multimodal retrieval for vqa. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10939--10956

    2024

  5. [13]

    Ghosh, Andrew D

    Dimosthenis Karatzas, Lluis Gomez - Bigorda, Anguelos Nicolaou, Suman K. Ghosh, Andrew D. Bagdanov, Masakazu Iwamura, Jiri Matas, Luk \' a s Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, Faisal Shafait, Seiichi Uchida, and Ernest Valveny. 2015. https://doi.org/10.1109/I...

    2015

  6. [14]

    Yupu Liang, Yaping Zhang, Cong Ma, Zhiyang Zhang, Yang Zhao, Lu Xiang, Chengqing Zong, and Yu Zhou. 2024. https://aclanthology.org/2024.naacl-long.392 Document image machine translation with dynamic multi-pre-trained models assembling . In Proceedings of the 2024 Conference of...

    2024

  7. [15]

    Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, and Xiang Bai. 2024. https://doi.org/10.48550/ARXIV.2403.04473 Textmonkey: An ocr-free large multimodal model for understanding document . CoRR, abs/2403.04473

    2024 doi arXiv

  8. [16]

    Cong Ma, Yaping Zhang, Mei Tu, Xu Han, Linghui Wu, Yang Zhao, and Yu Zhou. 2022. https://doi.org/10.1109/ICPR56361.2022.9956695 Improving end-to-end text image translation from the auxiliary text translation task . In 26th International Conference on Pattern Recognition, ICPR ...

    2022

  9. [17]

    Cong Ma, Yaping Zhang, Mei Tu, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023. https://doi.org/10.1007/978-3-031-41676-7\_28 Multi-teacher knowledge distillation for end-to-end text image machine translation . In Document Analysis and Recognition - ICDAR 2023 - 17th Internationa...

    2023 doi

  10. [18]

    Cong Ma, Yaping Zhang, Zhiyang Zhang, Yupu Liang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2024. https://aclanthology.org/2024.lrec-main.222/ Born a B aby N et with hierarchical parental supervision for end-to-end text image machine translation . In Proceedings of the 2024 Join...

    2024

  11. [19]

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. https://doi.org/10.18653/v1/2022.findings-acl.177 C hart QA : A benchmark for question answering about charts with visual and logical reasoning . In Findings of the Association for Computational Ling...

    2022 doi

  12. [20]

    Minesh Mathew, Viraj Bagal, Rub \` e n Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V. Jawahar. 2022. https://doi.org/10.1109/WACV51458.2022.00264 Infographicvqa . In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, Waikoloa, HI, USA, January 3-8...

    2022

  13. [21]

    Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. 2021. https://doi.org/10.1109/WACV48630.2021.00225 Docvqa: A dataset for VQA on document images . In IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Waikoloa, HI, USA, January 3-8, 2021 , pages 2199-...

    2021

  14. [22]

    Jisoo Mok, Jaeyoung Do, Sungjin Lee, Tara Taghavi, Seunghak Yu, and Sungroh Yoon. 2023. https://doi.org/10.18653/v1/2023.acl-long.703 Large-scale lifelong learning of in-context instructions and how to tackle it . In Proceedings of the 61st Annual Meeting of the Association fo...

    2023 doi

  15. [23]

    Liqiang Niu, Fandong Meng, and Jie Zhou. 2024. https://aclanthology.org/2024.lrec-main.1474 UMTIT: unifying recognition, translation, and generation for multimodal text image translation . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, ...

    2024

  16. [24]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

    2022

  17. [25]

    Wong, Xiaoshuai Sun, and Rongrong Ji

    Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek F. Wong, Xiaoshuai Sun, and Rongrong Ji. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.137 A ny T rans: Translate A ny T ext in the image with large scale models . In Findings of the Association for Comput...

    2024 doi

  18. [26]

    Lillicrap, Jean - Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, and et al

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean - Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, and et al. 2024. https://doi.org/10.48550/ARXIV.2403.05530 Gemini 1.5: Unlocking multimodal unde...

    2024 doi arXiv

  19. [27]

    Shuo Ren, Pu Jian, Zhenjiang Ren, Chunlin Leng, Can Xie, and Jiajun Zhang. 2025. Towards scientific intelligence: A survey of llm-based scientific agents. arXiv preprint arXiv:2503.24047

    2025

  20. [28]

    Nilesh P Sable, Priya Shelke, Ninad Deogaonkar, Nachiket Joshi, Rudra Kabadi, and Tushar Joshi. 2023. Doc-handler: Document scanner, manipulator, and translator based on image and natural language processing. In 2023 International Conference on Emerging Smart Computing and Inf...

    2023

  21. [29]

    Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, and Hao Wang. 2024. https://doi.org/10.48550/ARXIV.2404.16789 Continual learning of large language models: A comprehensive survey . CoRR, abs/2404.16789

    2024 doi arXiv

  22. [30]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. https://doi.org/10.48550/ARXIV.2409.1219...

    2024 doi arXiv

  23. [31]

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. 2024 a . https://doi.org/10.1007/978-3-031-73235-5\_23 Vary: Scaling up the vision vocabulary for large vision-language model . In Computer Vision - ECCV 202...

    2024 doi

  24. [32]

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, En Yu, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. 2024 b . https://doi.org/10.48550/ARXIV.2401.12503 Small language model meets with reinforced vision vocabulary . CoRR, abs/2401.12503

    2024 doi arXiv

  25. [33]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning...

    2022

  26. [34]

    Junhong Wu, Yang Zhao, Yangyifan Xu, Bing Liu, and Chengqing Zong. 2024. https://doi.org/10.48550/ARXIV.2410.13944 Boosting LLM translation skills without general ability loss via rationale distillation . CoRR, abs/2410.13944

    2024 doi arXiv

  27. [35]

    Wen Yang, Chong Li, Jiajun Zhang, and Chengqing Zong. 2023. Bigtranslate: Augmenting large language models with multilingual translation capability over 100 languages. arXiv preprint arXiv:2305.18098

    2023 arXiv

  28. [36]

    Wen Yang, Junhong Wu, Chen Wang, Chengqing Zong, and Jiajun Zhang. 2024 a . Language imbalance driven rewarding for multilingual self-improving. arXiv preprint arXiv:2410.08964

    2024 arXiv

  29. [37]

    Wen Yang, Junhong Wu, Chen Wang, Chengqing Zong, and Jiajun Zhang. 2025. Implicit cross-lingual rewarding for efficient multilingual preference alignment. arXiv preprint arXiv:2503.04647

    2025 arXiv

  30. [38]

    Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.58 Self-distillation bridges distribution gap in language model fine-tuning . In Proceedings of the 62nd Annual Meeting of the Association ...

    2024 doi

  31. [39]

    Cong Yao. 2023. https://doi.org/10.48550/ARXIV.2310.12430 Docxchain: A powerful open-source toolchain for document parsing and beyond . CoRR, abs/2310.12430

    2023 doi arXiv

  32. [40]

    Wenpeng Yin, Jia Li, and Caiming Xiong. 2022. https://doi.org/10.18653/v1/2022.acl-long.218 C on T in T in: Continual learning from task instructions . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 306...

    2022 doi

  33. [41]

    Ya - Qi Yu, Minghui Liao, Jihao Wu, Yongxin Liao, Xiaoyu Zheng, and Wei Zeng. 2024. https://doi.org/10.48550/ARXIV.2404.09204 Texthawk: Exploring efficient fine-grained perception of multimodal large language models . CoRR, abs/2404.09204

    2024 doi arXiv

  34. [42]

    Yunhao Zhang, Shaonan Wang, Xinyi Dong, Jiajun Yu, and Chengqing Zong. 2023 a . Navigating brain language representations: A comparative analysis of neural language models and psychologically plausible models. In Proceedings of the Annual Meeting of the Cognitive Science Socie...

    2023

  35. [43]

    Yunhao Zhang, Shaonan Wang, Nan Lin, Lingzhong Fan, and Chengqing Zong. 2025 a . https://doi.org/10.1016/j.neuroimage.2025.121096 A simple clustering approach to map the human brain's cortical semantic network organization during task . NeuroImage, 309:121096

    2025

  36. [44]

    Yunhao Zhang, Xiaohan Zhang, Chong Li, Shaonan Wang, and Chengqing Zong. 2024. Mulcogbench: A multi-modal cognitive benchmark dataset for evaluating chinese and english computational language models. arXiv preprint arXiv:2403.01116

    2024 arXiv

  37. [45]

    Zhiyang Zhang, Yaping Zhang, Yupu Liang, Cong Ma, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2025 b . https://doi.org/10.1109/TPAMI.2025.3530998 Understand layout and translate text: Unified feature-conductive end-to-end document image translation . IEEE Trans. Pattern ...

    2025

  38. [46]

    Zhiyang Zhang, Yaping Zhang, Yupu Liang, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.673 L ayout DIT : Layout-aware end-to-end document image translation with multi-step conductive decoder . In Findings of the Asso...

    2023 doi

  39. [47]

    Zhiyang Zhang, Yaping Zhang, Yupu Liang, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2025 c . https://aclanthology.org/2025.coling-main.723/ From chaotic OCR words to coherent document: A fine-to-coarse zoom-out network for complex-layout document image translation . In ...

    2025

  40. [48]

    Zhiyang Zhang, Yaping Zhang, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 c . A novel dataset and benchmark analysis on document image translation. In China Conference on Machine Translation, pages 103--115. Springer

    2023

  41. [49]

    Shaolin Zhu, Shangjie Li, Yikun Lei, and Deyi Xiong. 2023. https://doi.org/10.18653/v1/2023.acl-long.751 PEIT : Bridging the modality gap with pre-trained models for end-to-end image translation . In Proceedings of the 61st Annual Meeting of the Association for Computational L...

    2023 doi

  42. [50]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  43. [51]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.