Pith. sign in

REVIEW 3 major objections 5 minor 44 references

Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Single-to-mix modality alignment lets an image-only encoder absorb an MLLM's multimodal hidden states, giving a lightweight document translator cross-domain gains.

desk verdict Solid, well-ablated distillation method for document image MT; the mechanism holds up, but the cross-domain story needs a direct representation-fidelity check and stronger statistics. read the letter →

arxiv 2507.07572 v1 pith:6ERFC3PV submitted 2025-07-10 cs.CL cs.AIcs.CV

Yupu Liang , Yaping Zhang , Zhiyang Zhang , Yang Zhao , Lu Xiang , Chengqing Zong , Yu Zhou This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords documentimagemachinetranslationmultimodallargelanguagemodelmodalityalignmentknowledgedistillationcross-domaingeneralizationend-to-endvisual-textualrepresentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Document image machine translation (DIMT) -- translating text embedded in document images while keeping the layout -- struggles to generalize because training data are scarce and visual and textual information interact in complex ways. M4Doc's proposal is to let a small DIMT model borrow the visual-textual knowledge of a frozen multimodal large language model (MLLM) during training only, then discard the MLLM at inference. An image-only alignment encoder is trained to reproduce the MLLM's hidden states, which are computed from both the image and the ground-truth source text, so the lightweight decoder receives text-aware visual representations without the teacher's cost. If the method works as claimed, document-image translation no longer has to choose between the translation quality of a huge multimodal model and the speed of a compact one; the reported cross-domain zero-shot gain of roughly 5.5 BLEU over the strongest end-to-end baseline is the concrete payoff.

What carries the argument

The load-bearing object is the single-to-mix modality alignment encoder: a pre-trained Swin-Transformer vision encoder followed by two feed-forward projections that reshape the image-only representation to the sequence length and dimension of the teacher MLLM's last-layer hidden states. It is trained with the alignment loss $L_{\mathrm{align}} = 1 - \cos(H_{\mathrm{MLLM}}, H_{\mathrm{align}})$, where $H_{\mathrm{MLLM}} = \mathrm{MLLM}(I, X)$ is computed from image and ground-truth source text while $H_{\mathrm{align}}$ comes from the image alone. The total objective $\mathcal{L} = \alpha L_{\mathrm{align}} + L_{\mathrm{trans}}$ couples representation mimicry with translation likelihood, forcing the encoder to remain useful for generation, and at inference the aligned representation feeds an extra mix-modality cross-attention module in each decoder layer. The paper's comparison of alignment objectives (cosine similarity beats cross-entropy and mean squared error) shows that the choice of distance metric is part of the mechanism.

What would settle it

Take an unseen document domain (for instance, the DITrans political-report or ads-news test images) and compare the cosine similarity between the alignment encoder's image-only representation and the teacher's image-plus-text representation against the similarity obtained when the teacher receives only the image instead of the text. If the two similarity scores are statistically indistinguishable on the novel domain, or equivalently if a held-out domain shows no BLEU gain over the no-alignment baseline, then the claim that text-conditioned knowledge transfers breaks down. A complementary experiment would replace the teacher's ground-truth source text with predicted OCR text during training and check whether the gains persist.

Watch

Extended reading notes

Core claim

M4Doc claims that a frozen MLLM's text-conditioned representations can be distilled into an image-only encoder and that this substitution improves a lightweight document-image translation model. During training the teacher MLLM receives the document image together with its ground-truth source text and produces hidden states; the student alignment encoder sees only the image and is supervised to maximize cosine similarity with those states. The translation decoder then works from the student's aligned representation together with image features, and at inference the teacher is removed entirely. The paper's evidence is that the full model beats the strongest end-to-end baseline in-domain by several BLEU points and by a larger margin on zero-shot cross-domain documents, while ablations show that removing the alignment loss or removing the text input to the teacher both degrade translation quality.

Load-bearing premise

The load-bearing premise is that a mapping from a document image alone to the text-conditioned hidden states of the teacher model is learnable from limited training data and transfers to unseen layouts, fonts, and backgrounds; if the image-only encoder merely memorizes training-image teacher outputs, the reported cross-domain gains will not survive.

Editorial extensions

If this is right

  • M4Doc with a document-trained teacher reaches 42.98 BLEU on the in-domain DoTA test set and 18.18 BLEU on the zero-shot DITrans political-report set, compared with 38.68 and 12.64 for the strongest end-to-end baseline.
  • The same training recipe improves performance on long-context documents and on complex-layout images, with up to 5.58 BLEU gained on the complex-layout subset.
  • Directly fine-tuning the MLLM on the DIMT data produces worse translation than distilling its representations into the lightweight model, so the alignment route extracts more from the teacher than supervised fine-tuning does.
  • Because the teacher is discarded at inference, parameter count and per-page inference time stay close to the lightweight baseline's level while translation quality rises.
  • The approach also transfers to English-French and English-German document translation, not only English-Chinese.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's ablations point to the teacher's text input as the main carrier of the gain, which suggests the same single-to-mix alignment recipe could transfer to other image-to-text tasks with privileged text (scene-text translation, chart-to-text, formula recognition) whenever a text-conditioned teacher representation is available.
  • The four-teacher comparison hints that cross-domain gains scale with teacher quality on document images; a controlled sweep over teacher size and document pre-training data would make that relationship explicit and would guide teacher selection.
  • The method has not been tested with predicted OCR text replacing ground-truth source text during training; if that variant works, it would remove the privileged-text dependency and widen the method's applicability to settings without ground-truth source text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes M4Doc, a knowledge-transfer framework for document image machine translation (DIMT). A frozen multimodal large language model (MLLM) receives both a document image and its ground-truth source text, producing "mix-modality" hidden states; an image-only alignment encoder is trained, via a cosine-similarity loss, to reproduce those hidden states from the image alone. At inference the MLLM is discarded, and the aligned representations guide a lightweight Transformer decoder alongside a conventional image encoder. Experiments on DoTA and DITrans, including zero-shot cross-domain, long-context, and complex-layout scenarios, report consistent BLEU/COMET/STEDS gains over DIMTDA and several distillation baselines. Ablations isolate the contribution of the alignment loss and of the MLLM's text input, supporting the claim that the text-conditioned teacher signal is important. The code is publicly released.

Significance. If the empirical claims hold, M4Doc offers a practical recipe for transferring MLLM knowledge into a small, fast DIMT model, which is a meaningful contribution given the computational cost of MLLMs. The training loop is non-circular: the teacher is a frozen external model, and the student never sees the source text at inference. The paper's strengths include a well-motivated architecture, carefully chosen ablations (w/o Lalign, w/o MLLM Text Input), comparison against directly fine-tuned MLLMs, and release of code. However, the load-bearing cross-domain claim is not yet backed by a direct test of whether the image-only encoder reproduces the text-conditioned teacher representation on unseen layouts, and the statistical evidence is thin: all results come from single runs without error bars or significance tests, and the cross-domain test sets contain only 100 images each. The absence of several recent end-to-end DIMT baselines also weakens the stated state-of-the-art claim.

major comments (3)
  1. [§4.2.1, Table 3; §2.2, Eq. (4); §4.3, Table 5] The paper's headline result is cross-domain generalization, but the only ablation that tests the alignment mechanism (w/o Lalign, Table 5) is measured on the in-domain DoTA valid set. It is therefore not established that the cross-domain gains in Table 3 (e.g., 26.71 -> 34.26 BLEU on Political Report after fine-tuning) are attributable to the single-to-mix alignment rather than to the extra parameters of the alignment encoder or the Nougat-initialized Swin features. I request cross-domain versions of the w/o Lalign and w/o Alignment Encoder ablations, and a direct fidelity measurement of HAlign versus HMLLM on out-of-domain images (e.g., cosine similarity on DITrans images), since the T-SNE in Figure 4 is on an unspecified 100-sample set and does not demonstrate transfer to new layouts.
  2. [Tables 2, 3, 5, 6; Appendix A.1] All reported metric values come from a single run with no error bars, confidence intervals, or significance tests. The cross-domain test sets contain only 100 images per domain (Appendix A.1), so the zero-shot gain of 5.54 BLEU (Table 2, line 15 vs. 19) could be within run-to-run noise. Please report multiple seeds or bootstrap confidence intervals, and ideally paired significance tests, for the main results, and state the variance explicitly for the small cross-domain sets.
  3. [§3.3, §4.1, Conclusion] The conclusion claims state-of-the-art translation quality, but the end-to-end DIMT comparison includes only DIMTDA among document-level DIMT systems. The related work lists LayoutDIT (Zhang et al., 2023b) and the fine-to-coarse zoom-out network (Zhang et al., 2025b), both published before this submission; these are not compared. Either add these baselines (or other recent document-level DIMT models) to Table 2, or explicitly justify why they are excluded. Without this, the SOTA claim is not supported.
minor comments (5)
  1. [Section 2 title] The section title reads "M4Doc Mehod"; it should be "M4Doc Method".
  2. [Table 2 caption] The caption contains a duplicated phrase: "θ2 denotes the parameters of the parameters of the layout analysis model and sentence splitting model." Please remove the repetition.
  3. [§2.3, Eq. (6)] Equation (6) conditions on X, but at inference X is not available. Please clarify that X enters only through HAlign during training, and re-write the equation or add a sentence noting that the decoder input is HAlign, which is meant to approximate the text-conditioned representation without access to X.
  4. [Figure 4] The T-SNE visualization is described only as "100 samples" (Section 4.3.2). Please specify which dataset/split these samples come from, and ideally include an out-of-domain sample set to support the cross-domain claim.
  5. [References] There is a duplicated reference entry: Ma et al. 2024a and 2024b have the same title and venue. Please verify and merge or correct.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: M4Doc is standard teacher-student distillation with held-out evaluation.

full rationale

The paper's central derivation is a teacher-student distillation: a frozen MLLM produces text-conditioned hidden states H_MLLM = MLLM(I, X) (Eq. 1), and a trainable image-only alignment encoder produces H_Align = FFN(Swin(I)) (Eqs. 2-3), trained by L_align = 1 - Cos(H_MLLM, H_Align) (Eq. 4). At inference the MLLM is discarded and the decoder uses H_Align. This is not circular: the teacher is an external frozen model, the student never receives X at inference, and the reported gains are BLEU/COMET/STEDS on held-out DoTA and DITrans test sets, including zero-shot cross-domain evaluation (Table 2) and external baselines (ItNet, E2ETIT, PEIT, Seq-KD, MTKD, GPT-4o, Gemini). The ablations (Tables 5-6) confirm the alignment loss and text-conditioned teacher are responsible for the gains, which is the intended mechanism rather than a fitted artifact. Self-citations to DoTA, DITrans, DIMTDA, and LayoutDIT provide datasets and baselines but do not supply the load-bearing argument; the comparison to external methods and additional languages (En-Fr, En-De) anchors the evaluation independently. No equation reduces to its own input, and no fitted parameter is relabeled as a prediction. Therefore no significant circularity; score 1 reflecting minor reliance on author-created benchmarks.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim depends on a small set of hyperparameters (mainly alpha) and on standard transfer-learning assumptions. The paper does not introduce new entities or postulates beyond the alignment target, which is an existing MLLM's hidden state.

free parameters (1)
  • alpha = 1.0
    Weight of the alignment loss in L = alpha * L_align + L_trans. Tuned on the DoTA validation set (Appendix B.2) and set to 1.0.
assumptions (5)
  • domain assumption The representations of a frozen MLLM, conditioned on image and ground-truth text, are a suitable and learnable target for an image-only encoder.
    Invoked in Section 2.2 where HAlign is trained to mimic HMLLM via cosine similarity; if the target is not recoverable from the image, the method fails.
  • domain assumption The ground-truth source text is available as privileged information during training but not at inference, and its effect can be distilled into a vision-only student.
    The teacher uses X in Eq. 1; the student only uses I. The viability of this distillation is the core premise of the paper.
  • ad hoc to paper Cosine similarity over the full sequence of hidden states is an effective alignment objective.
    Chosen empirically over cross-entropy and MSE in Appendix B.1 (Table 8); the choice is made on the validation set, not derived from first principles.
  • domain assumption Initializing the vision encoders from Nougat's encoder and the decoder from a pre-trained text MT decoder benefits downstream performance.
    Section 2.4 states this initialization; standard transfer learning assumption, not independently justified in the paper.
  • domain assumption The metrics (BLEU, COMET, STEDS) and the automated pipeline for COMET adequately measure translation quality and structure preservation.
    Used throughout Section 3.1; the COMET pipeline with Trankit and Sentalign is an approximation that may affect conclusions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation." pith.science (2026). https://pith.science/paper/6ERFC3PV

@misc{pith2026250707572,
  author       = {Pith},
  title        = {Pith review of: Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ERFC3PV}},
  note         = {Machine review of arXiv:2507.07572}
}
read the original abstract

Document Image Machine Translation (DIMT) aims to translate text within document images, facing generalization challenges due to limited training data and the complex interplay between visual and textual information. To address these challenges, we introduce M4Doc, a novel single-to-mix modality alignment framework leveraging Multimodal Large Language Models (MLLMs). M4Doc aligns an image-only encoder with the multimodal representations of an MLLM, pre-trained on large-scale document image datasets. This alignment enables a lightweight DIMT model to learn crucial visual-textual correlations during training. During inference, M4Doc bypasses the MLLM, maintaining computational efficiency while benefiting from its multimodal knowledge. Comprehensive experiments demonstrate substantial improvements in translation quality, especially in cross-domain generalization and challenging document image scenarios.

Figures

Figures reproduced from arXiv: 2507.07572 by the authors.

Figure 1
Figure 1. Different test scenarios of end-to-end DIMT. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The diagram of the proposed M4Doc. During training, the alignment encoder learns to align with the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. BLEU scores of M4Doc models testing on different context length valid sets. Detailed data can be seen in Appendix C. 4.2 Generalization Ability towards Difficult DIMT Scenarios We pay special attention to challenging DIMT scenarios, where our model exhibits advantages through single-to-mix modality alignment. Conse￾quently, we conduct three sets of experiments. 4.2.1 Cross-domain We fine-tune end-to-end DIMT models … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: T-SNE visualization of different representa [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The output samples of M4Doc. For each image pair, the left is the input document image, and the right is the output translations in markdown format after rendering. It is better to zoom in for a clearer view. More samples can be seen in Appendix D. the MLLM. This demon…
Figure 6
Figure 6. Figure 6: BLEU and BLEU-PT scores of M4Doc (Vary￾toy) trained with different α values on the valid set. Detailed data can be seen in Appendix C. OCR this image. Prompts for Textmonkey <System Prompt> You are a helpful assistant. <User Prompt> Read all the text in the image. B De…
Figure 7
Figure 7. Figure 7: The output samples of M4Doc. For each image pair, the left is the input document image, and the right is [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: The output samples of M4Doc in the DoTA dataset. It is better to zoom in for a clearer view. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: The output samples of M4Doc in the political report subset of the DITrans dataset. It is better to zoom in [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: The output samples of M4Doc in the ads & news subset of the DITrans dataset. It is better to zoom in for [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 15 canonical work pages

  1. [1]

    Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. 2024. https://openreview.net/forum?id=fUtxNAKpdV Nougat: Neural optical understanding for academic documents . In The Twelfth International Conference on Learning Representations

    work page 2024

  2. [2]

    Boyu Guan, Yining Zhang, Yang Zhao, and Chengqing Zong. 2025. https://aclanthology.org/2025.coling-main.547/ T ri F ine: A large-scale dataset of vision-audio-subtitle for tri-modal machine translation and benchmark with fine-grained annotated tags . In Proceedings of the 31st International Conference on Computational Linguistics, pages 8215--8231, Abu Dh...

    2025

  3. [3]

    Ryota Hinami, Shonosuke Ishiwatari, Kazuhiko Yasuda, and Yusuke Matsui. 2021. Towards fully automated manga translation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12998--13008

    work page 2021

  4. [4]

    Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al. 2024. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. arXiv preprint arXiv:2403.12895

    arXiv 2024

  5. [5]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net

    2022

  6. [6]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    arXiv 2024

  7. [7]

    Puneet Jain, Orhan Firat, Qi Ge, and Sihang Liang. 2021. Image translation network

    work page 2021

  8. [8]

    Yoon Kim and Alexander M. Rush. 2016. https://doi.org/10.18653/v1/D16-1139 Sequence-level knowledge distillation . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1317--1327, Austin, Texas. Association for Computational Linguistics

    doi:10.18653/v1/d16-1139 2016

Show all 44 references
  1. [9]

    Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, Min Zhang, and Jinsong Su. 2024. Translatotron-v (ison): An end-to-end model for in-image machine translation. arXiv preprint arXiv:2407.02894

    2024 arXiv

  2. [10]

    Zhibin Lan, Jiawei Yu, Xiang Li, Wen Zhang, Jian Luan, Bin Wang, Degen Huang, and Jinsong Su. 2023. https://doi.org/10.18653/v1/2023.acl-long.192 Exploring better text image translation with multimodal codebook . In Proceedings of the 61st Annual Meeting of the Association for...

    2023 doi

  3. [11]

    Yupu Liang, Yaping Zhang, Cong Ma, Zhiyang Zhang, Yang Zhao, Lu Xiang, Chengqing Zong, and Yu Zhou. 2024. https://aclanthology.org/2024.naacl-long.392 Document image machine translation with dynamic multi-pre-trained models assembling . In Proceedings of the 2024 Conference of...

    2024

  4. [12]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 a . https://llava-vl.github.io/blog/2024-01-30-llava-next/ Llava-next: Improved reasoning, ocr, and world knowledge

    2024

  5. [13]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 34892--34916

    2023

  6. [14]

    Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, and Xiang Bai. 2024 b . Textmonkey: An ocr-free large multimodal model for understanding document. arXiv preprint arXiv:2403.04473

    2024 arXiv

  7. [15]

    Cong Ma, Xu Han, Linghui Wu, Yaping Zhang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 a . Modal contrastive learning based end-to-end text image machine translation. IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2023

  8. [16]

    Cong Ma, Yaping Zhang, Mei Tu, Xu Han, Linghui Wu, Yang Zhao, and Yu Zhou. 2022. Improving end-to-end text image translation from the auxiliary text translation task. In 2022 26th International Conference on Pattern Recognition (ICPR), pages 1664--1670. IEEE

    2022

  9. [17]

    Cong Ma, Yaping Zhang, Mei Tu, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 b . E2timt: Efficient and effective modal adapter for text image machine translation. In The 17th International Conference on Document Analysis and Recognition (ICDAR), pages 70--88

    2023

  10. [18]

    Cong Ma, Yaping Zhang, Mei Tu, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 c . Multi-teacher knowledge distillation for text image machine translation. In The 17th International Conference on Document Analysis and Recognition (ICDAR), pages 484--501

    2023

  11. [19]

    Cong Ma, Yaping Zhang, Zhiyang Zhang, Yupu Liang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2024 a . https://aclanthology.org/2024.lrec-main.222/ Born a B aby N et with hierarchical parental supervision for end-to-end text image machine translation . In Proceedings of the 2024 J...

    2024

  12. [20]

    Cong Ma, Yaping Zhang, Zhiyang Zhang, Yupu Liang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2024 b . https://aclanthology.org/2024.lrec-main.222 Born a B aby N et with hierarchical parental supervision for end-to-end text image machine translation . In Proceedings of the 2024 Jo...

    2024

  13. [21]

    Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021. https://doi.org/10.18653/v1/2021.eacl-demos.10 Trankit: A light-weight transformer-based toolkit for multilingual natural language processing . In Proceedings of the 16th Conference of the Europ...

    2021 doi

  14. [22]

    Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek F Wong, Xiaoshuai Sun, and Rongrong Ji. 2024. Anytrans: Translate anytext in the image with large scale models. arXiv preprint arXiv:2406.11432

    2024 arXiv

  15. [23]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.213 COMET : A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702, ...

    2020 doi

  16. [24]

    Nilesh P Sable, Priya Shelke, Ninad Deogaonkar, Nachiket Joshi, Rudra Kabadi, and Tushar Joshi. 2023. Doc-handler: Document scanner, manipulator, and translator based on image and natural language processing. In 2023 International Conference on Emerging Smart Computing and Inf...

    2023

  17. [25]

    Steinthor Steingrimsson, Hrafn Loftsson, and Andy Way. 2023. https://doi.org/10.18653/v1/2023.emnlp-demo.22 S ent A lign: Accurate and scalable sentence alignment . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations...

    2023 doi

  18. [26]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

    2024 arXiv

  19. [27]

    Yanzhi Tian, Xiang Li, Zeming Liu, Yuhang Guo, and Bin Wang. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.1004 In-image neural machine translation with segmented pixel sequence-to-sequence model . In Findings of the Association for Computational Linguistics: EMNLP 202...

    2023 doi

  20. [28]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  21. [29]

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. 2023. Vary: Scaling up the vision vocabulary for large vision-language models. arXiv preprint arXiv:2312.06109

    2023 arXiv

  22. [30]

    Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, En Yu, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. 2024. Small language model meets with reinforced vision vocabulary. arXiv preprint arXiv:2401.12503

    2024 arXiv

  23. [31]

    Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421, 9(1):1

    2023 arXiv

  24. [32]

    Cong Yao. 2023. Docxchain: A powerful open-source toolchain for document parsing and beyond. arXiv preprint arXiv:2310.12430

    2023 arXiv

  25. [33]

    Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Yuhao Dan, Chenlin Zhao, Guohai Xu, Chenliang Li, Junfeng Tian, et al. 2023. mplug-docowl: Modularized multimodal large language model for document understanding. arXiv preprint arXiv:2307.02499

    2023 arXiv

  26. [34]

    Ya-Qi Yu, Minghui Liao, Jihao Wu, Yongxin Liao, Xiaoyu Zheng, and Wei Zeng. 2024. Texthawk: Exploring efficient fine-grained perception of multimodal large language models. arXiv preprint arXiv:2404.09204

    2024 arXiv

  27. [35]

    Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, and Tong Sun. 2023 a . Llavar: Enhanced visual instruction tuning for text-rich image understanding. arXiv preprint arXiv:2306.17107

    2023 arXiv

  28. [36]

    Zhiyang Zhang, Yaping Zhang, Yupu Liang, Cong Ma, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2025 a . https://doi.org/10.1109/TPAMI.2025.3530998 Understand layout and translate text: Unified feature-conductive end-to-end document image translation . IEEE Trans. Pattern ...

    2025

  29. [37]

    Zhiyang Zhang, Yaping Zhang, Yupu Liang, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.673 L ayout DIT : Layout-aware end-to-end document image translation with multi-step conductive decoder . In Findings of the Asso...

    2023 doi

  30. [38]

    Zhiyang Zhang, Yaping Zhang, Yupu Liang, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2025 b . https://aclanthology.org/2025.coling-main.723/ From chaotic OCR words to coherent document: A fine-to-coarse zoom-out network for complex-layout document image translation . In ...

    2025

  31. [39]

    Zhiyang Zhang, Yaping Zhang, Lu Xiang, Yang Zhao, Yu Zhou, and Chengqing Zong. 2023 c . A novel dataset and benchmark analysis on document image translation. In China Conference on Machine Translation, pages 103--115. Springer

    2023

  32. [40]

    Shaolin Zhu, Shangjie Li, Yikun Lei, and Deyi Xiong. 2023. https://doi.org/10.18653/v1/2023.acl-long.751 PEIT : Bridging the modality gap with pre-trained models for end-to-end image translation . In Proceedings of the 61st Annual Meeting of the Association for Computational L...

    2023 doi

  33. [41]

    Wang Zhu, Alekh Agarwal, Mandar Joshi, Robin Jia, Jesse Thomason, and Kristina Toutanova. 2024. https://doi.org/10.18653/v1/2024.naacl-long.465 Efficient end-to-end visual document understanding with rationale distillation . In Proceedings of the 2024 Conference of the North A...

    2024 doi

  34. [42]

    Micha Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. 2016. https://aclanthology.org/L16-1561 The U nited N ations parallel corpus v1.0 . In Proceedings of the Tenth International Conference on Language Resources and Evaluation ( LREC '16) , pages 3530--3534, Portoro z ,...

    2016

  35. [43]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.