ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning

Ao Sun; Ching Wing Kwok; Jiaen Liu; Jianyuan Zhan; Sheng Fu; Sicheng Lai; Takatomo Saito; Wei Dai; Xudong Xiao; Yian Wang

REVIEW 4 major objections 4 minor 12 references

ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ViDove claims that feeding a whole video—visual cues, audio cues, and short- and long-term memory—through a team of specialized translation agents raises long-form subtitle translation quality by 28% in BLEU and 15% in SubER over…

desk verdict A genuinely useful system and benchmark, but the headline performance claim is not yet credible because the evaluation does not control the underlying models and leaves a real risk of domain-memory leakage from the test set itself. read the letter →

arxiv 2507.07306 v1 pith:Z2SD7AGA submitted 2025-07-09 cs.AI cs.CLeess.AS

Yichen Lu , Wei Dai , Jiaen Liu , Ching Wing Kwok , Zongheng Wu , Xudong Xiao , Ao Sun , Sheng Fu

show 4 more authors

Jianyuan Zhan Yian Wang Takatomo Saito Sicheng Lai

This is my paper · ORCID

classification cs.AIcs.CLeess.AS

keywords multimodalmachinetranslationagentvideosubtitlinglong-formmemory-augmentedreasoningretrieval-augmentedgenerationmulti-agentLLMsystemsDoveBench

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ViDove is a translation-agent system that takes a whole video as input rather than just text. The paper's claim is that giving the translator access to audio and visual cues—scene descriptions, speaker turns, audio events, and domain terminology stored in memory—makes long-form video subtitling substantially more accurate than text-only pipelines. On the new DoveBench benchmark, it reports a 28.8% BLEU improvement and a 15.5% SubER reduction over the strongest baseline, Whisper plus DelTA. The paper also releases DoveBench, 17 hours of human-annotated Chinese subtitles from Counter-Strike and StarCraft 2 fan-sub communities, to make this task measurable.

What carries the argument

The mechanism is a retrieval-augmented, multi-agent pipeline with a two-tier memory system. An auditory agent performs speaker diarization, transcription, audio-event detection, and emotion recognition; a vision agent extracts key frames and summarizes them into visual cues. Short-term memory holds translation history and per-chunk multimodal cues, while long-term memory holds domain knowledge and web knowledge. A translator agent retrieves from these memories to produce each subtitle, then a proofreader agent critiques terminology and fluency, and an editor agent makes final corrections while preserving subtitle timestamps.

What would settle it

Keep the LLM backend identical across conditions, remove any memory content drawn from DoveBench domains, and compare ViDove against Whisper plus DelTA on a new long-form video domain; if the BLEU and SubER advantages shrink to noise, the central claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that long-form video subtitling can be decomposed into perception and reasoning agents that share a memory, and that this decomposition beats both single multimodal LLMs and text-only translation agents. On DoveBench, ViDove reaches BLEU 23.51, BLEURT 19.55, and SubER 73.38; relative to Whisper plus DelTA, that is a BLEU gain of 28.8%, a BLEURT gain of 58.9%, and a SubER reduction of 15.5%. Ablations identify the proofreader agent and the domain-knowledge memory as the largest contributors, with the vision module helping mainly on entity-level terms such as names and objects. On the sentence-level BigVideo benchmark, the same system stays competitive, which the paper reads as evidence of generalizability.

Load-bearing premise

The load-bearing premise is that the score gap comes from ViDove's multimodal agent design rather than from its stronger LLM backends or from domain knowledge drawn from the same communities that supplied the test references.

Editorial extensions

If this is right

Long-form video subtitling can be automated end-to-end from raw video to aligned translated subtitles without a separately supplied source transcript.
Visual and auditory context can resolve terms that are ambiguous in text alone, such as game-specific unit names or in-video references.
A memory system with domain knowledge lets a translation agent keep terminology and style consistent across a multi-hour video.
DoveBench gives the field a shared, human-annotated testbed for long-form video subtitling, covering translation accuracy and subtitle alignment.
The ablations suggest that the proofreader-plus-editor post-editing step is a major source of quality, so critique-based refinement matters even when vision is not used.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A cleaner causal test would hold the LLM backend fixed and vary only the multimodal agents; the reported ablations do not separate backend strength from the ViDove design.
Because the domain-knowledge memory was built from the same fan-sub communities that produced DoveBench's ground truth, some of the reported gain may reflect benchmark affinity rather than architecture; testing on an unseen domain would measure the transfer.
The modular design suggests the visual and auditory cueing components could be reused with cheaper or stronger LLM backends, potentially extending to dubbing, live captioning, or accessibility subtitles.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you work on MT or subtitling, but treat the headline numbers with caution. The real contributions are the ViDove pipeline itself—MLLM-driven visual and auditory cues, a long/short-term memory system, and a three-agent translator/proofreader/editor loop—and DoveBench, a 17-hour, 50-video benchmark for long-form video subtitling that the field genuinely lacked. The architecture is sensible, the prompt design is careful, and the paper is honest that absolute scores are modest. That honesty does not extend far enough, though, into the evaluation.

The central problem is that the 28.8% BLEU and 15.5% SubER gains over Whisper+DelTA are not cleanly attributable to the design. ViDove uses GPT-4o and Gemini-2.5-Flash; DelTA's backend is not disclosed, and Whisper is doing the transcription. That is an apples-to-oranges comparison. There are no error bars, no significance tests, and the results come from a single run. On the external BigVideo benchmark, ViDove is merely competitive, not state-of-the-art, which further weakens the claim that multimodal memory is the decisive ingredient.

There is also the stress-test concern about contamination, and I think it lands. DoveBench is built from the FGA and StarPigeon fan-sub communities, the acknowledgments thank those same groups for providing ground truth, and the long-term memory stores "domain knowledge" capturing "specialized community language." If that memory index was populated from the same fan-sub translations that define the reference, ViDove is effectively retrieving reference-derived terminology while baselines are not. The proofreader log shows corrections like "spore crawler" → "孢子爬虫" and "Spire" → "飞龙塔," which look exactly like benchmark-specific vocabulary. The paper never tells us where the memory content came from, and the ablation only removes domain memory on a StarCraft subset, with a small BLEU drop (15.84→14.86) and no test of whether full-benchmark gains survive without it. This is a testable hypothesis, not a proven accusation, but the burden is on the authors to run the control.

Who is this for? Researchers building video translation systems will find the pipeline and benchmark useful as baselines, provided DoveBench is released and the memory provenance is clarified. The paper deserves a serious referee—the artifacts are real and the task is important—but the evaluation needs major revision: controlled backends, significance testing, full ablations, and an independent memory source. I would engage with it, but not cite it as evidence for the performance claim until those controls are in place.

Referee Report

4 major / 4 minor

Summary. This paper presents ViDove, a multimodal translation-agent pipeline for long-form video subtitling and translation. The system combines an auditory agent (diarization, ASR, audio cues), a vision agent (frame extraction, VLM summarization), a long-short term memory system with domain knowledge, and a Translator/Proofreader/Editor post-editing team. The authors also introduce DoveBench, a 50-video/17-hour Chinese-subtitle benchmark collected from Counter-Strike and StarCraft fan-sub groups. On DoveBench, ViDove is reported to outperform Whisper+DelTA by 28.8% BLEU and 15.5% SubER; ablations on a StarCraft subset are used to attribute gains to domain memory, vision, and the proofreader.

Significance. If the reported gains survive controlled evaluation, the paper makes a useful engineering contribution: it demonstrates a modular way to feed visual/audio evidence and persistent domain knowledge into LLM-based translation agents, and it provides a new benchmark and open code for long-form subtitling. The prompt specifications and proofreader logs in the appendix are useful for replication. However, at present the headline comparison is not controlled for model backend or information provenance, and the numerical support for the multimodal and memory claims is thin; these issues must be resolved before the central claim can be accepted.

major comments (4)

[§5.2, §5.3] The 28.8% BLEU / 15.5% SubER improvement is not attributable to the ViDove architecture as stated. ViDove's Translator and Editor run on GPT-4o and its auditory agent on Gemini-2.5-Flash, while Whisper+DelTA uses whisper-large-v3 plus DelTA with an unspecified LLM. A fair comparison needs either same-backbone baselines (e.g., DelTA and ViDove with identical LLMs) or a matched-backend ablation; the manuscript provides neither. Reporting the relative improvement as "over baselines" overstates what the design comparison establishes.
[§4, Appendix B, §3.5.2, Acknowledgments] The provenance of the domain-knowledge memory is not disclosed, and DoveBench's ground-truth annotations are supplied by the same fan-sub communities (FGA, StarPigeon) whose "specialized community language" the memory is designed to capture. Appendix A.3 feeds retrieved context and "term context" to the translator and proofreader, and Table 4 shows corrections of benchmark-specific terms such as "spore crawler" and "Spire". If the memory index was built from the same reference translations used for scoring, ViDove has access to answer-relevant terminology that Whisper+DelTA cannot retrieve, making the DoveBench comparison circular. The paper should state exactly what data populate M^l_domain, release the index or a sample, and report a no-domain-memory run on the full DoveBench test set (not just the StarCraft subset).
[Table 2, §5.4] The quantitative support for the vision module is weak. Removing the domain memory and vision together changes BLEU from 15.84 to 14.56 and leaves BLEURT essentially flat (17.11 to 17.50) on the StarCraft subset; the paper concedes that the visual module has limited impact on the metrics. Since multimodal grounding is a core claimed novelty, this needs either a more sensitive evaluation (e.g., human or targeted entity/terminology accuracy, or a subset where visual cues resolve lexical ambiguity) or a more cautious claim.
[§5.3, Table 1] All headline results are reported as single runs with no variance, confidence intervals, or significance tests, and no decoding details (temperature, random seed, number of samples) are given. Given the modest absolute BLEU/SubER values and the small number of videos (50), the reported differences could lie within run-to-run variability. The authors should report multiple runs or bootstrap intervals, and state the exact inference settings for all systems.

minor comments (4)

[Table 2 vs Table 1] Table 2's ViDove (full) BLEU of 15.84 differs from Table 1's 23.51; the text says the ablation is on a StarCraft subset and uses a "single-column setting", but this should be explained explicitly so readers do not infer inconsistency.
[References] The reference list contains malformed entries, e.g., "et al. 2025" in the Introduction, "Sulubacak et al. 2019" cited for SubSONAR (which is by Gaido et al.), and an anonymous ACL Rolling Review submission; these should be cleaned.
[Abstract] The abstract's "28% BLEU and 15% SubER improvement" should specify relative improvement and the comparison system (Whisper + DelTA) to avoid ambiguity.
[Appendix C.2.2] The statement that Qwen-2.5-Omni videos were manually segmented into ten-second clips because of context limits should be reported in Section 5.2 as part of the baseline's processing conditions, since manual segmentation is a potential confound.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation-chain circularity: ViDove's headline gains are empirical measurements, not consequences of the definitions; the main concern is a possible benchmark/memory provenance overlap, which is an evaluation confound rather than circularity.

full rationale

The claimed result is an empirical performance comparison on DoveBench and BigVideo, not a theorem derived from the system's definitions. ViDove's 28.8% BLEU and 15.5% SubER advantages over Whisper+DelTA are measurements from Table 1, and the paper's ablations (Table 2) attempt to attribute them to components. No fitted parameter is later renamed as a prediction, no uniqueness theorem from the authors' prior work is invoked, and no ansatz is smuggled in via citation. The self-citations (e.g., Lu et al. 2024a,b; Wu et al. 2024b) support component choices but are not load-bearing for the central claim. The strongest validity concern is that DoveBench is constructed from fan-sub communities (Appendix B.1: 'fazeclan galaxy archive' and 'StarPigeon Fan sub group') and ViDove's long-term domain memory is described as capturing 'specialized community language' (Section 3.5.2); if that memory was populated from the same fan-sub translations used as ground truth, the benchmark comparison would be confounded. However, the paper does not state that the domain memory was built from DoveBench's ground-truth annotations, so this remains a possible data-leakage/evaluation-design issue rather than a circularity provable from the text. I therefore find no significant circularity in the derivation chain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central performance claim rests on hand-selected domain knowledge and closed-source backbone choices (free parameters), and on assumptions that visual and audio summaries help translation, that retrieved domain knowledge is beneficial, and that standard metrics capture subtitle quality. These are engineering assumptions rather than mathematical axioms, but they are load-bearing for the empirical claim.

free parameters (3)

Domain knowledge base contents = curated from StarCraft 2 and Counter-Strike fan-sub groups
The long-term memory is populated with domain-specific terminology from the same communities that produced the DoveBench ground truth; the ablation shows removing it lowers BLEU, so the reported gains depend on this hand-selected resource.
Backend model selection = GPT-4o (translation/post-editing), Gemini-2.5-Flash (audio)
The system's performance is tied to the choice of closed-source LLMs, which are not controlled across baselines; the comparison mixes model capability with architectural contribution.
Context retrieval window = previous and next 5 segments
The translator and editor prompts retrieve up to 5 prior and 5 next segments; this window size is chosen by hand and not ablated in the paper.

assumptions (4)

domain assumption Visual and audio cues can be distilled into text summaries that improve translation quality
Core premise of the multimodal design (Sections 3.3 to 3.4); the ablation study in Section 5.4 finds vision has limited impact on BLEU and BLEURT, weakening this assumption for the headline metrics.
domain assumption Retrieved domain knowledge from long-term memory is accurate and beneficial for translation
The system relies on LLaMA-Index retrieval from a manually curated domain base; the quality and coverage of this base are not independently evaluated or released.
domain assumption Standard metrics (BLEU, BLEURT, SubER, sCOMET) adequately capture subtitle translation quality
The paper claims performance improvements using these metrics without human evaluation, and the authors themselves note the absolute scores remain modest.
domain assumption The DoveBench human annotations are correct and representative ground truth
The benchmark is introduced in this paper without inter-annotator agreement or external validation; annotations come from volunteer fan-sub groups.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning." pith.science (2026). https://pith.science/paper/Z2SD7AGA

@misc{pith2026250707306,
  author       = {Pith},
  title        = {Pith review of: ViDove: A Translation Agent System with Multimodal Context and Memory-Augmented Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z2SD7AGA}},
  note         = {Machine review of arXiv:2507.07306}
}

read the original abstract

LLM-based translation agents have achieved highly human-like translation results and are capable of handling longer and more complex contexts with greater efficiency. However, they are typically limited to text-only inputs. In this paper, we introduce ViDove, a translation agent system designed for multimodal input. Inspired by the workflow of human translators, ViDove leverages visual and contextual background information to enhance the translation process. Additionally, we integrate a multimodal memory system and long-short term memory modules enriched with domain-specific knowledge, enabling the agent to perform more accurately and adaptively in real-world scenarios. As a result, ViDove achieves significantly higher translation quality in both subtitle generation and general translation tasks, with a 28% improvement in BLEU scores and a 15% improvement in SubER compared to previous state-of-the-art baselines. Moreover, we introduce DoveBench, a new benchmark for long-form automatic video subtitling and translation, featuring 17 hours of high-quality, human-annotated data. Our code is available here: https://github.com/pigeonai-org/ViDove

Figures

Figures reproduced from arXiv: 2507.07306 by the authors.

**Figure 1.** Architecture of the ViDove Translation Agents System. The system consists of five modules: (i) Vision Agent L ∗ and (ii) Auditory Agent S extract multimodal cues; (iii) Translation Agent L utilizes memory M = {Ms ,Ml} for context-aware translation; (iv) a multi-agent post-editing module refines the output via collaboration; (v) Output Render generates final subtitles and video. • Multimodal Multi-Agent Collaboration… view at source ↗

**Figure 2.** User interface of ViDove [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

**Figure 3.** Vidove Video Cues Summarization for Translation Sample: The image portrays a surreal and dramatic illustration featuring a large, exaggerated face of a bearded man on the left, expressing concern or thoughtfulness. On the right, a dark, patterned background showcases large white flames, with small skeletal figures interacting with them in the foreground. The scene is rendered in a monochromatic color scheme. ORIGINA… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 4 canonical work pages

[1]

0:00:01,229

Each segment should: - Contain a natural sentence or phrase in Simplified Chinese, not too long. - Have a valid start and end time in the format ‘h:mm:ss,ms‘ (e.g., "0:00:01,229"). - Ensure the start time is less than the end time, and that each segment’s start time equals the previous segment’s end time (no overlap or gap). - If uncertain, round timestam...

work page
[2]

- Translate into natural, fluent Simplified Chinese

Translation Guidelines: - First, accurately understand the original audio content. - Translate into natural, fluent Simplified Chinese. - Retain the meaning and tone of the original speech. - Keep proper nouns and technical terms accurate. - Maintain sentence boundaries suitable for subtitle readability

work page
[3]

Simone Conia, Daniel Lee, Min Li, Umar Farooq Min- has, Saloni Potdar, and Yunyao Li

Qwen2-audio technical report.Preprint, arXiv:2407.10759. Simone Conia, Daniel Lee, Min Li, Umar Farooq Min- has, Saloni Potdar, and Yunyao Li. 2024. Towards cross-cultural machine translation with retrieval- augmented generation from multilingual knowledge graphs. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pa...

arXiv 2024
[5]

InProceedings of the 28th ACM International Conference on Multime- dia, MM ’20

Dynamic context-guided capsule network for multimodal machine translation. InProceedings of the 28th ACM International Conference on Multime- dia, MM ’20. ACM. Jerry Liu. 2022. LlamaIndex. Xinwei Long, Jiali Zeng, Fandong Meng, Zhiyuan Ma, Kaiyan Zhang, Bowen Zhou, and Jie Zhou. 2024. Generative multi-modal knowledge retrieval with large language models.P...

arXiv 2022
[6]

Low-resource machine translation through retrieval-augmented llm prompting: A study on the mambai language.Preprint, arXiv:2404.04809. et al. OpenAI. 2024. Gpt-4o system card.Preprint, arXiv:2410.21276. Anishka Peter, Mai Dang, Michael Liu, Joaquin Dominguez, and Nibhrat Lohia. 2024. Multi-agent translation team (matt): Enhancing low-resource language tra...

arXiv 2024
[8]

InProceedings of ACL

Bleurt: Learning robust metrics for text gener- ation. InProceedings of ACL. Huangjun Shen, Liangying Shao, Wenbo Li, Zhibin Lan, Zhanyu Liu, and Jinsong Su. 2024. A survey on multi-modal machine translation: Tasks, methods and challenges.Preprint, arXiv:2405.12669. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Natha...

arXiv 2024
[12]

翻译提供的视频中的说话内容到中文。只需要输出翻译内容原文，不要输出任何解释。

Notes: - Proper nouns and technical terms — remain accurate. - Sentence boundaries — avoid breaking at unnatural pauses. - Chinese grammar and natural fluency. Please provide the transcription and translation in the specified structured format. The output language must be Chinese. ” C.2 Qwen-2.5-Omni C.2.1 Prompts for DoveBench and BigVideo Qwen 2.5 Omni ...

work page
[2016]

pilum" in the source text seems to be a mistake or unclear. It might be intended to refer to

Google’s neural machine translation system: Bridging the gap between human and machine trans- lation.Preprint, arXiv:1609.08144. Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2024c. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation.Prep...

arXiv 2025

Show all 12 references

[2020]

InProceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 61–71, Online

BLEU might be guilty but references are not innocent. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing (EMNLP), pages 61–71, Online. Association for Computational Linguistics. Marco Gaido, Sara Papi, Matteo Negri, Mauro Cettolo, and Lu...

2020 arXiv
[2023]

Thibault Sellam, Dipanjan Das, and Ankur P Parikh

Tptu: Large language model-based ai agents for task planning and tool usage.Preprint, arXiv:2308.03427. Thibault Sellam, Dipanjan Das, and Ankur P Parikh
[2024]

Chen-Chi Chang, Chong-Fu Li, Chu-Hsuan Lee, and Hung-Shin Lee

Retrieving examples from memory for re- trieval augmented neural machine translation: A sys- tematic comparison.Preprint, arXiv:2404.02835. Chen-Chi Chang, Chong-Fu Li, Chu-Hsuan Lee, and Hung-Shin Lee. 2025. Enhancing low-resource mi- nority language translation with llms and...

2025 arXiv
[2025]

Maxime Bouthors, Josep Crego, and Francois Yvon

Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923. Maxime Bouthors, Josep Crego, and Francois Yvon

arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

0:00:01,229

Each segment should: - Contain a natural sentence or phrase in Simplified Chinese, not too long. - Have a valid start and end time in the format ‘h:mm:ss,ms‘ (e.g., "0:00:01,229"). - Ensure the start time is less than the end time, and that each segment’s start time equals the previous segment’s end time (no overlap or gap). - If uncertain, round timestam...

work page

[2] [2]

- Translate into natural, fluent Simplified Chinese

Translation Guidelines: - First, accurately understand the original audio content. - Translate into natural, fluent Simplified Chinese. - Retain the meaning and tone of the original speech. - Keep proper nouns and technical terms accurate. - Maintain sentence boundaries suitable for subtitle readability

work page

[3] [3]

Simone Conia, Daniel Lee, Min Li, Umar Farooq Min- has, Saloni Potdar, and Yunyao Li

Qwen2-audio technical report.Preprint, arXiv:2407.10759. Simone Conia, Daniel Lee, Min Li, Umar Farooq Min- has, Saloni Potdar, and Yunyao Li. 2024. Towards cross-cultural machine translation with retrieval- augmented generation from multilingual knowledge graphs. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pa...

arXiv 2024

[4] [5]

InProceedings of the 28th ACM International Conference on Multime- dia, MM ’20

Dynamic context-guided capsule network for multimodal machine translation. InProceedings of the 28th ACM International Conference on Multime- dia, MM ’20. ACM. Jerry Liu. 2022. LlamaIndex. Xinwei Long, Jiali Zeng, Fandong Meng, Zhiyuan Ma, Kaiyan Zhang, Bowen Zhou, and Jie Zhou. 2024. Generative multi-modal knowledge retrieval with large language models.P...

arXiv 2022

[5] [6]

Low-resource machine translation through retrieval-augmented llm prompting: A study on the mambai language.Preprint, arXiv:2404.04809. et al. OpenAI. 2024. Gpt-4o system card.Preprint, arXiv:2410.21276. Anishka Peter, Mai Dang, Michael Liu, Joaquin Dominguez, and Nibhrat Lohia. 2024. Multi-agent translation team (matt): Enhancing low-resource language tra...

arXiv 2024

[6] [8]

InProceedings of ACL

Bleurt: Learning robust metrics for text gener- ation. InProceedings of ACL. Huangjun Shen, Liangying Shao, Wenbo Li, Zhibin Lan, Zhanyu Liu, and Jinsong Su. 2024. A survey on multi-modal machine translation: Tasks, methods and challenges.Preprint, arXiv:2405.12669. Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Natha...

arXiv 2024

[7] [12]

翻译提供的视频中的说话内容到中文。只需要输出翻译内容原文，不要输出任何解释。

Notes: - Proper nouns and technical terms — remain accurate. - Sentence boundaries — avoid breaking at unnatural pauses. - Chinese grammar and natural fluency. Please provide the transcription and translation in the specified structured format. The output language must be Chinese. ” C.2 Qwen-2.5-Omni C.2.1 Prompts for DoveBench and BigVideo Qwen 2.5 Omni ...

work page

[8] [2016]

pilum" in the source text seems to be a mistake or unclear. It might be intended to refer to

Google’s neural machine translation system: Bridging the gap between human and machine trans- lation.Preprint, arXiv:1609.08144. Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2024c. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation.Prep...

arXiv 2025