REVIEW 1 cited by
Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this work, we provide the system description of our submission as part of the English to Lowres Multimodal Translation Task at the Workshop on Asian Translation (WAT2024). We introduce Chitranuvad, a multimodal model that effectively integrates Multilingual LLM and a vision module for Multimodal Translation. Our method uses a ViT image encoder to extract visual representations as visual token embeddings which are projected to the LLM space by an adapter layer and generates translation in an autoregressive fashion. We participated in all the three tracks (Image Captioning, Text only and Multimodal translation tasks) for Indic languages (ie. English translation to Hindi, Bengali and Malyalam) and achieved SOTA results for Hindi in all of them on the Challenge set while remaining competitive for the other languages in the shared task.
Forward citations
Cited by 1 Pith paper
-
Rethinking Multilingual Vision-Language Translation: Dataset, Evaluation, and Adaptation
The authors introduce AibTrans, a multilingual image-text translation benchmark, show that common translation metrics mislead on dense images, and find that balanced multilingual fine-tuning preserves generalization b...
Discussion (0). Continue with ORCID to comment.