Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation

Andrew Ho; Anthony Rinaldi; En-Shiun Annie Lee; Jonah Mackey; Rikki Hung; Sarubi Thillainathan; Shravan Nayak; Surangika Ranathunga; Yining Wang

Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.01382 v2 pith:TUXYSPVX submitted 2023-06-02 cs.CL

Shravan Nayak , Surangika Ranathunga , Sarubi Thillainathan , Rikki Hung , Anthony Rinaldi , Yining Wang , Jonah Mackey , Andrew Ho

show 1 more author

En-Shiun Annie Lee

This is my paper · ORCID

classification cs.CL

keywords datadomainfine-tuningmodelspmsswhendomain-specificitft

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS) models flounder when sufficient amounts of parallel data is not available for fine-tuning. This specifically holds for languages missing/under-represented in these models. The problem gets aggravated when the data comes from different domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of PMSS models is extremely beneficial for domain-specific NMT, especially when target domain data is limited/unavailable and the considered languages are missing or under-represented in the PMSS model. We quantify the domain-specific results variations using a domain-divergence test, and show that ITFT can mitigate the impact of domain divergence to some extent.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

SiTSE: Sinhala Text Simplification Dataset and Evaluation
cs.CL 2024-12 conditional novelty 6.0 of 10

A new Sinhala text simplification dataset with 3,000 human-written simplifications is released, and intermediate-task transfer learning on mT5/mBART beats prior zero-resource baselines.

Pith tools