Pith. sign in

REVIEW 3 major objections 6 minor 22 references

An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning mBART on TF-IDF-ranked synthetic pairs raises Vietnamese-Chinese test BLEU to 38.97.

desk verdict A modest low-resource MT system paper whose tables show real but small BLEU gains, whose abstract oversells an 8% transformer claim, and whose central TF-IDF selection step is not actually isolated by any ablation. read the letter →

arxiv 2501.19314 v1 pith:VG5QQDUG submitted 2025-01-31 cs.CL

Tran Ngoc Son , Nguyen Anh Tu , Nguyen Minh Tri This is my paper · ORCID
classification cs.CL
keywords machinetranslationlow-resourcelanguagesVietnamese-ChinesemBARTdatasynthesisback-translationTF-IDFdomainselectionmultilingualpre-trainedmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a three-step recipe for low-resource machine translation and tests it on Vietnamese-Chinese. An mBART-50 model is first fine-tuned on the existing parallel corpus to serve as an early-bird translator. TF-IDF scores pick 200,000 in-domain sentences from 25 million Vietnamese and 19 million Chinese monolingual sentences, and the early-bird model translates them to synthesize extra parallel pairs. Training on the original pairs plus these synthetic pairs lifts test BLEU to 38.97 for Vietnamese-to-Chinese and to 38.77 for Chinese-to-Vietnamese, beating the base mBART system on both directions. If the recipe is right, low-resource pairs can be improved with corpora that are easier to obtain than parallel data.

What carries the argument

The carrying machinery is mBART-50 plus a data-synthesis loop. mBART-50 is a multilingual sequence-to-sequence model pretrained with denoising objectives on 50 languages; the paper fine-tunes it on the parallel corpus to make an early-bird model, then uses TF-IDF (term frequency-inverse document frequency) scoring to rank monolingual sentences by similarity to the training domain. The early-bird model translates the 200,000 selected sentences for each language, producing synthetic parallel pairs that are merged with the original corpus for the final fine-tuning. The mechanism works by adding parallel supervision that requires no human translation effort and by focusing that supervision on sentences that resemble the domain of the real test data.

What would settle it

Train the same fine-tuning pipeline once with the TF-IDF-selected 200,000 monolingual sentences and once with 200,000 randomly chosen monolingual sentences; if the random-selection model matches or exceeds test BLEU 38.97, then the paper's domain-selection step is not the source of the improvement.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a multilingual pretrained sequence-to-sequence model can be turned into a better low-resource translator by (i) fine-tuning it once on the available parallel data, (ii) using TF-IDF ranking to pull 200,000 in-domain sentences out of about 44 million monolingual sentences, (iii) machine-translating those sentences with the first model to create a synthetic parallel corpus, and (iv) fine-tuning again on the original plus synthetic pairs. In the Vietnamese-Chinese evaluation this raises test BLEU from 38.22 to 38.97 in the Vietnamese-to-Chinese direction and from 35.58 to 38.77 in the Chinese-to-Vietnamese direction (the train-from-scratch variant reaches 38.90). The paper also reports an 8% improvement over a transformer model, though no transformer row appears in the result tables.

Load-bearing premise

The approach assumes that the 200,000 sentences chosen by TF-IDF domain similarity, after being machine-translated by the early-bird model, form training pairs whose quality and domain match improve the final model on the real test set.

Editorial extensions

If this is right

  • Low-resource translation pairs can get a measurable BLEU boost from monolingual text alone, without any new human-created parallel data.
  • The TF-IDF filter shrinks two very large monolingual corpora (about 44 million sentences total) to a 200,000-sentence working set, keeping the synthesis and fine-tuning stages computationally manageable.
  • The same early-bird model can supply synthetic parallel pairs for both translation directions, so the extra cost is one extra fine-tuning run per direction.
  • The final numbers land near or above a widely used commercial engine on the Chinese-to-Vietnamese public test set, while remaining below it on Vietnamese-to-Chinese.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the recipe transfers, it should work for any other language pair already covered by mBART-50; a natural next check is a pair like Vietnamese-Korean, which the paper mentions in passing but does not test.
  • Most of the measured gain sits in the Chinese-to-Vietnamese direction (3.19 BLEU over the base) while Vietnamese-to-Chinese moves only 0.75, so the method's benefit appears asymmetric across directions in these results.
  • Because BLEU measures surface overlap rather than meaning preservation, a human evaluation or a semantic similarity metric would be needed to confirm whether the synthetic-data gain corresponds to genuinely better translations.
  • The train-from-scratch variant (38.90 zh-vi) nearly matches the fine-tuned variant (38.77 zh-vi), suggesting the extra fine-tuning pass is not always the decisive step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a low-resource machine translation pipeline for Vietnamese-Chinese. Starting from the VLSP 2022 parallel corpus (300k pairs), the authors fine-tune mBART-50 to obtain an initial model, use TF-IDF to select 200k domain-related sentences from 25M Vietnamese and 19M Chinese monolingual sentences, translate those sentences with the initial model to synthesize parallel data, and then combine the synthesized data with the original parallel data to train the final model. Experiments report BLEU on the VLSP dev and test sets, comparing their system with Google Translate and the UET engine. The reported internal comparison shows that adding domain-selected synthesized data increases test BLEU from 38.22 to 38.97 for Vietnamese-to-Chinese and from 35.58 to 38.90 for Chinese-to-Vietnamese. The abstract additionally claims an 8% improvement over a transformer model, but no transformer baseline is presented in the paper.

Significance. If the observed gains are attributable to the proposed method, the contribution is a practical and low-cost data-augmentation recipe for a genuinely low-resource language pair: TF-IDF domain filtering followed by back-translation-style synthesis with an early model, using an off-the-shelf multilingual pretrained model. The paper has the concrete strength of evaluating on an external shared-task test set (VLSP 2022) and comparing against commercial systems, which is more informative than using only an internal split. The method itself is simple and potentially reproducible, and the reported hyperparameters are mostly explicit. The significance is moderate rather than high because the absolute gains are small (0.75 and 3.32 BLEU on the two directions) and the central attribution of the gains to the domain-selection mechanism is not established by the presented experiments.

major comments (3)
  1. [Abstract and Section 4.2] The abstract states the proposed scheme 'outperformed 8% compared to the transformer model', but no transformer baseline appears in Table 2 or Table 3. The tables compare only Google Translate, UET Engine, and variants of the authors' own mBART systems. This is a load-bearing claim in the abstract and cannot be verified from the reported results. Either add a transformer-from-scratch baseline trained on the same data and evaluation protocol, or remove/reword the claim.
  2. [Section 3.3.1 and Tables 2-3] The experiments do not isolate the TF-IDF selection mechanism, which is the main novel component. In Table 2, '+ Domain data selection' alone gives a test BLEU of 38.13, lower than the original dataset row (38.22); in Table 3, the selection-only gain is only +0.43. The larger gains appear only when synthesis is also added (+0.75 for vi-zh, +3.32 for zh-vi). Because the synthesis step adds 200k extra training pairs and presumably additional optimization steps, the observed improvement could be due to more data or training dynamics rather than to the TF-IDF domain filtering. A controlled ablation that replaces the TF-IDF-selected sentences with randomly selected monolingual sentences (or otherwise matches the amount of synthetic data) is needed to support the paper's central claim that the domain-selection step is beneficial.
  3. [Section 4.1 and Table 1] There is an inconsistency in the training schedule that affects interpretation of all internal comparisons. The text says 'For each model, we trained for 5 epochs and calculated the BLEU score after each epoch', while Table 1 reports 'Epochs 4'. The paper does not state whether every row in Tables 2 and 3 used the same number of epochs or the same early-stopping criterion, nor does it report a fine-tuning baseline on the original data alone that matches the training budget of the synthesis-augmented runs. Without matching training budgets, the improvements attributed to data synthesis may be confounded with additional epochs or optimization steps. Please clarify the epoch count and add a matched-budget fine-tuning baseline.
minor comments (6)
  1. [Section 3.1] Step 1 says 'Training a Vietnamese-English translation model with mBart-50 using the bilingual dataset', but the paper's task is Vietnamese-Chinese translation; this should be Vietnamese-Chinese.
  2. [Throughout] The model name is spelled inconsistently as 'mBART', 'mBart', and 'mBart-50'; please use one consistent spelling, for example mBART-50.
  3. [Section 4.2] The heading 'Experiemtal Results' contains a typo and should be 'Experimental Results'.
  4. [References] References Liu et al. 2020a and Liu et al. 2020b appear to be the same paper in different venues; cite a single published version and avoid duplicate entries.
  5. [Table 2 and Table 3] The footnote 'UET Engine4' appears inside Table 2 but no corresponding footnote text or external reference is provided; please add the citation or URL for the UET engine system.
  6. [Figure 1] The paper references Figure 1 to illustrate the proposed training flow, but the figure content is not described in the text; please ensure the figure is included and readable in the published version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are evaluated on held-out VLSP 2022 test data and external baselines, and the synthetic-data pipeline is an empirical recipe rather than a definitional or self-citational derivation.

full rationale

The paper's derivation chain is an empirical system description. The early-bird mBART model produces synthetic parallel data from TF-IDF-selected monolingual sentences; this synthetic data is then added to the original parallel corpus and the final model is scored on the public VLSP 2022 dev/test sets. Those test sets are external to the training and synthesis procedure, so the reported BLEU scores are not equal by construction to any fitted quantity or to the synthetic data generation step. The paper does not invoke a uniqueness theorem or any load-bearing self-citation: the cited related work on back-translation (e.g., Ngo et al. 2022b) is from other authors and is used as precedent, not as proof. The main weaknesses are experimental attribution, not circularity: the abstract's '8% compared to the transformer model' is not backed by a transformer row in Tables 2-3, and the benefit of TF-IDF selection over random selection is not isolated by an ablation (indeed, '+ Domain data selection' alone lowers vi-zh test BLEU in Table 2). These are concerns about underdetermination and unsupported comparison, not about the evaluation reducing to its own inputs. Hence a score of 0 is appropriate.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The empirical pipeline rests on standard assumptions about data selection and synthetic data quality rather than on mathematical axioms. No new entities are introduced; the only fitted values are hyperparameters and the hand-chosen monolingual selection budget.

free parameters (3)
  • TF-IDF sentence budget = 200,000 per language
    The number of monolingual sentences selected for back-translation is hand-chosen from 25M Vietnamese and 19M Chinese sentences; no ablation over this budget is reported, so the method's sensitivity to this choice is unknown.
  • Learning rate = 4e-5
    Tuned over five values and selected by validation BLEU; standard hyperparameter fitting.
  • Number of training epochs / early stopping point = 4 to 5, best by dev BLEU
    Section 4.1 says five epochs with best-dev selection, Table 1 lists four epochs; the chosen checkpoint is still a fitted choice.
assumptions (4)
  • domain assumption TF-IDF domain similarity identifies monolingual sentences whose synthetic translations improve the final model.
    Core selection step in Section 3.3.1; no direct validation that the selected sentences are in-domain or that selection alone helps, since Tables 2 and 3 show mixed results for domain selection without synthesis.
  • domain assumption Back-translations by the early bird mBART model are fluent and faithful enough to act as training data.
    The synthesis step in Section 3.1 assumes mBART-50 can produce usable target-side text; no human or automatic quality check of synthetic pairs is reported.
  • domain assumption The VLSP 2022 test set is a fair and representative evaluation of Vietnamese-Chinese translation quality.
    All conclusions rest on this single shared-task benchmark (Section 4.2); no error analysis or second test set is provided.
  • domain assumption The reported BLEU gains generalize beyond the specific dev/test split used for hyperparameter and epoch selection.
    The authors select learning rate and best epoch on dev BLEU and then report test BLEU once; this can overfit the dev set, and the exact multiple-comparison protocol is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese." pith.science (2026). https://pith.science/paper/VG5QQDUG

@misc{pith2026250119314,
  author       = {Pith},
  title        = {Pith review of: An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VG5QQDUG}},
  note         = {Machine review of arXiv:2501.19314}
}
read the original abstract

Despite the rise of recent neural networks in machine translation, those networks do not work well if the training data is insufficient. In this paper, we proposed an approach for machine translation in low-resource languages such as Vietnamese-Chinese. Our proposed method leveraged the power of the multilingual pre-trained language model (mBART) and both Vietnamese and Chinese monolingual corpus. Firstly, we built an early bird machine translation model using the bilingual training dataset. Secondly, we used TF-IDF technique to select sentences from the monolingual corpus which are the most related to domains of the parallel dataset. Finally, the first model was used to synthesize the augmented training data from the selected monolingual corpus for the translation model. Our proposed scheme showed that it outperformed 8% compared to the transformer model. The augmented dataset also pushed the model performance.

Figures

Figures reproduced from arXiv: 2501.19314 by the authors.

Figure 1
Figure 1. Flow of data processing and model training [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 8 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

    arXiv 2014

  4. [4]

    Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

    arXiv 2014

  5. [5]

    Le Duc Cuong and Trang Nguyen Thi Thu. 2020. https://aclanthology.org/2020.vlsp-1.13 V ietnamese- E nglish translation with transformer and back translation in VLSP 2020 machine translation shared task . In Proceedings of the 7th International Workshop on Vietnamese Language and Speech Processing, pages 64--70, Hanoi, Vietnam. Association for Computationa...

    work page 2020

  6. [6]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    arXiv 2018

  7. [7]

    Long Doan, Linh The Nguyen, Nguyen Luong Tran, Thai Hoang, and Dat Quoc Nguyen. 2021. Phomt: A high-quality and large-scale benchmark dataset for vietnamese-english machine translation. arXiv preprint arXiv:2110.12199

    work page Pith review arXiv 2021

  8. [8]

    Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. J. Mach. Learn. Res., 22(107):1--48

    work page 2021

Show all 22 references
  1. [9]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735--1780

    1997

  2. [10]

    Nal Kalchbrenner and Phil Blunsom. 2013. https://aclanthology.org/D13-1176 Recurrent continuous translation models . In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700--1709, Seattle, Washington, USA. Association for Computati...

    2013

  3. [11]

    Philipp Koehn and Rebecca Knowles. 2017. https://doi.org/10.18653/v1/W17-3204 Six challenges for neural machine translation . In Proceedings of the First Workshop on Neural Machine Translation, pages 28--39, Vancouver. Association for Computational Linguistics

    2017 doi

  4. [12]

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020 a . Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726--742

    2020

  5. [13]

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020 b . http://arxiv.org/abs/2001.08210 Multilingual denoising pre-training for neural machine translation . CoRR, abs/2001.08210

    2020 arXiv

  6. [14]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proceedings of the International Conference on Learning Representations (ICLR)

    2019

  7. [15]

    Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025

    2015 arXiv

  8. [16]

    Chinh Ngo, Trieu H Trinh, Long Phan, Hieu Tran, Tai Dang, Hieu Nguyen, Minh Nguyen, and Minh-Thang Luong. 2022 a . Mtet: Multi-domain translation for english and vietnamese. arXiv preprint arXiv:2210.05610

    2022 arXiv

  9. [17]

    Thi-Vinh Ngo, Phuong-Thai Nguyen, Thanh-Le Ha, Khac-Quy Dinh, and Le-Minh Nguyen. 2020. https://www.aclweb.org/anthology/2020.loresmt-1.8 Improving multilingual neural machine translation for low-resource languages: F rench, E nglish - V ietnamese . In Proceedings of the 3rd W...

    2020

  10. [18]

    Thi-Vinh Ngo, Phuong-Thai Nguyen, Van Vinh Nguyen, Thanh-Le Ha, and Le-Minh Nguyen. 2022 b . An efficient method for generating synthetic data for low-resource machine translation: An empirical study of chinese, japanese to vietnamese neural machine translation. Applied Artifi...

    2022

  11. [19]

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27

    2014

  12. [20]

    Yuqing Tang, Chau Tran, Xian Li, Peng - Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. http://arxiv.org/abs/2008.00401 Multilingual translation with extensible multilingual pretraining and finetuning . CoRR, abs/2008.00401

    2020 arXiv

  13. [21]

    Ngo Thi-Vinh, Nguyen Minh-Thuan, Nguyen Hoang Minh Cong, Nguyen Hoang-Quan, Nguyen Phuong-Thai, and Nguyen Van-Vinh. 2020. https://aclanthology.org/2020.vlsp-1.14 The UET - ICTU submissions to the VLSP 2020 news translation task . In Proceedings of the 7th International Worksh...

    2020

  14. [22]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.