Pith. sign in

REVIEW 10 cited by

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.04675 v4 pith:AGYF4FTE submitted 2023-04-10 cs.CL

Wenhao Zhu , Hongyi Liu , Qingxiu Dong , Jingjing Xu , Shujian Huang , Lingpeng Kong , Jiajun Chen , Lei Li This is my paper
classification cs.CL
keywords translationllmsexemplarslanguagelanguageslargeanalysisempirical
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT). In this paper, we systematically investigate the advantages and challenges of LLMs for MMT by answering two questions: 1) How well do LLMs perform in translating massive languages? 2) Which factors affect LLMs' performance in translation? We thoroughly evaluate eight popular LLMs, including ChatGPT and GPT-4. Our empirical results show that translation capabilities of LLMs are continually involving. GPT-4 has beat the strong supervised baseline NLLB in 40.91% of translation directions but still faces a large gap towards the commercial translation system like Google Translate, especially on low-resource languages. Through further analysis, we discover that LLMs exhibit new working patterns when used for MMT. First, LLM can acquire translation ability in a resource-efficient way and generate moderate translation even on zero-resource languages. Second, instruction semantics can surprisingly be ignored when given in-context exemplars. Third, cross-lingual exemplars can provide better task guidance for low-resource translation than exemplars in the same language pairs. Code will be released at: https://github.com/NJUNLP/MMT-LLM.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Label-aware diagnostic reflection plus two-stage outcome GRPO improves same-backbone IE F1 over SFT, with larger gains under relation-extraction domain shift.

  2. Dutch CrowS-Pairs: Adapting a Challenge Dataset for Measuring Social Biases in Language Models for Dutch

    cs.CL 2025-07 conditional novelty 6.0 of 10

    The paper presents a Dutch adaptation of the CrowS-Pairs bias benchmark and reports bias scores for seven masked and two autoregressive language models across nine demographic categories.

  3. Cascade: Token-Sharded Private LLM Inference

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Cascade performs LLM inference by sharding the token sequence across non-colluding nodes, claiming resistance to vocabulary-matching and learning-based reconstruction attacks while being orders of magnitude faster than SMPC.

  4. WETBench: A Benchmark for Detecting Task-Specific Machine-Generated Text on Wikipedia

    cs.CL 2025-07 conditional novelty 6.0 of 10

    WETBench shows that existing machine-generated text detectors, particularly zero-shot methods, underperform on task-specific Wikipedia editing scenarios, with supervised detectors averaging 78% accuracy and zero-shot ...

  5. Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework

    cs.HC 2025-06 conditional novelty 6.0 of 10

    Structured moral prompts, especially first-principles reasoning, improve LLM moral classification accuracy across 12 open models and four benchmarks, and reasoning distillation transfers these gains to a 3B model.

  6. Web-Browsing LLMs Can Access Social Media Profiles and Infer User Demographics

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Web-browsing LLMs can retrieve X profile content and infer demographics with above-chance accuracy in some cases, but the study's evidence is partly confounded by training-data memorization and a heavily reduced synth...

  7. Psychology-Driven Enhancement of Humour Translation

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A decomposition-and-recomposition prompt method for humor translation reports large gains on LLM-based metrics, but the evaluation lacks human validation and statistical checks.

  8. Information Loss in LLMs' Multilingual Translation: The Role of Training Data, Language Proximity, and Language Family

    cs.CL 2025-06 reject novelty 5.0 of 10

    Round-trip translation quality in GPT-4 and Llama 2 is jointly shaped by training data volume and language distance from English, with orthographic, phylogenetic, syntactic, and geographic distances as the strongest p...

  9. Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution

    cs.CR 2025-06 conditional novelty 5.0 of 10

    Prompt injection can make LLM agents leak personal data they observed while executing tasks, with measured attack success rates around 15-20 percent and password leakage much rarer.

  10. A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis

    cs.CL 2025-06 conditional novelty 4.0 of 10

    An LLM agent that reframes beam analysis as OpenSeesPy code generation reaches over 99 percent reliability on a small benchmark, but chiefly because the prompt contains a near-identical solved example.

Pith tools