Pith. sign in

REVIEW 4 cited by

Towards Making the Most of ChatGPT for Machine Translation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2303.13780 v4 pith:XCJEQBYW submitted 2023-03-24 cs.CL

Keqin Peng , Liang Ding , Qihuang Zhong , Li Shen , Xuebo Liu , Min Zhang , Yuanxin Ouyang , Dacheng Tao This is my paper · ORCID
classification cs.CL
keywords chatgpttranslationpromptsinformationperformancetemperaturedomaintasks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

ChatGPT shows remarkable capabilities for machine translation (MT). Several prior studies have shown that it achieves comparable results to commercial systems for high-resource languages, but lags behind in complex tasks, e.g., low-resource and distant-language-pairs translation. However, they usually adopt simple prompts which can not fully elicit the capability of ChatGPT. In this paper, we aim to further mine ChatGPT's translation ability by revisiting several aspects: temperature, task information, and domain information, and correspondingly propose an optimal temperature setting and two (simple but effective) prompts: Task-Specific Prompts (TSP) and Domain-Specific Prompts (DSP). We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information can further improve ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community. We also explore the effects of advanced in-context learning strategies and find a (negative but interesting) observation: the powerful chain-of-thought prompt leads to word-by-word translation behavior, thus bringing significant translation degradation.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Optimization is Better than Generation: Optimizing Commit Message Leveraging Human-written Commit Message

    cs.SE 2025-01 conditional novelty 6.0 of 10

    A commit-message optimization method that starts from human-written messages and uses GPT-4 plus automated evaluators outperforms message generation and completion methods on three of four quality metrics.

  2. Who Gets Recommended? Investigating Gender, Race, and Country Disparities in Paper Recommendations from Large Language Models

    cs.IR 2024-12 reject novelty 5.0 of 10

    LLM recommendations of important AI research favor recent, well-cited, team-authored papers, but do not measurably over-represent male, white, or developed-country scholars relative to a human-curated benchmark.

  3. Large Language Models for Security Operations Centers: A Comprehensive Survey

    cs.CR 2025-09 conditional novelty 4.0 of 10

    A systematic review of 138 papers classifying LLM applications in SOC workflows by phase, model family, datasets, and maturity.

  4. Bangla-Bayanno: A 52K-Pair Bengali Visual Question Answering Dataset with LLM-Assisted Translation Refinement

    cs.CL 2025-08 conditional novelty 4.0 of 10

    A 52,650-pair Bengali VQA dataset built by translating VQA v2 with GPT-4, claimed as the largest open-source Bangla benchmark but weakly validated.

Pith tools