REVIEW 18 cited by
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
This report provides a preliminary evaluation of ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. We adopt the prompts advised by ChatGPT to trigger its translation ability and find that the candidate prompts generally work well with minor performance differences. By evaluating on a number of benchmark test sets, we find that ChatGPT performs competitively with commercial translation products (e.g., Google Translate) on high-resource European languages but lags behind significantly on low-resource or distant languages. As for the translation robustness, ChatGPT does not perform as well as the commercial systems on biomedical abstracts or Reddit comments but exhibits good results on spoken language. Further, we explore an interesting strategy named $\mathbf{pivot~prompting}$ for distant languages, which asks ChatGPT to translate the source sentence into a high-resource pivot language before into the target language, improving the translation performance noticeably. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly boosted, becoming comparable to commercial translation products, even for distant languages. Human analysis on Google Translate and ChatGPT suggests that ChatGPT with GPT-3.5 tends to generate more hallucinations and mis-translation errors while that with GPT-4 makes the least errors. In other words, ChatGPT has already become a good translator. Please refer to our Github project for more details: https://github.com/wxjiao/Is-ChatGPT-A-Good-Translator
Forward citations
Cited by 18 Pith papers
-
On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lens
LLMs underperform on culturally loaded Chinese-to-Japanese translation, while both human and automatic evaluation become unstable when culture is the main variable.
-
ML2B: Benchmarking LLMs on Cross-Lingual ML Pipeline Generation
A new open-source benchmark evaluates LLM-generated end-to-end ML pipelines from Kaggle competition descriptions translated into 13 languages, with 6 private tasks to limit data leakage.
-
Mind the Language Gap in Digital Humanities: LLM-Aided Translation of SKOS Thesauri
WOKIE combines external translation services with LLM-based refinement to translate SKOS thesauri, improving translation quality and boosting ontology matching F1 scores.
-
Decoding Machine Translationese in English-Chinese News: LLMs vs. NMTs
Machine-translated English-to-Chinese news differs from original Chinese news in measurable ways, and LLM and NMT outputs can be partially but not fully distinguished by linguistic features.
-
Can Peter Pan Survive MT? A Stylometric Study of LLMs, NMTs, and HTs in Children's Literature Translation
LLM translations of Peter Pan sit stylistically closer to human translations than NMT outputs do on several child-literature features, but the prompting strategy and possible training-data overlap partly explain the c...
-
BTPD: A Multilingual Hand-curated Dataset of Bengali Transnational Political Discourse Across Online Communities
The paper presents BTPD, a new multilingual dataset of 2,235 hand-curated Bengali political posts from three online platforms, along with a descriptive topic overview.
-
Exploring In-context Example Generation for Machine Translation
DAT generates query-specific in-context translation examples using only an LLM, improving English-to-low-resource translation over zero-shot in most tested languages.
-
Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning
On Swiss legal translation, reinforcement learning with a ChrF reward improves small open models more than supervised fine-tuning, but frontier reasoning models still score higher.
-
$M^2PO$: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation
M2PO combines a QE-plus-alignment reward with a dynamic curriculum and multi-pair DPO loss, and reports WMT21-22 gains for a 7B translation model, but the abstract's WMT23/24 9B parity claims are unsupported.
-
ChatGPT for Code Refactoring: Analyzing Topics, Interaction, and Effective Prompts
An empirical study of developer-ChatGPT refactoring chats yields a 25-theme taxonomy, apology/affirmation signals, and a structured prompt template that reduces conversation turns.
-
Hunyuan-MT Technical Report
Hunyuan-MT and Chimera, a 7B open-source translation model and its multi-candidate fusion variant, claim state-of-the-art multilingual translation including Mandarin to minority languages, with open weights.
-
Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters
A 7B open-weight translation model matches or outperforms far larger commercial systems across 28 languages in automatic and human evaluations.
-
Psychology-Driven Enhancement of Humour Translation
A decomposition-and-recomposition prompt method for humor translation reports large gains on LLM-based metrics, but the evaluation lacks human validation and statistical checks.
-
Information Loss in LLMs' Multilingual Translation: The Role of Training Data, Language Proximity, and Language Family
Round-trip translation quality in GPT-4 and Llama 2 is jointly shaped by training data volume and language distance from English, with orthographic, phylogenetic, syntactic, and geographic distances as the strongest p...
-
Exploring and Mitigating Fawning Hallucinations in Large Language Models
A contrastive decoding method that contrasts a misleading prompt against a neutral rewrite reduces fawning hallucinations in LLMs, though most of the gain comes from the neutral prompt itself.
-
Bangla-Bayanno: A 52K-Pair Bengali Visual Question Answering Dataset with LLM-Assisted Translation Refinement
A 52,650-pair Bengali VQA dataset built by translating VQA v2 with GPT-4, claimed as the largest open-source Bangla benchmark but weakly validated.
-
Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directions
LLM robustness research is organized into adversarial robustness, out-of-distribution robustness, and evaluation, with an accompanying GitHub collection of papers.
-
Semiotic Reconstruction of Destination Expectation Constructs An LLM-Driven Computational Paradigm for Social Media Tourism Analytics
Using GPT-4 to label Chinese travel posts, the study claims leisure and social expectations predict social media engagement more than nature or emotional expectations.
Discussion (0). Sign in to comment.