{"id":"4855f5d5-600e-488b-a7b5-64344dc06bc3","arxiv_id":"2508.11831","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Sequential fine-tuning improves target-language euphemism detection in some pairs and models, but the effect is pair-dependent and often no larger than the run-to-run noise.","lead":"This paper tests whether training a model to recognize euphemisms in one language first, then in a second language, helps low-resource targets. The answer is mixed: some pairs improve, especially in mBERT, but many differences are tiny and not tested for statistical significance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significance testing or variance reporting; highlighted gains like EN→YO (+0.003) are within run-to-run noise, and aggregate XLM-R Turkish L2 gains are negative, so the central claim is unsupported.","rationale":"The reader's weakest_assumption was that the 5-trial average macro-F1 is stable, so differences of +0.003 are treated as transfer gains. I agree this is the most load-bearing concern: the paper's central claim relies entirely on point estimates with no variance quantification. My analysis adds a concrete aggravating detail: for XLM-R, the aggregate Turkish L2 results across all four source languages are, if anything, negative (-0.0035 average), undercutting the 'especially for Turkish' claim in the abstract and conclusion. This is not a matter of theoretical correctness but of empirical support: the study does not demonstrate that sequential fine-tuning robustly improves L2 performance beyond random variation. I considered other concerns, such as the potential confound of extra L1 data versus sequential order, but the simultaneous fine-tuning comparisons mitigate that. The statistical-reliability issue is primary. The paper's own Limitations section openly acknowledges the limited number of trials, which is consistent with the reader's conditional verdict. My read does not move the verdict: CONDITIONAL remains appropriate, contingent on the authors providing variance measures, significance tests, and a balanced presentation of negative results. The proposed concrete test—recomputing paired differences and CIs from existing per-trial data—would settle whether the highlighted gains are real.","tokens_in":9084,"tokens_out":7939,"duration_ms":85021,"concrete_test":"Re-analyze the existing per-trial macro-F1 scores for all sequential and monolingual baseline runs (the authors have these but do not report them). For each (model, L1, L2) condition, compute the paired difference (sequential L2 F1 − baseline L2 F1) across the 5 trials, report the 95% confidence interval, and run a Wilcoxon signed-rank test or paired bootstrap. Aggregate differences across all source languages for each L2. If the aggregate CI for XLM-R Turkish includes zero or is negative, the claim 'especially for Turkish' is not supported. Additionally, report the per-trial standard deviation for EN→YO to check whether +0.003 is within noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that sequential fine-tuning with a high-resource L1 improves L2 euphemism detection, especially for low-resource languages like Yoruba and Turkish. The evidence consists of 5-trial average macro-F1 scores without error bars, confidence intervals, or significance tests (Section 4.2, 5.3). The showcases in Section 5.3.1 include EN→YO at 0.812 vs 0.809 baseline (+0.003) and EN→TR at 0.801 vs 0.790 (+0.011) — differences likely within run-to-run variance given test sets of only ~200–300 examples (10% of ~2–3k dataset). More concretely, for XLM-R the claim that Turkish is a beneficiary is contradicted by the aggregate of all four source languages: L2 TR scores are 0.801 (EN), 0.777 (ES), 0.792 (ZH), 0.776 (YO) against a baseline of 0.790, yielding an average difference of −0.0035. Only EN→TR is positive, and it is cherry-picked. The Limitations section admits only 5 trials per pair and no systematic training-duration tests, so the uncertainty is acknowledged but not quantified. Without per-trial scores or paired tests, the observed gains could be random artifacts; the qualitative conclusion could flip under a different random split.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper empirically compares sequential fine-tuning (first L1, then L2), simultaneous fine-tuning, and monolingual baselines for euphemism detection in English, Spanish, Chinese, Turkish, and Yoruba, using XLM-R and mBERT. It reports 5-trial average macro-F1 for all directed language pairs and interprets the results as showing that sequential fine-tuning with a high-resource source language improves L2 euphemism detection, especially for low-resource languages such as Yoruba and Turkish, while XLM-R yields larger gains but more catastrophic forgetting and mBERT gives smaller but more stable gains. The contribution is an empirical mapping of cross-lingual transfer patterns for a figurative-language task, using publicly available PETs datasets.","tokens_in":9480,"tokens_out":6702,"duration_ms":68618,"significance":"If the central claim were robust, the paper would provide a lightweight training recipe for low-resource figurative-language NLP, requiring no translation or additional data. The study has concrete strengths: it uses publicly released euphemism datasets, evaluates all directed pairs between five typologically and resource-diverse languages, compares two multilingual encoders, and releases code and data via GitHub. It also addresses an understudied setting, sequential cross-lingual fine-tuning for figurative language. However, the absence of uncertainty quantification and the aggregate evidence for one of the headline languages mean the quantitative conclusions are not yet established at the level the paper claims.","major_comments":[{"comment":"The manuscript reports only 5-trial average macro-F1 scores, with no standard deviations, confidence intervals, or significance tests. The highlighted gains in §5.3.1 are within plausible run-to-run noise: EN→YO improves by 0.003 (0.812 vs. 0.809) and EN→TR by 0.011 (0.801 vs. 0.790). Given that each test set is roughly 10% of 2.4k–3.2k examples per language, these differences cannot be interpreted without per-trial or paired variance estimates. Because Section 6 directly bases the central claim on these values, the paper currently overstates its support.","section":"§4.2, §5.3.1, Table 4"},{"comment":"The paper repeatedly singles out Turkish as a beneficiary of sequential fine-tuning, but the aggregate XLM-R evidence contradicts this. For L2=TR, the four source-language results are EN 0.801, ES 0.777, ZH 0.792, YO 0.776 against a 0.790 monolingual baseline; the average is therefore below baseline, and only EN→TR is positive. A similar issue affects the 'majority of experiments' phrasing: for XLM-R, only 9 of the 20 directed L2 pairs exceed the baseline. The conclusions should be restricted to the specific pairs that actually improve, or backed by paired tests across all pairs.","section":"Table 4, §5.3.1, §6"},{"comment":"The claim that XLM-R is substantially more prone to catastrophic forgetting than mBERT is inferred from two selected cases (YO→EN: 0.490, YO→ZH: 0.432) plus one agreement coefficient. The paper does not report a systematic L1-retention metric across all 20 sequential runs, nor does it distinguish forgetting from inherent trial variance. The Limitations section itself acknowledges only 5 trials per pair and no systematic training-duration tests. Please provide per-trial scores and at least a paired test for L1 drops to support this conclusion.","section":"§5.3.2, Table 4, Limitations"}],"minor_comments":[{"comment":"The text says 'Table 4 reports the results' for simultaneous fine-tuning, but the corresponding table is Table 3.","section":"§5.2"},{"comment":"The phrase 'always-euphemistic PETs appeared only in training or testing' is ambiguous: it should clarify whether each such PET is assigned exclusively to training or exclusively to testing, and how this was controlled across trials.","section":"§4.2"},{"comment":"The reported Cohen's Kappa values (0.145 and 0.667) are not fully explained. Specify what predictions are being compared and over which instances.","section":"§5.3.2"},{"comment":"'Mandarin Chinese' is used in the introduction while the rest of the paper uses 'Chinese'; align terminology.","section":"§1"},{"comment":"There is a typographical error in the Gururangan et al. reference: 'Don‘t' and 'InProceed' should be corrected.","section":"References"},{"comment":"Minor typos include 'Krippendorf's' (should be 'Krippendorff's'), 'Glowbe' (should be 'GloWbE'), and 'Github' (should be 'GitHub').","section":"Miscellaneous"},{"comment":"The 'Model Archetype' figure is never described or referenced in the main text; consider adding an explanatory paragraph.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a worthwhile question and the experimental design is sensible, but the quantitative support for the main claim is currently too weak. I would require the authors to report per-trial scores, standard deviations or confidence intervals, and paired significance tests, and to revise the Turkish-related conclusions in light of the aggregate Table 4 results. The use of datasets from the same research group is not circular, since they are independently published benchmarks, but the manuscript should more clearly state how the present runs differ from prior task-report results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the empirical study is worth reading, but the central claim as written in the abstract and conclusion is not supported by the paper's own table. What's actually new: a systematic comparison of sequential vs simultaneous vs monolingual fine-tuning for euphemism detection across five languages, including a Turkish set. The authors also test directionality and report forgetting. That's useful for a small subfield. They ship code and data, which is good.\n\nThe main problem is the evidence for the headline. Table 4 shows that for XLM-R only 9 of 20 directed L2 pairs beat the monolingual baseline; 11 are worse. The aggregate gain is essentially zero. The specific examples in Section 5.3.1 include EN to YO (+0.003) and EN to TR (+0.011). Five trials, no standard deviations or significance tests, test sets around 200-300 examples: those differences are noise. The stress test is correct: for Turkish, the average XLM-R L2 score across all four source languages is slightly below the baseline. So Turkish is not a success story.\n\nThe paper is internally inconsistent in another way: the abstract says XLM-R achieves larger gains, but the table shows mBERT improving in 16 of 20 pairs with much larger gains (e.g., EN to TR +0.060, ZH to TR +0.038). XLM-R's only noticeable gains are a few small bumps and one YO to ZH jump, which is cherry-picked.\n\nWhat does work? The catastrophic forgetting result is real and dramatic: YO to EN drops EN from 0.821 to 0.490, YO to ZH drops ZH from 0.878 to 0.432. That's far beyond plausible noise. So the paper's real contribution is showing that sequential fine-tuning can destroy previously learned languages when the source is a low-resource L1 in XLM-R, while mBERT is more robust. The pretraining-coverage vs typology observation is interesting but underpowered with only five languages.\n\nThe limitations section is honest and acknowledges the 5-trial constraint. But the conclusion doesn't match the limitations; it repeats the overclaim. This is fixable in revision: report per-trial F1, run paired tests (or at least show all runs), and reframe the conclusion around the model-specific contrast: mBERT benefits modestly, XLM-R mostly doesn't.\n\nVerdict: deserves a serious referee because the question matters and the data are real, but it needs major revision before acceptance. I would not cite the current version as evidence for transfer gains.","headline":"Overclaimed central finding; the actual data show mBERT benefits from sequential fine-tuning while XLM-R mostly does not, and the highlighted low-resource gains are within noise.","tokens_in":101,"tokens_out":4636,"would_cite":false,"duration_ms":76245,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sequential fine-tuning from a high-resource language improves euphemism detection in target languages, especially low-resource ones.","keywords":["euphemism detection","sequential fine-tuning","cross-lingual transfer","low-resource languages","multilingual transformers","figurative language","XLM-R","mBERT"],"falsifier":"Re-run the sequential and monolingual experiments with many more random splits, say 30, and compute confidence intervals on the sequential-minus-monolingual macro-F1 difference for each language pair. If the intervals for the claimed gains (e.g., English-to-Yoruba +0.003, English-to-Turkish +0.011) include zero, the central transfer claim is not supported.","tokens_in":9028,"feed_emoji":"🌍","tokens_out":8749,"duration_ms":87220,"temperature":0.7,"pith_summary":"This paper asks whether a standard two-stage training recipe—first fine-tune a multilingual model on euphemism detection in one language, then continue fine-tuning on a second language—can transfer figurative-language knowledge across languages. The authors test every ordered pair among English, Spanish, Chinese, Turkish, and Yoruba, on two multilingual transformers, XLM-R and mBERT, and compare sequential fine-tuning against monolingual and simultaneous fine-tuning baselines. The central claim is that sequential fine-tuning from a higher-resource source language improves target-language euphemism detection, especially for low-resource targets like Yoruba and Turkish, without translation or additional labeled data. The paper also finds that transfer success is not predicted by typological similarity; pretraining coverage and dataset structure matter more. If right, the result offers a lightweight adaptation strategy for low-resource figurative-language tasks.","feed_headline":"Sequential fine-tuning boosts low-resource euphemism detection","feed_subtitle":"Training first on a high-resource language, then a low-resource one, beats monolingual baselines—though gains are small and model-dependent.","key_machinery":"The central mechanism is sequential fine-tuning: take an off-the-shelf multilingual transformer, fine-tune it on the source language (L1) to peak validation macro-F1, then continue fine-tuning on the target language (L2) and test on both. The task object is the Potentially Euphemistic Term (PET), a word or phrase labeled as euphemistic or not in context; always-euphemistic PETs are split between training and testing so the model must judge from context rather than memorize terms. Sequential fine-tuning carries the cross-lingual transfer, and the PET split makes the transfer test clean.","core_discovery":"The paper claims that sequentially fine-tuning a multilingual transformer—training first on a source language L1 to its best validation macro-F1, then continuing on a target language L2—raises L2 euphemism detection above the monolingual baseline in most language pairs, with the clearest gains when L1 is high-resource and L2 is low-resource. Simultaneous fine-tuning, by contrast, stays close to baseline. XLM-R produces larger L2 gains but suffers severe catastrophic forgetting of L1 when L1 is low-resource; mBERT is more stable but yields smaller improvements. Typological similarity does not predict transfer: distant pairs such as Spanish-to-Yoruba and English-to-Turkish show gains, while En","pith_inferences":["Because the reported gains are small and based on only five trials per pair, a decisive open question is whether the differences survive multi-seed replication with significance testing.","The paper leaves untested whether sequential fine-tuning acts as a cheap substitute for missing pretraining data; if so, low-resource languages absent from a model's pretraining corpus should consistently improve as L2 targets.","The PET split implies transfer must be contextual rather than lexical-memorized, so a natural extension is to test whether gains persist when source and target PET sets have no translation overlap.","The results may plausibly extend to other figurative-language tasks beyond euphemisms, but that extrapolation is not established by this paper."],"forward_implications":["Sequential fine-tuning can improve euphemism detection in a low-resource target language using only labeled data from a high-resource source language and standard hyperparameters.","When the source language is low-resource, the same recipe can seriously degrade source-language performance, especially on XLM-R, so the choice of which language to train first matters.","mBERT is a more stable choice across language pairs, though its gains are smaller than XLM-R's.","Typological closeness is not a reliable guide for selecting transfer pairs; pretraining coverage and dataset balance should guide the choice.","Sequential fine-tuning produces more distinct differences from baseline than simultaneous fine-tuning, which generally stays near monolingual performance."],"supporting_citations":[{"why":"Introduces the PET (potentially euphemistic terms) framing and the English corpus that anchors the task.","marker":"Gavidia et al., 2022"},{"why":"Extends the PET corpora to Spanish, Chinese, and Yoruba, providing the multilingual evaluation data.","marker":"Lee et al., 2023"},{"why":"Supplies the multilingual euphemism disambiguation datasets and annotation procedures used here.","marker":"Lee et al., 2024"},{"why":"Reports the multilingual euphemism detection task and serves as the data origin for the datasets.","marker":"Lee and Feldman, 2024"},{"why":"Contributes the Turkish euphemism dataset, making the Turkish experiments possible.","marker":"Biyik et al., 2024"},{"why":"Provides XLM-R, whose pretraining-coverage differences drive the observed transfer asymmetries.","marker":"Conneau et al., 2019"},{"why":"Provides mBERT, the comparison model with more uniform pretraining coverage.","marker":"Pires et al., 2019"},{"why":"Supplies the continued-fine-tuning rationale that sequential transfer builds on.","marker":"Gururangan et al., 2020"}],"fun_headline_variants":["Sequential fine-tuning lifts low-resource euphemism detection","L1-first fine-tuning yields low-resource euphemism gains","High-resource L1 first improves low-resource euphemism detection","Cross-lingual euphemism detection: sequential transfer helps","For euphemism detection, fine-tune high-resource before low-resource"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that averaging five random splits gives a stable macro-F1 estimate, so small positive differences (for example, English-to-Yoruba +0.003) can be read as real transfer gains rather than noise; the paper's own Limitations section says only five trials were run per pair and training durations were not systematically varied.","fun_headline_variants_meta":{"raw":{"variants":["Sequential fine-tuning lifts low-resource euphemism detection","L1-first fine-tuning yields low-resource euphemism gains","High-resource L1 first improves low-resource euphemism detection","Cross-lingual euphemism detection: sequential transfer helps","For euphemism detection, fine-tune high-resource before low-resource"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000977,"raw_usage":{"total_tokens":3972,"prompt_tokens":717,"completion_tokens":3255,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":3157}},"tokens_in":461,"tokens_out":3255,"duration_ms":25624,"temperature":1.0,"reasoning_tokens":3157,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:41:36.363304+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the sequential and monolingual experiments with many more random splits, say 30, and compute confidence intervals on the sequential-minus-monolingual macro-F1 difference for each language pair. If the intervals for the claimed gains (e.g., English-to-Yoruba +0.003, English-to-Turkish +0.011) include zero, the central transfer claim is not supported.","supporting_citations":[],"review_version":1}