{"id":"d1715ad1-f9ff-4d33-bee5-96d9c416fc69","arxiv_id":"2412.15453","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"Direct Preference Optimization on Llama-3 improves counterspeech generation on some metrics and languages, but the paper's own table contradicts the claim that it wins everywhere.","lead":"This paper trains Llama-3 with Direct Preference Optimization to generate counterspeech against hate speech, using GPT-4o-made negative examples. It reports that the DPO model beats supervised fine-tuning on some languages and metrics, though its own table shows the opposite for Spanish.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 contradicts the central claim: run3 is not best on all metrics in English (BLEU) and loses to SFT run2 on three metrics in Spanish; the claimed universal DPO advantage fails on the paper's own numbers.","rationale":"The paper's central claim is empirical and comparative: DPO-aligned run3 outperforms SFT runs. The strongest_claim repeats the paper's assertion that run3 wins across all metrics. This is directly checkable from Table 3, and it fails. For example, in Spanish, run3 trails run2 on ROUGE-L (45.3 vs 46.7), BLEU (33.4 vs 33.6), and BERTScore (80.5 vs 81.2); in English, run3's BLEU (40.3) is below run2's (42.1). The paper also says 'significantly outperform' but provides no significance tests, and with 100 test samples the observed gaps are easily within noise. The cross-lingual transfer assumption that the reader listed as weakest is important, but it only becomes relevant if the table first shows a consistent DPO advantage. Since the table does not, the primary load-bearing flaw is the internal data-claim mismatch. The reader did note this contradiction in the rationale, but the formal weakest_assumption points to transfer; hence partial agreement. A rejection is warranted because the main empirical claim is unsupported by the reported evidence.","tokens_in":10195,"tokens_out":4085,"duration_ms":30406,"concrete_test":"Reproduce Table 3 from per-example scores for all 100 test items per language and count, for every language-metric cell, whether run3 strictly exceeds both run1 and run2; any exception (e.g., Spanish ROUGE-L: run2 46.7 vs run3 45.3) falsifies the 'across all metrics' claim. Then run a paired bootstrap or Wilcoxon signed-rank test comparing run3 to run2 per language to test 'significantly outperform'; if no significant difference appears in Spanish or in English BLEU, the central claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that 'DPO-aligned models significantly outperform SFT baselines on CS benchmarks while scaling effectively to multiple languages' is contradicted by the paper's own Table 3. The strongest_claim asserts run3 'outperforms the other runs across all metrics,' but the table does not show this. In English, run3's BLEU is 40.3 versus run2's 42.1. In Spanish, run3 is not best on ROUGE-L (45.3 vs 46.7 for run2), BLEU (33.4 vs 33.6), BERTScore (80.5 vs 81.2), or JudgeLM (873.0 vs 894.5 for run1). The 'significantly outperform' wording also lacks any statistical test; with only 100 test examples per language and no significance reporting, the observed differences could be noise. The untested English-to-other-language transfer assumption is a real concern, but it is downstream of the more immediate problem: the evidence table fails to support the universal superiority claim even before cross-lingual validity is questioned.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a system for multilingual counter-speech generation submitted to a shared task. The authors fine-tune Llama-3 base and instruct models with supervised fine-tuning (SFT) and then apply Direct Preference Optimization (DPO) using ground-truth counter-narratives as preferred responses and GPT-4o-generated statements as rejected responses. Training and alignment are performed on English data only, and the resulting models are evaluated on English, Basque, Italian, and Spanish test sets using BLEU-2, ROUGE-L, BERTScore, JudgeLM, and novelty metrics. The paper claims that the DPO-aligned base model (run3) outperforms both SFT baselines across all metrics and languages, and concludes that DPO 'significantly outperform[s]' SFT while scaling to multiple languages.","tokens_in":10484,"tokens_out":4639,"duration_ms":33405,"significance":"If the claimed results held, the contribution would be a straightforward but useful demonstration that preference optimization improves counter-speech generation, with some evidence that English-only alignment transfers zero-shot to other languages. The paper's strengths are that it uses a standard DPO framework, includes knowledge-grounded prompts, and reports a fairly complete set of automatic metrics. However, the central empirical claim is contradicted by the paper's own Table 3, and the absence of significance testing or cross-lingual validation makes the reported advantages difficult to evaluate. As presented, the findings do not support the abstract's conclusions.","major_comments":[{"comment":"The central claim that run3, the DPO-aligned model, 'outperforms the other runs across all metrics' is factually contradicted by the paper's own table. In English, run3's BLEU is 40.3 versus run2's 42.1. In Spanish, run3 loses to run1 on all four headline metrics (JudgeLM 873.0 vs 894.5; ROUGE-L 45.3 vs 45.6; BLEU 33.4 vs 34.5; BERTScore 80.5 vs 80.6) and to run2 on ROUGE-L (45.3 vs 46.7) and BERTScore (80.5 vs 81.2). The abstract's statement that DPO-aligned models 'significantly outperform SFT baselines' is therefore not supported by the reported evidence.","section":"Section 5, Table 3; Abstract"},{"comment":"No statistical significance tests or confidence intervals are reported for any metric. With only 100 test examples per language, the observed differences between runs (for example, Spanish ROUGE-L 45.3 vs 45.6, or English BLEU 40.3 vs 42.1) may be within random variation. The word 'significantly' in the abstract and the ordering claims in Section 5 are unjustified without such analysis.","section":"Section 5"},{"comment":"The cross-lingual claim is not established. Since supervision and alignment are performed on English only, the paper needs to justify why the same model's metrics in Basque, Italian, and Spanish are meaningful evidence of 'scaling effectively to multiple languages.' No analysis of transfer is provided, such as a comparison with a model trained on the target languages, per-language qualitative examples in those languages, or a discussion of how the English prompt is handled for non-English inputs. Without this, the multilingual conclusions remain unsupported.","section":"Abstract and Section 5"},{"comment":"The DPO setup is underspecified. The paper does not report the number of preference pairs, whether the DPO dataset is per-language or English-only, the DPO beta parameter, or the exact prompt used for generating rejected answers (Figure 1 is referenced but not included). This prevents reproducibility and makes it difficult to assess whether the DPO training is properly configured.","section":"Sections 4.2 and 5"}],"minor_comments":[{"comment":"The metrics labeled 'Gen Len' and 'Novelty' are not defined in the text, and the meaning of 'AVG BLEU-2' and 'AVG ROUGE-L' as percentages (e.g., 48.3) is not clarified.","section":"Section 5"},{"comment":"The claim 'we are the first to investigate the effectiveness of model alignment approaches, particularly Direct Preference Optimization, for generating Counter Speech' is too strong, since the related work itself cites Hengle et al. (2024), which uses RLAIF for counterspeech generation; the scope of the novelty claim should be narrowed.","section":"Section 2"},{"comment":"There are typos in the prompt examples, such as 'survivos' for 'survivors' and 'Which other evidences' for 'What other evidence', which should be corrected.","section":"Appendix A.1"}],"recommendation":"reject","confidential_remarks":"The paper is a shared-task system description with a limited novel contribution, but the main reason for rejection is that its central empirical claim is internally contradicted by Table 3. Even if the claims were softened, the lack of significance testing and cross-lingual validation would leave the paper with little substance; a major revision would essentially require new experimental work. The editor may wish to consider whether a corrected version with reduced claims and proper statistical analysis could be appropriate for resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: this is a shared-task system paper that applies DPO to counterspeech generation, with GPT-4o generating the rejected samples. That combination is genuinely new as far as the cited prior work goes, and the paper is transparent about training details, prompts, and limitations. But the central claim, that DPO-aligned run3 outperforms all runs across all metrics, is contradicted by the paper's own Table 3. That's a load-bearing flaw, not a nitpick.\n\nWhat is actually new: DPO for counterspeech generation has not been done before in the cited literature; the Hengle et al. RLAIF work is adjacent but not DPO. Using GPT-4o to write negative samples is a small but useful twist. The appendix with sample outputs helps the reader see the qualitative differences. The paper also honestly states its constraints (no larger models, simple rejection prompt, manual criteria).\n\nThe soft spots are more than minor. In English, run3's BLEU is 40.3 vs run2's 42.1. In Basque, run3's JudgeLM is 1145 vs run2's 1158. In Spanish, run3 is not best on any of the four main metrics—run1 wins JudgeLM, run2 wins ROUGE-L, BLEU, and BERTScore. So the universal superiority claim fails on the numbers printed in the paper. There are no significance tests, and with 100 test examples per language, the differences could easily be noise. The multilingual claim rests on an untested assumption that English-only alignment transfers to Basque, Italian, and Spanish; the paper gives no cross-lingual validation. The training set is tiny (396 per language, 500 epochs), which invites overfitting, and checkpoint selection is done on the validation set without reporting variance.\n\nThe paper is coherent and honest enough in presentation that I wouldn't call it unserious. It looks like a typical workshop system description. But as a research claim, it overreaches. The authors could fix the claim, add error bars or significance tests, and temper the multilingual assertion; then it would be a solid system report.\n\nFor peer review: I would desk reject this version. The central claim is contradicted by the paper's own data, so a referee would spend the whole time on fundamental issues rather than advancing the work. The idea is worth exploring, but the evidence as presented isn't there.\n\nThat's my take. Let me know if you want to discuss.","headline":"DPO for counterspeech is worth a look, but the paper's central \"DPO wins everywhere\" claim dies on its own Table 3.","tokens_in":11002,"tokens_out":3667,"would_cite":false,"duration_ms":31803,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By aligning a language model with direct preference optimization, this paper produces counter-speech that outperforms supervised fine-tuning baselines across English, Basque, Italian, and Spanish.","keywords":["counter-speech generation","direct preference optimization","hate speech","LLM alignment","multilingual","knowledge grounding","supervised fine-tuning"],"falsifier":"A head-to-head evaluation in which native speakers of Basque, Italian, Spanish, and English rate DPO-aligned versus SFT counter-speech outputs for quality, persuasiveness, and appropriateness; if judges show no preference for the DPO outputs, the paper's central claim collapses.","tokens_in":10042,"feed_emoji":"💬","tokens_out":6975,"duration_ms":53224,"temperature":0.7,"pith_summary":"This paper asks whether Direct Preference Optimization (DPO) makes large language models generate better counter-speech — replies that challenge hate speech with facts and reasoning — than plain supervised fine-tuning (SFT). The authors fine-tune a Llama-3 base and an instruct variant, first with SFT and then with DPO, using the ground-truth counter-narratives as preferred responses and GPT-4o-generated supporting sentences as rejected ones, with background knowledge supplied as additional context. They report that the DPO-aligned base model scores highest across all measured metrics in English and that the same English-trained model also improves scores on Basque, Italian, and Spanish test sets. If correct, this offers a practical recipe for generating counter-speech in multiple languages without per-language training data.","feed_headline":"DPO-tuned LLM beats standard fine-tuning at countering hate speech","feed_subtitle":"English-only alignment yields better counter-narratives across four languages, including Basque, Italian, and Spanish.","key_machinery":"Direct Preference Optimization (DPO), a training objective that aligns a language model with pairwise human preferences without a separate reward model, is the central mechanism. The authors build a preference dataset from the ground-truth counter-narratives (preferred) and GPT-4o-generated hate-speech-supporting sentences (rejected), then continue training from an SFT checkpoint for 80 epochs. Knowledge grounding is added by giving each hate-speech/counter-narrative pair five background-knowledge sentences and instructing the model to base its response on them; the SFT model serves as the reference during DPO training.","core_discovery":"The central claim is that preference-based alignment with DPO, applied on top of a small amount of supervised fine-tuning, yields counter-narratives that are more assertive, more knowledge-grounded, and closer to human references than SFT-only generation. Table 3 shows the DPO-aligned Llama-3 base model (run3) achieving the best JudgeLM, BLEU, ROUGE-L, BERTScore, and novelty scores among the three runs in English, and also the best or near-best scores in Basque, Italian, and Spanish despite all supervision and alignment being done in English. The paper interprets this as evidence that DPO scales across languages and that grounded knowledge combined with preference alignment reduces generic, passive outputs.","pith_inferences":["A native-speaker evaluation in Basque, Italian, and Spanish would test whether the automatic-metric gains translate into human-perceived quality; the paper does not report such a study.","The rejected responses were generated with one simple prompt; varying the prompt or using a rejection-sampling loop could change the preference signal and likely shift DPO gains.","Because the training set is tiny and training runs for hundreds of epochs, an independent evaluation on a larger or held-out dataset would clarify how much of the improvement is robust versus overfit.","The metrics used measure similarity and machine-judged quality, not whether a reader's attitude actually changes; an intervention study would test the real-world premise."],"forward_implications":["DPO-aligned models can be trained on a small dataset of about 1,500 lines and still beat SFT baselines, making the method feasible in low-resource settings.","English-only alignment appears sufficient to improve automatic counter-speech scores in Basque, Italian, and Spanish, removing the need for per-language training data.","The DPO run also increases novelty and generation length, suggesting the model produces more varied and substantive responses rather than memorizing references.","Knowledge grounding combined with DPO produces counter-narratives that directly refute the hate speech rather than offering generic condemnations."],"supporting_citations":[{"why":"Introduces Direct Preference Optimization, the alignment method the paper applies to counter-speech generation.","marker":"Rafailov et al., 2023"},{"why":"Provides the Llama-3 base and instruct models used as the starting point for all runs.","marker":"Dubey et al., 2024"},{"why":"Establishes the knowledge-grounded counter-narrative generation approach that the paper builds on.","marker":"Chung et al., 2021"},{"why":"The model used to generate rejected counter-speech responses for the DPO preference pairs.","marker":"OpenAI, 2023"},{"why":"Supplies JudgeLM, the LLM-based ranking metric that reports the largest gains for the DPO run.","marker":"Zubiaga et al., 2024"},{"why":"Supplies BERTScore, one of the reference-similarity metrics used to compare runs.","marker":"Zhang et al., 2020"},{"why":"Supplies BLEU-2, a metric used in the evaluation table.","marker":"Papineni et al., 2002"},{"why":"Supplies ROUGE-L, another metric used to rank the runs.","marker":"Lin, 2004"}],"fun_headline_variants":["Preference-tuned LLMs write better counterspeech in 4 languages","English-only DPO training boosts counterspeech across languages","Knowledge-grounded DPO beats SFT for multilingual counterspeech","DPO beats SFT for hate speech counters, even across languages"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that English-only supervision and alignment transfer to Basque, Italian, and Spanish, and that the automatic metrics measure counter-speech quality equally in all four languages.","fun_headline_variants_meta":{"raw":{"variants":["Preference-tuned LLMs write better counterspeech in 4 languages","English-only DPO training boosts counterspeech across languages","Knowledge-grounded DPO beats SFT for multilingual counterspeech","DPO beats SFT for hate speech counters, even across languages"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001156,"raw_usage":{"total_tokens":4755,"prompt_tokens":875,"completion_tokens":3880,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":3808}},"tokens_in":491,"tokens_out":3880,"duration_ms":25425,"temperature":1.0,"reasoning_tokens":3808,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:24:42.848456+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A head-to-head evaluation in which native speakers of Basque, Italian, Spanish, and English rate DPO-aligned versus SFT counter-speech outputs for quality, persuasiveness, and appropriateness; if judges show no preference for the DPO outputs, the paper's central claim collapses.","supporting_citations":[],"review_version":1}