Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization

Aakash Mahalingam; Akankshya Kar; Chengtian Xu; Divya Chaudhary; Haoming Chen; Sahil Wadhwa

REVIEW 4 major objections 3 minor 1 cited by

Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization

T0 review · 4 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By aligning a language model with direct preference optimization, this paper produces counter-speech that outperforms supervised fine-tuning baselines across English, Basque, Italian, and Spanish.

desk verdict DPO for counterspeech is worth a look, but the paper's central "DPO wins everywhere" claim dies on its own Table 3. read the letter →

arxiv 2412.15453 v1 pith:LNAXV2WG submitted 2024-12-19 cs.CL cs.AI

Sahil Wadhwa , Chengtian Xu , Haoming Chen , Aakash Mahalingam , Akankshya Kar , Divya Chaudhary This is my paper · ORCID

classification cs.CLcs.AI

keywords counter-speechgenerationdirectpreferenceoptimizationhatespeechLLMalignmentmultilingualknowledgegroundingsupervisedfine-tuning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether Direct Preference Optimization (DPO) makes large language models generate better counter-speech — replies that challenge hate speech with facts and reasoning — than plain supervised fine-tuning (SFT). The authors fine-tune a Llama-3 base and an instruct variant, first with SFT and then with DPO, using the ground-truth counter-narratives as preferred responses and GPT-4o-generated supporting sentences as rejected ones, with background knowledge supplied as additional context. They report that the DPO-aligned base model scores highest across all measured metrics in English and that the same English-trained model also improves scores on Basque, Italian, and Spanish test sets. If correct, this offers a practical recipe for generating counter-speech in multiple languages without per-language training data.

What carries the argument

Direct Preference Optimization (DPO), a training objective that aligns a language model with pairwise human preferences without a separate reward model, is the central mechanism. The authors build a preference dataset from the ground-truth counter-narratives (preferred) and GPT-4o-generated hate-speech-supporting sentences (rejected), then continue training from an SFT checkpoint for 80 epochs. Knowledge grounding is added by giving each hate-speech/counter-narrative pair five background-knowledge sentences and instructing the model to base its response on them; the SFT model serves as the reference during DPO training.

What would settle it

A head-to-head evaluation in which native speakers of Basque, Italian, Spanish, and English rate DPO-aligned versus SFT counter-speech outputs for quality, persuasiveness, and appropriateness; if judges show no preference for the DPO outputs, the paper's central claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that preference-based alignment with DPO, applied on top of a small amount of supervised fine-tuning, yields counter-narratives that are more assertive, more knowledge-grounded, and closer to human references than SFT-only generation. Table 3 shows the DPO-aligned Llama-3 base model (run3) achieving the best JudgeLM, BLEU, ROUGE-L, BERTScore, and novelty scores among the three runs in English, and also the best or near-best scores in Basque, Italian, and Spanish despite all supervision and alignment being done in English. The paper interprets this as evidence that DPO scales across languages and that grounded knowledge combined with preference alignment reduces generic, passive outputs.

Load-bearing premise

The load-bearing premise is that English-only supervision and alignment transfer to Basque, Italian, and Spanish, and that the automatic metrics measure counter-speech quality equally in all four languages.

Editorial extensions

If this is right

DPO-aligned models can be trained on a small dataset of about 1,500 lines and still beat SFT baselines, making the method feasible in low-resource settings.
English-only alignment appears sufficient to improve automatic counter-speech scores in Basque, Italian, and Spanish, removing the need for per-language training data.
The DPO run also increases novelty and generation length, suggesting the model produces more varied and substantive responses rather than memorizing references.
Knowledge grounding combined with DPO produces counter-narratives that directly refute the hate speech rather than offering generic condemnations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A native-speaker evaluation in Basque, Italian, and Spanish would test whether the automatic-metric gains translate into human-perceived quality; the paper does not report such a study.
The rejected responses were generated with one simple prompt; varying the prompt or using a rejection-sampling loop could change the preference signal and likely shift DPO gains.
Because the training set is tiny and training runs for hundreds of epochs, an independent evaluation on a larger or held-out dataset would clarify how much of the improvement is robust versus overfit.
The metrics used measure similarity and machine-judged quality, not whether a reader's attitude actually changes; an intervention study would test the real-world premise.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The headline: this is a shared-task system paper that applies DPO to counterspeech generation, with GPT-4o generating the rejected samples. That combination is genuinely new as far as the cited prior work goes, and the paper is transparent about training details, prompts, and limitations. But the central claim, that DPO-aligned run3 outperforms all runs across all metrics, is contradicted by the paper's own Table 3. That's a load-bearing flaw, not a nitpick.

What is actually new: DPO for counterspeech generation has not been done before in the cited literature; the Hengle et al. RLAIF work is adjacent but not DPO. Using GPT-4o to write negative samples is a small but useful twist. The appendix with sample outputs helps the reader see the qualitative differences. The paper also honestly states its constraints (no larger models, simple rejection prompt, manual criteria).

The soft spots are more than minor. In English, run3's BLEU is 40.3 vs run2's 42.1. In Basque, run3's JudgeLM is 1145 vs run2's 1158. In Spanish, run3 is not best on any of the four main metrics—run1 wins JudgeLM, run2 wins ROUGE-L, BLEU, and BERTScore. So the universal superiority claim fails on the numbers printed in the paper. There are no significance tests, and with 100 test examples per language, the differences could easily be noise. The multilingual claim rests on an untested assumption that English-only alignment transfers to Basque, Italian, and Spanish; the paper gives no cross-lingual validation. The training set is tiny (396 per language, 500 epochs), which invites overfitting, and checkpoint selection is done on the validation set without reporting variance.

The paper is coherent and honest enough in presentation that I wouldn't call it unserious. It looks like a typical workshop system description. But as a research claim, it overreaches. The authors could fix the claim, add error bars or significance tests, and temper the multilingual assertion; then it would be a solid system report.

For peer review: I would desk reject this version. The central claim is contradicted by the paper's own data, so a referee would spend the whole time on fundamental issues rather than advancing the work. The idea is worth exploring, but the evidence as presented isn't there.

That's my take. Let me know if you want to discuss.

Referee Report

4 major / 3 minor

Summary. The paper describes a system for multilingual counter-speech generation submitted to a shared task. The authors fine-tune Llama-3 base and instruct models with supervised fine-tuning (SFT) and then apply Direct Preference Optimization (DPO) using ground-truth counter-narratives as preferred responses and GPT-4o-generated statements as rejected responses. Training and alignment are performed on English data only, and the resulting models are evaluated on English, Basque, Italian, and Spanish test sets using BLEU-2, ROUGE-L, BERTScore, JudgeLM, and novelty metrics. The paper claims that the DPO-aligned base model (run3) outperforms both SFT baselines across all metrics and languages, and concludes that DPO 'significantly outperform[s]' SFT while scaling to multiple languages.

Significance. If the claimed results held, the contribution would be a straightforward but useful demonstration that preference optimization improves counter-speech generation, with some evidence that English-only alignment transfers zero-shot to other languages. The paper's strengths are that it uses a standard DPO framework, includes knowledge-grounded prompts, and reports a fairly complete set of automatic metrics. However, the central empirical claim is contradicted by the paper's own Table 3, and the absence of significance testing or cross-lingual validation makes the reported advantages difficult to evaluate. As presented, the findings do not support the abstract's conclusions.

major comments (4)

[Section 5, Table 3; Abstract] The central claim that run3, the DPO-aligned model, 'outperforms the other runs across all metrics' is factually contradicted by the paper's own table. In English, run3's BLEU is 40.3 versus run2's 42.1. In Spanish, run3 loses to run1 on all four headline metrics (JudgeLM 873.0 vs 894.5; ROUGE-L 45.3 vs 45.6; BLEU 33.4 vs 34.5; BERTScore 80.5 vs 80.6) and to run2 on ROUGE-L (45.3 vs 46.7) and BERTScore (80.5 vs 81.2). The abstract's statement that DPO-aligned models 'significantly outperform SFT baselines' is therefore not supported by the reported evidence.
[Section 5] No statistical significance tests or confidence intervals are reported for any metric. With only 100 test examples per language, the observed differences between runs (for example, Spanish ROUGE-L 45.3 vs 45.6, or English BLEU 40.3 vs 42.1) may be within random variation. The word 'significantly' in the abstract and the ordering claims in Section 5 are unjustified without such analysis.
[Abstract and Section 5] The cross-lingual claim is not established. Since supervision and alignment are performed on English only, the paper needs to justify why the same model's metrics in Basque, Italian, and Spanish are meaningful evidence of 'scaling effectively to multiple languages.' No analysis of transfer is provided, such as a comparison with a model trained on the target languages, per-language qualitative examples in those languages, or a discussion of how the English prompt is handled for non-English inputs. Without this, the multilingual conclusions remain unsupported.
[Sections 4.2 and 5] The DPO setup is underspecified. The paper does not report the number of preference pairs, whether the DPO dataset is per-language or English-only, the DPO beta parameter, or the exact prompt used for generating rejected answers (Figure 1 is referenced but not included). This prevents reproducibility and makes it difficult to assess whether the DPO training is properly configured.

minor comments (3)

[Section 5] The metrics labeled 'Gen Len' and 'Novelty' are not defined in the text, and the meaning of 'AVG BLEU-2' and 'AVG ROUGE-L' as percentages (e.g., 48.3) is not clarified.
[Section 2] The claim 'we are the first to investigate the effectiveness of model alignment approaches, particularly Direct Preference Optimization, for generating Counter Speech' is too strong, since the related work itself cites Hengle et al. (2024), which uses RLAIF for counterspeech generation; the scope of the novelty claim should be narrowed.
[Appendix A.1] There are typos in the prompt examples, such as 'survivos' for 'survivors' and 'Which other evidences' for 'What other evidence', which should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical held-out comparison, and its central DPO claim does not reduce to its own inputs.

full rationale

This is an empirical systems and evaluation paper rather than a derivation chain. The claimed contribution is that DPO-aligned Llama-3 generates better counter-speech than SFT baselines, and the evidence is held-out test-set metrics (BLEU, ROUGE-L, BERTScore, JudgeLM) plus qualitative examples. The preferred DPO examples are human ground-truth counter-narratives from the training split, and the rejected examples are GPT-4o generations; neither is reused as the evaluation target, and the test split is separate from training and validation. No parameter is fitted to the test set and then reported as a prediction, and no equation-level reduction is present. The citations to DPO (Rafailov et al.) and JudgeLM (Zubiaga et al.) are external works by other authors, not self-citations, and there is no invocation of a self-authored uniqueness theorem or ansatz. The manuscript's own Table 3 does not fully support the broad statement that run3 outperforms all runs on all metrics (e.g., English BLEU and several Spanish metrics favor run1 or run2), but that is an internal-consistency and statistical-support concern, not a circularity concern. Honest non-finding is therefore appropriate: score 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central result rests on hand-chosen hyperparameters, an unvalidated assumption about cross-lingual transfer, and the reliability of automatic metrics and ground-truth labels. No new entities are introduced.

free parameters (7)

LoRA rank = 16
Rank of low-rank adaptation for SFT and DPO; chosen by hand.
LoRA alpha = 16
Scaling factor for LoRA; chosen by hand.
SFT learning rate = 2e-4
Learning rate for supervised fine-tuning.
DPO learning rate = 5e-4
Learning rate for Direct Preference Optimization stage.
Training epochs = 500 SFT, 80 DPO
High epoch count due to small training set; can cause overfitting.
Checkpoint epochs = 150 (run1), 200 (run2), 80 (run3)
Selected by validation performance, a model-selection choice.
Max sequence length = 640
Truncation length for training.

assumptions (5)

domain assumption English DPO alignment transfers to Basque, Italian, and Spanish.
Abstract and Section 5 score the same English-trained model in all languages with no cross-lingual validation. If false, the multilingual claim fails.
domain assumption GPT-4o-generated rejected answers are valid negative samples.
Section 4.2 uses these as the DPO rejected set; their quality and appropriateness are not assessed independently.
domain assumption JudgeLM, BLEU, ROUGE, and BERTScore measure counter-speech quality in all four languages.
Section 5 uses these metrics as the basis for the headline claim; no validation of their validity per language.
domain assumption The shared task ground-truth counter-narratives are of sufficient quality to serve as DPO preferred answers and evaluation references.
Section 3 introduces the dataset; no quality audit is reported.
domain assumption Training for up to 500 epochs on 396 examples does not overfit.
Section 5 reports 500 SFT epochs and 80 DPO epochs on a very small training set; overfitting is not addressed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization." pith.science (2026). https://pith.science/paper/LNAXV2WG

@misc{pith2026241215453,
  author       = {Pith},
  title        = {Pith review of: Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LNAXV2WG}},
  note         = {Machine review of arXiv:2412.15453}
}

read the original abstract

The automatic generation of counter-speech (CS) is a critical strategy for addressing hate speech by providing constructive and informed responses. However, existing methods often fail to generate high-quality, impactful, and scalable CS, particularly across diverse linguistic contexts. In this paper, we propose a novel methodology to enhance CS generation by aligning Large Language Models (LLMs) using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Our approach leverages DPO to align LLM outputs with human preferences, ensuring contextually appropriate and linguistically adaptable responses. Additionally, we incorporate knowledge grounding to enhance the factual accuracy and relevance of generated CS. Experimental results demonstrate that DPO-aligned models significantly outperform SFT baselines on CS benchmarks while scaling effectively to multiple languages. These findings highlight the potential of preference-based alignment techniques to advance CS generation across varied linguistic settings. The model supervision and alignment is done in English and the same model is used for reporting metrics across other languages like Basque, Italian, and Spanish.

Figures

Figures reproduced from arXiv: 2412.15453 by the authors.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech
cs.CL 2025-08 conditional novelty 6.0 of 10

NLP counterspeech research increasingly relies on recycled datasets and excludes the affected communities, according to a systematic review and NGO case study.

Reference graph

Works this paper leans on

16 extracted references · 8 canonical work pages · cited by 1 Pith paper

[3]

Preprint, arXiv:2407.21783

The llama 3 herd of models. Preprint, arXiv:2407.21783. He He, Anusha Balakrishnan, Mihail Eric, and Percy Liang

arXiv
[4]

Intent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF

Intent-conditioned and non-toxic counterspeech generation using multi-task instruction tuning with rlaif. Preprint, arXiv:2403.10088. Lingzi Hong, Pengcheng Luo, Eduardo Blanco, and Xi- aoying Song

work page Pith review arXiv
[5]

Preprint, arXiv:2403.17146

Outcome-constrained large lan- guage models for countering hate speech. Preprint, arXiv:2403.17146. Sameera Horawalavithana, Nazim Choudhury, John Skvoretz, and Adriana Iamnitchi

arXiv
[9]

Preprint, arXiv:2403.14938

On zero- shot counterspeech generation by llms. Preprint, arXiv:2403.14938. Carla Schieb and Mike Preuss

arXiv
[11]

(virtual); Conference Location: Online; Conference Date: October 16-17, 2020; Due to the Coronavirus (COVID-19) the conference was conducted virtually. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferre...

work page 2020
[12]

Preprint, arXiv:2307.09288

Llama 2: Open foundation and fine- tuned chat models. Preprint, arXiv:2307.09288. Luis Alfonso Ureña López, Rafael Valencia García, Salud M. Jiménez Zafra, Miguel Ángel García Cum- breras, Daniel García Baena, José Antonio Gar- cía Díaz, and Bharathi Raja Chakravarthi

arXiv
[13]

Procesamiento del lenguaje natu- ral, 71:371–381

Overview of hope at iberlef 2023: Multilingual hope speech detection. Procesamiento del lenguaje natu- ral, 71:371–381. Lingzhi Wang, Xingshan Zeng, Huang Hu, Kam-Fai Wong, and Daxin Jiang

work page 2023
[14]

In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 2127–2137, Punta Cana, Dominican Republic

Re-entry prediction for online conversations via self-supervised learning. In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 2127–2137, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi

work page 2021

Show all 16 references

[15]

Preprint, arXiv:1904.09675

Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. Irune Zubiaga, Aitor Soroa, and Rodrigo Agerri

1904 arXiv
[16]

No jobs should be given to anyone because of their colour or origin

A llm-based ranking method for the evaluation of automatic counter-narrative generation. Preprint, arXiv:2406.15227. A Appendix A.1 Training Prompt In this paper, two versions of the LLaMA-3 model were employed. Each model utilized a distinct prompt format during training. Pro...

arXiv
[2019]

CoRR, abs/1908.09203

Release strategies and the social impacts of language models. CoRR, abs/1908.09203. Dominik Stammbach and Elliott Ash. 2020-10. e- fever: Explanations and summaries for automated fact checking. In Proceedings of the 2020 Truth and Trust Online (TTO 2020), pages 32 – 43, Arling...

1908 arXiv
[2020]

Preprint, arXiv:1909.12940

Hope speech detec- tion: A computational analysis of the voice of peace. Preprint, arXiv:1909.12940. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu

1909 arXiv
[2021]

In Find- ings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 899–914, Online

Towards knowledge-grounded counter narrative generation for hate speech. In Find- ings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 899–914, Online. Associa- tion for Computational Linguistics. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, ...

2021
[2022]

Preprint, arXiv:2203.02155

Training language models to follow instructions with human feedback. Preprint, arXiv:2203.02155. Shriphani Palakodety, Ashiqur R. KhudaBukhsh, and Jaime G. Carbonell

arXiv
[2023]

In Find- ings of the Association for Computational Linguis- tics: EMNLP 2023 , pages 9759–9777, Singapore

Beyond denouncing hate: Strategies for countering implied biases and stereotypes in language. In Find- ings of the Association for Computational Linguis- tics: EMNLP 2023 , pages 9759–9777, Singapore. Association for Computational Linguistics. Long Ouyang, Jeff Wu, Xu Jiang, D...

2023
[2024]

In Find- ings of the Association for Computational Linguis- tics: NAACL 2024, pages 3480–3499, Mexico City, Mexico

NLP for counterspeech against hate: A survey and how-to guide. In Find- ings of the Association for Computational Linguis- tics: NAACL 2024, pages 3480–3499, Mexico City, Mexico. Association for Computational Linguistics. Bharathi Raja Chakravarthi

2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [3]

Preprint, arXiv:2407.21783

The llama 3 herd of models. Preprint, arXiv:2407.21783. He He, Anusha Balakrishnan, Mihail Eric, and Percy Liang

arXiv

[2] [4]

Intent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF

Intent-conditioned and non-toxic counterspeech generation using multi-task instruction tuning with rlaif. Preprint, arXiv:2403.10088. Lingzi Hong, Pengcheng Luo, Eduardo Blanco, and Xi- aoying Song

work page Pith review arXiv

[3] [5]

Preprint, arXiv:2403.17146

Outcome-constrained large lan- guage models for countering hate speech. Preprint, arXiv:2403.17146. Sameera Horawalavithana, Nazim Choudhury, John Skvoretz, and Adriana Iamnitchi

arXiv

[4] [9]

Preprint, arXiv:2403.14938

On zero- shot counterspeech generation by llms. Preprint, arXiv:2403.14938. Carla Schieb and Mike Preuss

arXiv

[5] [11]

(virtual); Conference Location: Online; Conference Date: October 16-17, 2020; Due to the Coronavirus (COVID-19) the conference was conducted virtually. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferre...

work page 2020

[6] [12]

Preprint, arXiv:2307.09288

Llama 2: Open foundation and fine- tuned chat models. Preprint, arXiv:2307.09288. Luis Alfonso Ureña López, Rafael Valencia García, Salud M. Jiménez Zafra, Miguel Ángel García Cum- breras, Daniel García Baena, José Antonio Gar- cía Díaz, and Bharathi Raja Chakravarthi

arXiv

[7] [13]

Procesamiento del lenguaje natu- ral, 71:371–381

Overview of hope at iberlef 2023: Multilingual hope speech detection. Procesamiento del lenguaje natu- ral, 71:371–381. Lingzhi Wang, Xingshan Zeng, Huang Hu, Kam-Fai Wong, and Daxin Jiang

work page 2023

[8] [14]

In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 2127–2137, Punta Cana, Dominican Republic

Re-entry prediction for online conversations via self-supervised learning. In Findings of the Association for Computational Linguistics: EMNLP 2021 , pages 2127–2137, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi

work page 2021