Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

CODEOFCONDUCT at Multilingual Counterspeech Generation: A Context-Aware Model for Robust Counterspeech Generation in Low-Resource Languages

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CODEOFCONDUCT, a simulated-annealing pipeline driven by an LLM judge, ranks first for Basque, second for Italian, and third for English and Spanish at MCG-COLING-2025.

desk verdict The reported SOTA is undermined by the paper's own methodology: generation on the test set and post-hoc run selection after seeing official scores. read the letter →

arxiv 2501.00713 v2 pith:XDYPIE6H submitted 2025-01-01 cs.CL

Michael Bennie , Bushi Xiao , Chryseis Xinyi Liu , Demi Zhang , Jian Meng , Alayo Tripp This is my paper · ORCID
classification cs.CL
keywords counterspeechgenerationhatespeechlow-resourcelanguagesBasquesimulatedannealingLLMjudgemultilingualNLPsharedtask
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that counterspeech generation for hate speech can be treated as a search problem rather than a single-pass text-generation problem, and that this search pays off most in low-resource languages. Its system, CODEOFCONDUCT, uses simulated annealing to mutate and expand candidate responses, an LLM judge to score every candidate, and a round-robin tournament to pick the best answers. On the shared task's official evaluation, the system ranked first for Basque, second for Italian, and third for English and Spanish, with all three Basque runs taking the top three positions. The authors contend this makes a no-fine-tuning, test-set-only recipe a viable path for languages where annotated counterspeech data and trained models are scarce. A reader should care because manual counterspeech does not scale, and an automated pipeline that wins in a structurally difficult language suggests the approach transfers.

What carries the argument

The mechanism that carries the argument is the simulated-annealing loop in Algorithm 1, combined with the JudgeLM scoring function. At each iteration the algorithm appends randomly sampled words to candidate counterspeech, scores the results with JudgeLM, selects survivors with probability $P(c')=T^{E(c')}/\sum_{c''\in S}T^{E(c'')}$, and uses LLMs to generate fresh variants from the selected candidates; the temperature $T$ and target score $S_{\mathrm{target}}$ control how long the search explores weak candidates. A second mechanism, the round-robin tournament in Algorithm 3, re-scores the top six candidates per hate-speech instance by pairwise comparison in both orders, averaging the results to reduce position bias and produce a final ranking. The paper then assembles the top-four ranked answers into four run files and uses the official shared-task evaluation script to choose which runs to submit, which is the step that produced the Basque run-4 anomaly.

What would settle it

Rerun the pipeline with the gold counter-narratives held out of the generation step and with run selection made only on the development split, then recompute the MCG-COLING-2025 leaderboard; if CODEOFCONDUCT does not keep its Basque top-three position under that protocol, the ranking claim is refuted. A second check would be native-speaker human ratings of the submitted Basque runs against lower-ranked systems, testing whether the JudgeLM scores correspond to perceived quality.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a simulated-annealing search over counterspeech candidates, scored by the LLM-based JudgeLM, yields top-ranked outputs in four typologically different languages without any training or additional data. Starting from the background-knowledge sentences supplied with each hate-speech instance, the algorithm appends words sampled from language-specific lists and from the hate speech itself, generates new variants with instruction-tuned LLMs, keeps high-scoring candidates through a Boltzmann-like selection rule, and then re-ranks the survivors in a pairwise round-robin tournament. This pipeline placed first in Basque, second in Italian, and third in English and Spanish in MCG-COLING-2025; for Basque, all three submitted runs outranked the other teams' runs. The paper also reports that its fourth-ranked Basque run scored higher than its second-ranked run under the official metric, which is why the final Basque submission used runs 1, 2, and 4.

Load-bearing premise

The load-bearing premise is that it is legitimate to generate answers from the held-out test set and to choose which runs to submit after seeing their scores on the official evaluation script; if that use of the test set is disallowed by the shared task, or if JudgeLM scores do not reflect counterspeech quality, the reported top rankings collapse.

Editorial extensions

If this is right

  • Low-resource languages can reach competitive counterspeech quality with no fine-tuning and no outside training data: the pipeline needs only the test hate-speech pairs and their background knowledge.
  • LLM-judge scores can act as a reward signal for discrete search, not just as a final evaluation metric, so generation quality can be improved by exploring and recombining candidate texts.
  • Pairwise tournament re-ranking changes the identity of the best answers, meaning the final output depends on the set of candidates being compared.
  • The official metric and the LLM judge can disagree about which run is best, as the Basque fourth-ranked run outscored the second-ranked run, so submission selection is a non-trivial part of the system.
  • The pipeline is computationally expensive—about ten GPU-hours per language on an A100—which limits its use in real-time moderation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If JudgeLM is a valid reward signal, the same annealing-plus-judge recipe should transfer to other text-generation tasks with cheap judge-based scoring, such as simplification or style transfer, where candidate mutation is easy to define.
  • The paper's use of the test set for both generation and run selection leaves the reported rankings dependent on task rules; an independent reproduction that holds out gold counter-narratives during generation and selects runs on development data would show whether the ranking reflects general quality.
  • The Basque run-4 anomaly hints that score differences of a few points between submitted runs may be within metric noise, so the leaderboard gaps over other teams deserve a direct human-evaluation check before being read as quality differences.
  • Because Basque required a separate judge (Judge-EUS) while other languages used JudgeLM, the cross-language comparability of scores is uncertain; the Basque first-place result may partly reflect judge-language fit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper describes CODEOFCONDUCT, a counterspeech generation system submitted to the MCG-COLING-2025 shared task. The method uses simulated annealing to iteratively mutate and score candidate responses with the LLM-based JudgeLM evaluator, followed by a round-robin re-ranking stage, and finally produces four run files per language from which the top three are selected for submission. The paper claims state-of-the-art performance, including first place for Basque and top-three ranks in English, Italian, and Spanish. The central evaluation claim is undermined by the procedure described in Sections 3.2 and 3.5: the testing set is used for generation and hyperparameter tuning, and the official evaluation script is run on the four candidate run files to select the three submitted runs. This constitutes selection on the held-out test labels and invalidates the reported leaderboard positions as evidence of generalizable performance.

Significance. If the reported rankings were valid, the simulated-annealing plus LLM-judge pipeline would be an interesting contribution to multilingual counterspeech generation, particularly for a low-resource language such as Basque. The manuscript is unusually transparent about its methodology, and it provides concrete implementation details such as algorithms, hyperparameter tables, model names, and computational cost. These strengths, however, do not compensate for the evaluation protocol: the paper explicitly states that it used the testing set to generate answers and that it ran the official evaluation script to choose the top three runs. Because the final submission is selected after observing test-set scores, the claimed state-of-the-art results cannot be interpreted as an unbiased comparison with other shared-task systems. The paper's contribution is therefore primarily a system description, not a validated empirical claim.

major comments (4)
  1. [§2.3 and §3.2] Section 2.3 describes the testing split as '100 HS-CN pairs (counter-narratives held out as blind test data)', while Section 3.2 states that the authors 'elected to only use the testing set (100 HS-CN pairs for each language) of the data provided to generate our answers.' If the released test files contain the gold counter-narratives, this is direct label leakage, because the generation process could access the reference responses. If the counter-narratives are not actually present, the notation 'HS-CN pairs' is seriously misleading. The paper must clarify exactly what fields are in the released test files; in the current form, the central evaluation claim is invalid under the first reading and unverifiable under the second.
  2. [§3.5] Stage 3 of the methodology says the authors 'ran the evaluation script given by MCG-COLING to find the top 3 runs' and then selected, for Basque, the first-, second-, and fourth-ranked answer files for submission. Running the official evaluation script on candidate submissions requires access to the test-set gold references, and selecting runs after observing those scores means the submitted runs are chosen by optimizing on the test labels. The resulting leaderboard positions are therefore not a valid measure of how the system would perform on unseen data. This directly undermines the abstract's claim of state-of-the-art performance across four languages and the claim that the Basque runs 'outclass all the other runs submitted by other groups.'
  3. [§3.3, §3.4, and §2.4] JudgeLM is used both as the objective function in the simulated-annealing search and in the round-robin re-ranking stage, and it is also listed as one of the official evaluation metrics of the shared task. Consequently, the reported high scores on JudgeLM-based metrics are not an independent assessment of quality: the generation and selection procedures are explicitly optimized toward the same evaluator that is later used to measure success. The paper should report results on metrics that were not used in any selection step, and it should discuss this circularity explicitly, since it affects the interpretation of all JudgeLM-related numbers in the paper.
  4. [§3.3] Hyperparameter tuning is described as being performed on 'a small subset of 4 HS instances (one from each language)' taken from the testing set, since Section 3.2 says no other data were used. Tuning hyperparameters on the test set, even on a small subset, is another form of test-set leakage and further weakens the claim that the reported results reflect generalization to the shared-task test data.
minor comments (4)
  1. [Table 2] Table 2 contains formatting and typographical errors, including 'V ersion' as a column header and 'Meta-Llamaz 3' as a model name; the parameter counts are missing and several entries appear corrupted.
  2. [§3.3, Algorithm 2] The selection probability P(c') = T^{E(c')} / sum T^{E(c'')} is described as favoring higher-scoring candidates, but this is only true when T > 1. The paper does not specify the initial temperature T0 or the increment ΔT, so the claimed monotonic relationship between score and selection probability is not guaranteed by the stated algorithm.
  3. [§3.5 and Figure 7] The text says that for Basque 'run 4 ... scored 3.5 points higher than the file for the second rank answers,' but Figure 7 is not described with enough detail to identify which scoring function produced that number; the figure should state the metric and the comparison procedure.
  4. [§4] The conclusion states that the system's success stems from factors such as the simulated-annealing approach handling Basque's agglutinative morphology, but no analysis or ablation is provided to isolate these factors from the selection effects described in Sections 3.2 and 3.5.

Circularity Check

2 steps flagged · score 7.0 of 10

Reported SOTA rankings are self-confirming: outputs are selected by JudgeLM, which is also the evaluation metric, and submitted runs are chosen by scoring them on the test set with the official script.

  1. fitted input called prediction [Section 3.3, Algorithm 2; Sections 2.4 and Abstract]
    "To prioritize higher-quality candidates while still allowing exploration of the search space, we compute selection probabilities using a Boltzmann-like distribution (Algorithm 2 from the appendix): P (c′) = T E(c′) P c′′∈S T E(c′′). ... Evaluation of the shared task employs both traditional metrics (BLEU, ROUGE, BERTScore, Novelty) and JudgeLM based on LLM."

    JudgeLM supplies the score E(c′) that drives both candidate selection in Stage 1 (via the Boltzmann probability) and final answer ranking in Stage 2's round-robin comparisons. Because JudgeLM is also an official evaluation metric, the high JudgeLM scores reported for the generated counterspeech are not independent evidence of quality: the procedure was explicitly constructed to maximize that same score. The effectiveness claim therefore reduces to a self-consistent optimization loop rather than an external prediction.

  2. fitted input called prediction [Sections 2.3, 3.2, and 3.5]
    "Testing: 100 HS-CN pairs (counter-narratives held out as blind test data). ... we elected to only use the testing set (100 HS-CN pairs for each language) of the data provided to generate our answers. ... we ran the evaluation script given by MCG-COLING to find the top 3 runs."

    The test split—the same split whose counter-narratives are held out as blind—is used both to generate the candidate outputs and, via the official evaluation script, to decide which three of the four candidate runs to submit. The reported top rankings are therefore a test-set selection maximum, not an out-of-sample prediction: the run files were scored with the official metric and the best-scoring ones were chosen. If the test files literally contain HS-CN pairs, the gold counter-narratives were accessible during generation; even without direct access, selecting runs by their test-set scores destroys the blindness of the evaluation and makes the reported leaderboard rank an artifact of fitting the submission choice to the evaluation labels.

full rationale

The paper's central claim is that CODEOFCONDUCT demonstrates state-of-the-art performance in the MCG-COLING-2025 shared task. Two parts of the derivation chain undermine this claim. First, JudgeLM is used both as the optimization objective (Stage 1 selection probabilities and Stage 2 round-robin ranking) and as an official evaluation metric; the high JudgeLM scores are therefore partly produced by the selection process, not independently measured. Second, the authors generated answers from the testing split and then ran the official MCG-COLING evaluation script to choose the top three of four runs; this is selection on the test set, so the leaderboard rank is a test-set maximum rather than a hold-out prediction. The paper even describes the test split as holding out counter-narratives as blind data while saying the system used the testing set for generation, which at minimum destroys the blindness of the evaluation and at worst exposes the system to gold counter-narratives. There is no meaningful self-citation chain here; the circularity is that the reported prediction reduces to optimizing the evaluation target on the evaluation data. Score 7 reflects that the central ranking claim is substantially compromised, though the generation method itself is not logically equivalent to its inputs by definition.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The pipeline rests on several unverified choices: JudgeLM as the objective, a custom Boltzmann-like acceptance rule, unreported temperature values, hand-picked LLMs, and the use of test-set scoring to select runs. None of these are derived from first principles, and most are not ablated. The system introduces no new physical or ontological entities, so the invented-entities ledger is empty.

free parameters (7)
  • Max iterations Nmax = 8
    Tuned in Table A.2 on 4 HS instances (one per language); 8 iterations with 6 candidates gave average high score 10.
  • Candidates per loop k = 6
    Tuned alongside Nmax on the same 4-instance subset.
  • Target score Starget = 9
    Chosen by the authors; answers scoring 10 were treated as meeting the target during tuning.
  • Initial temperature T0 and increment ΔT = Not reported
    Algorithm 1 depends on T0 and ΔT, but their values are never given.
  • Round-robin pool size = 6
    Section 3.4 says only top 6 answers were used in the quadratic tournament to limit cost.
  • Language-specific word lists = Not specified
    Stage 1 samples words from 'predefined word lists' and from the hate speech, but the lists and sampling counts are not provided.
  • Choice of LLM generators = 8B-72B models listed in Table 2
    The models are selected by hand; no comparison or ablation justifies the choices.
assumptions (5)
  • domain assumption JudgeLM and Judge-EUS scores are valid proxies for counterspeech quality
    Used in Stage 1 (E(c') selection) and Stage 2 (round-robin ranking); if judges reward lexical similarity over cultural or factual quality, the whole optimization target is misaligned.
  • domain assumption Background knowledge alone is a sufficient seed for generating counterspeech
    Algorithm 1 initializes c0 from the provided background knowledge; no external knowledge base or retrieval is used.
  • ad hoc to paper The test set can be used to generate answers and the official evaluation script can be run on candidate submissions
    Sections 3.2 and 3.5 explicitly use the testing set and the MCG-COLING evaluation script; this is only sound if gold counter-narratives are not seen and run selection is allowed.
  • ad hoc to paper Boltzmann-style selection with P(c') = T^{E(c')}/sum works for this search
    Algorithm 2 proposes this distribution without derivation; the unusual T^E form lacks the usual exp(E/T) normalization.
  • domain assumption Appending words sampled from the hate speech and word lists improves counterspeech
    Section 3.3 claims this enriches vocabulary and contextual grounding; no ablation tests this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CODEOFCONDUCT at Multilingual Counterspeech Generation: A Context-Aware Model for Robust Counterspeech Generation in Low-Resource Languages." pith.science (2026). https://pith.science/paper/XDYPIE6H

@misc{pith2026250100713,
  author       = {Pith},
  title        = {Pith review of: CODEOFCONDUCT at Multilingual Counterspeech Generation: A Context-Aware Model for Robust Counterspeech Generation in Low-Resource Languages},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XDYPIE6H}},
  note         = {Machine review of arXiv:2501.00713}
}
read the original abstract

This paper introduces a context-aware model for robust counterspeech generation, which achieved significant success in the MCG-COLING-2025 shared task. Our approach particularly excelled in low-resource language settings. By leveraging a simulated annealing algorithm fine-tuned on multilingual datasets, the model generates factually accurate responses to hate speech. We demonstrate state-of-the-art performance across four languages (Basque, English, Italian, and Spanish), with our system ranking first for Basque, second for Italian, and third for both English and Spanish. Notably, our model swept all three top positions for Basque, highlighting its effectiveness in low-resource scenarios. Evaluation of the shared task employs both traditional metrics (BLEU, ROUGE, BERTScore, Novelty) and JudgeLM based on LLM. We present a detailed analysis of our results, including an empirical evaluation of the model performance and comprehensive score distributions across evaluation metrics. This work contributes to the growing body of research on multilingual counterspeech generation, offering insights into developing robust models that can adapt to diverse linguistic and cultural contexts in the fight against online hate speech.

Figures

Figures reproduced from arXiv: 2501.00713 by the authors.

Figure 2
Figure 2. Box and whisker charts that compare the original scored value of a CS answer from stage 1 to the re-scored vales from stage 2. We used these optimized parameters to generate sentences for MCG-COLING 2025 where we set the target scores of each CS answer (Starget) to be 9. After generating answers for each of the 400 total instances of hate speech, we were left with a set of a 6,915 answers. The distribution of the in… view at source ↗
Figure 1
Figure 1. A histogram of scores E(c) for every counter￾speech generated from Algorithm 1 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Chart depicting the JudgeLM scores for each Basque run. Bars drawn in yellow represent the results from [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 7
Figure 7. Figure 7: Scoring that compares the 4 runs of generated [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 4
Figure 4. Figure 4: Chart depicting the ROUGE-L scores for each [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Chart depicting the BLEU scores for each [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Chart depicting the BERT scores for each [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech

    cs.CL 2025-08 conditional novelty 6.0 of 10

    NLP counterspeech research increasingly relies on recycled datasets and excludes the affected communities, according to a systematic review and NGO case study.

Reference graph

Works this paper leans on

21 extracted references · 10 canonical work pages · cited by 1 Pith paper

  1. [1]

    Rodrigo Agerri and et al. 2020. https://aclanthology.org/2020.lrec-1.588 Give your text representation models some love: the case for B asque . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4781--4788, Marseille, France. European Language Resources Association

    work page 2020

  2. [2]

    Bianca Cepollaro, Maxime Lepoutre, and Robert Mark Simpson. 2023. https://doi.org/10.1111/phc3.12890 Counterspeech . Philosophy Compass, 18(1):e12890

    doi:10.1111/phc3.12890 2023

  3. [3]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8440--8451

    work page 2020

  4. [4]

    Ona de Gibert, Naiara Perez, Aitor Garc \' a-Pablos, and Montse Cuadros. 2018. https://doi.org/10.18653/v1/W18-5102 Hate speech dataset from a white supremacy forum . In Proceedings of the 2nd Workshop on Abusive Language Online ( ALW 2) , pages 11--20, Brussels, Belgium. Association for Computational Linguistics

    doi:10.18653/v1/w18-5102 2018

  5. [5]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171--4186

    2019

  6. [6]

    Aamir Faisal, Muhammad Bilal Zafar, and Muhammad Imran Malik. 2021. Low-resource multilingual hate speech detection: A comparative study of classical and neural approaches. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 20(6):1--20

    work page 2021

  7. [7]

    Margherita Fanton, Helena Bonaldi, Serra Sinem Tekiro g lu, and Marco Guerini. 2021. https://doi.org/10.18653/v1/2021.acl-long.250 Human-in-the-loop for data collection: a multi-target counter narrative dataset to fight online hate speech . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internationa...

    doi:10.18653/v1/2021.acl-long.250 2021

  8. [8]

    Siqi Feng, Junjie Hu, Graham Neubig, and Yiming Yang. 2021. A survey of data augmentation approaches for nlp. In Proceedings of the 2021 Annual Meeting of the Association for Computational Linguistics (ACL), pages 45--55

    work page 2021

Show all 21 references
  1. [9]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74--81

    2004

  2. [10]

    Yonatan Lupu, Richard Sear, Nicolas Vel \'a squez, Rhys Leahy, Nicholas Johnson Restrepo, Beth Goldberg, and Neil F Johnson. 2023. https://doi.org/10.1371/journal.pone.0278511 Offline events and online hate . PloS one, 18(1):e0278511

    2023 doi

  3. [11]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311--318

    2002

  4. [12]

    Aashima Poudhar, Ioannis Konstas, and Gavin Abercrombie. 2024. https://doi.org/10.18653/v1/2024.woah-1.20 A strategy labelled dataset of counterspeech . In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 256--265, Mexico City, Mexico. Association f...

    2024 doi

  5. [13]

    Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4757--4766

    2019

  6. [14]

    Bertie Vidgen and Leon Derczynski. 2020. https://arxiv.org/abs/2004.01670 Directions in abusive language training data: Garbage in, garbage out . CoRR, abs/2004.01670

    2020 arXiv

  7. [15]

    Bertie Vidgen, Alex Harris, Dong Nguyen, Rebekah Tromble, Scott Hale, and Helen Margetts. 2019. https://doi.org/10.18653/v1/W19-3509 Challenges and frontiers in abusive content detection . In Proceedings of the Third Workshop on Abusive Language Online, pages 80--93, Florence,...

    2019 doi

  8. [16]

    Ke Wang and Xiaojun Wan. 2018. https://api.semanticscholar.org/CorpusID:51609768 Sentigan: Generating sentimental texts via mixture adversarial networks . In International Joint Conference on Artificial Intelligence

    2018

  9. [17]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019. https://arxiv.org/abs/1904.09675 Bertscore: Evaluating text generation with BERT . CoRR, abs/1904.09675

    2019 arXiv

  10. [18]

    Arkaitz Zubiaga. 2024. Judgelm: A llm-based ranking method for evaluating automatic counter-narrative generation. In Proceedings of COLING 2024

    2024

  11. [19]

    Irune Zubiaga, Aitor Soroa, and Rodrigo Agerri. 2024. https://api.semanticscholar.org/CorpusID:273190311 Ixa at refutes 2024: Leveraging language models for counter narrative generation . In IberLEF@SEPLN

    2024

  12. [20]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  13. [21]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.