Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement

Bo Xu; Hongfei Lin; Junyu Lu; Kaichun Wang; Kai Ma; Kelaiti Xiao; Liang Yang; Roy Ka-Wei Lee

REVIEW 3 major objections 6 minor 1 cited by

Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs give their most confident offensive-language verdicts on exactly the tweets human annotators disagree about.

desk verdict Solid agreement-stratified evaluation of LLMs on offensive language; the 'overconfident' conclusion is a stretch given the confidence proxy. read the letter →

arxiv 2502.06207 v3 pith:N43N3Y2L submitted 2025-02-10 cs.CL cs.AI

Junyu Lu , Kai Ma , Kaichun Wang , Kelaiti Xiao , Roy Ka-Wei Lee , Bo Xu , Liang Yang , Hongfei Lin This is my paper · ORCID

classification cs.CLcs.AI

keywords annotationdisagreementoffensivelanguagedetectionLLMconfidenceoverconfidenceself-consistencyinstructionfine-tuningfew-shotlearningMD-Agreement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether large language models behave like reliable judges of offensive language when the humans who labeled the data could not agree. It finds that they do not: on tweets where five annotators split, model accuracy falls below 65 percent, yet the models still emit high confidence, so an automated moderator would be most assertive precisely where human judgment is most divided. The paper further argues that this is not a fixed flaw: showing models disagreement samples in few-shot prompts or instruction fine-tuning improves both binary accuracy and the match between model confidence and the human agreement level. If this is right, treating offensive-language moderation as binary classification misses the central failure mode, and disagreement-aware training is a concrete repair.

What carries the argument

The load-bearing objects are the MD-Agreement dataset's soft labels and the self-consistency confidence estimate. MD-Agreement contains 10,753 tweets each annotated by five trained raters; soft labels are the averaged zero-one votes, bucketed into unanimous, mild, and weak agreement, and the corpus is used because prior validation showed its disagreements come from genuine ambiguity rather than labeling error. Confidence is derived by resampling each model at temperatures 0, 0.25, 0.5, 0.75, and 1 and averaging the binary predictions, following the self-consistency method. The alignment claim is carried by the mean squared error between that confidence and the soft label, supported by Spearman's rank correlation and Cohen's kappa; the few-shot and fine-tuning results show these metrics improving when disagreement samples enter training.

What would settle it

Take the low-agreement subset and measure each model's confidence with internal token probabilities or a proper scoring rule such as the Brier score or expected calibration error, instead of temperature-averaged hard predictions. If the paper's overconfidence claim is right, those calibrated confidences will still sit near 1.0 on tweets whose human agreement is only around 0.6; if the claim is a measurement artifact, the gap will shrink or disappear.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a pattern and a prescription. Across twelve open- and closed-source LLMs on the MD-Agreement corpus, zero-shot accuracy and F1 are strong on unanimously labeled tweets and collapse on weakly agreed tweets, with every model scoring below 65 percent on A0 samples and with models biased toward calling ambiguous content offensive. Confidence measured by self-consistency, averaging hard predictions over five temperature settings, stays high on low-agreement samples, so the mean squared error against soft labels roughly quadruples when moving from unanimous to weak agreement, and even the best model, GPT-4o, assigns high confidence across the whole agreement spectrum. The corollary the paper establishes is training-side: including disagreement samples in few-shot learning improves generalization, and instruction fine-tuning with a balanced mix of agreement levels, with medium-agreement A+ samples offering the best single-category trade-off, improves both detection accuracy and the alignment of model confidence with human disagreement.

Load-bearing premise

The overconfidence result assumes that the correct confidence for a model on a disputed tweet is exactly the fraction of human annotators who called it offensive, and that averaging the model's yes-no predictions over a few random samples faithfully measures its confidence; if either assumption fails, the gap between model confidence and human agreement is partly an artifact of comparing averaged binary outputs with fractional human labels.

Editorial extensions

If this is right

A moderation system that trusts an LLM's confidence is most likely to be wrong, and most firmly wrong, on borderline content, so confidence scores cannot be used on their own to route ambiguous cases to human review.
Low-agreement samples behave as a hard floor for scaling: increasing model size helps unanimous cases substantially but barely moves accuracy on ambiguous cases, so scaling alone will not resolve subjective disagreement.
Training data composition is a lever: mixing unanimous and disagreement samples beats any single agreement level, while including too many agreement categories can slightly reduce accuracy, so selection should balance clarity and ambiguity.
Fine-tuning with medium-agreement samples gives the best accuracy-calibration trade-off in-domain and also generalizes best out-of-distribution, suggesting that medium agreement is a useful proxy for moderate data complexity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If this result transfers to other subjective annotation tasks such as humor, misogyny, or stance detection, then the disagreement rate of a sample could serve as a cheap, annotation-free curriculum signal for selecting training data, since the paper shows agreement level tracks both accuracy and confidence alignment.
The paper compares averaged binary model outputs to averaged binary human labels; a stronger test of the overconfidence claim would use token-level probabilities or proper scoring rules such as the Brier score or expected calibration error, and until that is done the magnitude of the overconfidence gap may be partly a measurement choice, even though the accuracy collapse is not.
In deployment, the finding implies a two-stage moderation design: act automatically only on inputs whose agreement profile is learnable, and explicitly model annotation distributions rather than majority vote, which requires datasets with unaggregated labels.
A testable extension is that models trained on disagreement samples should show a sharper drop in false positives on ambiguous non-offensive tweets, since the paper identifies that category as the main error type.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look. The paper systematically evaluates 12 LLMs on MD-Agreement, stratifying by annotator agreement, and the headline accuracy result is real: every model drops sharply on low-agreement (A0) samples, and scaling doesn't fix it. That is a clean, reproducible finding, and the appendix replication with Qwen supports it. The linguistic-feature analysis (sarcasm and rhetorical questions being hardest) is a genuinely useful addition, and the authors are upfront about the single-dataset limitation.

The soft spot is the confidence story. They measure confidence as the mean of hard predictions across five temperatures and compare it to the mean of five binary human labels (Eqs. 1-2). That gives a distributional-mismatch score, not a calibration error. A model that says 'offensive' 5 times out of 5 on a 3-2 human split gets penalized 0.16 by their MSE, while the proper Brier against the individual labels would be 0.4. High kappa across temperatures is self-consistency, not overconfidence in the Bayesian sense. So 'overconfident' is too strong a word for what they measured. The abstract's claim that disagreement samples in training improve both accuracy and alignment is also not uniformly true: GPT-4o few-shot with A0 actually worsens A0 MSE (0.1923 to 0.1979 in Table 5). The Qwen replication goes the other way, which is interesting but undercuts the general claim.

The authors acknowledge the metric limitation and the single dataset. That helps, but the title and abstract still overstate. A revision that reframes the finding as 'LLMs do not reproduce the human label distribution on ambiguous items' rather than 'LLMs are overconfident' would be more honest, and adding a token-probability calibration measure would strengthen it. The OOD analysis in B.5 is thin, covering one dataset with accuracy only.

Bottom line: the empirical backbone is solid and worth refereeing. The central accuracy finding will be useful to anyone building moderation pipelines with LLMs. I'd send it to review with a clear request to revise the confidence interpretation.

Referee Report

3 major / 6 minor

Summary. The paper evaluates 12 large language models on the MD-Agreement offensive language dataset, which provides five binary annotations per tweet and three agreement levels (A++, A+, A0). In zero-shot classification, accuracy and F1 decline sharply as agreement decreases, with all models scoring below 65% accuracy on A0 samples. The authors define model confidence as the mean of five hard predictions sampled at temperatures 0, 0.25, 0.5, 0.75, and 1, and compute the MSE and Spearman correlation between this confidence and the soft label (the mean of the five human binary labels). They report that confidence alignment weakens on low-agreement items while models remain highly self-consistent under temperature resampling, which they interpret as overconfidence. In few-shot and instruction fine-tuning experiments, they find that including disagreement samples generally improves overall accuracy and, on some subsets, reduces the MSE, with Qwen replications and released code. The paper's central interpretive claim is that LLMs are overconfident on ambiguous inputs and that disagreement-aware training improves both accuracy and alignment with human judgment.

Significance. If the overconfidence finding holds, it has direct practical relevance for content moderation, because models would be systematically unreliable precisely where human judgment is split. The empirical sweep is a strength: twelve models from closed and open families, a well-established disagreement-annotated benchmark, transparent prompts, and replication experiments with Qwen models. The accuracy decline across agreement levels is consistent and robust. However, the paper's two headline claims—overconfidence in ambiguous cases and improved alignment from disagreement-aware training—both rest on a metric whose statistical interpretation is problematic. The current analysis does not yet establish that the observed behavior constitutes overconfidence in a calibrated sense, rather than a distributional mismatch between averaged binary model outputs and averaged binary human labels. The study is a useful contribution to the literature on LLM evaluation under annotation disagreement, but the central claim needs reanalysis with proper scoring rules or a clear reframing.

major comments (3)

[Section 3.2, Appendix A.2 (Eqs. 1-2)] The MSE in Eq. (2) compares the mean of five binary human labels to the mean of five binary model predictions. This is not a proper scoring rule for the individual-label prediction problem. For an item with three offensive and two non-offensive labels (y=0.6), a model outputting c=1 receives MSE (1-0.6)^2=0.16, whereas the Brier score computed against the five individual labels is 0.4. The metric therefore rewards extreme predictions on split samples, and the reported A0 MSE of approximately 0.19-0.23 reflects distributional mismatch between the model's sampling distribution and the human vote distribution, not necessarily overconfidence in the model's own beliefs. Because the abstract's 'overconfident' claim and the 'improved alignment' claims in Section 4 both rely on this metric, the authors should reanalyze the data with a proper scoring rule (e.g., per-label Brier score, expected calibration error, or reliability diagrams) or explicitly reframe the finding as a distributional comparison.
[Section 3.2, Figure 2] The use of Cohen's kappa across temperature resamples as evidence of overconfidence is not valid for that purpose. Kappa measures self-consistency: a model that always returns the same hard label at every temperature has kappa equal to 1 regardless of whether that label is correct or whether the item is genuinely ambiguous. The reported kappa above 0.75 on A0 shows only that temperature sampling does not change the binary decision; it does not show that the model's confidence exceeds its accuracy. To support the overconfidence conclusion, the authors need to compare confidence against accuracy within A0 (e.g., reliability curves or confidence-accuracy histograms) or demonstrate that the observed self-consistency exceeds what a calibrated model would exhibit on the same items.
[Section 4.1, Table 5] The abstract and Section 4 claim that training on disagreement samples improves alignment with human judgment, but on the A0 subset the few-shot results in Table 5 show the MSE increasing from 0.1923 (zero-shot) to 0.1979 with w/ A0 and to 0.2022 with w/ A++/0. The overall MSE reduction is driven primarily by the A++ and A+ subsets. The alignment-improvement claim should be qualified, or additional evidence such as a proper-score calibration analysis on A0 should be provided to show that disagreement-aware training actually improves calibration where agreement is weakest.

minor comments (6)

[Section 2.1, Table 1] The caption contains a typo, 'Eamples', which should be 'Examples'.
[Tables 3-6 and B1-B2] The column header 'Ao' appears in several tables and should be 'A0' for consistency with the notation defined in Section 2.1.
[Section 4.2] The text states 'using LLaMa3-7B as the backbone', but all tables list LLaMa3-8B; the exact model variant should be clarified.
[Appendix B.5, Table B3] The three rows labeled 'w/ A++' with different numerical values are likely intended to be 'w/ A++', 'w/ A+', and 'w/ A0'; the table labels should be corrected.
[Section 3.1, finding (4)] The text cites 'as shown in Figure B3' for accuracy on offensive and non-offensive subsets, but Appendix B.6's Figure B3 shows linguistic-feature performance; the figure reference should be corrected.
[Sections 4.1-4.2] The cross-references to Table 5 and Table 6 appear swapped: Section 4.1 describes few-shot GPT-4o results yet refers to Table 6, while Section 4.2 describes instruction fine-tuning yet refers to Table 5; the references should be fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the accuracy and alignment results are held-out empirical measurements, and the confidence metric is an explicit operational choice rather than a fitted input.

full rationale

The paper contains no fitted parameters, no derivation chain whose inputs equal its outputs, and no load-bearing self-citation. The central empirical claims are direct measurements: zero-shot accuracy on A++/A+/A0 subsets (Table 3), confidence computed as the mean of five temperature-resampled hard predictions (Appendix A.2, Eq. 1), and held-out test results after few-shot prompting or instruction fine-tuning (Tables 5, 6, B1, B2). The MSE alignment metric (Eq. 2) compares averaged model predictions to averaged human labels; this is an explicit, disclosed operationalization of 'alignment,' not a quantity that is fitted and then renamed as a prediction. The inference from high self-consistency (kappa > 0.75 on A0, Figure 2) to 'overconfidence' is an interpretive label applied to a measured quantity, and the paper acknowledges in its Limitations section that other uncertainty metrics exist and that logit-based calibration methods are future work. Because the accuracy drops and MSE gaps are evaluated on held-out test data and are not constructed from the same quantities used to define the conclusion, no step in the paper reduces to its own inputs by construction. The reader's concern about the improper scoring rule is a validity critique of the metric, not a circularity in the derivation.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central empirical claims do not depend on fitted constants. They rest on three domain assumptions about the dataset and metrics: soft labels are a valid intensity ground truth, self-consistency equals confidence, and MD-Agreement disagreement reflects ambiguity rather than noise. No invented entities are introduced.

assumptions (3)

domain assumption Soft-label averaging of five annotator hard labels yields a valid ground-truth measure of offensiveness intensity.
Section 2.1 and Eq. 1 in Appendix A.2 treat the averaged soft label as the quantity model confidence should match. If soft labels are not a valid intensity scale, the MSE-based alignment claims are weakened.
domain assumption Self-consistency across temperature resampling is a valid proxy for model confidence.
Section 3.2 and Appendix A.2 define confidence as the average of hard predictions over five temperature settings. The overconfidence conclusion depends on this equivalence. The authors acknowledge logit-based calibration as an alternative in the limitations.
domain assumption Annotation disagreements in MD-Agreement reflect inherent ambiguity rather than labeling error.
Section 2.1 cites Sandri et al. (2023) to argue that less than 0.5 percent of reviewed disagreement samples are annotation errors. If many disagreements were noise, interpreting model misalignment as overconfidence on subjective cases would be less grounded.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement." pith.science (2026). https://pith.science/paper/N43N3Y2L

@misc{pith2026250206207,
  author       = {Pith},
  title        = {Pith review of: Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N43N3Y2L}},
  note         = {Machine review of arXiv:2502.06207}
}

read the original abstract

Large Language Models (LLMs) have become essential for offensive language detection, yet their ability to handle annotation disagreement remains underexplored. Disagreement samples, which arise from subjective interpretations, pose a unique challenge due to their ambiguous nature. Understanding how LLMs process these cases, particularly their confidence levels, can offer insight into their alignment with human annotators. This study systematically evaluates the performance of multiple LLMs in detecting offensive language at varying levels of annotation agreement. We analyze binary classification accuracy, examine the relationship between model confidence and human disagreement, and explore how disagreement samples influence model decision-making during few-shot learning and instruction fine-tuning. Our findings reveal that LLMs struggle with low-agreement samples, often exhibiting overconfidence in these ambiguous cases. However, utilizing disagreement samples in training improves both detection accuracy and model alignment with human judgment. These insights provide a foundation for enhancing LLM-based offensive language detection in real-world moderation tasks.

Figures

Figures reproduced from arXiv: 2502.06207 by the authors.

**Figure 1.** Accuracy of LLMs on detecting offensive and non-offensive language with different degrees of annotation agreement. on A++ samples but only by 3.11% on A0 . Similarly, Mixtral and Qwen2.5 show no substantial gain in detecting disagreement samples despite increased parameters. Model scaling alone does not resolve ambiguity, suggesting that larger models lack the nuanced human reasoning required to navigate subjecti… view at source ↗

**Figure 2.** Self-consistency of several LLMs across varying degrees of annotation agreement with Cohen’s Kappa (κ) as the metric. closer alignment (Uma et al., 2021a; Leonardelli et al., 2023). Additionally, we employ Spearman’s Rank Correlation Coefficient (ρ) to assess statistical correlation. The detailed metric definitions are provided in Appendix A.2. The results are presented in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Context-Aware Content Moderation for German Newspaper Comments
cs.CL 2025-05 conditional novelty 4.0 of 10

LSTM and CNN models for German newspaper comment moderation improve when given article title and user history, while ChatGPT-3.5 zero-shot classification does not.

Reference graph

Works this paper leans on

56 extracted references · 18 canonical work pages · cited by 1 Pith paper

[1]

Lora Aroyo, Lucas Dixon, Nithum Thain, Olivia Redfield, and Rachel Rosen. 2019. https://doi.org/10.1145/3308560.3317083 Crowdsourcing subjective tasks: The case study of understanding toxicity in online discussions . In Companion of The 2019 World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019 , pages 1100--1105. ACM

work page arXiv 2019
[2]

Valerio Basile. 2020. https://ceur-ws.org/Vol-2776/paper-4.pdf It's the end of the gold standard as we know it. on the impact of pre-aggregation on the evaluation of highly subjective tasks . In Proceedings of the AIxIA 2020 Discussion Papers Workshop co-located with the the 19th International Conference of the Italian Association for Artificial Intellige...

work page 2020
[3]

Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, and Alexandra Uma. 2021. https://doi.org/10.18653/v1/2021.bppf-1.3 We need to consider disagreement in evaluation . In Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, pages 15--21, Online. Association for Computational Linguistics

doi:10.18653/v1/2021.bppf-1.3 2021
[4]

Connor Baumler, Anna Sotnikova, and Hal Daum \'e III. 2023. https://doi.org/10.18653/v1/2023.findings-acl.658 Which examples should be multiply annotated? active learning when annotators may disagree . In Findings of the Association for Computational Linguistics: ACL 2023, pages 10352--10371, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-acl.658 2023
[5]

Rui Cao and Roy Ka - Wei Lee. 2020. https://doi.org/10.18653/v1/2020.coling-main.557 Hategan: Adversarial generative-based data augmentation for hate speech detection . In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020 , pages 6327--6338. International Committee on...

work page doi:10.18653/v1/2020.coling-main.557 2020
[6]

Tommaso Caselli, Valerio Basile, Jelena Mitrovic, and Michael Granitzer. 2020. https://arxiv.org/abs/2010.12472 Hatebert: Retraining BERT for abusive language detection in english . CoRR, abs/2010.12472

arXiv 2020
[7]

Jiuhai Chen and Jonas Mueller. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.283 Quantifying uncertainty in answers from any language model and enhancing their trustworthiness . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024 , pages 5186-...

doi:10.18653/v1/2024.acl-long.283 2024
[8]

Portia Cooper, Mihai Surdeanu, and Eduardo Blanco. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.192 Hiding in plain sight: Tweets with hate speech masked by homoglyphs . In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023 , pages 2922--2929. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-emnlp.192 2023

Show all 56 references

[10]

Macy, and Ingmar Weber

Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. 2017. https://aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/15665 Automated hate speech detection and the problem of offensive language . In Proceedings of the Eleventh International Conference on Web and Soc...

2017
[11]

Naihao Deng, Xinliang Frederick Zhang, Siyang Liu, Winston Wu, Lu Wang, and Rada Mihalcea. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.832 You are what you annotate: Towards better models through annotator representations . In Findings of the Association for Computat...

2023 doi
[12]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/n19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Associat...

2019 doi
[13]

Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019. https://doi.org/10.18653/v1/D19-1461 Build it break it fix it for dialogue safety: Robustness from adversarial human attack . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language...

2019 doi
[14]

Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. 2021. https://doi.org/10.18653/V1/2021.EMNLP-MAIN.29 Latent hatred: A benchmark for understanding implicit hate speech . In Proceedings of the 2021 Conference ...

2021 doi
[15]

Yihe Fan, Yuxin Cao, Ziyu Zhao, Ziyao Liu, and Shaofeng Li. 2024. https://doi.org/10.48550/ARXIV.2404.05264 Unbridled icarus: A survey of the potential perils of image inputs in multimodal large language model security . CoRR, abs/2404.05264

2024 doi arXiv
[16]

Antigoni - Maria Founta, Constantinos Djouvas, Despoina Chatzakou, Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, Michael Sirivianos, and Nicolas Kourtellis. 2018. https://aaai.org/ocs/index.php/ICWSM/ICWSM18/paper/view/17909 Large scale crowdsourcing ...

2018
[17]

Tommaso Giorgi, Lorenzo Cima, Tiziano Fagni, Marco Avvenuti, and Stefano Cresci. 2024. https://doi.org/10.48550/ARXIV.2410.07991 Human and LLM biases in hate speech annotations: A socio-demographic analysis of annotators and targets . CoRR, abs/2410.07991

2024 doi arXiv
[18]

Weinberger

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. http://proceedings.mlr.press/v70/guo17a.html On calibration of modern neural networks . In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017...

2017
[19]

Laura Hanu and Unitary team . 2020. https://github.com/unitaryai/detoxify Detoxify . Github. https://github.com/unitaryai/detoxify

2020
[20]

Fan Huang, Haewoon Kwak, and Jisun An. 2023. https://doi.org/10.1145/3543873.3587368 Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech . In Companion Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX...

2023
[22]

Danette Ifert Johnson and Nicole Lewis. 2010. Perceptions of swearing in the work setting: An expectancy violations theory perspective. Communication Reports, 23(2):106--118

2010
[23]

Manfred Klenner, Anne G \" o hring, and Michael Amsler. 2020. https://ceur-ws.org/Vol-2624/paper10.pdf Harmonization sometimes harms . In Proceedings of the 5th Swiss Text Analytics Conference and the 16th Conference on Natural Language Processing, SwissText/KONVENS 2020, Zuri...

2020
[24]

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html Large language models are zero-shot reasoners . In Advances in Neural Informati...

2022
[25]

Deepak Kumar, Yousef AbuHashem, and Zakir Durumeric. 2024. https://doi.org/10.1609/ICWSM.V18I1.31358 Watch your language: Investigating content moderation with large language models . In Proceedings of the Eighteenth International AAAI Conference on Web and Social Media, ICWSM...

2024 doi
[26]

Elisa Leonardelli, Gavin Abercrombie, Dina Almanea, Valerio Basile, Tommaso Fornaciari, Barbara Plank, Verena Rieser, Alexandra Uma, and Massimo Poesio. 2023. https://doi.org/10.18653/V1/2023.SEMEVAL-1.314 Semeval-2023 task 11: Learning with disagreements (lewidi) . In Proceed...

2023 doi
[27]

Elisa Leonardelli, Stefano Menini, Alessio Palmero Aprosio, Marco Guerini, and Sara Tonelli. 2021. https://doi.org/10.18653/V1/2021.EMNLP-MAIN.822 Agreeing to disagree: Annotating offensive language datasets with annotators' disagreement . In Proceedings of the 2021 Conference...

2021 doi
[28]

Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://openreview.net/forum?id=8s8K2UZGTZ Teaching models to express their uncertainty in words . Trans. Mach. Learn. Res., 2022

2022
[29]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized BERT pretraining approach . CoRR, abs/1907.11692

2019 arXiv
[30]

Junyu Lu, Hongfei Lin, Xiaokun Zhang, Zhaoqing Li, Tongyue Zhang, Linlin Zong, Fenglong Ma, and Bo Xu. 2023 a . https://doi.org/10.1109/TASLP.2023.3294715 Hate speech detection via dual contrastive learning . IEEE ACM Trans. Audio Speech Lang. Process. , 31:2787--2795

2023
[31]

Junyu Lu, Bo Xu, Xiaokun Zhang, Changrong Min, Liang Yang, and Hongfei Lin. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.898 Facilitating fine-grained detection of chinese toxic language: Hierarchical taxonomy, resources, and benchmarks . In Proceedings of the 61st Annua...

2023 doi
[32]

Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. 2021. https://ojs.aaai.org/index.php/AAAI/article/view/17745 Hatexplain: A benchmark dataset for explainable hate speech detection . In Thirty-Fifth AAAI Conference on Artificial ...

2021
[33]

Changrong Min, Hongfei Lin, Ximing Li, He Zhao, Junyu Lu, Liang Yang, and Bo Xu. 2023. https://doi.org/10.1016/J.INFFUS.2023.03.015 Finding hate speech with auxiliary emotion detection from self-training multi-label learning perspective . Inf. Fusion, 96:214--223

2023 doi
[34]

Marmarelis, Frederic R

Negar Mokhberian, Myrl G. Marmarelis, Frederic R. Hopp, Valerio Basile, Fred Morstatter, and Kristina Lerman. 2024. https://doi.org/10.18653/V1/2024.NAACL-LONG.407 Capturing perspectives of crowdsourced annotators in subjective learning tasks . In Proceedings of the 2024 Confe...

2024 doi
[35]

Guanyi Mou, Pengyi Ye, and Kyumin Lee. 2020. https://doi.org/10.1145/3340531.3411990 SWE2: subword enriched and significant word emphasized framework for hate speech detection . In CIKM '20: The 29th ACM International Conference on Information and Knowledge Management, Virtual...

2020
[36]

Debora Nozza and Dirk Hovy. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.240 The state of profanity obfuscation in natural language processing scientific publications . In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2...

2023 doi
[37]

OpenAI. 2020. https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Language models are few-shot learners . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS...

2020
[38]

OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

2023 doi arXiv
[39]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

2022
[40]

Someen Park, Jaehoon Kim, Seungwan Jin, Sohyun Park, and Kyungsik Han. 2024. https://aclanthology.org/2024.emnlp-main.1166 PREDICT: multi-agent-based debate simulation for generalized hate speech detection . In Proceedings of the 2024 Conference on Empirical Methods in Natural...

2024
[41]

Ellie Pavlick and Tom Kwiatkowski. 2019. https://doi.org/10.1162/TACL\_A\_00293 Inherent disagreements in human textual inferences . Trans. Assoc. Comput. Linguistics, 7:677--694

2019 doi
[42]

Barbara Plank. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.731 The problem of human label variation: On ground truth in data, modeling and evaluation . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671--10682, Abu Dhab...

2022 doi
[43]

Sarthak Roy, Ashish Harshavardhan, Animesh Mukherjee, and Punyajoy Saha. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.407 Probing llms for hate speech detection: strengths and vulnerabilities . In Findings of the Association for Computational Linguistics: EMNLP 2023, ...

2023 doi
[44]

Marta Sandri, Elisa Leonardelli, Sara Tonelli, and Elisabetta Jezek. 2023. https://doi.org/10.18653/V1/2023.EACL-MAIN.178 Why don't you do it right? analysing annotators' disagreement in subjective tasks . In Proceedings of the 17th Conference of the European Chapter of the As...

2023 doi
[45]

Hajung Sohn and Hyunju Lee. 2019. https://doi.org/10.1109/ICDMW.2019.00084 MC-BERT4HATE: hate speech detection using multi-channel BERT for different languages and translations . In 2019 International Conference on Data Mining Workshops, ICDM Workshops 2019, Beijing, China, No...

2019
[46]

Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.330 Just ask for calibration: Strategies for eliciting calibrated confidence scores from language ...

2023 doi
[47]

Alexandra Uma, Dina Almanea, and Massimo Poesio. 2022. https://doi.org/10.3389/FRAI.2022.818451 Scaling and disagreements: Bias, noise, and ambiguity . Frontiers Artif. Intell., 5:818451

2022
[48]

Alexandra Uma, Tommaso Fornaciari, Anca Dumitrache, Tristan Miller, Jon Chamberlain, Barbara Plank, Edwin Simpson, and Massimo Poesio. 2021 a . https://doi.org/10.18653/v1/2021.semeval-1.41 S em E val-2021 task 12: Learning with disagreements . In Proceedings of the 15th Inter...

2021 doi
[49]

Alexandra Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, and Massimo Poesio. 2021 b . https://doi.org/10.1613/JAIR.1.12752 Learning from disagreement: A survey . J. Artif. Intell. Res., 72:1385--1470

2021 doi
[50]

Rabiul Awal, Kenny Tsu Wei Choo, and Roy Ka - Wei Lee

Han Wang, Ming Shan Hee, Md. Rabiul Awal, Kenny Tsu Wei Choo, and Roy Ka - Wei Lee. 2023 a . https://doi.org/10.24963/IJCAI.2023/694 Evaluating GPT-3 generated explanations for hateful content moderation . In Proceedings of the Thirty-Second International Joint Conference on A...

2023 doi
[51]

Wenbo Wang, Lu Chen, Krishnaprasad Thirunarayan, and Amit P. Sheth. 2014. https://doi.org/10.1145/2531602.2531734 Cursing in english on twitter . In Computer Supported Cooperative Work, CSCW '14, Baltimore, MD, USA, February 15-19, 2014 , pages 415--425. ACM

2014
[52]

Le, Ed H

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 b . https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confe...

2023
[53]

Tharindu Cyril Weerasooriya, Alexander Ororbia, Raj Bhensadadia, Ashiqur KhudaBukhsh, and Christopher Homan. 2023. https://doi.org/10.18653/v1/2023.findings-acl.287 Disagreement matters: Preserving label diversity by jointly modeling item and annotator label distributions with...

2023 doi
[54]

Jiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang, Chengfei Li, Jinfeng Bai, and Minlie Huang. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.84 Unveiling the implicit toxicity in large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language...

2023 doi
[55]

Rongwu Xu, Xuan Qi, Zehan Qi, Wei Xu, and Zhijiang Guo. 2024. https://doi.org/10.48550/ARXIV.2408.01419 Debateqa: Evaluating question answering on debatable knowledge . CoRR, abs/2408.01419

2024 doi arXiv
[56]

Min Zhang, Jianfeng He, Taoran Ji, and Chang - Tien Lu. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.652 Don't go to extremes: Revealing the excessive sensitivity and calibration limitations of llms in implicit hate speech detection . In Proceedings of the 62nd Annual Meeti...

2024 doi
[57]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[58]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Lora Aroyo, Lucas Dixon, Nithum Thain, Olivia Redfield, and Rachel Rosen. 2019. https://doi.org/10.1145/3308560.3317083 Crowdsourcing subjective tasks: The case study of understanding toxicity in online discussions . In Companion of The 2019 World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019 , pages 1100--1105. ACM

work page arXiv 2019

[2] [2]

Valerio Basile. 2020. https://ceur-ws.org/Vol-2776/paper-4.pdf It's the end of the gold standard as we know it. on the impact of pre-aggregation on the evaluation of highly subjective tasks . In Proceedings of the AIxIA 2020 Discussion Papers Workshop co-located with the the 19th International Conference of the Italian Association for Artificial Intellige...

work page 2020

[3] [3]

Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, and Alexandra Uma. 2021. https://doi.org/10.18653/v1/2021.bppf-1.3 We need to consider disagreement in evaluation . In Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future, pages 15--21, Online. Association for Computational Linguistics

doi:10.18653/v1/2021.bppf-1.3 2021

[4] [4]

Connor Baumler, Anna Sotnikova, and Hal Daum \'e III. 2023. https://doi.org/10.18653/v1/2023.findings-acl.658 Which examples should be multiply annotated? active learning when annotators may disagree . In Findings of the Association for Computational Linguistics: ACL 2023, pages 10352--10371, Toronto, Canada. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-acl.658 2023

[5] [5]

Rui Cao and Roy Ka - Wei Lee. 2020. https://doi.org/10.18653/v1/2020.coling-main.557 Hategan: Adversarial generative-based data augmentation for hate speech detection . In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020 , pages 6327--6338. International Committee on...

work page doi:10.18653/v1/2020.coling-main.557 2020

[6] [6]

Tommaso Caselli, Valerio Basile, Jelena Mitrovic, and Michael Granitzer. 2020. https://arxiv.org/abs/2010.12472 Hatebert: Retraining BERT for abusive language detection in english . CoRR, abs/2010.12472

arXiv 2020

[7] [7]

Jiuhai Chen and Jonas Mueller. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.283 Quantifying uncertainty in answers from any language model and enhancing their trustworthiness . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024 , pages 5186-...

doi:10.18653/v1/2024.acl-long.283 2024

[8] [8]

Portia Cooper, Mihai Surdeanu, and Eduardo Blanco. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.192 Hiding in plain sight: Tweets with hate speech masked by homoglyphs . In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023 , pages 2922--2929. Association for Computational Linguistics

doi:10.18653/v1/2023.findings-emnlp.192 2023