A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance

Dennis Assenmacher; Gabriella Lapesa; Matteo Melis

REVIEW 6 major objections 5 minor 12 references

A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance

T0 review · 6 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that the definition of hate speech supplied in a zero-shot prompt is a substantive variable that changes classifier performance, with the direction of the effect depending on the model.

desk verdict A genuinely useful taxonomy and definition collection, but the headline claim about Conceptual Elements overstates what the experiments can isolate; still worth a serious referee. read the letter →

arxiv 2506.18576 v1 pith:3LG2OWEL submitted 2025-06-23 cs.CL cs.CY

Matteo Melis , Gabriella Lapesa , Dennis Assenmacher This is my paper

classification cs.CLcs.CY

keywords hatespeechdetectionzero-shotclassificationdefinitionpromptingconceptualtaxonomylargelanguagemodelspromptsensitivityerroranalysisimplicit

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that hate speech is not a fixed target a model can be asked to detect: the definition placed in a zero-shot prompt is itself an experimental variable that changes classifier performance. To make definition choice controllable, the authors collect 20 published hate speech definitions, break them into 14 Conceptual Elements, and arrange them in a three-layer taxonomy, then recombine the elements into definitions that differ only in which elements they contain. Prompting three instruction-tuned LLMs on three datasets representing synthetic, human-in-the-loop, and real-world data, they find that more detailed definitions sometimes improve scores and sometimes lower them, and that the effect is not consistent across architectures. Their error analysis on HateCheck additionally indicates that adding a Conceptual Element aimed at a specific type of hate reduces errors on that type, such as explicitly mentioning exceptions to hate speech improving classification of misleading non-hate comments. If the claim holds, reported zero-shot hate speech scores are partly contingent on the definition selected, and the taxonomy offers a systematic way to construct and compare definitions.

What carries the argument

The central object is the three-layer taxonomy of 14 Conceptual Elements: four Foundational Elements (Form of Communication, Target, Problematic Content, Addressed Attributes) that form the Hate Speech Base definition; four Extensive Definitions (of Form of Communication, Target, Problematic Content, and a List of Addressed Attributes) that enrich those foundations; and six Accessory Elements including Possible Implications, Exceptions, and Implicit Hate Speech. The machinery's work is to make a definition decomposable and recombinable: each element has a fixed natural-language span, spans are reused verbatim across definitions, and definitions differ only in which spans are included. That design is what lets the experiments attribute performance differences to the presence or absence of conceptual content rather than to arbitrary wording.

What would settle it

Reprompt all models with several paraphrases of each Conceptual Element span while holding the underlying element fixed; if the macro-F1 differences across definitions vanish or reverse under rephrasing, the central claim that Conceptual Elements drive the effect is falsified. For the secondary claim, apply the Exceptions and Implicit Hate Speech elements to a dataset with corresponding fine-grained labels and test whether targeted error reductions appear outside HateCheck.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the definition of hate speech used in a zero-shot prompt is a substantive variable: prompting definitions built by adding or removing specific Conceptual Elements changes macro-F1 scores, and the direction of the change depends on the model and dataset. The paper further claims that definitions are modular, with the 14 Conceptual Elements combinable into a Hate Speech Base plus refinements, so the effect of a definition can be traced to the elements it contains. The error analysis adds that an element aimed at a particular type of hate reduces errors on that type: telling the model that offensive statements about personality traits are exceptions lowers errors on misleading non-hate items, and telling it that hate speech can be implicit lowers errors on implicitly hateful items, with the strongest evidence on the HateCheck suite.

Load-bearing premise

The experimental comparison assumes that the curated natural-language spans differ only in the Conceptual Elements they encode, so that performance differences come from conceptual content rather than from prompt wording, phrasing, or length.

Editorial extensions

If this is right

Reported zero-shot hate speech scores are definition-dependent, so benchmark comparisons that do not fix the definition are not directly comparable.
Definition prompting can be used to tune error profiles: models that reduce false negatives with detailed definitions fit high-recall moderation strategies, while models that primarily reduce false positives fit high-precision settings.
A definition can be evaluated module by module, letting researchers identify which Conceptual Element drives performance on which hate type before deployment.
The effect is not uniform across architectures, so a single best definition for hate speech detection should not be assumed to transfer across models.
Targeted errors respond to targeted elements, as seen when adding exceptions or implicit hate speech reduces errors on the corresponding HateCheck macro classes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the authors leave implicit is that the same modular technique could be applied to other contested constructs such as toxicity or misinformation, whose benchmark scores may be equally contingent on the definition baked into the prompt.
A testable extension is to apply the Implicit Hate Speech and Exceptions elements to a non-HateCheck dataset with fine-grained labels for implicit hate and misleading non-hate content; the paper predicts targeted error reductions on those classes, so their absence would show the effect is specific to HateCheck.
Because the authors concede that surface phrasing can matter even when the underlying elements are fixed, a natural follow-up is to paraphrase each Conceptual Element several times; if the performance differences vanish under rephrasing, the effect belongs to wording rather than conceptual content.
The taxonomy could help reconcile annotation standards by mapping each dataset's operational definition onto Conceptual Elements, allowing predictions about when classifiers trained on one definition will transfer to another.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The taxonomy and the curated definition collection are the real news here. Fourteen conceptual elements in three layers, derived from 20 external definitions, with reusable natural-language definitions in Appendix E: that is a genuinely useful scaffold for anyone building hate speech annotation or prompting studies. The experiments also go beyond Roy et al. in breadth, with three model families, three dataset types, and systematic combinations. Code and data are available. Credit where due.

The broad empirical claim survives: prompting different definitions changes zero-shot F1, and the direction of the effect varies by model. Flan-T5 reliably benefits from more detailed definitions; Mistral improves at the second stage; Llama-3 is flat or worse except one case. Those patterns are visible in the tables and consistent across datasets.

The soft spots are real but not fatal to the resource. The biggest is the CE-to-text confound, and the paper half-admits it. The authors' Own vs HSB comparison on MHS is the smoking gun: same encoded CEs, but phrasing alone shifts macro-F1 by 3.45 points for Llama-3, larger than most between-CE contrasts in Table 2. So the central claim phrased as effects of 'degree of specificity in terms of encoded elements' is not actually isolated. The Limitations section concedes that surface realization matters, but the abstract and conclusion still lean on the CE-specific reading. That gap should be fixed in revision.

Also: no significance tests or confidence intervals, and many F1 gaps are sub-point. Three runs with majority-consistency reporting is thin for the stronger claims. Step 2 selects its base definition on the same test set, making accessory-element comparisons conditional on a test-set-chosen baseline. Temperatures differ across models, and Llama-3's possible HateCheck leakage is acknowledged. The Sec 6 finding that adding an Implicit Hate Speech element helps the 'Special HS' macro class is close to tautological, since the prompt explicitly says 'implicit' and the macro class is defined as implicitly phrased hate; that finding should be presented as a sanity check, not a discovery.

Who is this for: people working on hate speech detection, prompt engineering, or construct operationalization. The taxonomy will be cited even if the experiments fade. It deserves a serious referee. I would send it to review, expecting a major revision that either adds uncertainty quantification and a held-out split, or reframes the central claim to 'definitions matter' while keeping the CE taxonomy as a proposal rather than a demonstrated causal driver.

Referee Report

6 major / 5 minor

Summary. The paper proposes a three-layer taxonomy of 14 Conceptual Elements (CEs) for hate speech definitions, derived from 20 definitions in the literature, social-media policies, official documents, and LLM-generated definitions. The authors construct a collection of definitions as modular combinations of CEs and use them as prompts in zero-shot binary hate-speech classification with three instruction-tuned LLMs (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.2, Flan-T5-XL) on three datasets (HateCheck, Learning from the Worst, Measuring Hate Speech). The main empirical claim is that varying the definition, specifically the degree of specificity in terms of encoded CEs, affects macro-F1 performance, but the effect is inconsistent across architectures. A secondary error analysis on HateCheck macro classes suggests that adding a CE aimed at a particular hate type (implicit hate, exceptions) reduces errors on the corresponding class. The paper also contributes a public code repository and appendices on robustness and sensitivity.

Significance. The taxonomy and the curated definition collection are valuable, transparent resources that could help researchers structure operational definitions of hate speech and compare benchmark results across definition choices. If the experimental claim were cleanly established, the paper would show that prompt definition is a substantive variable in zero-shot classification and that reported scores are contingent on the chosen definition. Strengths include the systematic recombination of CEs, the use of three datasets and three model families, and the appended robustness and sensitivity analyses. However, the quantitative evidence currently supports only a coarse version of the claim: definitions affect performance. The specific attribution to CE composition is confounded, and the cross-architecture comparison has methodological inconsistencies. The conceptual contribution stands on its own; the experimental conclusions require revision.

major comments (6)

[§3.3; Appendix E; §8] The central claim in the abstract attributes performance differences to 'a different degree of specificity in terms of encoded elements.' Section 3.3 states that the curated wording was chosen so that 'variation stems solely from different combinations of CEs,' but the definitions in Appendix E differ simultaneously in CE set, surface wording, and length. Section 8 concedes that 'semantically different realization of the Conceptual Elements could have had a different impact' and that surface-level phrasing can influence model behavior; the Own-vs-HSB comparison on MHS is a within-paper demonstration that identical CEs with different wording produce different F1. Therefore Tables 2 and 3 do not isolate CE specificity as the causal variable. Please add paraphrase-level controls (multiple surface realizations per CE combination, or matched templates that vary a single clause) and report the resulting variance, or restrict the claim to 'prompt wording affects performance.'
[§5.1; Tables 2 and 3] The paper reports macro-F1 point estimates from three runs (see also Appendix G) but no confidence intervals, error bars, or significance tests. Many contrasts are small, for example HateCheck Llama-3 HSB (80.02) vs HSB_EDT (80.14) in Table 2, and several Step 2 differences in Table 3 are on the order of 1 F1 point. The qualitative statements in Section 5.2, such as Mistral consistently improving in Step 2 or Flan-T5 never improving in Step 2, are not supported beyond descriptive trends. Please add paired significance tests or bootstrap confidence intervals across items and runs, or explicitly frame the results as exploratory.
[§4.1; §5.2; Table 3] In Step 2, the base definition is selected as the Step 1 definition with the highest macro-F1 on the same test set used for the Step 2 evaluation. This is a form of test-set peeking: the selection of HSB_EDT (or the relevant best definition per model/dataset) is informed by the labels of the very instances that are then used to measure the accessory-element effects. The Step 2 improvements in Table 3 and the corresponding claims in Section 5.2 may therefore be inflated. Please use a validation split for selecting the base definition, pre-register the base, or report selection-corrected estimates.
[Footnote 6; §5.2] The decoding setups differ across models: Mistral was run with constrained generation at temperature 0.7, while Llama-3 and Flan-T5 used temperature 0.95. The central claim includes the statement that the definition effect is 'not consistent across all architectures,' but cross-model differences in definition sensitivity (Section 5.2) could be driven by decoding temperature or constrained decoding rather than by architecture. Within-model comparisons are less affected because each model uses a fixed setup, but the architecture-level comparison should either match decoding settings across models or demonstrate robustness to temperature.
[§5.1; Table 2] Section 5.1 notes that Llama-3 performs best with no definition (NO) in two of three datasets, which the authors attribute to potential data leakage from HateCheck. If Llama-3 has been exposed to HateCheck during pretraining or instruction tuning, its HateCheck results in Tables 2 and 3 are not a clean zero-shot evaluation, and the cross-architecture pattern that includes Llama-3 is compromised. Please perform a contamination check (for example, exact or near-duplicate matching against known pretraining corpora, or evaluation on a newly collected held-out set) and rerun or qualify the Llama-3 results accordingly.
[§6; Table 4; Appendix J] The error-analysis claim that adding a targeted CE reduces errors on the corresponding macro class is partly built into the prompt. The IHS definition explicitly states that hate speech 'can also be implicit, portrayed as an indirect or coded language that uses Irony, Stereotypes, or Misinformation,' and the Special HS macro class in Appendix J is defined as implicitly phrased hate; similarly, the Exc definition enumerates what is not hate speech, matching the Misleading NHS class. The observed error reductions in Table 4 may therefore reflect keyword/instruction matching rather than a general conceptual effect. The authors acknowledge that this analysis is limited to HateCheck (Section 8), but the conclusion in Section 7 generalizes the finding. Please temper the claim or test it on held-out data for implicit hate and exceptions.

minor comments (5)

[Throughout] Model names and definition names are inconsistent across text and tables (e.g., 'LLama3' vs 'Llama-3', 'FlanT5' vs 'Flan-T5', 'HSB_EDFoC' vs 'HSB + EDFoC'); please standardize.
[Table 2 caption; Appendix K] There are several typos in tables and appendices, including 'Exstensive' in the Table 2 caption and 'couter_ref_nh' in Appendix K; a careful proofreading pass is needed.
[§4.1] The sampling description uses European decimal commas ('31,84%') and a comma in '3901,' which is inconsistent with the rest of the paper; please use uniform decimal notation.
[Appendices G-H] The robustness and sensitivity analyses are valuable but are only referenced as a sanity check in the main text; consider reporting the key robustness result (e.g., the NO condition being the frequent outlier) in the main Results section.
[References] Reference formatting is inconsistent, including 'V oin' in the Wigand reference and 'Plaza-del arco' capitalization; the citations should be harmonized with the venue style.

Circularity Check

1 steps flagged · score 4.0 of 10

Main benchmark result is independent, but the Sec. 6 finding that adding an Implicit Hate Speech CE improves Special HS performance is self-definitional: the prompt text names the exact property used to define the outcome class.

self definitional [Section 6 'Analysis of HateCheck functionalities'; Appendix J macro-class grouping; Table 1 / Appendix E IHS definition]
"Furthermore, we find that adding a CE specific to a class of instances reduces classification errors for that class. ... A similar effect is observed for implicit hate speech: clarifying that hate speech can be conveyed through coded language, irony, or sarcasm leads to performance gains in the Special HS class. This effect is most pronounced in Mistral, the only model that consistently improves in the second step."

The outcome class 'Special HS' is defined in Appendix J as 'data-points which are more implicitly phrased such as: "Never trust a single word said by a woman" or "I really do believe black people are just lesser beings"'. The IHS Conceptual Element is realized in the prompt as 'Hate Speech can also be implicit, portrayed as an indirect or coded language that uses Irony, Stereotypes, or Misinformation' (Table 1 and Appendix E). Thus the input condition literally contains the definition of the target phenomenon; improved performance on Special HS under +IHS is a direct consequence of telling the model what the test class contains, not an emergent empirical discovery about the taxonomy.

full rationale

The central experimental claim — that varying hate speech definitions, operationalized as different combinations of Conceptual Elements, affects zero-shot LLM performance in an architecture-dependent way — is not circular. The 14 Conceptual Elements are inductively derived from 20 external definitions (Section 3.1, Appendix A), and the F1 comparisons are run against external datasets (HateCheck, LFTW, MHS) with clearly specified prompts. The taxonomy is a structured summary of existing definitions, and the paper explicitly disclaims a one-to-one mapping between the original definitions and the experimental collection (Section 3.3), so the main benchmark result has independent empirical content. No load-bearing self-citation chain or imported uniqueness theorem is present. However, one secondary finding is close to tautological. In Section 6, the paper reports that adding the Implicit Hate Speech CE reduces errors on the Special HS macro class, but Special HS is defined by implicit phrasing while the IHS prompt text explicitly says that hate speech 'can also be implicit, portrayed as an indirect or coded language that uses Irony, Stereotypes, or Misinformation'. The alignment between the intervention and the outcome label is built in by the authors' own definitions, so this specific 'CE-specific error reduction' conclusion does not provide independent evidence for the taxonomy's predictive value. The paper's own Limitations section also concedes a wording confound: 'semantically different realization of the Conceptual Elements could have had a different impact on the models' performance... the Own definition contains the exact same CEs of the HSB definition, though leading to different results.' That concession weakens the causal attribution of F1 differences to CE content, but it is a validity threat rather than a circular derivation. Overall, the main derivation chain is self-contained; only the Section 6 error-analysis claim exhibits a self-definitional structure, yielding a moderate circularity score of 4.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The paper has no free parameters in the model-fitting sense, but it has several hand-chosen experimental design degrees of freedom: the wording of each CE, the selection of the Step 2 base definition using test performance, the random sampling of two datasets, and the differing temperatures. The central interpretive load is carried by assumptions that the dataset labels are valid ground truth, that the 20-definition coding is sound, and that the HateCheck macro-class grouping is meaningful. The taxonomy's CEs are grounded in 20 external definitions, so they are not invented from scratch, but their organization and wording are author-curated.

free parameters (4)

CE-to-language mapping = hand-written spans in Appendix E
Each Conceptual Element is realized by a specific natural language phrase chosen by the authors; the paper's Limitations (Sec 8) acknowledge different phrasings could change results.
Step 2 base definition = HSB_EDT for Llama-3 on HateCheck/LFTW; HSB_EDFoC_EDPC for MHS; etc. (Table 2)
The base definition for Step 2 was selected as the highest macro-F1 from Step 1 on the same test set, a data-dependent model selection without a held-out split.
Random sampling of LFTW/MHS = 3901 instances per dataset, 68.16% HS
Sampling matched HateCheck's class distribution; the seed is not fixed, so exact subsets are not reproducible.
Generation temperature = 0.95 for Llama-3 and Flan-T5; 0.7 for Mistral
Inconsistent decoding settings across models (Sec 4.1), a hand-chosen difference that could affect stability and F1.

assumptions (4)

domain assumption Dataset labels are valid ground truth
F1 is computed against dataset labels; if labels are noisy or use different construct definitions, the performance comparisons are distorted. Invoked throughout Sec 5.
domain assumption The 20 collected definitions are representative and the inductive coding into 14 CEs is sound
The taxonomy is built from this coding (Sec 3.1); no inter-annotator agreement or validation of the coding is reported.
domain assumption HateCheck macro-class grouping preserves meaningful categories
The 29 HateCheck functionalities are grouped into 5 macro classes (Appendix J); the grouping is author-defined and underlies the error-analysis claims in Sec 6.
domain assumption Three repeated runs suffice to stabilize F1 estimates
Each experiment repeated 3 times (Sec 5.1 footnote), but no variance or confidence intervals are reported for the F1 scores.

invented entities (1)

14 Conceptual Elements and the three-layer taxonomy independent evidence
purpose: To structure and recombine hate speech definitions into prompts
The CEs are derived from 20 external definitions (Sec 3.1, Appendix A), so they are grounded in external sources. However, their organization into a three-layer taxonomy and the wording of each CE is author-curated, so independent evidence is partial.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance." pith.science (2026). https://pith.science/paper/3LG2OWEL

@misc{pith2026250618576,
  author       = {Pith},
  title        = {Pith review of: A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3LG2OWEL}},
  note         = {Machine review of arXiv:2506.18576}
}

read the original abstract

Detecting harmful content is a crucial task in the landscape of NLP applications for Social Good, with hate speech being one of its most dangerous forms. But what do we mean by hate speech, how can we define it, and how does prompting different definitions of hate speech affect model performance? The contribution of this work is twofold. At the theoretical level, we address the ambiguity surrounding hate speech by collecting and analyzing existing definitions from the literature. We organize these definitions into a taxonomy of 14 Conceptual Elements-building blocks that capture different aspects of hate speech definitions, such as references to the target of hate (individual or groups) or of the potential consequences of it. At the experimental level, we employ the collection of definitions in a systematic zero-shot evaluation of three LLMs, on three hate speech datasets representing different types of data (synthetic, human-in-the-loop, and real-world). We find that choosing different definitions, i.e., definitions with a different degree of specificity in terms of encoded elements, impacts model performance, but this effect is not consistent across all architectures.

Figures

Figures reproduced from arXiv: 2506.18576 by the authors.

**Figure 2.** Distribution of errors across the three models on HateCheck. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Distribution of errors in LLama-3 across Hatecheck Classes [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (13 more)

**Figure 4.** Figure 4: Graphic visualization of the taxonomy. C Conceptual Elements in our Taxonomy: Definitions FOUNDATIONAL ELEMENTS: We define Foundational Elements as those Conceptual Elements that are required to build a meaningful definition of Hate Speech. In this category we find: Fo…

**Figure 5.** Figure 5: Confusion matrices of non-consistent answer between definitions in [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]

**Figure 6.** Figure 6: Confusion matrices of non-consistent answer between definitions in [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]

**Figure 7.** Figure 7: Confusion matrices of non-consistent answer between definitions in [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]

**Figure 8.** Figure 8: Distribution of errors across the three models on HateCheck (Step 1). [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]

**Figure 9.** Figure 9: Distribution of errors across the three models on HateCheck (Step 2). [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]

**Figure 10.** Figure 10: Distribution of errors across the three models on Learning from the Worst (Step 1). [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]

**Figure 11.** Figure 11: Distribution of errors across the three models on Learning from the Worst (Step 2). [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]

**Figure 12.** Figure 12: Distribution of errors across the three models on Measuring Hate Speech (Step 1). [PITH_FULL_IMAGE:figures/full_fig_p027_12.png]

**Figure 13.** Figure 13: Distribution of errors across the three models on Measuring Hate Speech (Step 2). [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]

**Figure 14.** Figure 14: Distribution of errors across Macro Classes [PITH_FULL_IMAGE:figures/full_fig_p030_14.png]

**Figure 15.** Figure 15: Distribution of errors across Macro Classes. [PITH_FULL_IMAGE:figures/full_fig_p030_15.png]

**Figure 16.** Figure 16: Distribution of errors across Macro Classes. [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages

[1]

If PC and AA are missing, the communication (FoC, T) is not Hate Speech, but it is just communica- tion

work page
[2]

this is bul***it

If T and AA are missing, the communication (FoC, PC) can still be offensive (or toxic), but not categorized as Hate Speech (i.e., “this is bul***it”, “Cauliflowers are fu**ing disgusting”)

work page
[3]

[POLITICIAN NAME] is the dumbest politician in the US

If AA is missing, the communication (FoC, T, PC) it is not Hate Speech but Offensive Language (“[POLITICIAN NAME] is the dumbest politician in the US")

work page
[4]

missing Conceptual Element

There are no cases in which there are only PC (FoC, PC, AA) and AA or T and AA (FoC, T, AA). This makes AA dependent from PC and T (other than from the FoC). It comes that, when it seems to have a case of this kind, actually AA overlaps with the apparent “missing Conceptual Element" (i.e., AA overlaps with PC “you are a [f-word]”). EXTENSIVE DEFINITIONS: ...

work page
[5]

In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022, pages 83–94

The measuring hate speech corpus: Leverag- ing rasch measurement theory for data perspectivism. In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022, pages 83–94. Indira Sen, Dennis Assenmacher, Mattia Samory, Is- abelle Augenstein, Wil Aalst, and Claudia Wagner

work page
[11]

undermines social cohesion, promotes division . . . in communities

social (sPI): it refers to the implication on the social level of one (or more) PC toward a T (i.e., “undermines social cohesion, promotes division . . . in communities”)

work page
[12]

can have serious consequences for individuals, often perpetuating discrimination, hostility, and violence

individual (iPI): it refers directly to the effects that one (or more) PC can have on the T (i.e., “can have serious consequences for individuals, often perpetuating discrimination, hostility, and violence”). Exceptions (Exc): provide information on what is not considered HS (i.e., ”attacks on people’s personality traits, ideas, or opinions”). Implicit Ha...

work page 2023
[2017]

CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network

Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 774–782, Taipei, Taiwan. Asian Federation of Natural Language Pro- cessing. S. Ghosh, M. Suri, P. Chiniya, U. Tyagi, S. Kumar, an...

work page Pith review arXiv 2023

Show all 12 references

[2019]

In Proceedings of the 13th International Work- shop on Semantic Evaluation , pages 54–63, Min- neapolis, Minnesota, USA

Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twit- ter. In Proceedings of the 13th International Work- shop on Semantic Evaluation , pages 54–63, Min- neapolis, Minnesota, USA. Association for Compu- tational Linguistics. Tilman Be...

2019 arXiv
[2022]

arXiv preprint

Scaling instruction-finetuned language models. arXiv preprint. Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech de- tection and the problem of offensive language. In Proceedings of the International AAAI Conference on Web and Social M...

2017 arXiv
[2023]

vocabulary and grammar depicting a common enemy,

People make better edits: Measuring the effi- cacy of LLM-generated counterfactually augmented data for harmful language detection. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10480–10504, Singapore. Association for Computat...

2023 arXiv
[2024]

ACM Transactions on the Web, 18(2):1–36

“hot” chatgpt: The promise of chatgpt in detecting and discriminating hateful, offensive, and toxic comments on social media. ACM Transactions on the Web, 18(2):1–36. Thomas Mandl, Sandip Modha, Gautam Kishore Shahi, Hiren Madhu, Shrey Satapara, Prasenjit Majumder, Johannes Sc...

2021 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

If PC and AA are missing, the communication (FoC, T) is not Hate Speech, but it is just communica- tion

work page

[2] [2]

this is bul***it

If T and AA are missing, the communication (FoC, PC) can still be offensive (or toxic), but not categorized as Hate Speech (i.e., “this is bul***it”, “Cauliflowers are fu**ing disgusting”)

work page

[3] [3]

[POLITICIAN NAME] is the dumbest politician in the US

If AA is missing, the communication (FoC, T, PC) it is not Hate Speech but Offensive Language (“[POLITICIAN NAME] is the dumbest politician in the US")

work page

[4] [4]

missing Conceptual Element

There are no cases in which there are only PC (FoC, PC, AA) and AA or T and AA (FoC, T, AA). This makes AA dependent from PC and T (other than from the FoC). It comes that, when it seems to have a case of this kind, actually AA overlaps with the apparent “missing Conceptual Element" (i.e., AA overlaps with PC “you are a [f-word]”). EXTENSIVE DEFINITIONS: ...

work page

[5] [5]

In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022, pages 83–94

The measuring hate speech corpus: Leverag- ing rasch measurement theory for data perspectivism. In Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022, pages 83–94. Indira Sen, Dennis Assenmacher, Mattia Samory, Is- abelle Augenstein, Wil Aalst, and Claudia Wagner

work page

[6] [11]

undermines social cohesion, promotes division . . . in communities

social (sPI): it refers to the implication on the social level of one (or more) PC toward a T (i.e., “undermines social cohesion, promotes division . . . in communities”)

work page

[7] [12]

can have serious consequences for individuals, often perpetuating discrimination, hostility, and violence

individual (iPI): it refers directly to the effects that one (or more) PC can have on the T (i.e., “can have serious consequences for individuals, often perpetuating discrimination, hostility, and violence”). Exceptions (Exc): provide information on what is not considered HS (i.e., ”attacks on people’s personality traits, ideas, or opinions”). Implicit Ha...

work page 2023

[8] [2017]

CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network

Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 774–782, Taipei, Taiwan. Asian Federation of Natural Language Pro- cessing. S. Ghosh, M. Suri, P. Chiniya, U. Tyagi, S. Kumar, an...

work page Pith review arXiv 2023