Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Instruction-tuned LLMs detect harmful sentences in long inputs best when about a quarter of the prompt is harmful; detection degrades as context lengthens, harm moves later in the input, or harm becomes implicit.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 11:13 UTC pith:RRV6D5WY

load-bearing objection A careful controlled stress-test mapping how LLMs extract harmful sentences from long synthetic inputs; the patterns are plausible but need error bars and a coherent-document check before they generalize. the 4 major comments →

arxiv 2510.05864 v2 pith:RRV6D5WY submitted 2025-10-07 cs.CL cs.CY

On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputs

Faeze Ghorbanpour , Alexander Fraser This is my paper
classification cs.CL cs.CY
keywords harmful content detectionlong-context LLMssensitivity analysisprevalence effectdilution effectposition biasimplicit vs explicit harminstruction-tuned models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that instruction-tuned LLMs, when asked to list harmful sentences embedded in long prompts, follow consistent, predictable patterns: their best precision-recall balance occurs when harmful content makes up about a quarter of the input; recall and precision drop as the prompt grows and harmful sentences are diluted; harmful sentences near the beginning are detected more reliably than those in the middle or end; and explicit harm is identified more reliably than implicit harm. These patterns hold across toxic, offensive, and hate content and across three open-weights instruction-tuned models. A sympathetic reader would care because content moderation, retrieval-augmented generation, and long-document filtering increasingly rely on LLMs to spot harmful spans inside extended mixed content, where sentence-level evaluations do not transfer directly.

Core claim

Using prompts built by randomly concatenating harmful and neutral sentences into numbered lists, the authors show that LLM sensitivity to harmful content is non-monotonic in prevalence: performance rises as the harmful fraction increases toward roughly 0.25 and then stagnates or declines as the fraction reaches 0.5, because the model over-predicts harm and loses accuracy on benign sentences. With the number of harmful sentences fixed, adding more neutral sentences lowers both recall and precision. Harmful sentences placed in the first third of the prompt are detected better than those in later regions, and explicit harm outperforms implicit harm on every dataset. The authors interpret this a

What carries the argument

The central instrument is a controlled stress-test prompt: sentences are sampled from existing datasets, numbered, randomly concatenated, and appended to an instruction asking the model to return the indices of harmful sentences. By varying four factors independently — total token length, harmful sentence ratio, harm type (explicit/implicit), and position region — the design isolates each factor's effect while holding others constant. The core metrics are macro-F1, predicted prevalence value (the fraction of sentences the model labels harmful), and harmful-class precision and recall; the contrast between predicted prevalence and F1 is what exposes the calibration-versus-localization gap.

Load-bearing premise

The results rest on prompts made by randomly shuffling independent sentences; if coherent, topically connected real-world inputs change how models weight harmful content, these prevalence, dilution, and position effects may not transfer.

What would settle it

Run the same localization protocol on coherent long documents — forum threads, reports, or multi-turn conversations — where harmful sentences appear naturally; if recall no longer drops with length, or if the performance peak moves away from roughly 25 percent harmful prevalence, the controlled stress-test pattern does not describe real inputs. A simpler check is to compare random-shuffle prompts to the same sentences reordered into a narrative with topic flow and anaphora.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Content moderation systems that rely on long-context LLMs will miss harmful sentences more often when those sentences are rare or buried among many benign ones.
  • In retrieval-augmented generation, harmful passages placed late in the retrieved context are more likely to slip through, so ranking or reordering that surfaces early or central content may reduce positional bias.
  • Models can often estimate how much harmful content exists even when they cannot name which sentences are harmful, a signal that could be used for triage before deeper inspection.
  • Explicit harm can be caught with reasonable reliability, but implicit hate and offense require either longer attention windows or complementary detectors.
  • The reported patterns are consistent across three model families and three harm categories, suggesting a common instruction-following behavior rather than a dataset-specific artifact.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the pattern transfers to coherent documents, safety teams could add a prevalence-calibration probe: ask the model how many harmful sentences exist before asking which ones, and use the discrepancy to flag low-confidence localization.
  • The position bias suggests a primacy effect for safety judgments; a testable extension is whether an explicit 'scan the whole input' instruction or attention-promoting formatting shifts detection toward later positions.
  • The random-concatenation setup removes discourse cues, so a natural next experiment is to graft harmful sentences into coherent narratives and check whether the quarter-prevalence optimum and the position bias survive.
  • Because predicted prevalence tracks true ratio while recall lags, a cheap two-pass strategy could be built: use the prevalence estimate to decide when to run a more thorough second pass over the input.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents a controlled stress-test evaluation of how instruction-tuned LLMs handle harmful sentences embedded in long inputs. The authors construct synthetic prompts by concatenating independent harmful and neutral sentences, varying four factors: prompt length (600–6,000 tokens in the main text, up to 30k in the appendix), harmful-to-neutral ratio (0.05–0.5), harm type (explicit/implicit/both), and placement (beginning/middle/end/all). They evaluate LLaMA-3-8B-Instruct, Qwen2.5-7B-Instruct, and Mistral-7B-Instruct-v0.3 on three datasets (IHC, OffensEval, JigsawToxic), asking the model to output the indices of harmful sentences. The main reported findings are: (i) non-monotonic prevalence sensitivity, with macro-F1 peaking around r=0.25; (ii) a dilution effect, with recall and precision degrading as more neutral sentences are added; (iii) a position effect, with beginning-placed harmful sentences detected most reliably; and (iv) higher sensitivity to explicit than implicit harm. The paper frames this as a complementary diagnostic to coherent-document evaluation and draws implications for moderation, RAG, and multi-turn safety.

Significance. If the findings hold, the paper contributes a systematic, factorial evaluation of long-context safety that goes beyond binary compliance scores, and the promised public framework could be reusable by safety teams. The design has genuine strengths: three models, three datasets, independent manipulation of several factors, and k=128 repetitions per setting. The reported patterns are consistent across models and datasets, which lends some robustness. However, the current evidence is weakened by the absence of any uncertainty quantification, the possibility that the headline prevalence peak is at least partly a macro-F1 artifact under class imbalance, and the entirely synthetic input construction, which the authors themselves concede may make the task easier. The paper is a useful stress-test study, but its broader claims about real long inputs are not yet supported.

major comments (4)
  1. [§4.3, §4.5, Fig. 1] The central non-monotonic prevalence claim—macro-F1 peaking at r≈0.25—is presented without any measure of variance. Although k=128 repetitions are run, §4.5 reports only pooled micro metrics, so the reader cannot tell whether the differences between r=0.1, 0.25, and 0.5 are meaningful or within sampling noise. Some curves in Fig. 1 appear to differ by only a few F1 points. The authors should provide per-run confidence intervals (e.g., bootstrap over the 128 runs), or a significance test, before asserting that the pattern is systematic.
  2. [§5.2, Fig. 1, §6] The prevalence peak may be a metric artifact. The authors themselves note that harmful precision and recall continue to improve as r increases, and the macro-F1 decline at high r comes from the non-harmful class. Under macro-F1, any classifier whose predicted prevalence roughly tracks the true ratio (as the PPV plots show) will tend to have lower macro-F1 at extreme r simply because one class dominates. To support the substantive claim of a 'moderate prevalence sweet spot' in §6, the paper should show that the non-monotonicity is not trivially produced by a prevalence-matching baseline, and should report a class-balance-robust metric such as Matthews correlation coefficient or balanced accuracy.
  3. [§3, Limitations] The prompts are random concatenations of independent sentences, and the Limitations section concedes that 'it may make the task easier for models, since they do not need to track harmful content in a flowing narrative.' The Discussion (§6) nonetheless draws 'direct implications' for RAG pipelines, content moderation, and multi-turn conversations—settings that involve coherent or discourse-structured inputs. Without at least a small coherent-document validation, the external validity of the central findings for these real-world applications remains unverified. The authors should either add such a condition or explicitly restrict the paper's claims to the synthetic stress-test regime.
  4. [§5.1, §5.2, Tables 1/3/4] The sentence-level baseline is not a controlled comparison with the long-context setting. The sentence-level prompt uses a binary yes/no format with a 4-shot prompt, while the long-context prompt asks for index extraction with a 1-shot prompt. Thus, claims such as 'multi-sentence prompting leads the model to approximate the underlying harm ratio more closely' (comparing PPV curves) confound context length with task format and instruction differences. This limits the interpretation of sentence-level versus long-context comparisons throughout §5.2.
minor comments (4)
  1. [Abstract and §4.3] The abstract states input lengths of 600–30,000 tokens, while the main experiments use 600–6,000 tokens (with 15k and 30k only in the appendix). Clarify in the abstract that the 30k results are appendix-only.
  2. [Fig. 3] The x-axis label 'JigsawT oxic' contains a typo, and the dataset order in Fig. 3 differs from other figures; harmonize for readability.
  3. [Appendix D–G] The text uses informal phrasing such as 'You can find their information on dilution analysis in Appendix E' and 'the same patterns are occurring'. Please revise to formal scientific style.
  4. [§4.5] The acronym PPV is defined as 'Predicted Prevalence Value', which may be confused with positive predictive value. Consider renaming to 'Predicted Prevalence' or explicitly stating the intended meaning to avoid ambiguity.

Circularity Check

0 steps flagged

No circularity: the reported patterns are direct measurements on external datasets and models, with no fitted parameters or self-referential construction.

full rationale

This is an empirical evaluation paper, not a derivation. The controlled variables (prompt length p, harm ratio r, harm region h, harm type t) are experimental design choices, not fitted parameters, and the conclusions are read directly off measured model outputs. Section 3 states: "we construct prompts artificially by randomly sampling harmful and non-harmful sentences," and the metrics in Section 4.5 (Macro-F1, PPV, precision, recall) are computed from the model's returned sentence indices against ground-truth indices with no fitted equations. Claims such as the prevalence peak near 0.25, dilution, the beginning-position advantage, and explicit-over-implicit advantage are empirical regularities observed across three external datasets (IHC, OffensEval, JigsawToxic) and three models (LLaMA-3, Qwen-2.5, Mistral); they are not obtained by solving equations that encode the claims. The paper's self-citations (Ghorbanpour et al. 2025a,b; Zhang et al. 2025) appear only as related-work references on sentence-level detection and are not load-bearing inputs to the current results. No fitted value is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The Limitation section's statement that the synthetic prompts "may make the task easier for models, since they do not need to track harmful content in a flowing narrative" is an external-validity caveat about transfer to coherent documents, not an admission of circularity. Thus the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

No fitted parameters or newly invented entities appear in this work. The central claims rest on the reliability of dataset annotations, on the validity of synthetic prompt construction as a stress test, and on the faithful parsing of model outputs. These are measurement assumptions rather than derived quantities.

axioms (3)
  • domain assumption Dataset labels in IHC, OffensEval, and JigsawToxic are treated as ground truth for harmful content and explicit/implicit type.
    All metrics and conclusions depend on the human annotations in these datasets being correct for the categories of harm being tested (§4.1).
  • domain assumption Randomly concatenating independently sampled sentences is a valid stress test that isolates the studied variables.
    The entire evaluation is built on synthetic prompts without discourse structure; the authors acknowledge this in §3 and Limitations.
  • domain assumption Model outputs in the form of comma-separated indices are parsed correctly and faithfully represent the model's detection decisions.
    The paper does not report parsing failures, malformed outputs, or any handling of outputs that do not follow the requested format (§4.4, §4.5).

pith-pipeline@v1.3.0-alltime-deepseek · 22645 in / 6802 out tokens · 62496 ms · 2026-08-04T11:13:16.875736+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) increasingly operate on long inputs, yet their behavior when harmful sentences are sparsely embedded within such inputs remains poorly understood. We present a sensitivity analysis that probes how LLMs extract harmful sentences embedded in long inputs. We construct long inputs by combining neutral and harmful sentences, and systematically vary four factors: input length (600--30,000 tokens), the proportion of harmful sentences (0.01--0.50), harm realization (explicit vs. implicit), and the position of harmful sentences within the input (beginning, middle, end), enabling a controlled stress-test evaluation. Experiments across toxic, offensive, and hate content, and across LLaMA-3.1, Qwen-2.5, and Mistral, reveal consistent patterns: sensitivity is non-monotonic with respect to harmful prevalence, peaking at moderate levels; sensitivity degrades as input length increases; harmful sentences placed earlier in the input are more strongly prioritized; and explicit harm is more reliably identified than implicit harm. These findings provide a systematic view of how LLMs prioritize harmful sentences in long input under controlled stress conditions, highlighting both emerging strengths and remaining challenges for safety-related use.

Figures

Figures reproduced from arXiv: 2510.05864 by Alexander Fraser, Faeze Ghorbanpour.

Figure 1
Figure 1. Figure 1: Prevalence analysis across datasets (IHC, OffensEval, JigsawToxic) with LLaMA-3. Each row shows [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Dilution analysis across datasets (IHC, OffensEval, JigsawToxic) with LLaMA-3. Each row reports Macro [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Region effect analysis with LLaMA-3 across datasets (IHC, OffensEval, JigsawToxic). Columns [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Type sensitivity analysis with LLaMA-3 across datasets (IHC, OffensEval, JigsawToxic). Columns [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Prevalence analysis across datasets (IHC, OffensEval, JigsawToxic) with Qwen-2.5. Each row shows [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Prevalence analysis across datasets (IHC, OffensEval, JigsawToxic) with Mistral. Each row shows Macro [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Dilution analysis across datasets (IHC, OffensEval, JigsawToxic) with Qwen-2.5. Each row reports Macro [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Dilution analysis across datasets (IHC, OffensEval, JigsawToxic) with Mistral. Each row reports Macro [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Region effect analysis with Qwen-2.5 across datasets (IHC, OffensEval, JigsawToxic). Columns [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Region effect analysis with Mistral across datasets (IHC, OffensEval, JigsawToxic). Columns correspond [PITH_FULL_IMAGE:figures/full_fig_p018_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Type sensitivity analysis with Qwen-2.5 across datasets (IHC, OffensEval, JigsawToxic). Columns [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Type sensitivity analysis with Mistral across datasets (IHC, OffensEval, JigsawToxic). Columns [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Oyster-II: Reinforcement Learning for Constructive Safety Alignment in Large Language Models

    cs.AI 2026-07 conditional novelty 6.0

    Zero-RL multi-stage constructive safety alignment with SERL and long-context training lets a 14B model match much larger models on safety without collapsing helpfulness or style.

Reference graph

Works this paper leans on

63 extracted references · 8 canonical work pages · cited by 1 Pith paper · 1 internal anchor

  1. [1]

    Ameeta Agrawal, Andy Dang, Sina Bagheri Nezhad, Rhitabrat Pokharel, and Russell Scheinberg. 2024. https://doi.org/10.18653/v1/2024.mrl-1.18 Evaluating multilingual long-context models for retrieval and reasoning . In Proceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024), pages 216--231, Miami, Florida, USA. Association for ...

    work page doi:10.18653/v1/2024.mrl-1.18 2024

  2. [2]

    Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, and Li Yang. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.19 ETC : Encoding long and structured inputs in transformers . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNL...

    doi:10.18653/v1/2020.emnlp-main.19 2020

  3. [3]

    Bang An, Shiyue Zhang, and Mark Dredze. 2025. https://doi.org/10.18653/v1/2025.naacl-long.281 RAG LLM s are not safer: A safety analysis of retrieval-augmented generation for large language models . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Vo...

    doi:10.18653/v1/2025.naacl-long.281 2025

  4. [4]

    Yang An, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei...

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.10671 2024

  5. [5]

    Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, et al. 2024. https://openreview.net/pdf?id=cw5mgd71jW Many-shot jailbreaking . Advances in Neural Information Processing Systems, 37:129696--129742

    2024

  6. [6]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. https://doi.org/10.18653/v1/2024.acl-long.172 L ong B ench: A bilingual, multitask benchmark for long context understanding . In Proceedings of the 62nd Annual Meeting of the Association for ...

    doi:10.18653/v1/2024.acl-long.172 2024

  7. [7]

    Parishad BehnamGhader, Nicholas Meade, and Siva Reddy. 2025. https://doi.org/10.18653/v1/2025.findings-acl.673 Exploiting instruction-following retrievers for malicious information retrieval . In Findings of the Association for Computational Linguistics: ACL 2025, pages 12962--12980, Vienna, Austria. Association for Computational Linguistics

    work page doi:10.18653/v1/2025.findings-acl.673 2025

  8. [8]

    Peters, and Arman Cohan

    Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. https://doi.org/10.48550/arXiv.2004.05150 Longformer: The long-document transformer . arXiv preprint arXiv:2004.05150

    Pith/arXiv arXiv doi:10.48550/arxiv.2004.05150 2020

  9. [9]

    Verma, Deepak Gupta, A

    Pronaya Bhattacharya, Vivek Kumar Prasad, A. Verma, Deepak Gupta, A. Sapsomboon, W. Viriyasitavat, and G. Dhiman. 2024. https://doi.org/10.1007/s11831-024-10115-5 Demystifying chatgpt: An in-depth survey of openai’s robust large language models . Archives of Computational Methods in Engineering, 31(5)

    work page doi:10.1007/s11831-024-10115-5 2024

  10. [10]

    Mazurek, Katie Shilton, and Hal Daum \'e Iii

    Yang Trista Cao, Lovely-Frances Domingo, Sarah Gilbert, Michelle L. Mazurek, Katie Shilton, and Hal Daum \'e Iii. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.209 Toxicity detection is NOT all you need: Measuring the gaps to supporting volunteer content moderators through a user-centric method . In Proceedings of the 2024 Conference on Empirical Meth...

    work page doi:10.18653/v1/2024.emnlp-main.209 2024

  11. [11]

    Tommaso Caselli, Valerio Basile, Jelena Mitrovi \'c , Inga Kartoziya, and Michael Granitzer. 2020. https://aclanthology.org/2020.lrec-1.760/ I feel offended, don ' t be abusive! implicit/explicit messages in offensive and abusive language . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6193--6202, Marseille, France. Eur...

    2020

  12. [12]

    Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean,...

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.11416 2022

  13. [13]

    cjadams, Jeffrey Sorensen, Julia Elliott, Lucas Dixon, Mark McDonald, nithum, and Will Cukierski. 2017. Toxic comment classification challenge. https://kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge. Kaggle

    2017

  14. [14]

    Marta R. Costa-jussà, Pierre Andrews, Mariano Coria Meglioli, Joy Chen, Joe Chuang, David Dale, Christophe Ropers, Alexandre Mourachko, Eduardo Sánchez, Holger Schwenk, Tuan Tran, Arina Turkatenko, and Carleigh Wood. 2025. https://aclanthology.org/2025.findings-acl.556.pdf Lcfo: Long context and long form output dataset and benchmarking . In Findings of t...

    2025

  15. [15]

    Remi Denton, Mark Díaz, Ian Kivlichan, Vinodkumar Prabhakaran, and Rachel Rosen. 2021. https://doi.org/10.48550/arXiv.2112.04554 Whose ground truth? accounting for individual and collective identities underlying dataset annotation

    Pith/arXiv arXiv doi:10.48550/arxiv.2112.04554 2021

  16. [16]

    Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019. https://doi.org/10.18653/v1/D19-1461 Build it break it fix it for dialogue safety: Robustness from adversarial human attack . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce...

    doi:10.18653/v1/d19-1461 2019

  17. [17]

    Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.29 Latent hatred: A benchmark for understanding implicit hate speech . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345--363, Online and ...

    doi:10.18653/v1/2021.emnlp-main.29 2021

  18. [18]

    Gemini Team . 2025. https://arxiv.org/abs/2507.06261 Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities . arXiv preprint arXiv:2507.06261

    Pith/arXiv arXiv 2025

  19. [19]

    Faeze Ghorbanpour, Daryna Dementieva, and Alexandar Fraser. 2025 a . https://aclanthology.org/2025.woah-1.39/ Can prompting LLM s unlock hate speech detection across languages? a zero-shot and few-shot study . In Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH), pages 413--425, Vienna, Austria. Association for Computational Linguistics

    2025

  20. [20]

    Faeze Ghorbanpour, Viktor Hangya, and Alexander Fraser. 2025 b . https://doi.org/10.18653/v1/2025.naacl-long.551 Fine-grained transfer learning for harmful content detection through label-specific soft prompt tuning . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Langua...

    doi:10.18653/v1/2025.naacl-long.551 2025

  21. [21]

    Aaron Grattafiori, Abhimanyu Dubey, et al. 2024. https://doi.org/10.48550/arXiv.2407.21783 The llama 3 herd of models . arXiv preprint arXiv:2407.21783

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.21783 2024

  22. [22]

    Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. https://doi.org/10.18653/v1/2022.acl-long.234 T oxi G en: A large-scale machine-generated dataset for adversarial and implicit hate speech detection . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P...

    doi:10.18653/v1/2022.acl-long.234 2022

  23. [23]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. https://openreview.net/forum?id=kIoBbc76Sy Ruler: What's the real context size of your long-context language models? In First Conference on Language Modeling (COLM 2024)

    2024

  24. [24]

    Mianqiu Huang, Xiaoran Liu, Shaojun Zhou, Mozhi Zhang, Qipeng Guo, Linyang Li, Pengyu Wang, Yang Gao, Chenkun Tan, Linlin Li, Qun Liu, Yaqian Zhou, Xipeng Qiu, and Xuanjing Huang. 2025. https://aclanthology.org/2025.llmsec-1.4/ L ong S afety: Enhance safety for long-context LLM s . In Proceedings of the The First Workshop on LLM Security (LLMSEC), pages 2...

    2025

  25. [25]

    Tao Huang. 2025. https://doi.org/10.1007/s10462-025-11328-1 Content moderation by llm: From accuracy to legitimacy . Artificial Intelligence Review, 58(10)

    doi:10.1007/s10462-025-11328-1 2025

  26. [26]

    Aman Jaiswal and Evangelos Milios. 2023. https://doi.org/10.48550/arXiv.2310.20558 Breaking the token barrier: Chunking and convolution for efficient long text classification with bert . arXiv preprint arXiv:2310.20558

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.20558 2023

  27. [27]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. https://doi.org/10.48550/arX...

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.06825 2023

  28. [28]

    Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan \" O . Arik. 2025. https://openreview.net/forum?id=oU3tpaR8fm Long-context llms meet rag: Overcoming challenges for long inputs in rag . In International Conference on Learning Representations (ICLR)

    2025

  29. [29]

    Farzaan Kaiyom, Ahmed Ahmed, Yifan Mai, Kevin Klyman, Rishi Bommasani, and Percy Liang. 2024. http://arxiv.org/abs/arXiv: not provided Helm safety: Towards standardized safety evaluations of language models

    2024

  30. [30]

    Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. 2024 a . https://proceedings.neurips.cc/paper_files/paper/2024/file/c0d62e70dbc659cc9bd44cbcf1cb652f-Paper-Datasets_and_Benchmarks_Track.pdf Babilong: Testing the limits of llms with long context reasoning-in-a-haystack . In Advances in Neural Info...

    2024

  31. [31]

    Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. 2024 b . https://proceedings.neurips.cc/paper_files/paper/2024/file/c0d62e70dbc659cc9bd44cbcf1cb652f-Paper-Datasets_and_Benchmarks_Track.pdf Babilong: Testing the limits of llms with long context reasoning-in-a-haystack . In Advances in Neural Info...

    2024

  32. [32]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. https://doi.org/10.1145/3600006.3613165 Efficient memory management for large language model serving with pagedattention . In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP '23, page 611–626, New York, ...

    arXiv 2023

  33. [33]

    Schuller

    Siddique Latif, Muhammad Usama, Mohammad Ibrahim Malik, and Björn W. Schuller. 2025. https://doi.org/10.1109/MCI.2024.3504833 Can large language models aid in annotating speech emotional data? uncovering new frontiers . IEEE Computational Intelligence Magazine, 20(1):66--77

    arXiv 2025

  34. [34]

    Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2024. https://doi.org/10.18653/v1/2024.acl-long.586 The dawn after the dark: An empirical study on factuality hallucination in large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    doi:10.18653/v1/2024.acl-long.586 2024

  35. [35]

    Kuan Li, Liwen Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Shuai Wang, and Minhao Cheng. 2025. https://icml.cc/virtual/2025/poster/46069 Lara: Benchmarking retrieval-augmented generation and long-context llms -- no silver bullet for lc or rag routing . In Proceedings of the 42nd International Conference on Machine Learning (ICML 2025)

    2025

  36. [36]

    Tian Li, Xiang Chen, Zhen Dong, Weijiang Yu, Yijun Yan, Kurt Keutzer, and Shanghang Zhang. 2022. https://doi.org/10.24963/ijcai.2022/585 Domain-adaptive text classification with structured knowledge from unlabeled data . In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI 2022), pages 4221--4227

    work page doi:10.24963/ijcai.2022/585 2022

  37. [37]

    Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.311 T oxic C hat: Unveiling hidden challenges of toxicity detection in real-world user- AI conversation . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4694--4702, Singapore. Associ...

    doi:10.18653/v1/2023.findings-emnlp.311 2023

  38. [38]

    Jiaheng Liu, Dawei Zhu, Zhiqi Bai, Yancheng He, Huanxuan Liao, Haoran Que, Zekun Wang, Chenchen Zhang, Ge Zhang, Jiebin Zhang, Yuanxing Zhang, Zhuo Chen, Hangyu Guo, Shilong Li, Ziqiang Liu, Yong Shan, Yifan Song, Jiayi Tian, Wenhao Wu, Zhejian Zhou, and et al. 2025. http://arxiv.org/abs/2503.17407 A comprehensive survey on long context language modeling ...

    arXiv 2025

  39. [39]

    Yida Lu, Jiale Cheng, Zhexin Zhang, Shiyao Cui, Cunxiang Wang, Xiaotao Gu, Yuxiao Dong, Jie Tang, Hongning Wang, and Minlie Huang. 2025. https://doi.org/10.18653/v1/2025.acl-long.1530 L ong S afety: Evaluating long-context safety of large language models . In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1...

    doi:10.18653/v1/2025.acl-long.1530 2025

  40. [40]

    Jitendra Singh Malik, Hezhe Qiao, Guansong Pang, and Anton van den Hengel. 2024. https://doi.org/10.1007/s41060-024-00650-6 Deep learning for hate speech detection: a comparative study . International Journal of Data Science and Analytics

    doi:10.1007/s41060-024-00650-6 2024

  41. [41]

    Sai Krishna Mendu, Harish Yenala, Aditi Gulati, Shanu Kumar, and Parag Agrawal. 2025. https://doi.org/10.24963/ijcai.2025/53 Towards safer pretraining: Analyzing and filtering harmful content in webscale datasets for responsible llms . In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2025), pages 466--474

    doi:10.24963/ijcai.2025/53 2025

  42. [42]

    John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. https://doi.org/10.18653/v1/2020.acl-main.396 Toxicity detection: Does context really matter? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4296--4305, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.acl-main.396 2020

  43. [43]

    Juan Manuel P \' e rez, Fernando Mar \' i a Luque, Daniel Zayat, Mart \' i n Kondratzky, Andr \' e s Moro, Paula Soledad Serrati, Julieta Zajac, Patricia Miguel, Natalia Debandi, Agust \' i n Gravano, and Valeria Cotik. 2023. https://doi.org/10.1109/ACCESS.2023.3258973 Assessing the impact of contextual information in hate speech detection . IEEE Access, ...

    arXiv 2023

  44. [44]

    Qwen Team . 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  45. [45]

    Tanmay Sen, Ansuman Das, and Mrinmay Sen. 2024. https://doi.org/10.48550/arXiv.2405.01577 Hatetinyllm: Hate speech detection using tiny large language models . arXiv preprint arXiv:2405.01577

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2405.01577 2024

  46. [46]

    Marco Antonio Stranisci and Christian Hardmeier. 2025. https://arxiv.org/abs/2503.05721 What are they filtering out? a survey of filtering strategies for harm reduction in pretraining datasets . arXiv preprint arXiv:2503.05721

    arXiv 2025

  47. [47]

    Bibek Upadhayay, Vahid Behzadan, and Amin Karbasi. 2024. http://arxiv.org/abs/2410.11272 Cognitive overload attack: Prompt injection for long context

    Pith/arXiv arXiv 2024

  48. [48]

    Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. 2021. https://doi.org/10.18653/v1/2021.acl-long.132 Learning from the worst: Dynamically generated datasets to improve online hate detection . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Langu...

    doi:10.18653/v1/2021.acl-long.132 2021

  49. [49]

    Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li

    Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2023 a . https://openreview.net/forum?id=kaHpo8OZw2 Decodingtrust: A comprehensive assessment of trustwort...

    2023

  50. [50]

    Han Wang, Ming Shan Hee, Md Rabiul Awal, Kenny Tsu Wei Choo, and Roy Ka-Wei Lee. 2023 b . https://doi.org/10.24963/ijcai.2023/694 Evaluating gpt-3 generated explanations for hateful content moderation . In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23), pages 6255--6263

    doi:10.24963/ijcai.2023/694 2023

  51. [51]

    Xindi Wang, Mahsa Salmani, Parsa Omidi, Xiangyu Ren, Mehdi Rezagholizadeh, and Armaghan Eshaghi. 2024 a . https://doi.org/10.24963/ijcai.2024/917 Beyond the limits: A survey of techniques to extend the context length in large language models . In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI-24), Survey T...

    doi:10.24963/ijcai.2024/917 2024

  52. [52]

    Yixu Wang, Yan Teng, Kexin Huang, Chengqi Lyu, Songyang Zhang, Wenwei Zhang, Xingjun Ma, Yu-Gang Jiang, Yu Qiao, and Yingchun Wang. 2024 b . https://doi.org/10.18653/v1/2024.naacl-long.263 Fake alignment: Are LLM s really aligned well? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hum...

    work page doi:10.18653/v1/2024.naacl-long.263 2024

  53. [53]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. https://www.aclweb.org/a...

    2020

  54. [54]

    Zachary Yang, Nicolas Grenon-Godbout, and Reihaneh Rabbany. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.663 Towards detecting contextual real-time toxicity for in-game chat . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9894--9906, Singapore. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.findings-emnlp.663 2023

  55. [55]

    Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Onta\ n on, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. https://doi.org/10.5555/3495724.3497174 Big bird: Transformers for longer sequences . In Advances in Neural Information Processing Systems 33 (NeurIPS 2020), pages 17283--17297

    arXiv 2020

  56. [56]

    Jiang Zhang, Qiong Wu, Yiming Xu, Cheng Cao, Zheng Du, and Konstantinos Psounis. 2024 a . https://doi.org/10.1609/aaai.v38i19.30178 Efficient toxic content detection by bootstrapping and distilling large language models . In Proceedings of the AAAI Conference on Artificial Intelligence, pages 21779--21787

    doi:10.1609/aaai.v38i19.30178 2024

  57. [57]

    Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. https://doi.org/10.18653/v1/P18-1125 Conversations gone awry: Detecting early signs of conversational failure . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    doi:10.18653/v1/p18-1125 2018

  58. [58]

    Yaqi Zhang, Viktor Hangya, and Alexander Fraser. 2025. https://aclanthology.org/2025.coling-main.188/ LLM sensitivity challenges in abusive language detection: Instruction-tuned vs. human feedback . In Proceedings of the 31st International Conference on Computational Linguistics, pages 2765--2780, Abu Dhabi, UAE. Association for Computational Linguistics

    2025

  59. [59]

    Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.830 S afety B ench: Evaluating the safety of large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)...

    doi:10.18653/v1/2024.acl-long.830 2024

  60. [60]

    Qingfei Zhao, Ruobing Wang, Yukuo Cen, Daren Zha, Shicheng Tan, Yuxiao Dong, and Jie Tang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1259 L ong RAG : A dual-perspective retrieval-augmented generation paradigm for long-context question answering . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22600...

    doi:10.18653/v1/2024.emnlp-main.1259 2024

  61. [61]

    Tianyi Zhuang, Chuqiao Kuang, Xiaoguang Li, Yihua Teng, Jihao Wu, Yasheng Wang, and Lifeng Shang. 2025. https://arxiv.org/abs/2502.17807 Docpuzzle: A process-aware benchmark for evaluating realistic long-context reasoning capabilities . arXiv preprint arXiv:2502.17807

    Pith/arXiv arXiv 2025

  62. [62]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  63. [63]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...