Large Language Models for Detection of Life-Threatening Texts

Campbell Wilson; Janis Dalins; Thanh Thi Nguyen

REVIEW 4 major objections 5 minor 18 references

Large Language Models for Detection of Life-Threatening Texts

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LoRA-fine-tuned 7B open-source LLMs—Mistral and Llama-2 in particular—consistently outperform TF-IDF, word embeddings, topic modeling, and even BERT at detecting life-threatening texts, and they do so without needing upsampling for class…

desk verdict Useful but narrow benchmark: LoRA-tuned 7B LLMs clearly beat BERT and classical baselines on a threatening-text task, but the 'life-threatening' framing outruns the dataset and the evaluation lacks error bars and per-source breakdown. read the letter →

arxiv 2506.10687 v1 pith:B6HJK5OV submitted 2025-06-12 cs.CL cs.AIcs.LG

Thanh Thi Nguyen , Campbell Wilson , Janis Dalins This is my paper

classification cs.CLcs.AIcs.LG

keywords largelanguagemodelslife-threateningtextdetectionLoRAfine-tuningclassimbalanceupsamplingclassificationMistral7BLlama-2

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that LoRA-fine-tuned 7B large language models—Mistral, Llama-2, and to a lesser extent Gemma—consistently outperform traditional text-classification methods (TF-IDF, word embeddings, topic modeling, and BERT) at detecting life-threatening texts in English. Across balanced, imbalanced, and extremely imbalanced datasets, Mistral and Llama-2 achieve the highest accuracy, F-scores, and AUC, with Gemma slightly behind. The paper also finds that upsampling the minority class helps traditional methods but gives little or no benefit to the LLMs, suggesting that LLMs handle class imbalance without resampling. If these results hold, effective threat-detection systems could be built with small fine-tuning datasets and no manual preprocessing, which matters for real-world moderation and crisis intervention.

What carries the argument

The central object is the LoRA (low-rank adaptation) fine-tuning update: for a frozen pretrained weight matrix $M_0 \in \mathbb{R}^{d \times k}$, the learned update is $\Delta M = M_A M_B$ with $M_A \in \mathbb{R}^{d \times r}$, $M_B \in \mathbb{R}^{r \times k}$, and rank $r=8$, applied to the forward pass as $h = M_0 x + \Delta M x$. This keeps the 7B models' pretrained weights fixed while learning only a small set of parameters, using cross-entropy loss with AdamW and a 128-token window. The evaluation machinery is the six constructed datasets: 915 threatening texts (606 from the dynamically generated hate speech dataset plus 309 from the TEL corpus) paired with 1,000, 5,000, or 10,000 non-threatening texts from two different sources, which creates the balanced, imbalanced, and extremely imbalanced scenarios used in the comparisons.

What would settle it

Evaluate the fine-tuned Mistral-7B and Llama-2-7B models on an expert-verified set of messages from crisis helplines or police threat reports; if their F1 and AUC do not clearly exceed TF-IDF and BERT on that set, the paper's claim of consistent superiority over traditional methods would not transfer to deployment.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that parameter-efficient fine-tuning of open-source 7B large language models via LoRA turns them into reliable detectors of life-threatening language. In comparisons across six datasets spanning balanced, imbalanced, and extremely imbalanced class ratios, Mistral-7B and Llama-2-7B achieve the highest accuracy, F1/F0.5/F2 scores, and AUC, with Gemma-7B slightly behind; all three surpass TF-IDF, GloVe, Word2Vec, LDA, LSI, and BERT-en-uncased. The paper also shows that upsampling the minority class improves the F-scores and AUC of traditional methods but leaves LLM performance essentially unchanged, which it reads as evidence that the LLMs handle class imbalance without resampling.

Load-bearing premise

The paper assumes that the 915 texts labeled as threatening—606 from a hate-speech dataset and 309 from a forensic linguistic corpus—are a representative and correctly labeled sample of real-life life-threatening language, yet it reports no manual verification or label-quality audit.

Editorial extensions

If this is right

A practical life-threatening language detector can be trained by LoRA fine-tuning a 7B open LLM on a few hundred labeled examples, with no preprocessing, lemmatization, or separate feature engineering.
Deployment on skewed real-world data does not require upsampling when using Mistral or Llama-2; their F1 and AUC remain high at 9% positive-class prevalence, while traditional methods degrade sharply without resampling.
Because the LLMs beat BERT, a smaller bidirectional transformer, the paper's results imply that model scale and pretraining diversity add real signal beyond contextual word representations for this task.
Evaluation of threat detectors should report F-scores and AUC alongside accuracy, since accuracy alone is near-saturated (above 91%) on extremely imbalanced datasets and hides the differences between methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the same pattern holds beyond English, the LoRA recipe could be dropped into low-resource languages where only a few dozen threatening messages exist; the paper identifies this as future work but does not test it.
The near-identical performance with and without upsampling suggests the LLMs are learning semantic markers of threat rather than relying on class-frequency priors; this could be probed by varying the positive/negative ratio in training and watching F1, or by attention analysis.
A deployment caveat the paper leaves implicit: the 915 positive examples come from hate speech and forensic corpora, so the detector may be tuned to the linguistic style of those corpora; whether it generalizes to crisis-helpline messages or police reports is an open empirical question.
The consistent Mistral/Llama-2 edge over BERT at the same 7B scale points to a practical cost-performance trade: LoRA fine-tuning one 7B model is computationally feasible on a single GPU, making this approach attainable for smaller organizations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, quick take on 2506.10687. The paper is a solid, conventional benchmark: LoRA-fine-tuned Mistral and Llama-2 7B outperform TF-IDF, GloVe/Word2Vec, LDA/LSI, and BERT on six balanced/imbalanced splits for threatening text detection. The margins are large (F1 often 20–60 points over BERT in the imbalanced sets), so the qualitative claim is almost certainly true as a statement about these datasets. The upsampling result—traditional methods benefit, LLMs barely move—is also credible and consistent across the tables.

It does what a benchmark should: clear methods, standard baselines, multiple metrics including F0.5/F2/AUC rather than just accuracy, and six scenarios that let you see how imbalance changes the ranking. The writeup is straightforward, and the citations to prior threatening-text work are appropriate. No code or data is released, which is a real reproducibility gap.

Soft spots, in rough order of importance. First, construct validity. The 915 positives are 606 'threatening' texts from Vidgen's dynamically generated hate speech corpus plus 309 from the TEL forensic corpus. Those are two different genres with different annotation procedures; 'threatening' in a hate-speech taxonomy is not the same as 'life-threatening' in a crisis or law-enforcement setting. The paper never reports results per source, so we don't know whether the LLM gains come mostly from the Vidgen subset. The title and abstract claim 'life-threatening' detection, which overreaches the data. This is a validity concern, not a computational error; the numbers are probably right for the positive class as constructed.

Second, the evaluation rests on a single random 90/10 split with no repeated runs, no confidence intervals, and no significance tests. With roughly 92 test positives, some of the smaller gaps could be noise, though the LLM-versus-BERT differences are large enough that most would survive resampling. Third, label quality is not audited; there is no manual verification or agreement statistic for the 915 positives.

None of these sink the paper's core comparative finding, but they bound what it can claim. It is a benchmark for threat detection, not a deployable system and not a rigorous causal story about LLM superiority. The stress-test worry about per-source results is legitimate; the paper should report Vidgen and TEL separately or justify pooling them.

Who should read it: people working on harmful or threatening language detection who want a quick reference on LoRA-tuned 7B models versus older baselines. It deserves a serious referee; the data validity and split robustness need to be addressed, and code and data should be required before acceptance.

Referee Report

4 major / 5 minor

Summary. The paper proposes fine-tuning three 7B open-source LLMs (Gemma, Mistral, Llama-2) with LoRA for binary classification of life-threatening text, comparing against TF-IDF, GloVe, Word2Vec (CBOW/skip-gram), LDA/LSI topic models, and BERT-en-uncased. The experimental design creates six datasets by pairing a fixed set of 915 threatening texts (606 from the dynamically generated hate speech corpus and 309 from the TEL corpus) with non-threatening texts sampled from two sources at three imbalance ratios. Results are reported as accuracy, F1/F0.5/F2, and AUC for a single 90/10 split, with and without upsampling. The paper concludes that Mistral and Llama-2 consistently outperform all traditional methods and handle imbalance without upsampling.

Significance. If the findings hold, the paper provides evidence that parameter-efficient fine-tuned 7B LLMs are strong detectors of threatening language with relatively little data and no upsampling, which is practically relevant for online safety and mental health applications. The study is commendable for comparing three LLMs against multiple classical baselines across balanced and imbalanced settings, and for reporting F-scores and AUC rather than accuracy alone. However, the significance is contingent on validating the positive class as genuinely life-threatening and on establishing that the reported performance is not an artifact of a single split or pretraining overlap.

major comments (4)

[Section 4, Table 1] The 915 positive examples are a union of 606 'threatening' texts from the dynamically generated hate speech dataset [18] and 309 texts from the TEL corpus [3], with no manual audit, inter-annotator agreement, or example-level verification reported. Because 'threatening' within a hate speech taxonomy is not the same as 'life-threatening' (e.g., it may include hyperbole or generic abuse), and because the two sources differ in genre and provenance, the aggregate benchmark does not by itself support the paper's claim to detect life-threatening language. The authors should report per-source performance, provide representative examples from both sources, and validate on an external set of explicitly life-threatening texts.
[Section 4, Table 1] The negative class is drawn from 'not-hate' texts in [18] (Datasets 1, 3, 5) and [6] (Datasets 2, 4, 6), but 'not-hate' is not equivalent to 'non-life-threatening.' A text can be not-hate yet contain self-harm or homicidal intent, and a life-threatening message need not be hateful. This label mismatch means the classifiers may be learning a hate-vs-not-hate boundary rather than a life-threatening-vs-benign boundary; the paper should discuss this and ideally re-annotate the negative class or use a corpus annotated for crisis/self-harm content.
[Section 4, last paragraph; Tables 2-6] All reported numbers come from a single random 90/10 split, with no standard deviations, repeated runs, or significance tests. The test sets are small (roughly 191 instances for Dataset 1, 1,091 for Dataset 6), and while the observed gaps between Mistral/Llama-2 and BERT/TF-IDF are substantial (e.g., Table 2: F1 of 91.62 vs 77.53 in Dataset 1), the claim of 'consistent' superiority needs either multiple random seeds with confidence intervals or paired tests such as McNemar's test. This is a load-bearing point for the conclusion that LLMs are uniformly better.
[Sections 2 and 4] The positive-class corpora are public and predate the pretraining of Gemma, Mistral, and Llama-2 (Vidgen et al. 2021; TEL corpus 2022), so test instances may appear in the pretraining data, potentially inflating LLM performance relative to BERT and bag-of-words baselines. The paper does not address this contamination risk. The authors should at least measure n-gram overlap or perplexity of test texts under the base models, or validate on newly collected time-stamped data, to rule out memorization as an alternative explanation for the large gaps.

minor comments (5)

[Equations (3) and (4)] The cross-entropy definition is garbled; l_n is not properly defined as the per-instance loss, the subscript n in exp(ˆy_{n,n}) appears duplicated, and w (class weights) is introduced without explanation. Replace with the standard softmax cross-entropy formula.
[Sections 2.2-2.5] Headings contain spacing artifacts ('W ord Embedding', 'GloV eGloVe', 'T opic Modeling', 'BER T'); these should be corrected.
[Section 3] The hyperparameter list omits several settings needed for reproducibility (e.g., warmup steps, weight decay, gradient accumulation, LoRA alpha and dropout, random seed). Add a full configuration table.
[Section 4] It is not stated whether any text preprocessing (tokenization, stopword removal, etc.) was applied for the non-LLM baselines, making the comparison hard to reproduce; specify the exact preprocessing for TF-IDF, embeddings, LDA/LSI, and BERT.
[Section 7] Future work mentions quantization but the paper does not report compute cost or inference latency; a brief note on practical resource requirements would help practitioners.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports direct held-out measurements on defined test splits, with no fitted quantity renamed as a prediction and no load-bearing self-citation.

full rationale

The paper contains no claimed derivation from first principles and no mechanism by which a test-label quantity is constructed from the model output. Each competing method is trained on a 90% split and evaluated on a disjoint 10% split, with all metrics (accuracy, F1/F0.5/F2, AUC) reported as direct measurements. No parameter is fitted to the test set and then renamed as a prediction; the LoRA update equations are standard background cited to Hu et al., not an input that encodes the comparative result. The central claim that Mistral and Llama-2 outperform traditional methods is an empirical observation on six datasets, not a theorem resting on a self-citation chain. The main legitimate concern is construct validity of the 915 positive examples, which combine 606 'threatening' samples from the Vidgen hate speech dataset with 309 TEL corpus texts and receive no label audit; however, that is an external-validity or benchmark-composition issue, not circularity, because the reported numbers remain correct measurements for the datasets as defined in Table 1. Accordingly, the circularity score is 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard supervised learning assumptions and the chosen hyperparameters; no new entities are introduced.

free parameters (5)

LoRA rank = 8
Set without systematic tuning; affects capacity of the learned update.
Learning rate = 2e-5
Standard AdamW value, chosen by hand.
Number of epochs = 10
Fixed for all models without early stopping or ablation.
Max sequence length = 128 tokens
Truncates/pads all inputs; may cut off long threatening messages.
Mini-batch size = 16
Fixed; affects optimization.

assumptions (3)

domain assumption Pretrained LLMs and BERT contain representations that LoRA fine-tuning can adapt to life-threatening text classification with small data.
The method assumes transfer learning works for this domain; not proven by the paper.
domain assumption Labels in the source corpora (TEL and dynamically generated hate speech) correctly distinguish threatening from non-threatening text.
Used as ground truth in Section 4; no manual audit.
domain assumption The 90/10 random split produces test sets representative of the class distribution.
Single split without stratification or repeated sampling.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Large Language Models for Detection of Life-Threatening Texts." pith.science (2026). https://pith.science/paper/B6HJK5OV

@misc{pith2026250610687,
  author       = {Pith},
  title        = {Pith review of: Large Language Models for Detection of Life-Threatening Texts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B6HJK5OV}},
  note         = {Machine review of arXiv:2506.10687}
}

read the original abstract

Detecting life-threatening language is essential for safeguarding individuals in distress, promoting mental health and well-being, and preventing potential harm and loss of life. This paper presents an effective approach to identifying life-threatening texts using large language models (LLMs) and compares them with traditional methods such as bag of words, word embedding, topic modeling, and Bidirectional Encoder Representations from Transformers. We fine-tune three open-source LLMs including Gemma, Mistral, and Llama-2 using their 7B parameter variants on different datasets, which are constructed with class balance, imbalance, and extreme imbalance scenarios. Experimental results demonstrate a strong performance of LLMs against traditional methods. More specifically, Mistral and Llama-2 models are top performers in both balanced and imbalanced data scenarios while Gemma is slightly behind. We employ the upsampling technique to deal with the imbalanced data scenarios and demonstrate that while this method benefits traditional approaches, it does not have as much impact on LLMs. This study demonstrates a great potential of LLMs for real-world life-threatening language detection problems.

Figures

Figures reproduced from arXiv: 2506.10687 by the authors.

**Figure 1.** The method for fine-tuning a pretrained LLM for text classification tasks. Text data are first tokenized with an LLM tokenizer into token IDs, which are then used as inputs to the learnable PEFT model, constructed by transferring the original LLM weights to the Hugging Face format and loaded via the PEFT-LoRA mechanism. update of parameters for a pretrained weight matrix M0 ∈ R d×k is determined by the product of tw… view at source ↗

**Figure 2.** The difference between regular fine-tuning and LoRA fine-tuning. The LoRA matrices MA and MB approximate the weight update matrix ∆M, with the inner dimension rank r being a hyperparameter. term “labels” refers to the target y, while the “outputs” generated by the learnable PEFT model signify the logits yˆ. The cross-entropy loss function, averaged over the mini-batch size N, between the output yˆ and the target y … view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages

[18]

In: The 59th Annual Meeting of the ACL

Vidgen, B., Thrush, T., Waseem, Z., Kiela, D.: Learning from the worst: Dynam- ically generated datasets to improve online hate detection. In: The 59th Annual Meeting of the ACL. pp. 1667–1682 (2021)

work page 2021
[3]

Gales, T., Nini, A., Symonds, E.: The threatening English language (TEL) corpus (Jul 2022), https://doi.org/10.5281/zenodo.6815671

work page doi:10.5281/zenodo.6815671 2022
[6]

Hugging Face: Dataset card for Measuring Hate Speech (Jan 2023), https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech

work page 2023
[1]

CEUR Workshop Proceedings3159, 744–762 (2021)

Amjad, M., Zhila, A., Sidorov, G., Labunets, A., Butt, S., Amjad, H., Vitman, O., Gelbukh, A.: Overview of abusive and threatening language detection in Urdu at FIRE 2021. CEUR Workshop Proceedings3159, 744–762 (2021)

work page 2021
[2]

In: The North American Chapter of the ACL: Human Language Technologies

Devlin,J.,Chang,M.W.,Lee,K.,Toutanova,K.:Bert:Pre-trainingofdeepbidirec- tional transformers for language understanding. In: The North American Chapter of the ACL: Human Language Technologies. pp. 4171–4186 (2019)

work page 2019
[4]

Gemma Team, Google DeepMind: Gemma: Open models based on Gemini research and technology (Feb 2024), https://storage.googleapis.com/deepmind- media/gemma/gemma-report.pdf

work page 2024
[5]

arXiv preprint arXiv:2106.09685 (2021)

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

arXiv 2021
[7]

arXiv preprint arXiv:2310.06825 (2023)

Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.d.l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al.: Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)

arXiv 2023

Show all 18 references

[8]

In: Conference on EMNLP

Kudo, T., Richardson, J.: SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In: Conference on EMNLP. pp. 66–71 (2018)

2018
[9]

arXiv preprint arXiv:2303.15647 (2023)

Lialin, V., Deshpande, V., Rumshisky, A.: Scaling down to scale up: A guide to parameter-efficient fine-tuning. arXiv preprint arXiv:2303.15647 (2023)

2023 arXiv
[10]

https://github.com/huggingface/peft (2022)

Mangrulkar, S., Gugger, S., Debut, L.: PEFT: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft (2022)

2022
[11]

Applied Sciences12(20), 10342 (2022)

Mehmood, A., Farooq, M.S., Naseem, A., Rustam, F., Villar, M.G., Rodríguez, C.L., Ashraf, I.: Threatening Urdu language detection from tweets using machine learning. Applied Sciences12(20), 10342 (2022)

2022
[12]

IEEE Access11, 106503–106515 (2023)

Rehan, M., Malik, M.S.I., Jamjoom, M.M.: Fine-tuning transformer models using transfer learning for multilingual threatening text identification. IEEE Access11, 106503–106515 (2023)

2023
[13]

In: The 1st Workshop on Perspectivist Approaches to NLP @Language Resources and Evaluation Conference (LREC)

Sachdeva, P., Barreto, R., Bacon, G., Sahn, A., Von Vacano, C., Kennedy, C.: The measuring hate speech corpus: Leveraging Rasch measurement theory for data per- spectivism. In: The 1st Workshop on Perspectivist Approaches to NLP @Language Resources and Evaluation Conference (L...

2022
[14]

Neurocomputing568, 127063 (2024)

Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., Liu, Y.: RoFormer: Enhanced trans- former with rotary position embedding. Neurocomputing568, 127063 (2024)

2024
[15]

TensorFlow: Implementation of the BERT encoder API - Encoder (Oct 2020), https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4

2020
[16]

TensorFlow: Implementation of the BERT encoder API - Preprocessor (Oct 2020), https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3

2020
[17]

arXiv preprint arXiv:2307.09288 (2023)

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bash- lykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [18]

In: The 59th Annual Meeting of the ACL

Vidgen, B., Thrush, T., Waseem, Z., Kiela, D.: Learning from the worst: Dynam- ically generated datasets to improve online hate detection. In: The 59th Annual Meeting of the ACL. pp. 1667–1682 (2021)

work page 2021

[2] [3]

Gales, T., Nini, A., Symonds, E.: The threatening English language (TEL) corpus (Jul 2022), https://doi.org/10.5281/zenodo.6815671

work page doi:10.5281/zenodo.6815671 2022

[3] [6]

Hugging Face: Dataset card for Measuring Hate Speech (Jan 2023), https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech

work page 2023

[4] [1]

CEUR Workshop Proceedings3159, 744–762 (2021)

Amjad, M., Zhila, A., Sidorov, G., Labunets, A., Butt, S., Amjad, H., Vitman, O., Gelbukh, A.: Overview of abusive and threatening language detection in Urdu at FIRE 2021. CEUR Workshop Proceedings3159, 744–762 (2021)

work page 2021

[5] [2]

In: The North American Chapter of the ACL: Human Language Technologies

Devlin,J.,Chang,M.W.,Lee,K.,Toutanova,K.:Bert:Pre-trainingofdeepbidirec- tional transformers for language understanding. In: The North American Chapter of the ACL: Human Language Technologies. pp. 4171–4186 (2019)

work page 2019

[6] [4]

Gemma Team, Google DeepMind: Gemma: Open models based on Gemini research and technology (Feb 2024), https://storage.googleapis.com/deepmind- media/gemma/gemma-report.pdf

work page 2024

[7] [5]

arXiv preprint arXiv:2106.09685 (2021)

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

arXiv 2021

[8] [7]

arXiv preprint arXiv:2310.06825 (2023)

Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.d.l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al.: Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)

arXiv 2023