{"id":"9e6371b2-f0c6-4ffe-a79c-8dc7a7b574d5","arxiv_id":"2411.19832","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"X-Sensitive, a manually annotated 8,000-tweet dataset over six sensitive categories, lets fine-tuned language models beat off-the-shelf moderation APIs.","lead":"This paper introduces X-Sensitive, a manually annotated dataset of 8,000 tweets covering six sensitive categories: conflictual language, profanity, sexual content, drugs, self-harm, and spam. Fine-tuned language models trained on it outperform off-the-shelf moderation tools, suggesting that specialized datasets still matter for content moderation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fine-tuned models are trained and evaluated on the same permissive single-annotator labels; the claimed 10-15% advantage over off-the-shelf LLMs may be an artifact of this noise rather than a genuine detection improvement.","rationale":"The reader's weakest assumption identified the validity of the annotation labels as load-bearing, and I agree. The specific risk is that the permissive 'at least one yes' rule creates a training/evaluation distribution that fine-tuned models can memorize but off-the-shelf models have not been exposed to, so the reported advantage is potentially an artifact of the label noise. This concern directly undermines the abstract's headline claim of 'significant improvements' and the '10-15% overall' underperformance of proprietary models. The paper itself flags the aggregation methodology as a limitation, and the release of annotator-level labels makes a decisive test possible. Since the reader already assigned a CONDITIONAL verdict, my read does not move that verdict; the concrete test would determine whether the concern actually lands and whether the performance claims can be taken at face value.","tokens_in":15441,"tokens_out":4159,"duration_ms":37044,"concrete_test":"Download all_annotations.json from the X-Sensitive Hugging Face repository, rebuild binary and multi-label ground truth labels using a stricter rule (e.g., requiring at least 2/3 of annotators to say 'yes', or a majority vote with no 'not sure' counted as positive), and re-run the Table 2 evaluations for fine-tuned llama3-8b, tlm, RoBERTa and zero-/few-shot gpt-4o on the filtered test split. If the fine-tuned versus gpt-4o margin shrinks to non-significance or reverses, the central performance claim is an artifact of the single-positive label rule rather than a property of the dataset content.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 defines a positive label as arising when at least one annotator says 'yes' and the rest say 'yes' or 'not sure' (no direct 'no'). With Krippendorff's alpha at 0.49 (multi-label) and 0.56 (binary), this permissive rule means many tweets that most annotators were unsure about become positive training and test instances. The fine-tuned models in Section 5 are optimized to reproduce this exact noisy mapping, whereas off-the-shelf LLMs such as gpt-4o and LLaMA are not. Their lower scores in Table 2 may therefore reflect a mismatch with the annotation convention rather than a genuine deficit in detecting sensitive content. Moreover, the abstract's '10-15% overall' underperformance claim is not consistently supported by Table 2: the binary gap is 9.9 points (85.6 vs 75.7) and the multi-label gap is 4.9 points (69.8 vs 64.9), with no error bars or significance tests. The paper acknowledges in Section 8 that the aggregation methodology 'may be subject to differing opinions' and promises to release annotator-level labels, which are now linked in the Ethics statement, enabling a direct empirical test.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces X-Sensitive, a manually annotated multi-label dataset of 8,000 English tweets covering six sensitive content categories (conflictual language, profanity, sexually explicit content, drug-related content, self-harm, and spam). Tweets were collected using a keyword-expansion technique and annotated by crowdsourced workers, with labels aggregated via a permissive rule requiring at least one 'yes' and no direct 'no'. The authors evaluate fine-tuned models (RoBERTa, TimeLMs, Llama-3-8B) and off-the-shelf systems (zero-/few-shot LLMs and moderation APIs) in binary and multi-label settings, reporting that fine-tuning yields substantial gains and that the best model, fine-tuned Llama-3-8B, reaches macro-F1 scores of 85.6 (binary) and 69.8 (multi-label). The paper also analyzes annotator demographics and category overlap, and releases the dataset, models, and annotator-level labels.","tokens_in":15726,"tokens_out":4026,"duration_ms":33663,"significance":"If the label-quality concerns are addressed, X-Sensitive would constitute a valuable unified resource for sensitive content moderation, covering under-studied categories such as self-harm and drugs alongside more commonly studied categories. The open release of the data, fine-tuned models, and annotator-level labels is a notable strength, as is the breadth of the evaluation across different model families and API-based systems. However, the central performance claim of a 10-15% advantage over proprietary LLMs is not fully supported by the reported numbers, and the low inter-annotator agreement combined with a permissive label aggregation rule raises questions about whether fine-tuned models are learning a robust notion of sensitivity or merely reproducing a noisy annotation convention. The paper's demographic analysis and class-overlap statistics are informative and add to the resource's potential utility.","major_comments":[{"comment":"The label aggregation rule assigns a positive label when at least one annotator answers 'yes' and the remaining annotators answer 'yes' or 'not sure', with no direct 'no'. This is a very permissive rule, especially given that Krippendorff's alpha is 0.49 in the multi-label setting and 0.56 in the binary setting. With three annotators, a tweet with one 'yes' and two 'not sure' becomes a positive training and test instance. Since the fine-tuned models are trained to reproduce this mapping, their advantage over off-the-shelf LLMs may partly reflect the models' adaptation to this particular label convention rather than to a more robust notion of sensitive content. The paper should report results on high-agreement subsets (e.g., unanimous or majority-vote labels) and/or evaluate with the annotator-level labels now linked in the Ethics statement, to demonstrate that the main findings are not an artifact of this permissive aggregation rule.","section":"§3.3"},{"comment":"The abstract states that proprietary OpenAI models 'underperform by 10-15% overall', but the numbers in Table 2 do not consistently support this. The largest gap between the best fine-tuned model (llama3-8b) and the best zero-shot model (gpt-4o) is 9.9 points in the binary setting (85.6 vs 75.7) and 4.9 points in the multi-label setting (69.8 vs 64.9). No confidence intervals, standard deviations, or statistical significance tests are reported, and the multi-label gap is far below 10%. The claim should be revised to match the reported numbers, and ideally supported with variance estimates or significance tests.","section":"Abstract and Table 2"},{"comment":"The comparison with out-of-the-box systems is weakened by taxonomy mismatch. Perspective and OpenAI moderation API categories are mapped to X-Sensitive categories, but the mapping is partial (e.g., OpenAI's 'violence' categories are not mapped to Conflictual, and Perspective's 'THREAT' and 'INSULT' are not mapped). The paper acknowledges that the systems are 'not fully comparable', but the headline conclusion is nonetheless based on these comparisons. Please provide a complete mapping table, consider evaluating on a reduced label set that is common to all systems, and report which labels were excluded and how partial matches were treated.","section":"§4.2.3 and Table 2"}],"minor_comments":[{"comment":"The paragraph under 'Sexually Explicit Content' describes a classifier training procedure (using hatespeechdata.com and Perspective API) that is out of place in the taxonomy definition; move it to the data collection or model development section.","section":"§3.1"},{"comment":"The statement that llama3-8b achieves the best results 'notably achieved without any hyper-parameter tuning' is unclear given §4.2.1 says Ray Tune was used for the other fine-tuned models; clarify whether llama3-8b used default hyperparameters or a separate procedure.","section":"§5.1"},{"comment":"There is a typo in the column header: 'Profaninty' should be 'Profanity'. Also, consider using consistent model naming across the table and text (e.g., 'chat-gpt' vs 'gpt-4o').","section":"Table 2"},{"comment":"The Limitations section says 'we plan to release all the collected annotations', but the Ethics statement already links annotator-level labels; update the limitation to reflect that the release has occurred.","section":"§8"},{"comment":"In the multi-label prompt template, '6 sexually explicit' is missing a comma; this is a minor formatting issue but should be corrected for reproducibility.","section":"Appendix C.3"}],"recommendation":"major_revision","confidential_remarks":"The resource is potentially useful and the release of annotator-level labels is exemplary. However, the paper's headline quantitative claim is not fully supported by the reported numbers, and the label-noise issue is central enough that I would not recommend acceptance without a revision that addresses these points. The authors should be encouraged to rerun or supplement the evaluation with majority-vote or high-agreement labels and to report uncertainty estimates."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look, but the abstract oversells the model comparison. The real contribution is X-Sensitive, a new 8,000-tweet multi-label benchmark covering six sensitive categories, including self-harm, drugs, and spam, which are under-served by existing toxic-language-focused datasets. The annotation guidelines are grounded in platform community guidelines, the collection strategy tries to avoid keyword overfitting, and the authors release the dataset, the raw annotator-level labels, and fine-tuned models. That is a solid, genuinely useful package.\n\nThe soft spot is the labelling rule. A tweet is positive if at least one of at least three annotators says yes and the rest say yes or 'not sure' – no direct 'no' is allowed. With Krippendorff's alpha at 0.49 (multi-label) and 0.56 (binary), this permissive rule means a lot of the 'positive' instances rest on a single confident annotator with uncertainty from the others. The fine-tuned models are trained to reproduce exactly this noisy mapping, while off-the-shelf LLMs are not, so their lower scores partly reflect a mismatch in annotation convention rather than a genuine detection deficit. That said, the authors release the per-annotator labels, so anyone can re-aggregate under a stricter rule and test this directly. The Limitations section also flags the aggregation methodology as debatable. So this is a fixable concern, not a fatal flaw.\n\nThe other problem is the '10-15%' claim. Looking at Table 2, the best fine-tuned model beats zero-shot gpt-4o by 9.9 F1 points in the binary setting and 4.9 points in the multi-label setting. Against few-shot gpt-4o the gaps shrink to 2.3 and 1.9. The 10-15% only holds for the OpenAI moderation API or some of the smaller zero-shot baselines. The abstract should be rewritten to match the actual numbers. There are also no error bars, and no significance tests, so the fine-grained class-level differences should be read cautiously.\n\nThe dataset itself is the main deliverable, and for that the paper is a genuine contribution. The model comparison is secondary and currently over-claimed. With a revised abstract, a robustness check on the annotation aggregation, and some error bars, this would be a solid benchmark resource. I'd send it to peer review and ask for those changes. The raw labels being public makes the key check cheap.","headline":"A genuinely useful multi-label sensitive-content dataset with a serious annotation-aggregation caveat, and an abstract whose 10-15% claim is not backed by the table.","tokens_in":16214,"tokens_out":2857,"would_cite":true,"duration_ms":23745,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new 8,000-tweet benchmark shows fine-tuned models beat off-the-shelf content moderators by 10-15 points.","keywords":["sensitive content detection","content moderation","multi-label classification","fine-tuning","X-Sensitive dataset","social media","crowd annotation","off-the-shelf LLM evaluation"],"falsifier":"Re-annotate a random sample of about 200 test tweets with a panel of five trained moderators using a strict majority rule, then compare their labels to X-Sensitive. If per-category agreement is close to the original annotators' internal agreement (alpha around 0.49-0.56), the reported F1 differences between systems are partly measuring label noise, and the rankings could change on a cleaner gold standard.","tokens_in":15260,"feed_emoji":"🛡️","tokens_out":11709,"duration_ms":84313,"temperature":0.7,"pith_summary":"The paper's central claim is that the bottleneck in detecting sensitive content on social media is the absence of a unified, multi-category training resource, not model size. To close this gap it introduces X-Sensitive, a manually annotated dataset of 8,000 tweets spanning six sensitive categories: conflictual language, profanity, sexually explicit material, drug-related content, self-harm, and spam. Fine-tuning an 8-billion-parameter language model on this dataset reaches 85.6 macro-F1 in binary detection and 69.8 in multi-label detection, outperforming open and proprietary off-the-shelf models by 10-15 points overall. A 355-million-parameter model trained on social media comes within about 2 points of the best model, which the paper reads as evidence that specialised data, not scale, drives performance. If this holds, the dataset offers a practical, privacy-preserving alternative to moderation APIs that cannot be customised to specific sensitive categories.","feed_headline":"Fine-tuned models beat off-the-shelf AI on sensitive-content detection","feed_subtitle":"A new 8,000-tweet benchmark lifts fine-tuned Llama-3-8B 10-15 points over GPT-4o and moderation APIs.","key_machinery":"The central object is the X-Sensitive dataset itself: 8,000 tweets, multi-label, covering conflictual language, profanity, sexually explicit material, drug-related content, self-harm, and spam. It is constructed to avoid the usual weaknesses of keyword-scraped sets by expanding seed word lists with tweet-trained word embeddings, manually filtering low-precision terms, and holding out roughly half the test set so those tweets share no collection keywords with the training split. The carrying mechanism is the combination of a consistent taxonomy drawn from platform community guidelines, a recall-oriented annotation rule, and the use of the same collection and labelling pipeline across all six categories; this is what lets one model be trained and evaluated on all categories at once rather than on separate, inconsistently built datasets.","core_discovery":"On its own terms, the paper establishes that a purpose-built, multi-label dataset makes specialised sensitive-content classifiers substantially better than general-purpose moderation systems. X-Sensitive labels 8,000 tweets with six categories, collected through embedding-expanded keyword lists and annotated by crowd workers using a recall-oriented rule: a tweet receives a label when at least one annotator answers yes and the rest answer yes or not sure, so no direct veto is allowed. The paper reports that fine-tuned Llama-3-8B is the best system overall, with macro-F1 (the average of per-class F1 scores) of 85.6 in the binary setting and 69.8 in the multi-label setting; gpt-4o in a few-shot setting reaches 83.3 and 67.9, while out-of-the-box moderation APIs fall further behind. The authors also find that fine-tuning helps every model tested and that a much smaller social-media-pretrained model is only about 2 points behind, which supports their argument that the value lies in the dataset itself.","pith_inferences":["Beyond the paper: the no-veto label rule deliberately trades precision for recall, so a model trained on X-Sensitive will tend to over-flag borderline tweets, meaning more false positives for human moderators to review.","Beyond the paper: because annotator agreement is only moderate (alpha 0.49 multi-label, 0.56 binary), some part of the measured gap between fine-tuned and off-the-shelf models could be the fine-tuned models learning the labelling rule rather than the categories themselves; an expert re-annotation study would separate these.","Beyond the paper: the same collection pipeline could be ported to other languages or platforms to test whether the fine-tuning advantage persists, since the paper itself notes that the dataset is English-only and drawn from a single platform."],"forward_implications":["Fine-tuned models built on X-Sensitive can run locally, giving researchers a privacy-preserving alternative to sending data to external moderation APIs.","The 10-15 point gap between fine-tuned and off-the-shelf systems implies that general-purpose and commercial models need category-specific training data before they can be relied on for moderation of self-harm, drug, or spam content.","The close performance of the 355-million-parameter model shows that a moderately sized, social-media-pretrained model is a practical option when compute is limited.","The multi-label design captures overlaps such as profanity combined with conflictual or sexually explicit language, so a single system can flag multiple harms in one tweet.","The keyword-disjoint test split gives a more realistic estimate of performance on unseen vocabulary than earlier keyword-based datasets."],"supporting_citations":[{"why":"Supplies the Llama-3-8B model used for fine-tuning and for the zero-/few-shot comparisons.","marker":"AI@Meta, 2024"},{"why":"Provides the Transformers implementation used to fine-tune the masked language models.","marker":"Wolf et al., 2020"},{"why":"Provides the tweet-trained GloVe embeddings that drive the keyword-expansion algorithm.","marker":"Pennington et al., 2014"},{"why":"Supplies spam-labelled tweets and n-gram seeds used to build the spam category.","marker":"Founta et al., 2018"},{"why":"Supplies self-harm keyword seeds and the pro-eating-disorder research used to define that category.","marker":"Chancellor et al., 2016b"},{"why":"Grounds the taxonomy of sensitive categories in platform community guidelines.","marker":"Scheuerman et al., 2021"},{"why":"Perspective API is one of the out-of-the-box production baselines the dataset is evaluated against.","marker":"Google, 2023"},{"why":"Llama Guard 2 is the specialised off-the-shelf safety classifier used as another baseline.","marker":"Inan et al., 2023"},{"why":"Defines the reliability coefficient used to report annotator agreement on X-Sensitive.","marker":"Krippendorff, 2011"}],"fun_headline_variants":["Fine-tuned Llama-3-8B beats GPT-4o by 10-15% on sensitive content","New 8,000-tweet dataset lifts fine-tuned models over GPT-4o and APIs","Fine-tuned Llama tops GPT-4o and moderation APIs on sensitive tweets","Six-category benchmark improves sensitive-content detection for fine-tuned LLMs","Moderation APIs underperform fine-tuned Llama by 10-15% on sensitive content"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the crowd labels in X-Sensitive are accurate enough to serve as ground truth, even though annotator agreement is moderate (Krippendorff's alpha 0.49 multi-label, 0.56 binary) and a single 'yes' with no outright 'no' is enough to assign a label.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuned Llama-3-8B beats GPT-4o by 10-15% on sensitive content","New 8,000-tweet dataset lifts fine-tuned models over GPT-4o and APIs","Fine-tuned Llama tops GPT-4o and moderation APIs on sensitive tweets","Six-category benchmark improves sensitive-content detection for fine-tuned LLMs","Moderation APIs underperform fine-tuned Llama by 10-15% on sensitive content"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00209,"raw_usage":{"total_tokens":8137,"prompt_tokens":968,"completion_tokens":7169,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":7050}},"tokens_in":584,"tokens_out":7169,"duration_ms":41948,"temperature":1.0,"reasoning_tokens":7050,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:45:47.100171+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random sample of about 200 test tweets with a panel of five trained moderators using a strict majority rule, then compare their labels to X-Sensitive. If per-category agreement is close to the original annotators' internal agreement (alpha around 0.49-0.56), the reported F1 differences between systems are partly measuring label noise, and the rankings could change on a cleaner gold standard.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the taxonomy of sensitive categories in platform community guidelines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Perspective API is one of the out-of-the-box production baselines the dataset is evaluated against."}],"review_version":1}