Pith. sign in

REVIEW 4 major objections 6 minor 13 references

Towards an LLM-based method for quantifying the sexual content in song lyrics

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A reproducible LLM scoring procedure for lyrics, calibrated against human labels, measures nine thematic dimensions at scale; applied to 1,259 reggaeton songs it finds explicit sexual content roughly doubled from 2002 to 2025 while…

desk verdict A transparent, well-scoped proof-of-concept for LLM-based lyric scoring with released code, prompt, and corpus; the main soft spot is the thin human-validation layer underneath every score. read the letter →

arxiv 2608.08885 v1 pith:23Z4ZZ75 submitted 2026-08-09 physics.soc-ph cs.CLcs.SD

Ignacio M. Sticco This is my paper · ORCID
classification physics.soc-phcs.CLcs.SD
keywords LLMscoringsonglyricsreggaetonsexualexplicitnessthematiccontentanalysisSpotifyexplicitflaglongitudinalmeasurementmethod
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to turn the familiar claim that reggaeton lyrics are saturated with sexual content into something measurable. It proposes a reproducible method in which a large language model scores each song on nine thematic dimensions on a 5-point ordinal scale, using a scoring prompt that is calibrated against hand-labeled songs before being applied. Applied to 1,259 songs by 12 reggaeton artists released between 2002 and 2025, the method produces three headline findings: sexually suggestive content is nearly twice as prevalent as sexually explicit content; explicit content rises over the two decades while suggestiveness stays flat; and Spotify's explicit flag marks only 26.8% of the songs the method scores as sexually explicit. If the scores are valid measurements, the paper provides a reusable, auditable measurement layer for content analysis of song lyrics, not limited to sexual content.

What carries the argument

The load-bearing mechanism is the scoring prompt, a fixed instruction text given to gpt-5.1 for each song, which defines nine thematic dimensions and anchors each point of a 0–4 ordinal scale with examples. The key conceptual distinction is between sexual suggestiveness (evoking sexuality through metaphor or connotation) and sexual explicitness (naming sexual acts, body parts, or behavior directly); for these two dimensions the model must supply a written justification that is stored for auditing. The prompt is calibrated by hand-labeling about ten songs, comparing the model's scores to those labels, revising the prompt until aligned, and then checking alignment on five held-out songs. Per-song raw scores are rescaled to a 0–1 range, a composite sexual score is defined as the average of the two normalized sexual dimensions, and duplicate songs are deduplicated both within and across artists. This prompt-plus-calibration loop is what carries the claim that the resulting numbers are measurements rather than arbitrary model outputs.

What would settle it

Take a random sample of, say, 100 songs from the released corpus, have two or more independent human raters score them under the paper's dimension definitions, and compare human scores to the model's scores with an agreement measure such as weighted kappa or intraclass correlation. If human–model agreement is no better than chance, or if independent human coding finds that most of the 456 songs Spotify leaves unflagged are not sexually explicit, the central measurement claim fails.

Watch

Extended reading notes

Core claim

The paper's central discovery, stated on its own terms, is that a single LLM scoring protocol, calibrated on a small hand-labeled sample, can assign usable ordinal scores for nine thematic dimensions of song lyrics, and that those scores support quantitative answers to questions about magnitude that qualitative studies could only assert. The method deliberately separates sexual suggestiveness from sexual explicitness and finds the two are related but not interchangeable ($r=0.47$): suggestiveness has the highest mean of the nine dimensions (0.45) while explicitness is about half as prevalent (0.27). Over 2002–2025 the fitted sexual-explicitness score roughly doubles, from 0.15 to 0.31 ($r=0.39$, $p=0.065$), and the rise survives restricting the variation to each artist's own catalogue (0.064 per decade; 95% CI $[0.028, 0.130]$), while suggestiveness shows no detectable trend ($r=0.18$, $p=0.40$). The method also finds that of 623 songs scored as containing explicit sexual content, Spotify's flag marks only 167 (26.8%), leaving 456 songs, or 36.2% of the corpus, unflagged.

Load-bearing premise

The entire analysis rests on the assumption that the LLM's numeric scores really measure the intended themes after a calibration step that uses about ten hand-labeled songs and a check on five, with no reported agreement statistic or confidence interval.

Editorial extensions

If this is right

  • If the scores are valid, the rise in reggaeton's sexual content from 2002 to 2025 is concentrated in explicitness, not suggestiveness: the fitted explicit score roughly doubles while suggestiveness remains statistically flat.
  • Spotify's explicit flag is not a reliable proxy for sexual explicitness in this corpus, since it misses about 73% of the songs the method scores as sexually explicit.
  • Artists within the genre differ in kind, not just degree: Plan B's profile is dominated by both sexual dimensions, Daddy Yankee's by party/nightlife with low explicitness, and Camilo's by romantic emotion with explicitness nearly absent.
  • The same calibrated prompt-and-scoring procedure can be redirected to other coding schemes and other lyric corpora or languages, with a new calibration cycle for each context.
  • Some longitudinal trends are compositional rather than behavioral: romantic emotion and street crime trends do not survive artist-demeaned analysis, while explicitness and substance use do.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because Spotify's flag is supplied by rights-holders with no published criteria, the large unflagged gap suggests that content-moderation or age-rating systems built on such flags will under-rate sexual explicitness in Spanish-language catalogs; the paper does not itself test platform-level consequences.
  • The calibration sample of about ten songs and a five-song held-out check is too small to certify the scores as ground truth; a formal inter-rater study on a larger random sample would be needed before reusing the released scores as a benchmark.
  • The suggestiveness/explicitness distinction could be imported into longitudinal studies of other lyric corpora, such as English-language pop or hip-hop, to ask whether the documented rise in explicit content comes from direct naming or from innuendo.
  • Because the paper notes that gpt-5.1 is a model alias that the provider can repoint, exact numerical replication of the released scores requires pinning a model snapshot; the paper's own numbers should be treated as version-specific.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a three-step, open-source method for scoring thematic content in song lyrics using an LLM: (1) collect lyrics and metadata from Spotify and Genius with a manual review pass, (2) score each song on nine ordinal dimensions (0-4) using gpt-5.1 with a prompt calibrated against hand-labeled songs, and (3) normalize, deduplicate, and assemble a composite sexual score. The method is applied to 1,259 reggaeton songs by 12 artists spanning 2002-2025. The empirical analyses describe the corpus, compare artists, trace longitudinal trends, and compare the method's sexual-explicitness score against Spotify's explicit flag. The main findings are that sexual suggestiveness is about twice as prevalent as sexual explicitness, artists differ markedly (composite scores 0.14 to 0.56), explicit sexual content rises over time while suggestiveness stays flat, and Spotify's flag marks only 26.8% of songs the method scores as sexually explicit. The paper releases the data collection code, the scoring prompt, and the corpus.

Significance. If the scoring method is valid, the paper makes a valuable methodological contribution: a reusable, auditable, and adaptable measurement layer for content analysis of lyrics, complementing existing hand-coding and word-frequency approaches. The explicit/suggestive distinction, the artist-demeaned trend analysis, and the external comparison with Spotify's flag are useful and go beyond prior work. The transparency in releasing code, prompt, and corpus, as well as the candid discussion of limitations (model specificity, language bias, unpinned alias), are strengths. However, the significance depends heavily on whether the LLM scores can be trusted as measurements, which the current validation evidence does not adequately establish; the empirical claims, while plausible, inherit this foundational uncertainty.

major comments (4)
  1. [Section 2.2]
  2. [Section 3.4 / Figure 10]
  3. [Section 2.4]
  4. [Section 3.3]
minor comments (6)
  1. [Abstract and Section 3.4]
  2. [Section 2.2]
  3. [Section 3.2]
  4. [Section 3.3]
  5. [Section 3.4]
  6. [Section 2.1]

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the composite index is explicitly defined, calibration uses external human labels, and no result reduces to a fitted parameter or self-citation.

full rationale

I walked the claimed derivation chain. The paper's methodological contribution is a prompt-calibrated LLM rating procedure; its empirical results are descriptive summaries of the resulting scores. The one constructed measure, sexualcomposite, is defined in Section 2.3 as the average of explicit_norm and suggestive_norm; the paper explicitly notes in Section 3.1 that its correlations with those components follow by construction and omits it from the correlation matrix. That is a transparent definition, not a disguised prediction. The calibration step in Section 2.2 tunes the prompt against about ten hand-labeled songs and checks alignment on five held-out songs: the human labels are an external input, and the full-corpus scores are not fitted to the Section 3 outcomes. The Spotify comparison is an external benchmark against a separate binary flag. There are no self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The unverified model alias and tiny calibration/holdout samples are validity and reproducibility concerns—they do not make any equation or claimed result equivalent to its inputs by construction. Therefore no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or formal entities. Its unstated premises are measurement-oriented: that LLM scores correspond to the intended constructs, that the lyrics corpus is faithful, and that the chosen thematic dimensions and analysis thresholds are reasonable. The free parameters are the analytical thresholds and era split that shape the headline empirical claims.

free parameters (2)
  • Two-era split year = 2013
    Used for the 2002-2012 versus 2013-2025 comparison in Section 3.3; chosen near the corpus temporal midpoint rather than prespecified.
  • Explicit score threshold = explicit_norm > 0
    Binary grouping for the Spotify flag comparison in Section 3.4; any non-zero model score counts as explicit, which directly shapes the 'three of four go unflagged' finding.
assumptions (4)
  • domain assumption LLM scores approximate human thematic judgments after prompt calibration
    The entire empirical section treats gpt-5.1 scores as measurements; validation rests on about ten calibration songs and five held-out songs without reported agreement statistics in Section 2.2.
  • domain assumption Lyrics retrieved from Genius and corrected manually are faithful to the recorded songs
    Step 1 relies on Genius as the primary source, with regular-expression cleaning and manual cross-checking against alternative lyric sites, as described in Section 2.1.
  • ad hoc to paper The nine dimensions and 0-4 ordinal scale are a valid operationalization of thematic content
    The dimensions are defined in the scoring prompt and are not independently validated against external instruments; they are tailored to this study and the calibration cycle in Section 2.2.
  • domain assumption Yearly means with linear trends describe corpus-level change
    Trends are fit to yearly means rather than individual songs, and the paper itself notes that artist composition changes across years; artist-demeaned regressions are used for only two of the trends in Section 3.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards an LLM-based method for quantifying the sexual content in song lyrics." pith.science (2026). https://pith.science/paper/23Z4ZZ75

@misc{pith2026260808885,
  author       = {Pith},
  title        = {Pith review of: Towards an LLM-based method for quantifying the sexual content in song lyrics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23Z4ZZ75}},
  note         = {Machine review of arXiv:2608.08885}
}
read the original abstract

Reggaeton is one of the most widely consumed music genres in the world, and its lyrics are commonly regarded as highly sexualized. This claim rests mostly on qualitative studies and on small-scale quantitative ones. This paper has two goals. First, we present a reproducible method that uses a large language model to quantify thematic content in song lyrics along several independent dimensions. The method is not restricted to sexual content. Second, we apply it to a corpus of 1,259 songs by 12 reggaeton artists released between 2002 and 2025. The analysis covers four topics: a dataset characterization, a per-artist comparison, an analysis of how the dimensions change over time, and a comparison between our sexual-explicitness score and Spotify's own explicit flag. We release the data collection code, the scoring prompt, and the corpus, so that other researchers can replicate the approach or apply it to their own lyrics datasets.

Figures

Figures reproduced from arXiv: 2608.08885 by the authors.

Figure 1
Figure 1. Scoring pipeline for Step 2. Each song’s lyrics are passed individually to an LLM together [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Number of songs per year, colored by artist. Bars are stacked, so the height of each bar [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Distribution of all nine thematic dimensions and the composite sexual score, ordered by [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Pairwise correlations between all nine thematic dimensions. Rows and columns are ordered [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Distribution of the composite sexual score by artist, ranked by mean (labeled). The 12 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Mean score on seven thematic dimensions for four representative artists. Each vertex is [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Word cloud from the lyrics of Camilo (n=43 songs). Word size is proportional to raw [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Word cloud from the lyrics of Plan B (n=46 songs). Built in the same way as Figure 7. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Yearly mean (marker and line) and linear trend (dashed) for four thematic dimensions, [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Songs by agreement between Spotify’s explicit flag and the method’s sexual-explicitness score. Block area is proportional to the number of songs. A song counts as explicit for the method when its sexual-explicitness score is non-zero. We think this gap is the clearest…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

  1. [5]

    From joy to fear: A bench- mark of emotion estimation in pop song lyrics.arXiv preprint arXiv:2509.05617,

    Shay Dahary, Avi Edana, Alexander Apartsin, and Yehudit Aperstein. From joy to fear: A bench- mark of emotion estimation in pop song lyrics.arXiv preprint arXiv:2509.05617,

    arXiv

  2. [6]

    DSVS at MiSonGyny 2025: Multiple instance learning for misogyny speech detection in song lyrics

    Sergio Damián-Sandoval and David Vázquez-Santana. DSVS at MiSonGyny 2025: Multiple instance learning for misogyny speech detection in song lyrics. InProceedings of the Iberian Languages Evaluation Forum (IberLEF 2025), CEUR Workshop Proceedings, Vol. 4098,

    work page 2025

  3. [1996]

    María José Hellín García

    doi: 10.1037/0022-3514.70.3.491. María José Hellín García. It all comes down to sex: Metaphorical animalisation in reggaeton discourse. In Eliecer Crespo-Fernández, editor,Discourse Studies in Public Communication, vol- ume 92 ofDiscourse Approaches to Politics, Society and Culture, pages 152–176. John Benjamins, Amsterdam,

    doi:10.1037/0022-3514.70.3.491

  4. [2006]

    Language mod- els for longitudinal analysis of abusive content in Billboard Music Charts.arXiv preprint arXiv:2510.06266,

    15 Rohitash Chandra, Yathin Suresh, Divyansh Raj Sinha, and Sanchit Jindal. Language mod- els for longitudinal analysis of abusive content in Billboard Music Charts.arXiv preprint arXiv:2510.06266,

    work page arXiv

  5. [2007]

    Priscilla Carballo Villagra

    doi: 10.1300/J086v18n04_02. Priscilla Carballo Villagra. Música y violencia simbólica.Revista de la Facultad de Trabajo Social, 22(22):28–43,

    work page doi:10.1300/j086v18n04_02

  6. [2017]

    Garrido-Merchán

    Dolores Zamacola Sánchez de Lamadrid and Eduardo C. Garrido-Merchán. Fine-tuning large lan- guage models for automatic detection of sexually explicit content in Spanish-Language song lyrics. arXiv preprint arXiv:2602.05485,

    work page arXiv

  7. [2018]

    Brook Bretthauer, Toni Schindler Zimmerman, and James H

    doi: 10.5354/0719-529X.2018.51797. Brook Bretthauer, Toni Schindler Zimmerman, and James H. Banning. A feminist analysis of popular music: Power over, objectification of, and violence against women.Journal of Feminist Family Therapy, 18(4):29–51,

    work page arXiv 2018

  8. [2020]

    doi: 10.14198/fem.2020.35.11. OpenAI. GPT-5.1 Instant and GPT-5.1 Thinking System Card Addendum.https://openai.com/ index/gpt-5-system-card-addendum-gpt-5-1/, November

    work page doi:10.14198/fem.2020.35.11 2020

Show all 13 references
  1. [2021]

    Marie-France Merlyn

    doi: 10.1075/dapsac.92.07gar. Marie-France Merlyn. Dime lo que escuchas y te diré quién eres: Representaciones de la mujer en las 100 canciones de reggaetón más populares en 2018.Feminismo/s, (35):291–320,

    2018 doi

  2. [2022]

    Luisa Fernanda Espinal Ramírez, Silvia Díaz-Fernández, and Johnny Orejuela

    doi: 10.2478/genst-2023-0005. Luisa Fernanda Espinal Ramírez, Silvia Díaz-Fernández, and Johnny Orejuela. Tendencias inves- tigativas sobre el reggaetón y horizontes de conocimiento futuro de un fenómeno sociocultural en expansión.Cuadernos de Música Iberoamericana, 37:415–444,

    2023 doi

  3. [2024]

    Ángel Torres-Toukoumidis, Camila O

    doi: 10.46377/dilemas.v11i3.4167. Ángel Torres-Toukoumidis, Camila O. Cando Quezada, José Pontón, and Isidro Marín-Gutiérrez. Computational analysis of Latin music songs through tokenization: Case of female artists and reggaeton. InCommunication and Applied Technologies: Proce...

    doi

  4. [2025]

    Karina Arévalo, Emilia Chellew, Isabel Figueroa-Cofré, Adonai Arancibia-Villablanca, and Simone Schmied

    doi: 10.7764/res.2025.56.5. Karina Arévalo, Emilia Chellew, Isabel Figueroa-Cofré, Adonai Arancibia-Villablanca, and Simone Schmied. Ni pobre diabla, ni candy: Violencia de género del reggaetón.Revista de Sociología, 33 (1):7–23,

    2025 doi

  5. [2026]

    González, and Thamar Solorio

    16 Yigeng Zhang, Mahsa Shafaei, Fabio A. González, and Thamar Solorio. Positive and risky message assessment for music products. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 1...

    2024

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.