Annotators presented with LLM label suggestions adopt them heavily, shifting gold labels and inflating reported LLM F1 scores by up to .35, even when humans review each suggestion.
Helping at my church made me feel like I was making a difference even though the world was going crazy
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CY 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks
Annotators presented with LLM label suggestions adopt them heavily, shifting gold labels and inflating reported LLM F1 scores by up to .35, even when humans review each suggestion.