LLM-as-judge scoring of multi-round lateral thinking tasks can be fooled by answer leakage and question substitution, so response-based metrics may overstate reasoning ability.
Olson, Yoshua Bengio, and Karim Jerbi
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Evaluation Hallucination in Multi-Round Incomplete Information Lateral-Driven Reasoning Tasks
LLM-as-judge scoring of multi-round lateral thinking tasks can be fooled by answer leakage and question substitution, so response-based metrics may overstate reasoning ability.