ProcCtrlBench introduces an ontology of 11 defect types across 4 categories plus control preservation metrics to evaluate LLM coding agent trajectories on 200 cases from AndroidBench, TerminalBench, and SWE-bench-Verified.
Proceedings of the 2021 conference of the North American chapter of the Association for Computational Linguistics: human language technologies , pages=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Frontier AI benchmarks lose discriminatory power as models saturate easy items, concentrating valid signal in scarce expert-authored hard-tail items whose replacement cost rises convexly with capability.
citing papers explorer
-
ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents
ProcCtrlBench introduces an ontology of 11 defect types across 4 categories plus control preservation metrics to evaluate LLM coding agent trajectories on 200 cases from AndroidBench, TerminalBench, and SWE-bench-Verified.
-
The Benchmark Ceiling: Human Judgment, Evaluation Scarcity, and the Political Economy of AI Capability Measurement
Frontier AI benchmarks lose discriminatory power as models saturate easy items, concentrating valid signal in scarce expert-authored hard-tail items whose replacement cost rises convexly with capability.