A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:WAY6RWYNrecord.jsonopen to challenge →
classification
cs.CL
keywords
accuracymethodologysystemsdata-to-textevaluationgeneratedgoldstandard
read the original abstract
Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy of generated texts, which is intended to serve as a gold-standard for accuracy evaluations of data-to-text systems. We use our methodology to evaluate the accuracy of computer generated basketball summaries. We then show how our gold standard evaluation can be used to validate automated metrics
This paper has not been read by Pith yet.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.