REVIEW 2 cited by
Fill in the BLANC: Human-free quality estimation of document summaries
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present BLANC, a new approach to the automatic estimation of document summary quality. Our goal is to measure the functional performance of a summary with an objective, reproducible, and fully automated method. Our approach achieves this by measuring the performance boost gained by a pre-trained language model with access to a document summary while carrying out its language understanding task on the document's text. We present evidence that BLANC scores have as good correlation with human evaluations as do the ROUGE family of summary quality measurements. And unlike ROUGE, the BLANC method does not require human-written reference summaries, allowing for fully human-free summary quality estimation.
Forward citations
Cited by 2 Pith papers
-
NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysis
A new large corpus of Indian court cases and a legal LLaMA model report very high judgment-prediction accuracy, but the evaluation leaks the outcome from the input text.
-
Survey on Abstractive Text Summarization: Dataset, Models, and Metrics
A survey of abstractive text summarization models, datasets, and metrics, accompanied by a small experimental run of public transformer checkpoints on short, long, and multi-document samples.
Discussion (0). Continue with ORCID to comment.