REVIEW 5 cited by
FLUTE: Figurative Language Understanding through Textual Explanations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Figurative language understanding has been recently framed as a recognizing textual entailment (RTE) task (a.k.a. natural language inference, or NLI). However, similar to classical RTE/NLI datasets, the current benchmarks suffer from spurious correlations and annotation artifacts. To tackle this problem, work on NLI has built explanation-based datasets such as e-SNLI, allowing us to probe whether language models are right for the right reasons.Yet no such data exists for figurative language, making it harder to assess genuine understanding of such expressions. To address this issue, we release FLUTE, a dataset of 9,000 figurative NLI instances with explanations, spanning four categories: Sarcasm, Simile, Metaphor, and Idioms. We collect the data through a model-in-the-loop framework based on GPT-3, crowd workers, and expert annotators. We show how utilizing GPT-3 in conjunction with human annotators (novices and experts) can aid in scaling up the creation of datasets even for such complex linguistic phenomena as figurative language. The baseline performance of the T5 model fine-tuned on FLUTE shows that our dataset can bring us a step closer to developing models that understand figurative language through textual explanations.
Forward citations
Cited by 5 Pith papers
-
Mining Contextualized Visual Associations from Images for Creativity Understanding
A scalable pipeline generates 1.7 million increasingly abstract MSCOCO captions, and fine-tuning CLIP on them improves zero-shot retrieval in poetry and metaphor tasks.
-
Fann or Flop: A Multigenre, Multiera Benchmark for Arabic Poetry Understanding in LLMs
The new Fann or Flop benchmark measures LLM comprehension of Arabic poetry through expert-written verse explanations and shows current LLMs perform poorly on interpretive depth.
-
Class Distillation with Mahalanobis Contrast: An Efficient Training Paradigm for Pragmatic Language Understanding Tasks
A Mahalanobis-distance contrast loss plus a Beta-distribution decision rule lets small-parameter models match or beat larger LLMs on sexism, sarcasm, and metaphor detection with one epoch of training.
-
Rhetorical Text-to-Image Generation via Two-layer Diffusion Policy Optimization
Rhet2Pix combines staged LLM prompt decomposition with a discounted PPO fine-tuning scheme for Stable Diffusion, claiming strong rhetorical text-to-image generation, but the quantitative evidence is circular and undefined.
-
Was that Sarcasm?: A Literature Survey on Sarcasm Detection
A literature review that catalogs sarcasm detection datasets, word-embedding strategies, and neural models, but adds no new experimental results.
Discussion (0). Continue with ORCID to comment.