REVIEW 4 cited by
AmbigQA: Answering Ambiguous Open-domain Questions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Ambiguity is inherent to open-domain question answering; especially when exploring new topics, it can be difficult to ask questions that have a single, unambiguous answer. In this paper, we introduce AmbigQA, a new open-domain question answering task which involves finding every plausible answer, and then rewriting the question for each one to resolve the ambiguity. To study this task, we construct AmbigNQ, a dataset covering 14,042 questions from NQ-open, an existing open-domain QA benchmark. We find that over half of the questions in NQ-open are ambiguous, with diverse sources of ambiguity such as event and entity references. We also present strong baseline models for AmbigQA which we show benefit from weakly supervised learning that incorporates NQ-open, strongly suggesting our new task and data will support significant future research effort. Our data and baselines are available at https://nlp.cs.washington.edu/ambigqa.
Forward citations
Cited by 4 Pith papers
-
Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning
RRPO formulates document reranking as a sequential MDP and optimizes a pointwise reranker with PPO using LLM generation rewards and a reference-anchored deterministic baseline.
-
Which LLMs Get the Joke? Probing Non-STEM Reasoning Abilities with HumorBench
HumorBench scores LLM explanations of cartoon jokes against expert-written objective elements and finds reasoning skills transfer from STEM benchmarks, while extra thinking tokens help only some models.
-
PRGB Benchmark: A Robust Placeholder-Assisted Algorithm for Benchmarking Retrieval-Augmented Generation
PRGB introduces a placeholder-based, fine-grained RAG benchmark that evaluates LLMs on filtering, combination, and multi-hop reasoning, with English and Chinese datasets.
-
Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information
Per the abstract, large reasoning models systematically fail to ask for missing information on under-specified math problems, a skill standard benchmarks never test.
Discussion (0). Sign in to comment.