REVIEW 8 cited by
QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Meetings are a key component of human collaboration. As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed. However, it is hard to create a single short summary that covers all the content of a long meeting involving multiple people and topics. In order to satisfy the needs of different types of users, we define a new query-based multi-domain meeting summarization task, where models have to select and summarize relevant spans of meetings in response to a query, and we introduce QMSum, a new benchmark for this task. QMSum consists of 1,808 query-summary pairs over 232 meetings in multiple domains. Besides, we investigate a locate-then-summarize method and evaluate a set of strong summarization baselines on the task. Experimental results and manual analysis reveal that QMSum presents significant challenges in long meeting summarization for future research. Dataset is available at \url{https://github.com/Yale-LILY/QMSum}.
Forward citations
Cited by 8 Pith papers
-
Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges
Per-bias selection of a cross-family LLM auditor lifts biased-judgment accuracy from 0.805/0.824 baselines to 0.884.
-
EvolKV: Evolutionary KV Cache Compression for LLM Inference
CMA-ES search over per-layer KV cache budgets beats uniform and pyramidal compression heuristics on LongBench, NIAH, RULER, and GSM8K, and edges past the full cache on one code dataset at 1.5% of the budget.
-
On Robustness and Reliability of Benchmark-Based Evaluation of LLMs
Paraphrasing benchmark questions keeps LLM rankings stable but reduces their accuracy, suggesting static benchmarks overestimate model robustness.
-
Teaching Language Models To Gather Information Proactively
Rewarding questions for eliciting genuinely new information trains a small model to outperform larger models at proactive clarification and downstream writing quality.
-
TalkLess: Blending Extractive and Abstractive Speech Summarization for Editing Speech to Preserve Content and Style
TalkLess blends extractive and abstractive speech summarization through LLM candidate generation and a weighted scoring function, then converts transcript edits to audio with VoiceCraft, evaluating favorably against a...
-
"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Models
LLMs ground answers in early context far more than later context, and chain-of-thought prompting or reasoning models reduce contextual grounding rather than improving it.
-
HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
HoPE replaces RoPE's sine/cosine rotations with hyperbolic functions plus an exponential damping term to enforce monotonic attention decay, but the claimed consistent superiority and the 'RoPE as special case' theorem...
-
Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey
A survey that classifies AI agent evaluation benchmarks along environment and capability axes, and proposes five traits that distinguish agents from chatbots.
Discussion (0). Sign in to comment.