MSRS: Evaluating Multi-Source Retrieval-Augmented Generation

Arman Cohan; Kejian Shi; Rohan Phanse; Wencai Zhang; Yijie Zhou; Yilun Zhao; Yixin Liu

REVIEW 3 major objections 5 minor 12 references

MSRS: Evaluating Multi-Source Retrieval-Augmented Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that realistic RAG needs benchmarks where answers span many documents, and its two new benchmarks show retrieval is the binding constraint while reasoning models are the current fix for synthesis.

desk verdict Useful benchmark contribution with a genuinely new construction pipeline, but the Story subset's query decontextualization may leak the target story via plot clues, so the retrieval-dependence claims are conditional on a leakage test. read the letter →

arxiv 2508.20867 v1 pith:LFQR77OO submitted 2025-08-28 cs.CL

Rohan Phanse , Yijie Zhou , Kejian Shi , Wencai Zhang , Yixin Liu , Yilun Zhao , Arman Cohan This is my paper

classification cs.CL

keywords multi-sourceretrieval-augmentedgenerationopen-domainmulti-documentsummarizationRAGbenchmarkconstructionquerydecontextualizationreasoningLLMsretrievalevaluationlong-formsynthesisoracle

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces two benchmarks, MSRS-Story and MSRS-Meet, built to test retrieval-augmented systems on the realistic task of pulling together information scattered across many documents and writing a long-form answer. The central finding is that in this setting generation quality tracks retrieval quality closely, and no tested retriever solves the problem: even the best retriever misses a large share of relevant documents, and the harder meeting-summarization benchmark defeats most dense retrievers outright. Even when the correct documents are handed to the generator, standard LLMs produce incomplete summaries, while reasoning-oriented models make a large, consistent improvement. If the benchmarks are valid, they give the field a way to separate retrieval failures from synthesis failures in multi-source RAG, and they point at reasoning models as the current best answer to the synthesis half.

What carries the argument

The construction framework that turns long-context single-document summarization datasets into open-domain multi-source RAG benchmarks. For MSRS-Story, chapters of fiction become corpus documents and queries are decontextualized with an LLM so they are self-contained but do not reveal the answer. For MSRS-Meet, similar queries from meeting transcripts are clustered and merged with their summaries into single query-summary pairs spanning multiple meetings. Each example keeps gold documents and gold summaries, so retrieval quality (NDCG, MAP, recall) and generation quality (ROUGE, BERTScore, G-EVAL) can be measured independently, and an oracle setting isolates the synthesis step from retrieval

What would settle it

Run retrieval and oracle generation on MSRS-Story with all story-identifying details (proper nouns, author, title references) stripped from the decontextualized queries. If a strong dense retriever's NDCG holds up, the queries encode genuine multi-document content needs; if it collapses toward BM25's level, the benchmarks measure story-identity lookup rather than multi-source retrieval. Separately, in the oracle setting, prompt a standard LLM to reason step by step before summarizing; if its G-EVAL matches the reasoning models, the reported reasoning advantage is a prompting effect rather than

Watch

Extended reading notes

Core claim

The paper claims that open-domain multi-document summarization, evaluated through controlled RAG benchmarks with gold relevance labels and gold summaries, reveals two separate bottlenecks. Retrieval is the first bottleneck: effectiveness varies sharply across tasks — on the meeting benchmark a sparse retriever rivals or beats most dense retrievers, and even the strongest systems top out well short of perfect retrieval. Synthesis is the second bottleneck: with perfect (oracle) retrieval, standard LLMs still omit major and minor details, and reasoning models beat them by up to roughly 11 G-EVAL points by writing more abstractive, complete answers. The claim is that these two bottlenecks are di

Load-bearing premise

The rewritten and merged queries preserve the original information needs without revealing the answer or making retrieval artificially easy; this premise is only spot-checked by human annotation on 160 of the 1,071 examples.

Editorial extensions

If this is right

Retrieval research for RAG should be benchmarked on multi-source tasks: single-source and factoid settings understate the difficulty, since the best retriever on MSRS-Meet reaches only about 33 NDCG.
Pipeline improvements that boost retrieval (reranking, stronger dense encoders) translate directly into better final summaries, making retrieval quality a reliable lever on generation quality.
Reasoning-oriented generators are a concrete fix for the synthesis step, gaining 7-11 G-EVAL points over the best standard LLMs when given the oracle documents.
Long-context models that skip retrieval still underperform retrieval-equipped pipelines even at 1M-token context windows, so retrieval remains necessary rather than optional.
The benchmarks enable component-level diagnosis: a RAG system that fails on MSRS can be classified as retrieval-limited or synthesis-limited, guiding where to invest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The framework suggests a general recipe: any long-context QA or summarization dataset with queries tied to identifiable subdocuments can be converted into a multi-source RAG benchmark, which could cheaply extend coverage to legal, medical, or scientific literature with gold multi-document labels.
The decontextualization results imply that query rewriting may be as important to real-world RAG performance as retriever choice; the paper's own Table 11 shows a 28-point NDCG swing on one retriever, but this is not foregrounded in the abstract.
The reasoning-model advantage in the oracle setting raises a testable hypothesis: the gain comes from reasoning tokens being spent on coverage of all required sub-topics, and a standard LLM given an explicit 'cover every chapter or meeting theme' instruction may narrow the gap.
Since reasoning models score higher on G-EVAL while scoring lower on ROUGE-2 and BERTScore, the benchmark's headline conclusions depend on the choice of evaluation metric; a reference-free factuality or coverage metric would be a useful stress test of the ranking.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a good benchmark paper, and the construction pipeline is the real contribution. I'd send it to review. But the MSRS-Story queries—decontextualized by appending a plot summary—may leak which story the query refers to, which could inflate retrieval numbers and soften the headline claims. The paper should run a leakage test.

The new thing here is not the task (open-domain MDS is established) but how the datasets are built: rewriting original SQuALITY queries instead of synthesizing pseudo-queries, and clustering and merging QMSum query-summary pairs into multi-meeting examples. That directly addresses a known weakness in prior benchmarks. The experimental sweep is wide—many retrievers, many generators, reranking, oracle, no-docs, long-context—and the human validation on summary quality is done carefully, though on a small sample.

The soft spot is the one the stress-test flags. Appendix F shows the rewritten query for 'describe the setting' becomes 'In a story where two men navigate a tense and dangerous situation involving a power source that could revive a struggling planet, describe the setting.' That plot clue may uniquely identify the story in a 127-story corpus. Table 11 shows decontextualization lifts gemini-embedding NDCG from 43.86 to 72.33. Some of that gain is expected—the original SQuALITY queries are absurdly underspecified ('What is the plot?')—but the paper doesn't demonstrate that the gain comes from deictic resolution rather than story identification. The human validation (160 of 1,071 examples) checks summary attributes, not whether the query leaks the source story. This is an empirical question with a straightforward test: mask the decontextualized clue and see if retrieval degrades, or ask annotators whether the rewritten query reveals the story. Without that, the Story subset's retrieval difficulty and the retrieval-to-generation dependence are harder to interpret.

The reasoning-model claim is the other soft spot. The gains (up to +11 G-EVAL) come from a single-run GPT-4o judge with no variance reported, and ROUGE-2 and BERTScore move in the opposite direction. The paper explains the divergence as more abstractive output, which is plausible, but 'significantly outperform' is resting on G-EVAL alone. That needs either multiple judge models, a human evaluation, or at least a statement about variance.

Nothing here is fatal. The Meet subset is less affected by the leakage concern and appears genuinely hard. The benchmarks will likely be used. For peer review: yes, send it out. The leakage question is exactly what a good reviewer should push on, and the paper is strong enough to survive an honest request for that evidence.

Who benefits: people building or evaluating multi-source RAG and deep-research-style summarization systems. The paper is written straight, no over-claiming except the 'significantly' on reasoning models, and the limitations section is honest.

Referee Report

3 major / 5 minor

Summary. The paper introduces MSRS-Story and MSRS-Meet, two benchmarks for multi-source retrieval-augmented generation, built by decontextualizing SQuALITY queries and by clustering/merging QMSum query-summary pairs. The authors evaluate a wide range of sparse and dense retrievers, with and without reranking, and a wide range of standard and reasoning LLMs as generators. Their central findings are: (1) retrieval effectiveness varies greatly across the two tasks; (2) generation quality is strongly dependent on retrieval quality; (3) multi-source synthesis remains difficult even with oracle retrieval; and (4) reasoning models significantly outperform standard LLMs in the oracle synthesis setting. The paper includes additional analyses of data contamination, multi-document necessity, long-context models, and oracle error types.

Significance. If the benchmark construction is valid, MSRS is a useful contribution: it targets a realistic and under-served setting (open-domain multi-document synthesis), releases code and data, and provides a systematic decomposition of retrieval and generation failures. The inclusion of long-context and no-retrieval control conditions is a strength, as is the attempt to validate the data with human annotation. However, the central empirical claims rest on two load-bearing assumptions that are not sufficiently verified: that the decontextualized MSRS-Story queries are genuine open-domain information needs rather than story-identification clues, and that single-run G-EVAL scores can support the claim that reasoning models significantly outperform standard LLMs. If the leakage concern is confirmed, the retrieval-dependence and task-variation findings are substantially weakened; if the G-EVAL concern is not addressed, the headline reasoning-model claim is not supported by the evidence as presented.

major comments (3)

[§3.2 / App. B.3 / Table 11 / App. F] The decontextualized queries in MSRS-Story may leak the target story through the appended 'vague story description.' The Appendix F example, 'In a story where two men navigate a tense and dangerous situation involving a power source that could revive a struggling planet, describe the setting,' is a distinctive plot clue that can identify one of only 127 stories. Table 11 shows that decontextualization raises gemini-embedding NDCG from 43.86 to 72.33 and R@8 from 34.91 to 57.46; BM25 NDCG rises from 24.49 to 28.75. The human validation in Table 3 (160/1,071 samples) rates summary attributes such as coherence and coverage, not whether queries leak the story identity or artificially simplify retrieval. Because the paper's central claims about retrieval effectiveness and task variation depend on MSRS-Story measuring genuine multi-source retrieval, this needs a leakage test: e.g., use the rew
[§5.4 / Table 8] The claim that reasoning models 'significantly outperform' standard LLMs is based on single-run G-EVAL scores without variance, confidence intervals, or significance tests. The G-EVAL gains conflict with lexical metrics: for GPT-5 vs GPT-4o in the oracle setting, MSRS-Story G-EVAL is 81.07 vs 68.22 (+12.85) but ROUGE-2 falls from 10.70 to 8.71 and BERTScore from 85.57 to 84.35; MSRS-Meet shows G-EVAL +7.11 but ROUGE-2 −4.55 and BERTScore −2.44. The paper explains this as more abstractive summaries, but no human preference data or abstractiveness-controlled analysis is provided. The oracle error analysis in Appendix B.4 covers only DeepSeek-V3 and GPT-4o, not reasoning models. Since this is a headline contribution, please report multiple runs or judge resampling with standard errors, pairwise significance tests, and/or human evaluation comparing reasoning and non-reasoning outputs.
[§3.3 / Table 3 / Appendix A] The MSRS-Meet construction assumes that LLM-merged query-summary pairs preserve the original information needs and that the merged summary is a valid gold target. The clustering parameters (similarity threshold θ, cluster size bounds 2–6) are free choices, and the merged summary is itself generated by an LLM from existing QMSum pairs. Human validation covers only 80 of 436 Meet examples and rates attributes like coherence and aggregation, not whether the merged query captures the same information need as the original queries or whether the merged gold summary is supported by the source meetings. The Appendix A example shows similar queries about remote-control functions merged into a query about 'function and object,' potentially shifting the coverage of the gold summary. This matters for every generation score on MSRS-Meet and for the task-variation comparison. Please report inter-annot

minor comments (5)

[Limitations] The Limitations section contains a typo ('argentic' should be 'agentic') and asserts that 'the nature does not undermine the value of our work' without argument. Please revise to acknowledge the specific risks raised by automated query construction.
[App. B.2] In the long-context evaluation, MSRS-Meet documents are truncated to the first 4,800 tokens each. Truncation may omit relevant later content, and this should be acknowledged in the main text as a caveat on the long-context comparison.
[§5.2 / App. C.6] The No Documents contamination setting for MSRS-Story gives the model the story title and author. Since frontier LLMs may have memorized these public-domain stories, the no-retrieval baseline may not be a clean test. Consider a condition without title/author.
[Tables 4–7] Column headers such as 'M EET' are typeset inconsistently; use the same formatting as 'Story'. Also, many comparisons are reported to two decimals with no indication of run-to-run variability.
[App. C.5] A G-EVAL consistency prompt is included but the main results only use the relevance rubric. Either report consistency scores or remove the unused prompt.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MSRS is an empirical benchmark-construction and evaluation paper; no central claim reduces to its inputs by construction.

full rationale

The paper's central claims are empirical observations from running retrieval and generation systems on a newly constructed benchmark, not derived quantities fitted from the same data. The MSRS-Story query rewriting step uses GPT-4o-mini with the full story to append a vague description, but this is a data-construction decision; the later retrieval and generation scores are measured outcomes, not definitions of the queries. Likewise, the decontextualization ablation (Appendix B.3) compares original SQuALITY queries to rewritten ones and reports retrieval differences; this is an empirical sanity check, not a fitted parameter renamed as a prediction. The use of GPT-4o as the G-EVAL judge while GPT-4o is also a generator is a shared-ecosystem evaluation-validity concern, but it is not circular: the judge score is not defined as, or derived from, the generator's identity, and the paper also reports ROUGE and BERTScore. The self-citations (e.g., Giorgi et al. 2023, which shares an author) are used only to describe limitations of prior open-domain MDS benchmarks and do not carry the weight of the paper's conclusions. No uniqueness theorem, ansatz smuggled by citation, or renaming of a known result is present. Therefore the paper is self-contained against external benchmarks for its main empirical claims, and the appropriate circularity score is 0.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central claims rest on assumptions about the quality and representativeness of the source datasets, the faithfulness of LLM-based rewriting and merging, and the validity of the evaluation metrics. Several hand-chosen hyperparameters (cluster threshold, cluster size bounds, K, truncation) shape the benchmarks and results.

free parameters (6)

QMSum query-cluster similarity threshold theta = not reported
Controls which QMSum queries are merged into one MSRS-Meet example (Section 3.3, Appendix A); a higher theta produces smaller clusters and changes the multi-source requirement.
QMSum cluster size bounds = min=2, max=6
Hand-selected constraints on merged clusters; affects granularity and number of examples (Appendix A).
Retrieval depth K = K=8 (Story), K=3 (Meet)
Set to average oracle document count (Appendix D.1); determines the operating point and hence reported retrieval and generation scores.
Dense retriever chunk size and aggregation = 1000-token chunks, size-weighted averaging
Applied to all dense retrievers (Appendix D.1); engineering choice that affects embedding quality and retrieval results.
Generation truncation limits = 4096 tokens for Llama 2; 32k for MSRS-Meet; 4800 tokens per document for long-context test
Truncation choices in Appendices D.2 and B.2 that can artificially lower or equalize generation scores.
Sampling hyperparameters = temperature=0.7, top-p=0.9, max output 600 tokens
Standard generation settings (Appendix D.2); needed to reproduce runs, and can influence output diversity and scores.

assumptions (7)

domain assumption SQuALITY and QMSum gold summaries are accurate, complete, and correctly aligned with their queries
Source datasets are trusted as ground truth; if these summaries are noisy, the benchmark's gold outputs are noisy (Section 3.1).
ad hoc to paper SQuALITY questions, after decontextualization, still have the same information need and the original gold summaries remain valid references
GPT-4o-mini rewrites queries using the full story (Section 3.2, Appendix C.2); this could change the question or leak the story identity, and human validation covers only 160 examples (Table 3).
ad hoc to paper QMSum clusters and LLM-merged summaries preserve the original content and produce coherent multi-meeting query-summary pairs
Merging by similarity threshold and LLM (Section 3.3, Appendix A) may introduce hallucinations or drop information; no large-scale verification is reported.
domain assumption Chapter segmentation using natural web delimiters yields coherent documents
Used to create MSRS-Story documents (Section 3.2); malformed splits could break retrieval or synthesis.
domain assumption Gold oracle document sets are complete and sufficient for the rewritten queries
Oracle setting feeds all source gold documents to generators (Section 5.3); if relevant info is missing, oracle generation scores underestimate model capability.
domain assumption G-EVAL with GPT-4o as judge and the relevance rubric measures summary quality
All headline generation comparisons use this metric (Appendix D.3.2); it may favor certain styles and lacks large-scale human correlation here.
domain assumption Standard IR metrics with gold documents provide correct relevance labels
Retrieval evaluation assumes gold docs are the only relevant docs and that relevance is binary (Appendix D.3.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of MSRS: Evaluating Multi-Source Retrieval-Augmented Generation." pith.science (2026). https://pith.science/paper/LFQR77OO

@misc{pith2026250820867,
  author       = {Pith},
  title        = {Pith review of: MSRS: Evaluating Multi-Source Retrieval-Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LFQR77OO}},
  note         = {Machine review of arXiv:2508.20867}
}

read the original abstract

Retrieval-augmented systems are typically evaluated in settings where information required to answer the query can be found within a single source or the answer is short-form or factoid-based. However, many real-world applications demand the ability to integrate and summarize information scattered across multiple sources, where no single source is sufficient to respond to the user's question. In such settings, the retrieval component of a RAG pipeline must recognize a variety of relevance signals, and the generation component must connect and synthesize information across multiple sources. We present a scalable framework for constructing evaluation benchmarks that challenge RAG systems to integrate information across distinct sources and generate long-form responses. Using our framework, we build two new benchmarks on Multi-Source Retrieval and Synthesis: MSRS-Story and MSRS-Meet, representing narrative synthesis and summarization tasks, respectively, that require retrieval from large collections. Our extensive experiments with various RAG pipelines -- including sparse and dense retrievers combined with frontier LLMs -- reveal that generation quality is highly dependent on retrieval effectiveness, which varies greatly by task. While multi-source synthesis proves challenging even in an oracle retrieval setting, we find that reasoning models significantly outperform standard LLMs at this distinct step.

Figures

Figures reproduced from arXiv: 2508.20867 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

[1]

Read the query carefully to understand the information needed

work page
[2]

Read through the documents to assess their relevance to the query

work page
[3]

Read the summary and proceed to evaluate the entire example on the 6 criteria above, assigning a score from 1-5 for each criterion

work page
[4]

For each score, provide a brief justification noting key observations and examples from the summary and documents

work page
[5]

Ten years ago,

Add further comments if there is any. F Dataset Example We show one example of each subset of the MSRS dataset in Table 15 and Table 16. 25 Published as a conference paper at COLM 2025 Field Content Query In a story where two men navigate a tense and dangerous situation involving a power source that could revive a struggling planet, describe the setting o...

work page 2025
[7]

A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation

URL https://openreview.net/forum?id=SkeHuCVFDr. Yilong Zhao and Daifeng Li. A large language model-based framework for semi-structured tender document retrieval-augmented generation. ArXiv, abs/2410.09077, 2024. URL https://api.semanticscholar.org/CorpusID:273346484. Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan Le Bras, Taira Anderson, Jonatha...

work page Pith review arXiv 2024
[1999]

Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel R

URL https://trec.nist.gov/pubs/trec8/papers/overview_8.pdf. Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel R. Bowman. SQuALITY: Building a long-document summarization dataset the hard way. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1139–1156. Association for Computational Linguist...

arXiv 2022
[2020]

Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang

URL https://api.semanticscholar.org/CorpusID:218718706. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. ArXiv, abs/2308.03281, 2023. URL https://api.semanticscholar.org/CorpusID:260682258. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summ...

arXiv 2023

Show all 12 references

[2021]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al

URL https://api.semanticscholar.org/CorpusID:237532173. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. ArXiv, abs/2412.15115, 2024. URL https://api.semanticscholar.org/C...

2024 arXiv
[2023]

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E

URL https://api.semanticscholar.org/CorpusID:263672150. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Haotong Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. Proceedi...

2023 arXiv
[2025]

Huayang Li, Yixuan Su, Deng Cai, Yan Wang, and Lemao Liu

URL https://openreview.net/forum?id=lgsyLSsDRe. Huayang Li, Yixuan Su, Deng Cai, Yan Wang, and Lemao Liu. A survey on retrieval-augmented text generation. ArXiv, abs/2202.01110, 2022. URL https://api. semanticscholar.org/CorpusID:246472929. Wei Li, Xinyan Xiao, Jiachen Liu, Hu...

2022 arXiv
[5277]

URL https://aclanthology

Association for Computational Linguistics, July 2025. URL https://aclanthology. org/2025.acl-long.263/. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2...

2025 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Read the query carefully to understand the information needed

work page

[2] [2]

Read through the documents to assess their relevance to the query

work page

[3] [3]

Read the summary and proceed to evaluate the entire example on the 6 criteria above, assigning a score from 1-5 for each criterion

work page

[4] [4]

For each score, provide a brief justification noting key observations and examples from the summary and documents

work page

[5] [5]

Ten years ago,

Add further comments if there is any. F Dataset Example We show one example of each subset of the MSRS dataset in Table 15 and Table 16. 25 Published as a conference paper at COLM 2025 Field Content Query In a story where two men navigate a tense and dangerous situation involving a power source that could revive a struggling planet, describe the setting o...

work page 2025

[6] [7]

A Large Language Model-based Framework for Semi-Structured Tender Document Retrieval-Augmented Generation

URL https://openreview.net/forum?id=SkeHuCVFDr. Yilong Zhao and Daifeng Li. A large language model-based framework for semi-structured tender document retrieval-augmented generation. ArXiv, abs/2410.09077, 2024. URL https://api.semanticscholar.org/CorpusID:273346484. Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan Le Bras, Taira Anderson, Jonatha...

work page Pith review arXiv 2024

[7] [1999]

Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel R

URL https://trec.nist.gov/pubs/trec8/papers/overview_8.pdf. Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel R. Bowman. SQuALITY: Building a long-document summarization dataset the hard way. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1139–1156. Association for Computational Linguist...

arXiv 2022

[8] [2020]

Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang

URL https://api.semanticscholar.org/CorpusID:218718706. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. ArXiv, abs/2308.03281, 2023. URL https://api.semanticscholar.org/CorpusID:260682258. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summ...

arXiv 2023