Pith. sign in

REVIEW 4 cited by

Context Embeddings for Efficient Answer Generation in RAG

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.09252 v3 pith:TW5N6RG2 submitted 2024-07-12 cs.CL cs.IR

David Rau , Shuai Wang , Hervé Déjean , Stéphane Clinchant This is my paper · ORCID
classification cs.CLcs.IR
keywords timecontextallowsanswercompressiondecodinggenerationmethod
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Retrieval-Augmented Generation (RAG) allows overcoming the limited knowledge of LLMs by extending the input with external information. As a consequence, the contextual inputs to the model become much longer which slows down decoding time directly translating to the time a user has to wait for an answer. We address this challenge by presenting COCOM, an effective context compression method, reducing long contexts to only a handful of Context Embeddings speeding up the generation time by a large margin. Our method allows for different compression rates trading off decoding time for answer quality. Compared to earlier methods, COCOM allows for handling multiple contexts more effectively, significantly reducing decoding time for long inputs. Our method demonstrates a speed-up of up to 5.69 $\times$ while achieving higher performance compared to existing efficient context compression methods.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agents

    cs.CR 2025-10 reject novelty 6.0 of 10

    The paper claims prompt compression is a new attack surface, but the abstract's COMA attack never appears in the body and the body's SoftCom requires white-box access.

  2. MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft Prompts

    cs.AI 2025-10 conditional novelty 6.0 of 10

    MHA-RAG encodes retrieved exemplars into order-invariant soft prompts via multi-head attention, claiming ~20-point effective-accuracy gains over RAG at ~10x lower inference FLOPs.

  3. Reranking with Compressed Document Representation

    cs.IR 2025-05 conditional novelty 6.0 of 10

    A reranker trained on 8-token PISCO document embeddings plus a short query achieves near-identical nDCG@10 to full-text rerankers on BeIR and TREC-DL while running up to 16x faster.

  4. Thermodynamic Prediction Enabled by Automatic Dataset Building and Machine Learning

    cond-mat.mtrl-sci 2025-07 conditional novelty 5.0 of 10

    An LLM pipeline extracts thermodynamic data from literature and the resulting dataset trains a CatBoost model that predicts mineral formation enthalpy with a mean absolute error of 46.5 kJ/mol.

Pith tools