REVIEW 19 cited by
Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) augmented with external data have demonstrated remarkable capabilities in completing real-world tasks. Techniques for integrating external data into LLMs, such as Retrieval-Augmented Generation (RAG) and fine-tuning, are gaining increasing attention and widespread application. Nonetheless, the effective deployment of data-augmented LLMs across various specialized fields presents substantial challenges. These challenges encompass a wide range of issues, from retrieving relevant data and accurately interpreting user intent to fully harnessing the reasoning capabilities of LLMs for complex tasks. We believe that there is no one-size-fits-all solution for data-augmented LLM applications. In practice, underperformance often arises from a failure to correctly identify the core focus of a task or because the task inherently requires a blend of multiple capabilities that must be disentangled for better resolution. In this survey, we propose a RAG task categorization method, classifying user queries into four levels based on the type of external data required and primary focus of the task: explicit fact queries, implicit fact queries, interpretable rationale queries, and hidden rationale queries. We define these levels of queries, provide relevant datasets, and summarize the key challenges and most effective techniques for addressing these challenges. Finally, we discuss three main forms of integrating external data into LLMs: context, small model, and fine-tuning, highlighting their respective strengths, limitations, and the types of problems they are suited to solve. This work aims to help readers thoroughly understand and decompose the data requirements and key bottlenecks in building LLM applications, offering solutions to the different challenges and serving as a guide to systematically developing such applications.
Forward citations
Cited by 19 Pith papers
-
Systematic Evaluation of Knowledge Graph Repair with Large Language Models
A systematic VIO-based framework generates SHACL-violating graph test cases and shows that LLM repair systems perform best with concise, violation-focused prompts.
-
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Dual global+latent states with hierarchical episodic merging enable reflexive, low-latency long-video agents that beat iterative reasoning baselines on accuracy and efficiency.
-
Open Data Synthesis For Deep Research
InfoSeek automatically synthesizes hierarchical constraint-satisfaction QA data from webpages; a 3B agent trained on it reaches 16.5% accuracy on BrowseComp-Plus, outperforming Qwen3-32B and Gemini 2.5 Flash.
-
PG-Agent: An Agent Powered by Page Graph
An MLLM GUI agent that stores past episodes as a page graph and retrieves action guidelines from it improves step success on three benchmarks.
-
MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning
The paper advertises a self-evolving tool-using agent that beats baselines, but the body contains a different math paper, leaving the performance claims unsupported.
-
Knowledge Conceptualization Impacts RAG Efficacy
An empirical study showing that both schema complexity and representation format affect how well GPT-4o generates SPARQL queries from competency questions, with mixed results across two knowledge graph families.
-
EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora
EraRAG uses hyperplane-based locality-sensitive hashing to build a hierarchical retrieval graph whose affected regions only are re-summarized when new documents arrive, cutting update cost by up to an order of magnitude.
-
Harnessing the Power of Reinforcement Learning for Language-Model-Based Information Retriever via Query-Document Co-Augmentation
A bidirectional RL framework jointly optimizes LLM query and document augmentation and improves retrieval accuracy on BEIR benchmarks beyond query-only rewriting.
-
DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent
A synthetic benchmark of 9,000 multi-hop web-research questions with difficulty tiers and teacher-generated search trajectories, plus an open-source RL training framework that reportedly lets 3B-parameter agents beat ...
-
Enterprise Large Language Model Evaluation Benchmark
A 14-task enterprise LLM benchmark built mostly from GPT-4o-generated labels and scored by GPT-4o-as-judge shows open-source models closing the reasoning gap, but the dataset is not public and the evaluation is partly...
-
FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation
FlexRAG is a modular, open-source RAG framework with text, multimodal, and web retrieval, plus evaluation tools and efficient memory-mapped indexing.
-
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
SCR uses retrieval-augmented generation to fetch refusal examples that block jailbreak attacks, but the reported advantages are partly overstated.
-
Large Language Models for Next-Generation Wireless Network Management: A Survey and Tutorial
A survey and tutorial that organizes LLM-enabled wireless network optimization into formulation, solution, and verification stages, with case studies drawn from the authors' own prior papers.
-
GPL-SLAM: A Laser SLAM Framework with Gaussian Process Based Extended Landmarks
A laser SLAM framework that models each object as a Gaussian-process contour, updated recursively and inferred jointly with the robot pose in a Bayesian framework.
-
UrbanMind: Towards Urban General Intelligence via Tool-Enhanced Retrieval-Augmented Generation and Multilevel Optimization
The paper introduces UrbanMind, a tool-enhanced RAG framework with a multilevel optimization formulation for continual adaptation in urban AI, but offers only qualitative prototype results.
-
BLOCKS: Blockchain-supported Cross-Silo Knowledge Sharing for Efficient LLM Services
BLOCKS combines a Cosmos-based blockchain, a reputation mechanism, and a priority cache to let LLMs retrieve prompts from untrusted knowledge silos.
-
Method-Based Reasoning for Large Language Models: Extraction, Reuse, and Continuous Improvement
An external store of problem-solution prompt pairs is retrieved and injected into LLM queries, and toy experiments claim this improves factual checking and generalizes across made-up software names.
-
Assessing RAG and HyDE on 1B vs. 4B-Parameter Gemma LLMs for Personal Assistants Integretion
For small Gemma models, RAG outperformed HyDE on latency and personal-data accuracy, while HyDE improved semantic relevance at the cost of speed and reliability.
-
Private GPTs for LLM-driven testing in software development and machine learning
Structured Gherkin prompts increased executability and pass rate of LLM-generated tests in two small benchmarks compared to plain natural-language prompts.
Discussion (0). Sign in to comment.