Pith. sign in

REVIEW 5 cited by

LettuceDetect: A Hallucination Detection Framework for RAG Applications

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.17125 v1 pith:MRZWJWMM submitted 2025-02-24 cs.CL cs.AI

Ádám Kovács , Gábor Recski This is my paper · ORCID
classification cs.CLcs.AI
keywords detectionencoder-basedlettucedetectmodelsapplicationscontextframeworkhallucination
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Retrieval Augmented Generation (RAG) systems remain vulnerable to hallucinated answers despite incorporating external knowledge sources. We present LettuceDetect a framework that addresses two critical limitations in existing hallucination detection methods: (1) the context window constraints of traditional encoder-based methods, and (2) the computational inefficiency of LLM based approaches. Building on ModernBERT's extended context capabilities (up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach outperforms all previous encoder-based models and most prompt-based models, while being approximately 30 times smaller than the best models. LettuceDetect is a token-classification model that processes context-question-answer triples, allowing for the identification of unsupported claims at the token level. Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for example-level detection, which is a 14.8% improvement over Luna, the previous state-of-the-art encoder-based architecture. Additionally, the system can process 30 to 60 examples per second on a single GPU, making it more practical for real-world RAG applications.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems

    cs.AI 2026-07 conditional novelty 6.0 of 10

    SIRIN unifies probing, uncertainty, and judge-style hallucination detectors with query answerability under one pluggable toolkit, evaluation pipeline, and interactive span-highlighting UI.

  2. Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP)

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Perturbing retrieved context and scoring likelihood drops plus JSD yields a training-free span-level RAG hallucination detector competitive with chunk-level NLI and stronger than perplexity.

  3. Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG

    cs.CL 2026-03 conditional novelty 6.0 of 10

    A claim-by-claim hierarchical verifier improves RAG hallucination detection over baselines, and a re-annotated benchmark finds 1.68x more hallucinated cases than the original labels.

  4. FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Fine-tuning small language models on synthetic financial errors yields high detection and editing scores, but the evaluation is limited to synthetic data from the same pipeline.

  5. CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection

    cs.CL 2025-06 conditional novelty 5.0 of 10

    CLATTER, a structured decompose-attribute-aggregate reasoning prompt, improves NLI-based hallucination detection for reasoning models by about 3.8 accuracy points on average, with inconsistent results across models an...

Pith tools