Pith Number

pith:KJ3GLN3I

pith:2023:KJ3GLN3IMWW72RZIYRM4HPVTLB

not attested not anchored not stored refs resolved

Ragas: Automated Evaluation of Retrieval Augmented Generation

Jithin James, Luis Espinosa-Anke, Shahul Es, Steven Schockaert

Ragas supplies reference-free metrics that score context relevance, faithfulness to retrieved passages, and answer relevance in RAG pipelines.

arxiv:2309.15217 v2 · 2023-09-26 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{KJ3GLN3IMWW72RZIYRM4HPVTLB}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We put forward a suite of metrics which can be used to evaluate these different dimensions without having to rely on ground truth human annotations.

C2weakest assumption

That LLM-based judgments of context relevance, faithfulness, and answer relevance will correlate sufficiently with human judgments across domains and models without additional calibration or validation data.

C3one line summary

Ragas supplies reference-free metrics for measuring context relevance, faithfulness to retrieved passages, and answer quality in RAG pipelines.

References

3 extracted · 3 resolved · 2 Pith anchors

[1] Sparks of Artificial General Intelligence: Early experiments with GPT-4 2022 · arXiv:2303.12712

[2] Halueval: A large-scale hallucination evaluation benchmark for large language models 2020

[3] SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models 2023 · arXiv:2303.08896

Formal links

3 machine-checked theorem links

Cited by

25 papers in Pith

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-Augmented Generation for Large Language Models: A Survey

A Survey on Retrieval-Augmented Text Generation for Large Language Models

RAG-Enabled Intent Reasoning for Application-Network Interaction

DOTRAG: Retrieval-Time Reasoning Along Paths

Receipt and verification

First computed	2026-05-17T23:38:46.488030Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

527665b76865adfd4728c459c3beb358694b06fe0e9c77004eb0ebc4a2cdddcc

Aliases

arxiv: 2309.15217 · arxiv_version: 2309.15217v2 · doi: 10.48550/arxiv.2309.15217 · pith_short_12: KJ3GLN3IMWW7 · pith_short_16: KJ3GLN3IMWW72RZI · pith_short_8: KJ3GLN3I

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/KJ3GLN3IMWW72RZIYRM4HPVTLB \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 527665b76865adfd4728c459c3beb358694b06fe0e9c77004eb0ebc4a2cdddcc

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "007146ebe8160da410be50d017592d232248f0d29c844a95b1aa4da35c6cb3ef",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2023-09-26T19:23:54Z",
    "title_canon_sha256": "91b21454bfac08ab0267ff0ab7c5697b9c78babb8efb3a88d2060faa54199c57"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2309.15217",
    "kind": "arxiv",
    "version": 2
  }
}