Pith Number

pith:INFD73OG

pith:2026:INFD73OGFCUGVWSPGCDJ6CWNGU

not attested not anchored not stored refs resolved

Deepchecks: Evaluating Retrieval-Augmented Generation (RAG)

Alex Zaikman, Assaf Gerner, Jonatan Liberman, Lior Rokach, Liron Hamra, Nadav Barak, Neal Harow, Netta Madvil, Noam Bresler, Philip Tannor, Rotem Brazilay, Shay Tsadok, Shir Chorev, Yaron Friedman

Deepchecks introduces a comprehensive framework for evaluating Retrieval-Augmented Generation systems through multi-faceted analysis, root cause identification, and production monitoring.

arxiv:2605.14488 v1 · 2026-05-14 · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{INFD73OGFCUGVWSPGCDJ6CWNGU}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Deepchecks' evaluation framework addresses RAG applications evaluation through a multi-faceted approach, root cause analysis and production monitoring. By ensuring alignment with application-specific requirements, Deepchecks framework provides a robust foundation for assessing reliability, relevance, and user satisfaction in RAG systems.

C2weakest assumption

That a multi-faceted approach with root cause analysis and production monitoring can effectively handle the stochastic nature of outputs and the interplay between retrieval and generation components to provide robust, aligned evaluations.

C3one line summary

Deepchecks is a new multi-faceted evaluation framework for RAG that incorporates root cause analysis and production monitoring to assess reliability, relevance, and user satisfaction.

References

20 extracted · 20 resolved · 1 Pith anchors

[1] Amazon Web Services: New RAG evaluation and llm-as-a-judge ca- pabilities in Amazon Bedrock.AWS Blog(2025), retrieved from https://aws.amazon.com/blogs/aws/new-rag-evaluation-and-llm-as-a- judge-capab 2025

[2] Arize AI: Llms as judges: A comprehensive survey on LLM-based evaluation methods.Arize AI Blog(2025), retrieved from https://arize.com/blog/llm- as-judge-survey-paper/ 2025

[3] arXiv preprint arXiv:2407.00072 (2024) 2024

[4] Proceedings of the 2015 Con- ference on Empirical Methods in Natural Language Processing pp 2015

[5] Con- fident AI Blog (2024), https://www.confident-ai.com/blog/why-llm-as-a- judge-is-the-best-llm-evaluation-method 2024

Receipt and verification

First computed	2026-05-17T23:39:06.463862Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

434a3fedc628a86ada4f30869f0acd353421537189cb38d920258ec3f97d4116

Aliases

arxiv: 2605.14488 · arxiv_version: 2605.14488v1 · doi: 10.48550/arxiv.2605.14488 · pith_short_12: INFD73OGFCUG · pith_short_16: INFD73OGFCUGVWSP · pith_short_8: INFD73OG

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/INFD73OGFCUGVWSPGCDJ6CWNGU \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 434a3fedc628a86ada4f30869f0acd353421537189cb38d920258ec3f97d4116

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "045acd3ced2cb19e760e1b2a377ae413ebbfd901e825b4b8f89d645bf5b13bf2",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-05-14T07:27:50Z",
    "title_canon_sha256": "b7d93b94fffaca55c4ac9b2ecc75b8588349264d59ee05655590375d86ffad7f"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.14488",
    "kind": "arxiv",
    "version": 1
  }
}