Pith. sign in

Paper Citation Record · LEDGER

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems

As of 22 August 2026, this Paper Citation Record lists 58 of 58 outbound references and 0 inbound Pith citation observations for arXiv:2606.23403.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2606.23403 v1

Coverage vector

measured 58 of 58 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-06-26T08:16:51.419186Z

measured 58 of 58 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-21T06:32:19.484+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

58 of 58 outbound references displayed

  • verified exact0
  • verified fuzzy0
  • unresolved57
  • parse uncertain0
  • malformed identifier0
  • metadata mismatch1

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation b9d634bb-1aa5-4246-b089-d41e889d9839 · outbound

This paper cites InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI), pages 8038–8047.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI), pages 8038–8047

Reference 1

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:0e5c078150c9ee607b5e566746b8d4f3086f24cec9f3b1fbe4be9eea713de9a6

Observation 9385460b-5003-428e-a545-8412e7ab3789 · outbound

This paper cites InIn- ternational Conference on Learning Representations.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems InIn- ternational Conference on Learning Representations

Reference 2

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:bc39ff8cd888a257f2e7d5c11fe4f15f31d29e838e6ee76014782c40dfa51507

Observation fe034b2e-076a-4f39-80f7-6982ff3fc2ce · outbound

This paper cites LLM Evaluators Recognize and Favor Their Own Generations.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems LLM Evaluators Recognize and Favor Their Own Generations

Reference 3

Resolution
metadata mismatch
local_arxiv, observed 2026-07-04T10:59:46.352976Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:ec078f535786d4009a52999bbb6cdbbc30cd10f75ece2577112ac2cfe650b762

Observation 2082e843-62ca-4684-9400-bb47d41f987e · outbound

This paper cites In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 314–324.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 314–324

Reference 4

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:dfa0b1d74ce151b12a7f51defe314b42c533c022ad722233eae49f442ce19c46

Observation d52e223a-0cb1-4576-a001-5fedc5bd7f90 · outbound

This paper cites not found.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems not found

Reference 5

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:e5f329e36bc2a41d0d0ccf9c41dd383ac415e936790ee2af44145844d5273303

Observation 5244f29b-229b-4513-ae43-3bc6ff95b978 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 6

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:6a605894bd09efc81d115ac1f5358f92aabf9b459ca7f28d5b26f03561eaa3d8

Observation 16bee449-ae78-40b0-b9c6-d332c5ab0223 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 7

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:e61b64a18079fc35cebf38e19ed266921895564e30a237ebb04210e32c830d94

Observation afda7c72-b074-479c-8c7a-05c5df708c88 · outbound

This paper cites PRIORITY ORDER:.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems PRIORITY ORDER:

Reference 8

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:e4f3c5f4efa8e8121c81ba7bcfe72a4da174ed6540351a3cfd8a5b740c8579e4

Observation 7c1d9e57-2039-4550-8310-ffc1bae4ceac · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 9

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:cd099fd86c74ed85e07aaa7349c6587d2ae18435b75d03f00f17ca2659ee9333

Observation 2d0fa759-220c-4de8-ae29-8d6348086d58 · outbound

This paper cites processing.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems processing

Reference 10

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:2454084463796937be49cf944ab8fb10e114db67784f2d7024a00b8830a88dc7

Observation acdfb9c5-657c-42a3-bdb4-9a6bd02725de · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 11

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:68087f41456607d4f4ac747758abe2d850e2dbb7a99ab069ba78c0f1fb1ef69d

Observation b9c59398-2666-4a3e-9e93-daecf62f81f2 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 12

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:406eaf57ecd8876d7e2045e44a02f945e739e7365959a2ce3343a953d9a557b0

Observation 8695bb48-c448-4057-a74a-b90895b3facf · outbound

This paper cites Document Retrieval.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Document Retrieval

Reference 13

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:93ff746b264830caac836f3030a00f94f1f808912a853143d52aa79725850d17

Observation 44e922b5-a7c7-4d71-b863-48a1084410be · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 14

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:e07b003a07d47b74f575e9e89524cb6b2b52ea9333020cc17600cc967613bd9d

Observation f5fc8336-46f6-49d0-9e24-5dc4c33582a1 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 15

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:15a067a89b1f7b7ca950f57be01ce9d0728ee65039752ba9dad05027b640d3c6

Observation 7cb07c13-af5d-4c4b-a489-2a12244b03ea · outbound

This paper cites If two groups of files implement different domain pipelines -- different domain vocabulary, inputs/outputs, or end- to-end flow -- they MUST be separate subsystems.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems If two groups of files implement different domain pipelines -- different domain vocabulary, inputs/outputs, or end- to-end flow -- they MUST be separate subsystems

Reference 16

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:d70291f07fb684a4babd3ed3957943634e625b17bcde134aff3d86c0ff2f2c8c

Observation 2cd7a54a-3130-4d1c-b125-00506edf373a · outbound

This paper cites Domain Logic.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Domain Logic

Reference 17

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:ed6b796ded89824238b3b5771983c7845694fcf8b30f0b533aeba5148947ec0c

Observation 1f345420-598f-4004-b90b-e477bc25b15b · outbound

This paper cites Return ONLY valid JSON matching the schema.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Return ONLY valid JSON matching the schema

Reference 18

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:789e49c81777fdce93a65c6e824d89d3b9ff4c5d66d4bd3013089d97f1c16b11

Observation 7a79fa0f-8f4b-4cfd-8d8c-97e397e20827 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 19

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:6fc345efaeb05a29b87f1db12b9e5cd1d7d605d6287d9da6cad05158e2cb5447

Observation 807d18ca-9631-486d-bb19-9c6fcc51a60c · outbound

This paper cites data_flow: subsystemA.output -> subsystemB.input.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems data_flow: subsystemA.output -> subsystemB.input

Reference 20

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:193ce510d3129b10195bb170a0e48b639a92eb34a5f81fac4d1c3b89adb4ba6b

Observation c3e6600b-3916-401b-b08f-bd7229fbf228 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 21

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:0c8cd63a857462b2dcd9730f486a185af9ca36055f5969f66cad1368d6921791

Observation 77c09481-a0d1-4ef2-9722-f26b61a26030 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 22

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:744556559dd7547275474295bb497834d59d939d5db4847f2cf71a965a037fa2

Observation 5f47d3c7-1e85-4b62-b9bf-0d3c67fef34f · outbound

This paper cites No subsystem may be isolated.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems No subsystem may be isolated

Reference 23

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:6406be2c11adf5edee41c6b00fed19fbd7be10164989dd7b6526dde3e5cdeffe

Observation b61ac0cd-ed5a-497f-85f6-7e8bebae0ef2 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 24

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:96ed98fdfd7977a9c9e849473b48fb158d50f5abc138af6d22bacdea0db756d2

Observation e30329ae-cacf-4582-a9ed-ad82cbceba3f · outbound

This paper cites Return ONLY valid JSON matching the schema.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Return ONLY valid JSON matching the schema

Reference 25

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:12015c70fb8fa9c1dc27bf222ad18c2a29ca5069ddb0ca29cd4b27870de2543e

Observation dd0eee61-2bad-4d2c-a0e8-865a4f56c273 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 26

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:5dbb97e2f0327ce9e988d1e5c9e8da7437ca455c8540d812226706c319744b83

Observation b763b471-171d-4914-8af5-d333d6f769a7 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 27

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:7f1ef18408b0b0552ed3037669ca7fe5a91134fb12b3672a55a1c7357fcbe70d

Observation 4cdd8c16-b7e2-4b1f-984c-7b4eea37028d · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 28

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:a769b12400a788c13e140578a7b2f8761bd0188a1d7032f51fcb91acb02961e6

Observation 6472e3f6-b88d-4831-8885-a870903891a7 · outbound

This paper cites ai_core.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems ai_core

Reference 29

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:56cbd4e9fead34274cb15de1c8b3ff7ae9e93378b1597da17a92176f05f1a151

Observation d11d616b-2a2c-4889-8e48-6c96e5b3aa01 · outbound

This paper cites Assume the synthesizer made mistakes.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Assume the synthesizer made mistakes

Reference 30

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:e799f8ea9af5258ff3bc37d16d0d96280612ed7ac5664b89c45842272a42d19a

Observation 00c3e734-0511-4f0c-83ac-39bfce00f771 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 31

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:022885031509887e81e1067b6f4c468e32cc41cce28220b0bb582d2e3d8c6c70

Observation a84e660d-0943-4e4f-9503-85d3a24a5251 · outbound

This paper cites prompt_chain.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems prompt_chain

Reference 32

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:c962664755a64bd25d47d6cc2640368ab4f0bda353b015735589fd965208a861

Observation 0a83fca3-0e48-4852-bf1e-5fff39f07afd · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 33

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:bdc4f3d77670145dc555d52d0ea939049103668f897bbb40784c3d20b9f4c762

Observation 652eb75d-37e4-4f85-9157-c65d96496261 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 34

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:212a944c6cda08d894f33ce1c2644744d20c24e96a4931e0b3e0426fe90658ab

Observation cc617c8b-cdb8-4667-b37f-b08ca2497a5e · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 35

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:7058fe8c0d121c26edd2fd75220159cec8edd031d29739a24a661a06e2801f70

Observation 47ef7195-539e-4717-b8a8-a48b7b4ad0b2 · outbound

This paper cites evidence_match.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems evidence_match

Reference 36

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:6fed8fb975bde042dc5ab9b3ca1c41040f4596bf8cf861932bb0e73b9e57d6b5

Observation ade26859-77f3-46e9-a01e-6d8147e510cf · outbound

This paper cites prompt_chain.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems prompt_chain

Reference 37

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:1922a3295d6f4c8c396ab85d8cdb3676c630e20ce8097e65c371acacb4954424

Observation 2e7287a5-22c8-41d3-86a6-8e6d930c85eb · outbound

This paper cites none" A component's pipelinePattern is INDEPENDENT of its patternStage -- e.g. a RAG retriever has pipelinePattern=.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems none" A component's pipelinePattern is INDEPENDENT of its patternStage -- e.g. a RAG retriever has pipelinePattern=

Reference 38

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:91d26561f165750631cdb4d73345a631142f756605fd69ca92c7916ba3a56b76

Observation a5ee1ed8-f3f8-4ab5-9f8c-56df4023070e · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 39

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:fced4fcae5d1a2dbd8e0767cab57cee82cdf0ac0311c772910cdbd8b5473638f

Observation 3253a48f-6c5b-4ed5-b218-0eb828497fcf · outbound

This paper cites non-empty retrieval result.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems non-empty retrieval result

Reference 40

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:4f527eb852dfea74a0ec7fd22c1a7aed995e6c384b09e70aceecd60df3c8495f

Observation a750804b-411d-462b-bd2c-eb44c578e8e3 · outbound

This paper cites compare to ground truth.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems compare to ground truth

Reference 41

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:9e5e4eccdf9c0af41933e3c0719e4a693cbdf055d94a75d0bd80d932f2d42b90

Observation 76caf7a8-d3b8-4f4b-9cc3-95bb2eca2dbb · outbound

This paper cites function call count.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems function call count

Reference 42

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:d11ddcd5065672138c7253ecc28c8b960854137b32ad189c70b573ee886deb31

Observation 0464f3e4-a418-4e2a-955c-72438513e414 · outbound

This paper cites Deterministic counters/ rates/percentiles already arrive via the PRE-CHOSEN METRICS block (user prompt) and are merged automatically -- do NOT duplicate them.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Deterministic counters/ rates/percentiles already arrive via the PRE-CHOSEN METRICS block (user prompt) and are merged automatically -- do NOT duplicate them

Reference 43

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:e0ef43618f63a0c6f2331cbb558852a64f3b28c9f726d3f5bd9ae33fc3ef0c3f

Observation 335e1cb8-7128-4cce-a660-51e865c93e55 · outbound

This paper cites accuracy.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems accuracy

Reference 44

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:3e12f3fe952fc54b89bc0cbda3c4f3ed5635739c3a2804068a33c9164e584aa5

Observation 2597d384-b023-4bf1-86e2-350e94a0e0b4 · outbound

This paper cites Relevancy and clarity are separate metrics, not one.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Relevancy and clarity are separate metrics, not one

Reference 45

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:36947a6b300f655dbeccc3467e55ec607179fe40185fbebf712c8ef40415f236

Observation 0dc9067b-b8f1-4d64-a2ed-3695d6883328 · outbound

This paper cites Use LLM-as-Judge ONLY for semantic quality that cannot be computed deterministically.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Use LLM-as-Judge ONLY for semantic quality that cannot be computed deterministically

Reference 46

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:686a34446b322e2ff01befc17161d3ac2affdb73820a8766423c3e7b0580ba18

Observation e4c9dc26-57a7-4447-ae3c-99ec0fefd729 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 47

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:7a813d5c588d0fa0a7f0542853eb8935c5398b0844b3aaf7af799a00001eb55a

Observation 1eee4597-af8c-42f2-b874-a217bd99c790 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 48

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:0ee7f535544bdc96d3c435c9dab66d2bdae79e53729e4b426b8308f09c55d1a9

Observation 279ca602-b6e4-482d-8297-0f9d904c0a4c · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 49

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:5d89933b736d84495df13b7f670083ff0b3e3665c41acc91105894c392d4f276

Observation cac67077-871d-49bb-b80b-4e757c05a2bf · outbound

This paper cites traceability = citation coverage + link strength + evidence breadth), design a composite metric with weighted sub-metrics and a formula.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems traceability = citation coverage + link strength + evidence breadth), design a composite metric with weighted sub-metrics and a formula

Reference 50

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:37b3c9f0ca1246954ae7e1da320aef23eccca041273da17afb4641c0842d6af2

Observation ca64771c-3ae1-4a43-98cf-362df3cb83f6 · outbound

This paper cites an unresolved cited work.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Unresolved cited work

Reference 51

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:09f0aa7e76c7a2d8d1fe2ccf7491f3b514c8c91ec1ebbdcb49e1c5cfc8eabcf6

Observation 81304400-0d42-4a0a-a09a-ddbea6c588d7 · outbound

This paper cites Accuracy + Latency is NOT a valid composite.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Accuracy + Latency is NOT a valid composite

Reference 52

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:a2950b1a5ec7f91dbd8d73920e6cb24aa9d8ba43974cce630c51739935e44221

Observation f0ac5948-29d8-4cf0-ada2-4ce706463183 · outbound

This paper cites X_grounding.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems X_grounding

Reference 53

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:435bc6df1393e913a11a7e3701983914e349531670b392f7088f73d3472d71f2

Observation 1e8f2ac2-3e51-49d6-b62d-c96be2a3b5d5 · outbound

This paper cites cost_per_quality.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems cost_per_quality

Reference 54

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:29395bb57894a542b96b91585b2ff6a38e7670431b0a8901c91aac158a0462b4

Observation 206b15f9-735d-4138-bd4c-96a198d45b3a · outbound

This paper cites Explain how they correlate -- e.g.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Explain how they correlate -- e.g

Reference 55

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:f5196e99bdaa65a9a27112db5fd38675dd53fb4b7a11eead8781cced9d66df55

Observation 5bb85000-6fc1-47c0-be44-ece3227d8ed7 · outbound

This paper cites For each gap, explain what is missing and suggest how to address it (e.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems For each gap, explain what is missing and suggest how to address it (e

Reference 56

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:1db02028e402e895043ffe6714696717c8c15b78524b19ee8323ea8cc0c91e57

Observation 683e5bfc-c23e-40b5-ac44-dd93c4e31b55 · outbound

This paper cites > 500ms for 5 minutes.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems > 500ms for 5 minutes

Reference 57

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:a2ae888fecaf222f86c2a02bd57fa44da10390182df789b09dfdc328d38a5b37

Observation f3112069-1aa5-4d8b-8814-4ae6c3e80c31 · outbound

This paper cites Reference actual source files when available.

Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems Reference actual source files when available

Reference 58

Resolution
unresolved
no resolver link, observed 2026-06-26T08:16:51.419186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-06-26T08:16:51.419186Z digest=sha256:c23e8d57f41dea532dbcee83d449a87952d56c3c00cfe9e3679ae02be4aed220

Pith citing papers

No inbound Pith citation observations are available.