Paper Citation Record · LEDGER

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations

As of 20 August 2026, this Paper Citation Record lists 27 of 27 outbound references and 0 inbound Pith citation observations for arXiv:2509.10963.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2509.10963 v1

Coverage vector

measured 27 of 27 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-04T17:26:38.394751Z

measured 27 of 27 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-20T06:33:59.587034+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

27 of 27 outbound references displayed

verified exact0
verified fuzzy0
unresolved27
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation c83ea906-353f-44e0-b624-c9fe36a54cbd · outbound

This paper cites A., and Sheikh, J.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations A., and Sheikh, J

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.274545Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6f44d130-7c88-43dc-91fd-b7eabc4621cb · outbound

This paper cites GPT-4 Technical Report.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations GPT-4 Technical Report

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.281877Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 92eb883e-14b0-40b7-ba1f-cf10bb00367e · outbound

This paper cites PCS-UQ: Uncertainty Quantification via the Predictability-Computability-Stability Framework.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations PCS-UQ: Uncertainty Quantification via the Predictability-Computability-Stability Framework

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.286778Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 262ac843-df59-4488-b0a3-78a6a830d550 · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.291673Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5ff41350-95a1-4dd6-a4b6-e4441380da6f · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.295941Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ee658f95-f988-4f5e-81c7-455f8ada20ba · outbound

This paper cites Premise Order Matters in Reasoning with Large Language Models.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Premise Order Matters in Reasoning with Large Language Models

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.300315Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b4326488-5829-44d2-8156-94dab54f8307 · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.305563Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 17d0b253-d3ad-4e5a-b560-e53b2b545bf9 · outbound

This paper cites Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.309880Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 60088fa0-5e18-4d49-aeaf-b07d58503228 · outbound

This paper cites The Llama 3 Herd of Models.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations The Llama 3 Herd of Models

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.314279Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 38076c9e-4f16-45cd-a402-be49e2cfe0ee · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.319011Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f019bd5b-25b9-4d3a-af87-9a9adb3d788a · outbound

This paper cites A Statistical Turing Test for Generative Models.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations A Statistical Turing Test for Generative Models

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.323143Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a54931fd-ea48-473d-a5de-b2c9e84bb2d0 · outbound

This paper cites From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.328095Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4e6bcd10-9aa7-49db-b159-50e9b757ee75 · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.332606Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0936f829-08fc-4804-b0d8-bb73ab1b9fba · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.337094Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7f977a3c-2a57-46c3-823a-b8f29db191db · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.341371Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a649924f-008f-4c42-9079-f22fba09622d · outbound

This paper cites Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.345872Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d2966f35-8df5-4396-8403-1ab3f0feb77c · outbound

This paper cites MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.350247Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 797b5eb6-1460-48b0-b778-10461e3117c3 · outbound

This paper cites Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.354877Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0bba18bf-9cd8-46cb-8a49-697560c21547 · outbound

This paper cites M., Terano, H.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations M., Terano, H

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.359620Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b8dbb191-40e5-4b80-948f-30f759369290 · outbound

This paper cites J., Ryan, P.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations J., Ryan, P

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.363860Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 02110994-59fe-4827-af19-80ee369c0289 · outbound

This paper cites Detecting Pretraining Data from Large Language Models.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Detecting Pretraining Data from Large Language Models

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.368248Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 46b0ba09-038a-44ab-81fe-2b23fdf69ec1 · outbound

This paper cites an unresolved cited work.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Unresolved cited work

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.372460Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e96bdd8e-e364-4eb7-95ea-4207c7361227 · outbound

This paper cites A Short Survey of Viewing Large Language Models in Legal Aspect.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations A Short Survey of Viewing Large Language Models in Legal Aspect

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.376545Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5e31537c-3a1f-4ce6-902d-dcf2b6ab02da · outbound

This paper cites Gemini: A Family of Highly Capable Multimodal Models.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Gemini: A Family of Highly Capable Multimodal Models

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.381255Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3e159ec2-9d47-49dd-9ef2-f4cfb13fcec1 · outbound

This paper cites Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.385609Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0026d3b8-97b2-4d65-801b-3de412dd33f2 · outbound

This paper cites and Barter, R.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations and Barter, R

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.390712Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 227be055-c831-4a01-808e-66ccb3da17de · outbound

This paper cites CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges.

Testing for LLM response differences: the case of a composite null consisting of semantically irrelevant query perturbations CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-04T17:26:38.394751Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.