Paper Citation Record · LEDGER

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing

As of 8 August 2026, this Paper Citation Record lists 48 of 48 outbound references and 0 inbound Pith citation observations for arXiv:2505.21701.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2505.21701 v2

Coverage vector

measured 48 of 48 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-07T13:30:26.011659Z

measured 48 of 48 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-08T06:32:00.761636+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

48 of 48 outbound references displayed

verified exact2
verified fuzzy0
unresolved46
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 2160ebd6-abce-4875-83f5-87a73113b2ca · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.155132Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e53a55d4-f1da-4d84-9162-b6b42e6b789f · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.303604Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c9c3e8f5-c30e-4081-b815-cf2976966d9a · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.391545Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 77c8f1b6-40f0-47a4-8ddd-301fdbfb763b · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.485820Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cbec1905-a5b6-49e2-a906-4208630f06d6 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.603860Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3605274f-f4a7-41ba-87ac-4312d1bdc3a3 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.694922Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 74b9a17d-9390-49aa-bdb4-563d5e50c6f7 · outbound

This paper cites Training Verifiers to Solve Math Word Problems.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Training Verifiers to Solve Math Word Problems

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.804142Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1adbb8d7-da61-4734-9806-3da64e4019bc · outbound

This paper cites The Llama 3 Herd of Models.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing The Llama 3 Herd of Models

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:21.933229Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5ce9d337-bcc7-49f7-98c4-534c06419a45 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 9

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:29.615941Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 0880a96d-7711-4dcc-881f-a06f3988a28f · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 10

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:29.408245Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation ab3bc857-5625-4002-bf24-85e733781fc0 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:22.244837Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0a8c34cd-9589-4be2-8415-9688a77d4999 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 12

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:29.201428Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 568fd548-2739-4e76-87c0-1645c5c6f353 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:22.391721Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9e12e19d-2e05-4462-ada4-80f275cc6182 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:22.500327Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 08491af3-aa22-4ec1-bc1b-c36c08d492ce · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:22.634899Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 31e482bb-f1c3-49c1-972d-c66f12327d56 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 16

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:29.046954Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation d6972f44-6f35-44e9-afe1-6b208568a12b · outbound

This paper cites Mistral 7B.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Mistral 7B

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:22.786432Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 19dcc578-cfb2-45b4-85ee-312744fb35ef · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 18

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:28.892340Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 41206786-842c-44f9-8f6e-b72adfc5f355 · outbound

This paper cites Language Models (Mostly) Know What They Know.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Language Models (Mostly) Know What They Know

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:22.967772Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9e77c766-f904-41e7-9422-4b843e01c190 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.078685Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 85a568e3-3f4a-4089-b8f9-c5cd10ccbe12 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:28.735952Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation e3d7337e-c5f2-402c-a494-dcd852d7b7f3 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.271856Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 253195a5-fe29-40aa-b867-579fa7073948 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.339406Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6524e2e9-6c6e-473d-a380-152a3b2dc499 · outbound

This paper cites SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.435924Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 76b378aa-c90e-447b-bb72-ae22eca02069 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 25

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:28.579969Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 18003982-0d7d-4531-96d0-785666ad9529 · outbound

This paper cites 2 OLMo 2 Furious.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing 2 OLMo 2 Furious

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.644760Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 15008fa0-a9be-4dae-9f4a-9273b4c6ecb8 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.724903Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a1a6059d-5697-4af0-a386-3b74480e64bb · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.814905Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3b362466-1122-41e4-b8c2-8c275f5079cb · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.932117Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bc57a897-5770-470e-a2eb-ccaa147eba21 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:23.980953Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 94f922f4-98bb-4fc2-9143-93475eb6e68c · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.062471Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3fcd9f06-e351-45b3-a36e-a498ebbe2847 · outbound

This paper cites Large Language Models are Inconsistent and Biased Evaluators.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Large Language Models are Inconsistent and Biased Evaluators

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.165124Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 68c98458-5e97-4284-a943-da939749d7a2 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 33

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:28.467355Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 41a10021-f421-493d-a498-2864acc502ad · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.271956Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 49b491ab-41b3-47f3-bbdd-40598e3f823f · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.341479Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 24b71c23-db40-408d-8e42-4934950d5e81 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 36

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:28.024749Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation cb581d2f-5735-442e-aea0-3be5fedc44c2 · outbound

This paper cites Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.644826Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 36ee7bc2-f65f-4727-b1b7-96a75465fd4d · outbound

This paper cites Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.734842Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 32f1f14c-4a65-4585-be33-4edbf40541f9 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 39

Resolution: verified exact; doi, observed 2026-08-07T13:30:27.014760Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 87287119-bbe8-42a4-9509-1bcf24dcb504 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 40

Resolution: unresolved; raw_fallback, observed 2026-08-07T13:30:27.674930Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 3509225c-0ef7-4b5f-a713-76e4a1114a3e · outbound

This paper cites Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:24.985286Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8c761e32-763f-49b0-882a-188400c7297a · outbound

This paper cites Knowledge Circuits in Pretrained Transformers.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Knowledge Circuits in Pretrained Transformers

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:25.090194Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 99d5dd6d-3005-4703-a2d3-4bcaaeaeea8b · outbound

This paper cites o rg Schl \.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing o rg Schl \

Reference 43

Resolution: verified exact; doi, observed 2026-08-07T13:30:26.564838Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 3dfac7d4-82c4-46ca-b101-05746a328c39 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 44

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:25.474900Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation edb885ce-99e3-4baa-9c96-49e6b861da4b · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:25.665721Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c1cebbc7-1f4c-452d-b967-5aaa63611604 · outbound

This paper cites an unresolved cited work.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing Unresolved cited work

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:25.799721Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 863bda5e-a294-444c-a119-264bdfa7a4e3 · outbound

This paper cites online" 'onlinestring :=.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing online" 'onlinestring :=

Reference 47

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:25.944752Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3220b2df-653f-4b3e-870a-8abe5405c5e2 · outbound

This paper cites write newline.

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing write newline

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-07T13:30:26.011659Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.