Paper Citation Record · LEDGER

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

As of 19 August 2026, this Paper Citation Record lists 70 of 70 outbound references and 5 inbound Pith citation observations for arXiv:2508.13144.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2508.13144 v1

Coverage vector

measured 70 of 70 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-15T17:21:06.866133Z

measured 75 of 75 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-19T06:32:44.657259+00:00

measured 5 of 5 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-15T14:19:01.315611Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-02T23:47:27.665680Z

Reference resolution

70 of 70 outbound references displayed

verified exact1
verified fuzzy16
unresolved52
parse uncertain0
malformed identifier1
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation e05913ba-f248-470b-8ce7-89afcc758094 · outbound

This paper cites Program Synthesis with Large Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Program Synthesis with Large Language Models

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.602037Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5bed2fbe-23cb-4657-8495-93c4718f35d4 · outbound

This paper cites An empirical investigation of statistical significance in NLP.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation An empirical investigation of statistical significance in NLP

Reference 2

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.648346Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation a3c03f77-dfbd-46e9-b42f-b5aad8f57ab6 · outbound

This paper cites Establishing Task Scaling Laws via Compute-Efficient Model Ladders.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Establishing Task Scaling Laws via Compute-Efficient Model Ladders

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.610860Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fdbc956b-b11b-446a-8cb6-66e797f35396 · outbound

This paper cites Piqa: Reasoning about physical commonsense in natural language.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Piqa: Reasoning about physical commonsense in natural language

Reference 4

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.637264Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation b1035626-9902-496a-8b1b-58858639b199 · outbound

This paper cites Does your data spark joy? Performance gains from domain upsampling at the end of training.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Does your data spark joy? Performance gains from domain upsampling at the end of training

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.619163Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9ecacb2e-2e88-4c62-b564-f43dbd4ca89c · outbound

This paper cites Position: Don't Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Position: Don't Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.623462Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 67ec5ff3-50be-4280-ba5e-f485d514b922 · outbound

This paper cites With little power comes great responsibility.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation With little power comes great responsibility

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.627643Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4f4b48fe-7993-4785-8546-de05fc20e669 · outbound

This paper cites Evaluating Large Language Models Trained on Code.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Evaluating Large Language Models Trained on Code

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.631354Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 399d888d-29f1-4d1f-8c83-57a8af153571 · outbound

This paper cites A Hitchhiker's Guide to Scaling Law Estimation.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation A Hitchhiker's Guide to Scaling Law Estimation

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.635018Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d74f2461-158f-444f-917f-42dc1d26a352 · outbound

This paper cites Boolq: Exploring the surprising difficulty of natural yes/no questions.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Boolq: Exploring the surprising difficulty of natural yes/no questions

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.626502Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 6d534e44-cf20-48f6-8125-ccec2d3e7422 · outbound

This paper cites Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.642868Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 71e8644f-a36e-4011-aaa5-e94db02090b5 · outbound

This paper cites Training Verifiers to Solve Math Word Problems.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Training Verifiers to Solve Math Word Problems

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.646553Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6f107d14-e154-4053-9744-c53c6b3497de · outbound

This paper cites Un- derspecification presents challenges for credibility in modern machine learning.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Un- derspecification presents challenges for credibility in modern machine learning

Reference 13

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.616189Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation b2512895-8f04-436e-91a2-f72e947e82d9 · outbound

This paper cites Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.654290Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 12920284-d0f9-4e04-a45f-19beed62d836 · outbound

This paper cites Understanding Emergent Abilities of Language Models from the Loss Perspective.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Understanding Emergent Abilities of Language Models from the Loss Perspective

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.657960Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 849ef073-95a7-4bbc-a4b5-9b742438d1f2 · outbound

This paper cites Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs

Reference 16

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.604615Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation bfd72767-cbdc-4198-aad9-74f43d1e77d0 · outbound

This paper cites The Llama 3 Herd of Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation The Llama 3 Herd of Models

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.665186Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7debad2f-c7a6-4d3a-98aa-4ffde8f26d3c · outbound

This paper cites Open llm leaderboard v2.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Open llm leaderboard v2

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.668985Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7b6af821-46f4-4882-aa93-fa37548a0574 · outbound

This paper cites Language models scale reliably with over-training and on downstream tasks.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Language models scale reliably with over-training and on downstream tasks

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.672554Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation caa45890-2edc-4fcc-8a50-4ad35b422e7a · outbound

This paper cites The Pile: An 800GB Dataset of Diverse Text for Language Modeling.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.676181Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6c1b5926-b73b-485e-8ead-9f20cf4e7070 · outbound

This paper cites Are We Done with MMLU?.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Are We Done with MMLU?

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.679877Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation eacdda79-bdab-4511-b107-0f9eb807efe3 · outbound

This paper cites OLMES: A Standard for Language Model Evaluations.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation OLMES: A Standard for Language Model Evaluations

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.683770Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6fff7c1b-047d-497e-a0e2-c18809fa2ee9 · outbound

This paper cites Measuring Massive Multitask Language Understanding.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Measuring Massive Multitask Language Understanding

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.687583Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e42f60ea-7c58-44ff-bc85-99b4c3a4d298 · outbound

This paper cites Training Compute-Optimal Large Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Training Compute-Optimal Large Language Models

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.691083Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 759da69e-87db-4833-8e14-b6a9b1819c95 · outbound

This paper cites Compression Represents Intelligence Linearly.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Compression Represents Intelligence Linearly

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.694837Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b0faf913-cc1d-4d17-8d23-7038fcbda701 · outbound

This paper cites Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension

Reference 26

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.586179Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation c92b553a-ee50-4228-a748-5a746c658b45 · outbound

This paper cites Scaling Laws for Neural Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Scaling Laws for Neural Language Models

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.702386Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 09caf088-d013-419e-b24f-aabf172c6bd2 · outbound

This paper cites Natural questions: A benchmark for question answering research.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Natural questions: A benchmark for question answering research

Reference 28

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.574010Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 6fcf2374-916f-40f2-92c3-b7af8d0e773c · outbound

This paper cites Finetasks: Finding signal in a haystack of 200+ multilingual tasks, 2024.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Finetasks: Finding signal in a haystack of 200+ multilingual tasks, 2024

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.561859Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 65d90a86-b989-419c-92b8-db89f68b9bff · outbound

This paper cites Solving Quantitative Reasoning Problems with Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Solving Quantitative Reasoning Problems with Language Models

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.713094Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 98757a67-a984-4d43-b884-dd5987894c53 · outbound

This paper cites Datacomp-lm: In search of the next generation of training sets for language models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Datacomp-lm: In search of the next generation of training sets for language models

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.716687Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation aa6ef03c-ca95-4aad-8484-a58bc1bed9f9 · outbound

This paper cites AutoBencher: Towards Declarative Benchmark Construction.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation AutoBencher: Towards Declarative Benchmark Construction

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.720504Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e98de564-0e70-4497-adb2-4db9a41f2a3f · outbound

This paper cites Holistic Evaluation of Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Holistic Evaluation of Language Models

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.724061Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c82f593d-909f-402b-8ef2-7b9b487607c0 · outbound

This paper cites Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.727477Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 43ea5e26-5e14-4592-b5f7-3f0a0caa4e0f · outbound

This paper cites RegMix: Data Mixture as Regression for Language Model Pre-training.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation RegMix: Data Mixture as Regression for Language Model Pre-training

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.730725Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f30632b9-3531-4029-9b0c-61a24be3029f · outbound

This paper cites Quantifying Variance in Evaluation Benchmarks.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Quantifying Variance in Evaluation Benchmarks

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.734304Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation baa20cb0-0ce7-45a0-8161-c2b529ff449b · outbound

This paper cites Paloma: A Benchmark for Evaluating Language Model Fit.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Paloma: A Benchmark for Evaluating Language Model Fit

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.737675Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a67e8339-a73f-44f7-ac98-7979e9858feb · outbound

This paper cites Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A

Reference 38

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.535539Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 2c6cce69-e09c-4b38-8ec7-892d8b10648d · outbound

This paper cites Can a suit of armor conduct electricity? a new dataset for open book question answering.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Can a suit of armor conduct electricity? a new dataset for open book question answering

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.745035Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2b0035f8-99b7-4487-a703-a34fe4963d10 · outbound

This paper cites Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.748522Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 838eefdc-f0c6-4ddd-9cd8-1d4b4a766fee · outbound

This paper cites GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.752982Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 617c3e2f-5a1d-4520-a575-3613e1b2bccf · outbound

This paper cites 2 OLMo 2 Furious.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation 2 OLMo 2 Furious

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.757147Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 95ab2b1f-7a23-473b-b5df-61ae2d3b73fc · outbound

This paper cites Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering

Reference 43

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.517487Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 27abd2b6-a631-4563-bf5d-8cddd35aee25 · outbound

This paper cites Reconciling Kaplan and Chinchilla Scaling Laws.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Reconciling Kaplan and Chinchilla Scaling Laws

Reference 44

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.764274Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2a652f7d-4536-4514-8b40-cfeab25c0d00 · outbound

This paper cites tinyBenchmarks: evaluating LLMs with fewer examples.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation tinyBenchmarks: evaluating LLMs with fewer examples

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.769157Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cb7de8f9-3a79-4321-8963-74a5d054c843 · outbound

This paper cites VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.773082Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ee5b7608-cf6c-42c0-a1c5-5987eb979b4f · outbound

This paper cites Squad: 100,000+ questions for machine comprehension of text.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Squad: 100,000+ questions for machine comprehension of text

Reference 47

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.776888Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5eec718c-2a99-4bbf-be7c-768780257b2a · outbound

This paper cites an unresolved cited work.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Unresolved cited work

Reference 48

Resolution: unresolved; raw_fallback, observed 2026-08-15T17:21:07.497106Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation cc21d9c2-fd19-4815-bfeb-ed14ac6b8347 · outbound

This paper cites GPQA: A Graduate-Level Google-Proof Q&A Benchmark.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Reference 49

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.784311Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7392c061-e889-4e90-8d64-3eed331ce2fe · outbound

This paper cites Compute Optimal Scaling of Skills: Knowledge vs Reasoning.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Compute Optimal Scaling of Skills: Knowledge vs Reasoning

Reference 50

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.788592Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a4d2d35a-d8a7-4a6f-aa61-f6a0224d030c · outbound

This paper cites Observational scaling laws and the predictability of langauge model performance.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Observational scaling laws and the predictability of langauge model performance

Reference 51

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.484796Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 32ddd0d9-6f4d-4ce6-bf56-80815b7030da · outbound

This paper cites Winogrande: An adversarial winograd schema challenge at scale.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Winogrande: An adversarial winograd schema challenge at scale

Reference 52

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.465361Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation c0a280ef-0a83-49ee-92a1-93eeeb8c8f34 · outbound

This paper cites Social iqa: Commonsense reasoning about social interactions.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Social iqa: Commonsense reasoning about social interactions

Reference 53

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.452734Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 39f6ecb1-ebb6-4953-90e6-af124b5c14eb · outbound

This paper cites Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?

Reference 54

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.804001Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 685cb3a3-87e0-4fc2-8c59-257766f9a5a6 · outbound

This paper cites Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

Reference 55

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.808213Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 04727587-5435-47c6-9a54-2a8a9b07f2fd · outbound

This paper cites Predictive Data Selection: The Data That Predicts Is the Data That Teaches.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Predictive Data Selection: The Data That Predicts Is the Data That Teaches

Reference 56

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.812208Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6ee9b21c-bb66-41d5-bc41-4c5df5b00050 · outbound

This paper cites Predicting Emergent Capabilities by Finetuning.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Predicting Emergent Capabilities by Finetuning

Reference 57

Resolution: verified exact; local_arxiv, observed 2026-08-15T17:21:06.992504Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation eb372f24-0521-4aaa-b5ff-3b5628717d1e · outbound

This paper cites Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Reference 58

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.821162Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8ef02911-d648-40d4-bcdf-fdbbcd2c706e · outbound

This paper cites Commonsenseqa: A question answering challenge targeting commonsense knowledge.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Commonsenseqa: A question answering challenge targeting commonsense knowledge

Reference 59

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.439453Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 372e5ce7-554d-48dd-98f9-629558701eda · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 60

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.829382Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ed837778-35c6-4578-94e9-b864220a6874 · outbound

This paper cites 200,000+ jeopardy! questions.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation 200,000+ jeopardy! questions

Reference 61

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.426367Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation a4dd801b-6fec-4047-856c-bb21901d5adb · outbound

This paper cites Do Large Language Model Benchmarks Test Reliability?.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Do Large Language Model Benchmarks Test Reliability?

Reference 62

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.836637Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8e3817de-6df9-4aa0-b7b4-89753cc2fb02 · outbound

This paper cites Wang, Alex Gu, Lovish Madaan, Dieuwke Hupkes, Jiawei Liu, Yuxiang Wei, Naman Jain, Yuhang Lai, Sten Sootla, Ofir Press, Baptiste Rozière, and Gabriel Synnaeve.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Wang, Alex Gu, Lovish Madaan, Dieuwke Hupkes, Jiawei Liu, Yuxiang Wei, Naman Jain, Yuhang Lai, Sten Sootla, Ofir Press, Baptiste Rozière, and Gabriel Synnaeve

Reference 63

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T17:21:07.412450Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 1366c2c9-b3ff-4405-befe-c5117d9a1edf · outbound

This paper cites Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

Reference 64

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.843954Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7ee62304-b34c-4477-b14b-5f925c0dd973 · outbound

This paper cites Emergent Abilities of Large Language Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Emergent Abilities of Large Language Models

Reference 65

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.847630Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7baf326e-4d8a-48b6-8546-a51bfaddc91e · outbound

This paper cites Organize the Web: Constructing Domains Enhances Pre-Training Data Curation.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Organize the Web: Constructing Domains Enhances Pre-Training Data Curation

Reference 66

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.851539Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2cd86841-44aa-4f4a-8e0c-0c1f4b79bc3d · outbound

This paper cites Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions

Reference 67

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.855163Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e4ece66d-d26b-492a-9649-4306861d56fc · outbound

This paper cites Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

Reference 68

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.858923Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8d8b5102-915b-4c9c-99ad-3fa46c50b6de · outbound

This paper cites Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, 2019.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, 2019

Reference 69

Resolution: unresolved; no resolver link, observed 2026-08-15T17:21:06.862717Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7828047d-0797-4a6d-9b9a-37881801c231 · outbound

This paper cites AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models.

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

Reference 70

Resolution: malformed identifier; no resolver link, observed 2026-08-15T17:21:06.866133Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 5836d679-f4c9-4ecd-9e88-f0cc6685728c · inbound

Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation cites this paper.

Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

Reference 11

Resolution: metadata mismatch; arxiv_id, observed 2026-06-28T22:52:45.619149Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation e143dbd7-b379-4fcb-8923-851e2092691d · inbound

Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation cites this paper.

Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

Reference 21

Resolution: verified exact; arxiv_id, observed 2026-07-02T23:47:27.667294Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 28aed6d8-f367-48d4-94ae-562d17e30cdc · inbound

DataComp-VLM: Improved Open Datasets for Vision-Language Models cites this paper.

DataComp-VLM: Improved Open Datasets for Vision-Language Models Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

Reference 97

Resolution: verified exact; arxiv_id, observed 2026-07-01T15:45:47.697916Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation cb4686b8-c5f2-4d3c-b97c-306c93e5f914 · inbound

DataComp-VLM: Improved Open Datasets for Vision-Language Models cites this paper.

DataComp-VLM: Improved Open Datasets for Vision-Language Models Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

Reference 97

Resolution: verified exact; arxiv_id, observed 2026-07-02T21:17:24.071487Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-19T06:32:44.657259+00:00.

Observation 9da7396a-b91e-48bb-8ee5-177e866cf05f · inbound

Deployment Decision Reliability: A Generalizability-Theory Framework for Sizing Long-Horizon Agent Evaluations cites this paper.

Deployment Decision Reliability: A Generalizability-Theory Framework for Sizing Long-Horizon Agent Evaluations Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation

Reference 1972

Resolution: unresolved; no resolver link, observed 2026-08-15T14:19:01.315611Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.