Paper Citation Record · LEDGER

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks

As of 19 August 2026, this Paper Citation Record lists 22 of 22 outbound references and 0 inbound Pith citation observations for arXiv:2607.27518.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2607.27518 v1

Coverage vector

measured 22 of 22 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-01T06:25:00.076693Z

measured 22 of 22 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-19T06:32:44.657259+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

22 of 22 outbound references displayed

verified exact0
verified fuzzy0
unresolved22
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 43e11603-7f26-4f4f-afbb-59b226ee71f0 · outbound

This paper cites MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.623173Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ba71c2e3-2283-457c-ab5a-d02ceb5ff26f · outbound

This paper cites Seven simple steps for log analysis in AI systems.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Seven simple steps for log analysis in AI systems

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.709724Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6627a295-fbbe-4c4e-b302-db9fccb0b6bb · outbound

This paper cites Measuring AI agents’ progress on multi-step cyber attack scenarios.arXiv preprint arXiv:2603.11214,.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Measuring AI agents’ progress on multi-step cyber attack scenarios.arXiv preprint arXiv:2603.11214,

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.795365Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 51a99686-3a9b-4f65-b9fa-34da2d4abaa7 · outbound

This paper cites Sayash Kapoor.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Sayash Kapoor

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.886679Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e7088df3-e805-46ef-8379-0e1d7896948d · outbound

This paper cites Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.053330Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 505b22e1-49f5-48b5-9e56-cafd480d89e8 · outbound

This paper cites KernelBench: Can LLMs Write Efficient GPU Kernels?.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks KernelBench: Can LLMs Write Efficient GPU Kernels?

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.127654Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7bfc0b18-c56c-42b3-ad83-053913b0618a · outbound

This paper cites Humanity's Last Exam.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Humanity's Last Exam

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.202308Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 37882ef8-f420-4dbd-af7c-29969ea7c76e · outbound

This paper cites PostTrainBench: CanLLMagentsautomateLLMpost-training?arXiv preprint arXiv:2603.08640,.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks PostTrainBench: CanLLMagentsautomateLLMpost-training?arXiv preprint arXiv:2603.08640,

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.266458Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3c739425-967f-4349-a8a0-56f03299ffe2 · outbound

This paper cites GPQA: A Graduate-Level Google-Proof Q&A Benchmark.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.336807Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5332bbed-fc19-4654-9342-38f9907a4c5a · outbound

This paper cites Zachary S.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Zachary S

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.409558Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8f278a99-1448-4012-a90e-41ba1e0aee7c · outbound

This paper cites CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.489925Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 232bb22d-d1e1-423c-958b-43a7223beb6b · outbound

This paper cites BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.579784Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c5a6f6b5-3bd7-4527-8feb-e3baa75a3d7e · outbound

This paper cites Automated Benchmark Auditing for AI Agents and Large Language Models.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Automated Benchmark Auditing for AI Agents and Large Language Models

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.673010Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 98176a1b-d30a-473d-84b3-eb363626bb36 · outbound

This paper cites ISBN 979-8-89176-251-0.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks ISBN 979-8-89176-251-0

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.745650Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation df392865-68cb-4dc7-9285-56fc59986a67 · outbound

This paper cites Establishing Best Practices for Building Rigorous Agentic Benchmarks.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Establishing Best Practices for Building Rigorous Agentic Benchmarks

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.912539Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 57a2c2a5-dab5-4a94-92ab-b33de042f2dc · outbound

This paper cites an unresolved cited work.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Unresolved cited work

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.995156Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6fcf916a-ec3f-4aeb-bb44-7d465402a2ee · outbound

This paper cites write to /app/out.html.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks write to /app/out.html

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-01T06:25:00.076693Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bf6fa028-3ef7-4606-80dc-77685dfdbf4d · outbound

This paper cites HellaSwag: Can a Machine Really Finish Your Sentence?.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks HellaSwag: Can a Machine Really Finish Your Sentence?

Reference 2019

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:59.835059Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fa861514-1852-499f-9692-3cfa92d50724 · outbound

This paper cites TruthfulQA: Measuring How Models Mimic Human Falsehoods.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks TruthfulQA: Measuring How Models Mimic Human Falsehoods

Reference 2022

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.973480Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5f0e2ef3-8c23-4cea-a165-92b98e898a93 · outbound

This paper cites Kao, Evangelia Spiliopoulou, and Adina Williams.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks Kao, Evangelia Spiliopoulou, and Adina Williams

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.541183Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7d7df7e5-3904-428d-a5f8-0c65f2366e4e · outbound

This paper cites SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.454362Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 048ba888-a8b2-4d5c-b8f5-12df41a89cb9 · outbound

This paper cites $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment.

Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment

Reference 2026

Resolution: unresolved; no resolver link, observed 2026-08-01T06:24:58.374494Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.