Paper Citation Record · LEDGER

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

As of 18 August 2026, this Paper Citation Record lists 64 of 64 outbound references and 64 inbound Pith citation observations for arXiv:2411.15114.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2411.15114 v2

Coverage vector

measured 64 of 64 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-12T14:33:13.231482Z

measured 128 of 128 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-17T06:30:58.91139+00:00

measured 64 of 64 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-16T12:40:14.612983Z

measured 1 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: pith, observed 2026-08-05T02:28:24.338817Z

Reference resolution

64 of 64 outbound references displayed

verified exact1
verified fuzzy17
unresolved44
parse uncertain0
malformed identifier2
metadata mismatch0

External citation measurements

2
pith, observed 2026-08-05T02:28:24.338817Z

Outbound references

Observation 93e50550-a810-40fc-a4a7-8283917db63c · outbound

This paper cites Evaluating Large Language Models Trained on Code.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Evaluating Large Language Models Trained on Code

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.069528Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9214f6e2-9035-4c84-9c81-1fd6d4366a98 · outbound

This paper cites Competition-level code generation with AlphaCode.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Competition-level code generation with AlphaCode

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.073102Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d68dc253-f549-4593-aebf-c77561f0f0ee · outbound

This paper cites Textbooks Are All You Need.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Textbooks Are All You Need

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.076189Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bea0344b-a812-44b7-aef2-106eb00a4911 · outbound

This paper cites The Llama 3 Herd of Models.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts The Llama 3 Herd of Models

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.078973Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 419e9304-59a9-477e-88c0-8b9dfed02aa1 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 5

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.703896Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 2f4af9d2-ff40-4e0e-9bec-fe388e460293 · outbound

This paper cites OpenHands: An Open Platform for AI Software Developers as Generalist Agents.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts OpenHands: An Open Platform for AI Software Developers as Generalist Agents

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.084827Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fb9d432a-3125-41da-b3f2-89fed418c256 · outbound

This paper cites Interviewing AI researchers on automation of AI r&d (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Interviewing AI researchers on automation of AI r&d (2024)

Reference 7

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.697544Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation d94f04f1-6f4d-4ba2-b655-c0a96dd5ba44 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 8

Resolution: verified exact; doi, observed 2026-08-12T14:33:13.252997Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 13b94930-6795-4441-aff6-0483157c500f · outbound

This paper cites Explosive growth from AI automation: A review of the arguments.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Explosive growth from AI automation: A review of the arguments

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.093306Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9108dd92-eba2-470c-b99b-80ec96d37abd · outbound

This paper cites OpenAI preparedness framework (beta).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts OpenAI preparedness framework (beta)

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.691722Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation daad44ba-e91c-40f9-8909-3730c0508d4a · outbound

This paper cites Frontier safety framework, version 1.0.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Frontier safety framework, version 1.0

Reference 11

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.685477Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation caa42555-43ef-46bc-9dda-6754402a07df · outbound

This paper cites Anthropic responsible scaling policy.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Anthropic responsible scaling policy

Reference 12

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.678875Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation cfac877e-9688-40de-b039-a5b053e7a108 · outbound

This paper cites Recital 110 of the eu artificial intelligence act (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Recital 110 of the eu artificial intelligence act (2024)

Reference 13

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.671828Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 8825082f-4866-415c-8950-0e5c94ab0ab6 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 14

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.663824Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation a2741764-7fae-47ce-9f96-def74a3d4ac0 · outbound

This paper cites The bletchley declaration on AI safety (2023).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts The bletchley declaration on AI safety (2023)

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.656928Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 2aebe51c-8d48-42b4-a261-88fb47af02e5 · outbound

This paper cites Frontier AI safety commitments, AI seoul summit 2024.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Frontier AI safety commitments, AI seoul summit 2024

Reference 16

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.650122Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 2d114eb8-0e65-4d24-81f6-aee212b8eaf4 · outbound

This paper cites Building an early warning system for llm-aided biological threat creation (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Building an early warning system for llm-aided biological threat creation (2024)

Reference 17

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.643125Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation bfcb303c-7e77-41d0-9a92-19acb60f329a · outbound

This paper cites CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.112305Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 21553f6f-d87d-4220-8f52-d8c10669d594 · outbound

This paper cites What a compute-centric framework says about takeoff speeds (2023).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts What a compute-centric framework says about takeoff speeds (2023)

Reference 19

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.636145Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 049cc47b-1736-4ae4-ba5f-dd9eddbf9ed7 · outbound

This paper cites Sabotage Evaluations for Frontier Models.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Sabotage Evaluations for Frontier Models

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.117927Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 43984ef0-bde5-411f-8d9a-bb6bf903d172 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.628799Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation b52f8e02-5694-430a-881c-ebe71f379032 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 22

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.622601Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation dda5e2a4-9525-41fa-bd72-aba38ab4de70 · outbound

This paper cites Raising the bar on swe-bench verified with claude 3.5 sonnet (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Raising the bar on swe-bench verified with claude 3.5 sonnet (2024)

Reference 23

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.616633Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 055c5b1d-80d5-4728-9bcd-93629c551826 · outbound

This paper cites Artificial Intelligence Index Report 2024.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Artificial Intelligence Index Report 2024

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.127921Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 26c207d8-7c32-4898-8e65-e5ff9de0db1e · outbound

This paper cites MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.130371Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 357ddd75-67dc-4f11-8700-e9dae2fa006f · outbound

This paper cites SciCode: A Research Coding Benchmark Curated by Scientists.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts SciCode: A Research Coding Benchmark Curated by Scientists

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.132899Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1f305af8-669c-4c5f-bd9c-842163806679 · outbound

This paper cites GPQA: A Graduate-Level Google-Proof Q&A Benchmark.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.135584Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1d91d83a-f5ba-4e82-9d6a-ddcc1cad57c0 · outbound

This paper cites GAIA: a benchmark for General AI Assistants.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts GAIA: a benchmark for General AI Assistants

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.138150Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5db7ccbf-5e43-407d-99a4-bbdb04f4c4d7 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 29

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.610070Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 33f1df9b-aaf8-44c7-b2d6-80125436611e · outbound

This paper cites WebArena: A Realistic Web Environment for Building Autonomous Agents.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts WebArena: A Realistic Web Environment for Building Autonomous Agents

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.143333Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 482e5e49-c928-48db-80dd-957436abcce4 · outbound

This paper cites MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.146051Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 70053717-4393-4d76-a4f4-d829c6c4311b · outbound

This paper cites DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.148692Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ab39c1ef-c811-4eb3-9f1c-5e419341688d · outbound

This paper cites H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.151881Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7add920a-c7e6-4f45-bd36-39f67fc91252 · outbound

This paper cites Training language models to follow instructions with human feedback.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Training language models to follow instructions with human feedback

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.154509Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9a730257-0cea-427b-94c2-65df035f2290 · outbound

This paper cites Constitutional AI: Harmlessness from AI Feedback.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Constitutional AI: Harmlessness from AI Feedback

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.157074Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 36689a9d-015e-49bd-955b-1b15cf741994 · outbound

This paper cites Nemotron-4 340B Technical Report.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Nemotron-4 340B Technical Report

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.159698Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation be93dd10-ed34-4eaf-a0ed-340d20b151c3 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.162197Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9306505f-d8a0-4dff-aa71-b43aee00c239 · outbound

This paper cites EvoPrompting: Language Models for Code-Level Neural Architecture Search.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts EvoPrompting: Language Models for Code-Level Neural Architecture Search

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.165447Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4cf5c454-0308-43ae-8b62-b4ad963cb9ed · outbound

This paper cites DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.167944Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0306d3bd-d6ea-4cfb-9ff7-ef2b4388d32f · outbound

This paper cites SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.170581Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 97edb651-32a6-4773-b83e-73d493d1b791 · outbound

This paper cites The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.173140Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d4886c2d-9919-4e81-a473-72b7b55735e7 · outbound

This paper cites Eureka: Human-Level Reward Design via Coding Large Language Models.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Eureka: Human-Level Reward Design via Coding Large Language Models

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.176588Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 619dfc4e-82ba-4856-ba1e-e93636af51f6 · outbound

This paper cites OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.179246Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 943925d1-4f81-4b0c-8197-e482e3e8d8bd · outbound

This paper cites Discovering Preference Optimization Algorithms with and for Large Language Models.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Discovering Preference Optimization Algorithms with and for Large Language Models

Reference 44

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.181972Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 374b2181-0d01-4879-a464-a963c73c8b91 · outbound

This paper cites SciAgent: Tool-augmented Language Models for Scientific Reasoning.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts SciAgent: Tool-augmented Language Models for Scientific Reasoning

Reference 45

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.184324Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ce8cc2a6-f7fa-485e-af96-2020ce6cf0ce · outbound

This paper cites ChemCrow: Augmenting large-language models with chemistry tools.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts ChemCrow: Augmenting large-language models with chemistry tools

Reference 46

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.186866Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b46ed719-5073-413b-b47a-0747638c7318 · outbound

This paper cites Scientific Large Language Models: A Survey on Biological & Chemical Domains.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Scientific Large Language Models: A Survey on Biological & Chemical Domains

Reference 47

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.189430Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9a49e1d6-cb3f-4d06-a864-f12545dc0181 · outbound

This paper cites AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.191545Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d7a33066-7114-4595-95a8-d5e6ea53ad7a · outbound

This paper cites Chip Placement with Deep Reinforcement Learning.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Chip Placement with Deep Reinforcement Learning

Reference 49

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.194082Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bd8aacd5-e317-4e49-aa2b-d1e8e4d17979 · outbound

This paper cites & Aydos, G.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts & Aydos, G

Reference 50

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.603105Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 27a19e35-bd54-415a-8ae2-b056bd5b6fbc · outbound

This paper cites Vivaria: Open-source platform for agent evaluations (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Vivaria: Open-source platform for agent evaluations (2024)

Reference 51

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.596263Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 2a51f86b-eb88-4539-b1a0-ff870c907679 · outbound

This paper cites Claude 3.5 Sonnet model card addendum (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Claude 3.5 Sonnet model card addendum (2024)

Reference 52

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.589134Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation b537316d-67d4-46f4-a304-82e7eff5addb · outbound

This paper cites OpenAI o1 system card (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts OpenAI o1 system card (2024)

Reference 53

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.580503Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation b7390dc8-00a2-4757-98b3-64235c222ea9 · outbound

This paper cites AIDE: Data science automation technical report (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts AIDE: Data science automation technical report (2024)

Reference 54

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.573410Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 7693bafe-b60a-40ed-85b6-4579cff93977 · outbound

This paper cites Details about METR’s preliminary evaluation of OpenAI o1-preview (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Details about METR’s preliminary evaluation of OpenAI o1-preview (2024)

Reference 55

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.565835Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation a4dbaa28-eb60-414f-96d8-590c9bce99bc · outbound

This paper cites Claude 3.5 Sonnet: Quality, performance & price analysis (2024).

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Claude 3.5 Sonnet: Quality, performance & price analysis (2024)

Reference 56

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T14:33:13.558707Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 4008e77c-c644-4c3c-a78f-76a5be9c7e79 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 57

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.552270Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 02f7ff6e-504e-4a93-bc58-097c5a6ada3c · outbound

This paper cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Direct Preference Optimization: Your Language Model is Secretly a Reward Model

Reference 58

Resolution: unresolved; no resolver link, observed 2026-08-12T14:33:13.215554Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b023ccb1-9588-4ffa-bd1e-97c18c884f76 · outbound

This paper cites Is this score similar to what you predicted or measured yourself, or does it come as a surprise?.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Is this score similar to what you predicted or measured yourself, or does it come as a surprise?

Reference 59

Resolution: malformed identifier; raw_fallback, observed 2026-08-12T14:33:13.545394Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 46c3d2b7-a08a-4178-868e-fe1ab97cb6d9 · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 60

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.536710Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 0f6e2c1e-2e63-4d88-9548-82a0c955aa1f · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 61

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.529316Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 6231b3b9-ee6a-4a25-baf1-34cc040b76dd · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 62

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.521984Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 9a6f4ce8-f8af-4924-a545-1e699ff2598a · outbound

This paper cites an unresolved cited work.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts Unresolved cited work

Reference 63

Resolution: unresolved; raw_fallback, observed 2026-08-12T14:33:13.514722Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation f6629272-2716-425d-b21e-14db69ae21ed · outbound

This paper cites cuda" "cuda.

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts cuda" "cuda

Reference 64

Resolution: malformed identifier; raw_fallback, observed 2026-08-12T14:33:13.507078Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Pith citing papers

Observation 4b89afad-6c15-4cd6-a734-fdf2480c86bc · inbound

Frontier Models are Capable of In-context Scheming cites this paper.

Frontier Models are Capable of In-context Scheming RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 35

Resolution: verified exact; arxiv_id, observed 2026-05-16T14:22:01.681340Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation d9fa4ac5-63e7-45de-986b-c9ba862334fb · inbound

Humanity's Last Exam cites this paper.

Humanity's Last Exam RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 60

Resolution: verified exact; arxiv_id, observed 2026-05-10T18:40:50.326504Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 7dddfdc1-9de0-4ea1-b0b6-ecb02f46e6d0 · inbound

The AI Agent Index cites this paper.

The AI Agent Index RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 84

Resolution: unresolved; no resolver link, observed 2026-08-09T14:48:35.872296Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0db8ffb1-e533-489d-b388-a0a9ea0c6fec · inbound

KernelBench: Can LLMs Write Efficient GPU Kernels? cites this paper.

KernelBench: Can LLMs Write Efficient GPU Kernels? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 42

Resolution: verified exact; arxiv_id, observed 2026-05-15T16:55:02.090533Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 915b7f2d-f33e-4ae4-9db6-045f0a38f059 · inbound

AI Behind Closed Doors: a Primer on The Governance of Internal Deployment cites this paper.

AI Behind Closed Doors: a Primer on The Governance of Internal Deployment RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-16T12:40:14.612983Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 18990c90-06f6-4e82-967c-1f3b67587aff · inbound

Bare Minimum Mitigations for Autonomous AI Development cites this paper.

Bare Minimum Mitigations for Autonomous AI Development RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-16T11:30:47.179402Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7f9a1d21-dc25-471a-8e49-39fd2441209b · inbound

Can AI Agents Design and Implement Drug Discovery Pipelines? cites this paper.

Can AI Agents Design and Implement Drug Discovery Pipelines? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-16T05:44:03.801916Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 19dcc8fb-763b-47e7-8ec9-771e5de241fe · inbound

AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions cites this paper.

AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 195

Resolution: unresolved; no resolver link, observed 2026-08-15T23:27:30.618243Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 35d19ee8-5d53-4ede-b2c2-bd9c4fc69316 · inbound

LLMs Outperform Experts on Challenging Biology Benchmarks cites this paper.

LLMs Outperform Experts on Challenging Biology Benchmarks RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-15T22:50:49.704789Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 79ec15b4-8781-468b-8d3e-beeea998a9e1 · inbound

TimeSeriesGym: A Scalable Benchmark for (Time Series) Machine Learning Engineering Agents cites this paper.

TimeSeriesGym: A Scalable Benchmark for (Time Series) Machine Learning Engineering Agents RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-15T20:18:59.964481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e5b67da1-0357-4675-8eb8-5547e5883c99 · inbound

Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks cites this paper.

Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 63

Resolution: unresolved; no resolver link, observed 2026-08-07T12:18:55.881071Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 603c9afb-9026-40fe-9f5a-dc6febb693fb · inbound

TextAtari: 100K Frames Game Playing with Language Agents cites this paper.

TextAtari: 100K Frames Game Playing with Language Agents RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 79

Resolution: unresolved; no resolver link, observed 2026-08-07T10:51:58.052862Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f650702d-9842-4d40-99ca-3e58f8cf9c10 · inbound

Deep Research Agents: A Systematic Examination And Roadmap cites this paper.

Deep Research Agents: A Systematic Examination And Roadmap RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 118

Resolution: unresolved; no resolver link, observed 2026-08-06T23:26:56.952422Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 36100b82-0a6a-453d-8fd8-7252d4bb2915 · inbound

Establishing Best Practices for Building Rigorous Agentic Benchmarks cites this paper.

Establishing Best Practices for Building Rigorous Agentic Benchmarks RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 79

Resolution: unresolved; no resolver link, observed 2026-08-06T20:24:27.421486Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b3799e82-49f1-4e93-b917-0b42741574ce · inbound

Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework cites this paper.

Evaluating the Critical Risks of Amazon's Nova Premier under the Frontier Model Safety Framework RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-06T19:39:59.279843Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a58694cb-4e26-4f00-876c-ee10cd3eb27b · inbound

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities cites this paper.

Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 89

Resolution: verified exact; arxiv_id, observed 2026-05-19T05:52:07.806483Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 9e58febf-03fe-4d5f-8648-635514eec14a · inbound

AblationBench: Evaluating Automated Planning of Ablations in Empirical AI Research cites this paper.

AblationBench: Evaluating Automated Planning of Ablations in Empirical AI Research RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 2022

Resolution: unresolved; no resolver link, observed 2026-08-06T19:00:08.585286Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 09cd1878-e82b-4063-bde5-4ff58152aeb2 · inbound

Exploring Design of Multi-Agent LLM Dialogues for Research Ideation cites this paper.

Exploring Design of Multi-Agent LLM Dialogues for Research Ideation RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-06T18:24:26.153839Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 203e8758-23fb-496e-bb2c-9753f4b9bdb4 · inbound

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity cites this paper.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-06T18:12:29.939412Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7eca4a44-634e-4288-828e-ad2fcd68d077 · inbound

Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety cites this paper.

Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 29

Resolution: metadata mismatch; arxiv_id, observed 2026-05-20T14:19:44.776446Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation f89e799f-9104-420c-9c93-7bfaa6b22443 · inbound

Scheming Ability in LLM-to-LLM Strategic Interactions cites this paper.

Scheming Ability in LLM-to-LLM Strategic Interactions RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 11

Resolution: verified exact; arxiv_id, observed 2026-05-18T07:51:03.835431Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 76b13409-8d2b-4366-afea-6a06c93a1a20 · inbound

EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies cites this paper.

EcoGym: Evaluating LLMs for Long-Horizon Plan-and-Execute in Interactive Economies RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 38

Resolution: verified exact; arxiv_id, observed 2026-05-16T05:57:24.639150Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 9126949c-9665-45a0-b476-7436e91deb07 · inbound

SPM-Bench: Benchmarking Large Language Models for Scanning Probe Microscopy cites this paper.

SPM-Bench: Benchmarking Large Language Models for Scanning Probe Microscopy RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-02T20:36:06.369914Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b030444d-0020-438e-a874-0edb4351c374 · inbound

Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimization cites this paper.

Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimization RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 24

Resolution: verified exact; arxiv_id, observed 2026-05-11T09:05:57.739286Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 8c7a22b9-3c70-4efa-97fd-886a14846ce8 · inbound

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration cites this paper.

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 48

Resolution: verified exact; arxiv_id, observed 2026-05-11T11:46:34.653244Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation c7d436cc-01f1-4907-92f6-2883a4edf091 · inbound

Risk Reporting for Developers' Internal AI Model Use cites this paper.

Risk Reporting for Developers' Internal AI Model Use RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 48

Resolution: verified exact; arxiv_id, observed 2026-05-11T23:16:16.444053Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 43f82950-7e26-4aa2-994c-4d56fa16925b · inbound

Principles and Guidelines for Randomized Controlled Trials in AI Evaluation cites this paper.

Principles and Guidelines for Randomized Controlled Trials in AI Evaluation RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 34

Resolution: metadata mismatch; arxiv_id, observed 2026-05-09T06:05:36.486105Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation b4e7854c-6681-4446-ae70-14c4cba12e9a · inbound

Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction cites this paper.

Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 21

Resolution: metadata mismatch; arxiv_id, observed 2026-05-15T06:05:06.762989Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 002c6092-2b3e-4304-a6b9-38cc4ca0c284 · inbound

FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale cites this paper.

FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 36

Resolution: verified exact; arxiv_id, observed 2026-05-15T02:03:29.062735Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation f16d3fd5-9469-4b23-8d75-671c0be93582 · inbound

FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics cites this paper.

FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 57

Resolution: verified exact; arxiv_id, observed 2026-05-20T14:28:21.448976Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 98f353b7-e769-4be9-910b-36a33c63fd71 · inbound

FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics cites this paper.

FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 57

Resolution: verified exact; arxiv_id, observed 2026-06-30T19:05:00.924661Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation f9630578-942a-41d2-9260-a9449fdd468e · inbound

AI for Auto-Research: Roadmap & User Guide cites this paper.

AI for Auto-Research: Roadmap & User Guide RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 222

Resolution: verified exact; arxiv_id, observed 2026-05-20T10:33:12.557130Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 5328db6c-78ad-45aa-86de-e3618d706e3f · inbound

AI for Auto-Research: Roadmap & User Guide cites this paper.

AI for Auto-Research: Roadmap & User Guide RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 221

Resolution: unresolved; no resolver link, observed 2026-08-02T13:43:49.951321Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 321ddc94-9675-4172-88d5-3f37ac4c7741 · inbound

How Far Are We From True Auto-Research? cites this paper.

How Far Are We From True Auto-Research? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 26

Resolution: metadata mismatch; arxiv_id, observed 2026-05-20T09:58:11.242622Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 95e6e6dd-7a45-4c86-a98d-45990b1871f9 · inbound

Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale cites this paper.

Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 37

Resolution: verified exact; arxiv_id, observed 2026-05-21T06:59:45.559572Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 23a23acf-0525-40c6-9917-a28501eda992 · inbound

Benchmarks are Not Enough: RAMP for Runtime Assessing of Agentic Models in Production Systems cites this paper.

Benchmarks are Not Enough: RAMP for Runtime Assessing of Agentic Models in Production Systems RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 26

Resolution: verified exact; arxiv_id, observed 2026-06-29T15:33:32.798892Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation abf9f5f6-6c96-483c-8388-2bd6d7e356b7 · inbound

The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development? cites this paper.

The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 1

Resolution: verified exact; arxiv_id, observed 2026-07-02T07:56:47.690735Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 52945ae3-1b6f-4313-b036-adb3f275750d · inbound

AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks? cites this paper.

AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 42

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T07:46:46.619488Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 9668af3d-f1e5-45f1-98eb-3e0245013815 · inbound

InquiTree: Evaluating AI Agents in the Scientific Inquiry Loop with Paper-Derived Research Trees cites this paper.

InquiTree: Evaluating AI Agents in the Scientific Inquiry Loop with Paper-Derived Research Trees RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 9

Resolution: metadata mismatch; arxiv_id, observed 2026-07-03T04:07:37.255049Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation efde55ab-60f9-4aae-8e1d-c230c3c7495a · inbound

Toward Generalist Autonomous Research via Hypothesis-Tree Refinement cites this paper.

Toward Generalist Autonomous Research via Hypothesis-Tree Refinement RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 160

Resolution: verified exact; arxiv_id, observed 2026-06-27T09:40:47.033546Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 4cc5987e-a720-497d-9c48-74c0d0c44ba6 · inbound

Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering cites this paper.

Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 48

Resolution: verified exact; arxiv_id, observed 2026-07-03T22:08:58.899735Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 5d39a32f-a09d-4ee2-a9dc-45b587457b01 · inbound

Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering cites this paper.

Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-02T11:06:03.847203Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4806bec1-2f3f-447a-b62b-400a84e06fa3 · inbound

Learning the ARTS of Search for Automated Discovery cites this paper.

Learning the ARTS of Search for Automated Discovery RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 54

Resolution: verified exact; arxiv_id, observed 2026-07-04T08:09:41.833473Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 1ac7c5ff-0cf2-4e78-b901-f62eaf2d6854 · inbound

Discovering Crystal Structure Prediction Algorithms with an AI Co-Scientist cites this paper.

Discovering Crystal Structure Prediction Algorithms with an AI Co-Scientist RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 20

Resolution: verified exact; arxiv_id, observed 2026-07-04T10:19:46.987416Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation a0aaa883-453e-4868-ac34-d39a298bd0e0 · inbound

MirrorCode: AI can rebuild entire programs from behavior alone cites this paper.

MirrorCode: AI can rebuild entire programs from behavior alone RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 23

Resolution: verified exact; arxiv_id, observed 2026-06-30T06:44:19.703971Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation c974db13-d3e9-4d21-8d11-6794418d9f38 · inbound

MirrorCode: AI can rebuild entire programs from behavior alone cites this paper.

MirrorCode: AI can rebuild entire programs from behavior alone RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-02T09:35:02.886229Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9c361421-912f-4c8e-8c9b-46ec8d5cb72a · inbound

Two AI Metrics Diverged: Will it Make All the Difference? cites this paper.

Two AI Metrics Diverged: Will it Make All the Difference? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 68

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T12:36:56.156858Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 4031574a-1bc3-4e7d-84a4-046c5a8f602e · inbound

RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications cites this paper.

RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 20

Resolution: verified exact; local_arxiv, observed 2026-07-08T07:04:44.520871Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-17T06:30:58.91139+00:00.

Observation 58fcc192-7e31-4da4-944f-af8b18859fb3 · inbound

RuBench: A Repository-Level Agentic Coding Benchmark with Natively Authored Russian Task Specifications cites this paper.

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-02T08:21:50.147998Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1296820b-a97b-4a1e-ab5e-dfdc4571568b · inbound

Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading cites this paper.

Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 44

Resolution: unresolved; no resolver link, observed 2026-07-13T05:28:45.311405Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c4e9e2a6-d05e-4c08-b8cf-89f4e4b97842 · inbound

Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading cites this paper.

Reference 44

Resolution: unresolved; no resolver link, observed 2026-07-14T15:24:58.589243Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4300af11-4056-4bf9-af7a-3b20f2f358d5 · inbound

When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation cites this paper.

When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 37

Resolution: unresolved; no resolver link, observed 2026-07-14T10:46:01.272433Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 45bc81b3-20ed-4ca4-a3a7-bf39ecc86496 · inbound

ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D cites this paper.

ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-01T12:50:00.490138Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4082e15f-b500-40b7-96f1-5feb288df4fa · inbound

Efficiency Matters in Autonomous Research cites this paper.

Efficiency Matters in Autonomous Research RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 18

Resolution: unresolved; no resolver link, observed 2026-07-31T09:14:06.207902Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d6f79981-5a6c-4191-b434-dd3ce80357e7 · inbound

Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation cites this paper.

Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 60

Resolution: unresolved; no resolver link, observed 2026-08-01T01:16:43.437896Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d0bf2f4c-2256-4be0-95f0-6a8f25daeea7 · inbound

Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs cites this paper.

Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 64

Resolution: unresolved; no resolver link, observed 2026-07-31T22:25:22.349139Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c77e66a2-c16c-433a-b39f-7b6101b3ccfc · inbound

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing cites this paper.

To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-03T01:30:17.866119Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a0cc8a4e-0e7d-46b8-a1fd-d9b77f400289 · inbound

Argus: A General-Purpose Agentic Runtime for Long-Horizon Reasoning cites this paper.

Argus: A General-Purpose Agentic Runtime for Long-Horizon Reasoning RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-06T04:16:45.749872Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fd24008f-7d94-46d1-b45c-723447228ba8 · inbound

Predicting Task Difficulty Without Rollouts cites this paper.

Predicting Task Difficulty Without Rollouts RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 1978

Resolution: unresolved; no resolver link, observed 2026-08-07T23:25:20.492888Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c3ee986a-611a-463a-941e-2e856cb77cad · inbound

HarnessOpt-Bench: Evaluating LLMs at Harness Optimization cites this paper.

HarnessOpt-Bench: Evaluating LLMs at Harness Optimization RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-07T06:02:25.759327Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b1c4f2d5-3f9b-4e58-a2ea-052cc44da010 · inbound

QuantumMind: Constraint-Grounded Agentic Reasoning for Speedup Analysis in Quantum Computing cites this paper.

QuantumMind: Constraint-Grounded Agentic Reasoning for Speedup Analysis in Quantum Computing RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-11T00:24:44.676649Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 06f8b810-3b52-4410-a6bc-6f129843f1ff · inbound

Evo-Bench: Can Language Models Improve Agent Harness? cites this paper.

Evo-Bench: Can Language Models Improve Agent Harness? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-11T23:44:24.561432Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 40910431-7f02-4f95-ab99-34b730e701de · inbound

Evo-Bench: Can Language Models Improve Agent Harness? cites this paper.

Evo-Bench: Can Language Models Improve Agent Harness? RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-14T04:22:26.843491Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b631cdce-8a01-4068-a59f-eb8bc494689e · inbound

VALG: An Agentic System for ML Theory Research cites this paper.

VALG: An Agentic System for ML Theory Research RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-15T17:44:14.918108Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.