Pith. sign in

Paper Citation Record · LEDGER

Investigating Test Overfitting on SWE-bench

As of 20 August 2026, this Paper Citation Record lists 23 of 23 outbound references and 2 inbound Pith citation observations for arXiv:2511.16858.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2511.16858 v3

Coverage vector

measured 23 of 23 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-05-17T19:57:06.015270Z

measured 25 of 25 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-20T06:33:59.587034+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-03T17:03:47.442858Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

23 of 23 outbound references displayed

  • verified exact11
  • verified fuzzy3
  • unresolved5
  • parse uncertain0
  • malformed identifier1
  • metadata mismatch3

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 3b508382-d91d-48cf-ae22-92843a0a0d3d · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 1

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.782947Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:9913f4054895046ae5a4407a013a7cc4fb4bc9d0f801ac452a387518110f2a92

Observation 2db7f73a-6e11-468d-b6f2-b729e96e90ae · outbound

This paper cites TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?.

Investigating Test Overfitting on SWE-bench TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?

Reference 2

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.761621Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:3429166344c4c776b1a456045a6fc3a1c3fed8510e5e4ff853ed4239778b1d74

Observation 5825aa92-aecc-452e-a5dc-a24e71cd1151 · outbound

This paper cites MASAI: Modular Architecture for Software-engineering AI Agents.

Investigating Test Overfitting on SWE-bench MASAI: Modular Architecture for Software-engineering AI Agents

Reference 3

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.766675Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:28db54b3b584d82b77772e3fb4ae1ccdd3242cff78fad59f281b68a07f661864

Observation c7ee6d44-dadb-4784-b2e1-0177e4f532c2 · outbound

This paper cites Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation.

Investigating Test Overfitting on SWE-bench Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

Reference 4

Resolution
metadata mismatch
arxiv_id, observed 2026-05-21T07:24:13.179886Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:de08edb92264b65cab513c081de42459ee989a561ed5b71290b8fcdf1035f485

Observation ab4433ae-abd7-4c07-a8a1-2968903a52df · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 5

Resolution
unresolved
raw_fallback, observed 2026-05-17T20:00:11.352875Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:47dd242ad82fa8403eec3eb4b4eff34af2406fe07e7831a6e61d9d416acc1674

Observation 457b3b2a-8ed2-483f-a3be-0c2d89db946e · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 6

Resolution
unresolved
raw_fallback, observed 2026-05-17T20:00:11.347606Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:4e24e5536c85eba4cea19ca29d9dcf835f08ce9fc46937549eb3b906168d8c61

Observation f0af13e9-53dc-4ba2-8189-3d86dcd7c6c9 · outbound

This paper cites Jimenez, John Yang, Kevin Liu, and Aleksander Madry.

Investigating Test Overfitting on SWE-bench Jimenez, John Yang, Kevin Liu, and Aleksander Madry

Reference 7

Resolution
verified fuzzy
raw_fallback, observed 2026-05-17T20:00:11.345076Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:0a753f6b1f003c53c4236289678b582806acefbfae33a71323280dbc7434110a

Observation 8f7f4a26-332f-4616-b31f-afa61ab3c5a9 · outbound

This paper cites CodeMonkeys: Scaling Test-Time Compute for Software Engineering.

Investigating Test Overfitting on SWE-bench CodeMonkeys: Scaling Test-Time Compute for Software Engineering

Reference 8

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.756853Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:d1ab3fcd37e2f0e5f2698c4a9d1016c99ea878a487a0055c2bfe3c1ebe9c0aa5

Observation 0d641755-7e23-4db2-970a-f0b3a7bc36d3 · outbound

This paper cites Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling.

Investigating Test Overfitting on SWE-bench Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling

Reference 9

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.807369Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:b80db4ca854cfb74e3a7e18444081670501fee75fe9cc7b22e8e142a548e97df

Observation 49b41f7a-1411-4e80-9d85-40255ac79a77 · outbound

This paper cites R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents.

Investigating Test Overfitting on SWE-bench R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents

Reference 10

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.787186Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:ea989e649a0f56aa34e0795cc4b47f48b2c7e866ddb4f8b8431b196ce5d1d535

Observation c71ace84-8ace-4c2d-9ba5-334a0b0990c8 · outbound

This paper cites Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan.

Investigating Test Overfitting on SWE-bench Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

Reference 11

Resolution
verified fuzzy
raw_fallback, observed 2026-05-17T20:00:11.339253Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:f05a5d85eec577311927fe6a432304b81107c78d88fa6e12510d3046a4480845

Observation 60105154-838c-4fa0-8809-38d795e310ca · outbound

This paper cites S*: Test Time Scaling for Code Generation.

Investigating Test Overfitting on SWE-bench S*: Test Time Scaling for Code Generation

Reference 12

Resolution
metadata mismatch
arxiv_id, observed 2026-05-17T20:00:10.752156Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:10d348c5a832ce459b7cb4662eae5972cd778bb3569b6a493ba8c3e0300dda30

Observation dc2425f3-b1d9-41f0-875c-2ebd1e2c61c2 · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 13

Resolution
unresolved
raw_fallback, observed 2026-05-17T20:00:11.341779Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:acd9a4a5c2eeeddbcd313b99b68399cf31b7f15c244eaf6c8132e6a8cb6116be

Observation ddc1d6b7-3879-4244-9570-c9fac789efbf · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 14

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.790859Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:38c9b0cc7d9373c49a14f43baa13e409c29f445586c936bca5ccebdfcca9e198

Observation f59e7034-a745-4bbd-8c5f-5a5e1ffc4189 · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 15

Resolution
unresolved
raw_fallback, observed 2026-05-17T20:00:11.350278Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:99375b42a9eac682d76aa3652b6b83d1e96f2fe5c01f0198f1f96eb03d92fb8f

Observation dd28ad85-74f2-4f8f-9adb-a6e4a6667629 · outbound

This paper cites SpecRover: Code Intent Extraction via LLMs.

Investigating Test Overfitting on SWE-bench SpecRover: Code Intent Extraction via LLMs

Reference 16

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.779241Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:deb72f50baeed490f711c49d63b3af510fd42288dfa93abd52813727c98a846d

Observation 2a50201e-08b4-465e-8b74-0127fa79453b · outbound

This paper cites Smith, Earl T.

Investigating Test Overfitting on SWE-bench Smith, Earl T

Reference 17

Resolution
metadata mismatch
arxiv_id, observed 2026-05-17T20:00:10.770775Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:44306421c72059425e7f783d6d18102eccbb95c014f850f1daa49d84cab6e54c

Observation 1d9ce753-8f06-4d73-b0fc-fad999d06723 · outbound

This paper cites Inference scaling f laws: The limits of llm resampling with imperfect verifiers.

Investigating Test Overfitting on SWE-bench Inference scaling f laws: The limits of llm resampling with imperfect verifiers

Reference 18

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.795573Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:e42ed80b3dfd7cf39252ba6ee41f77be5ebdf555511409c902b0f1594bcb8fd4

Observation 79fe9f2d-ff56-409a-8e58-c97f93e648d9 · outbound

This paper cites Co-evolving llm coder and unit tester via reinforcement learning.arXiv preprint arXiv:2506.03136.

Investigating Test Overfitting on SWE-bench Co-evolving llm coder and unit tester via reinforcement learning.arXiv preprint arXiv:2506.03136

Reference 19

Resolution
verified exact
arxiv_id, observed 2026-05-17T20:00:10.799641Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:fd1c4a78c5db7f6f1da31552b2946a942fdac36359dddf4d60488b9d874b80d5

Observation 009fbc01-4729-439a-8d17-94510f052d20 · outbound

This paper cites Demystifying llm-based software engineering agents.

Investigating Test Overfitting on SWE-bench Demystifying llm-based software engineering agents

Reference 20

Resolution
verified exact
doi, observed 2026-05-17T20:00:10.697613Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:6006a115f50c98ac17b50dc7a97a632051cf8ee00217ffe0c3ac6f3272d4773e

Observation 2bb9f469-b4b1-48bc-a1b0-b5b8d884b19c · outbound

This paper cites Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press.

Investigating Test Overfitting on SWE-bench Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

Reference 21

Resolution
verified fuzzy
raw_fallback, observed 2026-05-17T20:00:11.355638Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:5829e6832b48336e7bb324850b2645da67bb80bacb66ed79afbbf704b256c94d

Observation fbaf9ccb-f29a-4cf9-92d0-ce8ba8cb5636 · outbound

This paper cites an unresolved cited work.

Investigating Test Overfitting on SWE-bench Unresolved cited work

Reference 22

Resolution
unresolved
raw_fallback, observed 2026-05-17T20:00:11.357905Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:bb2707864cb4183cb10611b3eff1b9b6c33a1d639ee3be72fff5051d44a728aa

Observation 762e8cf3-448b-4dcf-9361-bd91c121a3e9 · outbound

This paper cites Refactoring Runaway.

Investigating Test Overfitting on SWE-bench Refactoring Runaway

Reference 23

Resolution
malformed identifier
arxiv_id, observed 2026-05-17T20:00:10.803500Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-20T06:33:59.587034+00:00.

source=pdf_text observed=2026-05-17T19:57:06.015270Z digest=sha256:ba1a6ddd7a05c3b49a201951a2309a832ccff9879bfa007f703804106c0c84aa

Pith citing papers

Observation 39aa7983-2c4a-4a50-8930-a2c75f031ac8 · inbound

Know Before Fix: QA-Driven Repository Knowledge Acquisition for Software Issue Resolution cites this paper.

Know Before Fix: QA-Driven Repository Knowledge Acquisition for Software Issue Resolution Investigating Test Overfitting on SWE-bench

Reference 38

Resolution
unresolved
no resolver link, observed 2026-07-14T06:59:44.621968Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-07-14T06:59:44.621968Z digest=sha256:db2bf497407cfbf2f33f1774cd4b1a845fb90b6432c4b1c9fd049df9297d2ee8

Observation d5eb9daa-19f0-41e1-8ea6-02c1dc52a707 · inbound

Automated Testing and Repair for Verified Compilers Generated by a Coding Agent cites this paper.

Automated Testing and Repair for Verified Compilers Generated by a Coding Agent Investigating Test Overfitting on SWE-bench

Reference 19

Resolution
unresolved
no resolver link, observed 2026-08-03T17:03:47.442858Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-03T17:03:47.442858Z digest=sha256:4a79ed2919eaaf9e9edc1ecbb4ab2132715d11f8088b3df3725f627f65511881