Paper Citation Record · LEDGER

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models

As of 13 August 2026, this Paper Citation Record lists 44 of 44 outbound references and 2 inbound Pith citation observations for arXiv:2411.18019.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2411.18019 v1

Coverage vector

measured 44 of 44 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-12T11:39:33.837756Z

measured 46 of 46 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-13T06:32:02.005865+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-02T16:27:43.489582Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-11T08:26:01.477245Z

Reference resolution

44 of 44 outbound references displayed

verified exact0
verified fuzzy1
unresolved43
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 09da401d-680c-41cb-9c88-4677228200fa · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.556342Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 218b4b6e-83fb-454b-90ab-4f796936a9cd · outbound

This paper cites Program Synthesis with Large Language Models.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Program Synthesis with Large Language Models

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.561080Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ff833103-d0a0-4aef-90a7-92706b999a49 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 3

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.330935Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 7c550b34-48bc-4c13-90f6-7edb3122a433 · outbound

This paper cites Evaluating Large Language Models Trained on Code.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Evaluating Large Language Models Trained on Code

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.569353Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c385cfc2-44c4-4d5b-aa5d-3efc4dc463ad · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.573403Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f6e0fc25-e88b-492b-a1cc-8fcd40f42907 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 6

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.312866Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation d36b637b-25cd-40f7-a11c-f28f7f5f6516 · outbound

This paper cites Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.581251Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0e32bc69-8746-42ce-84c1-3048914e325c · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 8

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.301704Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 8993847b-e09b-4c02-a348-b535ba8a88ff · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.588766Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3c90e457-6e5d-4901-8970-c3ae95b5f194 · outbound

This paper cites CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.593222Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b66db029-2a51-42cc-9860-3601c0dda63f · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.712047Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 603ebf6a-a91d-4a0e-8569-ee18c40581ee · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 12

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.270351Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation f9094f8b-fd9f-4cbd-b51f-8f7bd58ae0bc · outbound

This paper cites Measuring Coding Challenge Competence With APPS.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Measuring Coding Challenge Competence With APPS

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.723218Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3d4846ff-6186-436e-8bcb-25942094c996 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 14

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.257902Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation f91bd4fa-8d70-4197-8546-c6aa0eb04328 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.730593Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 138d75c1-91f8-4f0e-93b1-dfaa283e39ab · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 16

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.237292Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 4e0c54d3-92cd-429a-bede-39ee02206601 · outbound

This paper cites SWE-bench: Can Language Models Resolve Real-World GitHub Issues?.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.742992Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3cb517ee-ad6b-48d5-a850-253c1869d346 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.746615Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2841977a-9a7e-455b-b836-2c4e3972167a · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 19

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.216995Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 7794c252-cdcd-4f02-9bc9-8412265b1c88 · outbound

This paper cites CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.753114Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 69294333-50c2-4ef3-b636-1bf4817087f7 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 21

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.204800Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation b4c27d15-aa5c-44db-8fb6-0b7301f71394 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 22

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.192800Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 33acb935-17d1-4061-8383-46031e2b2343 · outbound

This paper cites CodeQA: A Question Answering Dataset for Source Code Comprehension.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models CodeQA: A Question Answering Dataset for Source Code Comprehension

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.762383Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b18bc12a-fdfd-4b29-9623-ea71967496e3 · outbound

This paper cites Large Language Model-Based Agents for Software Engineering: A Survey.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Large Language Model-Based Agents for Software Engineering: A Survey

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.766173Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 31f7ff52-f071-4e90-a145-3178924804fd · outbound

This paper cites MarsCode Agent: AI-native Automated Bug Fixing.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models MarsCode Agent: AI-native Automated Bug Fixing

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.769898Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b47db412-079e-494d-a814-8a89fa9c97d6 · outbound

This paper cites Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.773884Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 66788ad5-5064-4a54-b1d9-5af72ea5cec7 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.778162Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 671f294d-ae94-453c-a163-5aeb427b7e48 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 28

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.159665Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation aa3ec18d-c639-4023-9d2c-99dc5375fb90 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.790435Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1fdc7f2c-5d5b-4eb1-9228-2795fcff54a8 · outbound

This paper cites In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V

Reference 30

Resolution: verified fuzzy; raw_fallback, observed 2026-08-12T11:39:34.172802Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 0133477d-ebf4-49d4-b6a6-4910dcec5908 · outbound

This paper cites StarCoder: may the source be with you!.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models StarCoder: may the source be with you!

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.797995Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 27c3fe17-275b-4b4e-880c-c764f2c3589b · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.802325Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f09e517f-a269-43da-a9fa-f52a74cc2dbd · outbound

This paper cites AgentFL: Scaling LLM-based Fault Localization to Project-Level Context.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models AgentFL: Scaling LLM-based Fault Localization to Project-Level Context

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.793973Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 446a64cd-193c-4482-a7cc-e40d4b44be73 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 34

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.118431Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 2f37515f-e2c9-4d98-9dee-2af068903482 · outbound

This paper cites MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.814213Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 45ce0934-704d-4f33-9a3a-323af91f0e8d · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 36

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.130653Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation 496d94a6-a0c0-4c82-a9a9-25c5ece1129e · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.821566Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 51e64f52-0f11-41cc-b9b2-2539d36e359f · outbound

This paper cites Agentless: Demystifying LLM-based Software Engineering Agents.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Agentless: Demystifying LLM-based Software Engineering Agents

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.825804Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a524a3c0-1beb-4cf6-ac80-750ab6937de7 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 39

Resolution: unresolved; raw_fallback, observed 2026-08-12T11:39:34.106655Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation b1235a58-3ac0-4cf2-bb7c-6467c8b9fea3 · outbound

This paper cites an unresolved cited work.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Unresolved cited work

Reference 40

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.834103Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 00923c45-29b0-42ac-85a4-417bf0225f83 · outbound

This paper cites AutoCodeRover: Autonomous Program Improvement.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models AutoCodeRover: Autonomous Program Improvement

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.837756Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1be7cee2-c66c-424e-95eb-752600efba8c · outbound

This paper cites SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.829678Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5c952517-67fa-4ecd-a0d6-1a4c14c51d23 · outbound

This paper cites Large Language Models for Software Engineering: A Systematic Literature Review.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models Large Language Models for Software Engineering: A Systematic Literature Review

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.734856Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 24de3a1a-f3f7-46ff-8bf9-9a7ea1c3be3b · outbound

This paper cites In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering.

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-12T11:39:33.715985Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 399dd694-103d-440f-80a1-257581758b6a · inbound

Evaluating LLM Agents on Automated Software Analysis Tasks cites this paper.

Evaluating LLM Agents on Automated Software Analysis Tasks A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models

Reference 28

Resolution: verified exact; arxiv_id, observed 2026-05-11T08:26:01.479107Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-13T06:32:02.005865+00:00.

Observation f9bd4156-3400-4ce0-b5f5-ffdad8f9f162 · inbound

Evaluating LLM Agents on Automated Software Analysis Tasks cites this paper.

Evaluating LLM Agents on Automated Software Analysis Tasks A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-02T16:27:43.489582Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.