Paper Citation Record · LEDGER

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code

As of 10 August 2026, this Paper Citation Record lists 27 of 27 outbound references and 0 inbound Pith citation observations for arXiv:2608.00661.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2608.00661 v1

Coverage vector

measured 27 of 27 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-04T01:05:44.229751Z

measured 27 of 27 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-10T06:31:04.303077+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

27 of 27 outbound references displayed

verified exact0
verified fuzzy0
unresolved27
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 15cba260-eddc-4c60-888d-f72311fbdbe8 · outbound

This paper cites https://web.archive.org/web/20250430014530/https://www.cnbc.c om/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft -code-is-written-by-ai.html.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code https://web.archive.org/web/20250430014530/https://www.cnbc.c om/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft -code-is-written-by-ai.html

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:41.675067Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 33e7358e-1098-4ed8-b038-f3a6dac0321a · outbound

This paper cites Microsoft cto breaks down how he sees software developer jobs evolving in the next 5 years,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Microsoft cto breaks down how he sees software developer jobs evolving in the next 5 years,

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:41.751618Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c4267c5e-f440-42dd-982a-10b61c90636d · outbound

This paper cites The impact of ai on developer productivity: Evidence from github copilot,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code The impact of ai on developer productivity: Evidence from github copilot,

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:41.806283Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 63c78d1e-7012-4c52-af23-79978bb34b1c · outbound

This paper cites The effects of github copilot on computing students’ pro- gramming effectiveness, efficiency, and processes in brownfield coding tasks,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code The effects of github copilot on computing students’ pro- gramming effectiveness, efficiency, and processes in brownfield coding tasks,

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:41.894341Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 40e73a8e-6593-433d-90ab-521a6f8a380c · outbound

This paper cites Measuring coding challenge competence with apps,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Measuring coding challenge competence with apps,

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:41.981644Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 02c554f0-847a-4efa-9653-266f8cb0d0eb · outbound

This paper cites Beyond functional correctness: An empirical evaluation of large language models for text- to-code generation,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Beyond functional correctness: An empirical evaluation of large language models for text- to-code generation,

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.076119Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e540e0a8-7a88-4b51-b22f-80d0b6a3667a · outbound

This paper cites Exploring and evaluating hallucinations in llm-powered code generation,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Exploring and evaluating hallucinations in llm-powered code generation,

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.179583Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c79a2b6a-8d52-4197-adf3-520f7031fc04 · outbound

This paper cites Towards understanding the characteristics of code generation errors made by large language models,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Towards understanding the characteristics of code generation errors made by large language models,

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.273701Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d4d30bed-c8c7-4c0c-b68d-37316fe2a135 · outbound

This paper cites An empirical study of code generation errors made by large language models,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code An empirical study of code generation errors made by large language models,

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.375284Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 08904398-f20a-4d56-b733-bd39957c8e55 · outbound

This paper cites Bugs in large language models generated code: An empirical study,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Bugs in large language models generated code: An empirical study,

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.455980Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5c7eb81b-0e6f-4aaa-b5d0-da8c0a08f5f2 · outbound

This paper cites No need to lift a finger anymore? assessing the quality of code generation by chatgpt,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code No need to lift a finger anymore? assessing the quality of code generation by chatgpt,

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.569627Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c89a02d9-deb8-4ece-a8f0-61a8e2ff2d53 · outbound

This paper cites A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.673499Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9c76c0c5-00db-482f-a258-b5f56c21ae81 · outbound

This paper cites Assessing and analyzing the correctness of github copilot’s code suggestions,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Assessing and analyzing the correctness of github copilot’s code suggestions,

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.770880Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 16bb1223-c45c-414d-a8a3-2027d4653bb9 · outbound

This paper cites Gpt-4 technical report,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Gpt-4 technical report,

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:42.868042Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 75978369-e80b-4df0-ab41-b283685d77f5 · outbound

This paper cites DeepSeek-V3 Technical Report.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code DeepSeek-V3 Technical Report

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.031051Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 25b21f5f-1578-42f9-9ef3-0283fbcfdcd1 · outbound

This paper cites DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.179400Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d542d80a-b233-43b3-a295-3b6a23f6cc68 · outbound

This paper cites Qwen2.5-coder technical report,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Qwen2.5-coder technical report,

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.291943Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 59edae2c-ad76-45a4-ac5d-214238c1054a · outbound

This paper cites Incoder: A generative model for code infilling and synthesis,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Incoder: A generative model for code infilling and synthesis,

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.426461Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e87f8893-dafa-4dd2-bbd2-9411519eb854 · outbound

This paper cites Evaluating large language models trained on code,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Evaluating large language models trained on code,

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.563626Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b9dd5a41-8d70-4fd9-a21f-f294b5fd0d4c · outbound

This paper cites A large scale study of programming languages and code quality in github,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code A large scale study of programming languages and code quality in github,

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.649024Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d83f5a41-8d31-4bc2-9944-9ac88e88ac1e · outbound

This paper cites Orthogonal defect classification-a concept for in- process measurements,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Orthogonal defect classification-a concept for in- process measurements,

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.749086Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation eddd760d-3cf2-4af1-929f-f1b3127f923a · outbound

This paper cites Program synthesis with large language models,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Program synthesis with large language models,

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.807074Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9ea8050f-a33b-410e-9c72-78f987353dff · outbound

This paper cites Codexglue: A machine learning benchmark dataset for code understanding and generation,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Codexglue: A machine learning benchmark dataset for code understanding and generation,

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:43.878756Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0f48bb42-a000-4a5c-a932-9d11f02dcaab · outbound

This paper cites PROBE: Benchmarking Code Generation in Large Language Models.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code PROBE: Benchmarking Code Generation in Large Language Models

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:44.021875Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ac0aedb3-fe5c-4fcb-be57-6082c2817cae · outbound

This paper cites Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks,

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:44.092272Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3e0c5cb3-7c1e-488f-8062-83ea873859b7 · outbound

This paper cites The measurement of observer agreement for categorical data,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code The measurement of observer agreement for categorical data,

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:44.132381Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4704d800-7f0b-4228-91b7-d3baf708b730 · outbound

This paper cites Polyglot: An extensible framework to benchmark code translation with llms,.

Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code Polyglot: An extensible framework to benchmark code translation with llms,

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-04T01:05:44.229751Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.