Pith. sign in

Paper Citation Record · LEDGER

Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

As of 10 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 44 inbound Pith citation observations for arXiv:2312.04724.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2312.04724 v1

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 44 of 44 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-10T06:31:04.303077+00:00

measured 44 of 44 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-09T13:45:55.122362Z

measured 1 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Reference resolution

0 of 0 outbound references displayed

  • verified exact0
  • verified fuzzy0
  • unresolved0
  • parse uncertain0
  • malformed identifier0
  • metadata mismatch0

External citation measurements

18
arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation d71ed716-aa49-42ae-a6c4-491f44c60ad0 · inbound

StarCoder 2 and The Stack v2: The Next Generation cites this paper.

StarCoder 2 and The Stack v2: The Next Generation Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 164

Resolution
metadata mismatch
arxiv_id, observed 2026-05-12T17:28:22.675474Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=arxiv_source observed=2026-05-12T17:28:22.353355Z digest=sha256:7cb247a7056131f9c43cb5ed2972941672afaf8de9ee58dd5c4544eed30be573

Observation 4299a779-96e4-4419-9105-def9b99382a0 · inbound

Precision or Peril: A PoC of Python Code Quality from Quantized Large Language Models cites this paper.

Precision or Peril: A PoC of Python Code Quality from Quantized Large Language Models Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 16

Resolution
verified exact
arxiv_id, observed 2026-05-23T17:33:14.955539Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-23T17:30:54.204300Z digest=sha256:a2e83cbcf81494548aa3bce776f900f534fe942a948b6e0e03f72d399e0fd280

Observation f303afb9-da19-4460-9313-d870f3e455bd · inbound

Humanity's Last Exam cites this paper.

Humanity's Last Exam Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 10

Resolution
verified exact
arxiv_id, observed 2026-05-10T18:40:50.368040Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-10T18:40:50.139345Z digest=sha256:ae11003edd3fa3b1ed214ade1a33dae9556261908792f895500f05db019d2d68

Observation 301c7b7c-519f-472b-9145-8a07f717f1db · inbound

LLMSecConfig: An LLM-Based Approach for Fixing Software Container Misconfigurations cites this paper.

LLMSecConfig: An LLM-Based Approach for Fixing Software Container Misconfigurations Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 10

Resolution
unresolved
no resolver link, observed 2026-08-09T13:45:55.122362Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-09T13:45:55.122362Z digest=sha256:048c850dfed74013029360be8993add0e5377706b05f905a940307ee0b021c35

Observation 3e89f275-13dc-49a2-8358-52d8b34ab118 · inbound

Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models cites this paper.

Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-08T17:00:30.464049Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-08T17:00:30.464049Z digest=sha256:531b5976e7c107b5139c2da2f372d738183a7eae48f2a124d1f559c2a0b239de

Observation 259fba11-9ee5-4183-a195-facc5c07f03b · inbound

Training Language Models to Generate Quality Code with Program Analysis Feedback cites this paper.

Training Language Models to Generate Quality Code with Program Analysis Feedback Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-07T13:04:58.231629Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-07T13:04:58.231629Z digest=sha256:e0755663e759390452b6dfad932a5e8cbd7673317f4df99a94df71b804030071

Observation 6f606523-bef9-46e5-b411-10bf0bb1bb23 · inbound

MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment cites this paper.

MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 9

Resolution
unresolved
no resolver link, observed 2026-08-07T12:46:10.267103Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-07T12:46:10.267103Z digest=sha256:ce633f15478e5d2d6946fbeac3bec2a74f57d244324849c7581653986b0c478a

Observation 2a740e82-3223-4dfb-9470-ccff15569fbc · inbound

Developing a Risk Identification Framework for Foundation Model Uses cites this paper.

Developing a Risk Identification Framework for Foundation Model Uses Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 7

Resolution
unresolved
no resolver link, observed 2026-08-07T11:52:57.988743Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-07T11:52:57.988743Z digest=sha256:c7c28001d94eca2ef0572cbfc5a82798e055ae86c37eccebd0b274e3e6d45a14

Observation 9e5f17a6-a27d-47b9-b7eb-835e73519e26 · inbound

SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflows cites this paper.

SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflows Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 6

Resolution
unresolved
no resolver link, observed 2026-08-07T05:42:54.911928Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-07T05:42:54.911928Z digest=sha256:dbc97e8820d43b926c04926d21708a08c2bf3345339c0df2d40a3f02dc91a66b

Observation 78188daa-cfd8-411e-b556-0bf09ef0c4f5 · inbound

The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs cites this paper.

The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 57

Resolution
unresolved
no resolver link, observed 2026-08-07T10:17:26.763796Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-07T10:17:26.763796Z digest=sha256:d31e0588dde42f85412f9f71fa86342d93874491c80b941054d521068b809fa8

Observation 381c30dd-e45d-4868-af24-85cbb8ecd9f9 · inbound

Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation cites this paper.

Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 8

Resolution
unresolved
no resolver link, observed 2026-08-06T21:56:13.509418Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-06T21:56:13.509418Z digest=sha256:077df013760ff14c603e511c66d2f34b67b22a5f5c339d26f6a3ccdabadafa24

Observation cf6837e0-a1cf-456c-b41d-df3c068a6907 · inbound

MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation cites this paper.

MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-06T20:45:53.247909Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-06T20:45:53.247909Z digest=sha256:315b04cbad76ac9a17adbdb8c8efea996e83e2ec24f2921ca56e0109e3a54ed4

Observation 97794215-ada6-49f4-9f3e-40a02bb0b665 · inbound

Understanding the Supply Chain and Risks of Large Language Model Applications cites this paper.

Understanding the Supply Chain and Risks of Large Language Model Applications Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 2024

Resolution
unresolved
no resolver link, observed 2026-08-06T14:45:42.847281Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-06T14:45:42.847281Z digest=sha256:856ca80d0b2ffd9503be6135a9406cf5cbd37785e4d4d58f3a4b85a56396d50e

Observation 5dfe7ef9-3eb8-4fa5-8515-d47d7e71faec · inbound

Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security cites this paper.

Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-06T14:25:15.757169Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T14:25:15.757169Z digest=sha256:0033bcd3caf35fc59cffa5d78d420977dd72a1bc96f071d6528e14b5a896cb2a

Observation 24a861b7-a4e2-4ef7-8c4e-c1211673c999 · inbound

Towards terahertz nanomechanics cites this paper.

Towards terahertz nanomechanics Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-06T01:03:49.483750Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-06T01:03:49.483750Z digest=sha256:4d22379b80a32067f16a4504b57d149ec06b18a94d7eb9372e94253a92a1d1ee

Observation 75bcb548-44e5-45d4-8578-254e6d951780 · inbound

ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants cites this paper.

ASTRA: Autonomous Spatial-Temporal Red-teaming for AI Software Assistants Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
unresolved
no resolver link, observed 2026-08-06T01:04:32.435944Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=arxiv_source observed=2026-08-06T01:04:32.435944Z digest=sha256:ce5a4fde4f27a1211492063ec2e3199c6ce654f941a625c43e5f1ee5da7612fa

Observation 1f80b99b-ddea-4efa-bee6-62a473fe1d3f · inbound

Secure Code Generation at Scale with Reflexion cites this paper.

Secure Code Generation at Scale with Reflexion Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 11

Resolution
unresolved
no resolver link, observed 2026-08-03T23:50:49.056328Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-03T23:50:49.056328Z digest=sha256:2041b367dd26b93a5bc23081f6d80aca2155d35ddad032fe5f8449f797b276ed

Observation 375b5ec1-aada-443f-8522-c7944673692a · inbound

BEAVER: An Efficient Deterministic LLM Verifier cites this paper.

BEAVER: An Efficient Deterministic LLM Verifier Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 7

Resolution
metadata mismatch
arxiv_id, observed 2026-05-17T01:58:51.237959Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-17T01:58:44.719715Z digest=sha256:faba9b8bc5b21ff1afd134d9a9f299f29be496c60f9fe68f9b598d49e78a62ad

Observation 12076545-9314-47ab-8e36-96d721f8ed1e · inbound

Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software cites this paper.

Extracting Recurring Vulnerabilities from Black-Box LLM-Generated Software Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 2

Resolution
unresolved
no resolver link, observed 2026-08-03T05:18:42.953661Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-03T05:18:42.953661Z digest=sha256:77a9c4210ec9aa4ea92b7e2021003c7267ece5edbd07aedd3c95780e7ac9e42d

Observation 1c6d0de2-b199-4885-869e-e2348c008998 · inbound

"Tab, Tab, Bug": Security Pitfalls of Next Edit Suggestions in AI-Integrated IDEs cites this paper.

"Tab, Tab, Bug": Security Pitfalls of Next Edit Suggestions in AI-Integrated IDEs Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
metadata mismatch
arxiv_id, observed 2026-05-16T06:57:29.297748Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-16T06:53:13.235588Z digest=sha256:47287881364c0d6760e10edde2e4382827c6cd68bc7d53155496ba54ae0fee99

Observation 94d15de5-8ac2-4d03-9fd8-6d3157135cbb · inbound

Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code cites this paper.

Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 11

Resolution
metadata mismatch
arxiv_id, observed 2026-05-10T22:10:48.566036Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-10T20:12:03.118074Z digest=sha256:3e949a127b1cad585f975196c4a53576dc804a171329eb4ebe3d63aa1274a90e

Observation bebf90b3-ee10-42c9-9deb-daf54a51f640 · inbound

Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types cites this paper.

Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
metadata mismatch
arxiv_id, observed 2026-05-11T07:31:00.338647Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-10T17:08:25.471462Z digest=sha256:a7e24595b90593d51c0eb4a3f41983c8df38a617751662758588187afccabc17

Observation 7fbbb1e4-aa8e-49f4-b162-b551afa5aa3e · inbound

Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition cites this paper.

Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 52

Resolution
verified exact
arxiv_id, observed 2026-05-10T11:10:09.437088Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=arxiv_source observed=2026-05-10T04:55:43.987116Z digest=sha256:4ef3dff6164682d5254d19289105649d07aa590adfdf8ac6caad96c9f2157824

Observation 2c5d9f52-8c89-4d6e-9660-35e553710ce1 · inbound

Towards Optimal Agentic Architectures for Offensive Security Tasks cites this paper.

Towards Optimal Agentic Architectures for Offensive Security Tasks Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 1

Resolution
metadata mismatch
arxiv_id, observed 2026-05-11T12:16:03.146927Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-10T04:02:04.359269Z digest=sha256:b6ef1edd04e28ddfb83840dfb1037d371994e4d397dbb50b0280293a6b3c4f7b

Observation 3eda2f9c-237c-4b36-803e-1bdc8981c0d5 · inbound

A Validated Prompt Bank for Malicious Code Generation: Separating Executable Weapons from Security Knowledge in 1,554 Consensus-Labeled Prompts cites this paper.

A Validated Prompt Bank for Malicious Code Generation: Separating Executable Weapons from Security Knowledge in 1,554 Consensus-Labeled Prompts Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 47

Resolution
metadata mismatch
arxiv_id, observed 2026-05-09T06:40:43.797249Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-08T18:11:29.066362Z digest=sha256:07a8faec718a61c6a83a313e0c81889d3ee086b71ae737cad912fe727fd51041

Observation 89af28ed-bcf5-40d3-8611-5630b43ad2ef · inbound

Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours cites this paper.

Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 7

Resolution
metadata mismatch
arxiv_id, observed 2026-05-11T23:51:44.776533Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-07T16:06:18.057868Z digest=sha256:a3f7f4b050941e7ed6974bdc7bd400a6eaefd856afa46c4a8b4e42397b04b85c

Observation 15bad8b2-2d34-44a3-928e-b4a531ea6e94 · inbound

Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code cites this paper.

Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 10

Resolution
verified exact
arxiv_id, observed 2026-05-11T17:26:04.477648Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-08T17:37:51.790000Z digest=sha256:be07d98ba8f83e124ff3ec71b92ee96db8a8b1f9e611576eacb963a2d146a86b

Observation 593fda4e-e88b-4287-a655-07e4b936771f · inbound

SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization cites this paper.

SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 10

Resolution
verified exact
arxiv_id, observed 2026-05-12T08:26:24.588493Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-12T01:09:08.378040Z digest=sha256:86b29e57e50fd504200e93029041d925fe5aa770fcd116c2b88b950d314ae0c0

Observation d2708aee-a774-4683-95be-2aeb7caa45d7 · inbound

LLM-Agnostic Semantic Representation Attack cites this paper.

LLM-Agnostic Semantic Representation Attack Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 76

Resolution
metadata mismatch
arxiv_id, observed 2026-05-12T08:21:24.222423Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-12T01:14:08.629862Z digest=sha256:8593b02b6947f8157747e3d3cb4c6b50f1bee6485cc1656f76182730b5eeb07c

Observation de0b055b-e787-4943-bf8c-d0ea299bb904 · inbound

Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications cites this paper.

Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 7

Resolution
verified exact
arxiv_id, observed 2026-05-19T23:32:52.541609Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-19T23:30:43.364230Z digest=sha256:0d63dfa41b5c8d41a20a75d61127703cbb820a4d60e5412b772eada34bb5619b

Observation e9453341-ea23-434c-9bd0-857ce0aca0b2 · inbound

Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025) cites this paper.

Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025) Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 2

Resolution
metadata mismatch
arxiv_id, observed 2026-05-21T07:39:48.536733Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-21T07:38:43.363709Z digest=sha256:6d7be02a4b5eadd4ebe6f47f145d849666a2e015cdc288845792b8478e8a09e8

Observation 014b4ae9-6471-42c6-9e79-be13ffc29e50 · inbound

HIDBench: Benchmarking Large Language Models for Host-Based Intrusion Detection cites this paper.

HIDBench: Benchmarking Large Language Models for Host-Based Intrusion Detection Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 3

Resolution
metadata mismatch
arxiv_id, observed 2026-05-22T08:54:45.868682Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-22T08:52:34.079804Z digest=sha256:60fe0a8726b6f9c9c4828b41666b76f0e6c790be2d5aa185200c76edab086b8c

Observation 07a96601-9bbd-4e36-bdfd-14c9a7bae55b · inbound

Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety cites this paper.

Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 6

Resolution
verified exact
arxiv_id, observed 2026-05-22T05:51:07.706475Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-22T05:50:28.114140Z digest=sha256:db0d785ff88c4d8b869f9c52eff9c9d13dce4d0fa79c7146b441a13cd5e6f0ad

Observation 50142792-7fa7-41bc-bfa7-d3d3e189d660 · inbound

Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety cites this paper.

Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 6

Resolution
verified exact
arxiv_id, observed 2026-05-25T06:06:43.062405Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-25T06:05:27.736494Z digest=sha256:53b4962420db1f5e9830e77505ff3f4e143b6fc4207ccf28db67b5621e953c2d

Observation 8497cf12-ab6e-4c82-9ac5-566da15afb6a · inbound

Security of LLM-generated Code: A Comparative Analysis cites this paper.

Security of LLM-generated Code: A Comparative Analysis Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 15

Resolution
verified exact
arxiv_id, observed 2026-05-25T05:16:39.185029Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-05-25T05:16:26.372764Z digest=sha256:2d13eec9baa45b49f0d35d1066f4a288b48e1566fc226e539cc7386bf73a1bba

Observation 0e47e741-ab83-4a5a-aa3c-4c83b18a6e4d · inbound

SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces cites this paper.

SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 25

Resolution
verified exact
arxiv_id, observed 2026-07-01T21:36:15.066149Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=arxiv_source observed=2026-06-28T16:44:18.994680Z digest=sha256:d9a466a2faecab1e62419d6ba4059f4d4d85a99e6a9ca3ad0aa2734ca7504c66

Observation e7a4f50f-7229-44a5-a12a-25664d9e796a · inbound

Helpful or Harmful? Evaluating LLM-Assisted Vulnerability Patching via a Human Study cites this paper.

Helpful or Harmful? Evaluating LLM-Assisted Vulnerability Patching via a Human Study Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 4

Resolution
metadata mismatch
arxiv_id, observed 2026-07-04T21:10:09.288469Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-06-25T19:02:45.109478Z digest=sha256:a2add03c41ef9ac47eed5df8867d935b32d4304226001bf286471240977a814d

Observation 9d048307-d19b-422f-a380-eb09b9fd25ee · inbound

Direct Causation in International Humanitarian Law and the Challenge of AI-Mediated Civilian Cyber Operations cites this paper.

Direct Causation in International Humanitarian Law and the Challenge of AI-Mediated Civilian Cyber Operations Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 1

Resolution
metadata mismatch
arxiv_id, observed 2026-06-30T07:54:22.125211Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-10T06:31:04.303077+00:00.

source=pdf_text observed=2026-06-30T07:49:28.819402Z digest=sha256:c7a448e3d4d9e73cd9d404c208c66fbc6c02edfefebed3617c3b26bb68faafd2

Observation f2caab62-2233-4b1d-a14f-600eca1074a7 · inbound

Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale cites this paper.

Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 2

Resolution
unresolved
no resolver link, observed 2026-07-12T07:33:43.015966Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-07-12T07:33:43.015966Z digest=sha256:4ae6f8437a0e9c0ccf9adfcebfd3da2c61e8201329bcea316cd133d203d5e8bc

Observation 0d6179b5-8f42-4710-af49-7b22b97a1710 · inbound

Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents cites this paper.

Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 9

Resolution
unresolved
no resolver link, observed 2026-07-11T22:40:37.839133Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-07-11T22:40:37.839133Z digest=sha256:f64495d052cc8ce3cdb7112586c2a26797b08bd08c7343e99a74b7899879273b

Observation c4b6b2a4-983a-46fb-a945-a3a224793c59 · inbound

Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents cites this paper.

Refused in Chat, Written in Code: Workflow-Level Jailbreak Construction in IDE Coding Agents Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 9

Resolution
unresolved
no resolver link, observed 2026-07-13T07:01:49.222325Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-07-13T07:01:49.222325Z digest=sha256:6da1e355aa59ac0615704ab21d906b5175d0e4d0819109b915db3f5a118e1f9a

Observation 237e36ec-cebd-45cf-9c43-8a1d2b584881 · inbound

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response cites this paper.

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 24

Resolution
unresolved
no resolver link, observed 2026-08-01T02:40:29.099157Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-01T02:40:29.099157Z digest=sha256:7331a5ca7b651c31ae1d6869cf757ac56751e2f56767ad1b6858e1c687b463e7

Observation c2eb78e9-4e54-4e10-a6a0-07da0442523b · inbound

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response cites this paper.

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 25

Resolution
unresolved
no resolver link, observed 2026-08-04T01:33:17.643282Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-04T01:33:17.643282Z digest=sha256:1e1af3b0ed97792654e339426e5b7e2f9b24000926335c0de5bb57b11aa5445d

Observation 7cdc2bd7-d8d6-4bb1-8996-da6ea68649a7 · inbound

Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic) cites this paper.

Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic) Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

Reference 9

Resolution
unresolved
no resolver link, observed 2026-08-08T19:58:17.193944Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

source=pdf_text observed=2026-08-08T19:58:17.193944Z digest=sha256:2323f7845cfe3d40dff70d0ce5813792cc06dd0e06488939637338a8e0c06f7b