REVIEW 6 cited by
Constrained Decoding for Secure Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Code Large Language Models (Code LLMs) have been increasingly used by developers to boost productivity, but they often generate vulnerable code. Thus, there is an urgent need to ensure that code generated by Code LLMs is correct and secure. Previous research has primarily focused on generating secure code, overlooking the fact that secure code also needs to be correct. This oversight can lead to a false sense of security. Currently, the community lacks a method to measure actual progress in this area, and we need solutions that address both security and correctness of code generation. This paper introduces a new benchmark, CodeGuard+, along with two new metrics, to measure Code LLMs' ability to generate both secure and correct code. Using our new evaluation methods, we show that the state-of-the-art defense technique, prefix tuning, may not be as strong as previously believed, since it generates secure code but sacrifices functional correctness. We also demonstrate that different decoding methods significantly affect the security of Code LLMs. Furthermore, we explore a new defense direction: constrained decoding for secure code generation. We propose new constrained decoding techniques to generate secure code. Our results reveal that constrained decoding is more effective than prefix tuning to improve the security of Code LLMs, without requiring a specialized training dataset. Moreover, our evaluations over eight state-of-the-art Code LLMs show that constrained decoding has strong performance to improve the security of Code LLMs, and our technique outperforms GPT-4.
Forward citations
Cited by 6 Pith papers
-
HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
A 924-task benchmark reveals that LLMs generate functionally correct but security-incomplete hardware code, and that prompting with security hints substantially improves security pass rates.
-
Training Language Models to Generate Quality Code with Program Analysis Feedback
REAL trains LLMs with rewards from program analysis and unit tests, improving joint functionality and code quality over prior methods.
-
Functional and Secure Code Generation with Task Vectors
LPO-derived Secure-Anchored task vectors raise simultaneous functional-and-secure code rates by 2.1–36 pp on six coding LLMs with near-zero inference overhead.
-
SCGAgent: Recreating the Benefits of Reasoning Models for Secure Code Generation with Agentic Workflows
An agentic workflow with security guidelines and LLM-generated unit tests improves secure code generation on CWEval C tasks from 61% to 76% Func-Sec@1 with Sonnet-3.7, at roughly 98% of its original functionality.
-
CoGate: Confidence-Gated Co-Decoding for Secure Code Generation
Gating a security expert by its own confidence (max-prob or entropy) fixes inverted acceptance in co-decoding and improves joint functional-security code generation, especially on OOD CWEs.
-
Secure Code Generation at Scale with Reflexion
Three-round reflexion prompting raises LLM-generated code security from 70.74% to 79.43% as measured by the ICD static detector on the Instruct Prime benchmark.
Discussion (0). Continue with ORCID to comment.