REVIEW 9 cited by
CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite the significant progress made in code generation with large language models, challenges persist, especially with hardware description languages such as Verilog. This paper first presents an analysis of fine-tuned LLMs on Verilog coding, with synthetic data from prior methods. We identify two main issues: difficulties in handling non-textual representations (Karnaugh maps, state-transition diagrams and waveforms) and significant variability during training with models randomly making "minor" mistakes. To address these limitations, we enhance data curation by creating correct-by-construction data targeting non-textual representations. Additionally, we introduce an automated framework that generates error reports from various model checkpoints and injects these errors into open-source code to create targeted code repair data. Our fine-tuned Starcoder2-15B outperforms prior state-of-the-art results by 3.8%, 10.9%, 6.6% for pass@1 on VerilogEval-Machine, VerilogEval-Human, and RTLLM.
Forward citations
Cited by 9 Pith papers
-
A Progressive Approach to Synthesizable RTL Design Generation Using LLMs
VeriRefine boosts LLM-generated RTL correctness to 94.0% on RTLLM v2.0 and 98.1% on VerilogEval-Human v2 by refining and auditing a per-signal intermediate representation before code generation.
-
VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning
VERIRL applies reinforcement learning with a trace-back rescore mechanism and sample-balanced weighting to achieve state-of-the-art Verilog code generation.
-
RealBench: Benchmarking Verilog Generation Models with Real-World IP Designs
RealBench measures LLM Verilog generation on complex open-source IP cores with formal verification, and all tested models score near zero on full system designs.
-
Spec2RTL-Agent: Automated Hardware Code Generation from Complex Specifications Using LLM Agent Systems
On three NIST crypto standards (AES, DSS, HMAC), Spec2RTL-Agent generates RTL via a multi-agent pipeline from pseudocode to Python to synthesizable C++, reporting 3/3 correct designs with about 4.3 human interventions...
-
ScaleRTL: Scaling LLMs with Reasoning Data and Test-Time Compute for Accurate RTL Code Generation
A reasoning LLM fine-tuned on 3.5B tokens of RTL chain-of-thought data, with iterative test-time self-correction, sets new state-of-the-art results on VerilogEval and RTLLM.
-
hdl2v: A Code Translation Dataset for Enhanced LLM Verilog Generation
A new dataset of VHDL, Chisel, and PyMTL3 code paired with compiled Verilog improves LLM Verilog generation by up to 23% on VerilogEvalV2 after fine-tuning.
-
DecoRTL: A Run-time Decoding Framework for RTL Code Generation with LLMs
DecoRTL combines token-class-aware temperature adjustment with contrastive top-K reranking to improve synthesizability and functional correctness of LLM-generated Verilog.
-
VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generation
A new pipeline and dataset of 20,392 synthesis-checked Verilog modules for LLM fine-tuning is presented, claimed to be the largest high-quality dataset of its kind.
-
QiMeng: Fully Automated Hardware and Software Design for Processor Chip
QiMeng is a proposed three-layer architecture for automating processor hardware and software design, with several published components but no integrated implementation yet.
Discussion (0). Sign in to comment.