Pith. sign in

REVIEW 9 cited by

SynCode: LLM Generation with Grammar Augmentation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.01632 v4 pith:SVLX5XHI submitted 2024-03-03 cs.LG cs.FLcs.PLcs.SE

Shubham Ugare , Tarun Suresh , Hangoo Kang , Sasa Misailovic , Gagandeep Singh This is my paper · ORCID
classification cs.LGcs.FLcs.PLcs.SE
keywords syncodegenerationlanguagellmscodeefficientgrammarjson
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

LLMs are widely used in complex AI applications. These applications underscore the need for LLM outputs to adhere to a specific format, for their integration with other components in the systems. Typically the format rules e.g., for data serialization formats such as JSON, YAML, or Code in Programming Language are expressed as context-free grammar (CFG). Due to the hallucinations and unreliability of LLMs, instructing LLMs to adhere to specified syntax becomes an increasingly important challenge. We present SynCode, a novel framework for efficient and general syntactical decoding with LLMs, to address this challenge. SynCode ensures soundness and completeness with respect to the CFG of a formal language, effectively retaining valid tokens while filtering out invalid ones. SynCode uses an offline-constructed, efficient lookup table, the DFA mask store, derived from the DFA of the language's grammar for efficient generation. SynCode seamlessly integrates with any language defined by CFG, as evidenced by experiments focusing on generating JSON, Python, and Go outputs. Our experiments evaluating the effectiveness of SynCode for JSON generation demonstrate that SynCode eliminates all syntax errors and significantly outperforms state-of-the-art baselines. Furthermore, our results underscore how SynCode significantly reduces 96.07% of syntax errors in generated Python and Go code, showcasing its substantial impact on enhancing syntactical precision in LLM generation. Our code is available at https://github.com/uiuc-focal-lab/syncode

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DINGO: Constrained Inference for Diffusion LLMs

    cs.LG 2025-05 conditional novelty 7.0 of 10

    DINGO is a dynamic programming decoder over a token-level DFA that makes diffusion LLM outputs obey regular expressions while maximizing the product of per-position probabilities.

  2. Type-Driven Prompt Programming: From Typed Interfaces to a Calculus of Constraints

    cs.PL 2025-08 conditional novelty 6.0 of 10

    The paper proposes a not-yet-complete dependently typed calculus for prompt programming with probabilistic refinements, and identifies gaps in constraint expressiveness and optimization algorithms.

  3. Syntax Repair as Language Intersection

    cs.FL 2025-07 conditional novelty 6.0 of 10

    Bounded syntax repair is formalized as the intersection of a context-free grammar with an acyclic Levenshtein automaton, and the paper shows this intersection can be built, enumerated, and reranked efficiently.

  4. Memory Offloading for Large Language Model Inference with Latency SLO Guarantees

    cs.DC 2025-02 conditional novelty 6.0 of 10

    Select-N meets LLM latency SLOs by choosing and continuously adjusting an offloading interval that controls how much model state lives in CPU memory, increasing host memory usage and throughput versus prior offloading...

  5. Optimizing Datasets for Code Summarization: Is Code-Comment Coherence Enough?

    cs.SE 2025-02 conditional novelty 6.0 of 10

    Filtering code summarization training data by code-comment coherence gives no better results than random selection, and halving the training set does not hurt performance.

  6. Format Sensitivity Index: Token-Controlled Prompt Wrapper Robustness and Schema Compliance in LLM Benchmarking

    cs.AI 2026-05 conditional novelty 5.5 of 10

    Across 140k generations, wrapper-only formatting changes accuracy by over 30× across models, largely via parseability failures, so single-wrapper scores are statistically fragile.

  7. LLM-as-a-Judge Scores Are Unreliable Optimization Signals in Closed-Loop Table Recognition

    cs.CL 2026-07 conditional novelty 5.0 of 10

    Reference-free LLM judge scores failed to select better table-extraction outputs over eight regeneration iterations on FinTabNet and OmniDocBench; keeping the first output was safest.

  8. Syntactic Control of Language Models by Posterior Inference

    cs.CL 2025-06 conditional novelty 4.0 of 10

    SMC sampling with a tetratagger shaping function and a POS-bigram proposal lifts constituency-tree match from 12-35% to about 93% F1 on GPT2-large and Llama3-8B.

  9. Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A grammar-aligned MCMC sampler with prefix-resampling proposals and Metropolis-Hastings acceptance converges to the constrained language model distribution and improves fuzzing seed coverage.

Pith tools