Pith. sign in

REVIEW 3 cited by

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.13382 v2 pith:ZDYSUKMT submitted 2023-12-20 cs.CL cs.AIcs.PL

Arnav Singhvi , Manish Shetty , Shangyin Tan , Christopher Potts , Koushik Sen , Matei Zaharia , Omar Khattab This is my paper · ORCID
classification cs.CLcs.AIcs.PL
keywords assertionsdspyconstraintsmodelprogrammingcomputationallanguagestrategies
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Chaining language model (LM) calls as composable modules is fueling a new way of programming, but ensuring LMs adhere to important constraints requires heuristic "prompt engineering". We introduce LM Assertions, a programming construct for expressing computational constraints that LMs should satisfy. We integrate our constructs into the recent DSPy programming model for LMs, and present new strategies that allow DSPy to compile programs with LM Assertions into more reliable and accurate systems. We also propose strategies to use assertions at inference time for automatic self-refinement with LMs. We report on four diverse case studies for text generation and find that LM Assertions improve not only compliance with imposed rules but also downstream task performance, passing constraints up to 164% more often and generating up to 37% more higher-quality responses. Our reference implementation of LM Assertions is integrated into DSPy at https://github.com/stanfordnlp/dspy

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FoodTaxo: Generating Food Taxonomies with Large Language Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    LLM-based iterative prompting can complete and generate taxonomies from known concepts, competitive on some benchmarks but unreliable for inner-node placement.

  2. Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

    cs.CL 2026-01 conditional novelty 5.0 of 10

    Fine-tuned LLMs can reconstruct item characteristic curves from multiple-choice item text, giving useful estimates of IRT difficulty and discrimination without live student response data.

  3. Data Diversification Methods In Alignment Enhance Math Performance In LLMs

    cs.AI 2025-07 reject novelty 4.0 of 10

    DTS, which generates diverse solution strategies before writing solutions, improves GSM8K by 7.1 points and MATH by 4.2 points over an untuned base model at 1.03x baseline compute.

Pith tools