SPRI: Aligning Large Language Models with Context-Situated Principles

Hongli Zhan; Junyi Jessy Li; Mikhail Yurochkin; Muneeza Azmat; Raya Horesh

REVIEW 3 major objections 5 minor 1 cited by

SPRI: Aligning Large Language Models with Context-Situated Principles

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SPRI generates per-query principles that rival expert-written alignment guidance.

desk verdict A real increment in per-query alignment principles, but the TruthfulQA headline claim is contradicted by the paper's own Table 4 and needs qualification before this is ready. read the letter →

arxiv 2502.03397 v2 pith:AIL6UC65 submitted 2025-02-05 cs.CL cs.AI

Hongli Zhan , Muneeza Azmat , Raya Horesh , Junyi Jessy Li , Mikhail Yurochkin This is my paper · ORCID

classification cs.CLcs.AI

keywords LLMalignmentprinciple-basedconstitutionalAILLM-as-a-judgecognitivereappraisalsyntheticdatagenerationcritique-refineTruthfulQA

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SPRI, a framework that replaces generic, hand-written alignment principles with automatically generated ones tailored to each individual input query. Its central claim is that these context-situated principles can achieve performance on par with expert-crafted principles, even for a task as complex as psychologically grounded emotional support, while requiring minimal or no human effort. The paper demonstrates this claim in three settings: cognitive reappraisal, where SPRI matches oracle constitution guidance; LLM-as-a-judge, where SPRI-generated rubrics beat prior instance-agnostic frameworks; and synthetic data generation, where fine-tuning on SPRI outputs substantially improves TruthfulQA. If the claim holds, alignment can be scaled to domains that currently depend on costly human expertise.

What carries the argument

The central mechanism is the two-stage iterative critique-refine loop with a threshold-based stopping rule. Stage I synthesizes context-situated principles K for the user input T and refines them until the critic model assigns a score of at least 4 on a 1-5 Likert scale; Stage II generates and refines a response R against those principles until the same critic threshold is met, with a maximum of four iterations in each stage. The same principles serve as the evaluation criteria in Stage II, making the framework self-contained apart from optional seed examples. This loop is what replaces human-authored constitutions.

What would settle it

A direct test is to sample the queries where SPRI's critic assigns a score of 4 or higher and have human experts independently rate the same principles and responses. If the critic's 'good enough' judgments frequently disagree with expert judgments—for example, marking vague or generic principles as sufficient—the claimed equivalence to oracle guidance would not survive.

Watch

Extended reading notes

Core claim

SPRI's central discovery is that a two-stage critique-refine loop lets a base language model and a separate critic model jointly produce and use principles that are specific to each user query. In Stage I the base model generates initial principles for the query, the critic scores and critiques them, and the base model refines them until the critic gives a score of at least 4 or four iterations pass. In Stage II the same loop is repeated for the response, with the finalized principles serving as the rubrics for the critic's scoring. The paper shows that this procedure produces reappraisal responses that match oracle-guided outputs, evaluation rubrics whose correlation with human labels outperforms generic rubrics, and SFT data that yields large gains on truthfulness benchmarks.

Load-bearing premise

The entire method rests on the critic model's 1-to-5 score being a trustworthy measure of whether the generated principles are genuinely useful and the response genuinely adheres to them; if that score is miscalibrated or rewards superficial traits, the loop stops early on weak guidance.

Editorial extensions

If this is right

In domains where expert-written principles exist, SPRI can substitute for them with comparable measured alignment and empathy, reducing the need for contracted specialists.
SPRI-generated instance-specific rubrics improve LLM-as-a-judge correlation with human ratings over generic and domain-level rubrics, with a gap to oracle rubrics of only 0.07 Pearson correlation.
Fine-tuning base models on SPRI-generated synthetic data raises TruthfulQA scores by an average of 24.76% over off-the-shelf models and 19.09% over oracle-response fine-tuned models.
The critic model does not need to be larger than the base model, so the framework is usable with smaller open-weights models.
SPRI requires fewer model calls than self-refine on complex tasks while performing better, so the per-query overhead is modest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to apply SPRI to non-textual or multimodal inputs, where per-instance principles could guide vision-language responses.
The framework's dependence on the critic's 1-to-5 score suggests an upper bound: if the critic cannot detect a flawed principle, the loop stops on a weak guide; calibrating the critic to human judgments of principle quality would likely strengthen the method.
SPRI could be used as a data-generation engine for RLHF preference pairs, where the generated principles provide a per-query rationale for which response is better.
One testable prediction is that SPRI's advantage grows as instances become more heterogeneous, because generic principles degrade exactly where per-query tailoring matters most.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea here is worth your time: SPRI generates context-specific principles for each input on the fly, using a critic model to iteratively refine both the principles and the response. That per-query combination is genuinely new relative to Constitutional AI, ConstitutionalExperts, and ConstitutionMaker, and the paper evaluates it across three different use cases with real care. The cognitive reappraisal results (Table 1) are the strongest evidence: SPRI closes most of the gap to oracle psychological constitutions using only one seed example, and the ablation in Table 3 shows that seeds matter but even generic seeds beat no seeds. The BiGGen Bench rubric generation results (Table 2) are also convincing; SPRI beats the MT-Bench and FLASK baselines consistently and gets within 0.07 Pearson correlation of human-curated rubrics. The cost tables and the released code are a plus, and the paper is honest about stopping criteria and iteration counts.

The soft spot is the truthfulness claim, and it is not minor. Section 5.3 says SPRI 'consistently outperforms the off-the-shelf model' on TruthfulQA, but Table 4 shows the opposite for two of the three instruction-tuned models: Mistral-7B-v0.3-Instruct drops from 66.11% off-the-shelf to 56.43% (Dolly) and 57.99% (MixInstruct), and Gemma-2-9B-it also regresses on MixInstruct (59.75% vs 60.47%). The abstract's 'substantial improvement on truthfulness' is only true for base models. This is an internal contradiction, not a disagreement with external consensus. The fix is straightforward: qualify the claim to base models, or report per-model rather than blanket averages. The absence of confidence intervals across all three result tables also makes it hard to know which differences are real, and the reliance on Prometheus-2's 1–5 scores as a stopping target is never calibrated against human judgments of principle quality. On the reappraisal seed side, using one oracle RESORT constitution as the seed and then evaluating with RESORT-based criteria does create some circularity, though the paper discloses it and the ablation partially addresses it.

Who should read this: anyone working on scalable alignment, LLM-as-a-judge, or synthetic SFT data. The reappraisal and rubric experiments are solid enough to deserve referee time. My recommendation: send it to peer review, but require revision that fixes the TruthfulQA claim, adds variance estimates, and discusses critic calibration. The framework is not groundbreaking, but it is a clean, reproducible step forward.

Referee Report

3 major / 5 minor

Summary. The paper introduces SPRI, a two-stage framework in which a base LLM generates context-specific guiding principles for each user query, a critic model scores and refines those principles until a threshold is met, and the refined principles then guide response generation, which is also critic-refined. SPRI is evaluated on three tasks: cognitive reappraisal against expert-written RESORT constitutions (§4.1), instance-specific rubric generation on BiGGen Bench (§4.2), and synthetic SFT data for TruthfulQA and related benchmarks (§5). The paper claims on-par performance with expert-crafted principles in the first two tasks and substantial truthfulness gains from SPRI-generated SFT data, with code and model generations released.

Significance. The three task settings are well chosen and cover generation, evaluation, and data synthesis, and the released code plus the cost analysis in Appendix B are valuable. The ablation over seed conditions in Table 3 is a strength because it lets the reader see how much the oracle seed contributes. If fully supported, SPRI would be a practical contribution to alignment and LLM-as-a-judge research. However, the claim in §5.3 is internally contradicted by Table 4 for two instruction-tuned models, and the §4.1 result relies on an expert seed drawn from the same taxonomy used as the evaluation metric. These issues must be resolved before the broad conclusions in the abstract and conclusion can be accepted.

major comments (3)

[§4.1, Table 1, Table 3] The main reappraisal experiment is partly circular and not robust to its own ablation. §4.1 'SPRI Method' states that SPRI is given a single oracle RESORT constitution as a seed example, while 'Evaluation & Criteria' uses 'Alignment with Reappraisal Constitutions' as the key metric. The seed and the yardstick thus come from the same expert-knowledge family. Table 3 shows that removing the seed ('-seed=[none]') lowers GPT-4o-mini alignment to 7.67, below the vanilla baseline of 7.90, so the sentence 'SPRI consistently outperforms methods that lack access to oracle principles' is not robust to the ablation. In addition, no significance test against the oracle condition is reported, so the headline 'on-par with oracle guidance' is not statistically established. Please report the no-seed condition as primary, or justify why one expert seed is within the 'minimal human effort' budget, and add explicit non-inferiority or equivalence tests against the oracle condition.
[§5.3, Table 4] The blanket truthfulness claim is contradicted by the paper's own table. §5.3 states that 'SPRI consistently outperforms the off-the-shelf model as well as other synthetic response generation methods on the TruthfulQA dataset,' but Table 4 shows that Mistral-7B-v0.3-Instruct scores 66.11% off-the-shelf, while SPRI fine-tuned on Dolly scores 56.43% and on MixInstruct scores 57.99%. Gemma-2-9B-it also regresses on MixInstruct (59.75% vs. 60.47%). The abstract's 'substantial improvement on truthfulness' and the conclusion's 'improves synthetic data generation' are therefore internally inconsistent with the reported numbers. The claim should be restricted to base models, where the gains are consistent, or the post-trained regressions should be explained.
[§3, Eqs. (2) and (5)] The stopping rule in both stages is a critic score of at least 4 on a 1-5 Likert scale, but no calibration evidence is provided that Prometheus-2's scores track human judgments of principle quality or response-principle alignment. If the critic rewards superficial properties such as length or generic reassuring phrasing, SPRI could stop early on weak principles or over-refine on strong ones, and the apparent parity with oracle conditions in §4.1 and §4.2 could be an artifact of the critic. Please include a small human validation of the critic's scores on generated principles and responses, or an explicit sensitivity analysis to the threshold τ.

minor comments (5)

[§5.1] The dataset definition has a typographical error: D = {(p1, r1), ..., (pn, rn} should be D = {(p1, r1), ..., (pn, rn)}.
[Algorithm 1] The instruction 'Reset M, C' is not defined; please state whether this clears the dialogue history of the base and critic models between iterations.
[§4.1] The phrase 'SPRI consistently outperforms methods that lack access to oracle principles' should be qualified as 'SPRI with an oracle seed', since the no-seed ablation in Table 3 shows a regression for GPT-4o-mini.
[Table 1] The column headers 'Scale of10' and 'Scale of5' are missing spaces and should be 'Scale of 10' and 'Scale of 5'.
[Appendix B] The cost tables report 'Total Cost' without stating the unit or the number of responses over which the cost is averaged; please clarify the unit in the captions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: SPRI's pipeline is algorithmic and self-contained, the seed/evaluation overlaps are ablated rather than definitional, and the TruthfulQA contradiction is a correctness issue, not a circularity.

full rationale

SPRI's derivation chain is algorithmic rather than self-referential: Stage I generates principles K0 from the input plus a seed and iteratively critiques/refines them, and Stage II generates responses guided by Kfinal and refines them against Kfinal. There is no equation in which a predicted quantity is defined as its own input. The closest overlap is in Section 4.1, where SPRI is given 'a single oracle RESORT constitution as the seed example' and evaluated on 'Alignment with Reappraisal Constitutions' defined by the same RESORT framework. This is a legitimate leakage concern, but it is not circular by construction: the paper never sets Kfinal equal to the seed, and its own ablation in Table 3 shows that no-seed SPRI still outperforms vanilla and self-refine on most reappraisal cells, so the headline result is not forced by the seed. Similarly, Section 4.2 seeds SPRI with three gold BiGGen Bench rubrics per capability but evaluates on disjoint instances; that is few-shot in-context learning from the benchmark's format, not fitting to the evaluated labels. The SFT experiments in Section 5 train on Dolly/MixInstruct and evaluate on TruthfulQA, so the truthfulness gain is not fitted to the benchmark. The main defensible concern in the manuscript is empirical rather than circular: Section 5.3's blanket statement that 'SPRI consistently outperforms the off-the-shelf model as well as other synthetic response generation methods on the TruthfulQA dataset' is contradicted by Table 4 for Mistral-7B-v0.3-Instruct (66.11% off-the-shelf vs 56.43% and 57.99% after SPRI SFT) and for Gemma-2-9B-it on MixInstruct (59.75% vs 60.47%). That is an internal-consistency or correctness problem, not a circularity. No circular step is present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No invented entities. The framework introduces no new theoretical constructs. The free parameters are standard hyperparameters, not fitted to the target results. The main assumptions are about the validity of LLM-generated principles and critic scores.

free parameters (3)

score_threshold_tau = 4
Stopping threshold for principle and response critique loops; chosen by hand, not fitted.
max_iterations_nmax = 4
Maximum number of refinement rounds in both stages; chosen by hand.
seed_examples_per_task = 1 (reappraisal), 3 (BiGGen), 6 (SFT)
Number of seed (instruction, principle) pairs supplied to the base model; task-specific and hand-chosen.

assumptions (3)

domain assumption Generated principles can effectively steer LLM responses for complex tasks.
The whole SPRI pipeline assumes that a short set of principles written for one query will improve response quality; the experiments support this but do not prove it generally.
domain assumption Prometheus-2-8x7B provides valid numerical scores (1-5) for principle usefulness and response-principle adherence.
The loop stops at score >= 4; no calibration against human judgments of principle quality is provided (Section 3, Eqs. 2 and 5).
domain assumption The automatic evaluator in Section 4.1 (from Zhan et al. 2024) correlates with professional psychologist ratings.
The paper relies on this cited correlation to claim on-par performance with expert principles; it is not re-validated here.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SPRI: Aligning Large Language Models with Context-Situated Principles." pith.science (2026). https://pith.science/paper/AIL6UC65

@misc{pith2026250203397,
  author       = {Pith},
  title        = {Pith review of: SPRI: Aligning Large Language Models with Context-Situated Principles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AIL6UC65}},
  note         = {Machine review of arXiv:2502.03397}
}

read the original abstract

Aligning Large Language Models to integrate and reflect human values, especially for tasks that demand intricate human oversight, is arduous since it is resource-intensive and time-consuming to depend on human expertise for context-specific guidance. Prior work has utilized predefined sets of rules or principles to steer the behavior of models (Bai et al., 2022; Sun et al., 2023). However, these principles tend to be generic, making it challenging to adapt them to each individual input query or context. In this work, we present Situated-PRInciples (SPRI), a framework requiring minimal or no human effort that is designed to automatically generate guiding principles in real-time for each input query and utilize them to align each response. We evaluate SPRI on three tasks, and show that 1) SPRI can derive principles in a complex domain-specific task that leads to on-par performance as expert-crafted ones; 2) SPRI-generated principles lead to instance-specific rubrics that outperform prior LLM-as-a-judge frameworks; 3) using SPRI to generate synthetic SFT data leads to substantial improvement on truthfulness. We release our code and model generations at https://github.com/honglizhan/SPRI-public.

Figures

Figures reproduced from arXiv: 2502.03397 by the authors.

**Figure 1.** Using SPRI, GPT-4o-mini can generate situated and detailed principles to guide the response to a person narrating in distress. Compared with generic rules (Bai et al., 2022b) and human-expert-crafted principles (Zhan et al., 2024), SPRI requires minimal to no human efforts yet produces context-specific guidance for every query at hand. achieve the desired level of performance. For example, Zhan et al. (2024) showed… view at source ↗

**Figure 2.** Overview for SPRI, which consists of two stages: 1) producing a set of principles specifically tailored to the user’s input T, and 2) utilizing the generated principles to guide the response to T. Both stages include a critique-refine process involving a separate critic model, which aims to scrutinize the fitness of the principles to T and the final responses’ adherence to the generated principles. et al. (2022b) em… view at source ↗

**Figure 3.** The 6 default seed principles used in the SPRI framework. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Enhancing Scientific Visual Question Answering through Multimodal Reasoning and Ensemble Modeling
cs.CV 2025-07 conditional novelty 4.0 of 10

On the SciVQA 2025 benchmark, an InternVL3 model with optimized prompts and chain-of-thought instructions reaches ROUGE-1 and ROUGE-L F1 of 0.740, and a figure-type-aware ensemble ranks 5th.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · cited by 1 Pith paper

[5]

Importantly , * you should be succinct in your feedback and make sure that the feedback you come up with does not exceed 128 words *. ### Instruction to Evaluate : { Fill in Pprinciple-gen here } [ Question : { orig_question }] ### Principles to Evaluate : { orig_principle } ### Score Rubrics : On a scale of 1 to 5 , to what extent are the principles usef...

work page
[6]

Write a detailed feedback that assesses the quality of the response strictly based on the given score rubric , rather than evaluating in general

work page
[7]

You should refer to the score rubric

After writing the feedback , write a score that is an integer between 1 and 5. You should refer to the score rubric

work page
[8]

Feedback : ( write a feedback based on the evaluation criteria ) [ RESULT ] ( an integer number between 1 and 5)

The output format should look as follows : " Feedback : ( write a feedback based on the evaluation criteria ) [ RESULT ] ( an integer number between 1 and 5) "

work page
[9]

Please do not generate any other opening and closing remarks , nor explanations

work page
[10]

get over

Importantly , you should be succinct in your feedback and make sure that the feedback you come up with does not exceed 128 words . ### Instruction to Evaluate : You are an expert at following instructions . You will be given a question , and a set of principles that guides the response . You need to generate a response to the question that adheres closely...

work page 2024
[11]

Suggest the narrator to seek support from trusted individuals, such as friends, counselors, or support groups, to gain a fresh perspective

work page
[12]

Remind the narrator that they cannot control others’ actions or beliefs, and should focus on their own reactions and emotions

work page

Show all 12 references

[13]

Validate the narrator’s feelings and experiences, and assure them that they are not alone in their struggles
[14]

Congratulations!

Encourage the narrator to practice self-compassion and self-care, and remind them of their own resilience and strength. – Critique of the Principles (using Prometheus-2-8x7B; Iter 1): The response effectively encourages the narrator to prioritize their mental health, which is ...
[15]

Japanese Village

Examples from Dolly (i) • User input: Give me ideas of ways to add a spicy kick to my pho. • Oracle Response: You can spice up a bowl of pho by adding some sriracha sauce, fresh jalapenos, pickled jalapenos, or chili oil. • SPRI-Generated Principles, Iter 1 : When suggesting w...
[16]

global warming

Examples from MixInstruct (i) • User input: Is it true that if you don’t exercise your body will become weaker? • Oracle Response: Yes, it is true that if you don’t exercise your body will become weaker. Regular exercise helps to build and maintain muscle strength, bone densit...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [5]

Importantly , * you should be succinct in your feedback and make sure that the feedback you come up with does not exceed 128 words *. ### Instruction to Evaluate : { Fill in Pprinciple-gen here } [ Question : { orig_question }] ### Principles to Evaluate : { orig_principle } ### Score Rubrics : On a scale of 1 to 5 , to what extent are the principles usef...

work page

[2] [6]

Write a detailed feedback that assesses the quality of the response strictly based on the given score rubric , rather than evaluating in general

work page

[3] [7]

You should refer to the score rubric

After writing the feedback , write a score that is an integer between 1 and 5. You should refer to the score rubric

work page

[4] [8]

Feedback : ( write a feedback based on the evaluation criteria ) [ RESULT ] ( an integer number between 1 and 5)

The output format should look as follows : " Feedback : ( write a feedback based on the evaluation criteria ) [ RESULT ] ( an integer number between 1 and 5) "

work page

[5] [9]

Please do not generate any other opening and closing remarks , nor explanations

work page

[6] [10]

get over

Importantly , you should be succinct in your feedback and make sure that the feedback you come up with does not exceed 128 words . ### Instruction to Evaluate : You are an expert at following instructions . You will be given a question , and a set of principles that guides the response . You need to generate a response to the question that adheres closely...

work page 2024

[7] [11]

Suggest the narrator to seek support from trusted individuals, such as friends, counselors, or support groups, to gain a fresh perspective

work page

[8] [12]

Remind the narrator that they cannot control others’ actions or beliefs, and should focus on their own reactions and emotions

work page