Pith. sign in

REVIEW 3 major objections 6 minor 47 references

Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Representation theorems in decision theory can turn an AI model's own responses into complete, label-free rationality checks, with continuous penalties that vanish exactly when behavior is rationalizable.

desk verdict Sound decision-theory math with genuinely new lemmas, but the LLM story needs a caveat about response stochasticity before it fully works. read the letter →

arxiv 2608.05015 v1 pith:WDJDRMH4 submitted 2026-08-05 econ.TH cs.AIcs.LG

Isaiah Andrews This is my paper
classification econ.THcs.AIcs.LG MSC 91B0691B1691B08
keywords rationalityrepresentationtheoremsrevealedpreferencelabel-freeevaluationlargelanguagemodelsdeFinetticoherenceGARPsubjectiveexpectedutility
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that classic if-and-only-if representation theorems in decision theory can serve as label-free evaluation and regularization tools for large language models. Because these theorems guarantee that axiom compliance is equivalent to rationalizability by a well-defined objective, checking a model's own responses to synthetic choice problems exhaustively tests the relevant rationality standard. The paper develops three instantiations: de Finetti coherence, Afriat's revealed-preference test, and an Echenique-Saito test for subjective expected utility, each yielding a continuous penalty that is zero exactly when behavior is rationalizable. A sympathetic reader would care because this offers a principled way to measure and penalize irrationality without ground-truth labels, and because passing the checks rules out any further rationality-based rejection on the same elicited data.

What carries the argument

The machinery is the representation theorem itself: a finite set of axioms that is necessary and sufficient for behavior to be rationalizable by a well-defined objective. The paper uses three: de Finetti's coherence theorem, which equates avoiding sure loss with extendability to a finitely additive probability measure, checked by a linear program whose optimal value is the Dutch-book magnitude $L(p)$; Afriat's theorem, which equates the Generalized Axiom of Revealed Preference with the existence of a continuous, monotone, concave utility rationalizing budget-set choices, measured by $1-\mathrm{CCEI}$; and the Echenique-Saito strong axiom of revealed subjective expected utility, which equates a price-product restriction on state-contingent choices with subjective expected utility maximization, measured by $1-E$ where $E$ is the largest $e$ such that the data satisfy $e$-SARSEU. These penalties are zero exactly when the corresponding rationalization exists.

What would settle it

Take a model that always reports the true probability $p(E)$ for each event in a partition, then query it repeatedly under temperature $>0$; if the de Finetti linear program is run on a single sampled response set, the Dutch-book magnitude will typically exceed zero even though the underlying reports are coherent, showing the check falsely rejects a rational model under sampling noise.

Watch

Extended reading notes

Core claim

The central claim is that the bidirectional structure of representation theorems supplies a complete, label-free rationality check for AI systems. If a model's responses satisfy the axioms, then a rationalizing objective — a probability measure, a utility function, or a belief-utility pair — is guaranteed to exist; if the axioms fail, no such objective exists. The paper shows that each check is a continuous, polynomial-time-computable penalty: the maximum guaranteed profit of a Dutch book for de Finetti coherence, the complement of the Critical Cost Efficiency Index for Afriat's theorem, and a new continuous index $1-E$ for the SARSEU condition of Echenique and Saito, computed through a linear program. Passing all axioms means the model cannot be rejected on rationality grounds by any further test of the same data.

Load-bearing premise

The paper assumes a model's responses to the same choice problem form a fixed, deterministic dataset belonging to one decision-maker; in practice, sampling with nonzero temperature makes responses stochastic, so axiom violations could be artifacts of response noise rather than genuine irrationality.

Editorial extensions

If this is right

  • If applied to an LLM, driving the de Finetti penalty to zero guarantees the model's reported probabilities cannot be Dutch-booked and are consistent with some probability measure.
  • Driving $1-\mathrm{CCEI}$ to zero guarantees the model's budget-set choices are utility-maximizing for a continuous, monotone, concave utility, so no revealed-preference test can refute the rationality of those choices.
  • Driving $1-E$ to zero guarantees the model's portfolio choices under uncertainty are jointly rationalized by a single prior and a concave utility, unifying beliefs and preferences in the test.
  • Because the checks are label-free, they can be run at scale on synthetic choice problems, exposing a model's rationality to far more tests than human-subject studies allow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same construction could be applied to other representation theorems, such as maxmin expected utility or variational preferences, to give continuous penalties that penalize deviations from those richer standards.
  • The penalties could be used as regularizers inside standard training pipelines, jointly with reward signals, since coherence is value-neutral and complements rather than replaces alignment objectives.
  • A testable extension is to average responses across multiple samples or use a deterministic decoding before computing the penalties, to separate genuine incoherence from sampling noise.
  • The exhaustiveness claim is limited to the elicited data; out-of-sample or differently phrased queries could still reveal violations, so the checks are a lower bound on irrationality, not a global certification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper argues that classical if-and-only-if representation theorems from decision theory can be turned into label-free rationality tests and regularizers for large language models by checking axioms on synthetic choice problems. Three instantiations are developed: de Finetti's coherence theorem yields a Dutch-book linear program penalty L(p); Afriat's theorem yields the penalty 1−CCEI, with a generic no-ties lemma ensuring CCEI=1 if and only if the data satisfy GARP; and the Echenique–Saito SARSEU theorem yields the penalty 1−E, with an LP-based characterization and a continuity lemma. The paper claims that passing the axioms exhausts the implications of the relevant rationality standard for the elicited data, and that the penalties are continuous, computable in polynomial time, and usable for evaluation and regularization.

Significance. If the methodological gap identified below were closed, the proposal would offer a principled, theory-grounded way to measure one dimension of LLM coherence without ground-truth labels, complementing calibration and RLHF rather than replacing them. The paper imports the representation theorems correctly, gives explicit computational procedures, and proves two nontrivial lemmas (Lemma 1 and Lemma 2) with detailed appendix proofs. The exhaustive-guarantee conclusion is logically valid for a fixed dataset. The main threat to practical significance is that LLM responses are stochastic and prompt-sensitive, so the penalties are not stable properties of the model unless the unit of analysis is carefully defined; this is a scope limitation, not a flaw in the mathematical core.

major comments (3)
  1. [Section 5 and Section 3, Lemma 1] The manuscript nowhere addresses sampling randomness in LLM outputs. With temperature > 0, identical prompts can yield different responses on repeated draws, so the penalties L(p), 1−CCEI, and 1−E are random variables; the same model can pass on one draw and fail on another. The 'if and only if' theorems apply to a fixed realized dataset, so the exhaustive guarantee does not attach to the model's behavior. The assumption in Lemma 1 that 'each choice x_t depends only on observation t's prices p_t and income w_t' is compatible with a stochastic policy, but it does not make the penalty a stable model-level quantity. The paper should either restrict evaluation to deterministic decoding and define the object of evaluation as the prompt-to-response function, or develop a distributional treatment (e.g., expected penalty with confidence intervals, or testing based on multiple samples) and state which guarantees survive that treatment.
  2. [Section 5 and Section 7] The handling of paraphrase sensitivity creates an ambiguity in the unit of analysis. Section 5 says that the same formal choice problem described in different words may elicit different responses and that violations from paraphrases 'count.' That means the evaluation object is a collection of datasets generated by different prompts, not a single decision-maker's dataset. The exhaustive guarantee holds only for one fixed dataset, and the paper does not specify how to aggregate across paraphrases and roles to obtain a model-level rationality verdict. The role-fixing advice ('fix a role for each batch') is a practical heuristic, but it does not by itself define the model-level property being measured.
  3. [Section 3, penalty paragraph] The abstract and Section 3 describe the penalties as continuous, but 1−CCEI is not continuous at datasets with exact budget ties: as the paper itself notes just before Lemma 1, CCEI can equal 1 when GARP fails, and an arbitrarily small perturbation of a tied dataset can change CCEI discontinuously. Lemma 1 provides a generic no-ties condition under which CCEI=1 iff GARP, but it does not establish continuity of the penalty. If 1−CCEI is proposed as a regularizer for gradient-based training, the continuity claim should be stated with the needed genericity or smoothness caveat, or a smoothed version of the penalty should be defined.
minor comments (6)
  1. [Section 3] The sentence 'CCEI is continuous in the model's choices' should be qualified by the same no-ties condition used in Lemma 1; as written it conflicts with the paper's own discussion of exact budget equality.
  2. [Section 4] The reference to 'a generalization of their results to settings with state-dependent utility' should cite the specific proposition or theorem in Echenique and Saito (2015), since this is the basis for the claimed extension.
  3. [Section 5] In the adversarial problem-generation sentence, 'including the output from previous penalty computations' is vague; it should say how previous penalties are used, for example as features or as part of the proposing model's objective.
  4. [Section 3] The phrase 'Seror (2024) extend' should be 'extends.'
  5. [Section 7] Limitation 1 says the rationalizing utility 'could be arbitrary'; more precisely, it is unidentified from finite data and could be poorly aligned, which is the substantive concern.
  6. [Section 2] The paper says the approach requires 'no external labels,' but mapping an LLM's natural-language probability statements to real numbers in [0,1] already requires an interpretation layer; this should be acknowledged in the implementation discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the penalties and exhaustive guarantees are derived from external representation theorems, with no fitted parameters and no load-bearing self-citations.

full rationale

The paper proposes penalty functionals L(p), 1-CCEI, and 1-E that are defined directly from de Finetti's coherence theorem, Afriat's theorem, and Echenique-Saito's SARSEU theorem, none of which are authored by the paper's author. The only original technical results, Lemma 1 and Lemma 2, are proved in the appendix from stated assumptions (generic prices and the LP characterization of SARSEU), not assumed as inputs. The central 'exhaustive' claim follows logically from the if-and-only-if structure: if a dataset is rationalizable, then every necessary condition of rationalizability holds, so no further rationality test on the same data can reject it. No parameter is fitted to data, and no prediction is a renamed input. The paper's own limitations in Section 7 (coherence is not enough, choice of axiom system matters, monetary-payoff restriction) and implementation discussion in Section 5 (role-fixing and paraphrase sensitivity) are validity and scope concerns rather than circularity: the penalties are not fitted to any response distribution. The absence of a treatment of sampling stochasticity is a substantive correctness risk for the LLM application, but it does not make the derivation circular, because the penalties are still defined by external theorems and not by the data they evaluate. No circular step can be exhibited from the paper's equations or citations.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities and fits no free parameters. The central claims rest entirely on classical representation theorems, which are external and well-established, plus the explicit domain assumptions of the two new lemmas.

assumptions (5)
  • standard math De Finetti's coherence theorem for finite prevision assignments: no Dutch book iff extendable to a finitely additive probability measure.
    Invoked in Section 2 to justify the Dutch book penalty L(p) as an exact coherence measure.
  • standard math Afriat's theorem: GARP iff finite budget data can be rationalized by a continuous, monotone, concave utility function, via the Afriat inequalities.
    Invoked in Section 3 to define the CCEI as a measure of departure from utility maximization.
  • standard math Echenique-Saito theorem: SARSEU iff portfolio data are rationalizable by subjective expected utility with a concave utility and a full-support prior.
    Invoked in Section 4 to justify the 1-E penalty for joint belief-preference coherence.
  • domain assumption Generic-price independence and budget exhaustion assumptions in Lemma 1.
    Assumed so that CCEI=1 iff GARP holds with probability one; requires prices drawn from continuous distributions independently across observations and exact budget exhaustion.
  • domain assumption Strict positivity of payoffs in Lemma 2(iii).
    The continuity of 1-E is proved only on the region where all payoffs x_t(s) > 0, so the penalty may be discontinuous when a payoff is zero.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems." pith.science (2026). https://pith.science/paper/WDJDRMH4

@misc{pith2026260805015,
  author       = {Pith},
  title        = {Pith review of: Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WDJDRMH4}},
  note         = {Machine review of arXiv:2608.05015}
}
read the original abstract

Representation theorems in decision theory establish that behavior satisfies certain axioms if and only if it can be rationalized by a well-defined objective. I argue that this ``if and only if'' structure provides a potentially useful foundation for label-free evaluation and regularization of LLMs and other AI systems. Axiom compliance can be checked from the model's own responses to synthetic choice problems, with no external labels or human feedback, and the penalties are readily computable. Because the axioms are necessary and sufficient, the resulting checks exhaust the implications of the relevant rationality standard for the elicited data: a model that passes cannot be rejected on rationality grounds by any further test of the same data. I discuss three instantiations: probabilistic coherence via a theorem of de Finetti, preference rationality via Afriat's theorem, and subjective expected utility via a theorem of Echenique and Saito (2015), each yielding a continuous penalty that is zero whenever behavior can be rationalized. Since coherence does not restrict which objective rationalizes behavior, these penalties complement rather than replace other evaluation and training signals.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 39 canonical work pages

  1. [1]

    Afriat, S. N. (1967). The construction of utility functions from expenditure data. International Economic Review , 8(1):67--77

    work page 1967

  2. [2]

    Afriat, S. N. (1973). On a system of inequalities in demand analysis: An extension of the classical method. International Economic Review , 14(2):460--472

    work page 1973

  3. [3]

    Aguiar, V. H. and Kashaev, N. (2026). GARP-EFM : Improving foundation models with revealed preference structure. arXiv preprint arXiv:2603.23993

    work page arXiv 2026

  4. [4]

    Ajayi, E., Chowdhury, A., and Lazar, S. (2026). Incoherent values? Probing LLM preferences through parametric variation. arXiv preprint arXiv:2606.21102

    work page Pith review arXiv 2026

  5. [5]

    C., Kang, D., et al

    Alur, R., Stadie, B. C., Kang, D., et al. (2025). AIA forecaster: Technical report. arXiv preprint arXiv:2511.07678

    arXiv 2025

  6. [6]

    and Richardson, K

    Betz, G. and Richardson, K. (2023). Probabilistic coherence, logical consistency, and bayesian learning: Neural language models as epistemic agents. PLOS ONE , 18(2):e0281372

    work page 2023

  7. [7]

    Chadwick, A., Kahng, A., and Kipper, J. (2025). Dutch books and money pumps: Rectifying vulnerabilities in LLMs through rationality. In Proceedings of the 5th International Conference on Human and Artificial Rationality (HAR) , Paris, France

    work page 2025

  8. [8]

    Chambers, C. P. and Echenique, F. (2016). Revealed Preference Theory . Econometric Society Monographs. Cambridge University Press, Cambridge

    work page 2016

Show all 47 references
  1. [9]

    Chandak, N., Goel, S., Prabhu, A., Hardt, M., and Geiping, J. (2025). Scaling open-ended reasoning to predict the future. arXiv preprint arXiv:2512.25070

    2025

  2. [10]

    X., Shan, Y., and Zhong, S

    Chen, Y., Liu, T. X., Shan, Y., and Zhong, S. (2023). The emergence of economic rationality of GPT . Proceedings of the National Academy of Sciences , 120(51):e2316205120

    2023

  3. [11]

    de Finetti, B. (1937). La pr\' e vision: ses lois logiques, ses sources subjectives. Annales de l'Institut Henri Poincar\' e , 7(1):1--68

    1937

  4. [12]

    de Finetti, B. (1974). Theory of Probability, volume 1 . John Wiley & Sons, New York

    1974

  5. [13]

    Dembo, A., Kariv, S., Polisson, M., and Quah, J. K.-H. (2026). Ever since Ellsberg . arXiv preprint arXiv:2607.09355

    2026 arXiv

  6. [14]

    Echenique, F. (2020). New developments in revealed preference theory: Decisions under risk, uncertainty, and intertemporal choice. Annual Review of Economics , 12:299--316

    2020

  7. [15]

    Echenique, F. (2021). On the meaning of the critical cost efficiency index. arXiv preprint arXiv:2109.06354

    2021 arXiv

  8. [16]

    Echenique, F., Imai, T., and Saito, K. (2023). Approximate expected utility rationalization. Journal of the European Economic Association , 21(5):1821--1864

    2023

  9. [17]

    Echenique, F., Lee, S., and Shum, M. (2011). The money pump as a measure of revealed preference violations. Journal of Political Economy , 119(6):1201--1223

    2011

  10. [18]

    and Saito, K

    Echenique, F. and Saito, K. (2015). Savage in the market. Econometrica , 83(4):1467--1495

    2015

  11. [19]

    Garrabrant, S., Benson-Tilsen, T., Critch, A., Soares, N., and Taylor, J. (2016). Logical induction. arXiv preprint arXiv:1609.03543

    2016 arXiv

  12. [20]

    D., Shapira, I., Vorobeychik, Y., and Wu, J

    Ge, L., Halpern, D., Micha, E., Procaccia, A. D., Shapira, I., Vorobeychik, Y., and Wu, J. (2024). Axioms for AI alignment from human feedback. In Advances in Neural Information Processing Systems , volume 38

    2024

  13. [21]

    and Schmeidler, D

    Gilboa, I. and Schmeidler, D. (1989). Maxmin expected utility with non-unique prior. Journal of Mathematical Economics , 18(2):141--153

    1989

  14. [22]

    and Raftery, A

    Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association , 102(477):359--378

    2007

  15. [23]

    Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning , pages 1321--1330

    2017

  16. [24]

    Gustafsson, J. E. (2022). Money-Pump Arguments . Elements in Decision Theory and Philosophy. Cambridge University Press, Cambridge

    2022

  17. [25]

    Hagendorff, T., Fabi, S., and Kosinski, M. (2023). Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in ChatGPT . Nature Computational Science , 3:833--838

    2023

  18. [26]

    Kim, S., Lee, S., Fong, E., Lee, H., and Lee, J. (2026). From drift to coherence: Stabilizing beliefs in LLMs . arXiv preprint arXiv:2606.17832

    2026 arXiv

  19. [27]

    Knight, F. H. (1921). Risk, Uncertainty and Profit . Houghton Mifflin, Boston

    1921

  20. [28]

    Lee, S.-W., Yang, S., Kwak, D., and Siegel, N. Y. (2025). Advancing event forecasting through massive training of large language models: Challenges, solutions, and broader impacts. arXiv preprint arXiv:2507.19477

    2025 arXiv

  21. [29]

    Maccheroni, F., Marinacci, M., and Rustichini, A. (2006). Ambiguity aversion, robustness, and the variational representation of preferences. Econometrica , 74(6):1447--1498

    2006

  22. [30]

    W., Ren, R., Phan, L., Mu, N., Khoja, A., Zhang, O., and Hendrycks, D

    Mazeika, M., Yin, X., Tamirisa, R., Lim, J., Lee, B. W., Ren, R., Phan, L., Mu, N., Khoja, A., Zhang, O., and Hendrycks, D. (2025). Utility engineering: Analyzing and controlling emergent value systems in AIs . In Advances in Neural Information Processing Systems

    2025

  23. [31]

    Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 35

    2022

  24. [32]

    Paleka, D., Pallavi Sudhir , A., Alvarez, A., Bhat, V., Shen, A., Wang, E., and Tram\` e r, F. (2025). Consistency checks for language model forecasters. In Proceedings of the 13th International Conference on Learning Representations

    2025

  25. [33]

    K.-H., and Renou, L

    Polisson, M., Quah, J. K.-H., and Renou, L. (2020). Revealed preferences over risk and uncertainty. American Economic Review , 110(6):1782--1820

    2020

  26. [34]

    Z., Ruis, L., Guo, Z

    Pres, I., Li, B. Z., Ruis, L., Guo, Z. C., Hu, K., Damani, M., Puri, I., Lubana, E. S., and Andreas, J. (2026). Position: It's time to optimize for self-consistency. Working paper, MIT CSAIL

    2026

  27. [35]

    Qiu, L., Sha, F., Allen, K., Kim, Y., Linzen, T., and van Steenkiste, S. (2026). Bayesian teaching enables probabilistic reasoning in large language models. Nature Communications

    2026

  28. [36]

    D., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems , 36

    2023

  29. [37]

    Raman, N., Lundy, T., Amouyal, S., Levine, Y., Leyton-Brown, K., and Tennenholtz, M. (2024). STEER : Assessing the economic rationality of large language models. In Proceedings of the 41st International Conference on Machine Learning

    2024

  30. [38]

    Savage, L. J. (1954). The Foundations of Statistics . John Wiley & Sons, New York

    1954

  31. [39]

    Seror, A. (2024). The moral mind(s) of large language models. arXiv preprint arXiv:2412.04476

    2024 arXiv

  32. [40]

    Smeulders, B., Cherchye, L., De Rock, B., and Spieksma, F. C. (2013). The money pump as a measure of revealed preference violations: A comment. Journal of Political Economy , 121(6):1248--1258

    2013

  33. [41]

    N., Banayeeanzade, A., Bolourani, A., Bahrani, F., Chaubey, A., Karimireddy, S

    Tak, A. N., Banayeeanzade, A., Bolourani, A., Bahrani, F., Chaubey, A., Karimireddy, S. P., Schwarz, N., and Gratch, J. (2026). Sparks of rationality: Do reasoning LLMs align with human judgment and choice? arXiv preprint arXiv:2601.22329

    2026

  34. [42]

    Tan, Z.-X., Carroll, M., Franklin, M., and Ashton, H. (2025). Beyond preferences in ai alignment. Philosophical Studies , 182:1813--1863

    2025

  35. [43]

    Varian, H. R. (1982). The nonparametric approach to demand analysis. Econometrica , 50(4):945--973

    1982

  36. [44]

    Wen, S. (2025). Economic rationality under specialization: Evidence of decision bias in AI agents. arXiv preprint arXiv:2501.18190

    2025 arXiv

  37. [45]

    Yamin, K., Tang, J., Cortes-Gomez, S., Sharma, A., Horvitz, E., and Wilder, B. (2026a). When agents say one thing and do another: Validating elicited beliefs from LLMs . arXiv preprint arXiv:2602.06286

    2026 arXiv

  38. [46]

    Yamin, K., Tang, J., Horvitz, E., and Wilder, B. (2026b). Can revealed preferences clarify LLM alignment and steering? arXiv preprint arXiv:2605.08556

    2026 arXiv

  39. [47]

    and Griffiths, T

    Zhu, J.-Q. and Griffiths, T. L. (2024). Incoherent probability judgments in large language models. In Proceedings of the 46th Annual Conference of the Cognitive Science Society

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.