Quantitative Relaxations of Arrow's Axioms

Daniel Brous; Martin T. Wells; Moon Duchin; Suvadip Sana

REVIEW 3 major objections 6 minor 12 references

Quantitative Relaxations of Arrow's Axioms

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A voting rule passes both relaxed axioms exactly when it is a dictatorship.

desk verdict A clean metric framework for quantifying IIA and Unanimity violations, but the abstract overclaims Borda's empirical edge in a way that the paper's own bootstrap numbers do not support. read the letter →

arxiv 2506.12961 v1 pith:BK5XTVIC submitted 2025-06-15 cs.GT

Suvadip Sana , Daniel Brous , Martin T. Wells , Moon Duchin This is my paper · ORCID

classification cs.GT MSC 91B1491B12

keywords votingrulessocialchoiceaxiomsimpossibilitytheoremindependenceofirrelevantalternativesunanimityBordaruleKendalltaudistanceBradley-Terrymodel

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper converts two classical axioms of social choice, independence of irrelevant alternatives and unanimity, from binary guarantees into profile-level scores. $\sigma_{IIA}(f,P)$ measures how much a voting rule's output ranking changes when a candidate is removed, and $\sigma_U(f,P)$ measures how well the output respects majority pairwise preferences. The paper proves that a rule scores $\sigma_{IIA}=1$ and $\sigma_U>0$ on every profile exactly when it is a dictatorship, so the classical impossibility theorem survives verbatim in the metric setting. Used as tools, these scores let five common rules be compared on thousands of real and simulated elections, and the Borda rule comes out with the best scores on both dimensions. This turns a long-standing critique of binary axiomatics into a practical way to grade election rules by their actual behavior.

What carries the argument

The machinery is a pair of continuous scores computed from a voting rule $f$ and a profile $P$. $\sigma_{IIA}$ uses Kendall-tau swap distance between the ranking $f(P)$ and the ranking produced after deleting each candidate one at a time; it equals 1 when deletion never changes the relative order among remaining candidates and 0 when every deletion reverses the order completely. $\sigma_U$ uses the pairwise comparison graph of majority margins: it is 1 when the output ranking is a topological sort of that graph, and it decreases toward 0 as the worst-aligned pair of candidates approaches a unanimous preference that the rule reverses. These definitions make the classical axioms exact limits of the metrics, which is what allows the quantitative impossibility theorem to be proved by direct appeal to the classical one.

What would settle it

For the empirical claim: re-run the Scottish and Bradley-Terry comparisons treating STV output as a winner set rather than a converted ranking, and check whether Borda's top scores persist. For the theoretical claim: exhibit any single non-dictatorial rule with $\sigma_{IIA}=1$ and $\sigma_U>0$ on every profile, which would contradict the quantitative impossibility theorem.

Watch

Extended reading notes

Core claim

The central claim is the quantitative impossibility theorem: a voting rule $f$ is a dictatorship if and only if $\sigma_{IIA}(f,P)=1$ and $\sigma_U(f,P)>0$ for all preference profiles $P$. The proof runs through a proposition showing that $\sigma_{IIA}\equiv 1$ is equivalent to classical IIA, and through the definition of $\sigma_U$, whose positivity rules out unanimity violations. Thus the theorem is a faithful quantitative restatement of the classical impossibility result, not a new weakening of it. The empirical discovery is that on 1,070 Scottish local-government elections and on Bradley-Terry synthetic profiles, Borda consistently records the highest average $\sigma_{IIA}$ and $\sigma_U$ among Plurality, 2-Approval, 3-Approval, Borda, and STV, a pattern the authors connect to a recent result that weakening IIA to allow preference intensity uniquely selects Borda.

Load-bearing premise

The empirical comparison assumes that converting STV winner sets into a complete ranking for the metrics does not distort the comparison; the authors themselves note that this representation is 'not fully aligned' with how the elections were run.

Editorial extensions

If this is right

For any fixed election profile, axiom compliance becomes a number in $[0,1]$ instead of a yes/no answer, so rules can be ranked by how closely they satisfy the axioms in that election.
The quantitative impossibility theorem implies that no non-dictatorial rule can maintain perfect candidate-removal stability and full unanimity respect across all possible profiles.
The paper's greedy topological-sort algorithm shows that a rule can be constructed in polynomial time to maximize $\sigma_U$ on any profile, so majoritarian alignment is an optimizable objective.
On both observed Scottish elections and synthetic Bradley-Terry profiles, Borda has the highest $\sigma_{IIA}$ and $\sigma_U$ among the five rules tested, supporting the view that Borda is a good compromise between stability and majority responsiveness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: because $\sigma_U$ depends only on $f(P)$ and the profile, it can be computed from a single output ranking, making it a cheap audit tool for deployed voting algorithms in multi-agent or LLM settings.
Editorial inference: the same template could be applied to other binary axioms such as monotonicity, producing a family of continuous axiom scores and a multi-dimensional axiom profile for each rule.
Editorial inference: if STV were scored as a winner set instead of the converted complete ranking used here, the relative standing of the rules could shift, so the Borda-dominance conclusion is tied to the ranking representation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The theoretical core is in good shape. The two metrics, sigma_IIA built on swap distance and sigma_U built on majority alignment, are natural and the equivalence results connecting them to classical IIA and Unanimity are correct. The greedy rule that provably maximizes sigma_U is a genuine contribution, and the proof of its optimality is clean. The quantitative Arrow theorem is, as the authors more or less admit, a direct corollary of Arrow's theorem via those equivalences, not a new impossibility result. That is fine, but it should not be oversold.

The empirical story is where the paper gets shaky. The abstract says Borda 'consistently receives the highest' sigma_IIA and sigma_U scores across observed and synthetic elections. The body is more careful, saying Borda 'most frequently' receives the highest scores, and the bootstrap analysis in Appendix B shows that for sigma_U, Borda and 3-Approval fall into the same statistical grouping. That means the confidence intervals overlap, so on the paper's own numbers, Borda is not distinguishable from 3-Approval on sigma_U. The 'consistently highest' claim in the abstract is stronger than the evidence, and this is the load-bearing empirical takeaway. No pairwise significance tests are reported, and the STV-to-ranking conversion, while acknowledged in Section 5.1 as 'not fully aligned' with the actual winner-set elections, could still distort the relative scores. These are fixable issues, but they need to be fixed before the empirical conclusions are stated so baldly.

What is genuinely new and useful: the swap-distance formulation of sigma_IIA (an improvement over the edit-distance version in Zhao et al.), the sigma_U metric, the optimization result, and the empirical sweep over Scottish and synthetic data. The metrics give people a concrete, profile-level way to talk about axiom compliance, and the code and data are public. The literature engagement is honest, including the acknowledgment of the prior edit-distance metric.

Who should read this: anyone working in computational social choice, and to a lesser extent AI alignment researchers who want graded measures of voting-rule behavior. It deserves a serious referee. I would send it to peer review, but with a clear request to the authors: align the abstract with the bootstrap results, report uncertainty in the rule comparisons, and either handle STV more carefully or qualify the claims about it.

My verdict: conditional acceptance, pending the empirical claims being scaled back to what the statistics actually support.

Referee Report

3 major / 6 minor

Summary. The paper proposes two real-valued metrics, σ_IIA and σ_U, that measure, for a given preference profile, how stable a voting rule is under candidate deletion and how well its output respects pairwise majority preferences. It shows that σ_IIA ≡ 1 is equivalent to classical IIA and that σ_U > 0 for all profiles is equivalent to classical Unanimity, yielding a quantitative restatement of Arrow's theorem: a rule is dictatorial if and only if both conditions hold. It also defines a greedy rule that maximizes σ_U and reports empirical comparisons of Plurality, 2-Approval, 3-Approval, Borda, and STV on Scottish local-election data and Bradley-Terry synthetic profiles.

Significance. If the empirical claims held as stated, the paper would provide an interpretable, profile-level framework for measuring axiom compliance in real elections and a useful comparison of common voting rules. Strengths include transparent definitions, a clean equivalence with Arrow's theorem, public code and data, and bootstrap uncertainty analysis. However, the headline empirical claim is stronger than the reported statistics: Table 2 shows ties, Appendix B groups Borda with 3-Approval for σ_U, and no pairwise significance tests are reported. The STV ranking conversion is also acknowledged to be misaligned with the rule's actual output. The theoretical result is a correct but direct corollary of Arrow's theorem rather than a quantitative trade-off bound, so the paper's incremental contribution rests mainly on the metrics and the empirical comparison, which currently need tightening.

major comments (3)

[Abstract; §4.1; Appendix B] The abstract's claim that the Borda rule 'consistently receives the highest σ_IIA and σ_U scores' is not supported by the reported statistics. Section 4.1 uses the weaker wording 'most frequently receives the highest,' Table 2 shows Borda tied with 2-Approval on both metrics for the example profile, and Appendix B reports bootstrap groupings for σ_U of {Borda, 3-Approval}, {2-Approval}, and {Plurality, STV}, meaning Borda and 3-Approval are not statistically separated. No pairwise significance tests are reported. Please either add formal comparisons or qualify the conclusion to state that Borda belongs to the top group rather than being consistently and uniquely highest.
[§2.2; §5.1] The empirical comparison treats STV as a complete-ranking rule by filling in winners and eliminated candidates, a conversion the authors themselves say in Section 5.1 is 'not fully aligned with the nature of the elections where it was used.' This conversion can change σ_IIA and σ_U values for STV and therefore the relative ordering of rules. Because the abstract's headline includes STV among the compared rules, please provide a sensitivity analysis for alternative ways of ranking STV outputs or remove STV from the headline comparison until the conversion is justified.
[Appendix A, proof of Lemma 4] The proof states that g is increasing on [0,1), h is decreasing, and hence g∘h is increasing on [0,n]; the composition of an increasing function with a decreasing function is decreasing, not increasing. The subsequent minimization formula g(h(max δ)) is what would follow from a decreasing composition, so the result appears salvageable, but the monotonicity claim must be corrected and the proof rechecked.

minor comments (6)

[§3.4] The Quantitative Arrow's Theorem should explicitly state the standing assumption m ≥ 3, since Arrow's theorem does not apply to fewer than three candidates.
[§4.2] The Bradley-Terry generation procedure is not fully specified; please provide the exact generative model, including how the Dirichlet parameter α maps to candidate strengths and how partial ballots are produced.
[Table 2] The text notes that Borda and 2-Approval give the same outcome on the example profile; the implication for the 'highest scores' wording should be acknowledged in the main text.
[Appendix B] The phrase 'compliment the descriptive box plots' should be 'complement the descriptive box plots.'
[§2.2] The sentence describing how candidates left when seats are filled are placed 'in between, in order of first-place votes when the process terminates' is vague; please clarify the tie-breaking and termination conventions for STV.
[§1.1] The comparison with Zhao et al. [2024b] would be clearer if the relationship between σ_IIA and their edit-distance-based similarity score were stated formally rather than only informally.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the quantitative Arrow theorem is an explicit restatement of Arrow's theorem, and the empirical comparisons are direct metric evaluations without fitted parameters.

full rationale

I found no load-bearing circular step. The Quantitative Arrow Theorem (Section 3.4) is proved by invoking Arrow's theorem directly after Proposition 2 establishes the equivalence between sigma_IIA=1 and classical IIA and after the observation that sigma_U>0 for all profiles is equivalent to classical Unanimity. The proof states 'By Arrow's impossibility theorem it follows that f must be dictatorial,' so the conclusion is imported from an external theorem, not from the metric definitions themselves. The empirical comparisons (Sections 4.1-4.2 and Appendix B) compute the defined metrics on real and synthetic profiles without fitting any parameter to the outcome being predicted, so no claim reduces to its input by construction. The abstract's 'consistently receives the highest' is stronger than the body's 'most frequently receives the highest' and than the bootstrap grouping {Borda, 3-Approval} for sigma_U, but that is an overstatement/calibration issue, not circularity. The VoteKit and dataset citations are tool/data credits and are not load-bearing argument. Section 5.1's STV-ranking limitation is an acknowledged scope restriction, not a circular step.

Assumptions & free parameters 0 free parameters · 5 assumptions · 2 invented entities

The central theoretical results are essentially definitional plus an application of a known theorem. The main empirical comparison is independent. No fitted constants appear; the alpha values in the simulation are scenario hyperparameters, not fitted to data.

assumptions (5)

domain assumption Arrow's impossibility theorem applies to the extended domain of partial rankings and to the candidate-deletion formulation of IIA.
Invoked in Section 3.4 to conclude dictatorship from sigma_IIA=1 and sigma_U>0. The classical theorem is proved for complete rankings, and the extension to partial ballots is assumed without proof.
domain assumption Voting rules are deterministic functions from profiles of partial ballots to complete tie-free rankings.
Definition in Section 2.1 requires complete output rankings, which excludes rules or representations that output ties or winner sets. This matters for the STV comparison.
domain assumption A rule satisfying f(P^C)=f(P)^C for all C is equivalent to Arrow's IIA.
Proposition 2 proves this under the stated domain, but the proof needs the rule to be defined for every candidate subset, not just the full candidate set.
ad hoc to paper STV output can be represented as a complete ranking for comparison with scoring rules.
Section 2.2 describes a ranking conversion for STV, and Section 5.1 acknowledges that this is not fully aligned with the winner-set output of real STV elections.
domain assumption Bradley-Terry with Dirichlet alpha generates realistic preference distributions.
Used for synthetic experiments in Section 4.2. The alpha values are scenario choices, not fitted to the Scottish data.

invented entities (2)

sigma_IIA metric
purpose: Quantify the degree of IIA violation as an average Kendall-tau distance under candidate deletions.
New formal construct; it has no external falsifiable prediction beyond the definition itself.
sigma_U metric
purpose: Quantify majority alignment of a voting rule outcome with pairwise preferences.
New formal construct; it has no external falsifiable prediction beyond the definition itself.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Quantitative Relaxations of Arrow's Axioms." pith.science (2026). https://pith.science/paper/BK5XTVIC

@misc{pith2026250612961,
  author       = {Pith},
  title        = {Pith review of: Quantitative Relaxations of Arrow's Axioms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BK5XTVIC}},
  note         = {Machine review of arXiv:2506.12961}
}

abstract

In this paper we develop a novel approach to relaxing Arrow's axioms for voting rules, addressing a long-standing critique in social choice theory. Classical axioms (often styled as fairness axioms or fairness criteria) are assessed in a binary manner, so that a voting rule fails the axiom if it fails in even one corner case. Many authors have proposed a probabilistic framework to soften the axiomatic approach. Instead of immediately passing to random preference profiles, we begin by measuring the degree to which an axiom is upheld or violated on a given profile. We focus on two foundational axioms-Independence of Irrelevant Alternatives (IIA) and Unanimity (U)-and extend them to take values in $[0,1]$. Our $\sigma_{IIA}$ measures the stability of a voting rule when candidates are removed from consideration, while $\sigma_{U}$ captures the degree to which the outcome respects majority preferences. Together, these metrics quantify how a voting rule navigates the fundamental trade-off highlighted by Arrow's Theorem. We show that $\sigma_{IIA}\equiv 1$ recovers classical IIA, and $\sigma_{U}>0$ recovers classical Unanimity, allowing a quantitative restatement of Arrow's Theorem. In the empirical part of the paper, we test these metrics on two kinds of data: a set of over 1000 ranked choice preference profiles from Scottish local elections, and a batch of synthetic preference profiles generated with a Bradley-Terry-type model. We use those to investigate four positional voting rules-Plurality, 2-Approval, 3-Approval, and the Borda rule-as well as the iterative rule known as Single Transferable Vote (STV). The Borda rule consistently receives the highest $\sigma_{IIA}$ and $\sigma_{U}$ scores across observed and synthetic elections. This compares interestingly with a recent result of Maskin showing that weakening IIA to include voter preference intensity uniquely selects Borda.

Figures

Figures reproduced from arXiv: 2506.12961 by the authors.

**Figure 2.** Boxplots of the σIIA and σU metrics for the five voting rules on the Scottish elections. Most elections have 6-9 candidates and are used to select 3-4 winners, and the plots are split out accordingly. As we might expect, elections with more candidates are more stable, but tend to contain more violations of pairwise preferences. 0.4 0.6 0.8 1.0 U 0.4 0.5 0.6 0.7 0.8 0.9 1.0 IIA Borda 0.4 0.6 0.8 1.0 U 3 Approval 0.4 … view at source ↗

**Figure 3.** Scatterplots of σIIA and σU on the Scottish data help us visualize the relationship between the two metrics. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Ward-level heatmaps showing the σIIA and σU metrics for Borda, Plurality, and STV, applied to the 2012 Scottish local elections. 4.2 Synthetic data This section reports the outcomes of trials run on synthetic preference profiles generated using a model derived from a B…

**Figure 5.** Figure 5: Boxplots of σIIA and σU for the five voting rules applied to profiles generated from a BradleyTerry model. We use Dirichlet parameters to vary candidate strength: α = 2, 3 give scenarios in which voters tend to have more uniform preferences among candidates, while α =…

**Figure 6.** Figure 6: Bootstrapped mean and 95 % confidence intervals for [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

**Figure 7.** Figure 7: σIIA distributions by years (2012,2017,2022) of five voting rules on Scottish local elections with 6–9 candidates and 3-4 seats 6 7 8 9 Number of Candidates 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Score U by Candidates (2012) 6 7 8 9 Number of Candidates U by Candidates (2017) 6 7…

**Figure 8.** Figure 8: σU distributions by years (2012, 2017,2022) of five voting rules on Scottish local elections with 6–9 candidates and 3-4 seats 15 [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 8 canonical work pages

[1]

A difficulty in the concept of social welfare

Kenneth J Arrow. A difficulty in the concept of social welfare. Journal of Political Economy, 58 0 (4): 0 328--346, 1950

work page 1950
[2]

Social choice should guide AI alignment in dealing with diverse human feedback

Vincent Conitzer, Rachel Freedman, Jobst Heitzig, Wesley H Holliday, Bob M Jacobs, Nathan Lambert, Milan Moss \'e , Eric Pacuit, Stuart Russell, Hailey Schoelkopf, et al. Social choice should guide AI alignment in dealing with diverse human feedback. arXiv preprint arXiv:2404.10271, 2024

arXiv 2024
[3]

VoteKit : Python package

Data and Democracy Lab . VoteKit : Python package. https://github.com/mggg/VoteKit, 2024. GitHub repository

work page 2024
[4]

Independence of irrelevant alternatives under the lens of pairwise distortion

Th \'e o Delemazure, J \'e r \^o me Lang, and Grzegorz Pierczy \'n ski. Independence of irrelevant alternatives under the lens of pairwise distortion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9645--9652, 2024

work page 2024
[5]

The probability of violating A rrow’s conditions

Keith L Dougherty and Jac C Heckelman. The probability of violating A rrow’s conditions. European Journal of Political Economy, 65: 0 101936, 2020

work page 2020
[6]

An Introduction to the Bootstrap

Bradley Efron and Robert J Tibshirani. An Introduction to the Bootstrap. Chapman and Hall/CRC, 1994

work page 1994
[7]

Borda’s rule and A rrow’s independence condition

Eric Maskin. Borda’s rule and A rrow’s independence condition. Journal of Political Economy, 133 0 (2): 0 385--420, 2025

work page 2025
[8]

Monotonicity anomalies in S cottish local government elections

David McCune and Adam Graham-Squire. Monotonicity anomalies in S cottish local government elections. Social Choice and Welfare, 63 0 (1): 0 69--101, 2024

work page 2024

Show all 12 references

[9]

AI alignment and social choice: Fundamental limitations and policy implications

Abhilash Mishra. AI alignment and social choice: Fundamental limitations and policy implications. arXiv preprint arXiv:2310.16048, 2023

2023 arXiv
[10]

The structure of the election-generating universe

T Nicolaus Tideman and Florenz Plassmann. The structure of the election-generating universe. Manuscript, 2010

2010
[11]

An electoral approach to diversify LLM -based multi-agent collective decision-making

Xiutian Zhao, Ke Wang, and Wei Peng. An electoral approach to diversify LLM -based multi-agent collective decision-making. arXiv preprint arXiv:2410.15168, 2024 a

2024 arXiv
[12]

Measuring the inconsistency of large language models in preferential ranking

Xiutian Zhao, Ke Wang, and Wei Peng. Measuring the inconsistency of large language models in preferential ranking. arXiv preprint arXiv:2410.08851, 2024 b

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

A difficulty in the concept of social welfare

Kenneth J Arrow. A difficulty in the concept of social welfare. Journal of Political Economy, 58 0 (4): 0 328--346, 1950

work page 1950

[2] [2]

Social choice should guide AI alignment in dealing with diverse human feedback

Vincent Conitzer, Rachel Freedman, Jobst Heitzig, Wesley H Holliday, Bob M Jacobs, Nathan Lambert, Milan Moss \'e , Eric Pacuit, Stuart Russell, Hailey Schoelkopf, et al. Social choice should guide AI alignment in dealing with diverse human feedback. arXiv preprint arXiv:2404.10271, 2024

arXiv 2024

[3] [3]

VoteKit : Python package

Data and Democracy Lab . VoteKit : Python package. https://github.com/mggg/VoteKit, 2024. GitHub repository

work page 2024

[4] [4]

Independence of irrelevant alternatives under the lens of pairwise distortion

Th \'e o Delemazure, J \'e r \^o me Lang, and Grzegorz Pierczy \'n ski. Independence of irrelevant alternatives under the lens of pairwise distortion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9645--9652, 2024

work page 2024

[5] [5]

The probability of violating A rrow’s conditions

Keith L Dougherty and Jac C Heckelman. The probability of violating A rrow’s conditions. European Journal of Political Economy, 65: 0 101936, 2020

work page 2020

[6] [6]

An Introduction to the Bootstrap

Bradley Efron and Robert J Tibshirani. An Introduction to the Bootstrap. Chapman and Hall/CRC, 1994

work page 1994

[7] [7]

Borda’s rule and A rrow’s independence condition

Eric Maskin. Borda’s rule and A rrow’s independence condition. Journal of Political Economy, 133 0 (2): 0 385--420, 2025

work page 2025

[8] [8]

Monotonicity anomalies in S cottish local government elections

David McCune and Adam Graham-Squire. Monotonicity anomalies in S cottish local government elections. Social Choice and Welfare, 63 0 (1): 0 69--101, 2024

work page 2024