Pith. sign in

REVIEW 3 major objections 2 minor 43 references

What Voting Rules Actually Do: A Data-Driven Analysis of Multi-Winner Voting

T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A multi-winner voting rule's worth can be measured by how often it violates axioms, and neural-network rules trained on that frequency metric outperform traditional rules.

desk verdict The submission is broken: the abstract promises a voting-rule paper, the full text is an unrelated computer vision paper, so there is nothing to referee. read the letter →

arxiv 2508.06454 v1 pith:QNLTF47Y submitted 2025-08-08 cs.AI cs.GT

Joshua Caiata , Ben Armstrong , Kate Larson This is my paper
classification cs.AIcs.GT MSC 91B1491B12
keywords multi-winnervotingcommitteeselectionaxiomaticsocialchoiceruleevaluationaxiomviolationfrequencydata-drivenneural-networkpreferencedistributions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that multi-winner voting rules should be judged by how often they violate axioms across sampled preference distributions, not by whether a bad profile can make them fail. On that frequency-based yardstick, the paper compares established committee-selection rules and then trains neural networks to act as voting rules with the explicit goal of minimizing axiom violations. The result it reports is that the learned rules beat the traditional rules on this metric, which the authors take as evidence that data-driven methods can evaluate existing voting systems and inform the design of new ones. This shifts axiomatic social choice from a binary, worst-case classification of rules to a quantitative, distribution-dependent comparison.

What carries the argument

The load-bearing object is the empirical violation-frequency framework: instead of checking whether a rule violates an axiom on any profile, the framework draws preference profiles from specified distributions and counts the share of profiles on which each axiom fails. This replaces a binary property with a measurable rate, making axiomatic behavior an optimization target rather than a classification. The second piece of machinery is a neural-network voting rule trained on the same distributions to minimize the violation rate, which is what lets the paper put learned committee selection head-to-head with hand-designed rules.

What would settle it

Take preference profiles from real elections or committee decisions, hold them out from the neural network's training, and count axiom violations for the learned rule and the traditional rules; if the learned rule's violation rate is not lower on those held-out real profiles, the claimed superiority collapses.

Watch

Extended reading notes

Core claim

The core discovery is that axiom satisfaction is a matter of degree. The paper constructs a framework that samples preference profiles from several distributions, applies a multi-winner voting rule to each profile, and records how frequently the rule violates a set of axioms. Used as the yardstick, this violation frequency turns the usual separation between satisfying and violating rules into a quantitative ranking that can differ across distributions. The paper then goes further, training neural networks to choose committees while optimizing directly against this violation rate, and reports that such learned rules outperform traditional multi-winner rules on the same frequency metric. In th

Load-bearing premise

The claim depends on the preference distributions sampled by the framework being representative of real committee-selection settings, and on a fair comparison between a neural network trained on those very distributions and traditional rules that were not optimized against the violation-frequency metric.

Editorial extensions

If this is right

  • Rule rankings become distribution-dependent: two rules with identical worst-case guarantees can have very different violation rates on the profiles that actually occur.
  • Axiom satisfaction becomes a quantifiable objective, so different normative properties can be traded off or optimized instead of checked off.
  • Neural-network voting rules trained on sampled preferences are a viable alternative to hand-designed rules, at least under the framework's frequency metric.
  • The approach provides a concrete case for continuing data-driven research in social choice and for designing voting systems tuned to specific settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Manuscript note: the full text supplied with this record is a different paper, so this pith is reconstructed from the abstract; the specific axioms, distributions, and network details would need to be checked against the actual voting-rule manuscript.
  • Editorial inference: if violation frequency becomes the accepted yardstick, rule choice becomes context-dependent, and the practical question shifts from 'which rule is best?' to 'which rule for which electorate?'
  • Editorial inference: a fair stress test would withhold the evaluation distributions from the neural network's training and add distributions it never saw, because a rule trained to minimize the measured violations is being compared with rules that were not trained on that objective.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The submission presents an abstract that promises a data-driven framework for evaluating multi-winner voting rules by the frequency with which they violate axioms under diverse preference distributions, with the headline claim that neural networks acting as voting rules can outperform traditional rules at minimizing axiom violations. The full text, however, is not this paper. It is an unrelated computer vision manuscript titled "TRUST: Leveraging Text Robustness for Unsupervised Domain Adaptation," by a different set of authors from Catania, Nottingham, and EPFL. The body contains only a UDA method based on CLIP-based pseudo-label uncertainty and a language-guided soft-contrastive loss, with experiments on DomainNet, GeoNet, and VisDA. No voting rule, axiom, preference profile, committee-selection problem, or preference distribution is defined anywhere in the provided text. The central claim of the abstract is therefore entirely absent from the artifact, and no evaluation of the abstract's assertions is possible.

Significance. Had the voting-rule framework been present with proper held-out evaluation, the frequency-based perspective on axioms could be a useful complement to worst-case analyses, and the claim that learned rules can outperform classical rules at minimizing violation frequencies is substantive and falsifiable. However, as submitted, none of that content is in the manuscript. The only technical material, the TRUST UDA method, is outside the paper's stated scope and cannot be credited toward the voting-rule claim. There are no machine-checked proofs, reproducible code, parameter-free derivations, or empirical results relevant to the claimed contribution.

major comments (3)
  1. [Title/Abstract vs. Full Text] The manuscript is internally inconsistent. The title and abstract describe a multi-winner voting analysis, but every section of the full text—including the title, equations (1)–(8), Figures 1–6, and Tables 1–4—concerns unsupervised domain adaptation for image classification. There is no definition of a multi-winner rule, no axiom, no preference profile or distribution, and no experiment measuring axiom violations. The central claim of the abstract is thus unverifiable from the submitted text.
  2. [§4, Tables 1–3] The claimed empirical result—"neural networks, acting as voting rules, can outperform traditional rules in minimizing axiom violations"—is not presented anywhere. Section 4 reports image classification accuracy on DomainNet-345, GeoNet, and VisDA, comparing TRUST to UDA baselines. There are no voting-rule baselines, no axiom-violation frequencies, no error bars on such frequencies, and no neural-network-as-voting-rule architecture or training objective. This is a load-bearing omission: the headline result is entirely absent.
  3. [Abstract, final sentence] A further concern that would need to be settled in any revision is circularity: if the neural network is trained to minimize the same axiom-violation metric on the same preference distributions used for evaluation, its advantage over rules not optimized against that metric is partly by construction. The submitted text provides no information about training distributions, evaluation distributions, or the axiom-violation loss, so this cannot be ruled out. This is secondary to the absence of the framework, but it should be addressed before any empirical claim is made.
minor comments (2)
  1. [Abstract] The abstract's reference to "diverse preference distributions" is never operationalized; no distribution family, sampling procedure, or number of profiles is given anywhere in the submitted text.
  2. [References] The bibliography contains no entries on voting theory or social choice. This is consistent with the full text being an unrelated computer vision paper and further confirms that the claimed voting-rule analysis does not appear in the artifact.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity demonstrable: the supplied full text is an unrelated paper, so the claimed voting-rule derivation chain is absent.

full rationale

The abstract describes a data-driven framework for evaluating multi-winner voting rules and claims that neural networks acting as voting rules can outperform traditional rules in minimizing axiom violations. However, the full text provided is an entirely different manuscript, an unsupervised domain adaptation paper titled 'TRUST', with no content on voting rules, axioms, preference distributions, committees, or neural voting rules. There are therefore no equations, algorithms, training procedures, or experimental comparisons that could be examined for a circular reduction. The reader's suspicion that the neural network may have been trained on the same axiom-violation metric used for evaluation is plausible in the abstract, but it is not established by any quoted text or equation, and the hard rule for circularity requires exhibiting a specific reduction (Eq. X = Eq. Y by construction, or a fitted parameter renamed as a prediction). No such reduction can be identified because the relevant sections are missing. The abstract's claim is unverifiable from the supplied artifact, and the mismatch is a serious integrity/completeness concern, but it is not a demonstrated circularity. Accordingly, the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central comparison rests on two unquantified degrees of freedom: the preference distributions chosen for evaluation and the training objective of the neural network. Neither is specified in the abstract, and both would need to be fixed before the headline claim can be reproduced or falsified. No new physical or ontological entities are proposed; the neural-network voting rule is a methodological construct, not an invented entity in the sense of an unverified postulated object.

free parameters (2)
  • Preference distribution family and sampling parameters = unspecified in abstract
    The frequency of axiom violations is measured 'across diverse preference distributions'; the choice of distributions and their parameters is the main unspoken degree of freedom and drives the whole rule comparison.
  • Neural network training objective = unspecified in abstract
    The network is trained to minimize axiom violations; which axioms enter the objective and how they are weighted determines the reported outperformance.
assumptions (3)
  • domain assumption The chosen fairness axioms are the right or sufficient criteria for comparing multi-winner rules
    The abstract evaluates 'axiom violations' as the performance measure; no justification of the axiom set appears in the abstract.
  • domain assumption Sampled preference distributions represent practical committee-selection settings
    The abstract claims behavior 'in practice' from evaluations on sampled distributions; representativeness is assumed, not argued, in the first paragraph of the abstract.
  • standard math Background theory of multi-winner voting rules and axiom satisfaction is correct
    The framework inherits the standard formal results of social choice without restating them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What Voting Rules Actually Do: A Data-Driven Analysis of Multi-Winner Voting." pith.science (2026). https://pith.science/paper/QNLTF47Y

@misc{pith2026250806454,
  author       = {Pith},
  title        = {Pith review of: What Voting Rules Actually Do: A Data-Driven Analysis of Multi-Winner Voting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QNLTF47Y}},
  note         = {Machine review of arXiv:2508.06454}
}
read the original abstract

Committee-selection problems arise in many contexts and applications, and there has been increasing interest within the social choice research community on identifying which properties are satisfied by different multi-winner voting rules. In this work, we propose a data-driven framework to evaluate how frequently voting rules violate axioms across diverse preference distributions in practice, shifting away from the binary perspective of axiom satisfaction given by worst-case analysis. Using this framework, we analyze the relationship between multi-winner voting rules and their axiomatic performance under several preference distributions. We then show that neural networks, acting as voting rules, can outperform traditional rules in minimizing axiom violations. Our results suggest that data-driven approaches to social choice can inform the design of new voting systems and support the continuation of data-driven research in social choice.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 36 canonical work pages

  1. [1]

    Learning to elect

    Cem Anil and Xuchan Bao. Learning to elect. Advances in Neural Information Processing Systems, 34: 0 8006--8017, 2021

    work page 2021

  2. [2]

    Optimal voting, 2025

    Ben Armstrong. Optimal voting, 2025. URL https://pypi.org/project/optimal-voting/

    work page 2025

  3. [3]

    Machine learning to strengthen democracy

    Ben Armstrong and Kate Larson. Machine learning to strengthen democracy. In NeurIPS Joint Workshop on AI for Social Good, 2019

    work page 2019

  4. [4]

    The Condorcet Principle for Multiwinner Elections: From Shortlisting to Proportionality

    Haris Aziz, Edith Elkind, Piotr Faliszewski, Martin Lackner, and Piotr Skowron. The C ondorcet principle for multiwinner elections: From shortlisting to proportionality. arXiv preprint arXiv:1701.08023, 2017

    work page Pith review arXiv 2017

  5. [5]

    On the rationale of group decision-making

    Duncan Black. On the rationale of group decision-making. Journal of Political Economy, 56: 0 23--34, 1948

    work page 1948

  6. [6]

    Putting a compass on the map of elections

    Niclas Boehmer, Robert Bredereck, Piotr Faliszewski, Rolf Niedermeier, and Stanis aw Szufa. Putting a compass on the map of elections. arXiv preprint arXiv:2105.07815, 2021

    arXiv 2021

  7. [7]

    Properties of the M allows model depending on the number of alternatives: A warning for an experimentalist

    Niclas Boehmer, Piotr Faliszewski, and Sonja Kraiczy. Properties of the M allows model depending on the number of alternatives: A warning for an experimentalist. In International Conference on Machine Learning, pages 2689--2711. PMLR, 2023

    work page 2023

  8. [8]

    Handbook of Computational Social Choice

    Felix Brandt, Vincent Conitzer, Ulle Endriss, J \'e r \^o me Lang, and Ariel D Procaccia. Handbook of Computational Social Choice. Cambridge University Press, 2016

    work page 2016

Show all 43 references
  1. [9]

    Robust and verifiable proportionality axioms for multiwinner voting

    Markus Brill and Jannik Peters. Robust and verifiable proportionality axioms for multiwinner voting. arXiv preprint arXiv:2302.01989, 2023

    2023 arXiv

  2. [10]

    Voting: A machine learning approach

    D \'a vid Burka, Clemens Puppe, L \'a szl \'o Szepesv \'a ry, and Attila Tasn \'a di. Voting: A machine learning approach. European Journal of Operational Research, 299 0 (3): 0 1003--1017, 2022

    2022

  3. [11]

    Eliciting single-peaked preferences using comparison queries

    Vincent Conitzer. Eliciting single-peaked preferences using comparison queries. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, pages 1--8, 2007

    2007

  4. [12]

    Social choice for AI alignment: Dealing with diverse human feedback

    Vincent Conitzer, Rachel Freedman, Jobst Heitzig, Wesley H Holliday, Bob M Jacobs, Nathan Lambert, Milan Moss \'e , Eric Pacuit, Stuart Russell, Hailey Schoelkopf, et al. Social choice for AI alignment: Dealing with diverse human feedback. arXiv preprint arXiv:2404.10271, 2024

    2024 arXiv

  5. [13]

    Prudent k -choice functions: Properties and algorithms

    Bernard Debord. Prudent k -choice functions: Properties and algorithms. Mathematical Social Sciences, 26 0 (1): 0 63--77, 1993

    1993

  6. [14]

    Testing axioms against human reward divisions in cooperative games

    Greg d'Eon and Kate Larson. Testing axioms against human reward divisions in cooperative games. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, pages 312--320, 2020

    2020

  7. [15]

    Voting Procedures

    M Dummett. Voting Procedures. Oxford University Press, 1984

    1984

  8. [16]

    U ber die statistik verketteter vorg \

    Florian Eggenberger and George P \'o lya. \"U ber die statistik verketteter vorg \"a nge. ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift f \"u r Angewandte Mathematik und Mechanik , 3 0 (4): 0 279--289, 1923

    1923

  9. [17]

    Restricted preference domains in social choice: Two perspectives

    Edith Elkind. Restricted preference domains in social choice: Two perspectives. In Algorithmic Game Theory: 11th International Symposium, SAGT 2018, Beijing, China, September 11-14, 2018, Proceedings 11, pages 12--18. Springer, 2018

    2018

  10. [18]

    Properties of multiwinner voting rules

    Edith Elkind, Piotr Faliszewski, Piotr Skowron, and Arkadii Slinko. Properties of multiwinner voting rules. Social Choice and Welfare, 48: 0 599--632, 2017

    2017

  11. [19]

    The Spatial Theory of Voting: An Introduction

    James M Enelow and Melvin J Hinich. The Spatial Theory of Voting: An Introduction. CUP Archive, 1984

    1984

  12. [20]

    Learning aggregation rules in participatory budgeting: A data-driven approach

    Roy Fairstein, Dan Vilenchik, and Kobi Gal. Learning aggregation rules in participatory budgeting: A data-driven approach. arXiv preprint arXiv:2412.01864, 2024

    2024 arXiv

  13. [21]

    Multiwinner voting: A new challenge for social choice theory

    Piotr Faliszewski, Piotr Skowron, Arkadii Slinko, and Nimrod Talmon. Multiwinner voting: A new challenge for social choice theory. Trends in C omputational S ocial C hoice , 74 0 (2017): 0 27--47, 2017

    2017

  14. [22]

    An experimental comparison of multiwinner voting rules on approval elections

    Piotr Faliszewski, Martin Lackner, Krzysztof Sornat, and Stanis aw Szufa. An experimental comparison of multiwinner voting rules on approval elections. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 2675--2683, 2023

    2023

  15. [23]

    Machine learning? I n MY election? I t's more likely than you think: Voting rules via neural networks

    Daniel Firebanks-Quevedo. Machine learning? I n MY election? I t's more likely than you think: Voting rules via neural networks. Thesis, 2020

    2020

  16. [24]

    Condorcet social choice functions

    Peter C Fishburn. Condorcet social choice functions. SIAM Journal on Applied Mathematics, 33 0 (3): 0 469--489, 1977

    1977

  17. [25]

    Smoothed analysis of social choice revisited

    Bailey Flanigan, Daniel Halpern, and Alexandros Psomas. Smoothed analysis of social choice revisited. In International Conference on Web and Internet Economics, pages 290--309. Springer, 2023

    2023

  18. [26]

    The C ondorcet criterion and committee selection

    William V Gehrlein. The C ondorcet criterion and committee selection. Mathematical Social Sciences, 10 0 (3): 0 199--209, 1985

    1985

  19. [27]

    Condorcet’s paradox and anonymous preference profiles

    William V Gehrlein and Peter C Fishburn. Condorcet’s paradox and anonymous preference profiles. Public Choice, pages 1--18, 1976

    1976

  20. [28]

    Deep learning for multi-facility location mechanism design

    Noah Golowich, Harikrishna Narasimhan, and David C Parkes. Deep learning for multi-facility location mechanism design. In IJCAI, pages 261--267, 2018

    2018

  21. [29]

    Les th \'e ories de l’int \'e r \^e t g \'e n \'e ral et le probl \`e me logique de l’agr \'e gation

    Georges Th \'e odule Guilbaud. Les th \'e ories de l’int \'e r \^e t g \'e n \'e ral et le probl \`e me logique de l’agr \'e gation. Economie appliqu \'e e , 5 0 (4): 0 501--584, 1952

    1952

  22. [30]

    Predicting the winners of B orda, K emeny and D odgson elections with supervised machine learning

    Hanna Kujawska, Marija Slavkovik, and Jan-Joachim R \"u ckmann. Predicting the winners of B orda, K emeny and D odgson elections with supervised machine learning. In Multi-Agent Systems and Agreement Technologies: 17th European Conference, EUMAS 2020, and 7th International Con...

    2020

  23. [31]

    Multi-Winner Voting with Approval Preferences

    Martin Lackner and Piotr Skowron. Multi-Winner Voting with Approval Preferences. Springer International Publishing, 2023. ISBN 9783031090165. doi:10.1007/978-3-031-09016-5. URL http://dx.doi.org/10.1007/978-3-031-09016-5

    2023 doi

  24. [32]

    Soft C ondorcet optimization

    Marc Lanctot, Kate Larson, Michael Kaisers, Quentin Berthet, Ian Gemp, Manfred Diaz, Roberto-Rafael Maura-Rivero, Yoram Bachrach, Anna Koop, and Doina Precup. Soft C ondorcet optimization. In International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025), 2025

    2025

  25. [33]

    Non-null ranking models

    Colin L Mallows. Non-null ranking models. i. Biometrika, 44 0 (1/2): 0 114--130, 1957

    1957

  26. [34]

    Deepvoting: Learning voting rules with tailored embeddings

    Leonardo Matone, Ben Abramowitz, Nicholas Mattei, and Avinash Balakrishnan. Deepvoting: Learning voting rules with tailored embeddings. arXiv preprint arXiv:2408.13630, 2024

    2024 arXiv

  27. [35]

    Preflib: A library for preferences

    Nicholas Mattei and Toby Walsh. Preflib: A library for preferences. In Algorithmic Decision Theory: Third International Conference, ADT 2013, Bruxelles, Belgium, November 12-14, 2013, Proceedings 3, pages 259--270. Springer, 2013

    2013

  28. [36]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019

    2019

  29. [37]

    Proportionality and the limits of welfarism

    Dominik Peters and Piotr Skowron. Proportionality and the limits of welfarism. In Proceedings of the 21st ACM Conference on Economics and Computation, pages 793--794, 2020

    2020

  30. [38]

    A proportional voting system for awards nominations resistant to voting blocs

    Jameson Quinn and Bruce Schneier. A proportional voting system for awards nominations resistant to voting blocs. Preprint per https://www. schneier. com/academic/archives/2016/05/a\_proportional\_votin. html. Accessed, 14, 2019

    2016

  31. [39]

    Voting by axioms

    Marie Christin Schmidtlein. Voting by axioms. Master's thesis. ILLC, University of Amsterdam, 2022

    2022

  32. [40]

    Voting by axioms

    Marie Christin Schmidtlein and Ulle Endriss. Voting by axioms. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 2067--2075, 2023

    2023

  33. [41]

    The single transferable vote

    Nicolaus Tideman. The single transferable vote. Journal of Economic Perspectives, 9 0 (1): 0 27--38, 1995

    1995

  34. [42]

    A theory of the learnable

    Leslie G Valiant. A theory of the learnable. Communications of the ACM, 27 0 (11): 0 1134--1142, 1984

    1984

  35. [43]

    Generating single peaked votes

    Toby Walsh. Generating single peaked votes. arXiv preprint arXiv:1503.02766, 2015

    2015 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.