REVIEW 4 major objections 5 minor 29 references
Fair Compromises in Participatory Budgeting: a Multi-Agent Deep Reinforcement Learning Approach
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Trained multi-agent reinforcement-learning voters find collective choices in participatory budgeting that are fairer and more satisfying than actual election results, and the route to compromise runs through lower-cost projects.
desk verdict Branching-DQN scaling is real, but the headline satisfaction gains are largely built into the reward, and the preference model is a guess—so treat the 'vote cheaper' advice as a model artifact until validated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is a branching deep Q-network voter model paired with an impact-area reward function. Cumulative ballots with ten tokens across 33 projects produce roughly 1.5 billion possible ballots, so the branching architecture splits the action into ten parallel token-assignment heads of 33 actions each, shrinking the action space to 330 and letting 1703 independent learning agents train in the same election. Each agent's reward (Eq. 1) sums, over winning projects, the logarithm of project cost multiplied by the overlap between the voter's favoured impact areas and the project's impact areas and by the voter's token share on that project; this encodes both a preference for projects aligned with the voter's issues and a preference for larger projects. The trained policies are compared against the actual votes and against untrained random agents on project satisfaction, cost satisfaction, and budget share.
What would settle it
A direct test would re-run the same training pipeline on several additional participatory budgeting elections with different cost distributions and compare the learnt token allocations to voters' stated reasons for voting; if the cheap-project shift and the fairness improvement do not appear there, the claimed route to fair compromise would be a property of these two datasets rather than of participatory budgeting reward structures.
Extended reading notes
Core claim
The paper's central claim is that when voter preferences are formed from impact-area overlaps and voters learn to maximise their own reward, the resulting collective choices are fairer and more satisfying than the actual election outcomes. In both elections and under both equal-shares and utilitarian-greedy aggregation, trained agents lower the Gini coefficient of satisfaction and raise utilitarian welfare; in Toulouse the egalitarian welfare becomes non-zero, meaning every modelled voter had at least one project they supported in the winning set. The paper attributes these improvements to a learnt compromise: trained agents allocate a higher proportion of tokens to small- and medium-cost projects than the human voters did, and this shift is what enables the fairer outcome.
Load-bearing premise
The load-bearing premise is that each voter's true preferences are adequately captured by assuming they favour every impact area of every project they voted for in the real election; if that assumption is wrong, the reported fairness and satisfaction gains measure the model's proxy for welfare rather than real voter welfare.
Editorial extensions
If this is right
- If the claim is correct, participatory budgeting elections can become fairer by including and promoting lower-cost projects, without altering the ballot aggregation rule.
- Policymakers can use the model's what-if experiments to adjust election design, such as project selection or cost structure, toward outcomes that leave fewer voters unrepresented.
- The branching architecture demonstrates that multi-agent reinforcement learning can scale to elections with more than a thousand voters, making simulation-based decision support feasible for real participatory budgeting.
- Voters could be informed that shifting support toward cheaper projects increases the proportion of the budget that goes to the issues they care about.
- A fairer outcome does not come at the cost of average satisfaction: trained agents raise utilitarian welfare in both elections and aggregation methods, even as the per-voter budget share decreases in some conditions.
Reading between the lines
- The lower-cost pattern may be specific to these two elections; re-running the pipeline on elections with very different project-cost distributions would show whether it is a general property of equal-shares and greedy aggregation or a dataset artifact.
- Because the reward function's log-cost term directly rewards voting for expensive projects, an ablation that removes or alters that term would reveal whether the cheap-project shift is caused by the reward design rather than by the aggregation rules.
- The preference assumption that every voter favours all impact areas of every project they voted for could be replaced with stated preferences or inverse reinforcement learning; the fairness comparison would then measure true voter welfare instead of a model proxy.
- The action-branching training approach could transfer to other combinatorial collective decisions, such as participatory budgeting at city scale, provided the non-stationarity of thousands of learning agents remains manageable.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multi-agent deep reinforcement learning model in which voting agents in participatory budgeting elections learn cumulative ballots under a reward function based on inferred impact-area preferences (Eq. 1). The model is applied to real data from Aarau 2023 and Toulouse 2019, using both utilitarian greedy and equal shares aggregation. The authors report that trained agents achieve higher project and cost satisfaction than the actual voters, that the collective choice is fairer in terms of Gini, egalitarian, and utilitarian measures (Table 4), and that the improvement is associated with a shift of votes towards cheaper projects (Figure 6). The paper also introduces a branching deep Q-network architecture to scale MARL to over 1,700 agents and positions the work as ethically aligned decision support for fair compromise.
Significance. If the findings are reliable, this is a genuinely useful decision-support tool: it scales multi-agent reinforcement learning to realistic participatory budgeting settings, uses real election data, and makes its code and data available. The branching architecture is a practical contribution, and the explicit discussion of model assumptions is commendable. However, the central policy conclusion—that cheaper project compromises produce fairer and more satisfying outcomes—rests on two load-bearing assumptions: that the reward function in Eq. 1 is a faithful model of voter utility, and that the welfare measures used in Table 4 are independent of the training objective. The current evidence does not yet separate a learned discovery from a restatement of the reward specification, so the external validity of the headline claim is not established.
major comments (4)
- [§3.2, Eq. (1) and §4.3] The headline satisfaction gains are substantially built into the training objective. Eq. (1) rewards a voter when a winning project belongs to an impact area the voter favours, scaled by the tokens the voter assigned to that project (a(w)/T). Section 4.3 defines project satisfaction repp as the fraction of the winning set that the voter voted for. Since voter preferences p(v) in Section 4.1 are inferred from the actual ballot as the union of impact areas of the projects the voter supported, the reward and repp measure overlapping events: the training signal encourages exactly the behaviour (voting for favoured projects that win) that is later counted as satisfaction. The comparison with actual voters, who never optimised Eq. (1), is therefore not an independent test of welfare. I recommend evaluating trained policies on a welfare measure that is not aligned with the training reward, for example a separately elicited preference profile or a different additive utility function, and reporting both sets of results.
- [§4.1] The preference model is load-bearing and currently unvalidated. Voter preferences p(v) are assumed to equal the union of impact areas of all projects the voter originally voted for; the paper itself calls this 'a crude assumption'. The reward function (Eq. 1), the satisfaction metrics, and the fairness comparisons in Table 4 all depend on this p(v). If real voters supported projects for reasons unrelated to the listed impact areas, or voted strategically, the model optimises a proxy objective and the conclusion that cheaper projects yield fairer outcomes does not transfer to the actual elections. This is the main external-validity risk, and it is not resolved by the current experiments. The paper should include either a sensitivity analysis over alternative preference models (e.g., sparse random subsets of impact areas, or single-issue voters) or an empirical validation of p(v) against additional survey data.
- [Table 4, §5] The fairness claims are reported as point estimates with no uncertainty quantification, despite 10 repetitions per condition being listed in Table 2. For example, the statement that 'Gini is smaller ... for all three welfare metrics' is a claim about the population of possible training runs, but the table shows no standard deviation, confidence interval, or significance test. It is possible that observed differences are within run-to-run noise, especially for the smaller egalitarian differences in Toulouse. Please report the distribution over repetitions and, ideally, paired tests across conditions.
- [§5, Figure 6] The causal conclusion 'a fairer more satisfying collective choice is possible through shifting votes to cheaper projects' is not directly supported by the evidence presented. Figure 6 shows a correlation: trained agents allocate more tokens to cheaper projects, and Table 4 shows fairer outcomes. But no experiment manipulates the cost composition of the winning set while holding voter preferences fixed, so alternative mechanisms—such as coordination on a broader project set, or the specific distribution of impact areas across cheap projects—could explain the fairness gains. A counterfactual experiment, such as training under a reward that penalises cheap projects or re-weighting the budget, would establish the cost-shift mechanism.
minor comments (5)
- [Table 1] Table 1 contains corrupted entries '/reve' in place of check/cross marks; please regenerate the table with the intended symbols.
- [§3.2] The loss function L = E[1/N Σ (r − Qd(s, ad))²] is ambiguous: please define d as the action-branch index and specify the distribution of transitions more precisely.
- [Table 4] The multi-level header of Table 4 is difficult to read; a two-row header with explicit column groups for Actual and MARL and for Equal shares and Greedy would help the reader match columns to conditions.
- [Figure 5] The text explaining cumulative satisfaction shares is confusing; add axis labels and a legend or caption that makes the cumulative interpretation explicit.
- [§4.1] The statement that Aarau 2023 was 'the first participatory budgeting election to use cumulative voting and the method of equal shares' should be supported by a citation or qualified.
Circularity Check
Satisfaction result partly restates the training objective; fairness and cost-compromise findings remain independent.
-
fitted input called prediction
[Section 3.2 (Eq. 1), Section 4.3, Section 5 (Result i)]
"R(v, W) = Σ_{w∈W} ( log(C(w)) × (|p(v)∩i(w)|/|p(v)|) × (|p(v)∩i(w)|/|i(w)|) × (a(w)/T) ) (1) ... Satisfaction (projects) repp = |a∩W| / |W| ... Satisfaction (cost) repc = c(a∩W) / c(W) ... Voting agents in the proposed multi-agent reinforcement learning model achieve a higher vote satisfaction than the one of voters in the actual election."
The reward in Eq. (1) includes a(w)/T, the voter's own token share assigned to each winning project, while the satisfaction metrics repp and repc are also functions of a∩W, the set of projects the voter voted for that win. Training agents to maximize Eq. (1) therefore directly optimizes a proxy for these satisfaction metrics, up to the multiplicative overlap and log-cost factors. Reporting that trained MARL voters achieve higher satisfaction than actual voters is thus in part a restatement of the training objective rather than an independent prediction. The fairness improvement and the shift to cheaper projects are not directly encoded in the reward and provide independent content.
full rationale
The paper's central fairness and cost-compromise claims are not circular. The Gini, egalitarian, and utilitarian comparisons in Table 4 are computed from satisfaction, cost satisfaction, and share, none of which is the agents' reward function; the shift toward lower-cost projects emerges from the budget constraint under additive rewards rather than being specified as a target. Self-citations in the paper are transparent and not load-bearing in a theorem-importing sense: reference [1] supplies the Aarau dataset, and references [11,12] supply the impact-area preference formulation, but the model assumptions are stated and the code/data are available. The main circularity concern is limited to the satisfaction headline: the reward's a(w)/T term is structurally aligned with the satisfaction metrics that count whether voted projects win, so part of the reported satisfaction gain is a consequence of optimizing a closely related objective. The fairness and cheaper-project findings, however, are not forced by the reward and are externally meaningful model outcomes.
Assumptions & free parameters
free parameters (1)
- Reward function cost weighting =
log(C(w)), natural logarithm, no fitted coefficient
assumptions (4)
- domain assumption Voter preferences are fully described by a fixed set of impact-area overlaps, with utility given by Eq. 1.
- domain assumption Voters favour every impact area of every project they voted for in the real election.
- domain assumption A purely self-interested reward, without collective or communication terms, is sufficient to model compromise-seeking behaviour.
- domain assumption Equal shares and utilitarian greedy are the relevant aggregation rules, and the real ballot constraint can be relaxed in the model.
Cite this review
Pith. "Pith review of Fair Compromises in Participatory Budgeting: a Multi-Agent Deep Reinforcement Learning Approach." pith.science (2026). https://pith.science/paper/FTBLBS3Y
@misc{pith2026250717433,
author = {Pith},
title = {Pith review of: Fair Compromises in Participatory Budgeting: a Multi-Agent Deep Reinforcement Learning Approach},
year = {2026},
howpublished = {\url{https://pith.science/paper/FTBLBS3Y}},
note = {Machine review of arXiv:2507.17433}
}
read the original abstract
Participatory budgeting is a method of collectively understanding and addressing spending priorities where citizens vote on how a budget is spent, it is regularly run to improve the fairness of the distribution of public funds. Participatory budgeting requires voters to make decisions on projects which can lead to ``choice overload". A multi-agent reinforcement learning approach to decision support can make decision making easier for voters by identifying voting strategies that increase the winning proportion of their vote. This novel approach can also support policymakers by highlighting aspects of election design that enable fair compromise on projects. This paper presents a novel, ethically aligned approach to decision support using multi-agent deep reinforcement learning modelling. This paper introduces a novel use of a branching neural network architecture to overcome scalability challenges of multi-agent reinforcement learning in a decentralized way. Fair compromises are found through optimising voter actions towards greater representation of voter preferences in the winning set. Experimental evaluation with real-world participatory budgeting data reveals a pattern in fair compromise: that it is achievable through projects with smaller cost.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Evangelos Pournaras, Srijoni Majumdar, Thomas Wellings, Joshua C. Yang, Fatemeh B. Heravan, Regula Hänggli Fricker, and Dirk Helbing. Upgrading democracies with fairer voting methods, 2025
work page 2025
-
[2]
Sheena S. Iyengar and Mark R. Lepper. When choice is demotivating: Can one desire too much of a good thing? Journal of Personality and Social Psychology, 79(6):995–1006, December 2000
work page 2000
-
[3]
Expect the worst! expectations and social interactive decision making
Cinzia Giorgetta, Alessandro Grecucci, Michele Graffeo, Nicolao Bonini, Roberta Ferrario, and Alan G Sanfey. Expect the worst! expectations and social interactive decision making. Brain sciences, 11(5):572, 2021
work page 2021
-
[4]
Dominik Dellermann, Philipp Ebel, Matthias Söllner, and Jan Marco Leimeister. Hybrid intelligence. Business & Information Systems Engineering, 61(5):637–643, 2019
work page 2019
-
[5]
The impossibility of automating ambiguity
Abeba Birhane. The impossibility of automating ambiguity. Artificial Life, 27(1):44–61, 2021
work page 2021
-
[6]
Compu- tational modelling of public policy: Reflections on practice
Nigel Gilbert, Petra Ahrweiler, Pete Barbrook-Johnson, Kavin Preethi Narasimhan, and Helen Wilkinson. Compu- tational modelling of public policy: Reflections on practice. Journal of Artificial Societies and Social Simulation, 21(1):14, 2018
work page 2018
-
[7]
R.S. Sutton and A.G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054–1054, 1998
work page 1998
-
[8]
Issues, principles or ideology? how young voters decide
David Denver and Gordon Hands. Issues, principles or ideology? how young voters decide. Electoral Studies, 9(1):19–36, 1990
work page 1990
Show all 29 references
-
[9]
Multi-agents reinforcement learning in iterative voting, 2019
Loujain Liekah and Umberto Grandi. Multi-agents reinforcement learning in iterative voting, 2019. 1-19
2019
-
[10]
Learning agents for iterative voting
Stephane Airiau, Umberto Grandi, and Filipo Studzinski Perotto. Learning agents for iterative voting. In International Conference on Algorithmic Decision Theory, pages 139–152. Springer, 2017
2017
-
[11]
Fair voting outcomes with impact and novelty compromises? unravelling biases in electing participatory budgeting winners
Sajan Maharjan, Srijoni Majumdar, and Evangelos Pournaras. Fair voting outcomes with impact and novelty compromises? unravelling biases in electing participatory budgeting winners. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, 382:2024...
2024
-
[12]
Consensus-based participatory budgeting for legitimacy: Decision support via multi-agent reinforcement learning
Srijoni Majumdar and Evangelos Pournaras. Consensus-based participatory budgeting for legitimacy: Decision support via multi-agent reinforcement learning. In Giuseppe Nicosia, Varun Ojha, Emanuele La Malfa, Gabriele La Malfa, Panos M. Pardalos, and Renato Umeton, editors, Mach...
2024
-
[13]
θ-learning: An algorithm for the self-organisation of collective self-governance
Asimina Mertzani, Josiah Ober, and Jeremy Pitt. θ-learning: An algorithm for the self-organisation of collective self-governance. In 2023 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS), pages 97–106, 2023. 10 A PREPRINT - SEPTEMBER 9, 2025
2023
-
[14]
Different modelling purposes
Bruce Edmonds. Different modelling purposes. In Simulating social complexity: A handbook , pages 39–58. Springer, 2017
2017
-
[15]
Albrecht, Filippos Christianos, and Lukas Schafer.Multi-Agent Reinforcement Learning: Foundations and Modern Approaches
Stefano V . Albrecht, Filippos Christianos, and Lukas Schafer.Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024
2024
-
[16]
Action branching architectures for deep reinforcement learning
Arash Tavakoli, Fabio Pardo, and Petar Kormushev. Action branching architectures for deep reinforcement learning. In Proceedings of the aaai conference on artificial intelligence, volume 32, 2018
2018
-
[17]
Cumulative voting: The value of minority shareholder voting rights
Sanjai Bhagat and James A Brickley. Cumulative voting: The value of minority shareholder voting rights. The Journal of Law and Economics, 27(2):339–365, 1984
1984
-
[18]
Proportional participatory budgeting with additive utilities
Dominik Peters, Grzegorz Pierczy´nski, and Piotr Skowron. Proportional participatory budgeting with additive utilities. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P.S. Liang, and J. Wortman Vaughan, editors,Advances in Neural Information Processing Systems, volume 34, pages 1...
2021
-
[19]
Rusu, Joel Veness, Marc G
V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Kirkeby Fidjeland, Georg Ostrovski, Stig Petersen, Charlie Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan W...
2015
-
[20]
The importance of experience replay database composition in deep reinforcement learning
Tim De Bruin, Jens Kober, Karl Tuyls, and Robert Babuška. The importance of experience replay database composition in deep reinforcement learning. In Deep reinforcement learning workshop, NIPS, 2015
2015
-
[21]
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010
2010
-
[22]
Prioritized experience replay
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016
2016
-
[23]
Communication-enabled deep reinforcement learning to optimise energy-efficiency in uav-assisted networks
Babatunji Omoniwa, Boris Galkin, and Ivana Dusparic. Communication-enabled deep reinforcement learning to optimise energy-efficiency in uav-assisted networks. Vehicular Communications, 43:100640, 2023
2023
-
[24]
Pabulib: A participatory budgeting library
Dariusz Stolicki, Stanislaw Szufa, and Nimrod Talmon. Pabulib: A participatory budgeting library. CoRR, abs/2012.06539, 2020
2012 arXiv
-
[25]
Welfare engineering in multiagent systems
Ulle Endriss and Nicolas Maudet. Welfare engineering in multiagent systems. In Andrea Omicini, Paolo Petta, and Jeremy Pitt, editors, Engineering Societies in the Agents World IV, pages 93–106, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg
2004
-
[26]
Fairness in long-term participatory budgeting
Martin Lackner, Jan Maly, and Simon Rey. Fairness in long-term participatory budgeting. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, pages 1566–1568, 2021
2021
-
[27]
The nature of the social agent
Kathleen Carley and Allen Newell. The nature of the social agent. The Journal of Mathematical Sociology , 19(4):221–262, 1994
1994
-
[28]
Gareth Polhill, Christina Semeniuk, and Frithjof Stöppler
Nanda Wijermans, Geeske Scholz, Émile Chappin, Alison Heppenstall, Tatiana Filatova, J. Gareth Polhill, Christina Semeniuk, and Frithjof Stöppler. Agent decision-making: The elephant in the room - enabling the justification of decision model fit in social-ecological models. En...
2023
-
[29]
Generating synthetic bitcoin transactions and predicting market price movement via inverse reinforcement learning and agent-based modeling
Kamwoo Lee, Sinan Ulkuatam, Peter Beling, and William Scherer. Generating synthetic bitcoin transactions and predicting market price movement via inverse reinforcement learning and agent-based modeling. Journal of Artificial Societies and Social Simulation, 21(3):5, 2018. 11
2018
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.