{"id":"c46ec64c-cb7e-45b7-afb2-d5dc93ca0ac3","arxiv_id":"2507.17433","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A branching deep Q-learning voting-agent model on real Aarau and Toulouse data finds that fairer participatory budgeting outcomes can emerge when voters coordinate on lower-cost projects.","lead":"This paper trains thousands of simulated voters with deep reinforcement learning to find voting strategies in two real participatory budgeting elections. The trained agents produce more equal satisfaction across voters, and they do it by shifting votes toward cheaper projects.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The key claim that cheaper-project compromises improve fairness and satisfaction depends entirely on the unvalidated preference model in Eq.","rationale":"The reader's conditional verdict identifies the Eq. 1 preference model as the weakest assumption, and I agree. My stress-test refines the mechanism: the same functional form appears in the reward (training objective) and in the satisfaction/share metrics used to measure fairness and satisfaction. This creates a circularity that is not fatal by itself—agents could still discover a genuinely fairer outcome—but it means the burden of proof is on the preference model. The paper openly admits this assumption is 'crude' (Section 4.1) and that it 'can be built on in future work,' which strengthens the concern. The available code and data permit the proposed ablation, which is why the concrete test is feasible. If the cheaper-project compromise is robust to multiple preference specifications, the conditional verdict can be upgraded; if not, the central claim should be downgraded or reframed as a statement about the model rather than about real elections. I do not see an internal inconsistency, and the branching architecture and scalability results are plausible, so the concern is external validity, not correctness of the RL method. The paper deserves credit for releasing code and data and for transparently stating its assumptions.","tokens_in":9951,"tokens_out":7388,"duration_ms":86253,"concrete_test":"Run the full Section 5 pipeline (training, validation, and evaluation) with three alternative preference specifications while keeping everything else fixed: (i) p(v) is the single impact area that receives the most tokens in voter v's real ballot, rather than the union of all voted projects' areas; (ii) p(v) is learned per voter via inverse reinforcement learning from actual ballots; (iii) p(v) is a randomly permuted assignment of impact areas as a null model. If the trained agents fail to produce lower Gini and higher satisfaction than actual voters under any of (i)–(iii), or if the shift toward cheaper projects in Figure 6 is no longer present, then the headline claim is an artifact of the Eq. 1 preference model. If the pattern persists across all three, the concern is substantially mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution—that trained MARL agents find fairer, more satisfying outcomes by shifting votes to cheaper projects—rests on Eq. 1 being a faithful utility model for real voters. In Section 4.1, each voter's preference set p(v) is inferred as the union of impact areas of all projects they voted for in the actual election. This is a strong, untested assumption: voters often vote strategically, single-issue, or for reasons orthogonal to the project's listed impact areas, so their ballot does not reveal a complete or accurate p(v). Eq. 1 also multiplies overlap by log(C(w)) and by the voter's own token share a(w)/T, introducing a cost-based utility and a dependence on the voter's action. The same structure is then used both as the training reward and, via satisfaction/share metrics, as the evaluation of fairness and satisfaction. Consequently, trained agents will necessarily score higher on those metrics than actual voters who never optimized Eq. 1, even if real voter welfare is unchanged. If the inferred preferences are wrong, Table 4's Gini/egalitarian/utilitarian comparisons and the recommendation to favour lower-cost projects do not transfer to actual participatory budgeting outcomes. This is an external-validity risk, not an internal inconsistency, but it is the weakest link in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multi-agent deep reinforcement learning model in which voting agents in participatory budgeting elections learn cumulative ballots under a reward function based on inferred impact-area preferences (Eq. 1). The model is applied to real data from Aarau 2023 and Toulouse 2019, using both utilitarian greedy and equal shares aggregation. The authors report that trained agents achieve higher project and cost satisfaction than the actual voters, that the collective choice is fairer in terms of Gini, egalitarian, and utilitarian measures (Table 4), and that the improvement is associated with a shift of votes towards cheaper projects (Figure 6). The paper also introduces a branching deep Q-network architecture to scale MARL to over 1,700 agents and positions the work as ethically aligned decision support for fair compromise.","tokens_in":10186,"tokens_out":6777,"duration_ms":77444,"significance":"If the findings are reliable, this is a genuinely useful decision-support tool: it scales multi-agent reinforcement learning to realistic participatory budgeting settings, uses real election data, and makes its code and data available. The branching architecture is a practical contribution, and the explicit discussion of model assumptions is commendable. However, the central policy conclusion—that cheaper project compromises produce fairer and more satisfying outcomes—rests on two load-bearing assumptions: that the reward function in Eq. 1 is a faithful model of voter utility, and that the welfare measures used in Table 4 are independent of the training objective. The current evidence does not yet separate a learned discovery from a restatement of the reward specification, so the external validity of the headline claim is not established.","major_comments":[{"comment":"The headline satisfaction gains are substantially built into the training objective. Eq. (1) rewards a voter when a winning project belongs to an impact area the voter favours, scaled by the tokens the voter assigned to that project (a(w)/T). Section 4.3 defines project satisfaction repp as the fraction of the winning set that the voter voted for. Since voter preferences p(v) in Section 4.1 are inferred from the actual ballot as the union of impact areas of the projects the voter supported, the reward and repp measure overlapping events: the training signal encourages exactly the behaviour (voting for favoured projects that win) that is later counted as satisfaction. The comparison with actual voters, who never optimised Eq. (1), is therefore not an independent test of welfare. I recommend evaluating trained policies on a welfare measure that is not aligned with the training reward, for example a separately elicited preference profile or a different additive utility function, and reporting both sets of results.","section":"§3.2, Eq. (1) and §4.3"},{"comment":"The preference model is load-bearing and currently unvalidated. Voter preferences p(v) are assumed to equal the union of impact areas of all projects the voter originally voted for; the paper itself calls this 'a crude assumption'. The reward function (Eq. 1), the satisfaction metrics, and the fairness comparisons in Table 4 all depend on this p(v). If real voters supported projects for reasons unrelated to the listed impact areas, or voted strategically, the model optimises a proxy objective and the conclusion that cheaper projects yield fairer outcomes does not transfer to the actual elections. This is the main external-validity risk, and it is not resolved by the current experiments. The paper should include either a sensitivity analysis over alternative preference models (e.g., sparse random subsets of impact areas, or single-issue voters) or an empirical validation of p(v) against additional survey data.","section":"§4.1"},{"comment":"The fairness claims are reported as point estimates with no uncertainty quantification, despite 10 repetitions per condition being listed in Table 2. For example, the statement that 'Gini is smaller ... for all three welfare metrics' is a claim about the population of possible training runs, but the table shows no standard deviation, confidence interval, or significance test. It is possible that observed differences are within run-to-run noise, especially for the smaller egalitarian differences in Toulouse. Please report the distribution over repetitions and, ideally, paired tests across conditions.","section":"Table 4, §5"},{"comment":"The causal conclusion 'a fairer more satisfying collective choice is possible through shifting votes to cheaper projects' is not directly supported by the evidence presented. Figure 6 shows a correlation: trained agents allocate more tokens to cheaper projects, and Table 4 shows fairer outcomes. But no experiment manipulates the cost composition of the winning set while holding voter preferences fixed, so alternative mechanisms—such as coordination on a broader project set, or the specific distribution of impact areas across cheap projects—could explain the fairness gains. A counterfactual experiment, such as training under a reward that penalises cheap projects or re-weighting the budget, would establish the cost-shift mechanism.","section":"§5, Figure 6"}],"minor_comments":[{"comment":"Table 1 contains corrupted entries '/reve' in place of check/cross marks; please regenerate the table with the intended symbols.","section":"Table 1"},{"comment":"The loss function L = E[1/N Σ (r − Qd(s, ad))²] is ambiguous: please define d as the action-branch index and specify the distribution of transitions more precisely.","section":"§3.2"},{"comment":"The multi-level header of Table 4 is difficult to read; a two-row header with explicit column groups for Actual and MARL and for Equal shares and Greedy would help the reader match columns to conditions.","section":"Table 4"},{"comment":"The text explaining cumulative satisfaction shares is confusing; add axis labels and a legend or caption that makes the cumulative interpretation explicit.","section":"Figure 5"},{"comment":"The statement that Aarau 2023 was 'the first participatory budgeting election to use cumulative voting and the method of equal shares' should be supported by a citation or qualified.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a multiagent systems or computational social choice venue, and the ethical framing is thoughtful. However, the current empirical core is vulnerable to the reward–satisfaction circularity, and the preference model is unvalidated. I would encourage the authors to either add out-of-sample validation or substantially weaken the policy recommendation to a conditional statement about the model. A revision that only fixes presentation would not address the load-bearing issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper does two genuinely new things: it applies branching DQN to cumulative participatory budgeting ballots, cutting the action space from ~1.5e9 to a few hundred, and it scales independent MARL voting agents to 1700+, where prior work used at most 100. The cost-compromise pattern—agents shift tokens to cheaper projects and the resulting winning sets have lower Gini—is an emergent result, not wired in. The authors also say outright that their preference model is crude, which is more honesty than many papers manage.\n\nThat said, the central comparison is weaker than it looks. Eq. 1 rewards a voter exactly when a project they favor and vote for wins, weighted by cost and token share. The Section 4.3 satisfaction metrics count the same event (voted projects winning). So trained agents are almost forced to score higher on satisfaction than actual voters, who never optimize that objective. The fairness (Gini) improvement is less circular—nobody rewards equality directly—but it is measured on the same reward-aligned metric, and Table 4 reports single numbers with no variance or significance over the 10 repetitions. Small issue: they relax the Aarau ballot rule that tokens must go to at least three projects; that should be checked.\n\nThe bigger external-validity problem, as the authors concede, is p(v): voters are assumed to favor every impact area of every project they voted for. That turns ballots into revealed preferences over impact areas, which is a strong and untested assumption. The recommendation that cities should include cheaper projects rests entirely on this proxy. If the proxy is wrong, the whole 'fair compromise' story doesn't transfer.\n\nWho is this for? People working on MARL for voting or decision support will want the branching architecture and the complexity analysis. Social choice people will want the fairness result but should treat it as an artifact of the model until robustness checks are done. I'd send it to peer review—the method is real, and the circularity can be fixed by reporting a satisfaction metric that isn't part of the reward, or by ablating the cost and share terms. But I wouldn't cite the policy conclusion as evidence yet.\n\nRecommendation: engage, but demand the robustness checks before accepting the headline.","headline":"Branching-DQN scaling is real, but the headline satisfaction gains are largely built into the reward, and the preference model is a guess—so treat the 'vote cheaper' advice as a model artifact until validated.","tokens_in":10696,"tokens_out":2738,"would_cite":false,"duration_ms":31697,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Trained multi-agent reinforcement-learning voters find collective choices in participatory budgeting that are fairer and more satisfying than actual election results, and the route to compromise runs through lower-cost projects.","keywords":["Multi-Agent Reinforcement Learning","Participatory Budgeting","Equal Shares","AI Ethics","Voting","AI Decision Support","Collective Intelligence","Deep Reinforcement Learning"],"falsifier":"A direct test would re-run the same training pipeline on several additional participatory budgeting elections with different cost distributions and compare the learnt token allocations to voters' stated reasons for voting; if the cheap-project shift and the fairness improvement do not appear there, the claimed route to fair compromise would be a property of these two datasets rather than of participatory budgeting reward structures.","tokens_in":9763,"feed_emoji":"🗳️","tokens_out":7165,"duration_ms":64011,"temperature":0.7,"pith_summary":"This paper claims that modelling voters as self-interested reinforcement learning agents can reveal fair compromises in participatory budgeting that real elections miss. Applied to the 2023 Aarau and 2019 Toulouse elections, the trained agents produce winning project sets with lower inequality of satisfaction, higher average satisfaction, and in some cases a guarantee that every voter has at least one supported project in the winning set. The common pattern behind these gains is a learnt shift of voting tokens toward smaller-cost projects. If the claim holds, the model works as a decision-support tool for policymakers and voters, showing that fairer collective choices are achievable without changing the aggregation rule.","feed_headline":"Trained voter models find fairer budgets via cheaper projects","feed_subtitle":"In two real elections, token shifts toward cheap projects lowered inequality and raised satisfaction.","key_machinery":"The mechanism that carries the argument is a branching deep Q-network voter model paired with an impact-area reward function. Cumulative ballots with ten tokens across 33 projects produce roughly 1.5 billion possible ballots, so the branching architecture splits the action into ten parallel token-assignment heads of 33 actions each, shrinking the action space to 330 and letting 1703 independent learning agents train in the same election. Each agent's reward (Eq. 1) sums, over winning projects, the logarithm of project cost multiplied by the overlap between the voter's favoured impact areas and the project's impact areas and by the voter's token share on that project; this encodes both a preference for projects aligned with the voter's issues and a preference for larger projects. The trained policies are compared against the actual votes and against untrained random agents on project satisfaction, cost satisfaction, and budget share.","core_discovery":"The paper's central claim is that when voter preferences are formed from impact-area overlaps and voters learn to maximise their own reward, the resulting collective choices are fairer and more satisfying than the actual election outcomes. In both elections and under both equal-shares and utilitarian-greedy aggregation, trained agents lower the Gini coefficient of satisfaction and raise utilitarian welfare; in Toulouse the egalitarian welfare becomes non-zero, meaning every modelled voter had at least one project they supported in the winning set. The paper attributes these improvements to a learnt compromise: trained agents allocate a higher proportion of tokens to small- and medium-cost projects than the human voters did, and this shift is what enables the fairer outcome.","pith_inferences":["The lower-cost pattern may be specific to these two elections; re-running the pipeline on elections with very different project-cost distributions would show whether it is a general property of equal-shares and greedy aggregation or a dataset artifact.","Because the reward function's log-cost term directly rewards voting for expensive projects, an ablation that removes or alters that term would reveal whether the cheap-project shift is caused by the reward design rather than by the aggregation rules.","The preference assumption that every voter favours all impact areas of every project they voted for could be replaced with stated preferences or inverse reinforcement learning; the fairness comparison would then measure true voter welfare instead of a model proxy.","The action-branching training approach could transfer to other combinatorial collective decisions, such as participatory budgeting at city scale, provided the non-stationarity of thousands of learning agents remains manageable."],"forward_implications":["If the claim is correct, participatory budgeting elections can become fairer by including and promoting lower-cost projects, without altering the ballot aggregation rule.","Policymakers can use the model's what-if experiments to adjust election design, such as project selection or cost structure, toward outcomes that leave fewer voters unrepresented.","The branching architecture demonstrates that multi-agent reinforcement learning can scale to elections with more than a thousand voters, making simulation-based decision support feasible for real participatory budgeting.","Voters could be informed that shifting support toward cheaper projects increases the proportion of the budget that goes to the issues they care about.","A fairer outcome does not come at the cost of average satisfaction: trained agents raise utilitarian welfare in both elections and aggregation methods, even as the per-voter budget share decreases in some conditions."],"supporting_citations":[{"why":"Supplies the Aarau 2023 election dataset, the first participatory budgeting election using cumulative ballots and equal shares.","marker":"[1]"},{"why":"Provides the method of equal shares aggregation rule used for the Aarau model and fairness comparisons.","marker":"[18]"},{"why":"Supplies the Toulouse 2019 election dataset and the data format used for both elections.","marker":"[24]"},{"why":"Introduces the action branching architecture that the paper adapts to shrink the cumulative-ballot action space.","marker":"[16]"},{"why":"Provides the deep Q-network algorithm that the branching voter model extends.","marker":"[19]"},{"why":"Supplies the issue-voting model of preference formation that grounds the impact-area reward function.","marker":"[8]"},{"why":"Defines the egalitarian and utilitarian social welfare measures used to compare fairness across outcomes.","marker":"[25]"},{"why":"Establishes the impact-area formulation for participatory budgeting agents that this paper extends to fair-compromise analysis.","marker":"[12]"}],"fun_headline_variants":["AI voters learn to favor cheaper projects for fairer budgets","Multi-agent RL uncovers fair compromise in participatory budgeting","Deep RL shifts votes to low-cost projects, boosting fairness","Reinforcement learning finds fairer budgets via project cost shift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each voter's true preferences are adequately captured by assuming they favour every impact area of every project they voted for in the real election; if that assumption is wrong, the reported fairness and satisfaction gains measure the model's proxy for welfare rather than real voter welfare.","fun_headline_variants_meta":{"raw":{"variants":["AI voters learn to favor cheaper projects for fairer budgets","Multi-agent RL uncovers fair compromise in participatory budgeting","Deep RL shifts votes to low-cost projects, boosting fairness","Reinforcement learning finds fairer budgets via project cost shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1307,"prompt_tokens":845,"completion_tokens":462,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":461,"completion_tokens_details":{"reasoning_tokens":395}},"tokens_in":461,"tokens_out":462,"duration_ms":5445,"temperature":1.0,"reasoning_tokens":395,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:48:07.070291+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would re-run the same training pipeline on several additional participatory budgeting elections with different cost distributions and compare the learnt token allocations to voters' stated reasons for voting; if the cheap-project shift and the fairness improvement do not appear there, the claimed route to fair compromise would be a property of these two datasets rather than of participatory budgeting reward structures.","supporting_citations":[{"cited_title":"Yang, Fatemeh B","cited_arxiv_id":null,"evidence_quote":"Supplies the Aarau 2023 election dataset, the first participatory budgeting election using cumulative ballots and equal shares."},{"cited_title":"Proportional participatory budgeting with additive utilities","cited_arxiv_id":null,"evidence_quote":"Provides the method of equal shares aggregation rule used for the Aarau model and fairness comparisons."},{"cited_title":"Pabulib: A Participatory Budgeting Library","cited_arxiv_id":"2012.06539","evidence_quote":"Supplies the Toulouse 2019 election dataset and the data format used for both elections."},{"cited_title":"Action branching architectures for deep reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Introduces the action branching architecture that the paper adapts to shrink the cumulative-ballot action space."},{"cited_title":"Rusu, Joel Veness, Marc G","cited_arxiv_id":null,"evidence_quote":"Provides the deep Q-network algorithm that the branching voter model extends."},{"cited_title":"Issues, principles or ideology? how young voters decide","cited_arxiv_id":null,"evidence_quote":"Supplies the issue-voting model of preference formation that grounds the impact-area reward function."},{"cited_title":"Welfare engineering in multiagent systems","cited_arxiv_id":null,"evidence_quote":"Defines the egalitarian and utilitarian social welfare measures used to compare fairness across outcomes."},{"cited_title":"Consensus-based participatory budgeting for legitimacy: Decision support via multi-agent reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Establishes the impact-area formulation for participatory budgeting agents that this paper extends to fair-compromise analysis."}],"review_version":1}