{"id":"51ae623c-2621-4947-bb9f-129d195ecab0","arxiv_id":"2412.01864","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A Set Transformer learns participatory budgeting aggregation rules from labeled examples and can approximate compromise rules such as a 30-70 AV-CC blend close to PAV.","lead":"This paper trains a neural network to act as a participatory budgeting aggregation rule, learning from examples of votes and chosen projects. It shows the approach can imitate known rules and blend conflicting objectives, suggesting a way for communities to design rules by demonstration.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'closely resembles PAV' claim rests only on average welfare/representation ratios; no outcome-level similarity to PAV is reported, so the learned compromise rule may not actually be PAV-like.","rationale":"The paper's practical value rests on two claims: (1) it can learn existing rules, and (2) it can learn compromise rules, with the AV-CC-p to PAV result being the paper's own illustrative payoff. The first claim is reasonably supported by the reported RMSE and Jaccard tables. The second is the component that most needs scrutiny. The current evidence for 'AV-CC-0.3 resembles PAV' is the proximity of one blue dot and one orange dot to a red square in Figure 2, where axes are zoomed and no error bars or outcome-level statistics are given. Because the authors admit the outputs differ from PAV, the only factual content is that average welfare and representation ratios coincide. Many decision rules could share those averages while behaving very differently per instance. The reader's weakest assumption about synthetic-to-real transfer is valid, but it is a separate concern: even if the synthetic data were perfectly representative, the PAV-resemblance claim would still lack outcome-level verification. Hence I focus on the latter. The proposed Jaccard test would directly resolve whether the compromise rule is PAV-like; if it fails, the conclusion should be softened, and the paper's contribution would be reduced to learning rules that match aggregate tradeoffs rather than specific aggregation rules. Since the reader already issued a CONDITIONAL verdict, this concern does not change the verdict; it sharpens the condition under which the PAV claim should be accepted.","tokens_in":16424,"tokens_out":8194,"duration_ms":78154,"concrete_test":"On the UNIQUE test sets, for each instance compute the Jaccard similarity between the bundle selected by the trained AV-CC-0.3 model and the PAV-optimal bundle, and compare it with the same model's Jaccard similarity to AV-optimal and CC-optimal bundles and to a random feasible bundle. Report the mean and per-instance distribution. If the AV-CC-0.3-vs-PAV overlap is not significantly higher than the AV-CC-0.3-vs-AV and vs-CC overlaps, or is within random-bundle noise, then the claim should be weakened to 'matches PAV's average welfare-representation tradeoff,' not 'closely resembles PAV.' Additionally, bootstrap the Figure 2 averages to check that the p=0.3 point is statistically distinct from neighboring p values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline contribution in Section 7 ('a 30%-70% blend produces a rule that closely resembles PAV') is supported only by the average welfare and representation ratios plotted in Figure 2. The paper explicitly disclaims that AV-CC-0.3 'does not mean that those two rules give the same outcome as PAV,' yet 'closely resembles' is used as a data-driven rationale for PAV. Average ratios can be matched by rules that select very different bundles; a rule is PAV-like only if its chosen outcomes are substantially aligned with PAV's. No outcome-level comparison (Jaccard, rank correlation, or overlap of funded bundles) between AV-CC-0.3 and PAV is reported. The RMSE tables in Section 6 are about matching AV/CC/PAV optima, not about comparing the learned compromise rule to PAV. Without outcome-level evidence, the PAV-resemblance claim is an unsupported extrapolation from aggregate statistics, and the 'closed-formula rationale' is an empirical fit, not a derivation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a neural-network framework for learning participatory budgeting (PB) aggregation rules from labeled examples. The authors adapt the Set Transformer architecture by moving the pooling-by-multihead-attention (PMA) layer before the encoder, so that the network input is a fixed-size project representation regardless of the number of voters, and they introduce implicit (cross-entropy to an optimal bundle) and explicit (surrogate losses for welfare and representation) training modes. They evaluate the approach on synthetic datasets (Euclidean, One-vs-Many, Trade-Off AV, Trade-Off CC) and on real Warsaw PB data (2017–2021), reporting that the ST+PMA model learns AV, CC, and PAV rules on 'unique' datasets with low RMSE and generalizes to larger instances. The paper also learns mixture rules AV-CC-p and claims that a 30%-70% blend of AV and CC examples produces a rule that 'closely resembles' PAV, presented as a data-driven and closed-form rationale for PAV's welfare-representation balance.","tokens_in":16626,"tokens_out":7004,"duration_ms":63405,"significance":"If the results hold, the paper makes a useful contribution to data-driven social choice: it demonstrates that a reasonably simple neural architecture can learn standard PB rules from example optimal outcomes and can scale to the voter counts of real PB instances, which prior work on learning voting rules did not achieve for the PB setting. The architectural change (PMA before the encoder) is a practical engineering contribution, and the mixture-learning experiment, if properly supported, would offer an interesting perspective on the relationship between AV, CC, and PAV. The paper is honest about two limitations (black-box explainability and lack of worst-case guarantees), and the experimental setup is described in enough detail (dataset generators, hyperparameters, tie analysis) to be replicable. However, the central quantitative claims need strengthening: the headline RMSE tables lack uncertainty quantification, one of the four described model variants is omitted from results, and the PAV-resemblance claim is supported only by aggregate welfare/representation ratios rather than outcome-level comparisons.","major_comments":[{"comment":"The claim that AV-CC-0.3 'closely resembles' PAV and offers a 'data-driven rationale' for PAV is supported only by average welfare and representation ratios. The manuscript itself states that this 'does not mean that those two rules give the same outcome as PAV,' and average ratios can be matched by rules that select very different bundles. To make the resemblance claim load-bearing, the paper should report outcome-level similarity metrics (e.g., Jaccard overlap or rank correlation between funded bundles of AV-CC-0.3 and PAV) with confidence intervals, or explicitly weaken the claim to state that the average welfare/representation tradeoff of PAV is reproduced by the mixture on the evaluated datasets. In addition, the phrase 'closed-formula rationale' is misleading: the orange points in Figure 2 come from the ILP with objective p·welfare+(1−p)·representation, which is not the PAV score; the coincidence at p=0.3 is an empirical observation on these datasets, not a derivation.","section":"Section 7, Figure 2; Contribution (4)"},{"comment":"The RMSE values in Tables 2 and 3 are computed from dataset-level average welfare and representation ratios, with no error bars, confidence intervals, or repeated-seed information. The appendix tables show high per-instance variance for several cells, e.g., Table 5 reports OVM small AV implicit representation ratio 0.824 ± 0.249 and TOCC small CC implicit welfare ratio 0.095 ± 0.103; these large standard deviations mean that the aggregate RMSE may be dominated by a few instances or runs. The paper should report bootstrap intervals over instances and multiple training seeds, or otherwise quantify the uncertainty of the RMSE claims.","section":"Section 6, Tables 2 and 3; Appendix Tables 4 and 5"},{"comment":"The paper states that 'for brevity' it does not report the ST model with explicit learning because 'its performance was like that of Random.' This is a negative result for one of the four model variants the paper claims to evaluate, and its omission weakens the comparison, especially because the explicit method shows different behavior on TIED datasets. Please include the ST-explicit results (at least in the appendix) or provide a concrete summary of its performance, so readers can verify the claim and understand why this variant fails.","section":"Section 6, first paragraph"},{"comment":"The generalization claim ('able to generalize from small-scale synthetic PB examples to large, real-world PB instances') rests on a single real-world collection (Warsaw) and on synthetic generators whose parameters are not varied in a sensitivity analysis. The paper should add cross-generator transfer experiments (e.g., train on EUC and test on OVM or TOCC, and vice versa) and, if feasible, evaluate on additional Pabulib datasets, to assess how sensitive the learned rules are to distribution shift. Without such experiments, a reader cannot tell whether the success on Warsaw reflects general representativeness of the synthetic benchmark or an accidental match to that particular city's instances.","section":"Section 5 and Appendix 1.A; Section 6.2"}],"minor_comments":[{"comment":"The caption says 'The first four points from the left correspond to p = 0, 0.01, 0.04, 0.8'; given the discussion of p ≤ 0.1, '0.8' is likely a typo for '0.08'. Please correct.","section":"Figure 2 caption"},{"comment":"The text 'Alongside AC-CC-p' should be 'Alongside AV-CC-p' to match the notation used everywhere else.","section":"Section 7"},{"comment":"The explicit representation loss uses a shifted sigmoid with parameters a=3 and b=6 that were chosen 'after testing a few candidates.' Please clarify how these values were selected and whether the results are robust to their choice, since the loss is an approximation of the true representation score.","section":"Section 4.2"},{"comment":"There are several typos and spacing inconsistencies, e.g., 'T rade-Off A V', 'TOA V', 'optimal soultions', 'mult-winner', and 'it possible possible to use it' in Section 8. A careful proofreading pass is needed.","section":"Section 5 and Appendix 1.A"},{"comment":"The notation introduces [k] but does not use it consistently; please either use it in the definitions of the scores or remove it.","section":"Section 1.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's core pipeline—learning AV/CC/PAV on UNIQUE datasets with ST+PMA—is reasonably supported and the real-data test is a plus. The main risk is overclaiming in Section 7: the 'closely resembles PAV' statement and the 'closed-formula rationale' are not backed by outcome-level evidence, and the missing uncertainty quantification makes the quantitative comparisons hard to interpret. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection. The omission of ST-explicit results is also a completeness issue that should be addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core contribution is real: the paper extends Anil and Bao's Set Transformer approach to PB, handling project costs and budgets, and the PMA-before-encoder change is a sensible, effective way to scale to many voters. On the UNIQUE datasets the learned rules match AV, CC, and PAV with low RMSE, and the method generalizes from small synthetic instances to real Warsaw data. That looks like solid work, although code and data are not released, which matters for verification. The mixture experiment is genuinely new: training on a blend of AV and CC examples yields a compromise rule, and the p=0.3 point landing near PAV's average welfare and representation ratios is a noteworthy observation.\n\nThe soft spots are local, not fatal. The stress-test is right that the 'closely resembles PAV' claim rests only on average ratios. Two rules can have identical average welfare and representation while selecting very different bundles, and the paper itself admits that AV-CC-0.3 does not mean the same outcomes as PAV. So the phrasing is an overstatement; it should say 'has a similar average welfare-representation trade-off.' Adding Jaccard or bundle-overlap statistics against PAV would settle this. Second, the headline RMSE tables in Section 6 have no error bars, and the text says the ST-explicit model performed like Random but gives no data. A reviewer should ask for that model to be either shown or explicitly dropped. Third, all training data is synthetic, and there is no sensitivity analysis of the generation parameters; the single real-world transfer test helps but does not fully address deployment risk. These are legitimate concerns, but they do not undermine the main learning result.\n\nNo circularity: this is supervised learning with known labels and held-out test data. The AV-CC-p finding is post-hoc, so it is not circular either, just under-supported as a claim about PAV's rationale.\n\nWho is this for? Computational social choice researchers interested in data-driven rule design, and ML people working on permutation-invariant networks for structured outputs. It deserves a serious referee. Recommendation: send it to peer review with a request for outcome-level comparison for the PAV claim, error bars on the main tables, and ideally code and data.","headline":"A genuinely useful extension of Set Transformer learning to participatory budgeting, with a PAV-resemblance claim that overstates what average ratios can support.","tokens_in":17137,"tokens_out":2131,"would_cite":true,"duration_ms":21270,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","91B12","91B14"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that a neural network can learn participatory budgeting aggregation rules from example votes and outcomes, reproducing standard rules and recovering Proportional Approval Voting as a 30/70 compromise between welfare and…","keywords":["participatory budgeting","aggregation rules","neural networks","Set Transformer","proportional approval voting","social welfare","representation","compromise rules"],"falsifier":"Run the network trained on synthetic data on published participatory budgeting elections from several cities other than Warsaw, comparing the welfare and representation ratios of its chosen bundles with those of the rule it was trained to imitate; if the ratios fall well short on average, or if the 30/70 AV/CC rule's average ratios on a fresh, diverse set of real instances diverge from Proportional Approval Voting's by more than the variance reported in the paper, the transfer and compromise claims would be falsified.","tokens_in":16208,"feed_emoji":"🗳️","tokens_out":9038,"duration_ms":76291,"temperature":0.7,"pith_summary":"The paper claims that participatory budgeting rules do not have to be hand-crafted formulas: a neural network can learn the rule from examples of votes, project costs, and chosen outcomes. Trained on small synthetic instances, the network reproduces the behavior of Approval Voting, Chamberlin–Courant, and Proportional Approval Voting, and it keeps that accuracy when applied to real Warsaw municipal budgets with thousands of voters. When the training data blends 30 percent AV outcomes with 70 percent CC outcomes, the learned rule lands on exactly the welfare-versus-representation trade-off that Proportional Approval Voting achieves, which the authors offer as a data-driven rationale for PAV's compromise position.","feed_headline":"Neural networks can learn participatory budgeting rules","feed_subtitle":"Trained on example votes, it reproduces standard rules and recovers Proportional Approval Voting as a 30/70 compromise.","key_machinery":"The load-bearing component is the ST+PMA architecture: a Set Transformer in which a pooling-by-multihead-attention (PMA) layer is moved from the decoder to before the encoder, compressing the voter-by-project approval matrix into a project-dimensional latent space so that the number of voters no longer affects the network's input size. Project costs, normalized so the budget is 1, are appended to this latent representation, and the network outputs a sigmoid score per project from which a feasible bundle is selected greedily. Training runs in two modes: implicit, where the cross-entropy loss compares the network's bundle to an optimal bundle from the target rule, and explicit, where the loss directly optimizes a smooth approximation of the welfare or representation score—the explicit mode being designed for datasets with many tied optima.","core_discovery":"The central discovery is that an aggregation rule for participatory budgeting can be embedded directly into a neural network's weights and then decoded on new, larger instances. Using a Set Transformer with a pooling-by-multihead-attention layer placed before the encoder, the network's input size depends only on the number of projects, not the number of voters, allowing training on instances with 10–900 voters and transfer to instances with up to 50,000. On the UNIQUE datasets, the learned rule attains total RMSE between 0.013 and 0.076 in the welfare-representation space, well below the 0.168–0.242 of sequential baselines, and it runs on the real-world Warsaw instances where the plain architecture exhausts GPU memory. The new observation is that a rule trained on a blend of 30% AV outcomes and 70% CC outcomes reproduces PAV's average welfare and representation ratios, giving a data-driven and closed-form rationale for PAV's balance of welfare and representation.","pith_inferences":["If the approach generalizes beyond Warsaw, it suggests a paradigm in which aggregation rules are validated empirically by behavior rather than derived axiomatically; one could test whether other known rules, such as the Method of Equal Shares, are also recovered by particular AV/CC mixture levels.","The paper's own caution that the network is a black box with no worst-case guarantees implies the safest deployment is as a decision-support tool for ordinary PB, with human veto, rather than as the final authority on public funds.","The p=0.3 coincidence invites a direct experiment: train networks on AV/CC mixtures with p ranging from 0 to 1 in fine increments and check whether the welfare-representation curve is exactly the PAV curve at any other p, or whether 0.3 is special.","A natural extension the paper leaves implicit is to train on mixtures of PAV and another proportional rule to see whether the learned compromise outperforms both parents on proportionality axioms such as justified representation."],"forward_implications":["PB organizers could define a desired rule by supplying examples of outcomes, without knowing or stating the rule's formula.","Rules learned on small synthetic instances transfer to large real elections, making the approach practical for municipal budgeting with tens of thousands of voters.","The same trained network can be fine-tuned as objectives shift, so a community's aggregation rule can evolve without redesigning a rule from scratch.","A compromise between competing objectives can be produced by mixing examples from each objective; the 30/70 AV/CC mix specifically recovers PAV's welfare-representation trade-off.","Because inference with the network is fast, the learned rule is usable in real-time settings such as multi-winner voting in games or sensor-network decisions."],"supporting_citations":[{"why":"supplies the multi-winner Set Transformer architecture that this paper adapts to participatory budgeting by moving the PMA layer and adding normalized project costs.","marker":"[1]"},{"why":"defines the Chamberlin–Courant representation score that serves as one of the target rules and one component of the compromise mixture.","marker":"[6]"},{"why":"provides the Trade-Off AV and Trade-Off CC datasets used to stress-test learning under many tied optimal outcomes.","marker":"[11]"},{"why":"the hand-crafted scoring-rule approach the paper contrasts with, motivating the data-driven alternative.","marker":"[12]"},{"why":"introduces the Set Transformer and its pooling-by-multihead-attention mechanism that the architecture builds on.","marker":"[19]"},{"why":"supplies the definition of the Proportional Approval Voting score that the learned 30/70 AV/CC blend is compared against.","marker":"[21]"},{"why":"provides the real-world Warsaw 2017–2021 participatory budgeting instances used as the transfer test set.","marker":"[30]"}],"fun_headline_variants":["AI learns participatory budgeting rules from votes","Neural nets decode aggregation rules for budget votes","Machine learning finds new voting rules for budgeting","Data-driven discovery of participatory budgeting rules","30/70 blend yields PAV rule in learned budgeting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The learned rules are trained on synthetic instances from four stylized generative models, so the approach rests on these models mimicking the approval patterns, project costs, and project interactions of real participatory budgeting settings where the rule would be deployed.","fun_headline_variants_meta":{"raw":{"variants":["AI learns participatory budgeting rules from votes","Neural nets decode aggregation rules for budget votes","Machine learning finds new voting rules for budgeting","Data-driven discovery of participatory budgeting rules","30/70 blend yields PAV rule in learned budgeting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000571,"raw_usage":{"total_tokens":2678,"prompt_tokens":899,"completion_tokens":1779,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":1711}},"tokens_in":515,"tokens_out":1779,"duration_ms":11944,"temperature":1.0,"reasoning_tokens":1711,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:51:02.919409+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the network trained on synthetic data on published participatory budgeting elections from several cities other than Warsaw, comparing the welfare and representation ratios of its chosen bundles with those of the rule it was trained to imitate; if the ratios fall well short on average, or if the 30/70 AV/CC rule's average ratios on a fresh, diverse set of real instances diverge from Proportional Approval Voting's by more than the variance reported in the paper, the transfer and compromise claims would be falsified.","supporting_citations":[{"cited_title":"Advances in Neural Information Processing Systems 34, 8006–8017 (2021)","cited_arxiv_id":null,"evidence_quote":"supplies the multi-winner Set Transformer architecture that this paper adapts to participatory budgeting by moving the PMA layer and adding normalized project costs."},{"cited_title":"American Political Sci- ence Review 77(3), 718–733 (1983)","cited_arxiv_id":null,"evidence_quote":"defines the Chamberlin–Courant representation score that serves as one of the target rules and one component of the compromise mixture."},{"cited_title":"representation in participatory budgeting","cited_arxiv_id":null,"evidence_quote":"provides the Trade-Off AV and Trade-Off CC datasets used to stress-test learning under many tied optimal outcomes."},{"cited_title":"Collective Decisions: Theory, Algorithms And Decision Support Systems pp","cited_arxiv_id":null,"evidence_quote":"the hand-crafted scoring-rule approach the paper contrasts with, motivating the data-driven alternative."},{"cited_title":"In: Inter- national conference on machine learning, pp","cited_arxiv_id":null,"evidence_quote":"introduces the Set Transformer and its pooling-by-multihead-attention mechanism that the architecture builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the definition of the Proportional Approval Voting score that the learned 30/70 AV/CC blend is compared against."}],"review_version":1}