{"id":"dbe6a00d-16de-449d-aa2a-d21b80ac296f","arxiv_id":"2607.18296","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A benchmark of four deep RL algorithms on Baghchal finds MuZero strongest and PPO most practical, but the reported per-role win rates are incompatible with the stated self-play protocol.","lead":"This paper trains DQN, REINFORCE, PPO, and MuZero on the asymmetric Nepali board game Baghchal and reports that MuZero wins most often in both roles, while PPO offers competitive results at lower compute. The headline win rates are internally inconsistent with the stated self-play evaluation setup, so the comparison is not yet trustworthy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II win rates violate zero-sum arithmetic: MuZero Tiger+Goat+Draw = 157%, so the evaluation protocol is undefined or misreported; headline comparison unsupported.","rationale":"I agree with the reader's weakest assumption. The internal inconsistency is not a subtle modeling choice but a violation of the zero-sum game structure. The paper's own Section III emphasizes self-play training, but evaluation is not described. The reported aggregate rates cannot be produced by any self-play evaluation where the same agent takes both roles. If the evaluation used different opponents per role, this must be stated; otherwise the comparison is meaningless. This alone justifies rejection. I also note the paper does not release code/data, so the check cannot be performed post hoc. The concrete test would settle whether the numbers are transcription errors or a deeper protocol flaw. No other concern is as decisive.","tokens_in":9028,"tokens_out":3347,"duration_ms":33018,"concrete_test":"Compute, for each row in Table II, S = Tiger Win + Goat Win + Draw. If S ≠ 100%, the rates cannot derive from a single paired evaluation. Then inspect the MuZero match log: for each episode, record the winner (Tiger, Goat, or Draw). Verify that exactly one side wins per game. If the log is unavailable, request the authors to release it; without it, Table II is not reproducible. A minimal independent check: train MuZero by self-play for 100k episodes, then play 1000 self-play games and compute role win rates; the sums must equal 100%.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—MuZero achieves the best win rates in both roles—rests on Table II. In a two-player zero-sum game, for any single evaluation setting, Tiger win rate + Goat win rate + draw rate must equal 100%. Yet the reported sums are: DQN 126%, REINFORCE 126%, PPO 145%, MuZero 157%. MuZero's 86% Tiger and 62% Goat with 9% draws is arithmetically impossible. The paper never specifies the evaluation opponent or whether the role win rates come from the same matches. The stated self-play methodology (Section III) implies each agent plays both roles against itself, in which case the two win rates are complements; they cannot both be high. Unless the authors define a coherent protocol (e.g., each role tested against a different baseline), the numbers cannot be interpreted, and the ranking of algorithms is unverifiable. This is a load-bearing flaw because it undercuts the paper's headline result before any algorithmic comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical comparison of four deep reinforcement learning algorithms—DQN, REINFORCE, PPO, and MuZero—applied to Baghchal, a two-player zero-sum asymmetric board game. The agents are trained via self-play and evaluated on both roles. The reported results claim that MuZero achieves the highest win rates in both roles (86% as Tiger, 62% as Goat), that PPO offers the best performance-to-computational-cost tradeoff, and that model-based planning via MCTS explains MuZero's superiority. Additional analyses cover training convergence, computational cost, and qualitative strategic behavior.","tokens_in":9209,"tokens_out":5109,"duration_ms":55219,"significance":"If its central claims were valid, the paper would contribute a useful benchmark for deep RL in asymmetric two-player games, an underexplored area; it also documents computational-resource comparisons and highlights a culturally significant game. The paper is one of the first to apply modern deep RL algorithms (including MuZero) to Baghchal, and the inclusion of training-time and inference-cost measurements is a strength. However, the headline quantitative result is undermined by an internally inconsistent table and an underspecified evaluation protocol, so the main claims cannot be accepted as presented.","major_comments":[{"comment":"The reported win rates violate the zero-sum arithmetic of Baghchal. Under any single evaluation setting, Tiger win rate + Goat win rate + draw rate must equal 100%. The table reports sums of 126% (DQN), 126% (REINFORCE), 145% (PPO), and 157% (MuZero). The MuZero row alone—86% Tiger, 62% Goat, 9% draw—is arithmetically impossible if the two role win rates come from the same matches. The paper never states whether the role win rates are measured against the same opponent, different opponents, or different evaluation protocols. Consequently the central claim in the abstract and Section IV.A that “MuZero achieves the highest performance in both roles” is unsupported by the data as reported.","section":"Section IV.A, Table II"},{"comment":"The evaluation protocol is not defined. The paper says each agent is trained by self-play over 100,000 games and that performance is assessed every 1000 games, but it does not state who the final trained agent is evaluated against, how many evaluation games were played, how draws are scored, how the per-role win rates are combined, or how the reported ± intervals are computed across seeds. Without this information the results in Tables II–IV are not reproducible, and the cross-algorithm ranking cannot be verified.","section":"Section III.D and Section IV.A"},{"comment":"The attribution of MuZero's performance to “model-based planning via Monte Carlo Tree Search” is not supported by a controlled comparison. The algorithms differ in many respects beyond the presence of a learned model and MCTS, including network architecture, target networks, GAE, entropy regularization, replay mechanism, and compute budget. No ablation isolates MCTS or the learned model. The claim in Section III.A.4 that MuZero can “look 8-10 moves ahead” is asserted without any measured search-depth or tree-statistics data; it appears to be an interpretation rather than a measured result.","section":"Abstract and Section IV.A"},{"comment":"The hand-designed reward structure is role-asymmetric (e.g., +10 capture reward for Tiger, +5 survival and +10 strategic-positioning rewards for Goat, −1 per-move penalty for Tiger). Since all algorithms are optimized under these rewards, the reward schedule itself may create the apparent differences in role difficulty and algorithmic preference. The paper reports no sensitivity analysis or reward-calibration checks, so the conclusion in Section IV.E that value-based methods are “biased toward the Tiger role” is confounded by the reward design.","section":"Table I"}],"minor_comments":[{"comment":"The displayed equations are garbled: symbols such as “�” replace mathematical notation, and Eq. (2) uses undefined notation ($G_t$, $b$, etc. are partially illegible). This makes the algorithm descriptions hard to follow.","section":"Section III.A"},{"comment":"The training description says agents are trained for 100,000 self-play games, while Table III reports convergence at 25,000–45,000 episodes. The relationship between these numbers is unclear; please clarify whether 100,000 is the maximum training budget or the total number of episodes.","section":"Section III.D"},{"comment":"The abstract states that algorithms are “trained on one side ... and evaluated on the other side,” while Section III says each program learns by playing against itself and is tested as both Tiger and Goat. These statements are inconsistent and should be reconciled.","section":"Abstract and Section III"},{"comment":"The manuscript references several figures (Fig. 1, 3, 4, 5, 6) but the provided text does not include the images. Ensure that all figures are embedded and legible in the submission.","section":"Figures"},{"comment":"The conclusion includes claims not derived from the reported experiments, such as the agent “doesn't really work out what the other player is doing” and “doesn't get harder ... depending on the human player.” These are limitations of the current system but are not measured or discussed in the results section.","section":"Section V"},{"comment":"The related-work discussion mentions prior Baghchal work by Luitel et al. and Thapa & Poudel, but no quantitative comparison with their reported results is given. Adding such a comparison would strengthen the paper.","section":"Section II"}],"recommendation":"reject","confidential_remarks":"The paper addresses an interesting niche and includes useful resource-usage measurements, but the central quantitative result in Table II is internally inconsistent under the stated self-play protocol. The evaluation opponent and protocol are not specified, making the headline ranking uninterpretable. This is a load-bearing flaw that cannot be fixed by local edits; a clarified and corrected evaluation, possibly with new experiments, is required. If the authors can provide that, a resubmission could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely new: it applies four deep RL families (DQN, REINFORCE, PPO, MuZero) to Baghchal, an asymmetric Nepali board game that the deep RL literature has mostly ignored. The environment design, the two-phase game structure, and the reward tables are clearly described, and the computational cost comparison (training time, memory, inference) is more than most RL papers bother to report. If the results held, this would be a useful empirical benchmark for asymmetric multi-agent learning.\n\nBut the central result does not hold. Table II reports, for MuZero, an 86% Tiger win rate, a 62% Goat win rate, and a 9% draw rate. In a two-player zero-sum game these three numbers must sum to 100% if they come from the same set of matches. 86 + 62 + 9 = 157. The same problem appears for every algorithm: DQN sums to 126%, REINFORCE to 126%, PPO to 145%. This is not a rounding issue; it means the evaluation protocol is either badly misreported or not defined at all. The paper never states who the opponent is during evaluation, how many games were played, or whether the Tiger and Goat win rates are measured against the same fixed baseline or against self-play. The methodology section says the agents train by self-play, but the reported numbers cannot come from self-play unless the two win rates are complements, which they are not.\n\nBecause of this, the headline claim that MuZero is best in both roles is uninterpretable. A reader cannot tell whether MuZero is beating a weak fixed opponent, whether the win rates come from different evaluation settings, or whether the table is simply wrong. The paper's secondary claims—PPO as a practical compromise, DQN's tiger bias—are also built on this broken table. The strategic behavior analysis is mostly anecdotal, with no systematic metrics or ablations to back up statements about planning depth.\n\nCredit where due: the gap in the literature is real, the game choice is culturally significant, and the authors have made a careful attempt at environment engineering. The writing is rough in places, but that is minor compared to the evaluation flaw.\n\nThis paper is not ready for peer review in its current form. The authors need to specify the evaluation protocol precisely, provide the code and data, and re-report the win rates so they are internally consistent. If they do that, the comparison could be a useful contribution. As it stands, a serious referee would have to reject it, and a desk reject is defensible.","headline":"First deep RL comparison on Baghchal, but the headline win rates are arithmetically impossible and the evaluation protocol is undefined.","tokens_in":9750,"tokens_out":1877,"would_cite":false,"duration_ms":20202,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MuZero, a model-based planner, outperforms three deep-RL algorithms on both roles in Baghchal.","keywords":["Baghchal","deep reinforcement learning","asymmetric games","MuZero","Monte Carlo Tree Search","PPO","role asymmetry","zero-sum board games"],"falsifier":"Take the trained MuZero agent and play it 100 games as Tiger and 100 games as Goat against the same fixed baseline (e.g., the best PPO agent or a scripted rule-based player), recording wins, losses, and draws. If the two per-role win rates plus the draw rate do not sum to roughly 100%, the headline ranking is not measuring what it claims.","tokens_in":8880,"feed_emoji":"🐅","tokens_out":3388,"duration_ms":31837,"temperature":0.7,"pith_summary":"The paper attempts to establish that modern deep reinforcement learning can handle asymmetric two-player board games, where the two sides have different pieces, moves, and win conditions. Using the Nepali game Baghchal as a testbed, the authors train DQN, REINFORCE, PPO, and MuZero by self-play, letting each agent learn both the Tiger and Goat roles. They report that MuZero achieves the highest win rates in both roles (86% as Tiger, 62% as Goat) and attribute this to its learned model and Monte Carlo Tree Search planning. PPO is presented as the best cost-performance compromise, while DQN is shown to favor the reward-rich Tiger role. If true, the result would show that model-based planning is especially valuable in asymmetric games with sparse, long-horizon rewards.","feed_headline":"MuZero beats three deep-RL rivals as both Tiger and Goat in Baghchal","feed_subtitle":"Model-based planning wins in an asymmetric Nepali board game; PPO offers nearly as strong play at a fraction of the compute.","key_machinery":"The load-bearing mechanism is MuZero's combination of a learned latent world model and Monte Carlo Tree Search (MCTS) guided by the PUCT rule, which lets the agent evaluate hypothetical futures without knowing the game rules. The authors also rely on a shared self-play training protocol for both roles, a custom PyTorch Baghchal environment with legal-move masking, and a reward table that gives the Tiger immediate capture rewards while the Goat receives sparse positional rewards.","core_discovery":"The paper's central claim is that MuZero, which plans up to 8–10 moves ahead through a learned latent model, learns the strongest play on both sides of Baghchal, exceeding the model-free baselines DQN, REINFORCE, and PPO. The reported numbers are an 86% win rate as the Tiger and 62% as the Goat, with a 9% draw rate. The authors attribute this to model-based planning via Monte Carlo Tree Search, and further claim that PPO offers a competitive alternative at a fraction of the computational cost, and that value-based DQN develops a strong bias toward the Tiger role because capture rewards are dense and immediate.","pith_inferences":["If the reported per-role win rates were measured against the same fixed opponent, they would be mutually inconsistent in a zero-sum game (86% Tiger + 62% Goat + 9% draw exceeds 100%), so the reader should ask what opponent each win rate is measured against before accepting the ranking.","A direct extension would be to train a single MuZero agent and test it against a fixed, human- or scripted-baseline opponent on both roles, yielding interpretable per-role numbers; the paper's self-play protocol leaves this unspecified.","The authors' own conclusion notes the agents do not model the opponent's reasoning and only learn via self-play; an interesting follow-up would be to test whether opponent-aware methods (e.g., MADDPG) improve Goat coordination, as the paper itself suggests."],"forward_implications":["If MuZero's margin over model-free methods is real, it implies that long-horizon planning is a decisive factor in asymmetric board games, where one side faces delayed and sparse rewards.","PPO's near-MuZero performance at roughly one quarter of the training time and far lower memory suggests that actor-critic methods are the pragmatic choice for resource-constrained settings.","The consistent Tiger advantage across all algorithms indicates that role asymmetry itself, not algorithm choice, drives much of the difficulty; the Goat's coordination problem is the harder learning task.","Baghchal can serve as a compact testbed for asymmetric multi-agent RL, complementing standard symmetric benchmarks like chess and Go.","The observed emergent strategies—pincer attacks, sacrifice, and tempo control for the Tiger; mobile blocking formations for the Goat—suggest that deep RL can rediscover human-style play in a culturally specific game."],"fun_headline_variants":["MuZero tops DQN, REINFORCE, PPO as both sides in Baghchal","MuZero wins 86% as Tiger, 62% as Goat in Baghchal","Model-based MuZero beats value-based DQN in Baghchal","Baghchal: MuZero wins both roles, PPO nearly as strong","MuZero bests three RL baselines in asymmetric Baghchal"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The evaluation protocol yields comparable per-role win rates; the paper never states who the opponent is during testing, and the reported Tiger, Goat, and draw rates (86%, 62%, 9%) cannot all hold against a single fixed opponent in a zero-sum game.","fun_headline_variants_meta":{"raw":{"variants":["MuZero tops DQN, REINFORCE, PPO as both sides in Baghchal","MuZero wins 86% as Tiger, 62% as Goat in Baghchal","Model-based MuZero beats value-based DQN in Baghchal","Baghchal: MuZero wins both roles, PPO nearly as strong","MuZero bests three RL baselines in asymmetric Baghchal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000385,"raw_usage":{"total_tokens":1891,"prompt_tokens":781,"completion_tokens":1110,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":1012}},"tokens_in":525,"tokens_out":1110,"duration_ms":8903,"temperature":1.0,"reasoning_tokens":1012,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T08:58:21.567042+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained MuZero agent and play it 100 games as Tiger and 100 games as Goat against the same fixed baseline (e.g., the best PPO agent or a scripted rule-based player), recording wins, losses, and draws. If the two per-role win rates plus the draw rate do not sum to roughly 100%, the headline ranking is not measuring what it claims.","supporting_citations":[],"review_version":1}