{"id":"df4f6d6d-d464-475f-9990-dcf6bfa98891","arxiv_id":"2608.06892","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A reinforcement-learning agent trained to schedule inter-QPU communication in distributed quantum circuits matches heuristic compilers on structured benchmarks, with small gains from lookahead rewards on random circuits.","lead":"Distributed quantum computers need compilers that decide when to move quantum information between connected processing units, and this paper trains a reinforcement-learning agent to make those decisions. The agent matches hand-built heuristic compilers on structured circuit families, though it falls behind on dense circuits and remains limited to small systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The EPR-efficiency claim rests on equating EPR cost with split count; if merges also consume entanglement, Table III and Fig. 5 do not support the central claim.","rationale":"After stress-testing the central claim, I converge with the reader's weakest-assumption analysis. Table III is the only quantitative support for 'matches state-of-the-art heuristics,' and it is expressed entirely in split counts. The paper's statement in Section III-B that EPR count is equivalent to split count is an assumption about the split/merge primitives, not a derived fact. Since the cited protocols for these primitives (Refs. [31], [33]) are entanglement-assisted in both directions, a missing merge cost could change the comparison; the absence of merge counts means no reader can verify this from the current data. I did not find a separate internal inconsistency that would overturn the framework's contribution: the MDP/PPO formulation is coherent, the heterogeneous graph encoding is detailed, and the claim is scoped to specific circuit families. However, the empirical claim is only as strong as the cost model, so the paper should remain conditional, with the Bell-pair count check requested above. The lack of error bars and per-seed variance (also noted by the reader) compounds the issue but is secondary to the metric-validity concern.","tokens_in":13133,"tokens_out":10110,"duration_ms":118544,"concrete_test":"Use the actual split/merge circuit decompositions from Ref. [33] (or [31]) to count Bell pairs consumed by both operations, then rerun the Table III benchmarks for Full RL, DISQCO, and Pytket-DQC, logging both split and merge counts for every compiled circuit. Recompute EPR cost as splits plus merges (or the measured Bell-pair count per plan) and compare the rankings on QAOA, QFT, Random, and Quantum Volume. If Full RL no longer matches or beats the heuristics on the structured benchmarks after including merge cost, the central claim fails; if the ranking is unchanged and merges are verified entanglement-free, the abstraction is sufficient for the paper's conclusion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is that the RL policy 'matches state-of-the-art heuristics' on communication cost, measured as the number of split operations (Section V-C, Table III). Section III-B defines the optimization objective as EPR-pair consumption and asserts that this is 'equivalent to the number of split operations under our communication model.' However, the split and merge primitives in Section II-A are the entanglement-assisted cat-entangler/disentangler operations of Refs. [31] and [33], and in those protocols the reverse operation (merge/disentangle) is not obviously entanglement-free. If a merge consumes an EPR pair, or if a qubit must be split again after a merge, then the true EPR cost is at least splits plus merges, and the reward r_split in Eq. (23) optimizes the wrong quantity. The paper never reports merge counts, so the comparisons against DISQCO and Pytket-DQC could be reversed by a missing term. This is load-bearing: the abstract's headline is about EPR-pair consumption, but the evidence in Table III and Figure 5 is about split operations only.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a reinforcement learning (RL) framework for distributed quantum circuit compilation. It formulates compilation as a constrained Markov Decision Process (MDP) over split and merge communication primitives, uses a heterogeneous graph to represent qubits, QPUs, and gates, and trains a policy with Proximal Policy Optimization. The policy is evaluated on QAOA, QFT, random, and quantum volume circuits against the Pytket-DQC and DISQCO heuristics, with the number of split operations used as the proxy for EPR-pair consumption. The authors report that the RL policy matches the heuristics on structured circuits (QAOA, QFT) and gives modest improvements on random circuits, while remaining clearly worse on quantum volume circuits. The paper claims that the framework is a flexible alternative to manual heuristics, with scalability identified as a remaining limitation.","tokens_in":13344,"tokens_out":7196,"duration_ms":73361,"significance":"If the empirical results are robust, the paper makes a meaningful contribution to distributed quantum compilation by demonstrating that an RL policy with graph-based state encoding can discover communication strategies comparable to specialized heuristics. The constrained-MDP formulation and the heterogeneous graph representation are carefully designed and constitute a solid methodological template. The paper does not rely on fitted constants for its central claim; the evaluation is a direct empirical comparison. However, the significance is mitigated by the lack of statistical rigor, the unvalidated equivalence between EPR cost and split count, and the large performance gap on quantum volume circuits. The framework is a promising approach, but the evidence presented does not yet fully support the strength of the stated claims.","major_comments":[{"comment":"The optimization objective in Section III-B equates EPR-pair consumption with the number of split operations (\"the number of EPR pairs consumed, which is equivalent to the number of split operations under our communication model\"). This equivalence is asserted without supporting justification. The merge/disentangler primitive introduced in Section II-A is not described as entanglement-free, and in the cited protocols (Refs. [31], [33]) the reverse operation is not explicitly cost-free in EPR pairs. If merges also consume entanglement, then Table III and Fig. 5 measure only part of the true communication cost, and the comparisons to DISQCO and Pytket-DQC could be reversed if the RL policy uses more merges. Please either justify the cost model with explicit references or report merge counts and use splits+merges as the EPR cost metric.","section":"III-B, II-A"},{"comment":"The empirical evaluation reports single average values with no error bars, number of seeds, or statistical tests. The central claims—\"matches state-of-the-art heuristics\" and \"lookahead reward shaping yielding modest improvements\"—rest on small differences (e.g., 39.0 vs 42.0 on random circuits). Without variance information, these differences could be noise. Please report mean ± standard deviation across multiple random seeds (at least 5–10) and, where relevant, a paired significance test (e.g., Wilcoxon signed-rank test) or effect size.","section":"VI (Table III, Fig. 5)"},{"comment":"The \"No-lookahead RL\" ablation is not defined. The text contrasts it with \"Progress-only RL\" (which removes the lookahead reward component), but it does not explain what \"future-horizon state lookahead\" means in the policy architecture. No such mechanism is introduced in Section IV. Without a precise definition, this ablation cannot be interpreted, and the claim about lookahead reward shaping is therefore unclear. Please specify exactly what is being ablated or remove this variant.","section":"VI-B"},{"comment":"The temporal communication model in Section III-C2 decrements only the source QPU's channel budget (Eqs. (9)–(11)), while the feasibility constraint in Eq. (14) requires available channels on both the source and destination QPUs. The destination's channel is never consumed, so the dynamic capacity limit is not enforced for the receiving side. This contradicts the stated contribution of enforcing \"physical capacity limits on concurrent communication primitives\" and could allow the policy to schedule more concurrent splits than the model intends. Please fix the channel accounting or clarify the intended semantics.","section":"III-C2, III-D1"}],"minor_comments":[{"comment":"The phrase \"a logical qubit is require to be uniquely localized\" contains a grammatical error; it should read \"is required to be uniquely localized.\"","section":"III-D2"},{"comment":"Reference [24] and reference [40] both cite the same publication (\"A multilevel framework for partitioning quantum circuits\"). Please deduplicate.","section":"References"},{"comment":"The heading \"V olume\" contains an extra space; it should be \"Volume.\"","section":"Table III"},{"comment":"The baselines assume unconstrained link availability while the RL framework enforces channel capacities. Although the paper states this does not alter EPR count logic, the differing problem settings make the comparison less direct; a sentence explaining why the capacity constraint does not bias the EPR count would help.","section":"V-C"},{"comment":"Reproducibility would be improved by reporting the PPO hyperparameters (learning rate, batch size, clip ratio, number of epochs) and releasing the evaluation code; only reward hyperparameters are given in Table II.","section":"IV-D"}],"recommendation":"major_revision","confidential_remarks":"The paper has a notable concentration of self-references (Refs. [19], [23], [24], [26], [40], [42]) and the DISQCO baseline is authored by a co-author (F. Burt). Given that the comparison against DISQCO is central to the evaluation, the editor may wish to give extra scrutiny to the fairness of the baseline configuration. The channel-accounting inconsistency in Section III-C2 also suggests that the claimed contribution of dynamic channel constraints should be re-examined. These issues are fixable, but they warrant attention in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick read on arXiv:2608.06892. The real contribution is the formulation: a constrained MDP for distributed compilation with per-QPU channel budgets, a heterogeneous graph state, and a hierarchical action decoder. RL for distributed compilation is not new—they cite [30]—but the channel-capacity constraint and the graph encoding are genuinely useful extensions. The paper is honest: it reports that the learned policy matches DISQCO on QAOA/QFT/random, does not match on quantum volume (28.7 vs 11.9 splits), and that lookahead reward helps only modestly. That is a fair empirical claim, and the benchmark table supports it.\n\nThe strong point is the clean problem setup and the reward ablation. The MDP is coherent, the action masking is carefully specified, and the model explicitly accounts for channel contention that the baselines ignore. The paper says this rather than pretending the comparison is apples to apples.\n\nSoft spots, in order. First, the EPR-efficiency claim is equated to split count; merges are never counted. The stress-test note about merge cost is the right question to ask. If a merge also consumes an EPR pair, or if baseline merge behavior differs, Table III is not measuring exactly what the abstract claims. It is a stated modeling limitation, but it is load-bearing for the headline. Second, no error bars, seeds, or statistical tests—the differences between RL reward variants are within noise. Third, scale is tiny: 2 QPUs, 4–8 qubits per QPU. That is acceptable for a first result, but the scalability bottleneck they acknowledge in the conclusion is doing real work. Fourth, the closest prior RL compiler [30] is cited but not compared, and the reward weights are hand-tuned. Neither is fatal, but both should be addressed before I rely on the quantitative ranking.\n\nWho this is for: people working on modular or multi-core quantum compilation who want a template for RL-based scheduling. It deserves a serious referee. It is not a breakthrough, but it is a competently executed, honestly reported framework with reproducible value if the code and policies ship. Send it to review with the expectation that the authors add variance reporting, include the [30] baseline, and analyze merge cost.","headline":"A solid, honestly reported RL compiler for distributed circuits that matches heuristics on structured benchmarks, loses on dense ones, and needs a merge-cost check before the EPR-efficiency claim fully holds.","tokens_in":13908,"tokens_out":2857,"would_cite":true,"duration_ms":30647,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["03.67.-a","03.67.Lx"],"model":"deepseek-v4-flash","headline":"A reinforcement-learning policy can schedule split and merge actions for distributed quantum compilation, matching hand-designed heuristics on structured circuits while operating under stricter channel-capacity constraints.","keywords":["distributed quantum computing","quantum circuit compilation","reinforcement learning","PPO","heterogeneous graph","EPR pair","split-merge communication","communication scheduling"],"falsifier":"Measure the physical EPR consumption of the split/merge protocol on a two-QPU architecture with noisy entanglement generation and a nonzero cost for merges and channel seconds. If the ordering of RL versus DISQCO on random circuits changes when merges are charged EPR pairs, then the paper's primary metric, split count equal to EPR pairs, is not a faithful resource measure.","tokens_in":12910,"feed_emoji":"⚛️","tokens_out":6763,"duration_ms":62760,"temperature":0.7,"pith_summary":"This paper tries to establish that distributed quantum circuit compilation can be treated as a constrained Markov decision process and solved by a reinforcement-learning policy that never sees hand-coded routing rules. The policy, a heterogeneous graph neural network trained with PPO, picks split and merge actions that update which QPUs hold each logical qubit; when the reward includes a lookahead term, the policy anticipates future gate locality. On QAOA and QFT benchmarks the policy's average split counts (4.0 on each) match the DISQCO baseline on QFT and trail Pytket-DQC and DISQCO on QAOA (both 3.0). On random circuits the full policy averages 39.0 splits, better than Pytket-DQC's 39.5 and modestly worse than DISQCO's 35.7, while on dense quantum-volume circuits it needs 28.7 splits versus DISQCO's 11.9. The paper's claim is therefore that RL is a workable, flexible alternative to heuristics, not that it dominates them.","feed_headline":"RL policy matches heuristics on structured quantum circuit splits","feed_subtitle":"A learned policy matches Pytket-DQC and DISQCO on QAOA and QFT; lookahead rewards help on random circuits.","key_machinery":"The split and merge primitives, the constrained MDP, the heterogeneous graph observation, and the hierarchical action decoder. A split consumes one EPR pair to make a logical qubit available at an additional QPU; a merge removes a redundant placement. Around these primitives the paper builds a constrained Markov decision process whose transitions update placement sets $Loc_t(q)$, QPU clocks, and channel budgets, and a heterogeneous graph over QPU, qubit, and gate nodes with typed edges for topology, presence, operands, dependencies, and co-location. The hierarchical decoder selects qubit, source, action type, and destination, with feasibility masks; the reward's lookahead term carries the delayed-locality signal.","core_discovery":"The central discovery is that a learned policy can solve the two-QPU compilation problem under explicit channel-capacity and qubit-capacity constraints with EPR efficiency comparable to dedicated heuristics, provided the state is a heterogeneous graph and the reward separates immediate progress from delayed locality. The action space is factorized into qubit, source QPU, split-or-merge, and destination, with feasibility masks; after each action all newly executed gates run, and the communication clock advances only when a source QPU exhausts its channel budget. The lookahead reward term $|G^{t+1}_{\\mathrm{exec}}| - |G^{t}_{\\mathrm{exec}}|$ is the mechanism that gives modest random-circuit gains. The reported numbers: QAOA and QFT average 4.0 splits (baselines 3.0 and 4.0–5.7), random circuits 39.0 (baselines 39.5 and 35.7), quantum volume 28.7 (baselines 17.5 and 11.9).","pith_inferences":["A direct comparison under the same channel-capacity model for all methods would isolate whether the quantum-volume gap (28.7 vs 11.9) comes from the RL policy's decision quality or from the stricter hardware assumptions the paper imposes on itself.","If EPR cost is accounted per merge and per failed or held channel rather than per split, the random-circuit ranking between RL and Pytket-DQC could change; the paper's metric is an abstraction, not a measured Bell-pair budget.","The factorized action decoder and heterogeneous graph are architecture-agnostic, so extending to more than two QPUs or heterogeneous link capacities is a direct next test; the paper lists training on circuit families as future work.","A natural calibration experiment: train the policy on quantum-volume circuits alone, without curriculum mixing, and see whether the split-count gap closes; if it does not, the bottleneck is reward signal rather than architecture."],"forward_implications":["The same policy, without retraining on circuit families, matches DISQCO's split count on QFT and comes within one split of it on QAOA, so manual routing heuristics are not necessary for these structured workloads.","Because the RL environment enforces per-QPU communication-channel budgets while Pytket-DQC and DISQCO are evaluated without them, the RL policy's competitive split counts are achieved under a stricter resource model.","The lookahead reward's small gain on random circuits (39.0 vs 42.0 for progress-only) shows that delayed-locality shaping helps exactly where decisions have deferred benefits, but it does not help on quantum volume (28.7 vs 24.7 for no-lookahead).","Dense, permutation-heavy circuits remain unsolved by this approach: the policy's action-value signal is too flat to coordinate globally, and scalability is the stated bottleneck."],"supporting_citations":[{"why":"Pytket-DQC repository, the baseline compilation library whose split counts the RL policy is compared against.","marker":"[41]"},{"why":"DISQCO repository, the other baseline library whose MLFM-R partitioner is used by default.","marker":"[42]"},{"why":"Multilevel framework for partitioning quantum circuits; supplies the MLFM-R partitioner and the random-circuit generator used in the benchmarks.","marker":"[40]"},{"why":"Distributing circuits over heterogeneous modular networks; basis of Pytket-DQC's heterogeneous hypergraph partitioner.","marker":"[25]"},{"why":"Entanglement-efficient distribution of quantum circuits; basis of DISQCO's approach.","marker":"[26]"},{"why":"Entanglement-efficient bipartite distributed quantum computing; defines the split/merge starting and ending processes that ground the communication model.","marker":"[33]"},{"why":"Prior reinforcement-learning formulation for a distributed-quantum compiler that this work extends.","marker":"[30]"},{"why":"Proximal policy optimization, the training algorithm used to update the policy network.","marker":"[35]"},{"why":"Automated distribution of quantum circuits via hypergraph partitioning; foundational heuristic behind Pytket-DQC.","marker":"[13]"}],"fun_headline_variants":["RL matches heuristics for distributed quantum circuit splits","Lookahead reward improves RL on random quantum circuits","Quantum compilation via RL: on par with heuristics on structured circuits","RL for distributed quantum compilation: matches heuristics, gains on random circuits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison to baselines treats one split as exactly one EPR pair and ignores the EPR cost of merges, re-splits, failed entanglement generation, and noise; if real entanglement-assisted communication does not follow that accounting, the reported EPR counts do not measure physical communication cost.","fun_headline_variants_meta":{"raw":{"variants":["RL matches heuristics for distributed quantum circuit splits","Lookahead reward improves RL on random quantum circuits","Quantum compilation via RL: on par with heuristics on structured circuits","RL for distributed quantum compilation: matches heuristics, gains on random circuits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001224,"raw_usage":{"total_tokens":5017,"prompt_tokens":915,"completion_tokens":4102,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":4033}},"tokens_in":531,"tokens_out":4102,"duration_ms":28561,"temperature":1.0,"reasoning_tokens":4033,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:51:58.241019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the physical EPR consumption of the split/merge protocol on a two-QPU architecture with noisy entanglement generation and a nonzero cost for merges and channel seconds. If the ordering of RL versus DISQCO on random circuits changes when merges are charged EPR pairs, then the paper's primary metric, split count equal to EPR pairs, is not a faithful resource measure.","supporting_citations":[{"cited_title":"CQCL/pytket- dqc","cited_arxiv_id":null,"evidence_quote":"Pytket-DQC repository, the baseline compilation library whose split counts the RL policy is compared against."},{"cited_title":"Disqco: Distributed quantum circuit optimisation","cited_arxiv_id":null,"evidence_quote":"DISQCO repository, the other baseline library whose MLFM-R partitioner is used by default."},{"cited_title":"Distributing circuits over heterogeneous, modular quantum computing network architectures,","cited_arxiv_id":null,"evidence_quote":"Distributing circuits over heterogeneous modular networks; basis of Pytket-DQC's heterogeneous hypergraph partitioner."},{"cited_title":"Entanglement-efficient dis- tribution of quantum circuits over large-scale quantum networks,","cited_arxiv_id":null,"evidence_quote":"Entanglement-efficient distribution of quantum circuits; basis of DISQCO's approach."},{"cited_title":"Entanglement-efficient bipartite-distributed quantum computing,","cited_arxiv_id":null,"evidence_quote":"Entanglement-efficient bipartite distributed quantum computing; defines the split/merge starting and ending processes that ground the communication model."},{"cited_title":"Compiler for distributed quantum computing: A reinforcement learning approach,","cited_arxiv_id":null,"evidence_quote":"Prior reinforcement-learning formulation for a distributed-quantum compiler that this work extends."},{"cited_title":"Automated distribution of quan- tum circuits via hypergraph partitioning,","cited_arxiv_id":null,"evidence_quote":"Automated distribution of quantum circuits via hypergraph partitioning; foundational heuristic behind Pytket-DQC."}],"review_version":1}