{"id":"87dd4fcd-d005-4b70-bcb4-0ca3dbf33ed7","arxiv_id":"2411.09784","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A PPO-trained agent finds that optimized projective measurements can disentangle small random Clifford circuits with far fewer projections than random measurement rates in MIPT studies.","lead":"The authors train a reinforcement learning agent to place measurements inside small random quantum circuits, with the goal of destroying all their entanglement using as few measurements as possible. Their simulations suggest an optimized set of measurements can disentangle a circuit with many fewer projections than the random measurement patterns studied in measurement-induced phase transition papers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reward function in Eq. (12) has the opposite monotonicity from Eq. (11), pushing the optimizer toward the trivial last-layer solution that the paper claims to avoid; as a result M(N,D,α) is not a valid measure of efficient disentanglement.","rationale":"The reader's weakest_assumption identifies the layer-dependent weights as making the optimization target a weighted cost rather than a raw count, which is correct and important. The stress-test goes further: Eq. (12) does not merely add weights, it inverts the intended layer preference. This is an internal inconsistency, checkable directly from the equations and figures, and it directly undermines the paper's central claims about optimality and the pattern of disentanglers. The exhaustive-search test for small N would settle whether the reported M(N,D,α) is close to the true raw minimum or is an artifact of the mis-signed reward. Because the issue is concrete and fixable (redefine f_l to increase with l, rerun, or report raw unweighted counts), a conditional verdict is appropriate rather than outright rejection: the authors should correct the reward definition and re-validate the headline numbers. The reader's verdict already conditional, and this concern strengthens rather than replaces that verdict.","tokens_in":9344,"tokens_out":7087,"duration_ms":73347,"concrete_test":"Exhaustively enumerate all measurement matrices P for N=4, D/2=3 (2^12 = 4096 configurations) and compute the final averaged entropy plus both costs C_0 = Σ_l m_l and C_α using Eq. (12) for α=0.1. Find (i) the true minimum of C_0, (ii) the true minimum of C_α, and (iii) the layer-averaged ⟨L⟩ of each optimizer. Compare with the PPO-reported M for the same size. If the C_α-minimizer has higher C_0 than the C_0-minimizer and places measurements on later layers, the weighted reward is biasing the result; additionally verify the sign of d f_l/dl and check whether PPO's ⟨L⟩ vs α agrees with the exact C_α optimizer or with the paper's text.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantity M(N,D,α) is obtained by optimizing the sparse reward R = 1 - C/(FN) with C = Σ_l f_l m_l (Eqs. 10, 13). The stated design (Eq. 11) is that deeper layers incur a larger penalty, with f_l increasing in l, specifically to prevent the trivial solution of measuring the final layer. However, the chosen function f_{l;α} = 2e^{-αl}/(1+e^{-αl}) in Eq. (12) is strictly decreasing in l for every α > 0, and Fig. 2 confirms this. Thus the implemented reward prefers measurements on later layers, exactly the trivial strategy the text says it avoids. The inconsistency propagates into the results: Section III states that 'the larger α is, the lesser the number of measurements placed on the later layers is,' but Fig. 5 shows ⟨L⟩_eps increasing with α, meaning measurements move later as α grows. Consequently M(N,D,α) is the count produced by a policy optimizing a cost with the opposite of the intended layer preference, so the learned patterns cannot be described as 'optimal' and the counts cannot support the claim of being 'drastically less' than MIPT random-measurement rates. Independently of the sign, the reward is a weighted cost rather than the raw number of projections, so the reported numbers are not unconstrained minima; the sign error makes the bias toward the trivial late-layer solution systematic.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formulates the problem of disentangling a random Clifford brick-wall circuit by placing projective measurements between unitary layers, and solves it with proximal policy optimization (PPO). The agent acts on a binary matrix of measurement locations; the sparse reward is nonzero only when the final averaged von Neumann entropy vanishes, and otherwise equals one minus a layer-weighted measurement cost. The paper reports that the number of measurements M(N,D,alpha) is approximately linear in the number of qubits N and saturates as a tanh function of the depth D, and that the measurement pattern has a nontrivial layer profile controlled by the penalty slope alpha. Based on these numbers, the abstract claims that the number of measurements required to disentangle a random circuit is drastically less than the measurement rates used in measurement-induced phase transition (MIPT) numerics.","tokens_in":9724,"tokens_out":6774,"duration_ms":72328,"significance":"The core idea -- replacing random measurement patterns in MIPT studies by optimized, deliberately placed disentanglers -- is a worthwhile and potentially useful research direction. The stabilizer-formalism entropy calculation is standard and efficient, and framing the placement problem as an RL game is natural and clearly explained. If the results were validated, they would provide a concrete, falsifiable quantitative statement about the overhead of disentangling random Clifford circuits, and the learned patterns could inform the design of disentangling protocols. However, the current manuscript does not establish these claims: the reward function contains a sign inconsistency with the stated design goal, the reported M is the count of a weighted-cost optimum rather than an unconstrained minimum, no quantitative MIPT baseline is supplied, and the PPO solutions are not certified as optimal even on small instances. These issues are load-bearing because they affect the definition of the central quantity M and the headline comparison to MIPT.","major_comments":[{"comment":"The weight function f_{l;alpha} = 2e^{-alpha l}/(1+e^{-alpha l}) is strictly decreasing in l for every alpha > 0, as Fig. 2 confirms. This contradicts the design statement in Eq. (11) that the weights are a monotonically increasing penalty rate as the layer gets deeper. Because C = sum_l f_l m_l in Eq. (10), the implemented reward makes measurements on later layers cheaper, so it encourages the trivial last-layer solution that the text says it is designed to avoid. Consequently the interpretation in Section III.A -- that larger alpha moves measurements away from later layers -- is opposite to the incentive encoded in the reward, and the reported M(N,D,alpha) is not produced by the objective described in the text.","section":"II.B.3, Eq. (12)"},{"comment":"The sparse reward is R = 1 - C/(F N), so among successful disentanglers the agent minimizes the weighted cost C = sum_l f_l m_l, not the number of projections M. Unless all f_l are equal (which happens only at alpha=0), the number M(N,D,alpha) is not the minimal number of measurements required to disentangle the circuit; it is the count belonging to a weighted-cost optimum. The abstract and Section III.A treat M as the number of measurements 'required,' and compare it with MIPT measurement rates, but the comparison is between a weighted cost optimum and an unweighted per-site rate. This mismatch affects the central quantitative claim of the paper.","section":"II.B.3, Eqs. (10) and (13)"},{"comment":"The headline claim that the number of measurements is 'drastically less than the numerical results of measurement-induced phase transition papers' is not supported by any quantitative baseline. The paper never states the critical measurement rate or the total measurement counts from the cited MIPT works for the same system sizes and circuit depths, nor does it define whether the comparison is per-site rate, total projections, or something else. Without this baseline, the abstract's central comparison is unverifiable.","section":"III.A and Appendix A"},{"comment":"The PPO policies are not certified to be optimal or near-optimal. Training uses 2x10^5 to 5x10^5 time steps, and the learning curves in Figs. 6 and 7 show mean episode reward but no convergence criterion or comparison against an exact solution. For the smallest circuits (e.g., N=3-5, D/2=2-4) an exhaustive enumeration of all measurement subsets would be feasible and would provide a ground-truth check of Eq. (14). Without such a check, the quantities labeled 'min|R' are at best heuristic upper bounds rather than the 'required' numbers claimed in the abstract.","section":"Appendix A and II.B.1"},{"comment":"The text states that 'the larger alpha is, the lesser the number of measurements placed on the later layers is,' but this is inconsistent with Eq. (12), where larger alpha makes later layers cheaper and should push measurements toward later layers. Figure 5 should be read carefully and the direction of the effect must be reconciled with the reward. As written, the textual claim, the equation, and the observed profile cannot all be correct; this directly undermines the paper's characterization of the pattern of optimal disentanglers.","section":"III.A, Eq. (21) and Fig. 5"}],"minor_comments":[{"comment":"The notation eta_{xi,xj} in Eq. (16) is introduced but never used anywhere else in the manuscript; either use it or remove it.","section":"Eq. (16)"},{"comment":"There are several typographical issues: 'Gottesmann' should be 'Gottesman', 'bipartions' in Eq. (9) should be 'bipartitions', and the author names contain garbled diacritics.","section":"I, Acknowledgements"},{"comment":"The notation 'min|R' in Eq. (14) is unclear. Please define precisely what the minimization is over (final configurations? policies? reward values?) and how the minimum is estimated from the PPO runs.","section":"Eq. (14)"},{"comment":"The table and Eq. (17) use inconsistent notation for the fit coefficients: Eq. (17) writes gamma^{N,alpha}_1(D) and gamma^{N,alpha}_2(D), while the text and Table I use gamma1 and gamma2 with different stated dependencies. Please make the notation uniform.","section":"III.A and Table I"},{"comment":"The caption says 'tanh and linear fit' but the table mixes two fit forms and two fitted quantities without clearly indicating which rows correspond to which form. A separate column or grouped layout would improve readability.","section":"Table I caption"},{"comment":"Reference [21] is an unpublished manuscript 'available upon request'; for reproducibility, please provide a public version or repository for the PyClifford package and the simulation code.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a promising direction but the central result is currently not established. The sign inconsistency in Eq. (12) is particularly serious because it changes the meaning of the reward and the alpha-dependence of all reported patterns. I would ask the authors to fix the reward definition, rerun the simulations, add an exact brute-force check on small system sizes, and provide the missing MIPT comparison before this can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, the core idea is worth a coffee conversation: treating measurement placement in brick-wall Clifford circuits as an RL game, with a stabilizer-based entropy reward, and letting PPO discover where to project. Second, the implementation has a sign error in the reward that reverses the paper's own design and the interpretation of the results.\n\nWhat's actually new: the MDP framing over binary measurement matrices is not in the MIPT literature I know, and the scaling observations (roughly linear in N, saturating in D) are the kind of descriptive data that could inform measurement-placement heuristics for error mitigation. The stabilizer entropy calculation is standard and correctly described.\n\nWhere it falls down. Eq. (11) says the layer penalty should increase with layer depth, to avoid the trivial 'measure everything on the last layer' solution. But Eq. (12) defines f_{l;α} = 2e^{-αl}/(1+e^{-αl}), which strictly decreases with l for every α>0. So the implemented cost rewards placing measurements later, exactly the trivial strategy the text says it prevents. Figure 5 then shows ⟨L⟩_eps rising with α, meaning measurements move later as α grows—directly contradicting the sentence in Sec. III that says 'the larger α is, the lesser the number of measurements placed on the later layers is.' That's an internal contradiction, not a stylistic quibble. Because M(N,D,α) is the count coming out of a policy that optimized the wrong side of the cost, it cannot support the paper's headline claim about being 'drastically less' than MIPT papers. There is also no quantitative MIPT baseline anywhere in the text, so the comparison is unbacked.\n\nOther soft spots: 'optimal' is asserted without certification—PPO is a heuristic, and there is no exact brute-force check even for N=3,4. The reward is a weighted count, not the raw number of projections, so the numbers are not unconstrained minima. Implementation details are sloppy: the Fig. 3 caption lists lr=0.1 while Appendix A says lr=10^{-3}, and a reward scale pr=50.0 appears nowhere in the main equations. No code or data are released.\n\nCredit where due: the stabilizer machinery is solid, the environment description is clear enough to reproduce, and the small-N results do demonstrate that RL can find product-state disentanglers. But the current analysis does not show that those disentanglers are efficient in the raw-count sense, and the α-pattern is an artifact of the mis-signed reward.\n\nThis is for someone working on MIPT or on RL for circuit synthesis, as a caution and a sketch. I would not send it to a serious journal in this form. My recommendation: return for major revision, ask the authors to fix the reward monotonicity, compare with exact search on small N, quote MIPT thresholds, and release the code. If the reworked version appears, it deserves a referee's time.","headline":"Nice RL framing for measurement placement, but the reward sign error reverses the intended objective and the headline claim lacks quantitative support.","tokens_in":10200,"tokens_out":8189,"would_cite":false,"duration_ms":80044,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that targeted projective measurements found by reinforcement learning can fully disentangle random Clifford circuits with far fewer projections than random-measurement MIPT studies, and that the optimal placement has a…","keywords":["reinforcement learning","disentanglers","projective measurements","random Clifford circuits","measurement-induced phase transition","entanglement entropy","stabilizer states","proximal policy optimization"],"falsifier":"Enumerate all $2^{N D/2}$ binary measurement configurations for the smallest circuits, such as $N=4$ and $D/2=3$, and find the least number of projections that makes the averaged von Neumann entropy vanish; if the true minimum is materially lower or higher than the PPO-reported count, the learned numbers are not the raw minimum.","tokens_in":9114,"feed_emoji":"⚛️","tokens_out":9536,"duration_ms":87253,"temperature":0.7,"pith_summary":"The paper is trying to establish that the number of measurements needed to fully disentangle a random brick-wall Clifford circuit is substantially smaller than what earlier measurement-induced phase transition simulations suggest, and that the optimal measurement positions form a structured pattern rather than a random one. The agent starts with no measurements, adds or removes one projection at a time, and receives a reward only when the final state is a product state, minus a weighted cost for the measurements used. The paper reports that the optimal count grows linearly with qubit number, saturates with circuit depth, and moves to earlier layers when late measurements are penalized. If these results hold, the minimum intervention needed to destroy entanglement is an optimizable quantity rather than a fixed random-measurement rate.","feed_headline":"Learned measurement patterns disentangle circuits with far fewer cuts","feed_subtitle":"Strategic placement needs far fewer projections than random measurement patterns used in MIPT numerics.","key_machinery":"The load-bearing object is the binary matrix $P$ of measurement positions, where $P_{ij}=1$ means a $Z$-projection is inserted at site $i$ between consecutive unitary layers. The state of the reinforcement-learning agent is this matrix, the action flips one bit to add or delete a measurement, and an episode ends when the averaged von Neumann entropy over all $N-1$ contiguous bipartitions vanishes. The sparse reward is $R = 1 - C/(F N)$, with $C=\\sum_l f_{l;\\alpha} m_l$ the layer-weighted measurement cost and $f_{l;\\alpha}=2e^{-\\alpha l}/(1+e^{-\\alpha l})$ a monotone penalty that is small for early layers and large for late ones. Clifford and stabilizer simulation, through the Gottesman-Knill representation and the stabilizer formula for Rényi entropy, makes each entropy evaluation polynomial rather than exponential, which is what allows the PPO loop, a policy-gradient method with clipped updates, to search the configuration space.","core_discovery":"The central discovery claimed is that a PPO-trained agent finds a configuration of projections that makes the averaged von Neumann entropy over all contiguous bipartitions vanish exactly, meaning the final state is a product state, and that the number of projections in this configuration, averaged over episodes, is much smaller than the measurement counts used in MIPT numerics. The paper reports that this number scales linearly with the number of qubits, saturates as a hyperbolic tangent with circuit depth, and has a layer-resolved profile that shifts from late layers to earlier layers as the penalty slope increases. The authors take this as evidence that there is a learnable, structured efficient disentangler that random-projection studies do not see.","pith_inferences":["Exact enumeration on small systems would settle whether the learned configurations are truly minimal; the paper does not report such a check.","Because the reward weights measurements by layer, the reported counts are not unconstrained raw minima, and an unweighted reward could change the comparison with measurement-induced phase transition rates.","If the structured patterns are robust, the same optimization could be tried on non-Clifford circuits or on specific target states, where stabilizer shortcuts are unavailable and the learned patterns would have to be tested directly.","The shift of measurements toward earlier layers under a steeper penalty hints that late-layer measurements are the most efficient per measurement, an effect that could be quantified by removing one learned measurement at a time."],"forward_implications":["For a fixed depth and penalty slope, the minimal number of projections grows linearly with qubit number, roughly one additional measurement per two added qubits.","For a fixed system size, the minimal number of projections saturates as circuit depth grows, following the saturation of entanglement entropy in unmeasured circuits.","The optimal measurement pattern is depth-dependent: with a flat penalty most projections sit on late layers, and as the penalty slope rises they move to earlier layers.","Randomly placed measurement patterns overestimate how many projections are needed to fully disentangle a random Clifford circuit.","The fully disentangled final state is a product state, since the averaged entropy over all contiguous bipartitions vanishes."],"supporting_citations":[{"why":"Supplies the PPO algorithm used to train the disentangler policy.","marker":"[1]"},{"why":"Provides the numerical measurement-induced phase transition results the paper compares its measurement counts against.","marker":"[3]"},{"why":"Supplies the stabilizer and Heisenberg representation that makes Clifford circuit simulation and entropy evaluation efficient.","marker":"[11]"},{"why":"Relates Rényi entropy to stabilizer subgroup size, the formula used to compute entanglement entropy of stabilizer states.","marker":"[12]"},{"why":"Supplies the environment interface used to define the binary-matrix Markov decision process.","marker":"[18]"},{"why":"Supplies the Clifford-simulation software used for random gate sampling and entropy computations.","marker":"[21]"},{"why":"Supplies the PPO implementation used for training the agent.","marker":"[22]"}],"fun_headline_variants":["RL-trained disentanglers need far fewer cuts to untangle circuits","PPO agent finds sparse measurement patterns to disentangle circuits","Structured projections disentangle circuits with fewer measurements","Reinforced learning yields efficient disentanglers for random circuits","Optimal measurement patterns from PPO cut circuit entanglement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported minimum measurement counts assume the trained policy converged to a near-optimal configuration and that minimizing the layer-weighted reward is the same as minimizing the raw number of projections.","fun_headline_variants_meta":{"raw":{"variants":["RL-trained disentanglers need far fewer cuts to untangle circuits","PPO agent finds sparse measurement patterns to disentangle circuits","Structured projections disentangle circuits with fewer measurements","Reinforced learning yields efficient disentanglers for random circuits","Optimal measurement patterns from PPO cut circuit entanglement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000234,"raw_usage":{"total_tokens":1475,"prompt_tokens":902,"completion_tokens":573,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":492}},"tokens_in":518,"tokens_out":573,"duration_ms":5818,"temperature":1.0,"reasoning_tokens":492,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:19:19.116750+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Enumerate all $2^{N D/2}$ binary measurement configurations for the smallest circuits, such as $N=4$ and $D/2=3$, and find the least number of projections that makes the averaged von Neumann entropy vanish; if the true minimum is materially lower or higher than the PPO-reported count, the learned numbers are not the raw minimum.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the PPO algorithm used to train the disentangler policy."},{"cited_title":"The sparse reward function is evaluated at the end of every episode","cited_arxiv_id":null,"evidence_quote":"Provides the numerical measurement-induced phase transition results the paper compares its measurement counts against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the environment interface used to define the binary-matrix Markov decision process."},{"cited_title":"Schulman, S","cited_arxiv_id":null,"evidence_quote":"Supplies the Clifford-simulation software used for random gate sampling and entropy computations."}],"review_version":1}