{"id":"7cb0dcd2-bd2b-4bb5-b645-be650875373d","arxiv_id":"2607.11796","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"An exact per-mode Gram instrument shows trained Mamba models migrate state usage with input via Bt, and two-pass scheduled pruning matches unpruned quality at half state budget.","lead":"Selective SSMs like Mamba re-allocate which internal modes carry the signal depending on the input, driven mainly by the write map Bt rather than the timestep. An exact Gram-based instrument measures this and shows input-scheduled pruning can match the unpruned model at half the state budget as headroom.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The instrument is validated to 1e-6–1e-7 against both reconstruction and deployed pruning (4,464 configs). Migration (ρ≈0.44–0.57) and the Bt mechanism (freeze lifts ρ to ~0.96) hold across Mamba-1, Falcon-Mamba-7B and Mamba-2 with frozen-signal and energy-matched controls. The only soft spot is the two-pass nature of the compression result, which the authors already treat as headroom rather than a deployed method. That matches the reader's weakest_assumption exactly; no stronger load-bearing concern appears. Verdict remains ACCEPT.","tokens_in":13545,"tokens_out":385,"duration_ms":3601,"concrete_test":"Re-run the half-budget scheduled column of Table 2 under a pure continuation protocol that never re-uses any token of the scored window for mask selection (e.g., mask from a disjoint preceding window of the same domain stream); if the scheduled-vs-unpruned gap CI still excludes zero, the headroom claim would need re-examination, otherwise the paper's own disclosure stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claims (exact instrument, input-driven migration via Bt, and two-pass headroom) are internally consistent, multi-model, and carefully bounded. The reader's weakest_assumption correctly flags that the compression result is an oracle upper bound, but the manuscript already discloses this repeatedly (abstract, §4.3, Limitations, Conclusion) and never claims a one-pass deployable saving. No hidden circularity, derivation error, or unsupported leap remains; the same-window control and out-of-window recovery fractions (2–26%) already quantify how far the bound sits from practice.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper introduces an exact instrument for measuring how selective SSMs (Mamba-1/2, Falcon-Mamba) use their diagonal state modes. Because A is diagonal, each channel’s output decomposes into per-mode contributions; a per-(layer, channel, window) Gram tensor then yields the exact squared output error of dropping any mode subset offline (Eq. 4). Validated to ~2e-7 against the reference kernel and to median ~1e-6 against deployed pruning over 4,464 configurations, the instrument is applied across the Mamba-1 family (130M–2.8B), Falcon-Mamba 7B, and Mamba-2. The authors report input-driven mode migration (ρ≈0.44–0.57 at the worst layers), attribute it primarily to the write map Bt via freeze counterfactuals (with energy-matched and Bt-only-live controls), and show that a two-pass, same-window scheduled pruning rule outperforms static, modal-HSV, and LAST rankings and, at half state budget, matches or slightly beats the unpruned model on held-out perplexity—explicitly framed as realizable headroom, not a deployed saving.","tokens_in":13780,"tokens_out":1342,"duration_ms":23066,"significance":"If the results hold, the paper supplies a rare exact, offline measurement tool for selective-SSM state usage and a clear mechanistic account of input-dependent reallocation (Bt-driven, not Δ-driven). That combination is useful both for interpretability and for any future compression work that currently relies on static or activity-based scores. Strengths that raise confidence include: (i) machine-checked numerical gates against the reference implementation and against deployed pruning; (ii) multi-scale, multi-architecture coverage including a deployed 7B model; (iii) Proposition 1 linking the diagonal gap to energy-weighted churn; (iv) explicit same-window vs out-of-window controls and recovery fractions (2–26%); and (v) public checkpoints, fixed seeds, and released code. The compression claim is carefully bounded as oracle headroom rather than a free one-pass saving, which is the right scientific stance.","major_comments":[{"comment":"§4.3 and Table 2: the end-to-end two-pass protocol needs a precise statement. When masks for all layers are read from a first (unpruned) pass and then applied jointly in a second pass, later layers see shifted inputs, so the first-pass energies are not exactly the energies of the pruned trajectory. Please state whether masks are (a) computed once from the unpruned forward and applied jointly, (b) recomputed layer-by-layer along the pruned trajectory, or (c) something else, and report a short sensitivity check. This is load-bearing for the “matches unpruned at half budget” numbers, which are otherwise the paper’s strongest quantitative claim.","section":null},{"comment":"Abstract, Table 2, and §4.3: the striking sub-unpruned margin is same-window only. Table 3 and the half-window control show that out-of-window selection does not beat the unpruned model and recovers only ~18–26% of the oracle gain. The abstract already says “we claim no deployed compute or memory saving,” but the main table caption and the sentence “at half the state budget it matches the unpruned model” still invite over-reading. Please make the oracle / same-window restriction equally prominent in the Table 2 caption and in the first sentence of the Consequence subsection so the central compression claim cannot be mis-cited as a one-pass result.","section":null}],"minor_comments":[{"comment":"§3.4, Proposition 1: the proof is for the diagonal (energy-only) gap; the text correctly notes full-Gram ρ can slightly exceed 1 (layers 21–22 of 130M). A one-sentence pointer in the proposition statement that the inequality is diagonal-only would prevent misreading.","section":null},{"comment":"Figure 1: the migrating-band shading (0.4–0.75) is helpful, but the text also cites 0.6–0.75 for small Mamba-1 models. Align the shaded interval with the prose or note that the shade is a compromise across scales.","section":null},{"comment":"§4.1: cross-domain vs within-domain churn ratios (1.15–1.32) are modest and below the pre-registered 1.5× pilot. The paper already treats churn as corroborating; a brief remark that the migration case rests on ρ and the freeze experiments (not on churn alone) would help readers who stop at the churn numbers.","section":null},{"comment":"Table 4 / GHOST comparison: the note that the released pipeline can silently no-op under fused kernels is valuable for reproducibility; consider elevating a one-line warning into the Reproducibility paragraph so others do not re-hit the same trap.","section":null},{"comment":"Notation: ¯A vs A, and the dual role of Δ (pole and write gain), are introduced cleanly in §3.1 but reappear densely in §4.2. A short reminder that Δ is mode-uniform within a channel would make the “Δ carries almost none” claim easier to parse on a first read.","section":null},{"comment":"Limitations already flags untested long contexts; given that migration concentrates in slow modes (Figure 3), a single longer-window (e.g., 4k) spot-check on one small model would strengthen the claim that the phenomenon is not an artifact of L=1024, even if left as a brief appendix note.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is unusually careful about not overselling the compression result; the two major comments are clarification/emphasis, not conceptual holes. Fit for a strong ML venue is good: exact instrument + multi-model mechanism + honest headroom framing. I would not block on the oracle nature of the scheduler given how repeatedly it is disclosed. No citation or novelty concerns stood out."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful part of this paper is the instrument, not the pruning headline. Because A is diagonal, the channel output is an exact sum of per-mode terms, so a small Gram per (layer, channel, window) gives the exact squared error of dropping any mode subset offline. They validate reconstruction to ~2e-7 against the reference and match deployed layer error to median ~1e-6 over thousands of configs. That is clean and reusable.\n\nWith it they show, across the Mamba-1 family, Falcon-Mamba 7B, and Mamba-2, that the modes carrying the signal move with the input: at the worst layers a per-window top-r set roughly halves the error of a fixed calibration set (ρ ~0.44–0.57). Freeze counterfactuals pin most of that on Bt; freezing Δ barely moves the gap. That undercuts activity-based pruning that scores on the timestep. Proposition 1 and the energy-weighted churn make the diagonal gap transparent. Code and public checkpoints are there; the math is elementary but correctly applied.\n\nSoft spots are real but already labeled. Exact offline Gram stops at Mamba-2’s gated norm, so they switch to online measurement there. The scheduled column that matches or beats the unpruned model at half state is a same-window two-pass oracle; every prefix or half-window estimator they tried recovers only a small fraction of the gain, and the sub-unpruned margin disappears out-of-window. They say this repeatedly and claim no deployed saving. Depth geometry is architecture-specific; the phenomenon and mechanism are not. Related work (GHOST, LAST, PerfMamba, classical reduction) is engaged fairly.\n\nThis is for people who prune or interpret selective SSMs, or who want a ground-truth usage measure instead of a static score. It deserves a serious referee. I would bring it to reading group and cite the instrument and the Bt result.","headline":"Exact Gram instrument for diagonal selective SSMs, multi-model Bt-driven mode migration, and honest two-pass headroom—solid measurement paper, not a free pruning win.","tokens_in":14387,"tokens_out":502,"would_cite":true,"duration_ms":4516,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Selective state-space models reallocate which modes carry the signal with every input, and that migration is driven by the write map, not the timestep.","keywords":["selective state-space models","Mamba","mode pruning","Gram tensor","input-driven migration","write map","state usage"],"falsifier":"If, on the same models and windows, freezing the write map Bt left the migration gap essentially unchanged while freezing the timestep collapsed it, or if a static energy ranking already matched the two-pass scheduled perplexity at half budget, the central claims on mechanism and headroom would fail.","tokens_in":14417,"feed_emoji":"🔄","tokens_out":769,"duration_ms":8244,"temperature":0.7,"pith_summary":"Selective state-space models such as Mamba keep a bank of fixed first-order modes and let a learned selection mechanism decide how strongly each mode is written and read. This paper builds an exact instrument that measures, for any trained layer and any input window, which modes actually carry the output and what error you pay if you drop any subset of them. The instrument shows that the important set of modes is not fixed: it migrates from context to context, so a static keep-set incurs roughly twice the error of a per-input choice at the most affected layers. Freezing each selective signal in turn pins the migration on the input-dependent write map, while the timestep that is usually treated as “selectivity” contributes almost nothing. A two-pass scheduler that simply keeps the modes the instrument measures on each window recovers that loss and, at half the state budget, matches the unpruned model on held-out perplexity from 130M parameters up to a deployed 7B model. The result is a concrete measurement of headroom rather than a deployed saving, because the scheduler still needs a first pass over the tokens it scores.","feed_headline":"Mamba reallocates its state modes with every input","feed_subtitle":"An exact instrument shows the write map drives the shift; half the modes can still match full accuracy","key_machinery":"The per-(layer, channel, window) Gram tensor built from the exact per-mode output contributions. Because the state matrix is diagonal, the squared output error of any kept set of modes is a quadratic form on that Gram matrix, giving a closed-form, offline error for every pruning budget.","core_discovery":"Because the state matrix is diagonal, each channel’s output is an exact sum of per-mode contributions. Accumulating those contributions into a small Gram tensor per layer, channel and window yields the exact squared output error of dropping any subset of modes, offline and at any budget. Using this instrument across the Mamba-1 family, Falcon-Mamba 7B and Mamba-2, the paper shows that trained models re-allocate their state with the input: at the most affected layers a per-input oracle roughly halves the error of any fixed mode set. Frozen-signal counterfactuals attribute the migration primarily to the input-dependent write map Bt; the timestep carries almost none of it. Input-scheduled pruni","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Exact instrument shows Mamba migrates modes with each input","Write map Bt drives Mamba's per-input state mode reallocation","Per-input mode oracle halves error of any fixed Mamba state set","Input-scheduled pruning matches full Mamba at half the modes","Diagonal state yields exact mode-drop error via per-window Gram"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That a two-pass oracle which reads the mode mask from the very tokens being scored counts as meaningful realizable headroom, even though every cheaper out-of-window or prefix estimator recovers only a small fraction of the gain.","fun_headline_variants_meta":{"raw":{"variants":["Exact instrument shows Mamba migrates modes with each input","Write map Bt drives Mamba's per-input state mode reallocation","Per-input mode oracle halves error of any fixed Mamba state set","Input-scheduled pruning matches full Mamba at half the modes","Diagonal state yields exact mode-drop error via per-window Gram"]},"model":"grok-4.5","effort":"low","cost_usd":0.004992,"raw_usage":{"total_tokens":1508,"prompt_tokens":966,"num_sources_used":0,"completion_tokens":92,"cost_in_usd_ticks":49920000,"prompt_tokens_details":{"text_tokens":966,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":450,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":966,"tokens_out":92,"duration_ms":4378,"temperature":1.0,"reasoning_tokens":450,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T03:06:53.300143+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"If, on the same models and windows, freezing the write map Bt left the migration gap essentially unchanged while freezing the timestep collapsed it, or if a static energy ranking already matched the two-pass scheduled perplexity at half budget, the central claims on mechanism and headroom would fail.","supporting_citations":[],"review_version":1}