{"id":"c6b55b13-8535-4468-a19a-51010623937c","arxiv_id":"2502.03006","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An augmented backward-corrected projector-splitting integrator (abc-PSI) trains rank-adaptive low-rank neural networks with one QR decomposition per step and a claimed convergence guarantee to locally optimal weights.","lead":"The paper introduces a rank-adaptive algorithm for low-rank neural network training that modifies the backward-corrected projector-splitting integrator with a basis augmentation step, cutting the number of QR decompositions per step from two to one. It proves a loss-descent bound and a claimed local-convergence guarantee, with experiments on MNIST and vision transformer fine-tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 5.9's proof uses a false identity: the augmented basis does not annihilate the projected-gradient residual, so the convergence theorem is unproven as written.","rationale":"The reader's weakest assumption identifies exactly the false identity in Theorem 5.9, and my independent re-derivation agrees. The paper's central mathematical claim is convergence to locally optimal weights, established only through Theorem 5.9. The step from (5.13) to (5.15) requires replacing bU bU^T grad l by P(Y)grad l, which the proof justifies with the identity following (5.15). That identity is algebraically false: the augmented left basis contains U and grad l V, but not the left residual (I - UU^T) grad l (I - VV^T). The 2x2 counterexample makes the failure concrete. Because this is an internal error rather than a disagreement with prior consensus, it is a correctness risk, not a stylistic one. Secondary concerns strengthen rather than weaken the verdict: the rank-stabilization assumption sum_t E[||Yt - bYt||] <= D is stated without proof, and the robust error bound in Theorem 5.6 contains a term vartheta/h whose scaling needs justification. None of this impugns the empirical contribution; the MNIST and ViT results may stand, but they do not supply the missing proof. The reader's REJECT verdict is therefore appropriate, and my stress-test does not change it.","tokens_in":23325,"tokens_out":14737,"duration_ms":136038,"concrete_test":"Perform a symbolic check of the identity after (5.15) in R^{2x2} with r=1: take Y0 = s e1 e1^T, grad l(Y0) = e2 e1^T + e2 e2^T, and h>0. Compute bU = orth([s e1, s e1 - h grad l(Y0)e1]) and evaluate bU bU^T (I - P(Y0)) grad l(Y0). If this is nonzero, the identity used in the proof of Theorem 5.9 is false. An optional follow-up is to verify whether the norm inequality ||bU bU^T grad l|| >= ||P(Y)grad l|| nevertheless rescues the Robbins-Monro argument in (5.15).","verdict_should_be":"UNCHANGED","load_bearing_attack":"At the equation following (5.15), the proof of Theorem 5.9 asserts bU_t bU_t^T (I - P(Y_{t-1})) grad l(Y_{t-1}) = bU_t bU_t^T (I - U_{t-1}U_{t-1}^T) grad l(Y_{t-1}) (I - V_{t-1}V_{t-1}^T) = 0. The first equality is just the definition of the tangent-space projector. The final equality is false in general. At iteration t, bU_t is an orthonormal basis of span([U_{t-1}S_{t-1}, U_{t-1}S_{t-1} - h grad l(Y_{t-1})V_{t-1}]), hence it contains U_{t-1} and grad l(Y_{t-1})V_{t-1}. The residual (I - P)grad l = (I - UU^T) grad l (I - VV^T) is right-orthogonal to V and, in general, has left components outside span(grad l V), so bU_t need not annihilate it. Concrete 2x2 counterexample: Y0 = s e1 e1^T and grad l(Y0) = e2 e1^T + e2 e2^T. Then P(Y0)grad l = e2 e1^T, the residual is e2 e2^T, and for h>0, bU = orth([s e1, s e1 - h e2]) spans R^2, so bU bU^T residual = e2 e2^T is nonzero. Hence bU bU^T grad l != P(Y)grad l, and the chain converting the descent bound (5.13) into the bound (5.15) breaks. The theorem is not established as written. A norm inequality may repair the proof, but it is not what is stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes abc-PSI, an augmented backward-corrected projector-splitting integrator for dynamical low-rank training (DLRT). The method adds a rank augmentation step to the backward-corrected PSI of [2], claiming rank-adaptivity while preserving descent and local convergence guarantees, and reduces the number of QR decompositions per training step from two to one. The theoretical part contains a descent lemma (Theorem 5.8), a robust error bound (Theorem 5.6), and a convergence-to-local-optimum theorem (Theorem 5.9). Experiments on MNIST and on fine-tuning a vision transformer on CIFAR-10/100 are presented to support the method's practicality.","tokens_in":23722,"tokens_out":7117,"duration_ms":54997,"significance":"If the proven guarantees hold as stated, the contribution is valuable: abc-PSI would be the first PSI-type DLRT method with a local convergence guarantee and rank adaptivity, at reduced per-iteration cost compared to BUG-type integrators. The descent argument (Theorem 5.8) is elementary and correct, and the numerical experiments are consistent with the method being practically useful. The construction is not circular and does not rely on fitted constants. However, the proof of the central convergence theorem (Theorem 5.9) uses a false identity, so the claimed local convergence guarantee is not established in this manuscript.","major_comments":[{"comment":"The equation following (5.15) asserts that bU_t bU_t^T (I - P(Y_{t-1})) grad l(Y_{t-1}) = 0. This identity is false in general. The augmented basis bU_t contains U_{t-1} and grad l(Y_{t-1})V_{t-1}, but the residual (I - P(Y_{t-1}))grad l(Y_{t-1}) = (I - U_{t-1}U_{t-1}^T) grad l(Y_{t-1}) (I - V_{t-1}V_{t-1}^T) is right-orthogonal to V_{t-1} and in general has left components outside span(grad l V), so the projection onto span(bU_t) need not vanish. A concrete counterexample is Y0 = s e1 e1^T and grad l(Y0) = e2 e1^T + e2 e2^T. Then P(Y0)grad l = e2 e1^T, the residual is e2 e2^T, and for h>0 the augmented basis bU = orth([s e1, s e1 - h e2]) spans R^2, so bU bU^T residual = e2 e2^T ≠ 0. Consequently, the identity bU_t bU_t^T grad l = P(Y_{t-1})grad l used to pass from (5.15) to the limit statement fails, and Theorem 5.9 is not proven as written. A norm estimate controlling the extra term could repair the argument, but it is not present in the manuscript.","section":"Section 5.7, proof of Theorem 5.9"},{"comment":"The local error after truncation is stated as h*epsilon + 3*cl*h^2 + vartheta/h, but the truncation is defined by ||bY1 - Y1|| <= vartheta in the paragraph before Theorem 5.6. Therefore the local error bound should contain vartheta, not vartheta/h. The global bound can still acquire a vartheta/h contribution after summing over T/h steps, but the local error expression is incorrect as written and should be corrected.","section":"Section 5.5, proof of Theorem 5.6"},{"comment":"The remark states that the robust error bound, loss descent, and convergence results are proven for the discrete Algorithm 4.1, yet the proofs of Theorems 5.6, 5.8, and 5.9 operate with the continuous evolution equations (4.1), including integrals over the step. The manuscript should either state clearly that the theory applies to the continuous-in-time integrator (4.1) and add a separate discretization analysis for Algorithm 4.1, or modify the proofs to cover the explicit Euler update (4.3) directly.","section":"Section 5.5, remark before Theorem 5.6"}],"minor_comments":[{"comment":"The proof contains a typo: 'Cauchy-Schwartz' should be 'Cauchy-Schwarz'.","section":"Section 5.4, Lemma 5.4"},{"comment":"The assumption sum_t E||Yt - bYt|| <= D is asserted with the gloss 'i.e., for sufficiently large t, the rank stabilizes', but no argument links the truncation rule of Section 4 to this boundedness. Since this assumption is not verified by the algorithm, the theorem's conclusion is conditional; the authors should either prove that the truncation rule enforces this condition or state explicitly that it is an additional regularity assumption.","section":"Theorem 5.9"},{"comment":"In Table 1, several bc-PSI entries are marked with '-' without explanation; the caption should state that these correspond to failed training runs, and ideally the number of failures across seeds should be reported for all entries.","section":"Tables 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The dependence on unpublished or concurrent work for supporting results ([21] is listed as 'TBA', [40] is GeoLoRA) makes independent verification harder; this is worth keeping in mind for the revision. The main issue is the unproven Theorem 5.9; the authors may need to either provide a corrected proof with additional structural assumptions on the gradient or downgrade the claim to a weaker statement. The numerical results are encouraging but do not compensate for the missing proof of the central theoretical guarantee."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Kusch, Schotthöfer, and Walter propose abc-PSI, an augmented backward-corrected projector-splitting integrator for low-rank training. The new construction is a rank-doubling augmentation of the bc-PSI plus a truncation step; it is genuinely new as a combination, and the algorithmic claim holds: one QR per step instead of two, with competitive MNIST and ViT fine-tuning results. The memory-footprint argument for the K/L steps is also credible. The descent inequality in Theorem 5.8 is clean and correctly proven.\n\nThe soft spot is load-bearing. The proof of Theorem 5.9 uses the identity bU_t bU_t^T (I - P(Y_{t-1})) grad l(Y_{t-1}) = 0, claiming the augmented basis annihilates the projected-gradient residual. That identity is false in general. bU_t spans the old K and its negative-gradient update, so it contains U_{t-1} and grad l(Y_{t-1})V_{t-1}, but the residual (I - U_{t-1}U_{t-1}^T) grad l (I - V_{t-1}V_{t-1}^T) is right-orthogonal to V and its left component can lie outside span(grad l V). A 2x2 counterexample (Y0 = s e1 e1^T, grad l = e2 e1^T + e2 e2^T) gives bU bU^T residual = e2 e2^T nonzero. So the chain from (5.13) to (5.15) breaks, and Theorem 5.9 is not established. Since 'convergence to a locally optimal solution' is the paper's headline claim, this matters. A norm inequality may repair the proof, but that repair is not in the current text.\n\nTwo smaller issues. The rank-stabilization assumption in Theorem 5.9 (sum of expected truncation errors bounded by D) is asserted without argument. And Theorem 5.6's proof has a local-error line with ϑ/h where the truncation term should be ϑ; the global ϑ/h is what should emerge from the Lady Windermere fan argument. Both are fixable.\n\nOn citation pattern: the authors lean on their own [21] (a 'TBA' preprint) and [40] (GeoLoRA), but the central construction is not circular, and the descent theorem is self-contained. I don't see invented entities or fitted constants.\n\nWho is this for? Researchers in low-rank training/fine-tuning who want a computationally cheaper rank-adaptive PSI variant. The empirical section is useful, and the descent guarantee is real.\n\nRecommendation: send this to serious peer review, not desk reject. The method is novel and the experiments show value. But ask for a corrected proof of Theorem 5.9 or an explicit weakening of the convergence claim before acceptance.","headline":"A useful rank-adaptive PSI variant with a correct descent bound, but the central convergence theorem rests on a false projection identity and is unproven as written.","tokens_in":24294,"tokens_out":5432,"would_cite":false,"duration_ms":42779,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","49Q12","65L05","65L20","65L70"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that augmenting the left basis in a backward-corrected projector-splitting integrator yields a rank-adaptive low-rank training method with guaranteed loss descent, local convergence to locally optimal weights, and only…","keywords":["dynamical low-rank approximation","projector-splitting integrator","low-rank neural network training","rank adaptivity","local convergence","parameter-efficient fine-tuning","stochastic gradient descent"],"falsifier":"For a small random low-rank matrix $Y=USV^\\top$, define the augmented basis $\\widehat U=\\mathrm{ortho}([U,K_1])$ with $K_1=US - h\\nabla\\ell(USV^\\top)V$, and compute $R=\\widehat U\\widehat U^\\top(I-P(Y))\\nabla\\ell(Y)$. If $\\|R\\|>0$ for some gradient whose residual has a component in $\\mathrm{span}(\\nabla\\ell(Y)V)$ orthogonal to $U$, the identity in the proof of Theorem 5.9 is false and the stated convergence guarantee needs an additional assumption.","tokens_in":23055,"feed_emoji":"🧮","tokens_out":7234,"duration_ms":56463,"temperature":0.7,"pith_summary":"This paper introduces abc-PSI, an augmented backward-corrected projector-splitting integrator for training neural networks with low-rank weights. The authors claim that adding the previous left basis to the updated K-factor before the L-step makes the loss decrease provably and lets the iterates converge to weights satisfying the local optimality condition for the low-rank manifold. The same augmentation enables dynamic rank adjustment through a singular-value truncation, and the algorithm needs only one QR decomposition per step instead of two. If the claims hold, low-rank training becomes simultaneously more robust, cheaper, and rank-adaptive, with the same per-step cost structure as gradient descent on the factorized matrices.","feed_headline":"One-QR integrator makes rank-adaptive low-rank training converge","feed_subtitle":"Augmented backward-corrected PSI claims guaranteed descent, local optimality, and fewer decompositions per step.","key_machinery":"The load-bearing object is the left-basis augmentation step. At each step $\\widehat U_{k+1}=\\mathrm{ortho}([U_k, K_{k+1}])$, where $K_{k+1}$ is the result of the K-step. Because the augmented basis contains both the old left basis and the updated K factor, the projection $P_{\\widehat U_{k+1}}=\\widehat U_{k+1}\\widehat U_{k+1}^\\top$ satisfies $P_{\\widehat U_{k+1}}U_k=U_k$ and $P_{\\widehat U_{k+1}}K_{k+1}=K_{k+1}$, so the problematic residual $(I-U_1U_1^\\top)Y_0$ from the bc-PSI bound vanishes. The same augmented basis is reused in the L-step initial condition $L(t_0)=V_0\\bar S_1^\\top$, and a truncated SVD then selects the next rank $r_{k+1}$ by asking that the omitted squared singular values stay below $\\tau\\|\\Sigma\\|$. The backward-corrected projection replaces the original PSI's backward-in-time S-step, so every substep moves forward in time.","core_discovery":"The paper's central claim is that the augmented backward-corrected PSI (Algorithm 4.1) is a projector-splitting integrator that combines rank adaptivity with guaranteed loss descent and local convergence. Its construction replaces the S-step of the PSI by the projection $\\bar S_1 = \\widehat U_1^\\top U_0 S_0$ from the backward-corrected PSI, and then augments the left basis to $\\widehat U_1 = \\mathrm{ortho}([U_0, K(t_1)])$, the orthonormal span of the old basis and the updated K-factor. This removes the term $\\|(I-U_1U_1^\\top)Y_0\\|$ that blocked descent guarantees for the non-augmented version; Theorem 5.8 then gives $\\ell(\\widehat Y_1)\\le \\ell(Y_0) - (1-h c_l/2) h \\|P_{\\widehat U_1}\\nabla\\ell(Y_0)\\|^2$, and Theorem 5.9 concludes $\\liminf_{T\\to\\infty}\\mathbb{E}[\\|P(Y_T)\\nabla\\ell(Y_T)\\|^2]=0$ under Robbins-Monro learning rates. A truncation step based on an SVD with tolerance $\\tau$ makes the rank dynamic, and each iteration costs one QR decomposition plus this SVD, versus two QR decompositions for BUG-type integrators.","pith_inferences":["If the asserted annihilation identity in Theorem 5.9 cannot be repaired, the practical algorithm may still work, but the local-convergence guarantee as stated would reduce to a conjecture; a numerical check on random instances would settle this quickly.","The same augmentation-then-truncate recipe could be carried over to other splitting integrators or to Tucker-format tensor training, where the analogous projection onto the span of old basis plus updated core would play the same role.","Because the algorithm touches only the K and L factorizations at any given time, with U, S, and V co-materialized only in the truncation step, an implementation that exploits this could reduce peak memory below what simultaneous-update low-rank methods achieve."],"forward_implications":["Rank-adaptive low-rank training can be run with a monotone loss decrease for learning rates $h\\le 2/c_l$, matching the descent behavior of BUG-based dynamical low-rank training.","Each training step costs one QR decomposition plus one SVD on matrices of rank at most $2r$, making abc-PSI cheaper per step than BUG-type integrators that need two QR decompositions.","The method applies to fine-tuning pretrained networks, with additive low-rank corrections evolved by the same integrator; the reported experiments reach the accuracy of established parameter-efficient adaptation baselines with fewer trainable parameters.","Local convergence to a point satisfying the manifold optimality condition $P(Y_\\star)\\nabla\\ell(Y_\\star)=0$ is asserted for stochastic gradients under Robbins-Monro step sizes and bounded cumulative truncation error."],"supporting_citations":[{"why":"Introduces the projector-splitting integrator that abc-PSI modifies.","marker":"[34]"},{"why":"Proves the robust error bound for the PSI that secures the corresponding bound for the backward-corrected variants.","marker":"[27]"},{"why":"Proposes the backward-corrected PSI that the augmentation step builds on.","marker":"[2]"},{"why":"Introduces the unconventional BUG integrator, the baseline whose per-step QR and SVD count abc-PSI compares against.","marker":"[8]"},{"why":"Supplies the rank-augmentation and singular-value truncation criterion adopted by the abc-PSI algorithm.","marker":"[6]"},{"why":"Provides the stochastic-gradient descent lemma and convergence framework used in Theorems 5.8 and 5.9.","marker":"[21]"},{"why":"Supplies the low-rank fine-tuning comparison and the style of local-convergence argument that Theorem 5.9 adapts.","marker":"[40]"},{"why":"Defines the manifold optimality criterion that characterizes the locally optimal weights reached by the method.","marker":"[37]"}],"fun_headline_variants":["Augmented PSI: one QR per step, provable loss descent","Fewer decompositions, same convergence: backward-corrected PSI","Dynamical low-rank training with one QR and robust descent","New integrator slashes QR calls while ensuring local optimality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The convergence proof in Theorem 5.9 assumes the augmented basis makes $\\widehat U_t\\widehat U_t^\\top(I-P(Y_{t-1}))\\nabla\\ell(Y_{t-1})=0$, meaning the part of the gradient left over after projecting onto the current tangent space is entirely annihilated by the augmented projection; if that identity fails, the proof's replacement of $\\widehat U_t\\widehat U_t^\\top\\nabla\\ell$ by $P(Y_{t-1})\\nabla\\ell$ does not go through.","fun_headline_variants_meta":{"raw":{"variants":["Augmented PSI: one QR per step, provable loss descent","Fewer decompositions, same convergence: backward-corrected PSI","Dynamical low-rank training with one QR and robust descent","New integrator slashes QR calls while ensuring local optimality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000612,"raw_usage":{"total_tokens":2862,"prompt_tokens":974,"completion_tokens":1888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":1813}},"tokens_in":590,"tokens_out":1888,"duration_ms":15971,"temperature":1.0,"reasoning_tokens":1813,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T10:14:14.031988+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a small random low-rank matrix $Y=USV^\\top$, define the augmented basis $\\widehat U=\\mathrm{ortho}([U,K_1])$ with $K_1=US - h\\nabla\\ell(USV^\\top)V$, and compute $R=\\widehat U\\widehat U^\\top(I-P(Y))\\nabla\\ell(Y)$. If $\\|R\\|>0$ for some gradient whose residual has a component in $\\mathrm{span}(\\nabla\\ell(Y)V)$ orthogonal to $U$, the identity in the proof of Theorem 5.9 is false and the stated convergence guarantee needs an additional assumption.","supporting_citations":[{"cited_title":"Hnatiuk, J","cited_arxiv_id":null,"evidence_quote":"Provides the stochastic-gradient descent lemma and convergence framework used in Theorems 5.8 and 5.9."},{"cited_title":"GeoLoRA: Geometric integration for parameter efficient fine-tuning","cited_arxiv_id":"2410.18720","evidence_quote":"Supplies the low-rank fine-tuning comparison and the style of local-convergence argument that Theorem 5.9 adapts."},{"cited_title":"Sato, Riemannian optimization and its applications , vol","cited_arxiv_id":null,"evidence_quote":"Defines the manifold optimality criterion that characterizes the locally optimal weights reached by the method."}],"review_version":1}