{"id":"87ac89f0-d97a-483b-9451-8d7513caabb9","arxiv_id":"2602.03082","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Projected neural ODEs, exponential-map layers, and final-layer projections yield exact feasibility and universal approximation on prox-regular manifolds, with learned flow-matching projections as a fallback.","lead":"This paper groups neural-network architectures that keep predictions on a prescribed curved space (sphere, disk, rotation group, protein frames) into two families: constraints enforced at every layer or only at the output. It proves approximation theorems for both and shows in experiments that projection-based layers achieve exact feasibility, often at a small cost in prediction accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2.5 does not justify the learned projections as used: it assumes a compact boundaryless manifold and full Riemannian measure, while the experiments use empirical, narrow-support data and a backward-ODE integrator that the theorem does not cover.","rationale":"The analytic projection/exponential results are largely supported: Table 1 shows exact feasibility (mean distances ~1e-8/1e-9) for analytic updates, and the continuous-time stability argument in Theorem 2.2 is qualitatively plausible. The load-bearing gap is in the learned-projection contribution. The reader's weakest assumption identifies the measure mismatch; I agree and add that Algorithm 1's backward-ODE integration is not the map proven in Theorem 2.5, so even with full measure the theorem is not a complete justification. This is an unsupported inference from a theorem to an algorithm, not a disagreement with consensus, and it is addressable by restricting claims or providing a support-dependent projection result. The verdict should stay CONDITIONAL: the analytic architectures remain viable, but the learned-projection claims need revision.","tokens_in":29023,"tokens_out":14772,"duration_ms":164439,"concrete_test":"On SO(3), train Algorithm 1 using samples drawn only from a small geodesic ball (not the full group), then evaluate on noisy inputs whose analytic SVD projection lies outside that ball. If the learned projector returns points in or near the training support rather than the SVD projection, the full-support Riemannian-measure hypothesis in Theorem 2.5 is essential and the paper's empirical use of the learned projector is not justified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.3 and the abstract claim that flow matching can serve as a data-based metric projection. The only supporting result, Theorem 2.5, requires M compact C^∞ without boundary and dμ equal to the induced Riemannian measure on all of M. The experiments violate this: SO(3)/CS training data come from a specific dynamics rather than the uniform (Haar) measure; the protein SE(3) data occupy a small subset of a noncompact group; the disk has boundary. For an empirical or narrowly supported μ, the small-time limit of x + t∇log u_t(x) is governed by the support of μ (nearest training point as t→0), not by P_M(x). No theorem in the paper covers this regime, and no limitation is stated. Also, Algorithm 1 integrates the learned reverse-time ODE over [0,T], whereas Theorem 2.5 proves only the single-step map x ↦ x + t∇log u_t(x); no argument connects the ODE endpoint to P_M(x). The displayed estimate in Theorem 2.5 is also misstated: the proof yields an O(t^{-1/2}) gradient error, so the stated uniform bound ≤ C_K cannot hold for all sufficiently small t, although the intended O(t^{1/2}) projection bound does follow after multiplying by t. Because Experiment 1 and Flow IAA/FAA rely on this learned projector as a drop-in replacement for the analytic one, the paper's data-based projection claim lacks theoretical support in the regime studied.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified framework for geometry-preserving neural architectures, organized by whether constraints are enforced at intermediate layers (IAA) or only at the output (FAA). The main theoretical claims are: (i) projected neural ODEs preserve a prescribed prox-regular set M and approximate target flows with an error exponential in the horizon (Theorem 2.2); (ii) final projection after a δ-close unconstrained approximator loses at most 2ε (Theorem 2.3); (iii) final exponential-map architectures give L2 universal approximation on geodesically complete manifolds (Theorem 2.4); and (iv) small-time heat-kernel/flow-matching scores approximate the metric projection up to O(t^{1/2}) (Theorem 2.5). The paper also contains an empirical comparison of IAA and FAA variants on S^2, the disk, SO(3), Cucker–Smale dynamics, and protein backbone data on SE(3), including learned projections via flow matching.","tokens_in":29344,"tokens_out":12011,"duration_ms":135382,"significance":"If the results were correct as stated, the paper would fill a genuine gap: it unifies projection-based and exponential-map architectures under projected dynamical systems, and it gives quantitative approximation statements for non-convex, prox-regular constraint sets. The final-exponential-map universality theorem (Theorem 2.4), which uses measurable selections and an L2 relaxation to bypass topological obstructions, is a nice contribution. The experimental comparison of IAA versus FAA is also useful, and the paper provides unusually detailed implementation appendices and reproducibility information. However, several load-bearing statements are currently not supported as written: an architecture equation that does not preserve M, a theorem whose proof does not match its statement, an ill-posed final-projection theorem, and a learned-projection theorem that is both misstated and inapplicable to the experiments. These issues are local and fixable in principle, so the paper merits major revision rather than rejection.","major_comments":[{"comment":"Equation (6) defines the projected IAA update as h_{ℓ+1}=h_ℓ+Δt P_M(f_θ(ℓ,h_ℓ)). If P_M is the metric projection onto M, this update does not keep states in M: h_ℓ is in M but adding a projected tangent/ambient vector moves it off M. The correct update, which is actually implemented in Appendix E.3, is h_{ℓ+1}=P_M(h_ℓ+Δt f_θ(ℓ,h_ℓ)). As written, the exact-feasibility claim for the projected IAA is unsupported. A related typo appears in Eq. (5), where P_{TM(x(t))} uses an undefined x(t) and should be P_{TM(y(t))}.","section":"Section 2.1, Eq. (6)"},{"comment":"The theorem states the bound sup ‖Φ_F−Φ_fθ‖ ≤ δ/L_α(e^{L_α T}−1) with L_α=L+4U^2/α+1, but the proof does not derive this inequality. The proof obtains d/dt ‖e(t)‖² ≤ a‖e(t)‖²+δ² with a=2L+2U^2/α+1, and concludes ‖e(T)‖² ≤ δ²/a(e^{aT}−1), i.e. ‖e(T)‖ ≤ δ sqrt((e^{aT}−1)/a). This is a different bound with a different exponent and prefactor; the final displayed formula in the proof does not match the theorem statement. The authors should either prove a Lipschitz estimate for the projected vector field that yields the stated bound, or correct the theorem to match the Gronwall argument in the proof.","section":"Section B.1, Theorem 2.2"},{"comment":"The statement is ill-posed as written. F is assumed to be a continuous map F:M→M, but the hypothesis ∥F(x)−f_θ(x)∥≤ε is quantified over all x∈Ω, where Ω is a neighborhood of M that is larger than M. For x∈Ω\\M, F(x) is undefined. The proof also uses F(x)∈M for arbitrary x∈Ω. The theorem can likely be repaired by assuming F:Ω→M, or by assuming F is defined on M and using an extension or a distance-to-M argument, but as stated the theorem does not have a well-defined content.","section":"Section 2.2, Theorem 2.3"},{"comment":"There are three independent problems. First, the displayed gradient bound is misstated: the proof yields ∇_x log u_t(x)+(x−P_M(x))/t = O(t^{-1/2}), so a uniform bound ≤C_K independent of t cannot hold for all sufficiently small t. What follows from the proof is O(t^{1/2}) for x+t∇_x log u_t(x)−P_M(x), not for the gradient error. Second, the hypotheses require M to be compact, boundaryless, and μ to be the induced Riemannian measure on all of M. The experiments use the disk (which has boundary) and empirical/narrow-support distributions on SO(3), SE(3), and protein data; for empirical μ the small-time limit of x+t∇_x log u_t(x) is controlled by the nearest training point, not by P_M(x). No limitation is stated. Third, Algorithm 1 integrates a learned reverse-time ODE over [0,T], whereas Theorem 2.5 analyzes only the single-step map x+t∇_x log u_t(x). The learned-projection claim therefore","section":"Section 2.3, Theorem 2.5"},{"comment":"The paper claims that the flow-matching conditional velocity satisfies v(t,x)=∇_x log μ_t(x), but this is false for the generative process defined in Algorithm 1, Y_t=X+v t with v∼N(m,I). For m=0 and a point mass at X=0, Y_t=tV with V∼N(0,I), so μ_t=N(0,t²I), v(t,x)=E[V|Y_t=x]=x/t, while ∇_x log μ_t(x)=−x/t². These are not equal; the general relation for this path is E[V|Y_t=x]=−t∇_x log μ_t(x) (up to the mean m). Consequently, training v_θ by flow matching does not estimate the score appearing in Theorem 2.5. The connection between the learned velocity and the heat-kernel projection estimate needs to be re-derived or the algorithm and theorem must be matched.","section":"Section 2.3, Algorithm 1"}],"minor_comments":[{"comment":"There are several typos: “we also the demonstrate” in the abstract; “Let M⊆R^d be a be a closed subset” in Section 2; Figure 1 caption says “Final Layer Augmented (IAA)” where it should be FAA. These should be corrected.","section":"Abstract and Introduction"},{"comment":"The text states that “C² manifolds have uniformly positive reach.” This is not true without compactness or a uniform tubular-neighborhood bound; a noncompact C² submanifold can have reach 0. The statement should be restricted to compact or otherwise bounded-geometry submanifolds.","section":"Section 2.1, preliminaries"},{"comment":"The notation defines P_A(x) as argmin_y ∥x−y∥ for all x, but the metric projection is set-valued in general and is only single-valued inside the reach. The later theorems assume the single-valued regime; the notation table should acknowledge this distinction.","section":"Notation, Table of symbols"},{"comment":"The theorem introduces a Borel measure μ on M, but the L² norm is taken over Ω⊂R^d. The measure should be specified as a measure on Ω, or the text should clarify how μ is extended. Also the sentence “the corresponding constrained approximation is defined by composition with the projection map” in the exponential FAA paragraph should say “exponential map.”","section":"Section 2.2, Theorem 2.4"},{"comment":"The learned-projection evaluation integrates the backward ODE with RK45 from t=T to t=0, while the training uses a time grid up to T. The relation between the integration horizon and the small-time regime of Theorem 2.5 is not discussed; this is relevant to the theoretical gap identified above.","section":"Appendix D.6"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid organizing idea and a serious experimental appendix, but the current version contains several load-bearing mathematical inconsistencies: an architecture equation that does not enforce feasibility, a theorem/proof mismatch, an ill-posed theorem statement, and a learned-projection theorem that is both misstated and inapplicable to the experiments. All of these appear fixable in a revision. I would encourage the editor to send the manuscript back for major revision rather than reject it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. First, the core idea is useful: placing projection or exponential-map updates either interleaved (IAA) or only at the output (FAA) under one prox-regular umbrella is a sensible way to organize the literature, and the approximation theorems, once corrected, will be worth having. Second, the paper in its current form has a gap between what it proves and what it claims.\n\nWhat's actually new and good: Theorem 2.4 is a clean measurable-selection result: on a geodesically complete manifold, any continuous map into M can be written as exp_p∘f with f measurable and bounded, and L2 approximation follows from Lipschitzness of exp_p. That's a solid generalization of the Euclidean-homeomorphism case. Theorem 2.3, modulo a domain issue, is a simple and correct 2ε bound. The numerical demonstration that analytic projection keeps states on the manifold to ~1e-8 is convincing. The Lie-group coordinate simplification (Eq. 10) is also practical.\n\nThe soft spots are real. Eq. (6) does not preserve M as written—it projects the vector field, not the state; the appendix implements the state projection, which is the right thing, but the main text is wrong. Theorem 2.2's stated bound Lα:=L+4U²/α+1 does not match the proof, which yields a different constant and a different exponent structure; the qualitative stability claim is plausible but the theorem as stated is false. Theorem 2.3 assumes ∥F(x)-fθ(x)∥≤ε for all x∈Ω, yet F is only defined on M; the statement needs to be restricted to M or F extended. Theorem 2.5 is the biggest issue: it proves a single-step map x+t∇log u_t(x)≈P_M(x) for compact boundaryless manifolds with the full Riemannian measure. The experiments use SO(3) trajectory data, a narrow SE(3) protein subset, and the disk with boundary—none satisfying the assumptions—and the learned projection is applied by integrating a backward ODE, which the theorem doesn't cover. Unsurprisingly, Flow IAA/FAA distances are ~1e-2 to 1e-0, no better than unconstrained. The abstract's claim that flow-matching 'can be used as data-based projections' is not supported. The abstract also says FAA outperforms in most tasks, but Table 1 shows FAA MSE is worse than the regular ResNet on most datasets.\n\nThis is a fixable paper. The framework is good, the theorems are close, and the errors are of the kind competent authors can repair. Who should read it: anyone building constraint-preserving network layers or studying neural ODEs on manifolds. Should a serious editor send it to review? Yes—the core content merits referee time. But my recommendation to the authors would be major revision, not acceptance.","headline":"A promising framework with several theorems that don't yet match their proofs and an overclaimed learned-projection story; fixable, but not ready as written.","tokens_in":29900,"tokens_out":5104,"would_cite":false,"duration_ms":53184,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","53Z50","58A05"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that interleaving projections or exponential-map updates between neural layers keeps every intermediate state on a prescribed manifold with boundary, and that the resulting approximation error can be controlled by explicit","keywords":["geometry-preserving architectures","projected neural ODEs","manifolds with boundary","metric projection","exponential map","flow matching","universal approximation","prox-regular sets"],"falsifier":"Train the flow-matching projector on a narrow empirical subset of SO(3), such as rotations near the identity, and test on rotations far from that subset; if the learned map does not move noisy inputs to their nearest rotation, the general learned-projection claim fails. Alternatively, on a full-measure sphere dataset, compute ∥x + t∇log u_t(x) - P_M(x)∥ as t→0; if it does not decay as O(t^(1/2)), Theorem 2.5's rate is contradicted.","tokens_in":28869,"feed_emoji":"📐","tokens_out":4290,"duration_ms":47890,"temperature":0.7,"pith_summary":"The paper tries to establish a unified theory for geometry-preserving neural architectures: both interleaving a projection or exponential update at every layer and applying a single geometric correction at the output can be seen as discretizations of projected dynamical systems on manifolds with boundary. If the learned vector field is δ-close to a target field, the endpoint error of a projected neural ODE is bounded by an explicit exponential-in-time expression; if only the final layer is projected, fidelity degrades by at most a factor of two. The paper also proves that a single exponential map from a base point can represent any continuous target in an L2 sense on geodesically complete manifolds, and that small-time heat-kernel or flow-matching scores approximate the metric projection up to O(t^(1/2)). A sympathetic reader would care because exact feasibility plus controlled approximation is what makes constrained neural architectures reliable for prediction on manifolds such as SO(3), SE(3), the sphere, and the disk.","feed_headline":"Projections at every layer keep neural outputs on the manifold","feed_subtitle":"New error bounds cover manifolds with boundary and a learned flow-matching projection when no analytic one exists.","key_machinery":"The central device is the projected dynamical system ẋ = P_{T_M(x)} F(t, x), in which the velocity is projected onto the tangent cone at the current point; its Euler discretization is the interleaved projection layer, and its intrinsic discretization is the Riemannian exponential update. The theory rests on uniformly prox-regular sets, meaning sets with positive reach so that every point within some distance has a unique nearest point on the set; the associated normal-cone inequality supplies the differential-inequality argument in the error bound. For the learned projection, small-time heat-kernel asymptotics make the score of the smoothed density point toward the metric projection, and the","core_discovery":"On the paper's own terms: projected neural ODEs, where the learnable vector field is projected onto the tangent cone of a prox-regular set at every instant, keep the set invariant by construction and can approximate any sufficiently regular target flow with error bounded by δ/Lα(e^{LαT} - 1). Geometry-preserving architectures built by projecting after each residual layer inherit this guarantee. Enforcing constraints only at the output costs at most a factor of two in approximation error, and exponential-map output layers achieve universal approximation in L2 on geodesically complete manifolds. When no analytic projection exists, a reverse-time flow-matching field learned from heat-kernel-smo","pith_inferences":["The learned-projection guarantee is proven for compact boundaryless submanifolds with training samples drawn from the full Riemannian measure; applying the flow-matching projector to datasets such as protein backbones on SE(3), where samples cover a narrow empirical subset, extends the claim beyond its proven regime.","Because the learned projection is accurate only for inputs within the reach and for small diffusion time, networks whose intermediate states drift far from the manifold may not be corrected; the paper's own experiments show flow-matching variants did not match analytic projections.","A testable extension is to train the flow-matching projector with a time schedule that adapts to local reach rather than a fixed small diffusion time, which could make learned projections viable on empirical manifold distributions.","The O(t^(1/2)) rate suggests a practical diagnostic: on held-out noisy inputs, measure ∥x + t∇log u_t(x) - P_M(x)∥; if it does not shrink like t^(1/2), the learned score has not converged to the metric projection."],"forward_implications":["Projected and exponential intermediate-layer architectures give exact feasibility on manifolds with boundary, not just in the limit, for every intermediate and final layer.","If a neural vector field approximates the target in sup-norm to δ, the endpoint approximation error is bounded by δ/Lα(e^{LαT} - 1), so fidelity degrades at worst exponentially in time and inversely with the reach of the constraint set.","Final-only projection is a safe simplification: it at most doubles the ambient approximation error, and it is computationally cheaper in the paper's experiments.","On any geodesically complete Riemannian manifold, a fixed base-point exponential map combined with an L2-close network can approximate any continuous target, avoiding topological obstructions that block uniform approximation."],"fun_headline_variants":["Project every step: neural ODEs stay on the manifold","New error bounds for geometry-preserving nets on boundaries","Learn missing projections via heat-kernel flow matching","Output-only constraint nets beat layer-wise in most tasks"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The learned projection that stands in for an analytic projection is guaranteed to be a metric projection only when the constraint set is a compact smooth manifold without boundary and the training data covers the whole manifold according to its Riemannian measure; the paper still applies it to empirical distributions over narrow subsets of the constraint set, where that guarantee does not hold.","fun_headline_variants_meta":{"raw":{"variants":["Project every step: neural ODEs stay on the manifold","New error bounds for geometry-preserving nets on boundaries","Learn missing projections via heat-kernel flow matching","Output-only constraint nets beat layer-wise in most tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000276,"raw_usage":{"total_tokens":1470,"prompt_tokens":721,"completion_tokens":749,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":697}},"tokens_in":465,"tokens_out":749,"duration_ms":8859,"temperature":1.0,"reasoning_tokens":697,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T05:06:15.472481+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the flow-matching projector on a narrow empirical subset of SO(3), such as rotations near the identity, and test on rotations far from that subset; if the learned map does not move noisy inputs to their nearest rotation, the general learned-projection claim fails. Alternatively, on a full-measure sphere dataset, compute ∥x + t∇log u_t(x) - P_M(x)∥ as t→0; if it does not decay as O(t^(1/2)), Theorem 2.5's rate is contradicted.","supporting_citations":[],"review_version":1}