{"id":"49a5d383-0ff1-47f1-8b8f-708232cdbedf","arxiv_id":"2505.11360","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ProjectNet learns a matrix-parameterized update rule that approximates optimization solutions in a few forward steps, and using this surrogate in end-to-end training cuts training time by 2 to 10 times while keeping decision costs competitive.","lead":"This paper introduces ProjectNet, a neural network that learns to quickly approximate the solution of an optimization problem, replacing the exact solver inside end-to-end decision-focused training. It reports 2 to 10 times faster training across newsvendor, electricity planning, and shortest path tasks, with competitive decision costs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The implemented L(u)=M M^T ρ(D)(M M^T)^{-1} is non-symmetric, so positive eigenvalues do not imply w^T L w ≥ 0; the convexity/strong-convexity basis of Propositions 2–3 and Theorem 2 is not satisfied by the actual ProjectNet.","rationale":"The reader's weakest assumption is exactly the load-bearing issue: the constructed L(u) is not symmetric and is not PSD in the quadratic-form sense required by the analysis. I agree this is the central gap. The same construction also undermines the Lipschitz bound in Theorem 2, because that proof bounds I - γL using a scalar eigenvalue while ignoring non-symmetry. The step-size mismatch in Proposition 3 is an additional independent issue, but it is secondary: even with a symmetric L, the proof would need to be restated for the actual update. These are correctness problems in the theoretical claims, not in the empirical method as such, and they are repairable. Since the reader already assigned CONDITIONAL for the same reason, my stress-test does not change the recommended verdict.","tokens_in":26793,"tokens_out":4919,"duration_ms":48281,"concrete_test":"Compute the minimum eigenvalue of the symmetric part of L as defined in Eq. (63) for a fixed small instance, e.g. M = [[1,1],[0,1]], D = diag(1,10), S = MM^T, L = S D S^{-1}. If min eig((L+L^T)/2) < 0 (here ≈ -8.73), the surrogate is indefinite and the PSD premise of Propositions 2–3 is violated. Additionally, evaluate this quantity on the trained ProjectNet's L(u) over the electricity test set; any negative eigenvalue across test inputs confirms the implemented model falls outside the paper's theoretical assumptions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The theoretical core treats r_u(w) = g_u(w) + (γ/2η) w^T L(u) w as a strongly convex, smooth surrogate and derives regret, exponential convergence, and generalization in Propositions 2–3 and Theorem 2 using eigenvalues of L. The construction actually implemented in §3.2.2, Eq. (63), is L(u) = S(u) ρ(D(u)) S(u)^{-1} with S = M M^T symmetric positive definite. Such a matrix has positive eigenvalues (the entries of ρ(D)), but it is generally non-symmetric, and positive eigenvalues do not imply w^T L w ≥ 0. Convexity of the surrogate requires the symmetric part (L+L^T)/2 to be PSD, which Eq. (63) does not enforce. Example: M = [[1,1],[0,1]] and D = diag(1,10) give L = [[-8,18],[-9,19]], whose symmetric part has eigenvalues ≈19.7 and ≈ -8.7; the quadratic form is indefinite. Therefore Proposition 3's strong-convexity bound σ_min(∇²r) ≥ (γ/η)σ_min(L) and Proposition 2's regret inequality involving w^T L w can fail in either direction for the trained architecture. Theorem 2's proof additionally relies on ∥I - γL∥ ≤ 1 - γσ_min, which again presumes a symmetric PSD L (and γσ_min ≤ 1). A separate step-size gap: the proof of Proposition 3 invokes the standard rate for stepsize 1/β, whereas the implemented update uses step η; without η = 1/β the stated rate does not follow. These flaws are repairable by symmetrizing L or enforcing PSD of (L+L^T)/2 and by redoing the step-size analysis, but as written the guarantees do not cover ProjectNet.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ProjectNet, a meta-optimization method for end-to-end learning in which the exact optimization oracle is replaced by a learned, differentiable approximation. The forecast u is fed into an unrolled update w_{t+1} = π_P(w_t − η∇g_u(w_t) − γ L(u)w_t), run for T iterations, with L(u) learned to minimize the empirical decision cost. The authors prove a regret bound (Proposition 2), exponential convergence for the surrogate (Proposition 3), a generalization bound (Theorem 2), and an approximation-impact decomposition (Theorem 1). They also show that point forecasts suffice for loss-type objectives (Propositions 1 and 4). Experiments on a two-stage cross-fulfillment newsvendor problem, a PJM electricity planning problem, the Warcraft shortest-path benchmark, and synthetic matching/newsvendor problems report 2–10x training-speed improvements over OptNet, CVXPY, and Vlastelica et al. with competitive task costs.","tokens_in":27189,"tokens_out":10270,"duration_ms":97656,"significance":"The central idea—eliminating exact solver calls from end-to-end training by learning a feasible, differentiable surrogate—is practically attractive, and the empirical work is a genuine strength: it uses real PJM data, established benchmarks, and careful runtime/accuracy comparisons. If the theoretical gaps are repaired, the paper would make a useful contribution to decision-focused learning. However, the current guarantees do not cover the architecture actually implemented and evaluated: the learned L(u) in Eq. (63) is not positive semidefinite in the quadratic-form sense required by Propositions 2–3 and Theorem 2, and the proofs use an exact projection while the implementation uses a finite-k approximate projection. These issues are repairable, but they are load-bearing for the paper's theoretical claims.","major_comments":[{"comment":"The construction L(u) = (M(u)M(u)^T) ρ(D(u)) (M(u)M(u)^T)^{-1} has positive eigenvalues because it is similar to the diagonal matrix ρ(D(u)), but it is not symmetric in general, and positive eigenvalues do not imply w^T L w ≥ 0. The analysis in §3.1 explicitly assumes L(u) is positive semidefinite, and Propositions 2–3 and Theorem 2 rely on w^T L w ≥ 0, on σ_min(L) as a strong-convexity parameter, and on the inequality ||I − γL|| ≤ 1 − γσ_min. For a concrete counterexample, M = [[1,1],[0,1]] and D = diag(1,10) give L = [[−8,18],[−9,19]], whose symmetric part has a negative eigenvalue and hence an indefinite quadratic form. Please modify the construction—for example, use L(u) = S ρ(D) S with S symmetric positive definite, or explicitly enforce PSD of (L+L^T)/2—and re-derive the affected bounds for that construction.","section":"§3.2.2, Eq. (63); §3.1"},{"comment":"The proof invokes the standard projected gradient descent rate for step size 1/β, but the implemented update in Eq. (44) is w_{t+1} = π_P(w_t − η∇g_u(w_t) − γL(u)w_t), which equals a projected gradient step on r_u(w) with step size η, not 1/β. The rate (35) therefore does not follow without a further condition linking η, γ, α, and the eigenvalues of L. Please state and prove the required step-size condition, or redo the convergence analysis for the actual update.","section":"§3.1, Proposition 3"},{"comment":"The displayed proof does not establish the claimed bound. Equation (14) rewrites the target quantity without producing an upper bound, and the final step asserts that E[g_u(w^*(f̂(x))) − g_u(ŵ(f^*(x)))] is non-positive from the optimality of f̂, whereas Eq. (9) only yields E[g_u(ŵ(f̂(x)))] ≤ E[g_u(ŵ(f^*(x)))]. The theorem statement is true by a standard add-and-subtract argument using ŵ(f̂) and ŵ(f*), but the proof as written must be corrected.","section":"§2, proof of Theorem 1 (Eqs. (12)–(17))"},{"comment":"Section 3.2.3 states that the architecture replaces the exact projection π_P with the finite-k approximate projection ~π_k, yet Propositions 2–3 and Theorem 2 are proven for the exact projection operator. The geometric convergence of ~π_k to π cited from Deutsch and Hundal is not incorporated into the regret, convergence, or generalization bounds. Since the implemented and evaluated ProjectNet uses ~π_k, the stated guarantees do not cover the actual model; the projection error should either be included in the bounds or the theory should be explicitly restricted to an exact-projection version.","section":"§3.2.3; §3.2.1, Eq. (62)"}],"minor_comments":[{"comment":"The theorem statement (42) gives λ_T ≤ √2 β γ D · (1 − (1 − γσ_min + α)^T)/(γσ_min − α), while the recurrence (57)–(58) in the proof yields λ_T ≤ γ D · (1 − (1 − γσ_min + α)^T)/(γσ_min − α) with no β or √2 factor; the statement and proof need to be reconciled.","section":"§3.1, Theorem 2"},{"comment":"Equation (51) repeats ŵ^{L1}_T on both sides of the norm; one occurrence should be ŵ^{L2}_T.","section":"§3.1, proof of Theorem 2, Eq. (51)"},{"comment":"Assumption 1 defines g_u as α-smooth and β-Lipschitz, but the proof of Theorem 2 and surrounding text refer to g_u as \"α-Lipschitz\"; the notation should be made consistent.","section":"§3.1, Assumption 1; §3.1, proof of Theorem 2"},{"comment":"These propositions are correct but follow almost immediately from the definition of a loss-type objective; presenting them as observations rather than as primary theoretical contributions would better match their depth.","section":"§2, Propositions 1 and 4"},{"comment":"The sentence \"We will see algorithmically how we might be able to ensure this [PSD]\" is not fulfilled by Eq. (63); either add the PSD-enforcing step or remove the forward promise.","section":"§3.2.2"},{"comment":"There are several typos and duplicated labels: \"ProjecetNet\" in the caption of Figure 7, \"sate of the art\" in the Introduction, \"K-nearnest neighbor\" in Appendix A.2, and \"Assumption 1\" is used twice (Section 3.1 and Section 4).","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The empirical contribution is credible and the speedups are worth publishing once the theory is brought into line with the implementation. The main risk is that the current manuscript overstates the guarantees: the non-symmetric L(u) construction and the approximate projection are not covered by the proofs. I would ask the authors to fix the PSD construction, correct the Proposition 3 step-size analysis, repair the Theorem 1 proof, and explicitly scope the theoretical claims to the architecture actually used, before any acceptance decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, ProjectNet—an unrolled projected-gradient network with a learned quadratic perturbation and Dykstra-style approximate projections—is a sensible, potentially useful contribution to decision-focused learning. Replacing expensive exact solves with a fast network forward pass is the right bottleneck to attack, and the 2–10x training speedups over OptNet and Vlastelica et al. across newsvendor, electricity scheduling, and shortest-path benchmarks are credible in shape, though no code or data are provided to confirm them. Second, the theory section does not actually cover the architecture that is implemented. The stress-test concern lands: in Eq. (63), L(u) = (MM^T) ρ(D)(MM^T)^{-1} is similar to a diagonal matrix with positive entries, but it is generally non-symmetric, so positive eigenvalues do not imply the quadratic form w^T L w ≥ 0. Propositions 2 and 3 and Theorem 2 each rely on L being PSD in this quadratic-form sense; as written, the guarantees apply to a symmetric PSD L, not to the constructed one. There is also a step-size gap: Proposition 3's proof invokes the standard projected-gradient rate for step size 1/β, while the implemented update uses η and γ separately. Both gaps are repairable—symmetrize L, constrain the symmetric part, and redo the step-size analysis—but they are real.\n\nWhat is genuinely new is the combination: a learned linear perturbation of the objective, unrolled projected gradient steps, and approximate differentiable projections for feasibility. Each ingredient has precedent (Wilder-style regularization, OptNet, DC3), but putting them together into a fast approximate solver that can be trained end-to-end is useful. The Dykstra projection trick is practical and well-motivated. The generalization theorem is a standard Rademacher composition argument, and the point-forecast propositions are elementary existence results that the paper does not overclaim.\n\nThe main practical weakness is lack of artifacts: no code, no data, no standard errors. For a paper whose headline claim is speedups, that is a serious omission. This is a paper for researchers who work on end-to-end decision-focused learning and differentiable optimization layers. It deserves a serious referee and a request for heavy revision: fix the PSD/step-size issues, release code and data, and report variance. That is a route to acceptance, not a desk reject.","headline":"A genuinely useful surrogate-optimizer architecture with a theory section that doesn't yet cover the implemented network; the empirical speedups are plausible but not yet reproducible.","tokens_in":27743,"tokens_out":4019,"would_cite":false,"duration_ms":39613,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","90C25","90C15"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a learned surrogate optimizer can replace exact solver calls inside end-to-end decision learning, with bounded regret and large speedups.","keywords":["end-to-end learning","decision-focused learning","meta-optimization","surrogate optimization","projected gradient descent","stochastic optimization","newsvendor problem","shortest path"],"falsifier":"Train a ProjectNet on any instance class and freeze it. For a sampled cost vector $u$, compute $m = \\min_{\\|w\\|_2=1} w^\\top L(u) w$ using the learned matrix. If $m<0$ for any $u$, the positive-semidefiniteness premise used in the convergence and generalization theorems is violated for the implemented network, so the bounds as stated do not apply. A weaker check: set $\\gamma=0$ and compare decision regret; if regret is unchanged, the learned $L(u)$ term is not doing the work claimed.","tokens_in":26533,"feed_emoji":"⚡","tokens_out":15643,"duration_ms":137272,"temperature":0.7,"pith_summary":"The paper is trying to establish that the expensive optimization step inside end-to-end decision learning can be replaced by a learned surrogate solver without sacrificing decision quality. Its ProjectNet architecture performs a small, fixed number of projected-gradient-like steps with a learned linear term $L(u)$, and enforces feasibility through differentiable alternating projections onto the individual constraints. If the claims hold, training an end-to-end model no longer requires solving an optimization problem at every gradient step; the paper reports 2–10x faster training on electricity scheduling, inventory, and shortest-path tasks, with decision costs within a few percent of exact end-to-end methods. The paper also proves that for loss-type stochastic objectives a point forecast suffices in place of a distributional forecast, keeping the downstream problem deterministic.","feed_headline":"A learned surrogate makes end-to-end decision training 2–10x faster","feed_subtitle":"ProjectNet swaps each expensive solver call for differentiable projection steps, holding decision cost near optimal.","key_machinery":"The mechanism is the learned linear term $L(u)$ inside the projected-gradient-like update $w_{t+1} = \\pi_P(w_t - \\eta \\nabla g_u(w_t) - \\gamma L(u) w_t)$, iterated $T$ times, with $\\pi_P$ implemented as a sequence of differentiable projections onto the individual constraints. Adding $L(u)$ makes the effective surrogate objective $g_u(w) + \\frac{\\gamma}{2\\eta} w^\\top L(u) w$, so a good $L(u)$ reshapes the optimization landscape: it accelerates convergence while the regret relative to the true solution is controlled by the largest eigenvalue of $L(u)$. The implementation parameterizes $L(u) = (M M^\\top)\\rho(D)(M M^\\top)^{-1}$ with $M$ upper triangular and $D$ diagonal, using a scaled sigmoid $\\rho$ to keep the eigenvalues in a target range; differentiability of this construction is what allows both the surrogate and the downstream forecast model to be trained by gradient descent. The convergence and generalization analyses both treat $L(u)$ as positive semidefinite with controlled eigenvalues.","core_discovery":"The paper's central claim is that one can learn a fast, differentiable approximation $\\hat{w}(u)$ to the solution of a convex optimization problem by training a network to run $T$ iterations of the update $w_{t+1} = \\pi_P(w_t - \\eta \\nabla g_u(w_t) - \\gamma L(u) w_t)$, and that this approximation can replace the exact solution map $w^*(u)$ inside end-to-end learning. The replacement is justified by a regret decomposition: the extra decision cost of training with the surrogate is bounded by how well the surrogate approximates the exact optimizer on the forecasts the learner actually produces, so the surrogate only needs to be accurate on the data distribution, not universally. The paper proves exponential convergence of the surrogate objective values, a regret bound proportional to $\\gamma \\sigma_{\\max}(L) D^2 / (2\\eta)$, and a Rademacher-complexity generalization bound for learning $L$ from data. It also proves that for loss-type stochastic objectives a point forecast is as good as a distributional forecast. The intended consequence is that end-to-end training can be run without a single call to an exact optimization solver, and the reported experiments support this with 2–10x training speedups across electricity scheduling, multi-warehouse inventory, and shortest-path problems.","pith_inferences":["Symmetrizing $L(u)$, for example replacing it with $(L(u)+L(u)^\\top)/2$, would make the implemented network satisfy the positive-semidefiniteness premise of the proofs without changing the architecture's differentiability; if this is done, the convergence and generalization bounds would apply to the exact network that runs in practice.","The learned update can be read as a learned preconditioner or curvature term; a natural extension is to let $L$ depend on the iteration count or on the current point $w_t$, which should express more powerful learned solvers at some cost in the generalization analysis.","Because the surrogate is differentiable and fast, it could be deployed at inference time as the decision rule itself, not only as a training substitute, whenever exact solver latency is prohibitive; this deployment use is left implicit in the paper.","The point-forecast sufficiency results suggest a direct comparative test for any loss-type stochastic problem: train the same end-to-end pipeline with point and distributional forecasts; the theory predicts equal optimal decisions, with the point-forecast version enjoying the complexity advantages."],"forward_implications":["End-to-end training on large decision problems no longer needs an optimization solver in the inner loop, because the ProjectNet forward pass replaces it and cuts per-epoch training time by roughly 2–10x in the studied problems.","Linear and combinatorial decision problems, whose true solution maps have zero or undefined gradients, become trainable end-to-end because the surrogate's gradient does not vanish.","The surrogate only needs to approximate the optimizer on the distribution of forecasts encountered during training, so ProjectNet can be trained on cost vectors without solving the nominal optimization problem.","For loss-type objectives, forecasts can be point-valued rather than distributional, keeping the downstream problem deterministic and potentially avoiding an exponential scenario space.","The generalization bound implies that the gap between empirical and expected decision cost shrinks at the usual $O(1/\\sqrt{N})$ rate for simple hypothesis classes of $L$."],"supporting_citations":[{"why":"It supplies the alternating projection algorithm that ProjectNet uses to enforce feasibility differentiably.","marker":"Dykstra (1983)"},{"why":"It proves the geometric convergence of these cyclic projections for polyhedral sets, which justifies using a finite number of projection steps.","marker":"Deutsch and Hundal (1994)"},{"why":"It provides the strong-convexity and smoothness convergence rate for gradient descent that Proposition 3 extends to the surrogate objective.","marker":"Boyd and Vandenberghe (2004)"},{"why":"It gives the Rademacher complexity risk bounds and vector contraction inequality behind Theorem 2's generalization guarantee.","marker":"Bartlett and Mendelson (2002)"},{"why":"It establishes the task-based end-to-end stochastic optimization setting and supplies the baseline and data generation used in the quadratic newsvendor experiments.","marker":"Donti et al. (2017a)"},{"why":"It defines the differentiable-optimization-layer baseline that ProjectNet is compared against on speed and decision cost.","marker":"Amos and Kolter (2017)"},{"why":"It supplies the blackbox-differentiation baseline and the shortest-path benchmark used in the computer-vision experiment.","marker":"Vlastelica et al. (2020)"}],"fun_headline_variants":["Fast neural solver accelerates end-to-end learning 2–10x","Meta-optimized networks replace solver calls in end-to-end training","Learned approximations speed up decision-focused training","Exponential convergence in learned optimization for end-to-end learning","Scalable end-to-end learning via differentiable surrogate optimizers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every learned matrix $L(u)$ is positive semidefinite, meaning $w^\\top L(u) w \\geq 0$ for every direction $w$; the network construction controls only eigenvalues and can return a non-symmetric matrix for which this property does not automatically hold.","fun_headline_variants_meta":{"raw":{"variants":["Fast neural solver accelerates end-to-end learning 2–10x","Meta-optimized networks replace solver calls in end-to-end training","Learned approximations speed up decision-focused training","Exponential convergence in learned optimization for end-to-end learning","Scalable end-to-end learning via differentiable surrogate optimizers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000887,"raw_usage":{"total_tokens":3905,"prompt_tokens":1100,"completion_tokens":2805,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":716,"completion_tokens_details":{"reasoning_tokens":2723}},"tokens_in":716,"tokens_out":2805,"duration_ms":21003,"temperature":1.0,"reasoning_tokens":2723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:55:07.773745+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a ProjectNet on any instance class and freeze it. For a sampled cost vector $u$, compute $m = \\min_{\\|w\\|_2=1} w^\\top L(u) w$ using the learned matrix. If $m<0$ for any $u$, the positive-semidefiniteness premise used in the convergence and generalization theorems is violated for the implemented network, so the bounds as stated do not apply. A weaker check: set $\\gamma=0$ and compare decision regret; if regret is unchanged, the learned $L(u)$ term is not doing the work claimed.","supporting_citations":[],"review_version":1}