REVIEW 4 major objections 6 minor 15 references
Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that a learned surrogate optimizer can replace exact solver calls inside end-to-end decision learning, with bounded regret and large speedups.
desk verdict A genuinely useful surrogate-optimizer architecture with a theory section that doesn't yet cover the implemented network; the empirical speedups are plausible but not yet reproducible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is the learned linear term $L(u)$ inside the projected-gradient-like update $w_{t+1} = \pi_P(w_t - \eta \nabla g_u(w_t) - \gamma L(u) w_t)$, iterated $T$ times, with $\pi_P$ implemented as a sequence of differentiable projections onto the individual constraints. Adding $L(u)$ makes the effective surrogate objective $g_u(w) + \frac{\gamma}{2\eta} w^\top L(u) w$, so a good $L(u)$ reshapes the optimization landscape: it accelerates convergence while the regret relative to the true solution is controlled by the largest eigenvalue of $L(u)$. The implementation parameterizes $L(u) = (M M^\top)\rho(D)(M M^\top)^{-1}$ with $M$ upper triangular and $D$ diagonal, using a scaled sigmoid $\rho$ to keep the eigenvalues in a target range; differentiability of this construction is what allows both the surrogate and the downstream forecast model to be trained by gradient descent. The convergence and generalization analyses both treat $L(u)$ as positive semidefinite with controlled eigenvalues.
What would settle it
Train a ProjectNet on any instance class and freeze it. For a sampled cost vector $u$, compute $m = \min_{\|w\|_2=1} w^\top L(u) w$ using the learned matrix. If $m<0$ for any $u$, the positive-semidefiniteness premise used in the convergence and generalization theorems is violated for the implemented network, so the bounds as stated do not apply. A weaker check: set $\gamma=0$ and compare decision regret; if regret is unchanged, the learned $L(u)$ term is not doing the work claimed.
Extended reading notes
Core claim
The paper's central claim is that one can learn a fast, differentiable approximation $\hat{w}(u)$ to the solution of a convex optimization problem by training a network to run $T$ iterations of the update $w_{t+1} = \pi_P(w_t - \eta \nabla g_u(w_t) - \gamma L(u) w_t)$, and that this approximation can replace the exact solution map $w^*(u)$ inside end-to-end learning. The replacement is justified by a regret decomposition: the extra decision cost of training with the surrogate is bounded by how well the surrogate approximates the exact optimizer on the forecasts the learner actually produces, so the surrogate only needs to be accurate on the data distribution, not universally. The paper proves exponential convergence of the surrogate objective values, a regret bound proportional to $\gamma \sigma_{\max}(L) D^2 / (2\eta)$, and a Rademacher-complexity generalization bound for learning $L$ from data. It also proves that for loss-type stochastic objectives a point forecast is as good as a distributional forecast. The intended consequence is that end-to-end training can be run without a single call to an exact optimization solver, and the reported experiments support this with 2–10x training speedups across electricity scheduling, multi-warehouse inventory, and shortest-path problems.
Load-bearing premise
The load-bearing premise is that every learned matrix $L(u)$ is positive semidefinite, meaning $w^\top L(u) w \geq 0$ for every direction $w$; the network construction controls only eigenvalues and can return a non-symmetric matrix for which this property does not automatically hold.
Editorial extensions
If this is right
- End-to-end training on large decision problems no longer needs an optimization solver in the inner loop, because the ProjectNet forward pass replaces it and cuts per-epoch training time by roughly 2–10x in the studied problems.
- Linear and combinatorial decision problems, whose true solution maps have zero or undefined gradients, become trainable end-to-end because the surrogate's gradient does not vanish.
- The surrogate only needs to approximate the optimizer on the distribution of forecasts encountered during training, so ProjectNet can be trained on cost vectors without solving the nominal optimization problem.
- For loss-type objectives, forecasts can be point-valued rather than distributional, keeping the downstream problem deterministic and potentially avoiding an exponential scenario space.
- The generalization bound implies that the gap between empirical and expected decision cost shrinks at the usual $O(1/\sqrt{N})$ rate for simple hypothesis classes of $L$.
Reading between the lines
- Symmetrizing $L(u)$, for example replacing it with $(L(u)+L(u)^\top)/2$, would make the implemented network satisfy the positive-semidefiniteness premise of the proofs without changing the architecture's differentiability; if this is done, the convergence and generalization bounds would apply to the exact network that runs in practice.
- The learned update can be read as a learned preconditioner or curvature term; a natural extension is to let $L$ depend on the iteration count or on the current point $w_t$, which should express more powerful learned solvers at some cost in the generalization analysis.
- Because the surrogate is differentiable and fast, it could be deployed at inference time as the decision rule itself, not only as a training substitute, whenever exact solver latency is prohibitive; this deployment use is left implicit in the paper.
- The point-forecast sufficiency results suggest a direct comparative test for any loss-type stochastic problem: train the same end-to-end pipeline with point and distributional forecasts; the theory predicts equal optimal decisions, with the point-forecast version enjoying the complexity advantages.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ProjectNet, a meta-optimization method for end-to-end learning in which the exact optimization oracle is replaced by a learned, differentiable approximation. The forecast u is fed into an unrolled update w_{t+1} = π_P(w_t − η∇g_u(w_t) − γ L(u)w_t), run for T iterations, with L(u) learned to minimize the empirical decision cost. The authors prove a regret bound (Proposition 2), exponential convergence for the surrogate (Proposition 3), a generalization bound (Theorem 2), and an approximation-impact decomposition (Theorem 1). They also show that point forecasts suffice for loss-type objectives (Propositions 1 and 4). Experiments on a two-stage cross-fulfillment newsvendor problem, a PJM electricity planning problem, the Warcraft shortest-path benchmark, and synthetic matching/newsvendor problems report 2–10x training-speed improvements over OptNet, CVXPY, and Vlastelica et al. with competitive task costs.
Significance. The central idea—eliminating exact solver calls from end-to-end training by learning a feasible, differentiable surrogate—is practically attractive, and the empirical work is a genuine strength: it uses real PJM data, established benchmarks, and careful runtime/accuracy comparisons. If the theoretical gaps are repaired, the paper would make a useful contribution to decision-focused learning. However, the current guarantees do not cover the architecture actually implemented and evaluated: the learned L(u) in Eq. (63) is not positive semidefinite in the quadratic-form sense required by Propositions 2–3 and Theorem 2, and the proofs use an exact projection while the implementation uses a finite-k approximate projection. These issues are repairable, but they are load-bearing for the paper's theoretical claims.
major comments (4)
- [§3.2.2, Eq. (63); §3.1] The construction L(u) = (M(u)M(u)^T) ρ(D(u)) (M(u)M(u)^T)^{-1} has positive eigenvalues because it is similar to the diagonal matrix ρ(D(u)), but it is not symmetric in general, and positive eigenvalues do not imply w^T L w ≥ 0. The analysis in §3.1 explicitly assumes L(u) is positive semidefinite, and Propositions 2–3 and Theorem 2 rely on w^T L w ≥ 0, on σ_min(L) as a strong-convexity parameter, and on the inequality ||I − γL|| ≤ 1 − γσ_min. For a concrete counterexample, M = [[1,1],[0,1]] and D = diag(1,10) give L = [[−8,18],[−9,19]], whose symmetric part has a negative eigenvalue and hence an indefinite quadratic form. Please modify the construction—for example, use L(u) = S ρ(D) S with S symmetric positive definite, or explicitly enforce PSD of (L+L^T)/2—and re-derive the affected bounds for that construction.
- [§3.1, Proposition 3] The proof invokes the standard projected gradient descent rate for step size 1/β, but the implemented update in Eq. (44) is w_{t+1} = π_P(w_t − η∇g_u(w_t) − γL(u)w_t), which equals a projected gradient step on r_u(w) with step size η, not 1/β. The rate (35) therefore does not follow without a further condition linking η, γ, α, and the eigenvalues of L. Please state and prove the required step-size condition, or redo the convergence analysis for the actual update.
- [§2, proof of Theorem 1 (Eqs. (12)–(17))] The displayed proof does not establish the claimed bound. Equation (14) rewrites the target quantity without producing an upper bound, and the final step asserts that E[g_u(w^*(f̂(x))) − g_u(ŵ(f^*(x)))] is non-positive from the optimality of f̂, whereas Eq. (9) only yields E[g_u(ŵ(f̂(x)))] ≤ E[g_u(ŵ(f^*(x)))]. The theorem statement is true by a standard add-and-subtract argument using ŵ(f̂) and ŵ(f*), but the proof as written must be corrected.
- [§3.2.3; §3.2.1, Eq. (62)] Section 3.2.3 states that the architecture replaces the exact projection π_P with the finite-k approximate projection ~π_k, yet Propositions 2–3 and Theorem 2 are proven for the exact projection operator. The geometric convergence of ~π_k to π cited from Deutsch and Hundal is not incorporated into the regret, convergence, or generalization bounds. Since the implemented and evaluated ProjectNet uses ~π_k, the stated guarantees do not cover the actual model; the projection error should either be included in the bounds or the theory should be explicitly restricted to an exact-projection version.
minor comments (6)
- [§3.1, Theorem 2] The theorem statement (42) gives λ_T ≤ √2 β γ D · (1 − (1 − γσ_min + α)^T)/(γσ_min − α), while the recurrence (57)–(58) in the proof yields λ_T ≤ γ D · (1 − (1 − γσ_min + α)^T)/(γσ_min − α) with no β or √2 factor; the statement and proof need to be reconciled.
- [§3.1, proof of Theorem 2, Eq. (51)] Equation (51) repeats ŵ^{L1}_T on both sides of the norm; one occurrence should be ŵ^{L2}_T.
- [§3.1, Assumption 1; §3.1, proof of Theorem 2] Assumption 1 defines g_u as α-smooth and β-Lipschitz, but the proof of Theorem 2 and surrounding text refer to g_u as "α-Lipschitz"; the notation should be made consistent.
- [§2, Propositions 1 and 4] These propositions are correct but follow almost immediately from the definition of a loss-type objective; presenting them as observations rather than as primary theoretical contributions would better match their depth.
- [§3.2.2] The sentence "We will see algorithmically how we might be able to ensure this [PSD]" is not fulfilled by Eq. (63); either add the PSD-enforcing step or remove the forward promise.
- [Throughout] There are several typos and duplicated labels: "ProjecetNet" in the caption of Figure 7, "sate of the art" in the Introduction, "K-nearnest neighbor" in Appendix A.2, and "Assumption 1" is used twice (Section 3.1 and Section 4).
Circularity Check
The central ProjectNet meta-optimization derivation is self-contained; only the point-forecast sufficiency propositions reduce to definitional existence arguments.
-
self definitional
[Section 2, Proposition 1 and its proof]
"Let d be the solution to the problem using distributional forecast D: d = arg min_w E_{u∼D}[g_u(w)]. Now consider making a forecast of exactly d. Then, w* = arg min_w g_d(w) = d, since g_d is a loss function."
The point forecast d is defined as the minimizer of the very distributional objective that the proposition claims a point forecast can reproduce. The conclusion w*(d)=d then follows immediately from the loss-function property u=argmin_w g_u(w). The proof is an existence argument by construction: it provides no way to obtain d without already solving the stochastic problem. The claim is true but definitional rather than derived from independent inputs.
-
self definitional
[Section 4, Proposition 4 and its proof]
"Let w* be the solution to the problem using distributional forecast D: w* = arg min_w E_{u∼D}[Z(w,u)]. Now consider making a forecast of exactly d = w*. Then, w* = arg min_w Z(w,d), since Z(·,·) is a loss function (i.e., the minimum is achieved at Z(w,w))."
As in Proposition 1, the candidate point forecast d is chosen to be the distributional solution itself. The equality arg min_w E_{u∼D}[Z(w,u)] = arg min_w Z(w,d) is then immediate from the loss property Z(w,w)=0. This is a definitional existence argument rather than a constructive derivation of a point forecast from features or data.
full rationale
The main ProjectNet derivation is not circular: the learned matrix L(u) is optimized against the empirical decision cost (Eq. 45) and its quality is measured on held-out data against independent baselines (OptNet, Vlastelica et al., CVXPY, predict-then-optimize), with no fitted parameter being renamed as a prediction. The paper contains no author self-citations, and the convergence, regret, and generalization arguments rely on standard external results (Boyd and Vandenberghe, Bartlett and Mendelson, Deutsch and Hundal). The only reduction-by-construction passages are the point-forecast sufficiency propositions for single- and two-stage loss-type problems, whose proofs select d to be the distributional solution and therefore hold by definition; these side results are not used to support the empirical speed or accuracy claims. The non-symmetry of L(u)=M M^T ρ(D)(M M^T)^{-1}, the step-size mismatch between eta and 1/beta in Proposition 3, and the use of ||I-gamma L||<=1-gamma sigma_min in Theorem 2 are correctness gaps between the stated assumptions and the implemented architecture, but they are proof gaps rather than circular reductions: the analysis explicitly assumes L is PSD, while the construction fails to enforce it. Overall score 1 reflects one minor definitional self-reduction in non load-bearing theoretical side claims.
Assumptions & free parameters
free parameters (5)
- gamma (perturbation coefficient) =
not reported in the paper.
- eta (projected gradient step size) =
not reported in the paper.
- T0 and T1 (unrolled iterations) =
T0 = 5 reported for the matching experiment; other values not fully reported.
- k (Dykstra projection iterations) =
not reported in the paper.
- Eigenvalue bounds lambda and lambda_bar =
not reported in the paper.
assumptions (7)
- domain assumption The objective function gu is alpha-smooth and beta-Lipschitz over the feasible region P.
- domain assumption The feasible region P is convex and the objective gu is convex.
- ad hoc to paper The learned matrix L(u) is positive semidefinite.
- domain assumption The two-stage model has relatively complete recourse.
- standard math Dykstra's cyclic projections converge geometrically on polyhedral sets.
- standard math The standard projected gradient descent rate for strongly convex and smooth functions applies.
- standard math Rademacher complexity bounds and the vector contraction inequality hold for the function class.
Cite this review
Pith. "Pith review of Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach." pith.science (2026). https://pith.science/paper/A2UTTZ34
@misc{pith2026250511360,
author = {Pith},
title = {Pith review of: Efficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach},
year = {2026},
howpublished = {\url{https://pith.science/paper/A2UTTZ34}},
note = {Machine review of arXiv:2505.11360}
}
read the original abstract
End-to-end learning has become a widely applicable and studied problem in training predictive ML models to be aware of their impact on downstream decision-making tasks. These end-to-end models often outperform traditional methods that separate training from the optimization and only myopically focus on prediction error. However, the computational complexity of end-to-end frameworks poses a significant challenge, particularly for large-scale problems. While training an ML model using gradient descent, each time we need to compute a gradient we must solve an expensive optimization problem. We present a meta-optimization method that learns efficient algorithms to approximate optimization problems, dramatically reducing computational overhead of solving the decision problem in general, an aspect we leverage in the training within the end-to-end framework. Our approach introduces a neural network architecture that near-optimally solves optimization problems while ensuring feasibility constraints through alternate projections. We prove exponential convergence, approximation guarantees, and generalization bounds for our learning method. This method offers superior computational efficiency, producing high-quality approximations faster and scaling better with problem size compared to existing techniques. Our approach applies to a wide range of optimization problems including deterministic, single-stage as well as two-stage stochastic optimization problems. We illustrate how our proposed method applies to (1) an electricity generation problem using real data from an electricity routing company coordinating the movement of electricity throughout 13 states, (2) a shortest path problem with a computer vision task of predicting edge costs from terrain maps, (3) a two-stage multi-warehouse cross-fulfillment newsvendor problem, as well as a variety of other newsvendor-like problems.
Reference graph
Works this paper leans on
-
[3]
We observe that the end-to-end methods based on ProjectNet and OptNet takes better advantage of the problem structure to provide lower-cost decisions. Crucially, the end-to-end method based on ProjectNet is computationally more efficient, 10 times faster to train than the OptNet framework which needs to solve the original optimization problem at each iter...
work page 2017
-
[11]
Interior Point Solving for LP-based prediction+optimisation
Jayanta Mandi and Tias Guns. Interior point solving for lp-based prediction+optimisation. ArXiv, abs/2010.13943,
work page Pith review arXiv 2010
-
[12]
Pointer Networks.arXiv e-prints, art
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer Networks.arXiv e-prints, art. arXiv:1506.03134, June
-
[15]
We define the projected gradient descent sequence of points wt+1 =π(wt +η·u), for edge weights u. We also train a ProjectNet model with T0 = 5 iterations, and compare the objective value of its solution for iterations up to T1 = 35 on testing data. See figure 11a. In particular, we measure the average relative regret of decisions. That is, given realized ...
work page 2017
-
[1979]
doi: https://doi.org/10.1016/0020-0190(79) 90152-2
ISSN 0020-0190. doi: https://doi.org/10.1016/0020-0190(79) 90152-2. URL https://www.sciencedirect.com/science/article/pii/0020019079901522. Priya Donti, Brandon Amos, and J Zico Kolter. Task-based end-to-end model learning in stochastic opti- mization. Advances in neural information processing systems , 30, 2017a. Priya Donti, Zico Kolter, and Brandon Amo...
-
[1985]
Yi-hao Kao, Benjamin Roy, and Xiang Yan
doi: 10.1007/BF00339943. Yi-hao Kao, Benjamin Roy, and Xiang Yan. Directed regression. In Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems , volume
-
[1988]
ISSN 0340-1200. doi: 10.1007/BF00363956. URL https://doi.org/10.1007/BF00363956. Cristian et al.: Meta-optimization for efficient end-to-end learning 36 Appendix Appendix A: Synthetic experiments A.1. Maximum Matching In what follows, we aim to show the improvement of this approach over using a traditional projected gradient method. We present computation...
-
[2002]
Neural combinatorial opti- mization with reinforcement learning
Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial opti- mization with reinforcement learning. arXiv preprint arXiv:1611.09940 ,
Show all 15 references
-
[2004]
The perils of learning before optimizing
Chris Cameron, Jason Hartford, Taylor Lundy, and Kevin Leyton-Brown. The perils of learning before optimizing. arXiv preprint arXiv:2106.10349 ,
-
[2009]
Learning to optimize
Ke Li and Jitendra Malik. Learning to optimize. arXiv preprint arXiv:1606.01885 ,
-
[2015]
Vlastelica, Anselm Paulus, V´ ıt Musil, G
Marin P. Vlastelica, Anselm Paulus, V´ ıt Musil, G. Martius, and Michal Rolinek. Differentiation of blackbox combinatorial solvers. ArXiv, abs/1912.02175,
1912 arXiv
-
[2016]
Online contextual decision-making with a smart predict-then-optimize method
Heyuan Liu and Paul Grigas. Online contextual decision-making with a smart predict-then-optimize method. arXiv preprint arXiv:2206.07316 ,
-
[2017]
Learning with differentiable perturbed optimizers
Quentin Berthet, Mathieu Blondel, Olivier Teboul, Marco Cuturi, Jean-Philippe Vert, and Francis Bach. Learning with differentiable perturbed optimizers. ArXiv, abs/2002.08676,
2002 arXiv
-
[2019]
Ali Ugur Guler, Emir Demirovic, Jeffrey Chan, James Bailey, Christopher Leckie, and Peter J Stuckey
URL http://arxiv.org/abs/1902.01785. Ali Ugur Guler, Emir Demirovic, Jeffrey Chan, James Bailey, Christopher Leckie, and Peter J Stuckey. Divide and learn: A divide and conquer approach for predict+ optimize. arXiv preprint arXiv:2012.02342 ,
1902 arXiv
-
[2021]
Richard L
URL https://arxiv.org/abs/2104.12225. Richard L. Dykstra. An algorithm for restricted least squares regression. Journal of the American Statistical Association, 78(384):837–842,
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.