{"id":"a12cfe92-c3c0-440e-8d27-fa48649a3b54","arxiv_id":"2504.14099","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CVXPYgen gains compiled C code generation for differentiating through LP/QP solution maps, with order-of-magnitude speedups over CVXPYlayers in the reported examples.","lead":"This paper adds a differentiation engine to CVXPYgen, turning parametrized convex optimization problems into compiled C solvers that also compute gradients of the solution with respect to the parameters. The three examples show 5x to 40x faster gradient evaluation than the general-purpose CVXPYlayers tool.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim requires correct derivatives, but the paper reports no validation of the computed gradients; timing speedups are irrelevant if the derivatives are wrong.","rationale":"The reader's weakest_assumption concerned differentiability and active-set stability, which is a theoretical limitation the authors explicitly acknowledge. My concern is complementary but more immediately load-bearing: even in the smooth, stable regime the paper provides no empirical evidence that the computed gradients are correct. Without a gradient-correctness check, the central claim 'supports evaluating the derivative' is unverified. The math in the paper seems plausible, and the open-source code could be tested, so this concern does not warrant rejection; it does warrant keeping the verdict at CONDITIONAL. I agree with the reader's overall assessment, but I would elevate the missing gradient validation to the primary reason for conditionality, alongside the acknowledged differentiability caveats.","tokens_in":16257,"tokens_out":4612,"duration_ms":43667,"concrete_test":"Reproduce the elastic net example at the initial parameter ω0; compute the gradient of the cross-validated RMSE with respect to ω using the generated backward pass, and compare against the gradient from CVXPYlayers and central finite differences on the same problem instance. Report the maximum relative error for each component. Then repeat for 100 random parameter vectors in a small neighborhood of ω0 where the active set is stable, and verify all relative errors are below 1e-5. If any component mismatches, the implementation has a correctness bug.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's contribution is differentiating through convex optimization, yet the numerical experiments only report timing and tuning convergence curves. There is no check of the computed gradients against finite differences, analytical derivatives, or an independent implementation. The mathematical derivation in §2.3 appears standard, but several implementation details—active set determination from a numerically approximate primal-dual solution, the regularization ϵ=1e-6, iterative refinement, and the low-rank LDL update algorithms 2 and 3—could all introduce errors. Without a gradient-correctness test, the reader cannot verify the central claim that the generated code actually differentiates correctly. Tuning convergence could occur even with biased or approximate gradients, so the reported speedups do not by themselves validate the derivative output. This is a load-bearing concern because the entire contribution rests on the correctness of the returned Jacobian.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the open-source code generator CVXPYgen so that, for parameterized convex optimization problems reducible to quadratic programs, it emits custom C code for both solving the problem and differentiating the parameter-to-solution map. The differentiation method follows the standard KKT/argmin-differentiation approach of OptNet, with a regularized quasidefinite linear system solve, iterative refinement, and low-rank LDL updates for active-set changes. The authors provide three application studies: tuning elastic-net hyperparameters and winsorization levels, tuning weights of an approximate dynamic programming controller, and tuning a portfolio optimization trading engine. In each case they report convergence of the tuning procedure and compare computation times against CVXPYlayers.","tokens_in":16404,"tokens_out":4367,"duration_ms":46378,"significance":"If the generated derivative code is correct, the paper describes a practically useful software contribution: it is, to my knowledge, the first code generator for convex optimization that supports differentiation, and the reported gradient-only speedups of roughly one order of magnitude are meaningful for embedded and desktop tuning applications. The paper is also commendable for making the code and experiments publicly available, for clearly presenting the DPP-to-QP reduction, and for grounding the low-rank factorization updates in established references. However, the central claim that the generated code differentiates 'correctly' is not directly verified anywhere in the manuscript, and this is the main obstacle to accepting the paper in its current form.","major_comments":[{"comment":"The numerical experiments report tuning convergence curves and timings, but they never validate the computed gradients. Since the implementation depends on approximate active-set identification from an OSQP solution, regularization with epsilon = 1e-6, iterative refinement, and low-rank LDL updates, any one of these steps could introduce a bias or an outright error in the returned Jacobian. Tuning curves can decrease even with biased gradients, so the reported speedups do not by themselves establish that the differentiation is correct. I ask the authors to add a direct validation section: compare the generated backward pass against finite differences of the forward solution on small random LPs and QPs, including cases with active-set changes and equality constraints; compare the CVXPYgen Jacobian with the CVXPYlayers/diffcp Jacobian on the same problem instances; and report the sensitivity of the derivative to epsilon and N_refine. This is load-bearing because the paper's central contribution is correct differentiation through optimization, not merely fast solution.","section":"§4 and §2.3"},{"comment":"There is a mismatch between the stated design variable and the variable actually being optimized in the ADP example. The text says the controller is designed by omega = Z with Omega = S^n_+, but the algorithm modifies the Cholesky factor L and later recovers Z = L L^T - P_lqr. No projection onto the set of L whose recovered Z is positive semidefinite is described, and the line-search/projection step of Algorithm 4 is stated in terms of omega. If the implementation tunes L instead of Z, then the experiment solves a different optimization problem from the one stated, and the reported result does not demonstrate tuning of Z in the stated design space. Please clarify the parameterization, state the actual design space used in the code, and explain how the projection in Algorithm 4 is applied to L (or adjust the text to make L the design variable).","section":"§4.2"}],"minor_comments":[{"comment":"The abstract states that the approach 'increases the computation speed by about an order of magnitude in most applications,' but the full-tuning speedups in Tables 1, 2, and 4 are approximately 2.4x, 5.9x, and 2.9x. Only the gradient-only timings are near an order of magnitude. Please qualify the speedup claim as applying to the gradient computation, with full-loop speedups being smaller due to Python overhead.","section":"Abstract and §4"},{"comment":"In the elastic net example, the text reports 'lambda approx 0.68 and gamma approx 0.80' after stating lambda = 10^mu and gamma = 10^nu. It would be clearer to report mu and nu, or to state explicitly that these are the recovered values of lambda and gamma.","section":"§4.1"},{"comment":"The phrase 'survivorship bias' should be 'survivorship bias' (or 'survivorship bias' is already the standard term; the manuscript reads 'survivership bias'). This is a typographical issue only.","section":"§4.3"},{"comment":"The timing comparisons do not state the hardware, compiler, or software versions used. Since the main empirical claim is a speed comparison, please report these details so the numbers are reproducible and interpretable.","section":"§4"},{"comment":"The sentence 'Note that we use the sign of y to determine constraint activity' is terse. The active set should be defined by primal equality A_i x = l_i or u_i; the dual sign is relevant for whether a constraint is correctly classified as active with a nonzero multiplier. Please expand this sentence so the reader knows precisely which OSQP outputs are thresholded and how ties are broken.","section":"§2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution with a standard mathematical core, but the missing gradient-correctness validation is a real risk for the central claim. The requested validation experiments are straightforward to add and should be within the scope of a revision. I would not recommend rejection, because the derivation is standard and the software is open source, so the correctness question can be settled with additional experiments. The ADP parameterization issue in Section 4.2 should also be resolved, as it currently makes one of the three headline experiments ill-posed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a solid tools paper with a real deliverable, but it skips a basic numerical check that should have been there—nowhere does it validate the computed gradients against finite differences, analytical derivatives, or an independent implementation. The math is standard, the code is open, and the speedups are plausible, but the central claim \"we differentiate correctly\" rests only on tuning convergence curves, which can look fine with biased gradients.\n\nWhat's new: Schaller and Boyd extend CVXPYgen so that, from a CVXPY/DPP problem description, you get custom C code that both solves the parametrized problem (LP/QP after canonicalization) and computes the derivative of the solution with respect to parameters. That specific combination—code generation plus differentiation—is new as far as I know. The differentiation itself is OptNet's KKT reasoning, and the low-rank LDL updates are Davis and Hager's, but the integration into a DPP pipeline with Python/CVXPYlayers wrappers is a useful engineering contribution. The three examples (elastic net tuning, ADP controller tuning, portfolio backtest) are realistic and show large speedups over CVXPYlayers, with code and data available.\n\nSoft spots, in proportion: the missing gradient validation is the most important. A few lines comparing the generated Jacobian against CVXPYlayers or finite differences on random parameter instances would close the loop. Without that, the speedups are impressive but the product is unverified. Second, the timing comparison mixes canonical solvers (OSQP for CVXPYgen, Clarabel for CVXPYlayers) and gives no hardware details or error bars, so the \"order of magnitude\" claim is not as clean as it looks. Third, in the ADP example the design variable is L (a Cholesky factor) while the stated design space is the PSD cone for Z; the projection step in their generic algorithm isn't spelled out for L, and the recovered Z = LLᵀ - P_lqr might leave the cone. That's a minor gap, easily patched. The paper is honest about the non-differentiability caveat, which I appreciate—no overclaiming there.\n\nWho it's for: anyone building differentiable optimization pipelines that need speed or embedded deployment. It's a tools paper, not a theory paper. The citation pattern is fine; the KKT and LDL sources are properly attributed.\n\nRecommendation: yes, send it to peer review. The missing gradient test is fixable and should be requested, not fatal. With that added and the timing made apples-to-apples, I'd accept it.","headline":"A genuinely useful software extension that combines codegen with argmin differentiation, but the missing gradient-correctness check keeps the central claim from being fully demonstrated.","tokens_in":16919,"tokens_out":2871,"would_cite":true,"duration_ms":29723,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C25","90C20","68N30"],"pacs":[],"model":"deepseek-v4-flash","headline":"C code now differentiates through convex optimization problems","keywords":["convex optimization","code generation","differentiation through optimization","quadratic programs","disciplined parametrized programming","low-rank factorization updates","gradient-based tuning","CVXPYgen"],"falsifier":"Solve a quadratic program family at a parameter value where a constraint just becomes active, compute the generated gradient of the solution, and compare it with a central finite-difference Jacobian of the solution map; a mismatch of order one as the parameter crosses the kink would show that the generated derivative is not a true derivative, and a tuning loop built on it can be expected to fail.","tokens_in":16057,"feed_emoji":"⚙️","tokens_out":4266,"duration_ms":35607,"temperature":0.7,"pith_summary":"This paper claims that a code generator can produce custom C source that not only solves but also differentiates through a parametrized convex optimization problem, and that this is the first such code generator for convex optimization. If the claim holds, users of CVXPY can compile a problem family once and then get both the optimal solution and its gradient with respect to parameters, with no interpreter overhead, making gradient-based tuning practical in embedded or repeated-solve settings. The paper demonstrates the claim on three tuning applications, reporting speedups of roughly an order of magnitude over CVXPYlayers. The central mechanism is differentiating the KKT optimality conditions of the underlying quadratic program and reusing low-rank factorization updates across repeated solves.","feed_headline":"C code now differentiates through convex optimization problems","feed_subtitle":"Generated solver-and-differentiator pairs cut gradient computation about tenfold in tuning tasks.","key_machinery":"The load-bearing object is the KKT system formed from the active constraint set at optimality. Differentiating it yields a linear system solved through a regularized quasidefinite matrix with iterative refinement, and low-rank row and column addition and deletion algorithms update the sparse LDL factorization when the active set changes between solves. This combination turns repeated differentiation into a sequence of cheap rank-one updates rather than full refactorizations, which is what makes the compiled differentiator fast.","core_discovery":"The paper's central discovery is that differentiating through a convex optimization problem can be compiled: for any disciplined parametrized programming problem that reduces to a quadratic program, the parameter-to-solution mapping can be differentiated by forming the KKT system at the active constraints, differentiating it, and propagating a gradient through the affine canonicalization and retrieval steps—all in generated C code. The authors extend CVXPYgen with a custom gradient computation, add CVXPY and CVXPYlayers interfaces, and report that the generated differentiator is about an order of magnitude faster than general-purpose CVXPYlayers in three applications: elastic net hyper-parameter tuning, approximate dynamic programming controller tuning, and financial portfolio optimization. They state that, to their knowledge, CVXPYgen is the first code generator for convex optimization that supports differentiation.","pith_inferences":["The low-rank update strategy should extend to second-order information: repeated Hessian-vector products through the solution map could reuse the same factorization machinery.","A testable extension is to relax the disciplined parametrized programming requirement to non-affine parameterizations by differentiating through the canonicalization itself, allowing problems like the original approximate dynamic programming formulation to be differentiated directly.","The speed advantage is likely to grow when the problem family is solved many times with few active-set changes, suggesting the method is especially suited to model-predictive control and online portfolio rebalancing, where consecutive instances differ by small parameter changes."],"forward_implications":["Users of CVXPY can generate, compile, and deploy a solver-and-differentiator pair for a problem family, enabling gradient-based parameter tuning without a Python interpreter in the loop.","The differentiator can be wired into autodifferentiation frameworks through the provided Python interface, so gradients of arbitrary scalar functions that contain optimization steps become available.","Because the generated code is C, it can run in embedded and hard real-time settings, where parser-solver approaches are infeasible.","The reported speedups imply that repeated gradient evaluations, which previously dominated tuning loops, stop being the bottleneck."],"supporting_citations":[{"why":"Supplies the QP argmin differentiation method, differentiating the KKT conditions, that the paper adapts.","marker":"[AK17]"},{"why":"The base CVXPYgen code generator that this work extends with gradient support.","marker":"[SBD+22]"},{"why":"Defines disciplined parametrized programming and the reverse-mode differentiation mapping convention used to propagate gradients through canonicalization, and provides the CVXPYlayers baseline.","marker":"[AAB+19]"},{"why":"Provides the canonical quadratic program form and the solver the generated code targets.","marker":"[SBG+20]"},{"why":"Supplies the sparse Cholesky modification algorithm used for rank-one downdates and updates.","marker":"[DH99]"},{"why":"Supplies the row and column addition and deletion variants used when the active set changes.","marker":"[DH05]"},{"why":"Establishes quasidefiniteness of the regularized KKT matrix and the existence of its LDL factorization.","marker":"[Van95]"},{"why":"Supports the iterative refinement steps that correct the regularization error in the system solve.","marker":"[CH18]"}],"fun_headline_variants":["Compile differentiators for convex optimization, 10x speedup","First C code generator that differentiates convex programs","Differentiate convex optimization in generated C, 10x faster","CVXPYgen now emits differentiators: order-of-magnitude speedup","Custom C differentiators for convex optimization problems"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The gradient code assumes that the parameter-to-solution map is locally unique and differentiable, and that the active constraint set used to build the KKT system is correctly identified and stable; when the solution is nonunique or at a kink, the returned gradient is not a true derivative.","fun_headline_variants_meta":{"raw":{"variants":["Compile differentiators for convex optimization, 10x speedup","First C code generator that differentiates convex programs","Differentiate convex optimization in generated C, 10x faster","CVXPYgen now emits differentiators: order-of-magnitude speedup","Custom C differentiators for convex optimization problems"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000425,"raw_usage":{"total_tokens":2157,"prompt_tokens":901,"completion_tokens":1256,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":1172}},"tokens_in":517,"tokens_out":1256,"duration_ms":8804,"temperature":1.0,"reasoning_tokens":1172,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:56:07.793000+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Solve a quadratic program family at a parameter value where a constraint just becomes active, compute the generated gradient of the solution, and compare it with a central finite-difference Jacobian of the solution map; a mismatch of order one as the parameter crosses the kink would show that the generated derivative is not a true derivative, and a tuning loop built on it can be expected to fail.","supporting_citations":[],"review_version":1}