REVIEW 15 cited by
Efficient and Modular Implicit Differentiation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Automatic differentiation (autodiff) has revolutionized machine learning. It allows to express complex computations by composing elementary ones in creative ways and removes the burden of computing their derivatives by hand. More recently, differentiation of optimization problem solutions has attracted widespread attention with applications such as optimization layers, and in bi-level problems such as hyper-parameter optimization and meta-learning. However, so far, implicit differentiation remained difficult to use for practitioners, as it often required case-by-case tedious mathematical derivations and implementations. In this paper, we propose automatic implicit differentiation, an efficient and modular approach for implicit differentiation of optimization problems. In our approach, the user defines directly in Python a function $F$ capturing the optimality conditions of the problem to be differentiated. Once this is done, we leverage autodiff of $F$ and the implicit function theorem to automatically differentiate the optimization problem. Our approach thus combines the benefits of implicit differentiation and autodiff. It is efficient as it can be added on top of any state-of-the-art solver and modular as the optimality condition specification is decoupled from the implicit differentiation mechanism. We show that seemingly simple principles allow to recover many existing implicit differentiation methods and create new ones easily. We demonstrate the ease of formulating and solving bi-level optimization problems using our framework. We also showcase an application to the sensitivity analysis of molecular dynamics.
Forward citations
Cited by 15 Pith papers
-
Delocalized Coupled-Cluster Theory for Polaron Structure and Dynamics
Delocalized coupled-cluster theory gives a scalable, systematically improvable variational description of polaron ground states, spectral functions, and optical conductivities from model Hamiltonians to ab initio materials.
-
AGN Reverberation Mapping with LITMUS: Fundamental Limits on lag Recovery Rates
A Bayesian re-analysis of 906 OzDES AGN finds that previous reverberation lags, especially for MgII and CIV, are often false positives, with only 3-28% of MgII sources showing simple lags.
-
jaxdae: A JAX-native Differentiable Solver for Differential-Algebraic Equations in Coupled Multi-physics
jaxdae provides the first JAX-native differentiable DAE solver, using a frozen-grid BDF-2 replay adjoint for reverse-mode gradients and XLA-fused batched sweeps.
-
BOSS-CLAM: Utilizing a Constrained Linear Absorption Model to Infer Stellar Parameters from BOSS Spectra
A new generative pipeline, BOSS-CLAM, infers temperature, gravity, metallicity, and alpha-abundance for 1,708,214 SDSS-V BOSS spectra and releases a validated clean catalog of 915,514 stars.
-
Implicit differentiation of tensor network algorithms
PEPS energy gradients can be computed by implicit differentiation of characteristic equations for the contraction environment, avoiding unstable subroutine backpropagation and reducing asymptotic cost.
-
Restricted Dynamic Geometric Complexity: Path-Space Reduction and M\"obius--Jacobi Response
Restricted dynamic geometric complexity measures the intrinsic affine-invariant path distance from an initial metric to a condition-number target when the metric family is structurally constrained, with exact LMI and ...
-
General Purpose Inverse Design of Heterogeneous Finite-Sized Assemblies
A differentiable wrapper around an analytical equilibrium-yield calculation enables gradient-based design of dimers, temperature-switchable shells, and size-controlled polymers, validated by molecular dynamics.
-
linrax: A JAX Compatible, Simplex Method Linear Program Solver
linrax implements the simplex method in JAX, enabling JIT-compiled, differentiable LP solving with support for degenerate constraints.
-
Adversarial Machine Learning Attacks on Financial Reporting via Maximum Violated Multi-Objective Attack
MVMO, a new weighted multi-objective attack, can inflate earnings and lower fraud scores in about 50 to 66 percent of firm-years, versus under 14 percent for standard attacks.
-
DiffCoTune: Differentiable Co-Tuning for Cross-domain Robot Control
DiffCoTune co-tunes simulator and controller parameters via gradient-based alternating optimization, improving sim-to-real transfer with fewer than five deployment rollouts.
-
Diff-FlowFSI: A GPU-Optimized Differentiable CFD Platform for High-Fidelity Turbulence and FSI Simulations
A fully differentiable, GPU-optimized CFD platform for turbulence and fluid-structure interaction is validated on benchmarks and demonstrated for inverse modeling and hybrid neural-physics learning.
-
Resource-Efficient Quantum Optimization via Higher-Order Encoding
HUBO encodings reduce qubit counts from n*m to n*ceil(log2 m) and cut CNOT counts by 89.6-100% in QAOA benchmarks on gate assignment, max k-colorable subgraph, and integer programming instances.
-
Unveiling the Atmosphere of HR 7672 B from the Near-Infrared High-Resolution Spectrum Using REACH/Subaru
High-resolution REACH spectra of the brown dwarf HR 7672 B reveal water and iron hydride absorption, and a preference for an optically thick cloud layer near 10^1.16 bar.
-
rodeo: Probabilistic Methods of Parameter Inference for Ordinary Differential Equations
rodeo is a JAX-based Python library that implements probabilistic ODE solvers and several Bayesian parameter inference methods with linear scaling in system size.
-
The Gittins Index: A Design Principle for Decision-Making Under Uncertainty
The Gittins index is presented as a general design principle that optimally solves many independent-chain decision problems and gives strong approximate solutions in Bayesian optimization and tail-latency scheduling.
Discussion (0). Continue with ORCID to comment.