REVIEW 8 cited by
Training of Physical Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Physical neural networks (PNNs) are a class of neural-like networks that leverage the properties of physical systems to perform computation. While PNNs are so far a niche research area with small-scale laboratory demonstrations, they are arguably one of the most underappreciated important opportunities in modern AI. Could we train AI models 1000x larger than current ones? Could we do this and also have them perform inference locally and privately on edge devices, such as smartphones or sensors? Research over the past few years has shown that the answer to all these questions is likely "yes, with enough research": PNNs could one day radically change what is possible and practical for AI systems. To do this will however require rethinking both how AI models work, and how they are trained - primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs at large scale, many methods including backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs, and so far no method has been shown to scale to the same scale and performance as the backpropagation algorithm widely used in deep learning today. However, this is rapidly changing, and a diverse ecosystem of training techniques provides clues for how PNNs may one day be utilized to create both more efficient realizations of current-scale AI models, and to enable unprecedented-scale models.
Forward citations
Cited by 8 Pith papers
-
Universality of physical neural networks with multivariate nonlinearity
A physical neural network is universal exactly when its multivariate nonlinear encoding function has no identically vanishing partial derivative.
-
An analog-electronic implementation of a harmonic oscillator recurrent neural network
An analog circuit implementing a four-node harmonic oscillator network preserves enough information to match its digital twin's sMNIST classification accuracy with a retrained linear readout.
-
Higher-Order Kuramoto Oscillator Network for Dense Associative Memory
A Kuramoto oscillator network with quartic Hebbian coupling stores patterns with superlinear capacity scaling and shows a tricritical point separating continuous from explosive retrieval.
-
Equilibrium Propagation for Dissipative Dynamics
An effective action with time-reversed trajectories extends equilibrium propagation to damped linear reciprocal networks, enabling temporal learning demonstrated on mechanical and RLC systems.
-
Learning long range dependencies through time reversal symmetry breaking
RHEL computes backpropagation-equivalent gradients for Hamiltonian recurrent networks using finite differences of time-reversed, nudged trajectories, and matches BPTT accuracy on sequence tasks up to 50k steps.
-
Learning with springs and sticks
A damped spring-and-stick lattice performs regression by energy relaxation, and a reported 'thermodynamic learning barrier' sets the minimum free energy needed for learning.
-
Fully analog end-to-end online training with real-time adaptibility on integrated photonic platform
A 2-weight photonic microring weight bank is trained online with multiplexed gradient descent, achieving adaptive classification, although the training loop relies on digital FPGA computation.
-
Solving the compute crisis with physics-based ASICs
A coalition of academic and industry researchers argues that chips exploiting natural physical dynamics, rather than enforcing digital abstractions, could dramatically cut AI computing costs.
Discussion (0). Sign in to comment.