REVIEW 1 cited by
On Dissipativity of Cross-Entropy Loss in Training ResNets
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The training of ResNets and neural ODEs can be formulated and analyzed from the perspective of optimal control. This paper proposes a dissipative formulation of the training of ResNets and neural ODEs for classification problems by including a variant of the cross-entropy as a regularization in the stage cost. Based on the dissipative formulation of the training, we prove that the trained ResNet exhibit the turnpike phenomenon. We then illustrate that the training exhibits the turnpike phenomenon by training on the two spirals and MNIST datasets. This can be used to find very shallow networks suitable for a given classification task.
Forward citations
Cited by 1 Pith paper
-
Towards an Optimal Control Perspective of ResNet Training
Training ResNets with a stage cost on intermediate outputs obtained through skip connections biases deep residual layers toward identity mappings and enables pruning with small accuracy loss on homogeneous models.
Discussion (0). Sign in to comment.