REVIEW 2 cited by
Neural Collapse with Cross-Entropy Loss
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
We consider the variational problem of cross-entropy loss with $n$ feature vectors on a unit hypersphere in $\mathbb{R}^d$. We prove that when $d \geq n - 1$, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that as $n \rightarrow \infty$ with fixed $d$, the minimizing points will distribute uniformly on the hypersphere and show a connection with the frame potential of Benedetto & Fickus.
Forward citations
Cited by 2 Pith papers
-
Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient
Action Collapse Policy Gradient (ACPG) fixes the action-selection layer to a simplex ETF and claims improved discrete-action RL performance, with a theory that only covers a weighted optimal-action imitation objective.
-
Memory-efficient Continual Learning with Neural Collapse Contrastive
A continual learning method that combines focal contrastive learning with fixed neural-collapse prototypes and a distillation loss achieves state-of-the-art accuracy in memory-free class- and task-incremental learning.
Discussion (0). Continue with ORCID to comment.