Neural networks exhibit grokking on small algorithmic datasets, achieving perfect generalization well after overfitting.
Adding gradient noise improves learning for very deep networks
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
representative citing papers
The regularized Transformer loss satisfies Villani's coercive energy criteria, yielding log-Sobolev constants C_LS ≤ λ^{-1} + d/λ² and finite-time convergence bounds for noisy SGD.
Language models show good calibration when asked to estimate the probability that their own answers are correct, with performance improving as models get larger.
ST-MoE introduces stability techniques for sparse expert models, allowing a 269B-parameter model to achieve state-of-the-art transfer learning results across reasoning, summarization, and QA tasks at the compute cost of a 32B dense model.
Ranked preference modeling outperforms imitation learning for language model alignment and scales more favorably with model size.
SignSGD with pre-sign dithering and a calibrated hybrid switch to SGD achieves 92.18% accuracy on CIFAR-10 with ResNet-18, outperforming pure SGD and SignSGD, plus better results than Adam on CIFAR-100.
Scalar-irreducible dynamics enable internally generated regime transitions in learning systems via feedback between fast dynamical variables and slow structural adaptation.
citing papers explorer
-
Language Models (Mostly) Know What They Know
Language models show good calibration when asked to estimate the probability that their own answers are correct, with performance improving as models get larger.
-
ST-MoE: Designing Stable and Transferable Sparse Expert Models
ST-MoE introduces stability techniques for sparse expert models, allowing a 269B-parameter model to achieve state-of-the-art transfer learning results across reasoning, summarization, and QA tasks at the compute cost of a 32B dense model.
-
A General Language Assistant as a Laboratory for Alignment
Ranked preference modeling outperforms imitation learning for language model alignment and scales more favorably with model size.