Pith. sign in

REVIEW 12 cited by

Prodigy: An Expeditiously Adaptive Parameter-Free Learner

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.06101 v4 pith:OELRUVXT submitted 2023-06-09 cs.LG cs.AImath.OCstat.ML

Konstantin Mishchenko , Aaron Defazio This is my paper
classification cs.LGcs.AImath.OCstat.ML
keywords trainingprodigyd-adaptationlearningrateadamadaptivedataset
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

We consider the problem of estimating the learning rate in adaptive methods, such as AdaGrad and Adam. We propose Prodigy, an algorithm that provably estimates the distance to the solution $D$, which is needed to set the learning rate optimally. At its core, Prodigy is a modification of the D-Adaptation method for learning-rate-free learning. It improves upon the convergence rate of D-Adaptation by a factor of $O(\sqrt{\log(D/d_0)})$, where $d_0$ is the initial estimate of $D$. We test Prodigy on 12 common logistic-regression benchmark datasets, VGG11 and ResNet-50 training on CIFAR10, ViT training on Imagenet, LSTM training on IWSLT14, DLRM training on Criteo dataset, VarNet on Knee MRI dataset, as well as RoBERTa and GPT transformer training on BookWiki. Our experimental results show that our approach consistently outperforms D-Adaptation and reaches test accuracy values close to that of hand-tuned Adam.

Discussion (0). Sign in to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration

    cs.LG 2025-06 conditional novelty 8.0 of 10

    A single proof unifies convergence analyses of AdaGrad-Norm, AdaGrad, ASGO, and DASGO under Hölder smoothness, and shows AdaGrad/DASGO can be accelerated with Nesterov momentum.

  2. Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization

    math.OC 2025-07 conditional novelty 7.0 of 10

    Accelerated GRAAL is the first adaptive first-order method that proves near-optimal accelerated complexity for convex L-smooth and (L0,L1)-smooth functions with geometric stepsize growth.

  3. Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Appearance pointers are compact tokens that let a diffusion transformer apply text, image, or combined prompts to specific image regions in a single pass.

  4. OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A meta-pipeline plus LMO four-axis view yields a dual taxonomy of 108 optimizers, and a multi-objective LLM/vision benchmark shows no single family dominates the quality–cost–memory frontier.

  5. Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    ART is a two-stage framework that initializes makeup transfer with pseudo-targets then refines via a reality-anchored differentiable cycle on real references, plus the new MF2K 2K-resolution makeup dataset.

  6. Decomposing Subject-Driven Image Generation via Intermediate Structural Prediction

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    A two-stage method predicts an intermediate Canny map for structure then renders the image conditioned on appearance and structure, paired with a 100k text-aware dataset, to improve detail preservation in subject-driv...

  7. Superbunched random fiber laser

    physics.optics 2026-03 unverdicted novelty 6.0 of 10

    A fiber-integrated random laser uses Rayleigh scattering, cascaded Brillouin scattering, and four-wave mixing to generate multi-wavelength superbunched light with g(2)(0) up to ~26 and improved temporal ghost imaging.

  8. Image Editing As Programs with Diffusion Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    IEAP decomposes complex editing instructions into atomic operations executed sequentially on a diffusion transformer, and reports state-of-the-art results on MagicBrush and AnyEdit.

  9. AlignGen: Boosting Personalized Image Generation with Cross-Modality Prior Alignment

    cs.CV 2025-05 conditional novelty 6.0 of 10

    AlginGen improves zero-shot personalized image generation by training a learnable token and a selective attention mask that align textual and visual priors, achieving the best balance of concept preservation and promp...

  10. AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent

    cs.LG 2025-05 conditional novelty 6.0 of 10

    AutoSGD runs three parallel SGD streams at nearby learning rates, uses paired noisy objective estimates to pick the winner, and is claimed to converge with little user tuning.

  11. ZENITH: Automated Gradient Norm Informed Stochastic Optimization

    cs.LG 2026-01 reject novelty 5.0 of 10

    ZENITH tunes the learning rate to the ratio of the current gradient norm to its historical peak and reports accuracy wins over 11 automatic optimizers on 6 classification benchmarks and MS COCO detection/segmentation.

  12. CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-step

    cs.CV 2025-07 conditional novelty 4.0 of 10

    CoT-Diff couples a multimodal LLM's step-by-step 3D layout reasoning into the diffusion denoising loop, claiming large gains in spatial alignment for text-to-image generation.

Pith tools