Pith. sign in

REVIEW 10 cited by

Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1909.09157 v2 pith:Y75BWMHG submitted 2019-09-19 cs.LG stat.ML

Aniruddh Raghu , Maithra Raghu , Samy Bengio , Oriol Vinyals This is my paper · ORCID
classification cs.LGstat.ML
keywords mamllearningfeatureloopreusealgorithmheadinner
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, from which the inner loop can efficiently learn new tasks. Despite MAML's popularity, a fundamental open question remains -- is the effectiveness of MAML due to the meta-initialization being primed for rapid learning (large, efficient changes in the representations) or due to feature reuse, with the meta initialization already containing high quality features? We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor. This leads to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML where we remove the inner loop for all but the (task-specific) head of a MAML-trained network. ANIL matches MAML's performance on benchmark few-shot image classification and RL and offers computational improvements over MAML. We further study the precise contributions of the head and body of the network, showing that performance on the test tasks is entirely determined by the quality of the learned features, and we can remove even the head of the network (the NIL algorithm). We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DUPLE: An Intelligent Cross-Deployment Recognition Framework for Fiber-Optic Perimeter Security under Scarce Target Labels

    cs.LG 2025-11 unverdicted novelty 6.0 of 10

    DUPLE, a dual-domain multi-prototype meta-learning framework with statistical guidance, improves cross-deployment fiber-optic event recognition on two private few-shot benchmarks.

  2. Disentangled Multi-Context Meta-Learning: Unlocking robust and Generalized Task Learning

    cs.RO 2025-09 conditional novelty 6.0 of 10

    DMCM learns one context vector per task factor and updates only the relevant vector, improving out-of-distribution robustness in sine regression and quadruped locomotion via context sharing.

  3. Learning to Imitate with Less: Efficient Individual Behavior Modeling in Chess

    cs.AI 2025-07 conditional novelty 6.0 of 10

    Maia4All models individual chess players' move choices from as few as 20 games by enriching a population-level model with prototype players and then initializing personal embeddings via prototype matching.

  4. CLaDMoP: Learning Transferrable Models from Successful Clinical Trials via LLMs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    CLaDMoP pre-trains a two-branch model on successful clinical trials with a pair-matching objective, then fine-tunes it to predict trial success, outperforming prior models on the TOP benchmark.

  5. From Trajectories to Instructions: Language-Conditioned Meta-Reinforcement Learning

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A language-instruction embedding can replace the gradient-based inner loop of MAML, yielding competitive BabyAI performance with lower per-iteration wall-clock time.

  6. Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track

    cs.LG 2025-06 conditional novelty 5.0 of 10

    ML conferences should create an official peer-reviewed track dedicated to refuting and critiquing previously published work.

  7. CCoMAML: Efficient Cattle Identification Using Cooperative Model-Agnostic Meta-Learning

    cs.CV 2025-09 conditional novelty 4.0 of 10

    CCoMAML, a Cooperative MAML variant with a CNN co-learner, reports strong few-shot cattle identification from muzzle images, but its test-set-tuned hyperparameters and best-split reporting weaken the result.

  8. Unsupervised Multi-Attention Meta Transformer for Rotating Machinery Fault Diagnosis

    cs.LG 2025-09 reject novelty 4.0 of 10

    MMT-FD combines time-frequency self-supervised alignment, multi-head attention, a Transformer encoder, and MAML meta-learning to reach around 93-99% fault-diagnosis accuracy with 1-10% labeled data in reported experiments.

  9. Domain-Generalization to Improve Learning in Meta-Learning Algorithms

    cs.LG 2025-08 reject novelty 4.0 of 10

    DGS-MAML layers gradient matching onto SharpMAML and claims O(1/T) convergence and tighter PAC-Bayes bounds, but the displayed theorems give O(1/sqrt T) under the paper's own parameter choices.

  10. Acquiring and Adapting Priors for Novel Tasks via Neural Meta-Architectures

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A meta-learning dissertation showing that distributed memory and hypernetworks can adapt to new tasks with few samples, applied to image classification, text-to-3D generation, and molecular binding prediction, with th...

Pith tools