pith. sign in

arxiv: 1802.03268 · v2 · pith:JEA2NXYAnew · submitted 2018-02-09 · 💻 cs.LG · cs.CL· cs.CV· cs.NE· stat.ML

Efficient Neural Architecture Search via Parameter Sharing

Hieu Pham , Melody Y. Guan , Barret Zoph , Quoc V. Le , Jeff Dean This is my paper
classification 💻 cs.LG cs.CLcs.CVcs.NEstat.ML
keywords enasarchitectureneuralmodelsearchsubgraphtestarchitectures
0
0 comments X
read the original abstract

We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. In ENAS, a controller learns to discover neural network architectures by searching for an optimal subgraph within a large computational graph. The controller is trained with policy gradient to select a subgraph that maximizes the expected reward on the validation set. Meanwhile the model corresponding to the selected subgraph is trained to minimize a canonical cross entropy loss. Thanks to parameter sharing between child models, ENAS is fast: it delivers strong empirical performances using much fewer GPU-hours than all existing automatic model design approaches, and notably, 1000x less expensive than standard Neural Architecture Search. On the Penn Treebank dataset, ENAS discovers a novel architecture that achieves a test perplexity of 55.8, establishing a new state-of-the-art among all methods without post-training processing. On the CIFAR-10 dataset, ENAS designs novel architectures that achieve a test error of 2.89%, which is on par with NASNet (Zoph et al., 2018), whose test error is 2.65%.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AGAN: Towards Automated Design of Generative Adversarial Networks

    cs.LG 2019-06 unverdicted novelty 8.0

    AGAN is the first neural architecture search method for GANs that discovers architectures outperforming state-of-the-art on CIFAR-10 unsupervised image generation and competitive on supervised tasks.

  2. Characterizing Learning in Deep Neural Networks using Tractable Algorithmic Complexity Analysis

    cs.LG 2026-05 unverdicted novelty 7.0

    QuBD extends algorithmic complexity estimation to quantized DNN weights, revealing that complexity decreases during learning, increases with overfitting, follows grokking patterns, and correlates with generalization.

  3. Auto-FP: An Experimental Study of Automated Feature Preprocessing for Tabular Data

    cs.LG 2023-10 unverdicted novelty 7.0

    Experimental comparison of 15 HPO and NAS algorithms for automated feature preprocessing on 45 tabular datasets finds evolution-based methods and random search as top performers.

  4. Switchable Normalization for Learning-to-Normalize Deep Representation

    cs.CV 2019-07 unverdicted novelty 7.0

    Switchable Normalization learns per-layer weights to combine channel, layer, and minibatch normalizers, claiming robustness to batch size and better results than fixed normalizers on ImageNet, COCO, CityScapes, ADE20K...

  5. Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning

    cs.LG 2019-06 unverdicted novelty 7.0

    Placeto learns generalizable RL policies for device placement via iterative improvements and graph embeddings, needing up to 6.1x fewer steps than prior methods and applying to unseen graphs without retraining.

  6. Video Action Recognition Via Neural Architecture Searching

    cs.CV 2019-07 unverdicted novelty 6.0

    Uses differentiable NAS with temporal segments and pseudo-3D operators to discover a video action recognition network that outperforms hand-designed models on UCF101 with ~1% of the parameters when trained from scratch.

  7. EnforceNet: Monocular Camera Localization in Large Scale Indoor Sparse LiDAR Point Cloud

    cs.CV 2019-07 unverdicted novelty 5.0

    EnforceNet achieves centimeter-level monocular camera localization in sparse LiDAR maps of indoor parking garages via a novel resistor module that improves generalization, accuracy, and training speed.

  8. EPNAS: Efficient Progressive Neural Architecture Search

    cs.LG 2019-07 unverdicted novelty 5.0

    EPNAS uses a progressive search policy with REINFORCE performance prediction to search neural architectures in parallel, supporting multiple resource constraints and outperforming ENAS and PNAS on CIFAR-10 and ImageNe...

  9. Self-Adaptive 2D-3D Ensemble of Fully Convolutional Networks for Medical Image Segmentation

    eess.IV 2019-07 unverdicted novelty 4.0

    Self-adaptive 2D-3D FCN ensemble optimized by multiobjective evolution for prostate segmentation on PROMISE12 achieves top-10 ranking with smaller size than prior auto-designed models.

  10. MLFriend: Interactive Prediction Task Recommendation for Event-Driven Time-Series Data

    cs.LG 2019-06 unverdicted novelty 4.0

    MLFriend enumerates prediction tasks for event-driven time-series data and interactively recommends useful ones, with evaluation on three datasets yielding 2885 tasks of which 722 were deemed useful by experts.

  11. Genetic Network Architecture Search

    cs.NE 2019-07 unverdicted novelty 3.0

    Genetic algorithm searches convolution cell architectures with weight sharing via SGD, reporting 96% accuracy on CIFAR10 and 80.1% on CIFAR100.