REVIEW 9 cited by
Single Path One-Shot Neural Architecture Search with Uniform Sampling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We revisit the one-shot Neural Architecture Search (NAS) paradigm and analyze its advantages over existing NAS approaches. Existing one-shot method, however, is hard to train and not yet effective on large scale datasets like ImageNet. This work propose a Single Path One-Shot model to address the challenge in the training. Our central idea is to construct a simplified supernet, where all architectures are single paths so that weight co-adaption problem is alleviated. Training is performed by uniform path sampling. All architectures (and their weights) are trained fully and equally. Comprehensive experiments verify that our approach is flexible and effective. It is easy to train and fast to search. It effortlessly supports complex search spaces (e.g., building blocks, channel, mixed-precision quantization) and different search constraints (e.g., FLOPs, latency). It is thus convenient to use for various needs. It achieves start-of-the-art performance on the large dataset ImageNet.
Forward citations
Cited by 9 Pith papers
-
Once-for-All: Train One Network and Specialize it for Efficient Deployment
A single once-for-all network, trained with progressive shrinking, can be specialized to many hardware platforms by selecting a sub-network, matching or beating separate per-device training.
-
Improving Routability Prediction via NAS Using a Smooth One-shot Augmented Predictor
A new neural architecture search pipeline that averages multiple one-shot networks and augments the predictor data achieves a 0.9802 ROC-AUC for chip routing hotspot detection.
-
HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking
HM-NAS reaches 2.41% test error on CIFAR-10 with 1.8M parameters and 1.8 GPU days, and 73.4% top-1 on ImageNet, by learning hierarchical masks over a weight-sharing supernet.
-
SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search
Replacing skip connections with a learnable 1x1 convolution (ELS) during weight-sharing supernet training stabilizes the supernet and improves architecture ranking, yielding the SCARLET family of ImageNet models.
-
Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
Two-stage and gradually decreasing quantization, stochastic precision sampling, and joint teacher-student distillation each improve low-bit CNN accuracy on ImageNet and CIFAR-100, with the largest gains when combined.
-
Searching Efficient Deep Architectures for Radar Target Detection using Monte-Carlo Tree Search
Monte-Carlo tree search found a radar detection network with 60% fewer parameters than a baseline U-Net at comparable detection performance.
-
Delta-NAS: Difference of Architecture Encoding for Predictor-based Evolutionary Neural Architecture Search
Delta-NAS encodes pairs of one-edit-away architectures as sparse differences and learns to predict the accuracy gap, using that predictor to guide evolutionary search.
-
MoGA: Searching Beyond MobileNetV3
MoGA uses weighted NSGA-II and a one-shot supernet to search mobile GPU-aware architectures, producing models that beat MobileNetV3 on ImageNet at similar mobile GPU latency with 200x less search cost than MnasNet.
-
Efficient Deep Neural Networks
A dissertation showing that deep learning can be made practical on edge devices through four complementary routes: model, data, hardware, and design efficiency.
Discussion (0). Continue with ORCID to comment.