REVIEW 5 cited by
SNAS: Stochastic Neural Architecture Search
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline. In this work, NAS is reformulated as an optimization problem on parameters of a joint distribution for the search space in a cell. To leverage the gradient information in generic differentiable loss for architecture search, a novel search gradient is proposed. We prove that this search gradient optimizes the same objective as reinforcement-learning-based NAS, but assigns credits to structural decisions more efficiently. This credit assignment is further augmented with locally decomposable reward to enforce a resource-efficient constraint. In experiments on CIFAR-10, SNAS takes less epochs to find a cell architecture with state-of-the-art accuracy than non-differentiable evolution-based and reinforcement-learning-based NAS, which is also transferable to ImageNet. It is also shown that child networks of SNAS can maintain the validation accuracy in searching, with which attention-based NAS requires parameter retraining to compete, exhibiting potentials to stride towards efficient NAS on big datasets. We have released our implementation at https://github.com/SNAS-Series/SNAS-Series.
Forward citations
Cited by 5 Pith papers
-
NADER: Neural Architecture Design via Multi-Agent Collaboration
NADER uses a multi-agent LLM team with a graph-based block representation and a reflection memory to iteratively propose and test modified neural architectures, claiming gains beyond NAS-Bench-201's optimum on CIFAR a...
-
HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning
HEP-NAS partitions a NAS supernet by connection hierarchies, trains the candidate branches with mutual distillation, and greedily keeps the best branch, reporting lower error than prior few-shot NAS methods on standar...
-
Differentiable Learning-to-Group Channels via Groupable Convolutional Neural Networks
A Kronecker-product binary mask makes the number of groups in each convolution layer learnable end-to-end, giving 0.5 to 1.7 percent top-1 ImageNet gains over ResNet and ResNeXt at comparable cost.
-
Scalable Reinforcement-Learning-Based Neural Architecture Search for Cancer Deep Learning Research
Reinforcement-learning-based neural architecture search finds smaller and faster neural networks with accuracy comparable to, or better than, manually designed networks on three cancer drug-response benchmarks.
-
Regularizing Differentiable Architecture Search with Smooth Activation
SA-DARTS regularizes DARTS by penalizing the mean of architecture weights, reporting results on benchmarks that are similar to, but not clearly better than, existing methods.
Discussion (0). Continue with ORCID to comment.