SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search

Bo Zhang; Qingyuan Li; Ruijun Xu; Xiangxiang Chu; Xudong Li

REVIEW 2 major objections 4 minor 38 references

SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper identifies skip connections as the destabilizing element in one-shot weight-sharing supernets and shows that replacing them during training with a removable learned 1x1 convolution restores ranking and yields a 76.9% ImageNet…

desk verdict A simple, effective stabilizer for skip connections in one-shot NAS, with a mathematical equivalence claim that is stronger than the proof supports. read the letter →

arxiv 1908.06022 v6 pith:MXHJ74FT submitted 2019-08-16 cs.LG cs.CVstat.ML

Xiangxiang Chu , Bo Zhang , Qingyuan Li , Ruijun Xu , Xudong Li This is my paper · ORCID

classification cs.LGcs.CVstat.ML

keywords neuralarchitecturesearchweight-sharingsupernetone-shotNASskipconnectionfeatureinconsistencyequivariantlearnablestabilizerevolutionaryimageclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that weight-sharing neural architecture search can be made scalable to variable network depths by fixing how skip connections are represented inside the supernet. Earlier single-path one-shot supernets deliberately leave skip connections out, because adding them destabilizes training and destroys the supernet's ability to rank candidate models. The paper diagnoses the cause: outputs of a skip connection are much less similar to outputs of the other choices, and the resulting feature inconsistency cascades through the layers. It then proposes the Equivariant Learnable Stabilizer (ELS), a $1\times1$ convolution with no batch-normalization or activation, trained in place of each skip and folded away afterward so that representational capacity is unchanged. With this stabilizer, supernet training converges reliably and ranking improves, and an evolutionary search yields the SCARLET models, of which SCARLET-A reaches 76.9% top-1 on ImageNet.

What carries the argument

The load-bearing object is the Equivariant Learnable Stabilizer (ELS), a learned $1\times1$ convolution with no batch-normalization and no activation, placed where the search space would put an identity skip. Its defining equivalence is Eq. (3): for the input $x^{c_l}_l$ to layer $l$ and every operation $o$ in layer $l+1$, $f^{l+1}_o(x^{c_l}_l)=f^{l+1}_o(f^l_{ELS}(x^{c_l}_l))$. The $1\times1$ map satisfies this because linear convolution commutes with composition: its weights can be folded into the following convolution's weight tensor, so the stabilizer changes no representational capacity. ELS therefore gives the search a homogeneous-feature training signal while leaving the final architecture free of the extra layer.

What would settle it

A direct check is to take a layer where the skip's output has $c_l$ channels while the other operations produce $c_{l+1}\neq c_l$ channels, feed one batch of random inputs through both the raw skip and the ELS into every next-layer operation, and compare $f^{l+1}_o(x^{c_l}_l)$ with $f^{l+1}_o(f^l_{ELS}(x^{c_l}_l))$; a difference above numerical precision at any mismatched layer would show that Lemma 1 does not cover the actual search space. Alternatively, one can compare the supernet's rank correlation on benchmark models when ELS is used only at matched-channel layers versus everywhere; a drop in the all-layers case would show the mismatch is load-bearing.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that the obstacle to one-shot search over variable depths is not skip connections as such but the feature heterogeneity they introduce. In a layer whose choices are inverted-bottleneck blocks and an identity skip, the skip's feature maps have dramatically lower cosine similarity with the other blocks' outputs; when the next layer samples one path, it receives inputs with different statistics depending on the previous choice, and the disturbance accumulates. Replacing the skip during supernet training with an equivariant learnable stabilizer removes the heterogeneity while preserving exactly the same representational capacity, because a $1\times1$ convolution composed with any following convolution can be absorbed into that convolution's weights (Lemma 1). The paper reports that the stabilized supernet is a far better evaluator: on a benchmark with known ground-truth labels, rank correlation rises from near zero to about 0.42 with ELS. Searching this supernet evolutionarily produces SCARLET-A, which reaches 76.9% ImageNet top-1 at 365M multiply-adds, and the paper shows transfer results on CIFAR-10 and object detection.

Load-bearing premise

The load-bearing premise is that Eq. (3) holds at every layer of the supernet, including layers where the skip carries a different number of output channels into the next block than the other choices do; the paper's proof covers only a $1\times1$ convolution followed by a $k\times k$ convolution with matching input channels, so a channel mismatch would leave the trained supernet evaluating a different architecture class than the final models.

Editorial extensions

If this is right

Skip connections can be included in one-shot search spaces without destabilizing supernet training, so searched models can differ in depth rather than only in per-cell choices.
Because ELS is removed after training, final architectures carry no extra parameter cost from the stabilizer, so the accuracy gain is not bought with added runtime.
Better supernet ranking makes the evolutionary search stage more reliable; the paper's ablation puts the gain from ELS in ranking, not in the search algorithm.
The reported result implies that competitive mobile-scale architectures can be found without proxy tasks at modest compute: SCARLET-A's 76.9% top-1 exceeds a 390M-FLOPs strong baseline using 365M FLOPs.
Non-equivariant stabilizers (e.g., adding ReLU) overestimate subnetworks, so the equivalence condition is not a formality but an experimentally visible requirement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Extension: the same linear-absorption argument should transfer to any search space where a candidate operation's input features can be homogenized by a linear map; one could, for example, use a learned $1\times1$ stabilizer for variable-width choices rather than only variable-depth ones.
Extension: measuring cross-choice feature similarity early in supernet training could serve as a cheap diagnostic for whether a given search space will rank well; the paper's cosine-similarity curves make that a testable preprocessing step rather than a post hoc explanation.
Extension: the channel-mismatch worry in Eq. (3) suggests a targeted experiment: restrict ELS to layers whose channel counts match and compare final search quality; if quality does not drop, the shortcut is harmless in practice, and if it does, the proof needs a refined statement.
Extension: the ranking improvement from near zero to about 0.42 implies that weight-sharing evaluations can be made informative with a much smaller fix than retraining every subnetwork, which may make one-shot search competitive as a cheap ranking oracle for broader model-selection tasks.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth reading: it is the first to seriously address what happens when you add skip connections to a single-path one-shot supernet, and the proposed fix—a learnable 1x1 convolution (ELS) that stands in for the skip during training—is simple and clearly effective. The empirical story is genuinely good: training curves stabilize, the Kendall tau on NAS-Bench-101 jumps from 0.01 to 0.42, and the final SCARLET models are trained from scratch, so the headline ImageNet numbers do not depend on the supernet proxy. That is real evidence, not hype.

The soft spot is the theory. Lemma 1 claims Equivalence, not just helpful stabilization, and the proof in Appendix A only covers a 1x1 conv followed by a kxk conv with matching channel counts. But the search spaces S1 and S2 include skip connections that change channel counts—Fig. 11 explicitly shows the skip output having half the width of the other choices—and the next block's first conv is shared across all choices. The "iteratively apply" line in the proof does not survive contact with MB nonlinearities, SE, or consecutive skips. The NAS-Bench-101 experiment, which is the one clean ranking test, adds a 1x1 projection to avoid channel mismatch, so it sidesteps exactly the case where the proof is missing. This is not a fatal flaw: the method clearly helps even if the equivalence is inexact, and the final architectures are not trained through the supernet. But the paper overclaims when it says ELS leaves the search space unchanged. A serious referee should ask the authors to either extend the proof to the channel-mismatched case or soften the claim to "approximate equivalence."

Also, the 0.6% gain over EfficientNet-B0 is a single training run with no error bars; not a big deal for this area, but worth noting.

Bottom line: this deserves peer review. The core idea is likely correct, the empirical work is solid, and the gaps are addressable. I would bring it to a reading group and probably cite it if I did NAS work. It just needs a more honest statement of what is proven and what is empirical.

Referee Report

2 major / 4 minor

Summary. The paper addresses weight-sharing one-shot NAS in search spaces with variable depth, where skip connections are added to the usual MobileNetV2-style blocks. The authors observe that a vanilla scalable supernet suffers from training instability and poor subnetwork ranking, and they attribute this to feature inconsistency between the skip connection and the other choice blocks. They propose replacing the skip with a learnable 1x1 convolution, the Equivariant Learnable Stabilizer (ELS), during supernet training. They claim (Lemma 1, Eq. 3) that this stabilizer does not change the representational capacity of any subnetwork, so it can be removed after training. Experiments show lower variance in training curves, improved Kendall Tau on NAS-Bench-101 (from 0.0118 to 0.421), and a family of SCARLET models with competitive ImageNet accuracies, the best of which (SCARLET-A) reaches 76.9% top-1 accuracy. The final models are trained from scratch after the search, so the headline accuracies do not depend on the supernet proxy.

Significance. If the equivalence claim holds, the paper contributes a practical fix for a real failure mode in one-shot NAS with variable depth: the ELS mechanism is simple, cost-free during search, and demonstrably improves ranking on an external benchmark. The empirical backbone is generally solid: the NAS-Bench-101 ranking experiment uses ground-truth accuracies; the final ImageNet models are re-trained from scratch; and the ablations (training curves, feature similarity, constrained optimization) directly support the main narrative. The central risk is the gap between the proof of Lemma 1 and the actual search spaces, which contain layers where the skip output channel count differs from the other choices. Since the entire ranking guarantee rests on Eq. (3), this gap is load-bearing and must be closed or the search space clarified before the claims can be accepted.

major comments (2)

[Section 4.1 and Appendix A, Eq. (3)] Lemma 1's proof covers only a 1x1 convolution followed by a kxk convolution with matching input channels. In the search spaces S1/S2, at channel-changing layers the skip output has c_l channels while every other choice block outputs c_{l+1} channels; the Fig. 11 caption explicitly states 'the channel size of feature map after Choice 6 ... is half of others'. For these layers the right-hand side of Eq. (3), f^o_{l+1}(x^{c_l}_l), is not a well-defined input to a shared next block whose first convolution is shaped for c_{l+1} channels, and the 'iteratively apply' argument cannot pass through the MB nonlinearities (expansion, depthwise, SE) because folding ELS into the next block would require changing that block's weights to a tensor different from the shared one. Since SCARLET-B and -C are reported to contain identity operations, including at late stages where channel counts change, the supernet may have ranked architectures that are not the same as those later trained from scratch. The only direct ranking benchmark in Section 6.2 deliberately adds a 1x1 projection to avoid channel mismatch, which sidesteps exactly this case. Please either extend the equivalence proof to mismatched layers, or state that skip connections are only allowed when c_l = c_{l+1} and rerun or justify the affected experiments.
[Section 3.1 and Appendix C.1] The text repeatedly calls the skip connection 'parameter-free', yet the vignette in Fig. 11 and the search-space description imply that a bare identity can be selected at layers where c_l differs from c_{l+1}. A bare identity cannot be followed by a next-layer block whose first convolution expects c_{l+1} input channels; the resulting computational graph is ill-defined. This makes it unclear how the vanilla scalable supernets in Fig. 3 were trained at all. Please specify the exact skip implementation (for example, a 1x1 projection at downsampling layers, or a restriction of skip choices to channel-matched layers) and confirm that the training curves and ranking experiments use that implementation. Without this clarification, the observed 'instability' could be an artifact of an invalid architecture rather than a genuine feature-inconsistency effect.

minor comments (4)

[Abstract and Introduction] There are several small wording issues, such as 'we are mainly concerned about finding models' (Introduction) and 'it creates a large range of perturbation' (Abstract); these should be polished.
[Section 4.1, Eq. (3)] The notation f^o_{l+1} is used for the next-layer operation on both sides of Eq. (3), but if f^o_{l+1} is a shared block that expects c_{l+1} input channels, the expression on the right side is ambiguous at mismatched layers; the paper should define the input channel dimension of f^o_{l+1} explicitly.
[Section 6.2, Fig. 7] The NAS-Bench-101 experiment samples 100 models for the Kendall Tau comparison; it would be helpful to report the variance across multiple seeds or at least the number of supernet training runs, since a single run at tau=0.421 could be optimistic.
[Appendix C.2, Table 7] Several hyperparameters in Table 7 (prm, pre, ppr, pM, pK-M) are not defined in the text or pseudocode; please add a brief description of the mutation/crossover probabilities.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the only FairNAS self-citation is motivational, and the ELS/ranking claims are independently checked against external benchmarks.

full rationale

The derivation chain is not circular. The load-bearing equivalence in Eq. 3 is not assumed from the target result: Lemma 1 states that a 1x1 convolution can act as an equivariant stabilizer, and Appendix A proves the factorization by explicitly constructing the weight tensor w3_{u,c,q,q} = sum_p w2_{p,c,q,q} w1_{u,p,1,1} (Eq. 8); for the non-skip operations this is a direct algebraic identity, not a re-statement of the claim. The ranking contribution is validated against the external NAS-Bench-101 benchmark (Section 6.2, Kendall tau 0.421 with ELS vs 0.0118 without), and the final SCARLET-A, -B, -C numbers in Table 2 are standalone models trained from scratch after ELS is removed, so no headline accuracy is a supernet score recycled as a prediction. The only author self-citation is FairNAS [6], used as motivation ('feature similarity is critical for single-path training') and as a training/search baseline; that premise does not force the ELS result, so it is not load-bearing. A real weakness exists but is a correctness or proof-gap issue, not circularity: Appendix A proves the factorization for a convolution whose direct input has c_l channels and then says the skip case follows by 'iteratively apply[ing]' the proof, whereas Fig. 11 and its caption explicitly show the skip output can be half the width of the other choice blocks (16 vs 32). Under-argument of a lemma is not the same as the claim reducing to its own input, so the circularity score remains low.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the standard one-shot proxy, the reparameterization property of convolutions, and an unstated implementation assumption about channel-mismatched skip connections. The hand-chosen search weights do not affect the ELS stabilization claim.

free parameters (2)

Objective weights (w_acc, w_madds, w_params) = 0.4, 0.4, 0.2
Hand-chosen for the weighted NSGA-II evolutionary search; they influence which Pareto-front models are selected but not the ELS stabilizer itself.
Search constraints (acc_min, madds_max) = 0.4, 500M
Hand-chosen thresholds to exclude trivial all-skip and over-cost models during evolution.

assumptions (3)

standard math A 1x1 convolution followed by a kxk convolution can be reparameterized as a single kxk convolution (Appendix A).
Used in the proof of Lemma 1 for the non-skip operations.
domain assumption The skip connection at a channel-mismatched layer is implementable in the supernet (e.g., via zero-padding or projection) and its capacity is comparable to ELS.
The paper does not describe how a 16-channel skip output is consumed by a next layer expecting 32 channels (Fig. 11), yet ELS is claimed equivalent to it.
domain assumption Inherited-weights evaluation of sampled subnetworks is a valid proxy for their stand-alone quality (the one-shot assumption).
Standard for the SPOS/FairNAS pipeline adopted in Section 2.1; the paper relies on this for the second-stage ranking.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search." pith.science (2026). https://pith.science/paper/MXHJ74FT

@misc{pith2026190806022,
  author       = {Pith},
  title        = {Pith review of: SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MXHJ74FT}},
  note         = {Machine review of arXiv:1908.06022}
}

read the original abstract

To discover powerful yet compact models is an important goal of neural architecture search. Previous two-stage one-shot approaches are limited by search space with a fixed depth. It seems handy to include an additional skip connection in the search space to make depths variable. However, it creates a large range of perturbation during supernet training and it has difficulty giving a confident ranking for subnetworks. In this paper, we discover that skip connections bring about significant feature inconsistency compared with other operations, which potentially degrades the supernet performance. Based on this observation, we tackle the problem by imposing an equivariant learnable stabilizer to homogenize such disparities. Experiments show that our proposed stabilizer helps to improve the supernet's convergence as well as ranking performance. With an evolutionary search backend that incorporates the stabilized supernet as an evaluator, we derive a family of state-of-the-art architectures, the SCARLET series of several depths, especially SCARLET-A obtains 76.9% top-1 accuracy on ImageNet. Code is available at https://github.com/xiaomi-automl/ScarletNAS.

Figures

Figures reproduced from arXiv: 1908.06022 by the authors.

**Figure 1.** ELS helps to calibrate feature inconsistencies in scalable supernets with boosted cosine similarity (compared to Fig.4) and reduced angles among feature vectors (from blue shade to red). The cosine similarity is calculated on the third layer’s output feature maps from 7 paralleled choice blocks (MobileNetV2’s blocks numbered 0 to 5 and a skip connection). Each block’s feature vectors are projected to 2-dimensional s… view at source ↗

**Figure 2.** Our proposed SCARLET-NAS pipeline where the scalable supernet is stabilized by ELS, which also provides good ranking ability compared with the vanilla approach. ELS is removed from the final subnetwork to train from scratch. Note SC and ELS can appear on each row (layer), only one is drawn for brevity. 2 Preliminary Background 2.1 Single-Path Supernet Training The weight-sharing mechanism is now widely applied in ne… view at source ↗

**Figure 3.** Training supernet with Single Path One-Shot [12] and FairNAS [6] on ImageNet with and without Equivariant Learnable Stabilizer (ELS) in search space S1. Top: The supernets with ELS enjoy better convergence (red thick lines) and small variance (red shaded area). Bottom: Histogram of randomly sampled 1k one-shot models’ accuracies. Supernets with ELS have an improved estimation of subnetworks. In particular, we need … view at source ↗

Figures from the paper (9 more)

**Figure 4.** Figure 4: Cosine similarity matrices of the third layer’s outputs (averaged on 32 channels of 28 × 28 feature maps) from 7 choice blocks of supernets trained without ELS. The average similarity is shown as x-axis at the top. The skip connection yields different feature maps from…

**Figure 3.** Figure 3: What’s worse, it makes big trouble for the supernet to predict submod [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]

**Figure 5.** Figure 5: The architectures of SCARLET-A, B and C (from top to bottom). Downsampling points are indicated by dashed lines. The stem and tail parts are omitted for brevity. Evolutionary Searching. We search proxylessly in S2 on ImageNet. The evolution covered 8400 models (a popu…

**Figure 6.** Figure 6: Ranking analysis is based on a subspace of NAS-Bench-101. (a) A cell is a stack of 5 nodes. An additional 1 × 1 conv projection is added before the first one to avoid channel mismatch. (b) For each node, we can select one operation from 3 choices. 40 60 80 Ground Truth…

**Figure 7.** Figure 7: Comparison of ground-truth vs. estimated accuracy correlation between the supernets trained with and without ELS, based on 100 sampled models from NASBench-101 [33]. ELS substantially boosts the ranking ability of the supernet. We train such a supernet with and withou…

**Figure 8.** Figure 8: Adding ReLU to ELS gives a wrong evaluation (overestimation) of a subnetwork [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

**Figure 9.** Figure 9: Ablation study on constrained optimization. Left: Pareto front of MultAdds vs. Accuracy. Right: The ratios of skip connections per epoch. 6.5 Component Analysis A supernet trained without ELS can’t deliver good search results. Using the same searching strategy NSGA-II,…

**Figure 10.** Figure 10: Constrained and weighted NSGA-II Pipeline. It starts with a uniform initialization (top left) with some constraints (red) to generate the initial population. The trained scalable supernet serves as a fast evaluator to decide the relative performance of each model so …

**Figure 11.** Figure 11: Learned low-level and high-level features for the supernet with and without ELS [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 36 canonical work pages

[1]

In: ICML

Bender, G., Kindermans, P.J., Zoph, B., Vasudevan, V., Le, Q.: Understanding and Simplifying One-Shot Architecture Search. In: ICML. pp. 549–558 (2018)

work page 2018
[2]

In: ICLR (2019)

Cai, H., Zhu, L., Han, S.: ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In: ICLR (2019)

work page 2019
[3]

In: ICCV (2019)

Chen, X., Xie, L., Wu, J., Tian, Q.: Progressive Diﬀerentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation. In: ICCV (2019)

work page 2019
[4]

In: CVPR

Chollet, F.: Xception: Deep Learning with Depthwise Separable Convolutions. In: CVPR. pp. 1251–1258 (2017)

work page 2017
[5]

In: ICASSP (2020)

Chu, X., Zhang, B., Xu, R.: MoGA: Searching Beyond MobileNetV3. In: ICASSP (2020)

work page 2020
[6]

arXiv preprint

Chu, X., Zhang, B., Xu, R., Li, J.: FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search. arXiv preprint. arXiv:1907.01845 (2019)

arXiv 2019
[7]

In: CVPR (2019)

Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learn- ing Augmentation Policies from Data. In: CVPR (2019)

work page 2019
[8]

IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002)

Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002)

work page 2002

Show all 38 references

[9]

In: CVPR

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large- Scale Hierarchical Image Database. In: CVPR. pp. 248–255. Ieee (2009)

2009
[10]

In: CVPR

Dong, X., Yang, Y.: Searching for a Robust Neural Architecture in Four GPU Hours. In: CVPR. pp. 1761–1770 (2019)

2019
[11]

In: AJCAI

Friedrich, T., Kroeger, T., Neumann, F.: Weighted Preferences in Evolutionary Multi-Objective Optimization. In: AJCAI. pp. 291–300. Springer (2011)

2011
[12]

arXiv preprint

Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., Sun, J.: Single Path One-Shot Neural Architecture Search with Uniform Sampling. arXiv preprint. arXiv:1904.00420 (2019)

2019 arXiv
[13]

In: CVPR

He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: CVPR. pp. 770–778 (2016)

2016
[14]

In: ICCV (2019)

Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al.: Searching for MobileNetV3. In: ICCV (2019)

2019
[15]

In: CVPR

Hu, J., Shen, L., Sun, G.: Squeeze-and-Excitation Networks. In: CVPR. pp. 7132– 7141 (2018)

2018
[16]

In: CVPR

Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely Connected Convolutional Networks. In: CVPR. pp. 4700–4708 (2017)

2017
[17]

Biometrika 30(1/2), 81–93 (1938)

Kendall, M.G.: A New Measure of Rank Correlation. Biometrika 30(1/2), 81–93 (1938)

1938
[18]

Kornblith, S., Shlens, J., Le, Q.V.: Do Better Imagenet Models Transfer Better? In: CVPR. pp. 2661–2671 (2019)

2019
[19]

Krizhevsky, A., Hinton, G., et al.: Learning Multiple Layers of Features from Tiny Images. Tech. rep., Citeseer (2009)

2009
[20]

In: ICCV (2017)

Lin, T.Y., Goyal, P., Girshick, R., He, K., Doll´ ar, P.: Focal Loss for Dense Object Detection. In: ICCV (2017)

2017
[21]

In: ECCV

Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ ar, P., Zitnick, C.L.: Microsoft COCO: Common Objects in Context. In: ECCV. pp. 740–755. Springer (2014)

2014
[22]

In: ICLR (2019) 16 Chu et al

Liu, H., Simonyan, K., Yang, Y.: DARTS: Diﬀerentiable Architecture Search. In: ICLR (2019) 16 Chu et al

2019
[23]

In: ICML (2018)

Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Eﬃcient Neural Architecture Search via Parameter Sharing. In: ICML (2018)

2018
[24]

In: CVPR

Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: In- verted Residuals and Linear Bottlenecks. In: CVPR. pp. 4510–4520 (2018)

2018
[25]

Overﬁtting

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A Simple Way to Prevent Neural Networks from. Overﬁtting. JMLR 15(1), 1929–1958 (2014)

2014
[26]

In: ECMLPKDD (2019)

Stamoulis, D., Ding, R., Wang, D., Lymberopoulos, D., Priyantha, B., Liu, J., Marculescu, D.: Single-Path NAS: Designing Hardware-Eﬃcient ConvNets in less than 4 Hours. In: ECMLPKDD (2019)

2019
[27]

In: AAAI (2017)

Szegedy, C., Ioﬀe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. In: AAAI (2017)

2017
[28]

In: CVPR (2019)

Tan, M., Chen, B., Pang, R., Vasudevan, V., Le, Q.V.: Mnasnet: Platform-Aware Neural Architecture Search for Mobile. In: CVPR (2019)

2019
[29]

In: ICML (2019)

Tan, M., Le, Q.V.: EﬃcientNet: Rethinking Model Scaling for Convolutional Neural Networks. In: ICML (2019)

2019
[30]

BMVC (2019)

Tan, M., Le., Q.V.: MixConv: Mixed Depthwise Convolutional Kernels. BMVC (2019)

2019
[31]

CVPR (2019)

Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., Jia, Y., Keutzer, K.: FBNet: Hardware-Aware Eﬃcient ConvNet Design via Diﬀerentiable Neural Architecture Search. CVPR (2019)

2019
[32]

In: CVPR

Xie, S., Girshick, R., Doll´ ar, P., Tu, Z., He, K.: Aggregated Residual Transforma- tions for Deep Neural Networks. In: CVPR. pp. 1492–1500 (2017)

2017
[33]

In: ICML

Ying, C., Klein, A., Christiansen, E., Real, E., Murphy, K., Hutter, F.: Nas-bench- 101: Towards reproducible neural architecture search. In: ICML. pp. 7105–7114 (2019)

2019
[34]

In: ICLR (2020)

Zela, A., Elsken, T., Saikia, T., Marrakchi, Y., Brox, T., Hutter, F.: Understanding and Robustifying Diﬀerentiable Architecture Search. In: ICLR (2020)

2020
[35]

In: CVPR (June 2018)

Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuﬄeNet: An Extremely Eﬃcient Convo- lutional Neural Network for Mobile Devices. In: CVPR (June 2018)

2018
[36]

In: CVPR

Zhang, X., Li, Z., Change Loy, C., Lin, D.: PolyNet: A Pursuit of Structural Di- versity in Very Deep Networks. In: CVPR. pp. 718–726 (2017)

2017
[37]

In: CVPR

Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning Transferable Architectures for Scalable Image Recognition. In: CVPR. pp. 8697–8710 (2018) SCARLET-NAS 17 A Proof Proof. First, we prove that Equation 3 (main text) holds for∀o∈{ 0, 1,...,n−2}. In this case, it’s suﬃcient ...

2018
[38]

In particular, S1 is represented as a block-level supernet with L = 19 layers of N = 7 choices each

is adopted as its backbone. In particular, S1 is represented as a block-level supernet with L = 19 layers of N = 7 choices each. Its total size is 7 19. The choices are, – MobileNetV2’s inverted bottleneck blocks [24] of two expansion rates ( x) in (3,6), three kernel sizes (y...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

In: ICML

Bender, G., Kindermans, P.J., Zoph, B., Vasudevan, V., Le, Q.: Understanding and Simplifying One-Shot Architecture Search. In: ICML. pp. 549–558 (2018)

work page 2018

[2] [2]

In: ICLR (2019)

Cai, H., Zhu, L., Han, S.: ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In: ICLR (2019)

work page 2019

[3] [3]

In: ICCV (2019)

Chen, X., Xie, L., Wu, J., Tian, Q.: Progressive Diﬀerentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation. In: ICCV (2019)

work page 2019

[4] [4]

In: CVPR

Chollet, F.: Xception: Deep Learning with Depthwise Separable Convolutions. In: CVPR. pp. 1251–1258 (2017)

work page 2017

[5] [5]

In: ICASSP (2020)

Chu, X., Zhang, B., Xu, R.: MoGA: Searching Beyond MobileNetV3. In: ICASSP (2020)

work page 2020

[6] [6]

arXiv preprint

Chu, X., Zhang, B., Xu, R., Li, J.: FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search. arXiv preprint. arXiv:1907.01845 (2019)

arXiv 2019

[7] [7]

In: CVPR (2019)

Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learn- ing Augmentation Policies from Data. In: CVPR (2019)

work page 2019

[8] [8]

IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002)

Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002)

work page 2002