The reviewed record of science sign in
Pith

arxiv: 2012.06400 · v2 · pith:3SAMT6U5 · submitted 2020-12-11 · cs.NE · cs.LG

Differential Evolution for Neural Architecture Search

Noor Awad , Neeratyoy Mallik , Frank Hutter This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:3SAMT6U5record.jsonopen to challenge →

classification cs.NE cs.LG
keywords searchstrategyevaluationsevolutionperformancearchitecturedifferentialfull
0
0 comments X
read the original abstract

Neural architecture search (NAS) methods rely on a search strategy for deciding which architectures to evaluate next and a performance estimation strategy for assessing their performance (e.g., using full evaluations, multi-fidelity evaluations, or the one-shot model). In this paper, we focus on the search strategy. We introduce the simple yet powerful evolutionary algorithm of differential evolution to the NAS community. Using the simplest performance evaluation strategy of full evaluations, we comprehensively compare this search strategy to regularized evolution and Bayesian optimization and demonstrate that it yields improved and more robust results for 13 tabular NAS benchmarks based on NAS-Bench-101, NAS-Bench-1Shot1, NAS-Bench-201 and NAS-HPO bench.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive

    cs.CL 2024-02 conditional novelty 6.0

    DPOP is a new loss function that prevents DPO from lowering preferred response likelihoods and outperforms standard DPO on diverse datasets, MT-Bench, and enables Smaug-72B to exceed 80% on the Open LLM Leaderboard.