pith. sign in

arxiv: 2605.20797 · v1 · pith:G3GIZLWInew · submitted 2026-05-20 · 💻 cs.LG

Beyond Numerical Features: CNN-Driven Algorithm Selection via Contour Plots for Continuous Black-Box Optimization

Yiliang Yuan , Xiang Shi , Mustafa Misir This is my paper

Pith reviewed 2026-05-21 06:16 UTC · model grok-4.3

classification 💻 cs.LG
keywords algorithm selectionblack-box optimizationcontour plotsconvolutional neural networkslandscape visualizationBBOB benchmarkper-instance selectionperformance prediction
0
0 comments X

The pith

Contour plots of probed landscapes let a CNN pick the best black-box optimizer from a portfolio.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes replacing handcrafted numerical landscape features with simple contour-map images for per-instance algorithm selection in continuous black-box optimization. A CNN regressor processes multiple views of these contour plots, either stacked or encoded and aggregated, to predict each solver's performance on a given instance and then selects the predicted winner. On the BBOB 2009 single-objective benchmark the image-based selectors beat the single best solver and match the accuracy of traditional feature-based methods. The same principle extends to a bi-objective setting using windowed contour views. The central idea is that spatial patterns visible in the plots carry enough information for accurate performance prediction without explicit Exploratory Landscape Analysis descriptors.

Core claim

The paper establishes that contour-map visualizations of probed optimization landscapes, when fed to a CNN regressor, enable competitive per-instance algorithm selection. The regressor predicts per-solver performance from multiple instance-specific contour views, and selection proceeds by choosing the solver with the best predicted value. On the standard BBOB 2009 single-objective protocol these selectors significantly outperform the single best solver and remain competitive with feature-based baselines; a follow-up bi-objective test under a DeepELA-style protocol shows similar competitiveness when windowed contour views are used.

What carries the argument

A CNN regressor that ingests stacked or aggregated contour-map visualizations of probed landscapes to predict per-solver performance values.

If this is right

  • Algorithm selection can be performed directly from visual probes of the landscape without computing any handcrafted numerical features.
  • Portfolio performance improves over the single best solver because the selector adapts to instance-specific visual patterns.
  • The same image-based pipeline remains competitive when extended from single-objective to bi-objective problems using windowed views.
  • Selection decisions become independent of the particular set of Exploratory Landscape Analysis features chosen in advance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If contour plots prove sufficient, future selectors could operate in real time by generating a small number of quick probes and rendering them as images.
  • The approach suggests that other vision models or even simpler image statistics might replace the CNN while preserving selection quality.
  • Instance representations could be shared across different optimization domains once they are expressed as standardized contour images rather than domain-specific numerical descriptors.

Load-bearing premise

Contour-map images of the landscapes contain enough spatial information for the CNN to accurately predict which solver will perform best on each instance.

What would settle it

A controlled test showing that replacing the contour plots with random images of the same size and color distribution produces selectors no better than random choice or the single best solver.

Figures

Figures reproduced from arXiv: 2605.20797 by Mustafa Misir, Xiang Shi, Yiliang Yuan.

Figure 1
Figure 1. Figure 1: A traditional performance prediction based Automated Algorithm Selection (AAS) process. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Contour plots for different problems. From left to right, the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the combined model. The stacked contour tensor is [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of the separate model. Each contour view is encoded [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: relERT distribution per algorithm across all function groups and dimensions. Algorithm 1 CNN-Based Algorithm Selection (Combined vs. Separate) Input: Contour-plot tensor X ∈ R k×r×r (SOO), dimen￾sion d, and k=5; portfolio A={a1, . . . , am}. Output: Selected algorithm a ∗ . Combined model: 1: z ← Φcnn(X) 2: P ←ˆ Ψ(z, d) ▷ P ∈ ˆ R m 3: a ∗ ← arg minα∈{1,...,m} Pˆ α ▷ SOO (relERT); use arg max for MOO (relHV… view at source ↗
read the original abstract

The present paper introduces a new representation-driven approach to per-instance algorithm selection, applied to black-box optimization, for automatically choosing the most promising solver from a fixed portfolio. Prior work in continuous optimization largely relies on numerical descriptors, including Exploratory Landscape Analysis features and learned embeddings such as Deep-ELA. This work studies a complementary representation: contour-map visualizations of probed landscapes. A CNN regressor takes multiple instance-specific contour views (stacked or encoded per view and aggregated) and predicts per-solver performance, enabling selection by the predicted best value. On the standard BBOB 2009 single-objective protocol, the resulting selectors significantly outperform the single best solver (SBS) and are competitive with feature-based baselines. A subsequent bi-objective evaluation under the DeepELA setting further indicates that the same image-based principle can be competitive when using windowed contour views. Overall, the results suggest that simple vision models can exploit spatial structure in probed landscapes for algorithm selection without handcrafted ELA features.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces a representation-driven approach to per-instance algorithm selection for continuous black-box optimization. It replaces numerical descriptors such as ELA features with contour-map visualizations of probed landscapes, which are fed (stacked or encoded and aggregated) into a CNN regressor to predict per-solver performance and select the best solver from a portfolio. On the BBOB 2009 single-objective protocol the resulting selectors are claimed to significantly outperform the single best solver and remain competitive with feature-based baselines; a follow-up bi-objective evaluation under the DeepELA setting is also reported.

Significance. If the empirical claims hold under scrutiny, the work demonstrates that standard vision models can exploit spatial structure in simple landscape visualizations for effective algorithm selection, offering a complementary route to handcrafted or learned numerical features and potentially reducing reliance on domain-specific feature engineering.

major comments (2)
  1. [§3] §3 (Landscape Visualization and CNN Input): the central claim that stacked/encoded contour views supply enough spatial information for the CNN to regress per-solver performance is load-bearing for the headline result of outperformance on BBOB 2009, yet the manuscript provides no quantification or ablation of information loss when 2-D slices or projections are used on instances up to 40 D; without this the competitiveness with numerical ELA baselines cannot be fully assessed.
  2. [§4] §4 (BBOB Experiments): the abstract and results section assert significant outperformance over SBS and competitiveness with baselines, but the text supplies no concrete numbers, error bars, statistical tests, or details on data splits and training protocol; these omissions prevent verification of the reported gains.
minor comments (2)
  1. [Abstract] Abstract: the statement 'significantly outperform' should be accompanied by a reference to the specific table or figure that reports the quantitative results.
  2. [§3] Notation: the description of how multiple contour views are 'stacked or encoded per view and aggregated' would benefit from an explicit diagram or pseudocode to clarify the input tensor construction.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address the major comments point by point below and have revised the paper accordingly to improve clarity and completeness.

read point-by-point responses

  1. Referee: [§3] §3 (Landscape Visualization and CNN Input): the central claim that stacked/encoded contour views supply enough spatial information for the CNN to regress per-solver performance is load-bearing for the headline result of outperformance on BBOB 2009, yet the manuscript provides no quantification or ablation of information loss when 2-D slices or projections are used on instances up to 40 D; without this the competitiveness with numerical ELA baselines cannot be fully assessed.

    Authors: We agree that an explicit quantification or ablation of information loss from 2D contour projections in dimensions up to 40D would strengthen the assessment of the approach relative to numerical ELA baselines. The original manuscript emphasizes empirical competitiveness on BBOB instances, but does not include a dedicated ablation on this aspect. We will add such an analysis in the revised version, for example by comparing CNN performance on contour inputs against variants that incorporate additional dimensionality-aware processing or by reporting performance degradation trends across dimensions. revision: yes

  2. Referee: [§4] §4 (BBOB Experiments): the abstract and results section assert significant outperformance over SBS and competitiveness with baselines, but the text supplies no concrete numbers, error bars, statistical tests, or details on data splits and training protocol; these omissions prevent verification of the reported gains.

    Authors: We acknowledge that the main text could more explicitly present the supporting numerical results, error bars, statistical tests, and protocol details to facilitate verification. The manuscript reports results via tables and figures that include performance metrics with variability indicators and uses statistical comparisons (such as Wilcoxon signed-rank tests) to substantiate claims of outperformance over the single best solver. To address the concern, we will expand the results section in the revision to include specific numerical values, explicit descriptions of the data splits (e.g., instance-wise cross-validation), training protocol, and direct references to the statistical outcomes. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical pipeline is self-contained

full rationale

The paper presents an empirical machine-learning pipeline: contour-map visualizations of probed BBOB landscapes are fed to a CNN regressor that predicts per-solver performance, after which selection is performed by argmax over the predictions. All performance claims are obtained by training on one set of instances and evaluating on held-out BBOB 2009 single-objective and bi-objective test suites, with direct comparison to the single-best solver and to numerical ELA/Deep-ELA baselines. No derivation, uniqueness theorem, or fitted parameter is invoked that reduces by construction to the target quantity; the method does not rely on self-citations for its central premise and is externally falsifiable on the public benchmark.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The approach rests on standard assumptions that CNNs can extract useful patterns from contour images and that BBOB benchmark instances are representative of real optimization problems; no ad-hoc free parameters or new entities are introduced beyond conventional CNN training.

axioms (2)
  • domain assumption Convolutional neural networks can learn meaningful spatial features from contour plot images of optimization landscapes.
    Implicit in the choice of CNN regressor on stacked or encoded contour views.
  • domain assumption The BBOB 2009 single-objective protocol provides a fair and representative testbed for comparing algorithm selectors.
    Invoked when reporting outperformance on this benchmark.

pith-pipeline@v0.9.0 · 5702 in / 1357 out tokens · 37899 ms · 2026-05-21T06:16:33.456596+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 24 canonical work pages

  1. [1]

    Bbob in- stance analysis: Landscape properties and algorithm performance across problem instances,

    F. X. Long, D. Vermetten, B. van Stein, and A. V . Kononova, “Bbob in- stance analysis: Landscape properties and algorithm performance across problem instances,” inInternational Conference on the Applications of Evolutionary Computation (Part of EvoStar). Springer, 2023, pp. 380– 395

    work page 2023

  2. [2]

    No free lunch theorems for optimization,

    D. Wolpert and W. Macready, “No free lunch theorems for optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 67– 82, 1997

    work page 1997

  3. [3]

    Automated algorithm selection: Survey and perspectives,

    P. Kerschke, H. H. Hoos, F. Neumann, and H. Trautmann, “Automated algorithm selection: Survey and perspectives,”Evolutionary computa- tion, vol. 27, no. 1, pp. 3–45, 2019

    work page 2019

  4. [4]

    Auto- mated algorithm selection in single-objective continuous optimization: a comparative study of deep learning and landscape analysis methods,

    R. P. Prager, M. V . Seiler, H. Trautmann, and P. Kerschke, “Auto- mated algorithm selection in single-objective continuous optimization: a comparative study of deep learning and landscape analysis methods,” inInternational Conference on Parallel Problem Solving from Nature (PPSN). Springer, 2022, pp. 3–17

    work page 2022

  5. [5]

    Automated algorithm selection on continuous black-box problems by combining exploratory landscape analysis and machine learning,

    P. Kerschke and H. Trautmann, “Automated algorithm selection on continuous black-box problems by combining exploratory landscape analysis and machine learning,”Evolutionary Computation, vol. 27, no. 1, pp. 99–127, 2019

    work page 2019

  6. [6]

    Deep-ela: Deep ex- ploratory landscape analysis with self-supervised pretrained transformers for single-and multi-objective continuous optimization problems,

    M. V . Seiler, P. Kerschke, and H. Trautmann, “Deep-ela: Deep ex- ploratory landscape analysis with self-supervised pretrained transformers for single-and multi-objective continuous optimization problems,”Evo- lutionary Computation, pp. 1–27, 2025

    work page 2025

  7. [7]

    Deep learn- ing for algorithm portfolios,

    A. Loreggia, Y . Malitsky, H. Samulowitz, and V . Saraswat, “Deep learn- ing for algorithm portfolios,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1, 2016

    work page 2016

  8. [8]

    Real-parameter black- box optimization benchmarking 2009: Noiseless functions definitions,

    S. Finck, N. Hansen, R. Ros, and A. Auger, “Real-parameter black- box optimization benchmarking 2009: Noiseless functions definitions,” INRIA, Tech. Rep. RR-6829, 2009, updated version as of February 2019. [Online]. Available: https://inria.hal.science/inria-00362633v2/document

    work page 2009

  9. [9]

    The complexity of constrained min-max optimization,

    C. Daskalakis, S. Skoulakis, and M. Zampetakis, “The complexity of constrained min-max optimization,” inProceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, 2021, pp. 1466– 1478

    work page 2021

  10. [10]

    Automated algorithm selection: from feature-based to feature-free approaches,

    M. Alissa, K. Sim, and E. Hart, “Automated algorithm selection: from feature-based to feature-free approaches,”Journal of Heuristics, vol. 29, no. 1, pp. 1–38, 2023

    work page 2023

  11. [11]

    Learned features vs. classical ela on affine bbob functions,

    M. Seiler, U. ˇSkvorc, G. Cenikj, C. Doerr, and H. Trautmann, “Learned features vs. classical ela on affine bbob functions,” inInternational Conference on Parallel Problem Solving from Nature (PPSN). Springer, 2024, pp. 137–153

    work page 2024

  12. [12]

    COCO: A platform for comparing continuous optimizers in a black- box setting,

    N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tu ˇsar, and D. Brockhoff, “COCO: A platform for comparing continuous optimizers in a black- box setting,”Optimization Methods and Software, vol. 36, pp. 114–144, 2021

    work page 2021

  13. [13]

    On the potential of automated algorithm configuration on multi-modal multi-objective optimization problems,

    J. Rook, H. Trautmann, J. Bossek, and C. Grimme, “On the potential of automated algorithm configuration on multi-modal multi-objective optimization problems,” inProceedings of the Genetic and Evolutionary Computation Conference Companion, 2022, pp. 356–359

    work page 2022

  14. [14]

    Comparison of multiobjective evolutionary algorithms: Empirical results,

    E. Zitzler, K. Deb, and L. Thiele, “Comparison of multiobjective evolutionary algorithms: Empirical results,”Evolutionary computation, vol. 8, no. 2, pp. 173–195, 2000

    work page 2000

  15. [15]

    Scalable test problems for evolutionary multiobjective optimization,

    K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, “Scalable test problems for evolutionary multiobjective optimization,” inEvolutionary multiob- jective optimization: theoretical advances and applications. Springer, 2005, pp. 105–145

    work page 2005

  16. [16]

    A novel scalable test problem suite for multimodal multiobjective optimization,

    C. Yue, B. Qu, K. Yu, J. Liang, and X. Li, “A novel scalable test problem suite for multimodal multiobjective optimization,”Swarm and Evolutionary Computation, vol. 48, pp. 62–71, 2019

    work page 2019

  17. [17]

    Using well- understood single-objective functions in multiobjective black-box op- timization test suites,

    D. Brockhoff, A. Auger, N. Hansen, and T. Tu ˇsar, “Using well- understood single-objective functions in multiobjective black-box op- timization test suites,”Evolutionary computation, vol. 30, no. 2, pp. 165–193, 2022

    work page 2022

  18. [18]

    A fast and elitist multiobjective genetic algorithm: Nsga-ii,

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii,”IEEE transactions on evolu- tionary computation, vol. 6, no. 2, pp. 182–197, 2002

    work page 2002

  19. [19]

    Sms-emoa: Multiobjective selection based on dominated hypervolume,

    N. Beume, B. Naujoks, and M. Emmerich, “Sms-emoa: Multiobjective selection based on dominated hypervolume,”European journal of oper- ational research, vol. 181, no. 3, pp. 1653–1669, 2007

    work page 2007

  20. [20]

    Moea/d: A multiobjective evolutionary algorithm based on decomposition,

    Q. Zhang and H. Li, “Moea/d: A multiobjective evolutionary algorithm based on decomposition,”IEEE Transactions on evolutionary computa- tion, vol. 11, no. 6, pp. 712–731, 2007

    work page 2007

  21. [21]

    Omni-optimizer: A procedure for single and multi-objective optimization,

    K. Deb and S. Tiwari, “Omni-optimizer: A procedure for single and multi-objective optimization,” inInternational conference on evolution- ary multi-criterion optimization. Springer, 2005, pp. 47–61

    work page 2005

  22. [22]

    An r package implementing the multi-objective land- scape explorer (mole),

    L. Sch ¨apermeier, “An r package implementing the multi-objective land- scape explorer (mole),” 2022

    work page 2022

  23. [23]

    Multimodality in multi- objective optimization–more boon than bane?

    C. Grimme, P. Kerschke, and H. Trautmann, “Multimodality in multi- objective optimization–more boon than bane?” inInternational Confer- ence on Evolutionary Multi-Criterion Optimization. Springer, 2019, pp. 126–138

    work page 2019

  24. [24]

    Hypervolume indicator gradient ascent multi-objective optimization,

    H. Wang, A. Deutz, T. B ¨ack, and M. Emmerich, “Hypervolume indicator gradient ascent multi-objective optimization,” inInternational confer- ence on evolutionary multi-criterion optimization. Springer, 2017, pp. 654–669

    work page 2017