{"id":"a1ac3f3a-88d3-47a5-a7a6-e00292c48dc2","arxiv_id":"2501.12881","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"An RL agent (DDQN) reads landscape features of a black-box problem and outputs a full DE configuration, tested on six BBOB2009 functions.","lead":"This paper trains a reinforcement learning agent to design a differential evolution algorithm for each black-box optimization problem, choosing initialization, mutation, crossover, and hyperparameters from problem landscape features. A smart generalist might read it as a step toward automated algorithm design, where optimization software configures itself per problem.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported zero standard deviation across 31 stochastic runs and the AEI formula's division by sigma make the headline empirical claim (AEI 18.00, rank first) internally inconsistent and unsupported.","rationale":"The reader's weakest_assumption focused on whether 62 ELA features computed from a random initial sample capture enough problem structure. That is a plausible concern about generalization, but it is a design risk rather than a demonstrated flaw: the paper's experiments could still support the central claim even if the features are imperfect. The zero-standard-deviation anomaly is a concrete internal inconsistency: for a stochastic algorithm with random initialization and mutation, independent runs cannot all produce identical floating-point outputs. This makes the reported variance and the Wilcoxon significance tests meaningless. Moreover, the AEI formula in Eq. (10) divides by sigma_*, so with sigma_* = 0 the AEI 18.00 is mathematically undefined. This directly undermines the strongest claim that rlDE 'ranks first' on AEI. The reader mentioned these anomalies in their rationale but did not select them as the weakest assumption; thus agreement is partial. A REJECT verdict is appropriate because the central empirical claim is not merely unproven but internally inconsistent, and the authors provide no code or data to resolve the discrepancy. The framework idea may still be promising, but the paper as written does not support its headline result without correction or release of reproducible code.","tokens_in":17155,"tokens_out":4169,"duration_ms":44717,"concrete_test":"Obtain the authors' code (or request it) and run rlDE on the six testing functions with 31 distinct random seeds, recording per-run best objective values. Compute per-function mean and standard deviation; if any standard deviation is nonzero, the reported zeros in Table III are erroneous. If all standard deviations remain exactly zero, inspect the seeding policy and verify whether the 31 runs are actually identical; if so, the runs are not independent and the Wilcoxon tests are void. Then recompute AEI using Eq. (10) with the actual sigma_* values; if any sigma_* equals zero, the AEI score cannot be computed, and the reported 18.00 is unsupported. Also compare the recomputed AEI with the paper's value to assess reproducibility.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim that rlDE achieves the best aggregated indicator (AEI 18.00, rank first) rests entirely on the experiment tables, yet Table III reports identical worst/best/median/mean values with zero standard deviation for every testing function. Because rlDE uses random initialization (Table II), stochastic mutation/crossover operators, and epsilon-greedy action selection, independent runs should exhibit nonzero variance; identical floating-point outcomes across 31 runs is not credible unless the runs share the same random seed. If they share a seed, the 31 runs are not independent, so the Wilcoxon rank-sum tests in Table IV are invalid. If they do not share a seed, the zero-variance results likely indicate a reporting or implementation error. Additionally, Eq. (10) defines Z_k^* = (1/N) sum_n v_{k,n}^* / sigma_*, dividing by the per-function standard deviation; when sigma_* = 0, as reported for rlDE on all six functions, AEI is undefined. The paper explicitly acknowledges zero standard deviations but calls this 'highly stable' rather than addressing the statistical breakdown. No code or data is provided to resolve this. The ELA-feature concern raised by the reader is a legitimate design risk, but the zero-variance inconsistency is more load-bearing because it directly invalidates the quantitative comparison that supports the paper's strongest claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces rlDE, a reinforcement-learning framework that uses exploratory landscape analysis (ELA) features as states and a double deep Q-network (DDQN) to automatically select initialization, mutation, crossover, and control parameters for differential evolution (DE). The agent is trained offline on a subset of BBOB2009 problems and then generates a tailored DE configuration for unseen problems. The main empirical claim is that rlDE outperforms four RL-assisted DE variants and four traditional DE variants on six BBOB2009 test functions, achieving the best aggregated evaluation indicator (AEI 18.00) and exhibiting zero standard deviation across 31 runs. The paper also discusses the motivation for using problem characteristics instead of population characteristics and compares the framework with hyper-heuristics and self-adaptive methods.","tokens_in":17430,"tokens_out":2626,"duration_ms":29788,"significance":"The idea of replacing per-generation adaptive operator selection with one-shot, problem-characteristic-based DE design is timely and aligns with the growing interest in automated algorithm design and meta-learning for black-box optimization. The paper's conceptual separation of the learning episode from the using episode, and its use of ELA features as state representations, are reasonable design choices that could inspire follow-up work. However, the headline empirical result—that rlDE ranks first with AEI 18.00—is not supported by the reported data: the zero-variance results in Table III are implausible for a stochastic algorithm, the AEI formula in Eq. (10) becomes undefined when the standard deviation is zero, and the claimed design-space size in the abstract is inconsistent with Table II. These issues are load-bearing because the paper's central claim rests on the quantitative comparison. If the experiments are rerun with proper independent seeded runs and the statistical analysis is repaired, the framework may still be a valuable proof-of-concept; as presented, the evidence for superiority is not credible.","major_comments":[{"comment":"Table III reports identical worst, best, median, and mean values with standard deviation 0.0 for all six BBOB2009 test functions across 31 runs. Since rlDE uses random initialization (Table II), stochastic mutation and crossover operators, and epsilon-greedy action selection, independent runs must exhibit nonzero variance unless the runs share a random seed. If they share a seed, the 31 runs are not independent and the Wilcoxon rank-sum tests in Table IV are invalid. If they do not share a seed, the reported zero variance indicates a reporting or implementation error. The manuscript explicitly notes this lack of variability and calls it 'highly stable,' but it does not address the statistical breakdown. This directly invalidates the quantitative comparison that supports the paper's strongest claim.","section":"V-B, Table III"},{"comment":"The AEI calculation uses Z-score normalization Z_k* = (1/N) sum_n v_{k,n}^* / sigma_*, dividing by the per-function standard deviation over repeated runs. For rlDE, Table III reports sigma_* = 0 on every testing function, so the Z scores and hence the AEI value of 18.00 are undefined under the formula as written. The paper gives no alternative handling for zero variance, and the displayed AEI ranking therefore lacks a rigorous basis.","section":"V-B, Eq. (10)"},{"comment":"The abstract states that the proof-of-concept considers a subset of 40 possible strategy combinations and parameter optimizations, but Table II defines 5 initialization strategies x 10 mutation strategies x 2 crossover strategies x 5 population sizes x 41 scale factors x 21 crossover rates = 430,500 combinations. Moreover, Section IV-C describes an output layer with only 5 neurons, which cannot encode that design space directly. The discrepancy between the stated action space and the architecture description makes it unclear what the agent actually outputs and how the 40-combination subset (or the full space) is represented. This needs clarification and a consistent action-encoding description.","section":"Abstract and IV-C, Table II"},{"comment":"The state s_t is computed only from ELA features on the initial random sample of points, and the paper does not provide a concrete test of whether these static features are sufficient to determine the best DE configuration. The authors themselves note in the conclusion that the RL agent's performance depends heavily on the breadth and representativeness of training problems, but they do not present an ablation or a baseline comparison (e.g., an agent using a fixed random state, or a version without ELA). Since the state representation is the core of the claimed mapping from problem characteristics to algorithm design, an experimental comparison is needed to substantiate that the ELA features, rather than the training distribution or reward scale, drive the reported performance.","section":"IV-C, Algorithm 5 and Section III"}],"minor_comments":[{"comment":"There are multiple typos in this section: 'Curvatyre' should be 'Curvature,' 'Generakized' should be 'Generalized,' 'Disperison' should be 'Dispersion,' and 'meta-modal' should be 'meta-model.' Also, the unresolved reference 'Section ??' in Section IV-C should be fixed.","section":"Section III"},{"comment":"The table caption and the accompanying text use 'rom' instead of 'from' and 'llustrates' instead of 'illustrates' in Figure 4. The notation 'vavg(vstd)' is inconsistent with the column headers and the main text; please unify the notation for the average and standard deviation.","section":"Section V-B, Table IV"},{"comment":"Figure 1 is described as showing the overall framework, but the text does not explain the roles of the three color-coded starting points or how information flows between the RL process and the DE process. Adding a short walkthrough of the figure would improve readability.","section":"Section IV-B, Figure 1"},{"comment":"The table lists DEDQN and DEDDQN as related work, but the main text does not clearly differentiate the proposed MADQN from these prior DQN-based methods beyond the multi-object action architecture. A sentence clarifying the novelty in action representation would help position the contribution.","section":"Section II-C, Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper proposes an interesting framework, but the experimental section has a credibility problem that cannot be fixed by local edits: the zero-standard-deviation results and the undefined AEI calculation call into question whether the reported experiments were conducted as described. I would encourage the editor to require the authors to provide the exact experimental setup, seeds, and code, and to rerun the comparison with statistically valid independent trials. The manuscript also appears to overstate the size of the design space versus the abstract; this should be checked for consistency. If the authors can supply reproducible experiments and repair the statistical analysis, the paper may be suitable for publication, but I cannot recommend acceptance on the current evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea in this paper is genuinely worth knowing. The authors use exploratory landscape analysis (ELA) to extract 62 features from a random sample of a black-box problem, feed those features as the state to a double DQN, and decode the action into a complete DE configuration — initialization, mutation, crossover, population size, F, and Cr — before evolution starts. That is a real departure from DEDQN, LDE, RL-HPSDE, and the rest of the RL-assisted DE line, which adapt operators or parameters online during the search. The framework is a sensible assembly of known components, and the paper positions it honestly as a proof-of-concept.\n\nThe problem is the empirical section. Table III reports identical worst, best, median, and mean values with zero standard deviation for all six testing functions across 31 runs. rlDE uses random initialization, stochastic mutation and crossover, and epsilon-greedy action selection. Independent runs producing bit-identical floating-point results is not credible unless every run used the same seed; if they did, the runs are dependent and the Wilcoxon tests in Table IV are invalid. If they did not, there is an implementation or reporting error. The paper acknowledges the zeros and calls the algorithm 'highly stable,' which is not a statistical argument.\n\nThe AEI formula in Eq. (10) divides by the per-function standard deviation. With sigma = 0 for all six functions, AEI is undefined. Yet the paper reports an AEI of 18.00 for rlDE and uses that to claim rank first. That is load-bearing: the headline performance claim rests on numbers that cannot be reproduced from the given equations.\n\nThere are also smaller inconsistencies. The abstract says 40 possible strategy combinations; Table II defines a design space of 430,500 combinations, and the output layer has 5 neurons, which does not obviously decode to that space. No code or data is provided, and the reward function is an unnormalized e^{-f*_t}. The ELA-feature question — whether static features from one sample suffice to predict the best DE — is a real design risk, but it is secondary to the zero-variance breakdown.\n\nI want to give credit where it is due. The writing is clear, the related work is adequate, and the conclusion acknowledges the predefined design space and dependence on training problems. This is a reasonable proof-of-concept, but the results as reported cannot be accepted.\n\nRecommendation: this deserves peer review rather than desk rejection because the idea is new and testable. But it should be returned for major revision: release code and data, rerun independent seeds with proper variance reporting, fix the AEI calculation, reconcile the design-space numbers, and add a hyper-heuristic baseline. I would not cite it until those issues are resolved.","headline":"Offline, per-problem DE design via RL and ELA is genuinely new, but zero-variance results and an undefined AEI make the empirical headline unsupportable.","tokens_in":17932,"tokens_out":2935,"would_cite":false,"duration_ms":28946,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An RL agent can auto-design differential evolution for unseen black-box problems.","keywords":["differential evolution","reinforcement learning","black-box optimization","exploratory landscape analysis","meta-learning","deep Q-network","algorithm design","BBOB2009"],"falsifier":"A decisive check is to grid-search the same design space on each of the 24 noiseless BBOB2009 functions and compare the best per-function configuration with the one the agent chooses from the 62-feature state; if functions that are close in feature space have very different best configurations, or if the agent's choices are consistently worse than the oracle, the state representation is insufficient to support the claimed mapping.","tokens_in":16937,"feed_emoji":"🧬","tokens_out":6919,"duration_ms":64325,"temperature":0.7,"pith_summary":"This paper proposes a framework, rlDE, in which a reinforcement-learning agent learns, offline, to assemble a complete differential evolution algorithm for a given black-box optimization problem. The agent reads a problem through 62 exploratory landscape analysis features—cheap statistics computed once from a single random sample—and outputs a configuration that includes an initialization strategy, a mutation strategy, a crossover strategy, and values for population size, scale factor, and crossover rate. The paper's claim is that this learned mapping, trained on a majority of the BBOB2009 noiseless functions, generalizes to unseen problems: on six held-out functions the resulting algorithm attains the best aggregated evaluation indicator (AEI 18.00) among all compared RL-assisted DE variants and is competitive with the best traditional DEs. If the claim holds, it would mean that algorithm design for black-box optimization can be automated by learning the relationship between cheap landscape statistics and effective algorithm configurations, rather than relying on expert knowledge or per-problem search.","feed_headline":"RL agent writes a custom optimizer for each black-box problem","feed_subtitle":"A reinforcement-learning agent reads cheap landscape features and selects a tailor-made differential evolution algorithm.","key_machinery":"The load-bearing mechanism is the multi-armed-bandit deep Q-network (MADQN), which extends DDQN so that a single state produces several Q-value vectors, one per design object (initialization, mutation, crossover, and control parameters), letting the agent choose multiple strategy types simultaneously. The state is provided by exploratory landscape analysis: 62 features computed from one random initial sample, covering y-distribution, levelset classification errors, linear-model fits, funnel-structure statistics, information content, dispersion, and PCA of the decision space, plus the problem dimension. The design space being searched is a subset of 40 strategy combinations, with population size among $\\{5D,7D,9D,11D,13D\\}$, scale factor in steps of $0.05$, and crossover rate in steps of $0.1$.","core_discovery":"The central discovery the paper argues for is that the relationship between a black-box problem's landscape and a well-performing differential evolution configuration can be meta-learned by a double deep Q-network. The agent is trained on triples of landscape-feature state, generated algorithm action, and observed performance reward, where the reward is simply $r_t = e^{-f_t^*}$ with $f_t^*$ the best objective value found. After training, the agent is frozen and applied to an unseen problem by computing its 62 features once and decoding the network's output into a full DE variant. On the six BBOB2009 test functions selected, the paper reports zero standard deviation across 31 runs for each function and an aggregated evaluation indicator of 18.00, ranking first among the compared algorithms; this is offered as evidence that the generated algorithms are stable and effective.","pith_inferences":["A direct test of the representation would be to compare the agent's chosen configuration against a per-problem oracle that grid-searches the same 40-combination design space; the gap between them measures how much information the 62 features actually carry.","The framework could be extended to a continuous design space, or to a reward that tracks entire convergence curves rather than the final objective, which might improve generalization when evaluation budgets vary.","Because the state is computed once at the start, the approach is insensitive to the search dynamics; a natural next step is to make part of the state update during evolution, though this would trade away the cost advantage the paper argues for."],"forward_implications":["For a previously unseen problem, rlDE produces a complete DE without any per-problem search, so the cost of algorithm design becomes one round of feature computation plus one network forward pass.","The learned agent outperforms RL-assisted DE variants that adapt operators during evolution, which suggests that offline design from problem features can be more effective than online adaptation from population statistics.","If the mapping generalizes beyond BBOB2009, the same meta-learning scheme could be applied to other evolutionary algorithms, since the framework only needs a parametrized algorithm family and a performance signal.","The reported zero variance across 31 runs implies the agent converges to a single deterministic configuration per test function, making the generated algorithms repeatable."],"supporting_citations":[{"why":"Supplies the deep Q-network architecture that MADQN extends with multi-armed-bandit outputs.","marker":"[20]"},{"why":"Provides the exploratory landscape analysis feature computation used to build the agent's state.","marker":"[40]"},{"why":"Supplies the pflacco package, the feature collection source from which the 62 cheap features are selected.","marker":"[47]"},{"why":"Supports the choice of which cheap ELA features are informative for characterizing problems.","marker":"[48]"},{"why":"Defines the BBOB2009 noiseless benchmark suite used for training and testing.","marker":"[59]"},{"why":"Provides the MetaBox platform, including the training/test split and the AEI aggregated evaluation indicator.","marker":"[60]"},{"why":"Defines differential evolution, the algorithm the framework is designed to assemble.","marker":"[24]"},{"why":"Serves as a compared RL-assisted DE baseline, representing the operator-selection approach.","marker":"[1]"}],"fun_headline_variants":["RL auto-designs differential evolution per problem","Reinforcement learning builds bespoke optimizers","Meta-learner selects DE strategy from landscape","Double DQN writes your optimization algorithm","Custom DE for black-box: RL does it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 62 static features, computed once from a single random sample, capture enough of a problem's structure to determine which differential evolution configuration will perform best.","fun_headline_variants_meta":{"raw":{"variants":["RL auto-designs differential evolution per problem","Reinforcement learning builds bespoke optimizers","Meta-learner selects DE strategy from landscape","Double DQN writes your optimization algorithm","Custom DE for black-box: RL does it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000477,"raw_usage":{"total_tokens":2339,"prompt_tokens":893,"completion_tokens":1446,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":1377}},"tokens_in":509,"tokens_out":1446,"duration_ms":10750,"temperature":1.0,"reasoning_tokens":1377,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:42:21.330638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check is to grid-search the same design space on each of the 24 noiseless BBOB2009 functions and compare the best per-function configuration with the one the agent chooses from the 62-feature state; if functions that are close in feature space have very different best configurations, or if the agent's choices are consistently worse than the oracle, the state representation is insufficient to support the claimed mapping.","supporting_citations":[{"cited_title":"Differential evolution with mixed mutation strategy based on deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Serves as a compared RL-assisted DE baseline, representing the operator-selection approach."},{"cited_title":"Human-level control through deep rein- forcement learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the deep Q-network architecture that MADQN extends with multi-armed-bandit outputs."},{"cited_title":"flaccogui: Exploratory landscape analysis for everyone,","cited_arxiv_id":null,"evidence_quote":"Provides the exploratory landscape analysis feature computation used to build the agent's state."},{"cited_title":"Pflacco: Feature-based landscape analysis of continuous and constrained optimization problems in python,","cited_arxiv_id":null,"evidence_quote":"Supplies the pflacco package, the feature collection source from which the 62 cheap features are selected."},{"cited_title":"Expressiveness and robustness of landscape features,","cited_arxiv_id":null,"evidence_quote":"Supports the choice of which cheap ELA features are informative for characterizing problems."},{"cited_title":"Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions,","cited_arxiv_id":null,"evidence_quote":"Defines the BBOB2009 noiseless benchmark suite used for training and testing."},{"cited_title":"Metabox: a benchmark platform for meta-black-box optimization with reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the MetaBox platform, including the training/test split and the AEI aggregated evaluation indicator."},{"cited_title":"Differential evolution–a simple and effi- cient heuristic for global optimization over continuous spaces,","cited_arxiv_id":null,"evidence_quote":"Defines differential evolution, the algorithm the framework is designed to assemble."}],"review_version":1}