QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution

Lili Pan; Renjun Xu; Yu Lu; Zhanchao Zhou; Zhenzhong Lan; Zijie Chen

REVIEW 3 major objections 7 minor 4 cited by

QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that QUBE, by scoring clusters of programs with a quality-plus-uncertainty criterion, finds better heuristics than FunSearch and EoH on online bin packing, TSP, and cap set search.

desk verdict A useful incremental extension of FunSearch whose central quality estimate is statistically under-supported, but the empirical story is honest enough to warrant a serious look. read the letter →

arxiv 2412.20694 v4 pith:77LTG2PQ submitted 2024-12-30 cs.NE cs.AIcs.CL

Zijie Chen , Zhanchao Zhou , Yu Lu , Renjun Xu , Lili Pan , Zhenzhong Lan This is my paper · ORCID

classification cs.NEcs.AIcs.CL

keywords heuristicdesignlargelanguagemodelsevolutionaryalgorithmsexploration-exploitationtrade-offupperconfidenceboundonlinebinpackingtravelingsalesmanproblemcapset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that FunSearch, an LLM-based evolutionary search for heuristics, is held back by ranking candidate programs only by their own score. QUBE replaces that priority with a quality-uncertainty criterion: each cluster of behaviorally equivalent programs is judged by the average score of the offspring it has produced, plus an uncertainty bonus that declines as the cluster is used more often. On four OR-Library bin packing datasets, the best QUBE heuristics have excess ratios 9.36% to 41.73% lower than the paper's FunSearch reproduction; on TSP100 the excess is 0.023% versus 0.029%; on cap set n=8 it finds a set of size 480 versus 464. The paper is explicit that it did not beat the original FunSearch's published cap set result, only its own reproduction.

What carries the argument

Quality-Uncertainty Trade-off Criterion (QUTC), built on Uncertainty-Inclusive Quality (UIQ): $$\tilde{Q}_t(C)=Q_t(C)+k\sqrt{\ln t/N_t(C)}.$$ Clusters group programs that behave identically on all test instances; Q_t(C) is the mean score of offspring generated by that cluster's parents. The uncertainty term makes unexplored clusters competitive, so the criterion balances exploitation and exploration, and top-UIQ clusters determine both parent selection and island reset.

What would settle it

A controlled comparison that changes only the credit assignment—for example, running QUBE with single-parent prompts, or with offspring credited only to the higher-scoring parent, on the OR3 bin packing benchmark—would settle whether UIQ's gains come from exploration or from the two-parent attribution. If the advantage over raw-score selection disappears under one-parent credit, the reported result rests on credit assignment rather than the uncertainty bonus.

Watch

Extended reading notes

Core claim

The central discovery is that a UCB-style priority criterion, not the LLM generator, is what limits FunSearch's efficiency. QUBE computes, for each cluster C, the mean score Q_t(C) of all samples generated using C's programs as parents, and adds k sqrt(ln t/N_t(C)) to favor rarely-used clusters. Parent selection and island reset then use this UIQ instead of raw sample score. The authors report that this change yields consistently lower excess ratios on OBP, better TSP solutions, and a larger cap set than their FunSearch reproduction, and that the gain persists when the backend LLM is swapped from OpenCoder-8B to Deepseek-coder-6.7B.

Load-bearing premise

The load-bearing premise is that a cluster's evolutionary quality is correctly measured by Eq. (1), which gives every offspring's score to each of its two parent clusters; the paper does not test this attribution against a one-parent or interaction-aware model. If the two parents interact in the LLM prompt, UIQ mis-ranks clusters and the selection mechanism loses its justification.

Editorial extensions

If this is right

Replacing raw-score priority with QUTC in FunSearch-style evolution yields better heuristics on online bin packing, TSP, and cap set search under the paper's settings.
The improvement is not tied to one LLM: it reproduces with OpenCoder-8B and with Deepseek-coder-6.7B on OR3.
The ablation isolates three sources of gain: offspring-mean quality beats raw score, the uncertainty bonus beats quality-only selection, and UIQ-based island reset beats score-based reset.
Because QUTC is a change to the priority criterion only, the same evolution loop can accept it without altering the LLM prompting or evaluation machinery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the parent-quality estimate in Eq. (1) transfers to other multi-parent evolutionary settings, QUTC could be applied to any LLM+EA method that selects parents by score, including prompt-based variants.
A natural test is to run QUBE on additional NP-hard heuristic discovery tasks, such as graph coloring or admissible sets, to see whether the UIQ advantage generalizes beyond bin packing, TSP, and cap sets.
The uncertainty coefficient k is currently searched per problem; an adaptive schedule that shrinks k over time might remove that tuning cost while preserving the exploration-exploitation balance.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, here's my read on QUBE. It's a reasonable, incremental improvement to FunSearch: replace score-based parent selection with a UCB-style cluster-level quality criterion. The cluster-level UIQ and QUTC are not in the original work, and the paper ships a same-hardware FunSearch reproduction, ablations, a second LLM check, code, and an honest limitations section. That's more than most LLM+EA papers do.

The experiments show a consistent gain on online bin packing against their FunSearch* baseline, and the ablation in Table 2 supports the broad claim that cluster-level quality beats raw score and that the uncertainty term helps. The TSP gains are real but tiny; cap set is a wash against the original FunSearch's 512, which they concede.

Now the soft spots, in proportion. The central mechanism rests on Qt(C), defined in Eq. (1) as the mean offspring score across all generations using any sample in C as a parent. Section 5.6 calls this an unbiased estimate of C's expected offspring quality. But every offspring has two parents -- the prompt asks the LLM to improve parent v1 with parent v0 as context -- and each offspring's score is assigned to both parent clusters. That only gives a marginal-quality estimate if the two parents contribute additively and independently, and the co-parent distribution is stationary. None of that is stated or tested. The stress-test note is right: the ablation varies the selection rule but never varies the credit-assignment rule, so the mechanism's statistical foundation is unsupported. I'd still give the method the benefit of the doubt on OBP because the empirical difference is consistent, but the paper's language overstates what Eq. (1) can mean.

Other issues: main Table 1 reports best-of-10 with no variance; k is tuned per benchmark on the target dataset; the exploration/exploitation metrics ('Recent Best Score', 'Recent Proportion of Change') are invented in the same paper and then used to validate the method, which is a mild circularity. None of these are fatal; they just make the headline 'steadily find better heuristics' weaker than it reads.

Bottom line: this is a paper for the LLM+EA community, not for a general audience. It deserves a serious referee. I would ask for a one-parent attribution ablation or a sensitivity test that co-varies credit assignment, plus variance reporting in the main tables.

Referee Report

3 major / 7 minor

Summary. The paper proposes QUBE, a modification of the FunSearch LLM+EA framework. QUBE replaces FunSearch's score-based priority criterion with the Quality-Uncertainty Trade-off Criterion (QUTC), which ranks clusters by the Uncertainty-Inclusive Quality (UIQ) metric from Eq. (2): a cluster's mean offspring score plus a UCB-style exploration bonus. The criterion is used both for parent selection and for island-reset decisions. Experiments cover online bin packing (OR and Weibull instances), cap set n=8, and TSP20/50/100, with comparisons to FunSearch, a reproduced FunSearch*, and EoH. The paper reports lower excess ratios on OBP and TSP and a larger cap set than FunSearch*, but not larger than the original FunSearch cap-set result; ablations on OR3 separate the quality and uncertainty terms, and a second-LLM check is included. The code is released.

Significance. If the QUTC mechanism is sound, the paper offers a concrete and inexpensive way to improve the exploitation-exploration balance of LLM-based program evolution, and the release of code plus the same-infrastructure FunSearch* baseline are genuine strengths. The OR3 ablation reports averages with standard deviations, and Table 3 shows the gain persists across two LLMs. At the same time, the central estimator's two-parent credit assignment is not validated (Section 4.2), the main results are best-of-10 without variance (Table 1), the uncertainty weight k is tuned on the target datasets (Appendix C), and the cap-set comparison does not surpass the original FunSearch. These gaps make the strong 'significant improvements' claim premature, but they are addressable by additional analysis and experiments.

major comments (3)

[Section 4.2, Eq. (1), Appendix F] The cluster-quality estimate Qt(C) is computed by crediting every offspring score to each of its two parents. Appendix F shows that each prompt contains two parent functions ('priority_v0' and 'priority_v1') and asks for an improvement of 'priority_v1', so Eq. (1) adds the same offspring score s(a) to both parent clusters. Section 5.6 calls Qt(C) 'an unbiased estimate of the expected outcome with offspring samples in C serving as parents,' but for the purpose of ranking clusters this is only unbiased under an additivity/independence assumption about the two parents' contributions, which is neither stated nor tested. If the v0 parent changes the difficulty or style of the improvement task, a cluster can be systematically over- or under-ranked depending on which clusters it is paired with, so QUTC's parent-selection and island-reset decisions rest on a quality estimate whose statistical meaning is unsupported. The ablation in Table 2 changes the quality measure and the reset rule but never varies the credit-assignment rule, so the core selection mechanism is not cleanly isolated. I recommend adding an experiment that varies the attribution scheme (e.g., one-parent prompts, or random fixed co-parents) and reporting whether QUTC cluster rankings and final performance change.
[Section 5.4, Table 1, Appendix C] The main results in Table 1 report only the best of 10 runs with no variance or confidence intervals, while the ablation in Table 2 reports averages and standard deviations. This is particularly problematic for TSP100, where the reported gap is 0.023% versus 0.029%; without variance information it is not possible to assess whether this difference is significant. In addition, the UIQ hyperparameter k is tuned on the same benchmark used for evaluation (Appendix C: k is searched on OR3 for the OR datasets, on Weibull5k for the Weibull datasets, and on cap set n=8 for the cap-set experiment), which makes the final comparison against FunSearch* optimistic. Please report variance or confidence intervals for all main settings and use a validation protocol for hyperparameter selection, or clearly disclose that k is selected on test data.
[Section 5.4 and Limitations] The cap-set result of 480 does not beat the original FunSearch result of 512, as the Limitations section concedes. Since the cap set is one of FunSearch's signature results, the paper's abstract and introduction overstate the case when claiming 'significant performance improvements' without qualification. The comparison against FunSearch* is still meaningful as a controlled reproduction, but the claim should be narrowed to the controlled setting, or the paper should provide a cap-set comparison under matched LLM, compute, and sample budget. As written, a reader can reasonably conclude that QUBE improves on FunSearch itself on all problems, which the reported evidence does not support.

minor comments (7)

[Section 4.1] The phrase 'hubristic evolution' appears to be a typo for 'heuristic evolution' and should be corrected.
[Appendix F] All three prompt templates contain the instruction 'do note answer anything else', which should read 'do not answer anything else'.
[Appendix C] The sentence 'we experimented kr within the range of 16 to 64' should refer to the hyperparameter k, and Table 7 is cited as 'Table ??' in the text.
[Table 2] The table header 'Parent UIIS' is unclear; the column likely refers to the island-reset strategy, and 'UIIS' appears to be a typo for 'UIQ' or similar. Please clarify the column labels.
[Section 3.3] The definition of 'Recent Proportion of Change' should state whether the token-level edit distance is computed on code tokens or characters, and whether the normalization is by the parent length or the offspring length; the current text is ambiguous.
[Section 4.2, Eq. (2)] The uncertainty term is undefined when N_t(C)=0, which can occur for newly created clusters that have never been selected as parents; please specify the initialization or handling of this case.
[Abstract] The sentence 'Our code are available' should be 'Our code is available'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QUBE's benchmark gains come from an independent evolution run, not from Eq. (1)-(2) by construction.

full rationale

The paper's central claim is that QUTC, built on UIQ in Eq. (2), improves parent selection and island reset relative to FunSearch. This is a proposed search objective, not a result that is defined to equal the reported benchmark. Table 1 compares the best heuristics found by the full LLM+EA process against FunSearch*, EoH, and the published FunSearch numbers; those excess ratios, cap-set sizes, and TSP gaps are generated by running the evolutionary system, not by algebraic substitution into Eq. (1) or Eq. (2). The hyperparameter k is explicitly searched on the target datasets (Appendix C), which is a disclosed test-set tuning concern, but no reported result is a renamed fit: for OR3, all searched k values beat FunSearch* in Table 5, so the main conclusion is not forced by parameter choice. The paper's use of its own 'Recent Best Score' and 'Recent Proportion of Change' metrics to diagnose FunSearch and then validate QUBE is a self-consistent diagnostic argument rather than a circular derivation, and it is supported by the independent benchmark comparisons. The skeptic's point that Qt(C) credits every two-parent offspring to both parent clusters is a statistical identifiability limitation, not a circular step: it does not make the final benchmark outputs equal to the input definitions. There are no load-bearing self-citations, and no claim in the paper reduces to its own assumptions by construction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the cluster-attribution assumption in Eq. (1), the UCB bonus structure, and per-benchmark tuning of k. No new physical or ontological entities are introduced; UIQ and QUTC are algorithmic scoring rules, not entities with independent falsifiable handles.

free parameters (3)

UIQ uncertainty weight k = 0.0008 (OR OBP), 0.0001 (Weibull OBP), 32.0 (cap set), 1e-5 (TSP).
Searched per benchmark in Appendix C to optimize final performance; it directly controls the exploration bonus that is the paper's main contribution.
sample-selection temperature Tprog = 1.0.
Set by hand in Table 4; controls within-cluster sampling probability and is not searched or justified further.
island reset interval Treset = 32,768 (OBP), 262,144 (cap set).
Chosen in Table 4, presumably inherited from FunSearch; it affects when underperforming islands are reset.

assumptions (3)

domain assumption Deterministic score s(c) on fixed test instances is a faithful measure of heuristic quality.
Used to define cluster scores and every comparison; standard in supervised evaluation, but not argued for in the paper.
ad hoc to paper Offspring quality can be attributed additively to each parent cluster.
Equation (1) charges every offspring's score to both parent clusters, although each new program is prompted with two parents; this untested modeling choice is what the UIQ ranking depends on.
ad hoc to paper Token-level edit distance between a generated sample and its nearest parent measures useful exploration.
Introduced in Section 3.3 as 'Recent Proportion of Change' and used as evidence that QUBE explores better; no validation that edit distance correlates with function-space exploration value.

how reviews work

0 comments

Cite this review

Pith. "Pith review of QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution." pith.science (2026). https://pith.science/paper/77LTG2PQ

@misc{pith2026241220694,
  author       = {Pith},
  title        = {Pith review of: QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/77LTG2PQ}},
  note         = {Machine review of arXiv:2412.20694}
}

read the original abstract

Solving NP-hard problems traditionally relies on heuristics, yet manually designing effective heuristics for complex problems remains a significant challenge. While recent advancements like FunSearch have shown that large language models (LLMs) can be integrated into evolutionary algorithms (EAs) for heuristic design, their potential is hindered by limitations in balancing exploitation and exploration. We introduce Quality-Uncertainty Balanced Evolution (QUBE), a novel approach that enhances LLM+EA methods by redefining the priority criterion within the FunSearch framework. QUBE employs the Quality-Uncertainty Trade-off Criterion (QUTC), based on our proposed Uncertainty-Inclusive Quality metric, to evaluate and guide the evolutionary process. Through extensive experiments on challenging NP-complete problems, QUBE demonstrates significant performance improvements over FunSearch and baseline methods. Our code are available at https://github.com/zzjchen/QUBE_code.

Figures

Figures reproduced from arXiv: 2412.20694 by the authors.

**Figure 2.** Illustration of QUBE. We manipulate the parent selection procedure in FunSearch’s evolution process. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** FunSearch has a consistently low “Recent [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Performance progress on online bin packing. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: More experiment results on cap set n=8 and [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
cs.AI 2026-07 conditional novelty 6.0 of 10

A survey of 1,250 papers organizes AI self-improvement along two axes—what is improved and loop closure—finding that demonstrated self-improvement strength tracks a verification hierarchy from formal verifiers down to...
An In-depth Study of LLM Contributions to the Bin Packing Problem
cs.AI 2025-10 accept novelty 6.0 of 10

The LLM-generated bin packing heuristics from Nature's FunSearch paper reduce to simple two-parameter threshold rules and don't constitute a mathematical discovery.
Explainable AI-assisted Optimization for Feynman Integral Reduction
hep-ph 2025-02 conditional novelty 6.0 of 10

FunSearch discovered a simple priority function for ordering IBP seeding integrals, reducing the number needed for multi-loop Feynman integral reductions by factors up to 3058.
Using Reasoning Models to Generate Search Heuristics that Solve Open Instances of Combinatorial Design Problems
cs.AI 2025-05 conditional novelty 5.0 of 10

LLM-generated search heuristics run through the CPro1 protocol with the reasoning model o3-mini-high produced verified constructions resolving open instances in 7 Handbook design families and newer problems.

Reference graph

Works this paper leans on

37 extracted references · 18 canonical work pages · cited by 4 Pith papers

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Peter Auer. 2002. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3(Nov):397--422

work page 2002
[4]

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732

arXiv 2021
[5]

John E. Beasley. 1990. Or-library: distributing test problems by electronic mail. Journal of the operational research society, 41(11):1069--1072

work page 1990
[6]

Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. 2021. Machine learning for combinatorial optimization: A methodological tour d'horizon. Eur. J. Oper. Res., 290(2):405--421

work page 2021
[7]

Shuvayan Brahmachary, Subodh M Joshi, Aniruddha Panda, Kaushik Koneripalli, Arun Kumar Sagotra, Harshil Patel, Ankush Sharma, Ameya D Jagtap, and Kaushic Kalyanaraman. 2024. Large language model-based evolutionary optimizer: Reasoning with elitism. arXiv preprint arXiv:2403.02054

arXiv 2024
[8]

Burke, Graham Kendall, Jim Newall, Emma Hart, Peter Ross, and Sonia Schulenburg

Edmund K. Burke, Graham Kendall, Jim Newall, Emma Hart, Peter Ross, and Sonia Schulenburg. 2003. Hyper-heuristics: An emerging direction in modern search technology. In Handbook of Metaheuristics, volume 57 of International Series in Operations Research & Management Science , pages 457--474. Kluwer / Springer

work page 2003

Show all 37 references

[9]

Angelica Chen, David Dohan, and David So. 2024. Evoprompting: language models for code-level neural architecture search. Advances in Neural Information Processing Systems, 36

2024
[10]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

2021 arXiv
[11]

Edward G Coffman Jr, Michael R Garey, and David S Johnson. 1984. Approximation algorithms for bin-packing—an updated survey. In Algorithm design for computer system design, pages 49--106. Springer

1984
[12]

Joshua Grochow. 2019. New applications of the polynomial method: the cap set conjecture and beyond. Bulletin of the American Mathematical Society, 56(1):29--64

2019
[13]

Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. Deepseek-coder: When the large language model meets programming - the rise of code intelligence. CoRR, abs/2401.14196

2024 arXiv
[14]

Andr \' e Hottung, Shunji Tanaka, and Kevin Tierney. 2020. Deep learning assisted heuristic tree search for the container pre-marshalling problem. Comput. Oper. Res., 113

2020
[15]

Siming Huang, Tianhao Cheng, Jason Klein Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J Yang, JH Liu, Chenchen Zhang, Linzheng Chai, et al. 2024. Opencoder: The open cookbook for top-tier code large language models. arXiv preprint arXiv:2411.04905

2024 arXiv
[16]

Benjamin Hudson, Qingbiao Li, Matthew Malencia, and Amanda Prorok. 2022. Graph neural network guided local search for the traveling salesperson problem. In ICLR . OpenReview.net

2022
[17]

Ya - Hui Jia, Yi Mei, and Mengjie Zhang. 2023. Learning heuristics with different representations for stochastic routing. IEEE Trans. Cybern. , 53(5):3205--3219

2023
[18]

Wouter Kool, Herke Van Hoof, and Max Welling. 2018. Attention, learn to solve routing problems! arXiv preprint arXiv:1803.08475

2018 arXiv
[19]

T.L Lai and Herbert Robbins. 1985. https://doi.org/10.1016/0196-8858(85)90002-8 Asymptotically efficient adaptive allocation rules . Adv. Appl. Math., 6(1):4–22

1985 doi
[20]

Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O. Stanley. 2024. https://doi.org/10.1007/978-981-99-3814-8_11 Evolution Through Large Models , pages 331--366. Springer Nature Singapore, Singapore

2024 doi
[21]

Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161

2023 arXiv
[22]

Fei Liu, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang. 2023. Algorithm evolution using large language model. arXiv preprint arXiv:2311.15249

2023 arXiv
[23]

Fei Liu, Tong Xialiang, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. 2024. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In Forty-first International Conference on Machine Learning

2024
[24]

Silvano Martello and Paolo Toth. 1990. Lower bounds and reduction procedures for the bin packing problem. Discrete applied mathematics, 28(1):59--70

1990
[25]

Yi Mei, Qi Chen, Andrew Lensen, Bing Xue, and Mengjie Zhang. 2023. Explainable artificial intelligence by genetic programming: A survey. IEEE Trans. Evol. Comput. , 27(3):621--641

2023
[26]

Elliot Meyerson, Mark J Nelson, Herbie Bradley, Adam Gaier, Arash Moradi, Amy K Hoover, and Joel Lehman. 2023. Language model crossover: Variation through few-shot prompting. arXiv preprint arXiv:2302.12170

2023
[27]

Muhammad Umair Nasir, Sam Earle, Julian Togelius, Steven James, and Christopher Cleghorn. 2024. Llmatic: neural architecture search via large language models and quality diversity optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1110--1118

2024
[28]

Gustafson, and Wolfgang Banzhaf

Michael O'Neill, Leonardo Vanneschi, Steven M. Gustafson, and Wolfgang Banzhaf. 2010. Open issues in genetic programming. Genet. Program. Evolvable Mach., 11(3-4):339--363

2010
[29]

Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. 2024. Mathematical discoveries from program search with large language models. Nature, 62...

2024
[30]

Terence Tao and Van H Vu. 2006. Additive combinatorics, volume 105. Cambridge University Press

2006
[31]

Christos Voudouris, Edward PK Tsang, and Abdullah Alsheddy. 2010. Guided local search. In Handbook of metaheuristics, pages 321--361. Springer

2010
[32]

Haorui Wang, Marta Skreta, Cher-Tian Ser, Wenhao Gao, Lingkai Kong, Felix Strieth-Kalthoff, Chenru Duan, Yuchen Zhuang, Yue Yu, Yanqiao Zhu, et al. 2024. Efficient evolutionary search over chemical space with large language models. arXiv preprint arXiv:2406.16976

2024 arXiv
[33]

Lilian Weng. 2020. https://lilianweng.github.io/posts/2020-06-07-exploration-drl/ Exploration strategies in deep reinforcement learning . lilianweng.github.io

2020
[34]

Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. 2024. Reevo: Large language models as hyper-heuristics with reflective evolution. arXiv preprint arXiv:2402.01145

2024 arXiv
[35]

Rui Zhang, Fei Liu, Xi Lin, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. 2024. Understanding the importance of evolutionary search in automated heuristic design with large language models. In International Conference on Parallel Problem Solving from Nature, pages 185--202. Springer

2024
[36]

Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. 2024. Sglang: Efficient execution of structured language model programs. arXiv preprint arXiv:2312.07104

2024 arXiv
[37]

Mingkai Zheng, Xiu Su, Shan You, Fei Wang, Chen Qian, Chang Xu, and Samuel Albanie. 2023. Can gpt-4 perform neural architecture search? arXiv preprint arXiv:2304.10970

2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Peter Auer. 2002. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3(Nov):397--422

work page 2002

[4] [4]

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732

arXiv 2021

[5] [5]

John E. Beasley. 1990. Or-library: distributing test problems by electronic mail. Journal of the operational research society, 41(11):1069--1072

work page 1990

[6] [6]

Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. 2021. Machine learning for combinatorial optimization: A methodological tour d'horizon. Eur. J. Oper. Res., 290(2):405--421

work page 2021

[7] [7]

Shuvayan Brahmachary, Subodh M Joshi, Aniruddha Panda, Kaushik Koneripalli, Arun Kumar Sagotra, Harshil Patel, Ankush Sharma, Ameya D Jagtap, and Kaushic Kalyanaraman. 2024. Large language model-based evolutionary optimizer: Reasoning with elitism. arXiv preprint arXiv:2403.02054

arXiv 2024

[8] [8]

Burke, Graham Kendall, Jim Newall, Emma Hart, Peter Ross, and Sonia Schulenburg

Edmund K. Burke, Graham Kendall, Jim Newall, Emma Hart, Peter Ross, and Sonia Schulenburg. 2003. Hyper-heuristics: An emerging direction in modern search technology. In Handbook of Metaheuristics, volume 57 of International Series in Operations Research & Management Science , pages 457--474. Kluwer / Springer

work page 2003