Evaluating the Effectiveness of Black-Box Prompt Optimization as the Scale of LLMs Continues to Grow

Jingyuan Yang; Rongjun Li; Yihang Wu; Zhan Xiao; Ziyu Zhou

REVIEW 4 major objections 5 minor 19 references

Evaluating the Effectiveness of Black-Box Prompt Optimization as the Scale of LLMs Continues to Grow

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Black-box prompt optimization delivers shrinking gains as language models grow, an inverse-scaling effect.

desk verdict Useful preliminary measurement, but the 'inverse scaling law' is not supported by the evidence; treat the claim as a hypothesis, not a result. read the letter →

arxiv 2505.08303 v1 pith:O254DF26 submitted 2025-05-13 cs.CL

Ziyu Zhou , Yihang Wu , Jingyuan Yang , Zhan Xiao , Rongjun Li This is my paper · ORCID

classification cs.CL

keywords black-boxpromptoptimizationinversescalinglawlargelanguagemodelsEvomodelscaleNLUbenchmarksNLG

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Black-box prompt optimization—automatically rewriting an instruction by treating the model as an unopenable box—has mostly been tested on 7B–14B models. This paper asks whether the same tricks still pay off on frontier-scale models and reports that they do not, by much: on DeepSeek V3 and Gemini 2.0 Flash, three optimizers improve four benchmarks by only about 0.86%–2.03% on average. It then argues that model scale itself is the cause, showing that EvoPrompt's gain on SST-5 drops from 12% at Qwen 7B to 5.9% at Qwen 72B to 1.1% at DeepSeek V3 671B. If this inverse-scaling pattern holds, automatic prompt optimization will matter less as models grow, shifting the practical value of prompt engineering toward smaller models and richer prompt content rather than lexical tweaks.

What carries the argument

The central object is the scaling curve produced by applying EvoPrompt—an evolutionary algorithm that rewrites prompts through crossover and mutation—to the Qwen 2.5 family at 7B, 14B, 32B, and 72B, with DeepSeek V3 (671B) as a further endpoint. The mechanism claimed is that the optimized prompts differ from initial prompts only in lexical detail, and such superficial changes matter less when the model's alignment is already strong. The argument treats the trend across model sizes as evidence that model scale, rather than optimizer choice or dataset, drives the diminishing returns.

What would settle it

Run EvoPrompt with, say, ten independent optimization runs on Qwen2.5-7B and Qwen2.5-72B on SST-5, computing paired bootstrap confidence intervals for the mean gain; if the 7B gain is not significantly larger than the 72B gain, the inverse-scaling claim is unsupported. Alternatively, repeat the Qwen-scale experiment on another LLM family spanning 7B to 72B and show flat or increasing gains with model size.

Watch

Extended reading notes

Core claim

The paper's central claim is an inverse scaling law: the performance gain from black-box prompt optimization declines as LLM scale increases. The key observations are the small average gains on two large-scale models (DeepSeek V3 and Gemini 2.0 Flash) across SST-5, AG's News, SAMSum, and ASSET, and the monotonic decline of EvoPrompt gains across the Qwen 2.5 series, illustrated by SST-5 accuracy: 12% at 7B, down to 5.9% at 72B, and 1.1% at DeepSeek V3 (671B). The authors propose a mechanism: larger models have stronger alignment and domain understanding, so explicit domain cues and synonym-level rephrasing, which help smaller models, become redundant.

Load-bearing premise

The conclusion rests on treating one run of EvoPrompt per model size, without error bars, as an accurate measure of optimization effectiveness, and on assuming that the Qwen 2.5 family's size differences, rather than dataset headroom or optimizer choices, are what drive the declining gains.

Editorial extensions

If this is right

For frontier models, automatic black-box prompt optimization may no longer justify its API cost, since the measured gains are often under one or two percent.
Benchmark comparisons across model sizes should control for prompt sensitivity, because the same optimized prompt yields very different gains at 7B versus 72B.
Smaller models, around 7B, remain the most promising targets for black-box prompt optimization in practical applications.
Prompt engineering effort may need to shift from lexical rewording toward richer context, demonstrations, or reasoning scaffolds, which the paper does not test.
Researchers evaluating new prompt optimizers should report results on large models separately, since averaging across scales can mask an inverse-scaling effect.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test would be to run EvoPrompt with multiple independent seeds on Qwen2.5-7B and Qwen2.5-72B, computing paired bootstrap confidence intervals; if the 12% versus 5.9% gap on SST-5 does not survive, the inverse-scaling claim reduces to noise.
The same decline may partly reflect benchmark saturation: larger models already sit near the ceiling on these tasks, leaving less headroom for any optimizer to add.
If robust alignment is the cause, similar declines should appear for white-box prompt-tuning methods that access model gradients, not just for black-box optimizers.
The authors' own limitations note that reasoning-oriented models were excluded; a natural next test is whether inverse scaling holds for chain-of-thought-style models, which may be more prompt-sensitive.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

You'll want to know: this paper asks a good question and reports a clean-sounding answer—black-box prompt optimization gains shrink as LLMs grow—but the evidence behind the answer is thinner than the framing suggests. The Qwen 2.5 scaling curve is new, but it rests on single runs of one optimizer with no variance estimates, and the top point comes from a different model family.

What's genuinely useful: the authors evaluate three black-box methods on DeepSeek V3 and Gemini 2.0 Flash across four datasets, and they show concrete examples of how optimized prompts differ by scale. The observation that large models barely benefit from lexical tweaks is consistent with prior work (Shirafuji et al.), and the paper extends that to a within-family trend. The writing is clear, and the Limitations section is honest about the 72B–671B gap.

The soft spots are load-bearing. Every number in Table 1 comes from a single run. A change like 56.0 to 56.4 on a 500-example test set is two examples; that's sampling noise. Figure 1 has no error bars or significance tests, so the 12% → 5.9% → 1.1% progression could easily be optimizer instability. EvoPrompt is stochastic; one trajectory per model doesn't establish a monotone trend. And attaching DeepSeek V3 as the 671B data point confounds scale with architecture and training pipeline. The authors also note the optimized prompts themselves differ across sizes, so the intervention changes, not just the model.

That said, the inverse scaling hypothesis is plausible and worth testing. The paper just doesn't have the measurement to support calling it a law. I'd assign it to a serious referee: the question is timely for practitioners, and the work is a reasonable pilot. But the referee should require repeated runs, confidence intervals, and a clean scaling study within a single family before accepting the central claim. As it stands, I'd file this as a preliminary report, not a citable result.

My recommendation for peer review: send it out—the question matters enough that a rigorous referee could push the authors toward a more defensible design. I'd expect heavy revision, or rejection as-is if the authors can't provide variance estimates and address the confounds.

Referee Report

4 major / 5 minor

Summary. The paper evaluates three black-box prompt optimization methods (EvoPrompt, ProTeGi, BPO) on two large-scale LLMs (DeepSeek V3 and Gemini 2.0 Flash) across four NLU and NLG datasets, reporting only limited performance gains. To explain these limited gains, the authors hypothesize that model scale is the primary factor, and they test this by applying EvoPrompt to the Qwen 2.5 series (7B–72B) plus a single external point from DeepSeek V3 (671B). Based on these experiments, the paper claims an 'inverse scaling law' in which the effectiveness of black-box prompt optimization declines as model size increases.

Significance. If robustly established, the claimed inverse-scaling law would be practically important: it would suggest that black-box prompt optimization yields diminishing returns as LLMs grow, guiding practitioners to allocate optimization effort only for smaller models. The paper also contributes a useful preliminary data point on the behavior of three optimizers on very large models. However, the current evidence is far from establishing a law. The central claim rests on unrepeated single runs without uncertainty estimates, on a single optimizer for the scaling curve, and on a cross-family comparison that confounds model scale with model family and training pipeline. These issues are load-bearing for the paper's main conclusion.

major comments (4)

[Section 4.2, Figure 1] The central inverse-scaling claim is supported only by single runs of EvoPrompt on each Qwen 2.5 model, with no standard deviations, confidence intervals, or significance tests. On SST-5, the reported improvement at the DeepSeek V3 point is 56.0 to 56.4, which is just two additional correct answers on a 500-example test set and lies within binomial sampling noise; the monotone decline from 12% to 5.9% to 1.1% could therefore reflect optimizer stochasticity rather than model scale. Please repeat the optimization with multiple seeds and report variance estimates and, where possible, paired significance tests.
[Section 4.2, Limitations] The scale comparison appends DeepSeek V3 (671B) to the Qwen 2.5 series, so model scale is confounded with model family, architecture, and training pipeline. The paper's own Limitations section explicitly acknowledges the untested gap between 72B and 671B and the differing model family; consequently, the final point in Figure 1 cannot be attributed to scale, and the label 'inverse-scaling phenomenon' is not warranted. The claim should either be restricted to the Qwen 2.5 family or supported by a controlled multi-family design that accounts for family identity.
[Section 4.1, Table 3] The scaling analysis uses only EvoPrompt, yet the abstract and Section 4.2 generalize to 'black-box prompt optimization methods'; this generalization is unsupported by the data. In addition, Table 3 shows that the optimized prompts differ qualitatively across model sizes (e.g., the AG's News prompts contain different instructions and wording), so the optimization process is not held constant while scale varies. The observed decline in gains could stem from differences in the discovered prompts rather than from model scale. Applying all three methods across the scale range and reporting prompt similarity or edit distance would help disentangle these factors.
[Section 3.3, Table 1] The claim that improvements on large-scale models are 'limited' is made without a comparison to smaller models under an identical setup within this section, and several entries in Table 1 are within the noise floor for a 500-example test set (e.g., DeepSeek V3 with BPO on AG's News decreases from 84.6 to 83.8, and SST-5 improvements of 0.4 points correspond to two examples). The average improvements of 0.86% to 2.03% need confidence intervals or statistical tests before they can be interpreted as nonzero effects.

minor comments (5)

[Limitations] The Limitations section contains a typo: 'treasoning-oriented LLMs' should be 'reasoning-oriented LLMs.'
[References] The references for Anil et al., Brown et al., and Hurst et al. contain 'and 1 others' placeholders; the full author lists should be provided.
[References] The dataset references for SST-5 and AG's News point to URLs with access dates; proper bibliographic entries (e.g., ACL Anthology or a formal publisher citation) should be used instead.
[Section 4.2] The phrase 'a distinct inverse-scaling phenomenon' is an interpretive label rather than a fitted scaling law; 'observed trend' would be more precise in the absence of a quantitative fit.
[Table 1] Table 1 does not explain how the 'Average % Increase' is computed; the authors should state whether it is relative to the initial score or an average of per-method relative changes.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the inverse-scaling conclusion is an empirical reading of measured gains, not a fitted or self-referential derivation.

full rationale

The paper is an empirical evaluation. It applies three existing black-box prompt optimizers to models of varying scale and reports measured accuracy, ROUGE-L, and SARI changes. No quantity is defined in terms of another, no parameter is fitted from a subset of data and then renamed as a prediction, and the inverse-scaling claim is an interpretation of plotted measured gains rather than an equation manufactured from the paper's own inputs. The cited optimization methods (EvoPrompt, ProTeGi, BPO) are prior external work, and the paper does not rely on a self-citation chain for its central claim. The acknowledged limitations, including the untested 72B-to-671B gap and the single-run design without variance estimates, concern statistical soundness and external validity rather than circularity. Therefore the paper receives a score of 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters or invented entities. Its empirical claims rest on assumptions about the representativeness of the selected methods, the sufficiency of single runs, and the comparability of relative improvements across datasets and model families.

assumptions (3)

domain assumption The three selected methods (EvoPrompt, ProTeGi, BPO) are representative of the class of black-box prompt optimization methods.
The paper generalizes its conclusions to black-box prompt optimization as a category, but the inverse scaling result uses only EvoPrompt.
domain assumption A single optimization run per configuration is sufficient to estimate the performance improvement.
No repeated trials or error bars are reported, yet the claim depends on the stability of the observed improvements.
domain assumption Model scale is the primary factor explaining the limited improvements, rather than model family, training data, or optimizer choice.
This is asserted in Section 4.2 without controlling for Qwen vs DeepSeek differences or for the different optimizer configurations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evaluating the Effectiveness of Black-Box Prompt Optimization as the Scale of LLMs Continues to Grow." pith.science (2026). https://pith.science/paper/O254DF26

@misc{pith2026250508303,
  author       = {Pith},
  title        = {Pith review of: Evaluating the Effectiveness of Black-Box Prompt Optimization as the Scale of LLMs Continues to Grow},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O254DF26}},
  note         = {Machine review of arXiv:2505.08303}
}

read the original abstract

Black-Box prompt optimization methods have emerged as a promising strategy for refining input prompts to better align large language models (LLMs), thereby enhancing their task performance. Although these methods have demonstrated encouraging results, most studies and experiments have primarily focused on smaller-scale models (e.g., 7B, 14B) or earlier versions (e.g., GPT-3.5) of LLMs. As the scale of LLMs continues to increase, such as with DeepSeek V3 (671B), it remains an open question whether these black-box optimization techniques will continue to yield significant performance improvements for models of such scale. In response to this, we select three well-known black-box optimization methods and evaluate them on large-scale LLMs (DeepSeek V3 and Gemini 2.0 Flash) across four NLU and NLG datasets. The results show that these black-box prompt optimization methods offer only limited improvements on these large-scale LLMs. Furthermore, we hypothesize that the scale of the model is the primary factor contributing to the limited benefits observed. To explore this hypothesis, we conducted experiments on LLMs of varying sizes (Qwen 2.5 series, ranging from 7B to 72B) and observed an inverse scaling law, wherein the effectiveness of black-box optimization methods diminished as the model size increased.

Figures

Figures reproduced from arXiv: 2505.08303 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 7 canonical work pages

[3]

Advances in Neural Information Processing Systems , 33:1877–1901

Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877–1901. Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang

work page 1901
[4]

arXiv preprint arXiv:2311.04155

Black-box prompt optimization: Aligning large language models without training them. arXiv preprint arXiv:2311.04155. Sarkar Snigdha Sarathi Das, Ryo Kamoi, Bo Pang, Yusen Zhang, Caiming Xiong, and Rui Zhang

arXiv
[6]

arXiv preprint arXiv:2412.19437

Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. DeepSeek-AI

arXiv
[9]

arXiv preprint arXiv:2309.08532

Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532. Aaron Hurst, Alec Lerer, AP Goucher, Alex Perelman, Aditya Ramesh, Arthur Clark, AJ Ostrow, and 1 others

arXiv
[10]

arXiv preprint arXiv:2410.21276

Gpt-4o system card. arXiv preprint arXiv:2410.21276. Brian Lester, Rami Al-Rfou, and Noah Constant

arXiv
[12]

gradient descent

Automatic prompt optimization with "gradient descent" and beam search. arXiv preprint arXiv:2305.03495. Taylor Shin, Yasaman Razeghi, Robert L Logan IV , Eric Wallace, and Sameer Singh

arXiv
[13]

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235

Autoprompt: Eliciting knowledge from language models with au- tomatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235. 5 Atsushi Shirafuji, Yutaka Watanobe, Takumi Ito, Makoto Morishita, Yuki Nakamura, Yusuke Oda, and Jun Suzuki

work page 2020
[14]

arXiv preprint arXiv:2306.14583

Exploring the robustness of large language models for solving programming problems. arXiv preprint arXiv:2306.14583. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts

arXiv

Show all 19 references

[16]

arXiv preprint arXiv:2309.03409

Large language models as optimizers. arXiv preprint arXiv:2309.03409. Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, and Xuanjing Huang

arXiv
[17]

arXiv preprint arXiv:2303.10420

A comprehen- sive capability analysis of gpt-3 and gpt-3.5 series models. arXiv preprint arXiv:2303.10420. Xiang Zhang, Junbo Zhao, and Yann Le- Cun

arXiv
[2013]

https://nlp.stanford.edu/sentiment/

Recursive deep models for semantic compositionality over a sentiment tree- bank. https://nlp.stanford.edu/sentiment/. Accessed: 2025-04-27. Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Gold- blum, Jonas Geiping, and Tom Goldstein

2025
[2015]

https: //github.com/mhjabreel/CharCnn_Keras/ tree/master/data/ag_news_csv

Character-level convolutional networks for text classification. https: //github.com/mhjabreel/CharCnn_Keras/ tree/master/data/ag_news_csv. Accessed: 2025-04-27. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba

2025
[2019]

arXiv preprint arXiv:1911.12237

Samsum corpus: A human- annotated dialogue dataset for abstractive summa- rization. arXiv preprint arXiv:1911.12237. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yu- jiu Yang

1911 arXiv
[2020]

In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), pages 4668–4679

Asset: A dataset for tuning and evalua- tion of sentence simplification models with multiple rewriting transformations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), pages 4668–4679. Associa- tion for Computational Lingu...

2020
[2021]

In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 3045–3059

The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 3045–3059. Xiang Lisa Li and Percy Liang

2021
[2022]

arXiv preprint arXiv:2211.01910

Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910. 6

arXiv
[2023]

arXiv preprint arXiv:2312.11805

Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Tom B. Brown, Benjamin Mann, Nick Ryder, and 1 others

arXiv
[2024]

arXiv preprint arXiv:2405.12406

Greater: Gradients over reasoning makes smaller language models strong prompt optimizers. arXiv preprint arXiv:2405.12406. DeepSeek-AI

arXiv
[2025]

arXiv preprint arXiv:2501.12948

Deepseek-r1: Incentivizing rea- soning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Alek- sander Wawer

arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [3]

Advances in Neural Information Processing Systems , 33:1877–1901

Language models are few-shot learners. Advances in Neural Information Processing Systems , 33:1877–1901. Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang

work page 1901

[2] [4]

arXiv preprint arXiv:2311.04155

Black-box prompt optimization: Aligning large language models without training them. arXiv preprint arXiv:2311.04155. Sarkar Snigdha Sarathi Das, Ryo Kamoi, Bo Pang, Yusen Zhang, Caiming Xiong, and Rui Zhang

arXiv

[3] [6]

arXiv preprint arXiv:2412.19437

Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. DeepSeek-AI

arXiv

[4] [9]

arXiv preprint arXiv:2309.08532

Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532. Aaron Hurst, Alec Lerer, AP Goucher, Alex Perelman, Aditya Ramesh, Arthur Clark, AJ Ostrow, and 1 others

arXiv

[5] [10]

arXiv preprint arXiv:2410.21276

Gpt-4o system card. arXiv preprint arXiv:2410.21276. Brian Lester, Rami Al-Rfou, and Noah Constant

arXiv

[6] [12]

gradient descent

Automatic prompt optimization with "gradient descent" and beam search. arXiv preprint arXiv:2305.03495. Taylor Shin, Yasaman Razeghi, Robert L Logan IV , Eric Wallace, and Sameer Singh

arXiv

[7] [13]

In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235

Autoprompt: Eliciting knowledge from language models with au- tomatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235. 5 Atsushi Shirafuji, Yutaka Watanobe, Takumi Ito, Makoto Morishita, Yuki Nakamura, Yusuke Oda, and Jun Suzuki

work page 2020

[8] [14]

arXiv preprint arXiv:2306.14583

Exploring the robustness of large language models for solving programming problems. arXiv preprint arXiv:2306.14583. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts

arXiv