Pith. sign in

REVIEW 4 major objections 5 minor 21 references

PRISM: A Predictive Protocol for Permutation Optimization via Landscape Diagnostics

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two cheap landscape statistics forecast when permutation search beats sampling.

desk verdict The exhaustive 720-order instruction map is a genuine empirical find; the pre-flight protocol is promising but its core claim that it predicts when search beats sampling fails on the paper's flagship case. read the letter →

arxiv 2608.08344 v1 pith:FGE2G5PE submitted 2026-08-08 cs.LG cs.AImath.OC

Blessings Mambwe This is my paper · ORCID
classification cs.LGcs.AImath.OC
keywords permutationoptimizationlandscapediagnosticsfitnessautocorrelationfitness-distancecorrelationinstructionorderinglanguagemodelpromptingsearchstrategyselectionpre-flightprotocol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the difficulty of a permutation optimization problem is not set by the size of the search space, but by measurable landscape structure. It introduces a pre-flight protocol, PRISM, that spends a small budget estimating one-move autocorrelation and fitness–distance correlation, then forecasts whether structured search, uniform sampling, or enumeration is the right executor. The protocol is validated on exact 720- and 5,040-order landscapes, on language-model instruction ordering where order alone moves accuracy from 6.3% to 96.9%, and on scientific and neural architecture benchmarks. A sympathetic reader would care because, if correct, it replaces universal optimizer claims with a cheap, recorded decision rule for when search is worth running at all.

What carries the argument

The central object is the two-statistic pre-flight. The one-move autocorrelation $\rho_1 = \operatorname{corr}(F(\pi), F(o(\pi)))$ estimates local fitness preservation under a move operator $o$, and the fitness–distance correlation $\operatorname{FDC} = \operatorname{corr}(F(\pi), d(\pi, \Omega))$ estimates global guidance toward the optimum set $\Omega$. These two numbers route the search to an executor—exact enumeration, uniform sampling, an elitist evolutionary search, a surrogate, or transfer—under a distinct-evaluation budget, with a forecast ledger that retains null results and misses. The paper also uses exact enumeration of small spaces ($7!$ and $6!$) to obtain ground-truth optima and audited hit rates rather than method-dependent best observations.

What would settle it

Take a fresh set of permutation optimization problems with known ground truth, such as random typed synthetic landscapes or new language-model instruction sets; run the pre-flight with the Table 1 bands unchanged and compare the predicted regime against the observed advantage of elitist search over uniform sampling at equal distinct-evaluation budgets. If forecast accuracy is at chance on these held-out problems, the central claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that two inexpensive diagnostics—the one-move fitness autocorrelation $\rho_1$ and the fitness–distance correlation FDC—separate permutation landscapes into guiding, near-random, and deceptive regimes, and that a protocol using these measurements can forecast, before the main budget is spent, whether an evolutionary searcher will beat uniform sampling. The paper shows this in exact ground truth: on the 5,040-order parity landscape, elitist search hits an optimum in 19 of 40 runs versus 30 of 40 for uniform sampling, matching the near-zero $\rho_1$ and weak FDC forecast. On the 720-order instruction landscape, $\rho_1 = 0.547$ and FDC $= -0.346$ predict insertion as the matched operator, yet dense optima make sampling competitive; both the success and the scoped null result are recorded as forecasts. The paper further claims that ordering structure transfers across model families and survives automated wording optimization, making ordering a distinct optimization coordinate.

Load-bearing premise

The whole forecast rests on the assumption that the empirical operating bands in Table 1 generalize to new landscapes, and that the proxy fitness–distance correlation, which uses a frozen best sample as the target, does not miss the true basin.

Editorial extensions

If this is right

  • On landscapes with near-zero local correlation, structured search can be worse than uniform sampling, so a correct sampling forecast counts as a success rather than a failed optimizer.
  • Practitioners can spend a small probe budget before the main search to choose between enumeration, sampling, and evolution, and record the forecast for later audit.
  • Ordering of fixed components should be treated as a separate optimization coordinate, alongside wording or content optimization.
  • The regime thresholds are empirical operating bands, not universal constants, so each new landscape family requires recalibration.
  • Exact ground truth at small $n$ exposes circular comparisons in which the optimizer's best observation is mistaken for the answer key.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-statistic protocol could be applied to other fixed-component ordering problems listed in the paper, such as compiler pass ordering, database join plans, or laboratory automation, where an evaluator can be defined; these are testable extensions, not validated findings.
  • On larger landscapes where exact optima are unavailable, the proxy FDC may degrade exactly where the protocol is most useful; an uncertainty-aware probe that allocates extra evaluations to stabilise the proxy would be a natural follow-up.
  • If the thresholds generalize, PRISM could become a standard pre-registration step for optimization studies, with $\rho_1$ and FDC reported before the search budget is spent.
  • The instruction-ordering result implies that prompt-optimization pipelines should explicitly search over order after content is fixed, rather than treating order as noise or calibrating it away.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. PRISM proposes a pre-flight protocol for permutation optimization: before spending the main search budget, it measures one-move autocorrelation rho1 and fitness-distance correlation FDC on a small probe set, classifies the landscape into guiding, near-random, or deceptive regimes, and selects an executor (exact enumeration, uniform sampling, evolutionary search, surrogate, or transfer-guided list). The paper claims four results: (C1) the two-statistic pre-flight separates operator-aligned, near-random, and deceptive regimes; (C2) exact enumeration exposes a case where structured search is worse than uniform sampling (the parity landscape); (C3) exhaustive evaluation of all 720 orderings of six fixed reasoning instructions shows a 6.3%–96.9% accuracy range with transferable position effects; and (C4) cross-domain studies cover scientific pipelines, tabular NAS forecasts, and RL pilots. The manuscript emphasizes retained forecasts, misses, and reproducibility, including a forecast ledger, bootstrap intervals, and a reported external inference cost below US$20.

Significance. If the forecasting claim held, PRISM would turn optimizer selection in permutation spaces from universal superiority arguments into measured, budgeted decisions. The paper's strengths are substantial: exact enumeration of all 720 instruction orderings on a fixed 32-question GSM8K subset; a 40-run parity audit with Wilson intervals showing uniform sampling outperforming elitist search; bootstrap intervals for transfer and baseline comparisons; 18 registered NAS forecasts with retained misses; and a detailed reproducibility and resource-accounting record. These support the negative result that structured search is not universally better than sampling and the empirical claim that instruction order is a real optimization coordinate. The main risk is the forecasting link: the paper does not yet demonstrate out-of-sample validity of its regime thresholds, and on the paper's cleanest guiding landscape the structured-search-versus-sampling comparison shows overlapping intervals, so the operational promise that PRISM 'identifies when structured search is likely to outperform random sampling' remains unsupported.

major comments (4)
  1. [§7, Table 5] The paper's headline operational claim—that PRISM identifies when structured search is likely to outperform random sampling—is contradicted by the paper's own cleanest guiding example. The six-instruction landscape has rho1 = 0.547 and exact FDC = -0.346, is classified as guiding, and insertion is selected, yet the 40-run audit gives evolutionary search 9.9 [7,13] versus uniform sampling 10.8 [8,14] distinct evaluations to optimum, with overlapping bootstrap intervals. Table 5 records the verdict as 'scoped', not correct. Since this is the one exact landscape with a guiding classification and a higher-powered audit, the manuscript needs either to revise the abstract's promise (e.g., to 'identify operator-aligned and near-random/deceptive regimes') or to add a pre-flight criterion—such as optimum density or a margin threshold—that would have predicted the overlap.
  2. [§5, §B.2, Table 1, Figure 4] The reported selector accuracy of 0.925 at B=100 is not established as a prediction. Table 1 describes the thresholds as 'empirical operating bands, not universal constants,' and Figure 4's shaded regions appear drawn around the plotted points used to report accuracy. No calibration holdout, cross-validation, or independent threshold-setting procedure is described, so the reader cannot distinguish a fit to these landscapes from a forecast for new landscapes. Please report how the bands were derived (for example, fixed a priori, optimized on a training subset, or fitted post hoc) and evaluate the selector on landscapes excluded from threshold construction.
  3. [§3, Limitations] The proxy FDC, defined in Eq. (4) as using the best frozen probe sample as the target when exact optima are unknown, is load-bearing for all non-enumerated forecasts (eight-module, MATH-500, NAS slices, RL pilots). The Limitations section concedes that this proxy 'can miss the relevant basin.' The paper does not quantify how often or by how much the proxy diverges from exact FDC on landscapes where both are computable, such as the six-instruction or parity surfaces. Please report a sensitivity analysis comparing proxy and exact FDC on enumerated landscapes, or state explicitly which forecasts would change under alternative proxy choices.
  4. [§6, Table 5] The parity falsification is a strong empirical result, but the statement that 'the pre-flight predicts this boundary before the search comparison is run' (Section 6) is difficult to audit because the decision bands appear to be defined after the fact. Please provide the pre-registered or timestamped forecast for the parity landscape, or make explicit that this case was used to set, rather than test, the thresholds. Without this information, the reader cannot tell whether the parity result is a confirmation of the protocol or an input to it.
minor comments (5)
  1. [§2, Protocol 1] Protocol 1, step 1 says to 'measure variance, tie density, and evaluator resolution,' but the paper does not define how evaluator resolution is quantified; please provide a formula or a reference to a defined measure.
  2. [Appendix A] Theorem A.2 says 'Under the conditions of Theorem A.1,' but the preceding statement is Proposition A.1; the cross-reference should be corrected.
  3. [Table 4] The rows for 'OPRO ordering search' and 'Wording level change' would benefit from stating the exact evaluation budgets and number of runs; Section 8 mentions five shared runs for the OPRO comparison, but this is not carried in the table.
  4. [Figure 4] The caption states that the shaded regions 'summarize the conservative rule in Table 1,' but the numerical thresholds are not given in either place; please add the boundary values in the caption or table so the regions are reproducible.
  5. [Abstract vs. §8] The abstract says 'cross-model experiments show that useful ordering structure can transfer,' while Section 8 reports that 'fine rank transfer varies'; please align the abstract's wording with the more qualified finding.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PRISM's diagnostics are classical and independently measured, forecasts are registered before search, and the paper retains its own falsifications.

full rationale

PRISM's two statistics are defined directly from the frozen evaluator (Eq. 3 and Eq. 4) and are not defined in terms of the outcomes they predict. The paper explicitly disclaims novelty of the ingredients: 'PRISM does not claim these ingredients as new' (Section 4). The central predictive link is tested prospectively: 'A tabular neural architecture search study ... registers 18 HIGH/LOW forecasts on fixed-operation multiset slices before search. Thirteen are correct' (Section 9), and the forecast ledger retains mismatches, e.g., the six-instruction landscape is recorded as 'scoped' with overlapping intervals (Table 5). This is the opposite of a fit: the guiding-regime forecast did not guarantee a speed advantage, and the paper reports that honest null. There are no load-bearing self-citations; the only invoked convergence result is an external standard theorem (Rudolph [4]) with a self-contained proof in Appendix A. The 'empirical operating bands' caveat in Table 1 and the FDC-proxy limitation in Section 10 are limitations on calibration and measurement, not evidence that the forecasts reduce to their inputs by construction. Although Figure 4's shaded regions summarize the rule and are drawn around the plotted points, the paper does not report a classification accuracy computed on those same points; the reported selector accuracy (B.2) concerns estimator choice rather than regime prediction, and the registered NAS and pipeline forecasts are out-of-sample by design. No equation or fitted parameter is renamed as a prediction, and no decision threshold is shown to be fitted on the validation points. Therefore no circular step can be exhibited under the required standard.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The protocol introduces no new physical entities. Its central claim rests on two hand-set decision statistics and the transferability of thresholds, plus small-n exact enumeration as ground truth.

free parameters (2)
  • Regime thresholds (rho1/FDC bands) = not numerically specified; qualitative bands in Table 1
    Table 1 states they are empirical operating bands, not universal constants; no calibration holdout or threshold registration is documented, so the classification rule is hand-set and potentially fit to the validation landscapes.
  • Probe budget B_p = 100 parent-child pairs in selector experiments; otherwise 'small fixed budget'
    Protocol 1 step 1 uses a small fixed probe budget; the choice of 100 pairs for selector reliability is heuristic, and forecast accuracy depends on it.
assumptions (4)
  • domain assumption Two statistics, one-move autocorrelation and FDC, are sufficient to classify permutation search difficulty
    Central premise of PRISM (Section 3, Protocol 1); the paper provides no proof that these two statistics determine whether structured search beats random sampling.
  • ad hoc to paper Empirical decision bands transfer across landscapes
    Table 1 thresholds are called empirical operating bands, not universal constants; transferability is assumed, not demonstrated on a threshold-calibration holdout.
  • domain assumption Exact ground truth at n=6 to n=8 supports claims about larger spaces
    Limitations (Section 10) concede evidence is concentrated at small n; the protocol's broader applicability assumes the small-n landscapes are representative.
  • domain assumption Language-model responses are reproducible under deterministic decoding and frozen pools
    The distinct-evaluation budget and cached response records assume reproducible evaluations; nondeterminism or API drift would change hit counts and transfer results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PRISM: A Predictive Protocol for Permutation Optimization via Landscape Diagnostics." pith.science (2026). https://pith.science/paper/FGE2G5PE

@misc{pith2026260808344,
  author       = {Pith},
  title        = {Pith review of: PRISM: A Predictive Protocol for Permutation Optimization via Landscape Diagnostics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FGE2G5PE}},
  note         = {Machine review of arXiv:2608.08344}
}
read the original abstract

Permutation optimization arises whenever the components of a system are fixed but their ordering affects performance. We introduce PRISM, a predictive protocol for permutation optimization that measures a fitness landscape before selecting a search strategy. PRISM uses inexpensive landscape diagnostics, including one-step move autocorrelation and fitness-distance correlation, to predict useful mutation operators, identify when structured search is likely to outperform random sampling, and detect regimes in which search provides little advantage. Across synthetic permutation landscapes, neural architecture benchmarks, scientific machine learning pipelines, and large-language-model instruction ordering, the protocol makes testable predictions about search behavior before optimization begins. Exhaustive instruction-ordering experiments reveal substantial performance variation induced solely by permutation, while cross-model experiments show that useful ordering structure can transfer across model families and task difficulty. Additional experiments demonstrate that instruction ordering remains consequential after prompt wording is optimized, indicating that content optimization and ordering optimization are complementary. The results position PRISM not as a universally superior optimizer, but as a framework for determining when permutation search is useful, which representation and operator should be used, and when simpler alternatives are preferable.

Figures

Figures reproduced from arXiv: 2608.08344 by the authors.

Figure 1
Figure 1. Order alone moves measured accuracy by 90.6 percentage points. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Objects required for a PRISM study. The landscape is not merely the search-space size: it is the scored structure induced by the evaluator and the chosen neighbourhood operators. The empirical landscape regimes are visualized in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Operator-to-landscape alignment. The conceptual illustration pairs absolute position with swap, precedence with insertion, and adjacency with inversion. It is a schematic, not a measured surface; the observed censoring and locality statistics are reported in the text and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The two pre-flight axes separate guiding, near-random, and deceptive regimes. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Parity is the central falsification case. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: ANSWER is favoured late and COMPUTE early. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Ordering structure transfers and is complementary to wording optimization. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Registered NAS forecast calibration by dataset and stratum. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Evolutionary executor loop. A population is evaluated, its best member is retained, tournament selection chooses parents, and the selected mutation operator produces the next generation. The convergence statements below apply to this executor, not to enumeration or uni…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 9 canonical work pages

  1. [1]

    Correlated and Uncorrelated Fitness Landscapes and How to Tell the Difference

    Edward Weinberger. “Correlated and Uncorrelated Fitness Landscapes and How to Tell the Difference”. In:Biological Cybernetics63 (1990), pp. 325–336.doi:10.1007/BF00202749

    work page doi:10.1007/bf00202749 1990

  2. [2]

    Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms

    Terry Jones and Stephanie Forrest. “Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms”. In:Proceedings of the Sixth International Conference on Genetic Algorithms. Morgan Kaufmann, 1995, pp. 184–192

    work page 1995

  3. [3]

    On Fitness Landscape Analysis of Permutation Problems: From Distance Metrics to Mutation Operator Selection

    Vincent A. Cicirello. “On Fitness Landscape Analysis of Permutation Problems: From Distance Metrics to Mutation Operator Selection”. In:Applied Sciences12.19 (2022), p. 9976.doi:10.3390/app12199976. arXiv:2208.11188 [cs.NE]

    work page Pith review arXiv 2022

  4. [4]

    Convergence Analysis of Canonical Genetic Algorithms

    Günter Rudolph. “Convergence Analysis of Canonical Genetic Algorithms”. In:IEEE Transactions on Neural Networks5.1 (1994), pp. 96–101.doi:10.1109/72.265964

    work page doi:10.1109/72.265964 1994

  5. [5]

    Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. “Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity”. In:Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2022, pp. 8086–8098...

    work page 2022

  6. [6]

    Calibrate Before Use: Improving Few-Shot Performance of Language Models

    Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. “Calibrate Before Use: Improving Few-Shot Performance of Language Models”. In:Proceedings of the 38th International Conference on Machine Learning. Vol. 139. Proceedings of Machine Learning Research. PMLR, 2021, pp. 12697–12706. url:https://proceedings.mlr.press/v139/zhao21c.html

    work page 2021

  7. [7]

    Lost in the Middle: How Language Models Use Long Contexts

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. “Lost in the Middle: How Language Models Use Long Contexts”. In:Transactions of the Association for Computational Linguistics12 (2024), pp. 157–173.doi:10.1162/tacl_a_00638

    doi:10.1162/tacl_a_00638 2024

  8. [8]

    Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying about Prompt Formatting

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. “Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying about Prompt Formatting”. In: The Twelfth International Conference on Learning Representations. 2024.url:https://openreview. net/forum?id=RIu5lyNXjT

    work page 2024

Show all 21 references
  1. [9]

    Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

    Pouya Pezeshkpour and Estevam Hruschka. “Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions”. In:Findings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, 2024, pp. 2006–2017.doi:10.186...

    2024 doi

  2. [10]

    Large Language Models Are Not Robust Multiple Choice Selectors

    Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. “Large Language Models Are Not Robust Multiple Choice Selectors”. In:The Twelfth International Conference on Learning Representations. 2024.url:https://openreview.net/forum?id=shr9PXz7T0

    2024

  3. [11]

    Hyunjong Ok and Jaeho Lee.Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models. 2026. arXiv:2601.14152 [cs.CL]

    2026 arXiv

  4. [12]

    The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning

    Yue Zhou, Henry Peng Zou, Barbara Di Eugenio, and Yang Zhang. “The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning”. In:Findings of the Association for Computational Linguistics: EACL 2026. Association for Computational Linguistics, 2026, pp. 4175–4185....

    2026

  5. [13]

    Le, and Denny Zhou.Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou.Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. 2022. arXiv: 2201.11903 [cs.CL]

    2022 arXiv

  6. [14]

    Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.Self-Consistency Improves Chain of Thought Reasoning in Language Models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.Self-Consistency Improves Chain of Thought Reasoning in Language Models. 2022. arXiv:2203.11171 [cs.CL]

    2022 arXiv

  7. [15]

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba.Large Language Models Are Human-Level Prompt Engineers. 2022. arXiv:2211.01910 [cs.LG]

    2022 arXiv

  8. [16]

    Le, Denny Zhou, and Xinyun Chen

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large Language Models as Optimizers. 2023. arXiv:2309.03409 [cs.LG]

    2023 arXiv

  9. [17]

    Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts.DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vard- hamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts.DSPy: Compiling Declarative Language Model Calls into Self-Impr...

    2023 arXiv

  10. [18]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rei Nakano, Christopher Hesse, and John Schulman.Training Verifiers to Solve Math Word Problems. 2021. arXiv:2110.14168 [cs.LG]

    2021 arXiv

  11. [19]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.Measuring Mathematical Problem Solving With the MATH Dataset. 2021. arXiv: 2103.03874 [cs.LG]

    2021 arXiv

  12. [20]

    Discovering Governing Equations from Data by Sparse Identification of Nonlinear Dynamical Systems

    Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. “Discovering Governing Equations from Data by Sparse Identification of Nonlinear Dynamical Systems”. In:Proceedings of the National Academy of Sciences113.15 (2016), pp. 3932–3937.doi:10.1073/pnas.1517384113

    2016 doi

  13. [21]

    Xuanyi Dong and Yi Yang.NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search. 2020. arXiv:2001.00326 [cs.CV]. 12 Figure 9: Evolutionary executor loop.A population is evaluated, its best member is retained, tournament selection chooses parents, and the ...

    2020 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.