Pith. sign in

REVIEW 3 major objections 4 minor 38 references

LLM-generated optimization models can be verified soundly — with a zero false-positive rate by design — by probing every numeric quantity as a typed slot, so a failed test is a certificate of unfaithfulness.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:20 UTC pith:AAMOMBLH

load-bearing objection Genuinely new sound falsification battery for LLM-generated MILPs, with real proofs and a transparent probe-scale caveat the abstract overstates. the 3 major comments →

arxiv 2607.16646 v1 pith:AAMOMBLH submitted 2026-07-18 cs.SE cs.AI

Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits

Haifeng Li , Mo Hai This is my paper
classification cs.SE cs.AI MSC 90C0590C1168Q60
keywords falsification-based verificationoptimization modelinglarge language modelsmetamorphic testinglinear programming dualitycomparative staticssound test batterydetection limits
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models that translate word problems into optimization models fail silently: the generated model runs and still formulates the wrong problem. This paper claims that such failures can be caught soundly, without any reference model or label, by treating every numeric quantity in the description as a typed slot (capacity, requirement, rate, cost, reward, ratio) and testing the candidate only through solver calls on slot-transformed instances. The battery's test classes — directions, curvature, crush probes, prohibitive limits, annihilation, and exchange — are each proved sound: any candidate faithful to the asserted slot roles passes, so a violation is a machine-checkable certificate of unfaithfulness and the false-positive rate is zero by construction. The paper also maps what this style of verification can never see (uniform objective rescaling, orbit-redundant constraints, single-slot linear misuse) and proves that fixed-threshold perturbation testers cannot be both sound and nontrivial. If the claims hold, generated optimization models can be checked against the structure of their own text description using only solver calls, with certified flags rather than heuristic alarms.

Core claim

The central claim is Theorem 1: in exact arithmetic, every Layer-A test with a certified precondition passes on every structure-faithful candidate, and Layer-B tests pass on every mixed-integer linear program; hence a violation certifies unfaithfulness and the battery's false-positive rate is zero by construction. The claim's force is that verification needs no reference model, label, or human: only solver calls on slot-transformed instances. The paper further proves detection limits — uniform positive objective scaling is invisible and decision-irrelevant, orbit-redundant rows are invisible relative to the probe orbit, and linear misuse of a single slot is invisible to single-slot probes bu

What carries the argument

The carrying object is the assertion set A — the typed slot table extracted from the natural-language description, where each numeric quantity's semantic role and entity are declared — against which role-faithfulness is defined. The identities that do the work are necessary conditions from LP theory: monotone comparative statics of the value function in right-hand sides and objective coefficients (via set inclusion and pointwise monotonicity), convexity/concavity of the LP value function (via duality), certified infeasibility at crush probes, the prohibitive-limit equality for pricing an activity out of the model (via a polyhedral finite-threshold argument), annihilation exactness, and excha

Load-bearing premise

The deployment-mode guarantee holds only if the typed slot assertions extracted from the natural-language text are correct — a slot misclassified by the extraction pass can make a faithful model flagged or an unfaithful one pass — and the implemented prohibitive-limit test is additionally sound only when the true polyhedral threshold lies within the probed scale range (M̄ ≤ 5M2 − 4M1), a condition the verifier does not certify at runtime.

What would settle it

Take a hand-written two-variable LP whose declared capacity slot is always slack over the battery's probe orbit, add a row that binds only far outside the orbit, and run the full battery: if the faithful model is flagged (false positive) or an omitted-orbit-redundant row passes, the orbit-relativity claim fails. More sharply, a single role-faithful MILP with a certified precondition on which a Layer-A predicate fails in exact arithmetic — for instance a cost-slot prohibitive limit where v(M) never equals v_e0 beyond any finite M — would falsify Theorem 1.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A failed Layer-A test with a certified precondition is a machine-checkable proof that the candidate is not faithful to the asserted slot roles, enabling deployment-time verification with no reference model.
  • Execution-accuracy scoring is blind in both directions: 40.4% of mutants that match the labeled optimal value are still convicted by the battery, so verification and evaluation are genuinely different problems.
  • Threshold-based presence testers have false-positive rates structurally bounded away from zero: for every relative threshold there exist faithful models that are flagged, and no threshold choice is simultaneously sound and nontrivial.
  • The battery doubles as a diagnostic: each failed test names the offending slot and property, and the reproducibility of the predicted detectability pattern (including its zeros) makes the verdicts usable as certificates rather than opinions.
  • The same machinery doubles as a benchmark auditor, isolating label and annotation defects in public benchmarks as a by-product.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The framework is a template: any parametric optimization family with provable necessary conditions (for example, convex programs with known value-function properties) can admit new sound test classes, so the battery can be extended beyond MILPs.
  • Because flags carry certificates with the offending slot and property, a natural next step is coupling verification to repair — steering regeneration or targeted patching instead of full re-prompting — which the paper explicitly leaves orthogonal.
  • The assertion-extraction dependency suggests a testable extension: measure battery soundness under adversarially mis-extracted slot tables; if extraction errors are systematic, battery flags could serve as feedback to refine extraction, making the false-positive guarantee conditional on a pass that the battery itself helps validate.
  • Orbit design becomes the practical lever: enlarging probe orbits to activate near-redundant rows can shrink the provable blind set, so verification power is tunable by adding certified probes rather than by relaxing soundness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops an oracle-free, falsification-based verification method for LLM-generated optimization models. It treats every numeric quantity in a problem description as a typed slot, maps a candidate MILP to a parametric family, and tests the candidate by solving slot-transformed instances and checking necessary conditions supplied by LP duality, comparative statics, and polyhedral limit theory. The theoretical core is a battery of test classes (directional, curvature, crush, prohibitive-limit, annihilation, exchange, and assertion-free Layer B), a soundness theorem, a characterization of invisible errors, an impossibility result for threshold-based presence testers, and a class-by-error detectability matrix. Experiments on NL4OPT and three other benchmark families report 0.0% false positives on 326 faithful seeds, substantial detection of certified mutants, and reproduction of the predicted detectability pattern. A two-pass LLM interface makes data binding explicit via an extracted typed slot table and a slot-referencing JSON IR.

Significance. This is a valuable and largely rigorous contribution. Its main strength is replacing heuristic threshold testing with provably sound necessary conditions, and complementing the positive results with a formal treatment of what oracle-free verification cannot see. The exact-arithmetic proofs are mostly complete, the machine-checkable/reproducible code and data artifacts are a real asset, and the detectability matrix yields falsifiable predictions that the experiments confirm, including the zeros. The central 'zero false positives by design' claim, however, is currently stated more strongly than the implemented A5 test warrants, and the deployment-mode guarantee is conditional on an unquantified assertion-extraction step. These issues are acknowledged in parts of the text but are not propagated to the abstract and Theorem 1, so the headline guarantee is overstated relative to the shipped implementation.

major comments (3)
  1. [§4.3, Remark 1, EC.1.14, Theorem 1(ii)] The implemented A5 prohibitive-limit rule is sound only under the probe-scale condition M̄ ≤ 5M2 − 4M1. The implementation does not certify or enforce this condition, and EC Table EC.3 explicitly shows faithful two-cost instances above the bound being rejected by the implemented rule. Theorem 1(ii) nevertheless states that tests A0–A3, A5–A7 pass on every role-faithful candidate whose preconditions are certified, and the abstract claims 'the false-positive rate is zero by design.' As written, Theorem 1(ii) is false for the implemented A5. This is a load-bearing gap in the central soundness guarantee. The manuscript must either implement a runtime check of the probe-scale condition (or an adaptive scale-selection rule), or restrict every soundness/zero-FP statement to the condition and report how often the condition is violated in the experimental corpora.
  2. [§6 Pass 1 and §7.6] Deployment-mode guarantees are relative to the assertion set A extracted by the first LLM pass. The paper acknowledges that extraction is 'not infallible,' but provides no measurement of extraction error, no ablation, and no upper bound. The headline 0.0% false-positive rate is measured on annotated NL4OPT seeds where A comes from ground-truth declarations, not from the extraction pass. A text-faithful candidate can therefore be convicted, or an unfaithful candidate can pass, if a slot's qtype, entity, or sense is misread. This is a central limitation for the deployment claims, not a peripheral one. I would like the authors to either report extraction quality on their benchmark pools or explicitly state in the abstract and Theorem 1 that all guarantees are conditional on a correct assertion set A.
  3. [§5.1, Proposition 8(i) and EC.1.9] Proposition 8(i) claims that a single-slot linear misuse θ_j ↦ λθ_j passes every single-slot probe, including the prohibitive-limit test. The proof asserts that the two-scale residual criterion is preserved because residuals 'vanish beyond M̄/λ and decay proportionally.' This is not correct for λ < 1. If the faithful value function has threshold M̄, the misused model has threshold M̄/λ, which is larger than M̄; the acceptance inequality r(M2) ≤ 4(r(M1) − r(M2)) is not invariant under the rescaling. A simple piecewise-linear residual g(M) = max(0, T − M) shows the criterion can fail when λ < 1 even if the original faithful model passes. The theoretical claim that single-slot misuse is invisible to all single-slot probes is therefore only true for the exact limit predicate, or for λ ≥ 1, not for the implemented finite-scale A5. The empirical M7 experiments use only λ = 10, so they do not c
minor comments (4)
  1. [EC.1.5] Typo: 'c_e >0 one's coordinates' should probably read 'all coordinates' or 'the relevant coordinates'; the current phrase is unclear.
  2. [EC Table EC.3] The table lists K = 10^10.0 three times with identical entries; this looks like a formatting or duplication error and should be cleaned up.
  3. [§7.3] The panel analysis treats M8 and M9 controls as faults in the main recall computation and only reports the M8/M9-excluded numbers as a robustness check. Since the paper itself proves M8 is decision-irrelevant and M9 is orbit-invisible, the M8/M9-excluded figures should arguably be the primary analysis, not a secondary one.
  4. [Table 3] The header/column alignment for 'Cascade' is easy to misread because on NL4OPT the cascade value equals the greedy value. A footnote explaining the fallback rules for filters (and why cascade can reduce to greedy) would help.

Circularity Check

0 steps flagged

No significant circularity: the battery's soundness and detectability results are proven theorems over an explicit role-faithfulness definition, with external empirical anchors; the acknowledged A5 and extraction caveats are correctness risks, not circular reasoning.

full rationale

The derivation chain is not circular. Definition 1 fixes role-faithfulness in terms of slot-entry positions (e.g., a capacity slot appears only as the RHS of a ≤ row), and Propositions 1–5 prove the Layer-A predicates as necessary conditions of that definition via set inclusion, LP duality, and polyhedral threshold arguments. The predicates are not restatements of the definition: role-faithfulness does not assert monotonicity, midpoint convexity, or exact prohibitive-limit equality; those are derived consequences. Theorem 1 then states a soundness theorem about the class of faithful candidates, and the 0.0% false-positive rate on the 326 NL4OPT seeds is an implementation sanity check against externally supplied ground-truth declarations, not a fitted value. The zeros in the detectability matrix (M8, M9) follow from Propositions 6–7 and are tested empirically rather than built into the definitions. The paper's own limitations are acknowledged rather than hidden: Remark 1 and EC.1.14 show the implemented A5 two-scale acceptance rule is only proven sound under M̄ ≤ 5M2 − 4M1, which is not runtime-certified, and §7.6 states that deployment-mode guarantees are relative to extracted assertions. These are genuine correctness/scope conditions but they are not circularity: they condition the guarantee on stated assumptions instead of assuming the target result. No load-bearing self-citations appear; the cited mathematical facts (Bertsimas/Tsitsiklis, Topkis, Milgrom/Roberts, etc.) are classical external results. The central theoretical contribution—a sound, oracle-free test battery with proven detection limits—is independent content, not a renaming of inputs or a self-citation chain.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 2 invented entities

The central claim rests on standard optimization theorems plus domain assumptions about how text maps to typed slots and how solver certificates are obtained. The main non-standard loads are the correctness of the extracted assertion set and the probe-scale condition for the implemented A5 limit test.

free parameters (2)
  • One-sided violation slack ε = 1e-5 (relative)
    Used in implemented predicates to avoid numerical false positives; hand-chosen, not fitted to data. It widens the pass region, affecting power more than soundness.
  • A5 two-scale probe scales (M1, M2) and contraction factor 4 = e.g., (1e6, 1e9) in synthetic validation; factor 4
    The implemented prohibitive-limit rule is sound only when the polyhedral threshold M̄ ≤ 5M2 − 4M1 (Remark 1, EC.1.14). The scales are user-chosen and bound the range of cost ratios the test can certify; faithful models outside this range can be falsely flagged.
axioms (6)
  • standard math Linear programming duality: LP value function is convex in RHS for minimization, optimal duals are subgradients, and parametric LP admits finitely many optimal bases.
    Used in Propositions 2 and 4 and Layer B tests (§4, E-Companion EC.1.2, EC.1.4).
  • standard math Feasible-set inclusion monotonicity: increasing a capacity RHS or decreasing a requirement/rate RHS enlarges the feasible set; MILP optimal values are weakly monotone under set inclusion.
    Basis of A1/A2 directional tests (Proposition 1, EC.1.1).
  • domain assumption The candidate is a parametric mixed-integer linear program of form (1), and every numeric quantity in the text is a typed slot entering only at its asserted position (role-faithfulness).
    Defines the verification problem (Definition 1). If the model has nonlinearities, uncertainty sets, or untyped numbers, the battery's predicates are not guaranteed.
  • domain assumption The assertion set A extracted from the natural-language description is correct in deployment mode.
    §6 Pass 1; §7.6 acknowledges extraction is 'not infallible.' If slot types or entities are misread, certificates and predicates are applied to the wrong structure.
  • domain assumption Certificates computed by solver calls—feasibility of x_e=0, infeasibility of crushed structure, congruence of entity bundles—are reliable and, in deployment mode, coincide with true properties for faithful candidates.
    Used in A4, A5, A7 and Theorem 1(ii). A noisy solver or wrong candidate-side certification could only reduce power for faithful candidates, but the unconditional zero-FP claim depends on this premise.
  • ad hoc to paper The implemented A5 two-scale rule is sound only under the probe-scale bound M̄ ≤ 5M2 − 4M1, which is not runtime-certified.
    Remark 1 and EC.1.14 state that data with cost ratios exceeding the probe range can defeat the rule, contradicting the abstract's unconditional 'zero by design' phrasing.
invented entities (2)
  • Typed slot table (slot identifiers, qtype, entity) no independent evidence
    purpose: Makes data binding explicit and checkable; the battery tests whether candidates respond to slot changes as the asserted roles dictate.
    A formal modeling artifact introduced in §3.1 and §6. It has no independent falsifiable handle outside the pipeline's own extraction pass; errors in it shift every downstream certificate.
  • Certified congruence / entity bundle no independent evidence
    purpose: Precondition for exchange tests A7; marks entities as interchangeable so swapping slot bundles must leave the value unchanged.
    Computed from candidate or declared structure (§4.4, Prop 5). If incorrectly certified, exchange tests could fire on faithful models; the paper relies on this certificate being correct in deployment.

pith-pipeline@v1.3.0-alltime-deepseek · 24426 in / 15714 out tokens · 158584 ms · 2026-08-01T20:20:13.377984+00:00 · methodology

0 comments
read the original abstract

Large language models now translate natural-language descriptions of decision problems into solver-ready optimization models, but they fail silently. A generated model often runs and still formulates the wrong problem. This paper develops a theory of falsification-based verification for this setting. Every numeric quantity in the description is a typed slot, and a candidate model is tested only through solver calls on slot-transformed instances; no reference model or label is consulted. From duality, comparative statics, and polyhedral limit arguments we derive a battery of test classes covering directions, curvature, crush probes, prohibitive limits, annihilation, and exchange. Every test is sound, so a violation certifies unfaithfulness and the false-positive rate is zero by design. We characterize what such verification can never see, give conditions under which the canonical error classes are detected with certainty, and prove that no fixed-threshold perturbation tester is simultaneously sound and nontrivial. Experiments on 326 ground-truth models from NL4OPT and four benchmark families confirm the theory. The battery attains a 0.0% false-positive rate against 54.9% for a threshold tester, detects 70.0% of certified conditional-class mutants, convicts 40.4% of the mutants invisible to execution-accuracy scoring, and reproduces the predicted detectability pattern including its zeros.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 15 linked inside Pith

  1. [1]

    Ramamonjison, Rindranirina and Yu, Timothy T. and Li, Raymond and Li, Haley and Carenini, Giuseppe and Ghaddar, Bissan and He, Shiqi and Mostajabdaveh, Mahdi and Banitalebi-Dehkordi, Amin and Zhou, Zirui and Zhang, Yong , title =. Proceedings of the NeurIPS 2022 Competition Track, PMLR , volume =

    2022

  2. [2]

    International Conference on Learning Representations (ICLR) , year =

    Xiao, Ziyang and Zhang, Dongxiang and Wu, Yangjun and Xu, Lilin and Wang, Yuan Jessica and Han, Xiongwei and Fu, Xiaojin and Zhong, Tao and Zeng, Jia and Song, Mingli and Chen, Gang , title =. International Conference on Learning Representations (ICLR) , year =

  3. [3]

    International Conference on Machine Learning (ICML) , year =

    AhmadiTeshnizi, Ali and Gao, Wenzhi and Udell, Madeleine , title =. International Conference on Machine Learning (ICML) , year =

  4. [4]

    Operations Research , year =

    Huang, Chenyu and Tang, Zhengyang and Hu, Shixi and Jiang, Ruoqing and Zheng, Xin and Ge, Dongdong and Wang, Benyou and Wang, Zizhuo , title =. Operations Research , year =

  5. [5]

    International Conference on Learning Representations (ICLR) , year =

    Jiang, Caigao and Shu, Xiang and Qian, Hong and Lu, Xingyu and Zhou, Jun and Zhou, Aimin and Yu, Yang , title =. International Conference on Learning Representations (ICLR) , year =

  6. [6]

    arXiv preprint arXiv:2502.11102 , year =

    Lu, Hongliang and Xie, Zhonglin and Wu, Yaoyu and Ren, Can and Chen, Yuxuan and Wen, Zaiwen , title =. arXiv preprint arXiv:2502.11102 , year =

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2505.11792 , year =

    Chen, Yitian and Xia, Jingfan and Shao, Siyu and Ge, Dongdong and Ye, Yinyu , title =. arXiv preprint arXiv:2505.11792 , year =

    arXiv

  8. [8]

    arXiv preprint arXiv:2602.15983 , year =

    Lian, Junbo Jacob and Sun, Yujun and Chen, Huiling and Zhang, Chaoyu and Qin, Hanzhang and Teo, Chung-Piaw , title =. arXiv preprint arXiv:2602.15983 , year =

    Pith/arXiv arXiv

  9. [9]

    arXiv preprint arXiv:2605.27630 , year =

    Xu, Yujia and Wang, Zhiheng and Dinh, Thi , title =. arXiv preprint arXiv:2605.27630 , year =

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2510.27610 , year =

    Wang, Zhuohan and Zhu, Ziwei and Li, Ziniu and Chen, Congliang and others , title =. arXiv preprint arXiv:2510.27610 , year =

    arXiv

  11. [11]

    arXiv preprint arXiv:2605.23966 , year =

    Fang, Ziyang and Wang, JinXi and Zhong, Jinghui and Ong, Yew-Soon , title =. arXiv preprint arXiv:2605.23966 , year =

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2605.29556 , year =

    Liu, Haoyang and Wang, Jie and Niu, Boxuan and Han, Xiongwei and others , title =. arXiv preprint arXiv:2605.29556 , year =

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2605.11738 , year =

    Li, Zhong and Guo, Zihan and Lu, Xiaohan and Wang, Juntao and others , title =. arXiv preprint arXiv:2605.11738 , year =

    Pith/arXiv arXiv

  14. [14]

    International Conference on Learning Representations (ICLR) , year =

    Huang, Jie and Chen, Xinyun and Mishra, Swaroop and Zheng, Huaixiu Steven and Yu, Adams Wei and Song, Xinying and Zhou, Denny , title =. International Conference on Learning Representations (ICLR) , year =

  15. [15]

    and Ramamonjison, Rindranirina and Carenini, Giuseppe and Zhou, Zirui and Zhang, Yong , title =

    Mostajabdaveh, Mahdi and Yu, Timothy T. and Ramamonjison, Rindranirina and Carenini, Giuseppe and Zhou, Zirui and Zhang, Yong , title =. INFOR: Information Systems and Operational Research , volume =

  16. [16]

    arXiv preprint arXiv:2604.00442 , year =

    Guan, Runda and Shen, Xiangqing and Zhang, Jiajun and Zhang, Yifan and others , title =. arXiv preprint arXiv:2604.00442 , year =

    arXiv

  17. [17]

    arXiv preprint arXiv:2606.04816 , year =

    Luo, Xizi and He, Changhong and Geng, Dongdong and Shi, Chenggong and others , title =. arXiv preprint arXiv:2606.04816 , year =

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2605.02545 , year =

    Zhao, Ruiqing and Li, Fengzhi and Zuo, Yuan and Liu, Rui and Liu, Yansong and Ma, Yunfei and Meng, Fanyu and Feng, Junlan , title =. arXiv preprint arXiv:2605.02545 , year =

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2606.25832 , year =

    Zhao, Ke and Di, Zixiang and Qian, Hong and Shu, Xiang and others , title =. arXiv preprint arXiv:2606.25832 , year =

    Pith/arXiv arXiv

  20. [20]

    arXiv preprint arXiv:2405.13144 , year =

    Huang, Xuhan and Shen, Qingning and Hu, Yan and Gao, Anningzhe and Wang, Benyou , title =. arXiv preprint arXiv:2405.13144 , year =

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2407.09887 , year =

    Yang, Zhicheng and Wang, Yiwei and Huang, Yinya and Guo, Zhijiang and Shi, Wei and Han, Xiongwei and Feng, Liang and Song, Linqi and Liang, Xiaodan and Tang, Jing , title =. arXiv preprint arXiv:2407.09887 , year =

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2602.22465 , year =

    Tso, Joseph and Schmittou, Preston and Huynh, Quan and Hutchins, Jibran , title =. arXiv preprint arXiv:2602.22465 , year =

    arXiv

  23. [23]

    arXiv preprint arXiv:2505.21775 , year =

    Klamkin, Michael and Deza, Arnaud and Cheng, Sikai and Zhao, Haoruo and others , title =. arXiv preprint arXiv:2505.21775 , year =

    Pith/arXiv arXiv

  24. [24]

    and Li, Can , title =

    Chen, Hao and Constante-Flores, Gonzalo E. and Li, Can , title =. INFORMS Journal on Data Science , year =

  25. [25]

    arXiv preprint arXiv:2511.16383 , year =

    Zadorojniy, Alexander and Wasserkrug, Segev and Farchi, Eitan , title =. arXiv preprint arXiv:2511.16383 , year =

    Pith/arXiv arXiv

  26. [26]

    Chen, Tsong Yueh and Kuo, Fei-Ching and Liu, Huai and Poon, Pak-Lok and Towey, Dave and Tse, T. H. and Zhou, Zhi Quan , title =. ACM Computing Surveys , volume =

  27. [27]

    and Ruiz-Cort

    Segura, Sergio and Fraser, Gordon and Sanchez, Ana B. and Ruiz-Cort. A survey on metamorphic testing , journal =

  28. [28]

    arXiv preprint arXiv:2406.06864 , year =

    Wang, Xiaoyin and Zhu, Dakai , title =. arXiv preprint arXiv:2406.06864 , year =

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2406.05397 , year =

    Li, Rui and Liu, Huai and Poon, Pak-Lok and Towey, Dave and Chen, Tsong Yueh and Zhou, Zhi Quan , title =. arXiv preprint arXiv:2406.05397 , year =

    Pith/arXiv arXiv

  30. [30]

    IEEE Transactions on Software Engineering , volume =

    Jia, Yue and Harman, Mark , title =. IEEE Transactions on Software Engineering , volume =

  31. [31]

    , title =

    Bertsimas, Dimitris and Tsitsiklis, John N. , title =

  32. [32]

    , title =

    Vanderbei, Robert J. , title =

  33. [33]

    , title =

    Topkis, Donald M. , title =

  34. [34]

    American Economic Review , volume =

    Milgrom, Paul and Roberts, John , title =. American Economic Review , volume =

  35. [35]

    Symmetry in integer linear programming , booktitle =

    Margot, Fran. Symmetry in integer linear programming , booktitle =

  36. [36]

    , title =

    Chinneck, John W. , title =

  37. [37]

    and Maragno, Donato and den Hertog, Dick , title =

    Fajemisin, Adejuyigbe O. and Maragno, Donato and den Hertog, Dick , title =. European Journal of Operational Research , volume =

  38. [38]

    European Journal of Operational Research , volume =

    Sadana, Utsav and Chenreddy, Abhilash and Delage, Erick and Forel, Alexandre and Frejinger, Emma and Vidal, Thibaut , title =. European Journal of Operational Research , volume =