REVIEW 5 major objections 4 minor 1 cited by
Performance report of heuristic algorithm that cracked the largest Gset Ising problems (G81 cut=14060)
T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A heuristic algorithm named Cosm reports record Max-Cut values on the three largest Gset instances — G72 at 7008, G77 at 9940, G81 at 14060 — reaching 99.9% quality hundreds to thousands of times faster than prior heuristics.
desk verdict New best cuts on Gset G72/77/81 with verifiable bitstrings is a real, checkable claim; the speed claims are the soft underbelly. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is Cosm itself, an iterative heuristic defined functionally here (its design is promised in a forthcoming publication), whose fundamental step is a single sweep; the target problems are weighted Max-Cut instances on a toroidal square grid with weights $\{-1, +1\}$, a structure known to be NP-hard. For the speed claims, the carrying identity is the repetition formula $r = \max\left(1,\ \frac{\log(0.01)}{\log(1-P_s)}\right)$, which converts a per-trial success probability $P_s$ into the number of independent trials needed to reach a target cut with 99% probability; sweeps-to-target and time-to-target are this $r$ times the per-trial sweep count and per-trial runtime. The record cuts themselves are carried by the published solution bitstrings, which let any reader plug bits into the Max-Cut objective and check 7008, 9940, and 14060 directly.
What would settle it
Take the G81 bitstring from Appendix E, expand it to 20,000 bits, and evaluate the weighted Max-Cut objective on the G81 instance from the Gset collection: if the cut is not 14060, the record claim fails. For the speed claims, once the algorithm is published, run several thousand independent trials at 3 million sweeps per trial; a success rate for cut 14060 significantly below 3/100 would invalidate the extrapolated time-to-target. For the optimality claim, compare the three bitstring cuts with the Gurobi-certified optimal values when they are released.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that an iterative heuristic named Cosm, whose fundamental operation is a single 'sweep' over the variables, attains higher cuts than ever previously reported on the three largest Gset instances: G72 cut=7008, G77 cut=9940, and G81 cut=14060, with corresponding Ising energies −14022, −19672, and −28086. Cosm also reaches 99.9% of the best known solution quality on G77 in 39 seconds and on G81 in 78 seconds, against published GES-PR baselines of 7 hours and 77 hours, respectively. The author states that the best solutions appear to be optimal and expects them to match Gurobi-certified optimal solutions once those are published, and includes solution bitstrings in appendices so the cuts can be independently validated. For the speed claims, success probabilities measured over 100 trials per instance are converted through a geometric-trial formula into sweeps-to-target and time-to-target; with a projected 2 ns per sweep on parallel hardware, 99.9% quality would be reached in about a millisecond.
Load-bearing premise
The speed and speedup claims, computed in Section III, rest on the premise that the success probabilities measured from 100 trials per instance (as low as 3/100 for the G81 record cut) describe a true, constant per-trial success rate with independent trials, against a comparison baseline taken from published results on a different CPU and implementation.
Editorial extensions
If this is right
- Independent verification of the three bitstrings would make G72, G77, and G81 the first instances of their size in the Gset family to be solved at or near optimality by a heuristic algorithm.
- The reported speedups imply that 99.9% solution quality on the largest Gset instances is reachable in seconds on a laptop, where the previous best heuristic needed hours.
- A parallelized Cosm implementation at a projected 2 ns per sweep would reach 99.9% quality in about a millisecond and the best-known cuts within about a second, a regime that would reset expectations for Ising-machine hardware.
- The improved G72 and G77 results over the author's earlier report are attributed mainly to resetting a single solver parameter using a more representative 5,000-variable tuning problem, suggesting the tuning protocol matters as much as the search moves.
Reading between the lines
- The speed comparison is not a controlled benchmark: the GES-PR times were published for a different CPU and code base, so the true speedup on identical hardware is untested; re-running GES-PR on the same machine would settle the gap.
- Because a single parameter change yielded 5–6x better sweeps-to-target, the algorithm's behavior appears highly sensitive to tuning, so the reported record probabilities (3/100 on G81) may shift across implementations and machines until Cosm is fully specified and stable.
- If the 14060 cut is confirmed optimal, the three largest Gset instances lose much of their power to separate heuristics from exact solvers, and the natural next test is whether Cosm transfers to larger, denser, or more heavily weighted Ising instances where optimality is not yet certified.
- The closing suggestion that disruptive performance can come from hardware-centric algorithm discovery is a research program rather than a demonstrated result; a direct test would be whether Cosm-style search also wins on problem families unrelated to toroidal grids.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reports that a heuristic algorithm called Cosm, implemented as a MATLAB proof-of-concept, attains higher Max-Cut values than previously reported on the three largest Gset instances: G72 (cut 7008), G77 (cut 9940), and G81 (cut 14060). The evidence for these numerical claims consists of three hexadecimal solution bitstrings in Appendices C-E. The paper also reports success probabilities, sweeps-to-target, and time-to-target for reaching 99.9% and 100% of these best cuts, and compares these values with published GES-PR time-to-target results to claim speedups of 655x (G77) and 3560x (G81). The abstract and Section III further state that the new best solutions 'appear to be optimal,' pending confirmation from an unpublished Gurobi run referenced as [13]. The Cosm algorithm itself is not described; the manuscript states it 'is to be published shortly after this report.'
Significance. If the three reported cut values are correct, this is a significant empirical result for a benchmark family that has been widely studied for 25 years. The decision to provide solution bitstrings is commendable because it makes the central numerical claims externally checkable without trusting the implementation. However, the absence of a verification script or checksum, the lack of an algorithm description, and the uncontrolled hardware comparison in the speed claims substantially limit the reproducibility of the paper's performance claims. The paper is a performance report rather than a complete algorithmic study; its value as a standalone publication depends on whether the reviewer and editor accept this format. The manuscript itself explicitly asks readers to perform validation, which is appropriate, but it does not provide the tooling to do so.
major comments (5)
- [Section III, Appendices C-E] The central cut claims (G72=7008, G77=9940, G81=14060) rest entirely on the three hex bitstrings, yet the manuscript provides no verification script, checksum, or a separate independently computed value of the objective function. A single transcription or line-wrap error would invalidate the headline result while leaving the surrounding narrative internally consistent. The paper should either (a) provide a machine-readable solution file and/or a short evaluation script, or (b) include an independently computed cut value or signature (e.g., a hash of the bitstring with a stated cut value). Without this, the principal numerical claims are not verifiable from the manuscript as submitted.
- [Section III, Methodology] The Cosm algorithm is not described. The text says the 'algorithm is to be published shortly after this report,' but the abstract and Section III make strong empirical claims that depend on the algorithm's details: solver parameters, neighbor structure, termination criteria, and the 'single solver parameter' whose improved setting is credited for 5-6x better sweeps-to-target. Without at least a high-level algorithmic description or pseudocode, the reported speed and quality results cannot be reproduced or independently assessed. This is a load-bearing omission for a paper whose contribution is a heuristic performance report.
- [Section III, Speed and Table I] The reported speedups against GES-PR (655x for G77, 3560x for G81) are not a controlled comparison. Table I compares Cosm-MATLAB TTT measured on an Intel Core Ultra 7 155H laptop to GES-PR TTT published for an Intel i7-3770 CPU, and the implementations differ in language, parallelization, and code maturity. The manuscript acknowledges this indirectly by calling Cosm-MATLAB a 'proof-of-concept,' but the abstract's 'orders of magnitude faster' claim is nonetheless presented as a headline result. A fair comparison would require rerunning both solvers on the same hardware or at least reporting normalized timings.
- [Section III, Equation (3) and Table I] The sweeps-to-target and time-to-target estimates rely on Eq. (3), which assumes independent trials with a constant success probability p_s. The success probabilities in Table I are estimated from only 100 trials (e.g., 3/100 for G81 100% cut), so the 99%-success-level extrapolation (r = log(0.01)/log(1-p_s)) has very wide confidence intervals. For the 3/100 case, r is 151, but the 95% confidence interval for p_s ranges from about 0.006 to 0.085, giving r between roughly 53 and 765. The paper does not report such uncertainties, making the stated sweeps-to-target (454M) and projected TTT (910 ms at 2 ns/sweep) appear spuriously precise.
- [Section III, Optimality] The abstract states that the new best solutions 'appear to be optimal,' but the the only support is an informal expectation about an unpublished Gurobi run in Ref. [13], which itself has not released solutions. The manuscript says 'It is our understanding and expectation... awaits official confirmation.' This is not evidence of optimality. The claim should be either removed from the abstract or explicitly labeled as an unverified conjecture, especially since a 'second-best' cut of 14058 is 99.986% of 14060 and the paper cites a common threshold for optimality [15] that is not met by 14058.
minor comments (4)
- [Section III, Equations (4), (5)] The definitions of STT and TTT use a variable name 't_t_trial' that is not defined in the text; it should be clarified as the average execution time per trial, which is then used in Table I.
- [Table I, G72 row] The 'State-of-the-art TTT from GES-PR' column says 'Not enough data' for G72 99.9% but then reports a speedup figure of 655x for G77 and 3560x for G81. For G72, no speedup is stated, which is fine, but the column header should be split or annotated to indicate that the 'Not enough data' entry means no comparison was made.
- [Appendix A, Figure 2] The visualization of G81 is very low resolution in the provided text and does not convey meaningful structure beyond the toroidal grid; consider replacing it with a labeled diagram showing the 100x200 periodic grid and representative edge weights.
- [Section II, References] The references list mixes arXiv preprints and peer-reviewed papers without consistent citation dates or DOIs; for example, Ref. [22] is a book chapter but is cited in text simply as 'GESPR – 2017 [22]' without a chapter or page range.
Circularity Check
No circular derivation: the reported G72/G77/G81 cuts are empirical outputs with externally checkable bitstrings, and the speed metrics are definitions from measured success probabilities.
full rationale
The paper makes no first-principles derivation that reduces to its own inputs. The central claims are (i) that Cosm attained cuts 7008, 9940, and 14060, supported by the bitstrings in Appendices C-E with instructions to evaluate objective function (1); and (ii) that time-to-target at 99.9% quality is much faster than GES-PR, computed from measured success probabilities via Eqs. (3)-(5). These are statistical extrapolations, not predictions forced by construction: p is an empirical frequency from 100 trials, and r is a defined function of p. Tuning algorithm parameters on G57 before running the large instances is a benchmark-family choice, not an equation-level reduction, and it does not determine the reported G72/77/81 cut values. The self-citation [10] is used only to compare against the author's previous implementation and does not carry the new-best-cut claim. The optimality discussion is explicitly an expectation awaiting Gurobi confirmation, not a derived consequence of the method. The algorithm itself is not yet published, which is a reproducibility limitation rather than circularity, and the bitstrings make the headline results independently checkable. No load-bearing step equates an output to an input by construction.
Assumptions & free parameters
free parameters (2)
- Cosm algorithm parameters =
not disclosed
- Number of sweeps per trial per instance =
G72: 80k (99.9%), 1.5M (100%); G77: 80k (99.9%), 2M (100%); G81: 100k (99.9%), 3M (100%)
assumptions (4)
- domain assumption The Max-Cut objective function (1) with weights in {-1,+1} is the correct formulation for the Gset instances.
- domain assumption The Gset instances G72, G77, and G81 are as publicly defined in Ref [1].
- domain assumption Trials are independent and have constant success probability, so Eq. (3) correctly computes the number of repetitions for 99% confidence.
- ad hoc to paper The G57 tuning instance is representative of the largest Gset problems, so the tuned parameters transfer to G72/77/81.
Cite this review
Pith. "Pith review of Performance report of heuristic algorithm that cracked the largest Gset Ising problems (G81 cut=14060)." pith.science (2026). https://pith.science/paper/C7MOWV7L
@misc{pith2026250518508,
author = {Pith},
title = {Pith review of: Performance report of heuristic algorithm that cracked the largest Gset Ising problems (G81 cut=14060)},
year = {2026},
howpublished = {\url{https://pith.science/paper/C7MOWV7L}},
note = {Machine review of arXiv:2505.18508}
}
read the original abstract
For the past 25 years, the Gset benchmark problems have challenged all manner of Ising and Max-Cut solvers. The largest of these problems have remained unsolved by any heuristic algorithm. In this report we provide data showing dramatically better speed and accuracy on these large sparse problems. Our newly discovered heuristic algorithm called Cosm reaches high (99.9% of best) solution quality orders of magnitude faster than the previous best heuristic solver results. Additionally, when afforded enough steps Cosm attains higher cuts than ever previously reported, specifically on instances G72 (cut=7008), G77 (cut=9940), and the 20,000-variable G81 (cut=14060). This report includes solution bitstrings so that the cuts can be independently validated. Remarkably, the new best solutions appear to be optimal. We believe the results are an early hint of disruptive opportunities for unconventional, hardware-centric approaches to algorithm discovery.
Figures
Forward citations
Cited by 1 Pith paper
-
Cosm: Collective Switched Motion for Fast and Accurate Sparse Ising Optimization
Cosm finds certified optimal cuts on Gset G72/G77/G81 and reduces best-known times-to-target on G61/G70 from hundreds of hours to 36–303 s via switched circular dynamics.
Reference graph
Works this paper leans on
-
[13]
K. Kalinin et al., “Analog Iterative Machine (AIM): using light to solve quadratic optimization problems with mixed variables,” arXiv:2304.12594, 2023. 4
arXiv 2023
-
[15]
On the emerging potential of quantum annealing hardware for combinatorial optimization ,
B. Tasseff, T. Albash, Z. Morrell, et al., “On the emerging potential of quantum annealing hardware for combinatorial optimization ,” J Heuristics 30, 325–358 (2024)
work page 2024
- [1]
-
[2]
Mixed linear and semidefinite programming for combinatorial and quadratic optimization,
S. J. Benson, Y. Yeb an d X. Zhang, “Mixed linear and semidefinite programming for combinatorial and quadratic optimization,” Optimization Methods and Software , 11(1–4), 515 – 544, 1999
work page 1999
-
[3]
Solving Sparse Semidefinite Programs Using the Dual Scaling Algorithm with an Iterative Solver,
C. Choi and Y. Ye, “Solving Sparse Semidefinite Programs Using the Dual Scaling Algorithm with an Iterative Solver,” manuscript, Department of Management Sciences, University of Iowa, Iowa City, IA 52242, USA, 2000. https://web.stanford.edu/~yyye/yyye/cgsdp1.pdf
work page 2000
-
[4]
Spinto: High -performance energy minimization in spin glasses,
H. J. García and I. L. Markov, “ Spinto: High -performance energy minimization in spin glasses, ” 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE 2010), Dresden, Germany, pp. 160-165, 2010
work page 2010
-
[5]
Rank-two relaxation heuristics for MAX CUT and other binary quadratic programs ,
S. Burer, R.D.C. Monteiro, Y. Zhang, “Rank-two relaxation heuristics for MAX CUT and other binary quadratic programs ,” SIAM J. on Optimization, 12: 503–521, 2002
work page 2002
-
[6]
Performance of quantum annealing inspired algorithms for combinatorial optimization problems,
QG. Zeng, XP. Cui, B. Liu, et al., “Performance of quantum annealing inspired algorithms for combinatorial optimization problems,” Commun Phys 7, 249 (2024)
work page 2024
Show all 24 references
-
[7]
Breakout Local Search for the Max -Cut problem,
U. Benlic and J.- K. Hao, “Breakout Local Search for the Max -Cut problem,” Engineering Applications of Artificial Intelligence, vol. 26, no. 3, March 2013
2013
-
[8]
Ising machines as hardware solvers of combinatorial optimization problems,
N. Mohseni, P. McMahon and T. Byrnes, “Ising machines as hardware solvers of combinatorial optimization problems,” Nature Reviews Physics, vol. 4, June 2022
2022
-
[9]
High -performance combinatorial optimization based on classical mechanics,
H. Goto et al., “High -performance combinatorial optimization based on classical mechanics,” Science Advances, vol. 7, no. 6, 2021
2021
-
[10]
Improved Sparse Ising Optimization,
K. M. Zick, “Improved Sparse Ising Optimization,” arXiv:2311.09275, 2023
2023 arXiv
-
[11]
McSparse: Exact Solutions of Sparse Maximum Cut and Sparse Unconstrained Binary Quadratic Optimization Problems ,
J. Charfreitag, M. Jünger, S. Mallach and P. Mutzel, “McSparse: Exact Solutions of Sparse Maximum Cut and Sparse Unconstrained Binary Quadratic Optimization Problems ,” 2022 Proceedings of the Symposium on Algorithm Engineering and Experiments (ALENEX), p. 54-66, 2022
2022
-
[12]
Faster exact solution of sparse MaxCut and QUBO problems ,
D. Rehfeldt, T. Koch and Y. Shinano, “Faster exact solution of sparse MaxCut and QUBO problems ,” Math. Prog. Comp. 15, 445– 470, 2023
2023
-
[14]
Team s of Global Equilibrium Search Algorithms for Solving the Weighted Maximum Cut Problem in Parallel,
Shylo et al., “Team s of Global Equilibrium Search Algorithms for Solving the Weighted Maximum Cut Problem in Parallel,” Cybernetics and Systems Analysis, vol. 51, p. 16–24, 2015
2015
-
[16]
Record Acceleration of the Two -Dimensional Ising Model Using a High -Performance Wafer- Scale Engine ,
D. V. Essendelft, H. Almolyki, W. Shi, T. Jordan, M.- Y. Wang and W. A. Saidi, “Record Acceleration of the Two -Dimensional Ising Model Using a High -Performance Wafer- Scale Engine ,” arXiv:2404.16990 [cs.AR], 2024
2024 arXiv
-
[17]
Efficient Cluster Algorithm for Spin Glasses in Any Space Dimension,
Z. Zhu, A. Ochoa and H. Katzgraber, “Efficient Cluster Algorithm for Spin Glasses in Any Space Dimension,” Phys. Rev. Lett. 115, 077201, 2015
2015
-
[18]
Physics -Inspired Optimization for Quadratic Uncon strained Problems Using a Digital Annealer,
M. Aramon, G. Rosenberg, E. Valiante, T. Miyazawa, H. Tamura and H. Katzgraber, “Physics -Inspired Optimization for Quadratic Uncon strained Problems Using a Digital Annealer,” Frontiers in Physics , vol. 7, 2019
2019
-
[19]
Scaling Advantage in Approximate Optimization with Quantum Annealing,
H. Munoz-Bauza and D. Lidar, “Scaling Advantage in Approximate Optimization with Quantum Annealing,” Phys. Rev. Lett. 134, 2025
2025
-
[20]
Direct comparison of stochastic driven nonlinear dynamical systems for combinatorial optimization,
J. Hou, A. Barzegar and H. G. Katzgraber, “Direct comparison of stochastic driven nonlinear dynamical systems for combinatorial optimization,” arXiv:2503.15427, 2025
2025 arXiv
-
[21]
Scalable Connectivity for Ising Machines: Dense to Sparse,
M M. H. Sajeeb, et al., “ Scalable Connectivity for Ising Machines: Dense to Sparse,” arXiv:2503.01177, 2025
2025 arXiv
-
[22]
Shylo and O.V
V.P. Shylo and O.V. Shylo. Algorithm Portfolios and Teams in Parallel Optimization. In: Butenko, S., Pardalos, P., Shylo, V. (eds) Optimization Methods and Applications. Springer Optimization and Its Applications, vol 130. Springer, Cham. 2017
2017
-
[23]
A memetic algorithm based on edge- state learning for max-cut,
Zhi-zhong Zeng, Zhi -peng lü, Xin -guo Yu, Qing -hua Wu, Y . Wang and Z. Zhou, “A memetic algorithm based on edge- state learning for max-cut,” Expert Systems with Applications, vol. 209, 2022
2022
-
[24]
A multiple search operator heuristic for the max- k-cut problem,
F. Ma and JK. Hao, “A multiple search operator heuristic for the max- k-cut problem,” Ann Oper Res 248, 365–403 (2017). 5 APPENDIX A Gset G81 Problem Structure Visualization APPENDIX B Cosm solution quality on problem G81 Some of the historic G81 milestones and other notable r...
2017
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.