REVIEW 3 major objections 5 minor 1 cited by
Mixed-Integer Linear Optimization via Learning-Based Two-Layer Large Neighborhood Search
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A two-layer scheme that hunts solutions inside its own neighborhoods beats single-layer LNS and top solvers, cutting primal integral by up to 96%.
desk verdict TLNS is a genuinely new recursive LNS idea, but its empirical edge over LNS is not cleanly established because the comparison uses unequal neighborhood sizes and no matched-k baseline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the two-layer fixation-neighborhood scheme. An auxiliary MILP defined by fixing variables (a fixing neighborhood) is itself solved by LNS rather than by an exact solver, and the presolve operator Q is what makes the nesting affordable: it compresses the outer auxiliary problem so that the inner LNS operates on a smaller model and each inner presolve is cheap. The neighborhood-selection policy is a lightweight graph transformer that adds linear global attention on top of two graph-convolutional half-layers, trained with contrastive loss where local branching supplies positive and negative action samples. This policy is shared by both layers, and the same trained model is applied to test instances larger than those used for training.
What would settle it
Measure T1p (presolve time of the outer auxiliary problem), T2p (presolve time of a single inner auxiliary problem), and H (number of inner LNS iterations) on the four benchmark families. If H * T2p is not substantially smaller than (H - 1) * T1p, the Remark 2 time saving does not hold and TLNS could be slower than a single-layer LNS with an equivalent small neighborhood; the reported gains would then be attributable to neighborhood size or hardware, not nesting. A direct test is to run TLNS with the inner LNS replaced by a single exact solve of the same small neighborhood and compare wall-clock time per unit primal-bound improvement.
Extended reading notes
Core claim
The paper's central claim is that replacing the exact solve of an auxiliary MILP with a heuristic LNS pass accelerates the whole search. Formally, TLNS applies LNS at two levels: in the outer layer it fixes a neighborhood F1 around the incumbent, forming an auxiliary problem A(M, x-bar, F1), then presolves that problem once into a reduced model P; the inner layer runs a standard LNS on P, with a count limit that enlarges the inner neighborhood whenever progress stalls; postsolve maps the inner solution back. The paper argues via Remark 1 that the inner heuristic exits the outer neighborhood faster than branch-and-bound, and via Remark 2 that the single outer presolve makes the total presolve time smaller than presolving each of H inner neighborhoods separately, saving (H - 1)T1p - H T2p when the inner presolve time T2p is much smaller than the outer T1p. Empirically, on Set Cover, Combinatorial Auction, Maximum Independent Set, and Minimum Vertex Cover instances, the learning-guided version CL-TLNS is reported to beat the single-layer CL-LNS by 26.5% to 66.1% in primal integral and to beat both Gurobi and SCIP by up to 96.1% in primal integral and 39.7% in final primal bound.
Load-bearing premise
The whole time-saving argument assumes that after the outer presolve, each inner auxiliary problem can be presolved much faster than the original auxiliary problem would be, so the nested scheme saves more time than it spends on extra inner iterations; the paper does not report presolve times T1p and T2p or inner iteration counts directly.
Editorial extensions
If this is right
- TLNS yields faster primal progress than single-layer LNS for both random and learned neighborhood heuristics on all four benchmarks, with primal-integral gains of 26.2% to 51.3% for the random version and 26.5% to 66.1% for the contrastive-learned version.
- CL-TLNS beats both Gurobi and SCIP (with aggressive internal heuristics) on all benchmarks, by up to 96.1% in primal integral and up to 39.7% in primal bound at the 1,000-second time limit.
- The simplified graph transformer (SGT) outperforms GCN and GAT in guiding TLNS and avoids GAT's out-of-memory failures on the Set Cover instances.
- The number of LNS layers can be increased: GRB(NoRel), CL-LNS(NoRel), and CL-TLNS have 0, 1, and 2 layers, respectively, and performance improves with each added layer.
- The method only needs small MILPs for the exact solver, reducing reliance on branch-and-bound for large problems and suggesting multi-layer LNS as a research direction.
Reading between the lines
- The paper's speedup rests on the unmeasured premise that inner presolve is much cheaper than outer presolve; if that premise fails, TLNS's advantage could vanish on other instance classes, so a direct measurement of T1p, T2p, and H on new benchmarks would settle when nesting helps.
- Because the graph transformer is trained on small instances (up to 6,000 variables) and applied to much larger ones (up to 100,000), the receptive-field argument suggests the method will degrade least on problems whose structure is scale-invariant, such as the random graphs used here; on highly irregular or tightly coupled instances, the fixed training distribution may limit transfer.
- The comparison to state-of-the-art MILP solvers uses a 1,000-second time limit and excludes cases where the solvers can close the gap; on smaller or easier instances where Gurobi or SCIP can prove optimality, the PB/PI gains would likely shrink, and the method's role is best seen as a primal heuristic for hard large instances.
- A recursive multi-layer version would forgo exact solves entirely; an immediate test is whether depth three or four continues to improve primal integral, or whether presolve time savings saturate as inner models become tiny.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-layer large neighborhood search (TLNS) for mixed-integer linear programs: the outer LNS creates an auxiliary MILP as usual, but instead of handing it to an exact solver, an inner LNS solves that auxiliary problem, so that only small sub-MILPs are given to off-the-shelf solvers. A simplified graph transformer (SGT) with linear attention is trained by contrastive learning to score variables for neighborhood construction. Experiments on four public benchmark families (set covering, combinatorial auctions, maximum independent set, minimum vertex cover) compare TLNS with single-layer LNS (both random and learned) and with Gurobi/SCIP, reporting primal-integral improvements up to 66% over LNS and up to 96% over the solvers.
Significance. If the reported improvements are robust, the two-layer recursion idea is a useful contribution: it reduces reliance on exact solvers inside LNS and may scale better to very large MILPs. The paper's SGT architecture with linear attention is a sensible way to extend GNN-based policies to large graphs, and the training/inference pipeline is described with enough detail to be reproducible. The paper also ships code and uses public benchmark generators, which supports independent verification. The main value, however, depends on whether the two-layer structure, rather than the larger neighborhoods assigned to TLNS, is responsible for the observed gains; for that reason the current evidence is suggestive but not conclusive.
major comments (3)
- [§4.2–4.3, Table 2] The claimed superiority of TLNS over LNS is confounded by unequal neighborhood sizes. In Table 2, TLNS is always assigned a larger outer neighborhood (k1) than the single-layer LNS neighborhood (k): e.g., on SC CL-LNS uses k=175 while CL-TLNS uses 500/120; on MIS 12,500 vs 30,000/7,000; on MVC 1,250 vs 5,500/1,000. Since the primal-integral metric rewards earlier progress, the larger k1 alone could explain much of the reported gain. The paper does not include a matched-k baseline (e.g., single-layer LNS with the union neighborhood F1∪F2 as described in Remark 2) nor an ablation that fixes k1 and toggles the inner LNS. Without such a control, the central claim that TLNS generally outperforms LNS is not supported.
- [Remark 2, §3.1] The efficiency argument of Remark 2 relies on the assumption that presolving the inner auxiliary problem A(P, y, F2^h) is much cheaper than presolving A(M, x, F1∪F2^h), i.e., T2p << T1p. This is an unverified assumption; the inner LNS may invoke the solver many times, and each invocation incurs its own presolve cost, so the total presolve time could be larger than T1p. The manuscript reports no measurements of T1p, T2p, or the number of inner iterations. Please provide empirical timing data or at least an ablation that reports presolve times and inner-iteration counts to substantiate the claim that the nested scheme saves presolving time.
- [§4.3–4.4, Table 3] The primal-bound (PB) improvements are small or negative on two of the four benchmarks. In Table 3, CL-TLNS PB on SC is 113.0 vs 112.7 for CL-LNS (0.2% worse), and on MVC the gain is only 0.07%. The conclusion that TLNS 'generally outperforms' LNS is therefore based primarily on the primal-integral (PI) metric. Since PI is normalized by best-known bounds and integrated over the runtime, a systematic difference in neighborhood sizes can dominate it. The authors should temper the general claim, report PB gaps or performance profiles, and explicitly separate the PI-based results from the PB-based results in the conclusions.
minor comments (5)
- [Definition 1] The text 'akaneighborhood size' has a missing space; it should read 'a neighborhood size'.
- [Table 4] The '–' entry for SCIP on MIS is not explained in the text; the discussion should mention that SCIP failed due to memory limits.
- [§4.1] The hardware description '12th Gen Intel(R) Core(TM) i9-12900K CPUs with Nvidia GeForce RTX 3090 GPUs' should clarify how many CPUs/GPUs are used per run (e.g., one CPU and one GPU per instance).
- [§4.2] The statement that SCIP is used as the underlying solver 'since the presolve operator in Gurobi is inaccessible' is unclear; Gurobi's presolve can be disabled, so the reason for not using Gurobi in LNS should be explained more precisely.
- [Table 3] The 'Gain' column sometimes takes negative values (e.g., SC PB -2.7%), which indicates worse performance for TLNS; the sign convention and interpretation should be stated explicitly.
Circularity Check
No significant circularity: TLNS's recursive construction and learned neighborhoods are not defined in terms of the benchmark outcomes; remaining issues are experimental confounds, not circular steps.
full rationale
The paper's derivation chain is an algorithmic proposal plus an empirical evaluation, not a formal derivation of its performance from its inputs. The TLNS recursion (Algorithm 2) solves the outer auxiliary MILP by an inner LNS; this is a genuine algorithmic construction and no equation in the paper defines the reported gains in terms of the learned policy's outputs or the benchmark values. Training uses SMALL instances and testing uses LARGE instances from the same generators, so the headline numbers are not curve-fits to the test set. The individually fine-tuned neighborhood sizes in Table 2 and the unmeasured presolve-time assumption in Remark 2 (T2p much smaller than T1p) are legitimate threats to the causal attribution of the gains, but they do not reduce the comparison to an identity by construction. The best-known objective used to normalize PI is drawn from all evaluated methods including CL-TLNS; while self-referential, it is a fixed precomputed constant per instance and does not by itself force CL-TLNS's PI advantage. The only apparent self-citation, [9], supplies a feature set and is not load-bearing for the two-layer recursion or for the central comparison. No circular step satisfying the evidence standard can be exhibited.
Assumptions & free parameters
free parameters (8)
- Neighborhood sizes for CL-LNS =
SC: 175; CA: 35000; MIS: 12500; MVC: 1250
- Neighborhood sizes for CL-TLNS (outer k1 / inner k2) =
SC: 500/120; CA: 60000/3000; MIS: 30000/7000; MVC: 5500/1000
- Neighborhood sizes for R-LNS =
SC: 4000; CA: 35000; MIS: 40000; MVC: 10000
- Neighborhood sizes for R-TLNS =
SC: 8000/1600; CA: 60000/3000; MIS: 70000/7000; MVC: 15000/1250
- Adaptive rates eta1 and eta2 =
not reported
- Local branching neighborhood sizes for data collection =
SC: 100; CA: 400; MIS: 500; MVC: 75
- Training hyperparameters =
tau=0.07, batch size=32, Adam lr=0.001, layers=2, hidden dim=32
- Sampling size for inference =
same as neighborhood sizes above
assumptions (7)
- domain assumption All variables are binary (xi in {0,1})
- domain assumption An initial feasible solution is available from Gurobi with solution limit 1
- domain assumption Presolve operator Q and postsolve Q^{-1} preserve correctness and are cheap enough
- domain assumption The inner LNS with count limit C and neighborhood size k2 returns a high-quality solution quickly
- domain assumption The graph transformer policy trained on SMALL instances generalizes to LARGE instances
- domain assumption LB expert actions provide good positive and negative samples for contrastive learning
- ad hoc to paper In Remark 2, T2p is much smaller than T1p, so TLNS saves (H-1)T1p - H T2p
Cite this review
Pith. "Pith review of Mixed-Integer Linear Optimization via Learning-Based Two-Layer Large Neighborhood Search." pith.science (2026). https://pith.science/paper/MXAKYZVB
@misc{pith2026241208206,
author = {Pith},
title = {Pith review of: Mixed-Integer Linear Optimization via Learning-Based Two-Layer Large Neighborhood Search},
year = {2026},
howpublished = {\url{https://pith.science/paper/MXAKYZVB}},
note = {Machine review of arXiv:2412.08206}
}
read the original abstract
Mixed-integer linear programs (MILPs) are extensively used to model practical problems such as planning and scheduling. A prominent method for solving MILPs is large neighborhood search (LNS), which iteratively seeks improved solutions within specific neighborhoods. Recent advancements have integrated machine learning techniques into LNS to guide the construction of these neighborhoods effectively. However, for large-scale MILPs, the search step in LNS becomes a computational bottleneck, relying on off-the-shelf solvers to optimize auxiliary MILPs of substantial size. To address this challenge, we introduce a two-layer LNS (TLNS) approach that employs LNS to solve both the original MILP and its auxiliary MILPs, necessitating the optimization of only small-sized MILPs using off-the-shelf solvers. Additionally, we incorporate a lightweight graph transformer model to inform neighborhood design. We conduct extensive computational experiments using public benchmarks. The results indicate that our learning-based TLNS approach achieves remarkable performance gains--up to 66% and 96% over LNS and state-of-the-art MILP solvers, respectively.
Figures
Forward citations
Cited by 1 Pith paper
-
RL-SPH: Learning to Achieve Feasible Solutions for Integer Linear Programs
RL-SPH is a reinforcement learning start primal heuristic that independently produces feasible solutions for ILPs with non-binary integers at 100% rate and with 28.6× lower primal gap than prior start heuristics.
Reference graph
Works this paper leans on
-
[1]
INFORMS Journal on Computing32(2), 473–506 (2020)
Achterberg, T., Bixby, R.E., Gu, Z., Rothberg, E., Weninger, D.: Presolve reductions in mixed integer programming. INFORMS Journal on Computing32(2), 473–506 (2020)
work page 2020
-
[2]
Mathematical Programming Computation 6, 33–54 (2014) 16 W
Berthold, T.: Rens: the optimal rounding. Mathematical Programming Computation 6, 33–54 (2014) 16 W. Liu et al
work page 2014
-
[3]
Bestuzheva, K., Besançon, M., Chen, W.K., Chmiela, A., Donkiewicz, T., van Doornmalen, J., Eifler, L., Gaul, O., Gamrath, G., Gleixner, A., Gottwald, L., Graczyk, C., Halbig, K., Hoen, A., Hojny, C., van der Hulst, R., Koch, T., Lübbecke, M., Maher, S.J., Matter, F., Mühmer, E., Müller, B., Pfetsch, M.E., Rehfeldt, D., Schlein, S., Schlösser, F., Serrano,...
work page 2021
-
[4]
Mathematical Programming102, 71–90 (2005)
Danna, E., Rothberg, E., Pape, C.L.: Exploring relaxation induced neighborhoods to improve mip solutions. Mathematical Programming102, 71–90 (2005)
work page 2005
-
[5]
Mathematical programming98, 23–47 (2003)
Fischetti, M., Lodi, A.: Local branching. Mathematical programming98, 23–47 (2003)
2003
-
[6]
In: NeurIPS 2021 Competitions and Demonstrations Track
Gasse, M., Bowly, S., Cappart, Q., Charfreitag, J., Charlin, L., Chételat, D., Chmiela, A., Dumouchelle, J., Gleixner, A., Kazachkov, A.M., et al.: The machine learning for combinatorial optimization competition (ml4co): Results and insights. In: NeurIPS 2021 Competitions and Demonstrations Track. pp. 220–231. PMLR (2022)
work page 2022
-
[7]
Advances in neural information processing systems 32 (2019)
Gasse, M., Chételat, D., Ferroni, N., Charlin, L., Lodi, A.: Exact combinatorial opti- mization with graph convolutional neural networks. Advances in neural information processing systems 32 (2019)
work page 2019
-
[8]
Gurobi Optimization, LLC: Gurobi Optimizer Reference Manual (2023),https: //www.gurobi.com
work page 2023
Show all 30 references
-
[9]
arXiv preprint arXiv:2302.05636 (2023)
Han, Q., Yang, L., Chen, Q., Zhou, X., Zhang, D., Wang, A., Sun, R., Luo, X.: A gnn-guided predict-and-search framework for mixed-integer linear programming. arXiv preprint arXiv:2302.05636 (2023)
2023 arXiv
-
[10]
Mathematical Programming Computation pp
Hendel, G.: Adaptive large neighborhood search for mixed integer programming. Mathematical Programming Computation pp. 1–37 (2022)
2022
-
[11]
In: International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research
Huang, T., Ferber, A., Tian, Y., Dilkina, B., Steiner, B.: Local branching relaxation heuristics for integer linear programs. In: International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research. pp. 96–113. Springer (2023)
2023
-
[12]
In: International Conference on Machine Learning
Huang, T., Ferber, A.M., Tian, Y., Dilkina, B., Steiner, B.: Searching large neigh- borhoods for integer linear programs with contrastive learning. In: International Conference on Machine Learning. pp. 13869–13890. PMLR (2023)
2023
-
[13]
arXiv preprint arXiv:1412.6980 (2014)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
2014 arXiv
-
[14]
arXiv preprint arXiv:1609.02907 (2016)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
2016 arXiv
-
[15]
In: Proceedings of the AAAI conference on artificial intelligence
Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the AAAI conference on artificial intelligence. vol. 32 (2018)
2018
-
[16]
Liu,D.,Fischetti,M.,Lodi,A.:Learningtosearchinlocalbranching.In:Proceedings of the aaai conference on artificial intelligence. vol. 36, pp. 3796–3803 (2022)
2022
-
[17]
Transportation Research Part B: Methodological45(5), 808–827 (2011)
Luathep, P., Sumalee, A., Lam, W.H., Li, Z.C., Lo, H.K.: Global optimization method for mixed transportation network design problem: a mixed-integer linear programming approach. Transportation Research Part B: Methodological45(5), 808–827 (2011)
2011
-
[18]
arXiv preprint arXiv:2012.13349 (2020) MILP via learning-based TLNS 17
Nair, V., Bartunov, S., Gimeno, F., Von Glehn, I., Lichocki, P., Lobov, I., O’Donoghue, B., Sonnerat, N., Tjandraatmadja, C., Wang, P., et al.: Solving mixed integer programs using neural networks. arXiv preprint arXiv:2012.13349 (2020) MILP via learning-based TLNS 17
2020 arXiv
-
[19]
Pochet, Y., Wolsey, L.A.: Production planning by mixed integer programming, vol. 149. Springer (2006)
2006
-
[20]
INFORMS Journal on Computing19(4), 534–541 (2007)
Rothberg, E.: An evolutionary algorithm for polishing mixed integer programming solutions. INFORMS Journal on Computing19(4), 534–541 (2007)
2007
-
[21]
In: International conference on principles and practice of constraint programming
Shaw, P.: Using constraint programming and local search methods to solve vehi- cle routing problems. In: International conference on principles and practice of constraint programming. pp. 417–431. Springer (1998)
1998
-
[22]
Advances in Neural Information Processing Systems 33, 20012–20023 (2020)
Song, J., Yue, Y., Dilkina, B., et al.: A general large neighborhood search framework for solving integer linear programs. Advances in Neural Information Processing Systems 33, 20012–20023 (2020)
2020
-
[23]
CoRRabs/2107.10201 (2021), https://arxiv.org/abs/2107.10201
Sonnerat, N., Wang, P., Ktena, I., Bartunov, S., Nair, V.: Learning a large neigh- borhood search algorithm for mixed integer programs. CoRRabs/2107.10201 (2021), https://arxiv.org/abs/2107.10201
2021 arXiv
-
[24]
SIAM (2002)
Toth, P., Vigo, D.: The vehicle routing problem. SIAM (2002)
2002
-
[25]
Advances in neural information processing systems 30 (2017)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
2017
-
[26]
arXiv preprint arXiv:1710.10903 (2017)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
2017 arXiv
-
[27]
Wolsey, L.A., Nemhauser, G.L.: Integer and combinatorial optimization, vol. 55. John Wiley & Sons (1999)
1999
-
[28]
Advances in Neural Information Processing Systems36 (2024)
Wu, Q., Zhao, W., Yang, C., Zhang, H., Nie, F., Jiang, H., Bian, Y., Yan, J.: Simplifying and empowering transformers for large-graph representations. Advances in Neural Information Processing Systems36 (2024)
2024
-
[29]
Advances in Neural Information Processing Systems34, 30075–30087 (2021)
Wu, Y., Song, W., Cao, Z., Zhang, J.: Learning large neighborhood search policy for integer programming. Advances in Neural Information Processing Systems34, 30075–30087 (2021)
2021
-
[30]
Neurocomputing519, 205–217 (2023)
Zhang, J., Liu, C., Li, X., Zhen, H.L., Yuan, M., Li, Y., Yan, J.: A survey for solving mixed integer programming via machine learning. Neurocomputing519, 205–217 (2023)
2023
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.