{"id":"3220faf8-a015-4953-91be-2ee63690c0a3","arxiv_id":"2412.14854","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Iterative surrogate-assisted multi-objective optimization with NSGA-II and neural network surrogates is tested on a 24-parameter car suspension design problem, reporting faster Pareto front approximation.","lead":"This paper tests an iterative shortcut for designing car suspensions with conflicting goals: a fast surrogate model guides a small number of expensive, realistic simulations. The authors compare neural networks versus radial basis functions and two optimizers on a 24-parameter suspension problem.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The stopping criterion in Algorithm 3 certifies front stability, not front accuracy; without a reference Pareto front or direct-optimization baseline, the near-optimality claim is unsupported.","rationale":"The reader's weakest assumption correctly identifies the decisive logical gap: Algorithm 3's convergence test only measures stability between surrogate fronts, not accuracy relative to the true Pareto front. This is the single most load-bearing issue because the paper's headline contributions—near-optimality, speedup, and superiority of ANN over RBF—all depend on the final front being genuinely close to PF. The RBF experiment is particularly telling: it converges in fewer iterations to a front the authors themselves deem worse, showing that the stopping rule can be satisfied at low-quality solutions. No experiment in the paper tests the final front against an independent reference: all plots compare iterations of the algorithm to each other, and the non-dominance test on line 15 only filters real samples, it does not establish coverage of PF. The paper is internally consistent and the method is plausible, but the evidence supplied is insufficient for the strength of the claims. I agree with the reader's conditional verdict: the framework may work, but acceptance of the near-optimality claim requires an external validation, ideally on a problem with a known Pareto front or at least a direct optimization baseline. The proposed benchmark test would settle whether the stopping criterion is merely convenient or actually meaningful. No change to the reader's verdict is warranted.","tokens_in":12357,"tokens_out":2252,"duration_ms":23500,"concrete_test":"Run Algorithm 3 with the same settings (ANN surrogate, s=20, hmin=2, NSGA-II) on a benchmark multi-objective problem with a known analytical Pareto front, e.g., ZDT1 or DTLZ2 in dimension 24, and compute the inverted generational distance (IGD) or true Hausdorff distance between the final surrogate front bPF and the true PF. If the algorithm terminates with h < hmin while the IGD remains large relative to the front's objective scale, the stopping criterion fails to certify near-optimality. Additionally, for the MBS case, evaluate all M=100 points of the final ANN surrogate front with the expensive model and compare their hypervolume against a direct NSGA-II run with the same total budget of S=120 true evaluations; if the direct run attains comparable or better hypervolume, the claimed speedup and 'best' status need revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the framework 'finds near-optimal solutions' and that ANN+NSGA-II is 'best' rests on Algorithm 3's stopping rule (lines 9-12): the loop terminates when the Hausdorff distance between two successive surrogate Pareto fronts, d_H(bPF^(j-1), bPF^(j)), falls below hmin = 2. This is a consistency check between two approximations of the same surrogate-based problem, not a check of closeness to the true Pareto front PF. A systematically biased surrogate can produce fronts that stabilize far away from PF, so small successive distance does not imply near-optimality. The paper's own RBF experiment (Section V-B) provides empirical evidence: Algorithm 3 stops after 6 iterations with a front the authors themselves judge worse than the ANN front, yet the same stopping rule is used to certify the ANN result. Additionally, all reported results are self-comparisons: there is no reference Pareto front, no repeated runs, no direct NSGA-II run on the expensive model, and no total wall-clock budget comparison. Consequently, 'significant speedup of several orders' and 'convergence to high-quality solutions' are not quantitatively established. The load-bearing condition for the central claim is that the final surrogate Pareto front is close to the true Pareto front, and this condition is never directly tested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a surrogate-assisted framework for expensive multi-objective optimization of multibody systems. Algorithm 3 alternates between training a surrogate (ANN or RBF), solving the surrogate MOP with NSGA-II (or MGDA), and using k-means clustering on the current surrogate Pareto set to select additional expensive samples. Convergence is declared when the Hausdorff distance between successive surrogate Pareto fronts falls below hmin. The method is demonstrated on a 24-parameter rear-suspension model with two objectives. The authors report that ANN surrogates with NSGA-II outperform RBFs and MGDA, requiring 11 iterations to converge, and conclude that the framework yields a speedup of several orders and convergence to high-quality solutions.","tokens_in":1283,"tokens_out":1528,"duration_ms":40930,"significance":"The algorithmic template is sensible and the paper addresses an important problem: how to allocate a limited budget of expensive simulations when approximating a Pareto front. The pseudocode and the MBS test case are concrete, and the paper explicitly compares two surrogate classes and two optimizer families. Strengths include a cheap convergence criterion, a clear description of the sampling strategy, and a nontrivial engineering test problem. However, the current evidence is almost entirely internal: all claims are relative to surrogate-based fronts generated by the same pipeline. There is no reference Pareto front, no direct-optimization baseline, no repeated runs, and no accounting of total wall-clock time. Thus the headline claims of near-optimality and several-orders speedup are not yet supported. The methodology is a plausible and useful contribution, but the experimental validation needs substantial strengthening.","major_comments":[{"comment":"The stopping criterion h = d_H(bPF^(j-1), bPF^(j)) < hmin is a self-comparison of two surrogate-generated fronts; it certifies consistency between successive approximations of the same surrogate problem, not closeness to the true Pareto front. A systematically biased surrogate can stabilize far from the true PF. The paper itself provides an internal counterexample in Section V-B: with the RBF surrogate, Algorithm 3 stops after 6 iterations with a front the authors judge worse than the ANN front, yet the same hmin = 2 criterion is used for the ANN result. The conclusion 'convergence to high-quality solutions' therefore is not directly supported. The revision should evaluate the final surrogate front on the true simulator and compare it against a reference Pareto front or a direct NSGA-II run on the expensive model with a comparable budget.","section":"Algorithm 3, lines 9-12"},{"comment":"There is no reference Pareto front and no non-surrogate baseline in the experiments. All reported comparisons are among variants of Algorithm 3, so the claim that ANN+NSGA-II is 'the best way to find near-optimal solutions' cannot be assessed without a ground-truth front. Please add a high-budget direct optimization baseline (for example, NSGA-II applied directly to the expensive model with a predefined evaluation budget, or a set of independently converged single- or multi-objective optima) and report the resulting front together with the surrogate fronts.","section":"Section V"},{"comment":"The claim of 'a significant speedup of several orders' is not quantified. No total wall-clock time for Algorithm 3 is reported against a direct-optimization baseline; statements such as 'approximately 30 seconds for a single function evaluation' and 'a few seconds for solving the surrogate MOP' do not by themselves establish the speedup, because training, retraining, clustering, and NSGA-II iterations consume time as well. In addition, NSGA-II is stochastic, yet no repeated runs or variance measures are reported for any experiment. The revision should report total wall-clock budgets, numbers of true evaluations, and statistics over multiple independent runs.","section":"Section V-C and Conclusion"}],"minor_comments":[{"comment":"The sentence 'We will experiment with various values for m' appears to refer to the sample size s or the batch size; please correct the variable and make the statement consistent with Algorithm 3.","section":"Section II-C"},{"comment":"There are several typos and grammatical errors: 'Guassian' should be 'Gaussian', 'imporoves' should be 'improves', and the sentence ending 'and a better approximated Pareto front' in Section V-C is incomplete.","section":"Section V"},{"comment":"The caption is inconsistent with the text: the caption says (c) and (d) illustrate computation times, while the text says (c) shows the number of iterations versus sample size; please harmonize the caption, axis labels, and the referenced quantities.","section":"Figure 10"},{"comment":"The sub-figures are small and the axes are not always labeled; adding consistent axis labels and legends would help the reader verify the claimed convergence behavior visually.","section":"Figure 8"},{"comment":"The statement 'we found hmin = 2 to be a good value' would be more informative with a short description of the range of thresholds tested and the effect of the threshold on the RBF and ANN results.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper presents an iterative back-and-forth between surrogate modeling and multi-objective optimization, using k-means sampling on the surrogate Pareto front to add points, and a Hausdorff-distance-based stopping rule. The test problem is a realistic 24-parameter rear suspension multibody system with two conflicting objectives. The comparison of ANN vs RBF surrogates, NSGA-II vs MGDA, and sample sizes is a useful engineering study.\n\nWhat's genuinely new is the specific combination of Pareto-informed k-means sampling with ANN surrogates and NSGA-II in a loop. That combination isn't in the cited prior work, and the paper documents behavior across iterations on a real, expensive simulator. That's worth something.\n\nHowever, the central claim that the framework 'finds near-optimal solutions' and achieves 'several orders of magnitude speedup' is not supported by the reported experiments. The stopping rule in Algorithm 3 compares two successive surrogate-based fronts. Small Hausdorff distance between them means the surrogate front has stabilized, not that it is close to the true Pareto front. A systematically biased surrogate can stabilize anywhere. The RBF experiment shows this: it stops after 6 iterations with a front the authors themselves judge worse than the ANN front. So the convergence criterion does not certify quality.\n\nWorse, there is no reference Pareto front from a high-budget direct optimization, no non-surrogate baseline, no repeated runs of the stochastic NSGA-II, and no total wall-clock budget comparison. The 'speedup' claim has no denominator. The final non-dominance test on the true samples grounds the sampled points, but it doesn't tell you how close the surrogate front is to the true front.\n\nThe framework is plausible, and the application is real, so I'd send it to review, but with a clear request to add a reference front (even on a test problem with known Pareto front), repeat runs, and report compute times. As it stands, the conclusions overreach the evidence. A serious referee could fix this with moderate revisions.\n\nSend it out, but expect the authors to need to do another round of experiments.","headline":"A plausible surrogate-assisted multi-objective framework on a real suspension problem, but the near-optimality claim rests on a self-referential stopping criterion and no reference front.","tokens_in":13147,"tokens_out":2233,"would_cite":false,"duration_ms":16745,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C29","90C59"],"pacs":[],"model":"deepseek-v4-flash","headline":"Coupling NSGA-II with repeatedly retrained neural surrogates finds near-optimal suspension designs with orders-of-magnitude fewer simulations.","keywords":["multi-objective optimization","surrogate modeling","multibody systems","NSGA-II","radial basis functions","artificial neural networks","Pareto front","Hausdorff distance"],"falsifier":"Evaluate the final surrogate Pareto front on the true suspension model and compare it with a reference Pareto front obtained by a very expensive high-generation NSGA-II run directly on the true model; if the true-evaluated surrogate front is dominated by the reference front, or if their Hausdorff distance is much larger than the threshold $h_{\\min}=2$ used to stop the loop, then the convergence criterion has not certified near-optimality.","tokens_in":12168,"feed_emoji":"🚗","tokens_out":8009,"duration_ms":62341,"temperature":0.7,"pith_summary":"Optimizing an expensive multibody system with several conflicting objectives is normally infeasible because every simulation takes roughly thirty seconds and a Pareto front needs many evaluations. This paper tries to establish that an adaptive loop between surrogate modeling and multi-objective optimization can make this tractable: train a cheap surrogate on a small Latin-hypercube sample, compute a surrogate Pareto front, use that front to choose where to sample next, retrain, and repeat until the front stops moving. The authors claim that, for a 24-parameter rear-suspension model with wheel-load and body-acceleration objectives, the combination of NSGA-II with a neural-network surrogate and Pareto-informed k-means sampling is the most promising configuration, giving a speedup of several orders of magnitude while still converging to high-quality solutions. A radial-basis-function surrogate converges in fewer iterations but to a worse front, and gradient-based multi-start is inferior in spread and time. If the claim holds, surrogate-assisted multi-objective optimization becomes a practical route for design problems where direct optimization is too costly.","feed_headline":"Surrogate loop cuts suspension design cost by orders of magnitude","feed_subtitle":"A stop rule based on front stability gives near-optimal trade-offs with orders fewer expensive simulations.","key_machinery":"The load-bearing mechanism is Algorithm 3, an iterative loop that alternates between building a surrogate of the expensive objective $f : \\mathbb{R}^{24} \\to \\mathbb{R}^2$ and solving the surrogate problem $(\\widehat{\\mathrm{MOP}})$ with NSGA-II. New expensive samples are placed by k-means clustering the current surrogate Pareto front, so each batch of simulations targets the region the surrogate believes is optimal. Convergence is judged by the Hausdorff distance $d_H(\\widehat{\\mathrm{PF}}^{(j-1)}, \\widehat{\\mathrm{PF}}^{(j)})$, the largest distance from any point of one set to the nearest point in the other set, between fronts of successive iterations; when that distance drops below $h_{\\min}$, the loop stops. A final non-dominance test on all true samples yields the reported Pareto set, turning a roughly thirty-second-per-evaluation model into a few-seconds-per-iteration optimization.","core_discovery":"The authors' central discovery is that a back-and-forth procedure—surrogate training, optimization on the surrogate, Pareto-informed resampling, retraining—solves the expensive multibody design problem efficiently and near-optimally. The recommended configuration is NSGA-II as the multi-objective solver with a fully connected neural network as the surrogate, using twenty new expensive samples per iteration chosen by k-means clustering of the previous surrogate Pareto front. The algorithm stops when the Hausdorff distance between successive surrogate Pareto fronts falls below a threshold; on this problem that took eleven iterations with a per-iteration sample size of twenty, and the resulting front is judged to be of high quality. The authors also report that radial-basis-function surrogates stop earlier but produce a worse front, and that larger sample sizes reduce iterations while increasing wall-clock time.","pith_inferences":["If the same loop were run with a fixed total budget but an adaptive per-iteration sample size, earlier iterations could explore coarsely and later iterations refine; the paper names adaptive sample sizes as future work, so this is an extension rather than a claim.","The convergence proxy is only as good as the surrogate's consistency: two successive surrogate fronts can be close to each other yet both be far from the true Pareto front, so a practical deployment should validate the final front with true evaluations.","Because the method treats the simulator as a black box, the same loop should transfer to other expensive simulation-based multi-objective design problems, such as structural or energy-system design, though the paper only demonstrates it on one suspension model.","A direct test of the claimed speedup would compare total wall-clock time against a standard NSGA-II run on the true model with the same budget and report the Hausdorff distance between the two fronts."],"forward_implications":["For this rear-suspension problem, NSGA-II combined with an ANN surrogate and Pareto-informed sampling replaces an infeasible optimization (about 30 seconds per evaluation) with a loop that converges in roughly ten to eleven iterations of twenty evaluations, a speedup of several orders of magnitude.","RBF surrogates converge in fewer iterations but to a visibly worse front, which the paper attributes to RBF sensitivity to unevenly distributed samples, making surrogate choice a decisive factor.","The stopping rule trades accuracy for cost: a smaller Hausdorff threshold improves the front but needs many more expensive iterations.","Larger per-iteration sample sizes cut the number of iterations but raise total computation time, so sample size must be balanced against the simulation budget.","The final Pareto front reported is the result of a non-dominance test on the true expensive samples, so the delivered set is grounded in real evaluations rather than surrogate predictions alone."],"supporting_citations":[{"why":"Supplies the NSGA-II algorithm used to solve the surrogate multi-objective problem in every iteration.","marker":"[7]"},{"why":"Provides the radial-basis-function trust-region surrogate context and the RBF baseline the paper compares against.","marker":"[2]"},{"why":"Defines the Hausdorff distance used as the convergence and stopping criterion between successive surrogate fronts.","marker":"[39]"},{"why":"Supplies the rear-axle multibody model that defines the design variables and the test case for the numerical study.","marker":"[38]"},{"why":"Provides the symbolic multibody software that generates the equations of motion for the expensive simulator.","marker":"[24]"},{"why":"Defines the wheel-load-fluctuation and body-acceleration objectives and the sinusoidal excitation used in the experiment.","marker":"[28]"},{"why":"Supplies the design-of-experiments background for the Latin-hypercube initial sampling.","marker":"[46]"},{"why":"Provides the multiple-gradient-descent algorithm (MGDA) used as the alternative optimizer that is found inferior.","marker":"[9]"}],"fun_headline_variants":["Back-and-forth surrogate loop yields near-optimal multibody designs","Iterative surrogate optimization finds Pareto front with fewer simulations","Surrogate retraining loop tackles expensive multibody design efficiently","Loop between surrogate and optimizer speeds complex design"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The procedure assumes that a small Hausdorff distance between two consecutive surrogate Pareto fronts means the front has converged to the true Pareto front; if the surrogate stays confidently wrong in the same region, the fronts can stabilize far from the true optimum, and the paper's RBF experiment shows that stopping early on this criterion does not by itself certify quality.","fun_headline_variants_meta":{"raw":{"variants":["Back-and-forth surrogate loop yields near-optimal multibody designs","Iterative surrogate optimization finds Pareto front with fewer simulations","Surrogate retraining loop tackles expensive multibody design efficiently","Loop between surrogate and optimizer speeds complex design"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1297,"prompt_tokens":882,"completion_tokens":415,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":498,"tokens_out":415,"duration_ms":3806,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:51:06.777199+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the final surrogate Pareto front on the true suspension model and compare it with a reference Pareto front obtained by a very expensive high-generation NSGA-II run directly on the true model; if the true-evaluated surrogate front is dominated by the reference front, or if their Hausdorff distance is much larger than the threshold $h_{\\min}=2$ used to stop the loop, then the convergence criterion has not certified near-optimality.","supporting_citations":[{"cited_title":"A fast and elitist multiobjective genetic algorithm: NSGA-II,","cited_arxiv_id":null,"evidence_quote":"Supplies the NSGA-II algorithm used to solve the surrogate multi-objective problem in every iteration."},{"cited_title":"Derivative-free multiobjective trust region descent method using radial basis function surrogate models,","cited_arxiv_id":null,"evidence_quote":"Provides the radial-basis-function trust-region surrogate context and the RBF baseline the paper compares against."},{"cited_title":"Using the averaged hausdorff distance as a performance measure in evolutionary multiobjective optimization,","cited_arxiv_id":null,"evidence_quote":"Defines the Hausdorff distance used as the convergence and stopping criterion between successive surrogate fronts."},{"cited_title":"Tire wear reduction based on an extended multibody rear axle model,","cited_arxiv_id":null,"evidence_quote":"Supplies the rear-axle multibody model that defines the design variables and the test case for the numerical study."},{"cited_title":"From Neweul to Neweul-M2: Symbolical equations of motion for multibody system analysis and synthesis,","cited_arxiv_id":null,"evidence_quote":"Provides the symbolic multibody software that generates the equations of motion for the expensive simulator."},{"cited_title":"Kraftfahrzeug, unebenheits- einpunktanregung,","cited_arxiv_id":null,"evidence_quote":"Defines the wheel-load-fluctuation and body-acceleration objectives and the sinusoidal excitation used in the experiment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the design-of-experiments background for the Latin-hypercube initial sampling."},{"cited_title":"Multiple-gradient descent algorithm (MGDA) for mul- tiobjective optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the multiple-gradient-descent algorithm (MGDA) used as the alternative optimizer that is found inferior."}],"review_version":1}