{"id":"3a1535b6-3bdb-4df0-b66e-3c2767b8b4fd","arxiv_id":"2411.09807","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Topological summaries of 2D loss landscape projections, saddle counts and average persistence, correlate with accuracy and Hessian metrics for ResNets and PINNs.","lead":"This paper applies topological data analysis, using merge trees and persistence diagrams, to measure the shape of neural network loss landscapes and relates these measurements to model accuracy and optimization difficulty. It shows that removing residual connections from a ResNet makes the loss landscape topologically more complex, and that a physics-informed neural network's landscape becomes more complex as the convection speed increases and training fails.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No sensitivity analysis for grid resolution or kNN connectivity; saddle counts and persistence may be graph artifacts, undermining the reported comparisons.","rationale":"The reader's weakest_assumption focuses on the 2D projection not preserving the topology of the full loss landscape. I partially agree, but the more immediate and testable vulnerability is that even the 2D landscape's computed topology is itself a function of discretization choices (grid resolution and kNN connectivity). The paper provides no evidence that the merge-tree saddle count or average persistence is stable under these choices. Because the central claim is entirely comparative—residual connections reduce saddles; increasing beta increases saddles and persistence—an instability in these metrics would invalidate the reported insights. My concern is distinct from the projection issue because removing the projection (e.g., going to 3D or 10D) would still leave the discretization dependence unresolved. The reader's CONDITIONAL verdict already acknowledges the need for additional validation, so I do not change it; I would make the requested conditions more explicit by demanding a discretization sensitivity analysis in addition to multi-seed replication and code release.","tokens_in":7586,"tokens_out":4805,"duration_ms":52092,"concrete_test":"Recompute the merge tree and 0-dimensional persistence diagram for the ResNet-20 (with/without residual connections) and PINN (beta=1..10) landscapes at 2x and 0.5x grid resolution and with k=4, 8, 16, and 32 in the kNN graph (Appendix A.1). For each configuration, compare the sign and ordering of (i) saddle count and (ii) average persistence against accuracy/error. If any configuration reverses the reported trend—more saddles for no-residual ResNet-20, or increasing saddles/persistence with beta—the claims are not robust to discretization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The quantitative backbone of the paper is the number of saddle points in the merge tree and the average persistence of the 0-dimensional persistence diagram. These quantities are not properties of the continuous loss landscape directly; they are computed from a discrete k-nearest-neighbor graph (k=8) over finite samples on a 2D projection (Appendix A.1). For a discrete graph, the merge tree and its saddle count depend on the graph's connectivity and sampling density. The paper does not report any sensitivity analysis with respect to grid resolution or k, nor any convergence check toward the continuous merge tree. Without such validation, the reported differences (e.g., ResNet-20 without residual connections having more saddles; PINN with beta=9 having higher persistence; Figs. 5 and 6) could be artifacts of the discretization rather than properties of the underlying loss landscape. Since every subsequent conclusion compares these counts, this unexamined discretization dependence is the most load-bearing weakness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using topological data analysis (TDA) — specifically merge trees and 0-dimensional persistence diagrams — to quantify the loss landscapes of neural networks. The authors construct 2D loss surfaces by perturbing trained model parameters along either random directions or the top two Hessian eigenvectors (Eq. 4), represent the sampled losses as an unstructured k-nearest-neighbor graph (k = 8 in Appendix A.1), and extract two scalar metrics: the number of saddle points in the merge tree and the average persistence of the 0-dimensional persistence diagram. These metrics are compared with traditional ML metrics (accuracy, absolute error, top Hessian eigenvalue, Hessian trace) for two case studies: ResNet-20 on CIFAR-10 with and without residual connections, and a physics-informed neural network (PINN) solving a 1D convection problem with increasing convection coefficient β. The central claims are that removing residual connections increases the number of saddle points and decreases average persistence, while increasing β increases both saddle count and average persistence, tracking the deterioration of PINN accuracy. The paper also highlights that merge trees reveal multiscale structure not visible in raw loss landscape visualizations.","tokens_in":7747,"tokens_out":3189,"duration_ms":34818,"significance":"If the empirical trends hold, the proposed TDA-based metrics would provide a useful quantitative complement to existing loss landscape visualization and Hessian-based analysis. The paper leverages well-established TDA libraries (TTK) and connects topological summaries to both performance metrics and local curvature metrics, which is a sensible and potentially reproducible framework. The use of four random seeds for the ResNet experiment is a positive feature. However, the evidence in the current manuscript is thin: the central quantitative claims rest on discrete graph summaries whose dependence on the kNN graph, grid resolution, and projection choice is not examined, and the PINN experiment uses a single seed. The claimed insights are plausible but not yet established at the level of a journal publication.","major_comments":[{"comment":"The saddle count and average persistence are computed on a k-nearest-neighbor graph with k = 8 over a finite 2D grid of loss samples from Eq. (4), but the paper reports no sensitivity analysis or convergence check with respect to k, grid resolution, or perturbation range. Since every subsequent comparison in Figs. 5 and 6 is based on these discrete quantities, the reported differences (e.g., more saddles without residual connections, higher persistence at β = 9) could in principle be artifacts of graph connectivity or sampling density rather than properties of the underlying continuous loss landscape. The authors should either add a sensitivity/convergence study or provide a formal argument that the chosen discretization is sufficient.","section":"Appendix A.1, Figs. 5 and 6"},{"comment":"The PINN experiment uses a fixed random seed (the Fig. 6 caption states seed = 0) and reports no error bars, no repeated trials, and no statistical significance tests, yet the text states as a general trend that increasing β increases saddle count and average persistence. A single optimization trajectory cannot support a claim about the relationship between β and the TDA metrics. Multiple seeds with reported variability are needed before the PINN conclusion can be considered robust.","section":"Section 3.2 and Fig. 6"},{"comment":"The TDA descriptors are computed on a 2D projection of the parameter space, and the future-work paragraph explicitly acknowledges that this is not the full space. Because the merge tree and persistence diagram of a projection need not reflect the topology of the high-dimensional loss function, the paper's framing as 'quantifying the topology of loss landscapes' is stronger than what the method supports. The authors should either justify that the chosen 2D projections preserve the relevant topological features (e.g., by comparing random and Hessian projections across repeated draws), or consistently re-frame the claims as properties of the 2D slices. As written, the paper oscillates between these two readings.","section":"Eq. (4) and Section 4"}],"minor_comments":[{"comment":"The definition of a sublevel set writes L−(v) = {x ∈ D; x ≤ v}; this should be {x ∈ D | f(x) ≤ v} for the scalar function f under consideration.","section":"Section 2"},{"comment":"The PINN loss in Eq. (3) is written with 1/Nu and 1/Nf prefactors but no square root or explicit weighting convention; please clarify whether the terms are mean squared errors and specify how the weights λi are set.","section":"Eq. (3)"},{"comment":"The caption states β ∈ [1..10], while the text and Fig. 2 show β = 1, 3, 5, 7, 9; please reconcile these ranges and specify exactly which values were used.","section":"Fig. 6 caption"},{"comment":"The caption reports four random seeds for each ResNet condition, but the figure as described does not show per-seed points or error bars; consider overlaying individual seeds or reporting standard deviations so the reader can assess the spread.","section":"Fig. 5 caption"},{"comment":"The statement that saddle nodes in the merge tree are 'degree-three nodes (each connecting two local minima and one other saddle point)' may be too restrictive; a merge tree can have nodes where more than two components merge at the same threshold. Clarify how such degeneracies are handled in the implementation.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads as a workshop-style empirical study with a plausible but not yet fully supported central claim. The main risk is that the quantitative TDA metrics are computed on discrete kNN graphs and 2D projections without sensitivity or convergence analysis; this is fixable but must be addressed. The single-seed PINN experiment is another concern that should be straightforward to remedy. I would encourage the editor to require these additions before considering the paper for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely new: it uses merge trees and persistence diagrams to quantify loss landscapes, not just visualize them. The saddle count and average persistence are simple summaries, but they do reveal structure that a raw plot hides—the ResNet-20 comparison in Fig. 1 is convincing, and the finding that removing residual connections creates more saddles and lower persistence aligns with prior smoothness arguments. The PINN experiment also shows sensible trends as beta increases, with saddle count and persistence tracking error and Hessian metrics. The writing is clear, the TDA background is accurate, and the citations to Li et al., Krishnapriyan et al., and Yao et al. are appropriate. This is a reasonable exploratory paper, and the central idea deserves a serious referee.\n\nThe soft spots are real, though. The stress-test concern is valid: the saddle count and persistence are computed on a k-nearest-neighbor graph with k=8, and the paper gives no sensitivity analysis for k, grid resolution, or perturbation range. The authors do state the choice in Appendix A.1, but they do not show that the reported differences are robust to these choices. Since every conclusion compares these numbers, this is a load-bearing weakness, not a minor omission. The PINN experiment uses a single seed (seed=0), so the trends across beta could be partly idiosyncratic; four seeds for ResNet help, but no error bars or significance tests are reported anywhere. The projection onto a 2D subspace is acknowledged in the future work section, but the assumption that it preserves the topology of interest is not tested. These are all fixable with additional experiments, but right now they limit the strength of the claims.\n\nOverall, the paper is honest and the trends are plausible, but it is not yet a rigorous demonstration. It would benefit from a sensitivity analysis of the TDA pipeline, multiple PINN seeds, and ideally code/data release so others can reproduce the numbers. I would send it to peer review with a request for major revision, not desk-reject it.\n\nFor a reading group: maybe—it is short and gives a quick look at TDA applied to ML, but the methodological gaps will be obvious to a critical audience.","headline":"A solid exploratory application of TDA to loss landscapes with some real insights, but the quantitative claims rest on unexamined discretization choices and thin experimental evidence.","tokens_in":8273,"tokens_out":1509,"would_cite":false,"duration_ms":17579,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["55N31","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"By quantifying the topology of loss landscapes with merge trees and persistence diagrams, this paper shows that saddle count and average persistence track model accuracy, curvature, and the onset of optimization failure in PINNs.","keywords":["loss landscape","topological data analysis","merge tree","persistence diagram","ResNet","physics-informed neural networks","Hessian","saddle points"],"falsifier":"A concrete check: compute merge trees on 3D and higher-dimensional subspaces, sampled along the top 3 to 10 Hessian eigenvectors, for the same trained ResNet-20 and PINN models. If the reported trends in saddle count and average persistence reverse or disappear as the subspace dimension grows, the 2D-projection topology is not representative of the true landscape. A complementary test is to vary the sampling density and the k-nearest-neighbor connectivity of the loss grid; if the relationships between saddle count, persistence, and accuracy or Hessian metrics change sign or vanish, the descriptors are artifacts of the discretization.","tokens_in":7402,"feed_emoji":"📐","tokens_out":8683,"duration_ms":78851,"temperature":0.7,"pith_summary":"The paper's claim is that the shape of a neural network's loss landscape can be measured, not just visualised, using two tools from topological data analysis: the merge tree and the persistence diagram. These tools count the landscape's saddle points and measure how long its valleys persist as a loss threshold is raised, yielding quantitative descriptors that can be compared across models. On ResNet-20, the method finds that removing residual connections produces a merge tree with more saddles and lower average persistence, even where the raw loss surface looks deceptively simple. On physics-informed neural networks solving a 1D convection equation, raising the wave speed parameter increases both saddle count and persistence, in step with rising error and Hessian-based measures of curvature. The paper argues that these topological summaries offer a reproducible, scale-free window into why some architectures and problems are easier to optimize than others.","feed_headline":"Topological tools expose hidden structure in loss landscapes","feed_subtitle":"Counting saddles and measuring persistence in merge trees tracks accuracy, curvature, and PINN failure.","key_machinery":"The central object is the merge tree of the loss function's sublevel sets, along with its 0-dimensional persistence diagram. As a threshold on loss is raised, new connected components are born at local minima and die at saddles; the merge tree records these events as degree-three nodes (saddles) and degree-one nodes (minima), while the persistence diagram plots each feature's birth against its death and measures its lifetime by distance to the diagonal. The paper counts the degree-three saddle nodes and averages the persistence values to get two scalar descriptors. These descriptors are what connect the topology of the landscape to accuracy, Hessian eigenvalue, and Hessian trace.","core_discovery":"The central discovery is that the 0-dimensional persistent homology of a loss landscape, encoded in a merge tree and persistence diagram, provides quantitative, reproducible descriptors that track model performance and learning dynamics. For ResNet-20, the paper reports that deleting residual connections increases the number of saddle points and decreases the average persistence, and these topological changes move oppositely to the model's accuracy, top Hessian eigenvalue, and Hessian trace. For PINNs, raising the convection coefficient from 1 to 9 makes the loss landscape more complex in both the saddle-count and persistence senses, and this complexity grows together with absolute error, top Hessian eigenvalue, and Hessian trace. The paper reads these paired results as evidence that topological summary statistics capture the shape of the loss function in a way that complements, and sometimes reveals more than, visual inspection.","pith_inferences":["A natural extension the authors leave implicit is to use topological descriptors as training-time diagnostics: monitoring saddle count and persistence during optimization could reveal when a run is about to fail, before error metrics rise.","The 2D-projection caveat suggests a stress test: if the same correlations hold in 3D and higher subspaces, the method becomes a practical tool; if not, the current results are a property of 2D slices. This is testable with existing software.","The same pipeline could be applied to loss landscapes of other structured models, such as graph neural networks or transformers, where the parameter space is too high-dimensional for visual inspection but low-dimensional projections are still feasible.","Because the persistence diagram is computed from sublevel sets, it is sensitive to the exact choice of connectivity graph; checking whether the results are stable under different k values would show how much of the signal is intrinsic."],"forward_implications":["For image classifiers like ResNet-20, saddle count and average persistence offer a quantitative, parameter-free way to measure landscape smoothness and to compare architectures without needing to inspect plots.","For physics-informed neural networks, the same two numbers can flag the onset of optimization failure, since they rise together with absolute error and Hessian-based curvature measures as the convection problem gets harder.","Because merge trees and persistence diagrams are defined for arbitrary-dimensional functions, the approach extends naturally to sampling along more than two directions, potentially revealing structure invisible in 2D slices.","The merge tree's scale-free nature means it captures structure across different loss scales automatically, removing the need to manually choose visualization ranges to see small-scale features."],"supporting_citations":[{"why":"Supplies the random-projection method for sampling 2D loss landscapes that the paper follows in Eq. (4).","marker":"[Li et al., 2018]"},{"why":"Supplies the Hessian-based directions and the randomized numerical linear algebra approach used to compute top eigenvalues and trace.","marker":"[Yao et al., 2020]"},{"why":"Defines the PINN convection benchmark and established that increasing the convection coefficient makes optimization harder, which this paper's topology metrics confirm.","marker":"[Krishnapriyan et al., 2021]"},{"why":"Provides the algorithm for computing merge trees (contour trees) in all dimensions, which the paper uses to extract saddle counts.","marker":"[Carr et al., 2003]"},{"why":"Formalizes persistent homology and the persistence diagram, the basis for the average persistence metric.","marker":"[Edelsbrunner and Harer, 2008]"},{"why":"Provides the topology software toolkit used to compute merge trees and persistence diagrams.","marker":"[Bin Masood et al., 2021]"},{"why":"Supplies the approximate k-nearest-neighbor graph construction used to connect sampled loss values into a graph for TDA.","marker":"[Dong et al., 2011]"}],"fun_headline_variants":["Loss landscape saddles track neural network accuracy","Saddle count and persistence reveal loss landscape topology","Topology of loss surfaces tracks neural net performance","Persistent homology quantifies loss landscape shape","Saddles in loss landscapes signal model performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a two-dimensional slice through parameter space, defined by two random or Hessian-based directions, preserves enough of the loss landscape's topology that merge trees and persistence diagrams computed on that slice reflect the actual high-dimensional landscape.","fun_headline_variants_meta":{"raw":{"variants":["Loss landscape saddles track neural network accuracy","Saddle count and persistence reveal loss landscape topology","Topology of loss surfaces tracks neural net performance","Persistent homology quantifies loss landscape shape","Saddles in loss landscapes signal model performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000869,"raw_usage":{"total_tokens":3733,"prompt_tokens":883,"completion_tokens":2850,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":2781}},"tokens_in":499,"tokens_out":2850,"duration_ms":22589,"temperature":1.0,"reasoning_tokens":2781,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:17:20.857989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: compute merge trees on 3D and higher-dimensional subspaces, sampled along the top 3 to 10 Hessian eigenvectors, for the same trained ResNet-20 and PINN models. If the reported trends in saddle count and average persistence reverse or disappear as the subspace dimension grows, the 2D-projection topology is not representative of the true landscape. A complementary test is to vary the sampling density and the k-nearest-neighbor connectivity of the loss grid; if the relationships between saddle count, persistence, and accuracy or Hessian metrics change sign or vanish, the descriptors are artifacts of the discretization.","supporting_citations":[{"cited_title":"PyHessian : Neural networks through the lens of the hessian","cited_arxiv_id":null,"evidence_quote":"Supplies the Hessian-based directions and the randomized numerical linear algebra approach used to compute top eigenvalues and trace."},{"cited_title":"Characterizing possible failure modes in physics-informed neural networks","cited_arxiv_id":null,"evidence_quote":"Defines the PINN convection benchmark and established that increasing the convection coefficient makes optimization harder, which this paper's topology metrics confirm."},{"cited_title":"Computing contour trees in all dimensions","cited_arxiv_id":null,"evidence_quote":"Provides the algorithm for computing merge trees (contour trees) in all dimensions, which the paper uses to extract saddle counts."},{"cited_title":"Persistent H omology-a S urvey","cited_arxiv_id":null,"evidence_quote":"Formalizes persistent homology and the persistence diagram, the basis for the average persistence metric."},{"cited_title":"An overview of the T opology T ool K it","cited_arxiv_id":null,"evidence_quote":"Provides the topology software toolkit used to compute merge trees and persistence diagrams."},{"cited_title":"Efficient k-nearest neighbor graph construction for generic similarity measures","cited_arxiv_id":null,"evidence_quote":"Supplies the approximate k-nearest-neighbor graph construction used to connect sampled loss values into a graph for TDA."}],"review_version":1}