{"id":"d5df4cd2-a1e5-43fa-b0aa-086cf1012395","arxiv_id":"2412.15279","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Functional connectomes of neural networks, analyzed with persistent graph homology and exact Wasserstein distances, cluster networks by regularization strategy and input class with above-chance purity.","lead":"This paper treats neuron activations in a trained neural network like brain regions in an fMRI scan, builds functional connectomes from correlations between neurons, and summarizes their topology with persistent graph homology. The framework produces exact Wasserstein statistics in near-linear time, making topological analysis of large networks practical.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Clustering purity alone does not establish that persistent-graph-homology topology, rather than trivial activation statistics, drives the reported separability; no permutation or raw-activation baseline is provided.","rationale":"The paper's technical machinery is credible: the O(n log n) computation of persistent graph homology, the closed-form Wasserstein statistics, and the runtime scaling are genuine contributions and are not the source of the concern. The weak point is the empirical bridge from clustering purity to the strong interpretability claim. The reader's weakest assumption identified exactly this gap, and my analysis confirms it: the reported separability could be an artifact of the clustering procedure or of trivial activation statistics rather than of the topological decomposition. A raw-activation baseline and a sorted-edge-weight baseline would directly test whether the MST/non-MST split adds anything beyond the correlation distribution, while a permutation test would establish whether the purity values are statistically meaningful. Since these tests are missing, the current evidence supports only a conditional acceptance: the method is promising and computationally useful, but the central interpretability claim requires sharper validation. This does not change the reader's CONDITIONAL verdict, so the recommendation is UNCHANGED.","tokens_in":11746,"tokens_out":6734,"duration_ms":73305,"concrete_test":"In Study 2, run the identical 20-trial k-means purity pipeline on (i) raw per-neuron activation vectors for the same 200 class-conditioned functional datasets and (ii) globally sorted lists of all |rho| edge weights from the same adjacency matrices, and run a permutation test that shuffles class labels among connectomes 1000 times before clustering. If raw activations or sorted edge weights match or exceed Top, or if observed Top purity falls inside the permutation null distribution, the topological characterization claim is not supported; otherwise the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 3: \"topological signals, as measured by persistent homology methods, prove to be an effective means of characterizing neural network functions\") rests entirely on unsupervised purity against predefined classes. Two unaddressed alternatives could explain the results without any topological mechanism. First, k-means always produces purity greater than 1/K even on label-random data, so Study 2 values of 0.46-0.57 against a nominal random baseline of 0.1 are not evidence of significance; no permutation test is reported, even though the Potential Impact section advertises permutation tests as a benefit. Second, the connectome is built from per-neuron activation correlations, so the separating signal may be carried by low-order activation statistics (e.g., mean or variance of activations) or by the global distribution of pairwise correlations, not by the MST/non-MST decomposition that persistent graph homology introduces. The Adj baseline uses the same correlations but not the topological decomposition, and Top does not consistently beat it (e.g., Study 2, Fashion-MNIST/Batch Norm: Top 0.49 vs SWK 0.60; MNIST/Batch Norm: Top 0.46 vs WD 0.50). Because the claimed \"deeper understanding of underlying mechanisms\" is inferred solely from label-cluster alignment, the interpretability conclusion is underdetermined without these controls.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a framework for characterizing neural network function by constructing functional connectomes (absolute Pearson correlations between neuron activations) and summarizing their topology with persistent graph homology. The authors derive closed-form expressions for Wasserstein distances, barycenters, variances, and gradients of birth/death values, and use centroid-based clustering (Top) to group networks by regularization strategy (Study 1) and by input class (Study 2). Experiments on MNIST, Fashion-MNIST, and CIFAR-10 compare Top with six baselines and include a runtime study. The paper claims that topological signals effectively characterize neural network functions and that Top offers scalable exact computation.","tokens_in":11995,"tokens_out":7879,"duration_ms":73392,"significance":"If the central claim holds, the paper offers a threshold-free and scalable representation of network behavior, which would be a useful addition to interpretability tools. The strengths include a clean definition of functional connectomes, exact near-linear-time computation, no fitted parameters in the representation, public code, and a runtime comparison showing orders-of-magnitude speedup over conventional persistence diagram methods. However, the empirical validation does not yet establish that the topological decomposition itself, rather than low-order activation statistics or the clustering algorithm, is responsible for the reported separability; the paper also overstates the performance of Top relative to baselines. These issues are fixable with additional controls and corrected claims.","major_comments":[{"comment":"The comparison of purity scores (0.46–0.57) as \"significantly better than the 0.1 score expected if clustering was made randomly\" is statistically invalid because k-means with K=10 always produces purity greater than 1/K on label-random data, and the reported values are not tested against a permutation null. The Potential Impact section advertises permutation tests, but none is performed. Please add permutation or label-shuffling tests and report p-values or null distributions for the purity scores.","section":"Section 3, Cluster analysis and Study 2"},{"comment":"The statement \"In most settings, the proposed Top method outperforms other baselines\" is not supported by the tables. For example, in Table 2, Fashion-MNIST Vanilla SWK achieves 0.64 vs Top 0.53; Fashion-MNIST Batch Norm SWK 0.60 vs Top 0.49; MNIST Batch Norm WD 0.50 vs Top 0.46; and in Table 1, MNIST All SWK 0.85 vs Top 0.78 and CIFAR-10 All WD 0.98/SWK 0.96 vs Top 0.88. The paper should either provide a statistical comparison or reframe the contribution as scalability, not clustering accuracy.","section":"Section 3, Tables 1 and 2"},{"comment":"The clustering signal could be carried by trivial activation statistics or by the global distribution of pairwise correlations rather than by the topological birth/death decomposition, but no control on raw activation vectors is provided. The Adj baseline uses the same correlations and is much worse than Top, which is encouraging, but a k-means baseline on raw per-neuron activation vectors (or on the full correlation vector) is needed to support the claim that the \"deeper understanding of underlying mechanisms\" comes from topology. Please add such a control.","section":"Section 2 and Section 3, Cluster analysis"},{"comment":"Many cells in Tables 1 and 2 are reported without standard deviations (e.g., Table 1: WD 0.75, SWK 0.85; Table 2: several entries), and no significance tests are used to compare methods across the 20 trials. Since the tables are the primary evidence for the clustering claims, the missing uncertainty quantification makes it impossible to assess whether the observed differences are reliable.","section":"Section 3, Method comparison and Tables 1-2"}],"minor_comments":[{"comment":"The claim that the exact p-Wasserstein distance equals the Lp distance between sorted birth/death vectors is cited to prior work but not derived; a brief proof sketch or an explicit statement of the equal-cardinality and no-diagonal conditions would make the paper more self-contained.","section":"Section 2, Persistence Statistics"},{"comment":"The notation W_p,B appears in text but the equation uses W_p,B(G(1),G(2)) with a semicolon in one place; unify the notation throughout.","section":"Section 2, Eq. (5) and surrounding text"},{"comment":"The sentence \"As is common in machine learning, since we know a computable formula...\" is awkward and should be rewritten for clarity.","section":"Section 2, Functions of Neural Networks"},{"comment":"In the reference to Xiao et al., the author name is typeset as \"V ollgraf\"; this should be \"Volgraf\". Also, the in-text citation to \"Songdechakraiwut et al. 2023\" in the persistent graph homology paragraph should be cross-checked against the reference list to ensure the intended source is unambiguous.","section":"References"},{"comment":"The figure caption says \"Persistence diagrams and statistics for each strategy\" but the figure itself is not included in the submitted text; ensure the figure is legible and the shaded regions are clearly labeled as Wasserstein standard deviation.","section":"Section 3, Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a potentially useful application of persistent graph homology to neural network interpretability, with strong computational claims and reproducible code. The main weaknesses are in the validation and in the overstatement of comparative performance; these are fixable with additional experiments (permutation tests, raw-activation baselines, significance testing) and by revising the claims. The theoretical foundations are largely imported from the authors' prior work, which is acceptable but should be made more explicit in the main text. The paper fits the scope of the journal, though the novelty relative to prior persistent graph homology papers should be framed as the neural-network application and scalability story."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take: this paper does something genuinely useful and mostly sound, but the headline interpretability claim is under-validated. The authors adapt their prior persistent graph homology framework to neural-network functional connectomes, and the main win is computational: they compute exact Wasserstein distances between connectomes in O(n log n), and the runtime plot shows their Top method handling graphs with thousands of nodes in a second, where conventional persistence baselines die at a few hundred. That's a real contribution, and it is new relative to Zhang et al., which thresholded and stayed cubic.\n\nThe math is fine, as far as I can tell. The birth/death sets from persistent graph homology reduce to sorted edge weights in the MST and its complement, so the equal-cardinality sorted matching gives exact Wasserstein distances. They cite the proofs from their earlier papers rather than reproducing them, which is acceptable for an application paper. The two studies—clustering by regularization strategy and by input class—are reasonable demonstrations, and the code and data are linked.\n\nHere is where I lower my enthusiasm. The central claim that topological signals characterize neural network function rests entirely on clustering purity. The paper advertises permutation tests as a benefit but runs none. The purity comparison to a nominal 0.1 random baseline is weak because k-means on random labels will beat 1/K; without a permutation test, values like 0.46–0.57 in Study 2 are not established as significant. The stress-test note also lands: there is no raw-activation baseline. The Adj baseline uses the same correlation matrices without the topological decomposition, and Top does not consistently beat it—in Study 2, SWK and WD often outperform Top, and Adj is sometimes not far behind. So the unique contribution of the topology itself is not nailed down.\n\nThat said, the paper is not overreaching in every sentence; in Study 2 they present Top as one of several effective topological methods, not the clear winner. The scalability claim is solid, and the framework could be immediately useful for monitoring training or comparing models. The gaps are fixable: add permutation tests, add a baseline on raw activation vectors or simple correlation statistics, and temper the interpretability language.\n\nI would send this to review. It deserves referee time, but I would expect major revision. The method is valuable, the runtime advantage is real, and the empirical weaknesses are addressable rather than fatal. A reader working on interpretability or TDA will get something out of it.","headline":"A scalable topological summary for neural-network functional connectomes with a real runtime win, but the empirical validation leans on clustering purity without significance tests or a raw-activation control.","tokens_in":12512,"tokens_out":2917,"would_cite":true,"duration_ms":29420,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["55N31","62R40","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a neural network's functional connectome—the correlation graph of hidden-neuron activations—carries topological signatures that distinguish how the network was regularized and which input class it is processing.","keywords":["functional connectome","neural network interpretability","persistent graph homology","Wasserstein distance","topological data analysis","centroid-based clustering","regularization analysis","brain-inspired computing"],"falsifier":"Permute the class labels of the functional dataset across many trials and recompute Top's clustering purity; if purity stays well above chance, the class separation reflects input statistics rather than the claimed class-specific activation structure. Alternatively, run the same clustering on raw activation vectors; if they match Top's purity, no topological signal is needed.","tokens_in":11531,"feed_emoji":"🧠","tokens_out":6528,"duration_ms":57667,"temperature":0.7,"pith_summary":"This paper argues that a neural network's behavior can be read from the topology of its functional connectome, a graph whose edges are correlations between hidden neurons' activations across a set of inputs. The authors show that analyzing this brain-inspired representation with persistent graph homology separates networks trained under different regularization strategies and separates the internal processing of different input classes, all in near-linear time without thresholding. If the claim holds, it gives researchers a scalable, threshold-free way to probe what a trained network has learned and how it processes information, borrowing tools from human brain connectomics.","feed_headline":"Neuron-activity topology separates training styles and input classes","feed_subtitle":"Brain-inspired connectome summaries reveal how regularization and stimuli shape network behavior—in near-linear time.","key_machinery":"Persistent graph homology, a threshold-free topological summary that records births of connected components and deaths of cycles as an edge-weight threshold rises; for a complete weighted graph it reduces to the sorted maximum-spanning-tree edge weights (births) and the sorted non-MST edge weights (deaths). The paper's mechanism is the closed-form Wasserstein distance on these sorted vectors, $W_{p,B}(G^{(1)},G^{(2)})=\\lVert\\mathbf{b}_{G^{(1)}}-\\mathbf{b}_{G^{(2)}}\\rVert_p$ and analogously for deaths, which makes the Wasserstein barycenter a coordinate-wise average and enables Lloyd-style centroid clustering in $O(n\\log n)$ time.","core_discovery":"The central claim is that topological signals measured by persistent homology on neural-network functional connectomes characterize neural network function. A functional connectome is the complete graph on hidden neurons with edge weights given by the absolute Pearson correlation of their activation vectors over a functional dataset. Persistent graph homology tracks connected components and cycles across all correlation thresholds, and the paper presents closed-form Wasserstein distance, barycenter, variance, and gradient statistics for these summaries. Empirically, clustering these summaries separates regularization strategies (batch norm, dropout, L2, vanilla) with high purity and separates per-class connectomes at levels far above chance, with exact computation for thousands of nodes in about a second.","pith_inferences":["Beyond the paper: the sorted birth and death vectors are a compressed, threshold-free fingerprint of a network's activation geometry; a natural test is whether these fingerprints track generalization, calibration, or adversarial robustness across training runs and seeds.","Beyond the paper: because the Wasserstein barycenter and variance are closed-form, one could build topological analogues of PCA or regression on connectomes, or monitor how topology evolves during training, none of which the paper demonstrates.","Beyond the paper: the purity-based validation does not rule out that simple activation statistics (e.g., mean or variance of activations) drive the clusters; a permutation test or a baseline on raw activation vectors would test whether topology adds signal beyond those statistics."],"forward_implications":["Regularization strategy leaves a detectable topological fingerprint: clustering connectomes from batch norm, dropout, L2, and vanilla training separates the four strategies with high purity, and each pairwise comparison against vanilla reaches purity near 1.0 in most datasets.","Different input classes are processed through distinct functional mechanisms: per-class functional connectomes cluster with purity 0.5–0.6 across ten classes, far above the 0.1 random baseline.","Exact topology is computable at scale: Top computes exact Wasserstein distances for connectomes with thousands of nodes and millions of edges in roughly one second, while conventional persistent-homology baselines stall at a few hundred nodes.","Closed-form Wasserstein statistics provide a gradient-based tool: because barycenter, variance, and distance gradients are analytic, the representation can plug into centroid clustering and, potentially, other gradient-optimized machine learning objectives.","The method works beyond toy settings: it remains effective on the convolutional CIFAR-10 network by restricting analysis to the final fully-connected layers."],"supporting_citations":[{"why":"Defines persistent graph homology birth/death sets and the closed-form Wasserstein distance that the framework builds on.","marker":"Songdechakraiwut and Chung 2023"},{"why":"Supplies the Wasserstein stability theorem that grounds the robustness benefits claimed for the distance.","marker":"Skraba and Turner 2023"},{"why":"Establishes the functional connectome workflow and the thresholding limitation the paper aims to remove.","marker":"Bullmore and Sporns 2009"},{"why":"Provides the Pearson-correlation functional connectome construction that the paper adapts to neural networks.","marker":"Fornito, Zalesky, and Bullmore 2016"},{"why":"Prior application of persistent homology to neural-network functional connectivities; the paper contrasts its threshold-free scalable approach with this cubic-time, thresholded predecessor.","marker":"Zhang et al. 2023"},{"why":"Documents the cubic time complexity of conventional persistent homology that motivates the O(n log n) method.","marker":"Otter et al. 2017"},{"why":"Defines the purity score used to evaluate clustering performance in both studies.","marker":"Manning, Raghavan, and Schütze 2008"}],"fun_headline_variants":["Topology of neural activity separates training styles and inputs","Brain-inspired homology fingerprints neural network training","Persistent homology classifies regularization and input types","Near-linear topological fingerprints separate network training","Connectome topology reveals training style and input class"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that unsupervised cluster purity against predefined classes is evidence that the topology captures meaningful functional mechanisms; if the clusters are separable by trivial activation statistics or by the particular functional dataset chosen, the interpretability claim does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Topology of neural activity separates training styles and inputs","Brain-inspired homology fingerprints neural network training","Persistent homology classifies regularization and input types","Near-linear topological fingerprints separate network training","Connectome topology reveals training style and input class"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000502,"raw_usage":{"total_tokens":2378,"prompt_tokens":795,"completion_tokens":1583,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":411,"completion_tokens_details":{"reasoning_tokens":1515}},"tokens_in":411,"tokens_out":1583,"duration_ms":10786,"temperature":1.0,"reasoning_tokens":1515,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:05:44.964317+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Permute the class labels of the functional dataset across many trials and recompute Top's clustering purity; if purity stays well above chance, the class separation reflects input statistics rather than the claimed class-specific activation structure. Alternatively, run the same clustering on raw activation vectors; if they match Top's purity, no topological signal is needed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Pearson-correlation functional connectome construction that the paper adapts to neural networks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior application of persistent homology to neural-network functional connectivities; the paper contrasts its threshold-free scalable approach with this cubic-time, thresholded predecessor."},{"cited_title":"A.; Tillmann, U.; Grindrod, P.; and Harrington, H","cited_arxiv_id":null,"evidence_quote":"Documents the cubic time complexity of conventional persistent homology that motivates the O(n log n) method."},{"cited_title":"D.; Raghavan, P.; and Schütze, H","cited_arxiv_id":null,"evidence_quote":"Defines the purity score used to evaluate clustering performance in both studies."}],"review_version":1}