{"id":"120d9cd8-85be-4300-81e1-7516aeb226c2","arxiv_id":"2501.14964","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MetSelect improves node classification on several heterophilic datasets, preserves accuracy in deeper GNNs, and softens poisoning attacks by choosing each node's prediction layer through variance-normalized prototype distances.","lead":"MetSelect picks a different GNN layer for each node, selecting the layer whose distance to a class prototype is smallest after variance normalization, instead of always using the last layer. The authors report accuracy gains on heterophilic graphs, better depth stability, and improved resistance to graph poisoning in a plug-and-play way.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equations (4)–(5) define l* as the layer that minimizes the softmax probability of the true or nearest class, which inverts the stated prototype-distance objective; this makes the central algorithm as written impossible to reconcile with the reported gains.","rationale":"The reader's CONDITIONAL verdict is appropriate, but the most load-bearing issue is more specific and more formal than the stated weakest assumption about prototype reliability: the selection rule in Eqs. (4)-(5) is inverted relative to the stated objective. This matters because l* appears in every downstream result—Table 2, the depth experiments, the robustness experiments, and Figure 5 are all defined through this selection rule. If Eq. (5) is implemented literally during training, the model is asked to minimize the softmax probability of the true class in the chosen layer, which contradicts the method's motivating claim and should not produce the reported consistent improvements. The likely explanation is a typo in the min/max direction, but no code is provided and Section 7.2 itself mislabels the loss used for deep experiments, so the reader cannot determine which variant was actually run. A two-variant reimplementation is a cheap and decisive check: one literal version and one corrected version. This would resolve whether the empirical claims support the algorithm as stated, support a corrected algorithm, or support neither. The other concerns—cross-layer comparability of Mahalanobis distances, covariance invertibility on small graphs, and the limited robustness evaluation—are secondary; they become important only after the selection rule is unambiguous. I therefore keep the verdict unchanged at CONDITIONAL, contingent on correcting the formal definition and making the code available for verification.","tokens_in":14715,"tokens_out":5207,"duration_ms":49334,"concrete_test":"Re-implement MetSelect from the manuscript on Cora with GCN at depth L=2, using the same train/val/test splits and hyperparameters, under two variants: (A) Eq. (5) as written, i.e., argmin over layers of exp(-d_true)/Z, and (B) the corrected objective, i.e., argmin over layers of d_true (equivalently argmax of exp(-d_true)/Z). Compare test accuracy, validation-selected epoch, and the layer distribution of Figure 5 against Table 2. If variant A reproduces the reported numbers, the method's success is not explained by prototype proximity as claimed. If only variant B reproduces them, Eqs. (4)–(5) must be corrected and the formal description updated before the claims can be assessed. The same check should be run with the authors' released code, once available.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The entire empirical program rests on l*(v) computed in Eqs. (4)–(5). In Eq. (5), l*(vi|yi)=argmin_l exp(-d_{i,yi})/sum_c exp(-d_{i,c}), where d is the Mahalanobis distance to the class prototype. Because exp(-d)/Z is monotonically decreasing in d, this selects the layer in which the true class is least probable under the softmax-distance decoding, not the layer closest to the true prototype. Eq. (4) has the same inversion: min_y exp(-d_{i,y})/Z picks the class farthest from the node, and argmin over layers then selects the layer with the lowest confidence in that farthest class. The prose says MetSelect minimizes the distance to class prototypes, and Algorithm 1 trains using l* from Eq. (5). Either the formal definition is a typo, in which case it should be argmin of d or argmax of exp(-d_true)/Z, or the method actually implemented is not the one described. All downstream results—accuracy gains, depth behavior, robustness, and the l* distribution of Figure 5—are attributed to this selection rule, so the paper cannot be verified without resolving the inversion. The missing code and the Section 7.2 mislabeled loss (Equation 6 versus Equation 7) compound the ambiguity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MetSelect, a method that selects a personalized GNN layer for each node by comparing variance-normalized distances to class prototypes across the layers of a pretrained GNN encoder. The method is intended to be used as a plug-and-play replacement for final-layer prediction: for each node it chooses the layer whose representation is closest to the corresponding class prototype, trains per-layer decoders, and uses the selected layer at inference time. Experiments on 10 datasets with GCN, GAT, and GIN report consistent accuracy gains, especially on heterophilic graphs, and additional experiments report improved depth stability and robustness to structural poisoning attacks.","tokens_in":14991,"tokens_out":6508,"duration_ms":69627,"significance":"If the method works as described, the contribution is practically useful: it offers a simple, architecture-agnostic way to improve an existing GNN's predictions without changing message passing, and it is backed by a broad evaluation (10 datasets, 3 base GNNs, ablations, depth and robustness studies). The paper also provides explicit time and space complexity analyses. However, the central selection rule is defined in a way that contradicts the stated objective, and Algorithm 1 as written does not compute the moments before using them. These issues affect every reported result, so the empirical claims cannot currently be verified as stated.","major_comments":[{"comment":"The selection rule in Eqs. (4) and (5) is inverted relative to the stated objective. The quantity being minimized is a softmax probability of the form exp(-d)/Z, which is monotonically decreasing in the Mahalanobis distance d. Thus Eq. (5), argmin_l of the true-class softmax probability, selects the layer in which the true class is least probable under the distance-based decoding, not the layer closest to the true class prototype. Eq. (4) is similarly problematic: min_y over the softmax probability picks the least probable class, and the outer argmin over layers then selects the layer with the lowest confidence in that least probable class. This is not a 'minimum distance to a class prototype' rule. Since Algorithm 1 and all downstream experiments use Eq. (5), the authors must either correct these equations to argmin_l d_{i,y_i} (or equivalently argmax_l of the true-class softmax probability) and re-examine the experiments, or clarify that the implemented method is different from the one described.","section":"Section 5, Eqs. (4) and (5)"},{"comment":"The training procedure as written is internally inconsistent. In each epoch, the loop over training nodes accumulates \\hat{\\mu} and \\hat{K}_{l,0} and also computes the loss using l* (lines 6–9), but the class means \\mu^{(l)}_c and covariances K_l are only updated afterward at lines 12–13. Consequently, the l* used in the loss is not computed from the moments of the current forward pass; in the first epoch those moments are zero. This contradicts the surrounding text, which states that the moments are found first and then used to minimize the loss. The algorithm needs to be restructured so that the moments are computed before l* is evaluated, and the description of the actual implementation must be made consistent with this order.","section":"Algorithm 1, lines 3–13"},{"comment":"The experimental reporting for the depth and robustness studies confuses the two loss functions. Section 7.2 says 'we use the distance-based loss function in Equation 6' and then refers to 'the linear loss (i.e., Equation 6)', but Eq. (6) is the personalized cross-entropy loss while Eq. (7) is the distance-based loss. The robustness section likewise states that the model was trained using Eq. (7). It is therefore unclear which loss was actually used for Figure 3 and Figure 4, and this ambiguity affects the reproducibility of the depth and robustness claims. Please specify the loss used in each experiment and correct the cross-references.","section":"Section 7.2 and Figures 3–4"}],"minor_comments":[{"comment":"The covariance formula drops the summation over nodes in the second term: as written, \\sum_c 1[y_i=c] (\\mu^{(l)}_c)(\\mu^{(l)}_c)^\\top is not summed over i. It should be \\sum_i \\sum_c 1[y_i=c](\\mu^{(l)}_c)(\\mu^{(l)}_c)^\\top, or equivalently \\sum_c N_c \\mu^{(l)}_c (\\mu^{(l)}_c)^\\top as used in Algorithm 1.","section":"Section 5, Eq. (3)"},{"comment":"The ablation baseline MetSelect-max should be revisited after Eqs. (4) and (5) are corrected; if the implemented selection is actually the argmin of a softmax probability, then what is called the 'polar opposite' baseline may coincide with the intended criterion. The interpretation of this ablation depends on resolving the definitional issue.","section":"Section 7.4, Table 2"},{"comment":"There are several small naming and typographical inconsistencies, including 'Citseer' in Table 2, 'Ogba' in Table 3 versus 'ogbn-arxiv' elsewhere, and 'Mahlanabois' in Section 5. These should be cleaned up.","section":"Section 6 and Table 1"},{"comment":"The paper honestly notes that proving a particular layer is optimal is extremely hard; this limitation is acceptable for an empirical method paper, but it should be tied to the choice of prototypes and covariance estimates. The authors could acknowledge more explicitly that their selection rule is a heuristic that depends on the quality of the training-set prototype estimates.","section":"Section 8"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the evaluation is broad, but the manuscript currently has a load-bearing inconsistency: the formal selection rule in Eqs. (4)–(5) is the opposite of the stated prototype-distance objective, and Algorithm 1 computes the moments after using them. These are not merely presentation issues; they make the reported results unverifiable. I would ask the authors to supply corrected equations, a corrected and precise training loop, the actual code, and a confirmation that the reported numbers were produced with the intended selection rule. If the intended rule is the one in the prose, the experiments need to be rerun or revalidated under that rule. The paper is not ready for acceptance in its current form, but the issues are local enough that a major revision could address them."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the idea is worth taking seriously: MetSelect picks a per-node GNN layer by Mahalanobis distance to class prototypes, and the reported accuracy gains on heterophilic datasets, depth preservation, and poisoning robustness are the kind of result practitioners would use. Second, the paper as written cannot be verified because equations (4) and (5) define l* as the argmin over layers of the softmax probability of the true (or nearest) class—which, since exp(-d) decreases with distance, selects the layer where the node is least confident about that class, exactly the opposite of the stated objective. Either the equations are misprinted or the implemented method is not the one described. Given that the authors include a 'MetSelect-max' baseline that inverts the distance, I suspect a typo, but the burden is on them to fix it and release code.\n\nWhat is genuinely new: the variance-normalized, per-node layer selection is not in NDLS or JK-Net, and the plug-and-play framing is useful. The experiments cover 10 datasets and three GNNs, and the depth and robustness results, though narrow (one GNN, two datasets for robustness), are suggestive.\n\nSoft spots beyond the equation inversion: Section 7.2 refers to Equation 6 as the distance-based loss, but Equation 6 is the cross-entropy loss; Equation 7 is the distance loss. The self-loop paragraph claims consistent gains, but Table 4 shows MetSelect losing to FinalSelect on six of ten datasets, including all four homophilic ones. And there is no code, despite the claim that it will be open-sourced after publication.\n\nWho is this for? GNN users who want a cheap post-hoc way to select a better embedding layer. The idea deserves a serious referee and, conditional on fixing the equations, releasing code, and scoping claims, could be a solid contribution. But in its current form I would not rely on the numbers, and I would not cite it as a method until the definition is corrected.\n\nRecommendation: send to peer review, but ask for major revision and code release.","headline":"A novel and plausible layer-selection heuristic, but the defining equations select the least-confident layer and the self-loop table contradicts the text; needs major corrections before the results can be trusted.","tokens_in":15545,"tokens_out":4840,"would_cite":false,"duration_ms":40793,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Every node has its own best GNN layer, and MetSelect finds it without retraining.","keywords":["graph neural network","node classification","personalized layer selection","metric learning","oversmoothing","heterophily","adversarial robustness","prototype distance"],"falsifier":"For a labeled graph, train a GNN, then separately record which layer, if used for prediction, actually labels each node correctly; if those oracle-best layers match MetSelect's choices no more often than chance, the claim that MetSelect finds the optimal layer is false.","tokens_in":14501,"feed_emoji":"🎯","tokens_out":10402,"duration_ms":77139,"temperature":0.7,"pith_summary":"The paper challenges the standard practice of classifying every node from a GNN's final layer. Different nodes in a graph may need different amounts of neighborhood smoothing, and forcing one layer on all nodes can hurt accuracy. MetSelect measures, for each node, how close its representation in each layer is to the class prototype after variance normalization, and picks the layer with the smallest normalized distance. The authors show this per-node layer choice improves accuracy on heterophilic datasets, allows GNNs to go much deeper without the usual accuracy collapse, and makes them more robust to training-time link poisoning. If the paper is right, existing GNN representations already contain enough information for better prediction once the right layer is chosen for each node.","feed_headline":"Per-node GNN layer choice lifts accuracy up to 20%","feed_subtitle":"Picking the layer nearest a class prototype per node improves accuracy, depth, and attack resistance.","key_machinery":"The core object is the variance-normalized Mahalanobis distance from a node's representation in a layer to the class prototypes in that layer: $d^{(l)}_{i,c} = (\\tilde{h}^{(l)}_i - \\mu^{(l)}_c)^\\top K_l^{-1} (\\tilde{h}^{(l)}_i - \\mu^{(l)}_c)$, where $\\mu^{(l)}_c$ is the mean embedding of labeled nodes of class $c$ and $K_l$ is that layer's empirical covariance. MetSelect selects the layer $l^*(v)$ that minimizes the softmax of these distances over classes. This object carries the argument because it makes representations from different layers comparable, letting the method pick a layer without modifying the GNN's weights.","core_discovery":"The paper claims that every node in a graph has its own best representation depth inside a fixed GNN, and that this layer can be found without retraining the GNN. MetSelect compares each node's representation in every layer with per-class prototype embeddings and picks the layer that minimizes the variance-normalized Mahalanobis distance to the class prototypes. Using the selected layer for prediction instead of the final layer improves node classification accuracy on heterophilic graphs, keeps accuracy from collapsing when the GNN is deepened to 128 layers, and reduces the damage from untargeted poisoning attacks. The method works in a plug-and-play way with GCN, GAT, and GIN encoders without altering their message-passing weights.","pith_inferences":["A testable extension would be to check whether the same prototype-distance rule can select not just depth but also other per-node choices, such as which aggregation operator or which feature subset to trust.","The robustness gain hints that adversarial structural perturbations often corrupt higher-order neighborhoods first, so a per-node layer choice acts as a cheap, unintended defense; this is an inference, not a claim of the paper.","If the method is right, a GNN's learned representations are already rich across layers and the bottleneck is the fixed decoding layer, which suggests that decoder-side personalization deserves more attention than architecture changes."],"forward_implications":["GNNs can be trained to much greater depth (up to 128 layers) without the usual oversmoothing collapse in accuracy, because each node ignores layers that have already smoothed away its signal.","Node classification on heterophilic graphs improves by up to 20% over the final-layer baseline, since the method can automatically fall back to a node's own features (layer 0) when neighbors carry misleading label information.","Models trained with per-node layer selection are substantially more robust to untargeted structural poisoning attacks (Mettack): GCN+MetSelect preserved test accuracy that the standard GCN lost.","The improvement is plug-and-play: the same GCN, GAT, or GIN encoder is used unchanged, with only the prediction layer and the training loss replaced."],"supporting_citations":[{"why":"Supplies the GCN encoder and the Cora, Citeseer, and Pubmed datasets used as base model and homophilic benchmarks.","marker":"Kipf & Welling, 2016"},{"why":"Provides the message-passing formulation (Eq. 1) that defines each layer's representation.","marker":"Gilmer et al., 2017"},{"why":"Introduces variance-normalized metric learning for adaptive density discrimination, the basis of MetSelect's normalized distance.","marker":"Rippel et al., 2015"},{"why":"Establishes distance-based prototype classification, which MetSelect adapts to compare prediction confidence across layers.","marker":"Salakhutdinov & Hinton, 2007"},{"why":"Defines the Mettack poisoning attack and the robustness evaluation used to show MetSelect's defense.","marker":"Zügner et al., 2018"},{"why":"NDLS is the node-specific smoothing baseline whose layer choices are compared against MetSelect.","marker":"Zhang et al., 2021"},{"why":"JKNet is the layer-aggregation baseline; its attention variant AttnSelect is adapted for comparison.","marker":"Xu et al., 2018b"},{"why":"Provides the six heterophilic datasets and the random train/val/test splits used in the experiments.","marker":"Pei et al., 2020"},{"why":"Supplies the ogbn-arxiv dataset, the largest benchmark in the evaluation.","marker":"Hu et al., 2020"}],"fun_headline_variants":["Per-node layer choice improves GNN accuracy and depth","GNNs choose a distinct layer per node to improve accuracy","Personalized depth per node makes GNNs deeper and attack-resistant","Variable-depth GNN layers beat fixed-depth on accuracy","Plug-and-play per-node layer selection improves GNNs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method relies on the assumption that the layer whose examples of a class are closest to the class average, after scaling by how spread out the layer is, will also be the best layer for predicting unlabeled nodes.","fun_headline_variants_meta":{"raw":{"variants":["Per-node layer choice improves GNN accuracy and depth","GNNs choose a distinct layer per node to improve accuracy","Personalized depth per node makes GNNs deeper and attack-resistant","Variable-depth GNN layers beat fixed-depth on accuracy","Plug-and-play per-node layer selection improves GNNs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001947,"raw_usage":{"total_tokens":7589,"prompt_tokens":891,"completion_tokens":6698,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":6626}},"tokens_in":507,"tokens_out":6698,"duration_ms":34431,"temperature":1.0,"reasoning_tokens":6626,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:45:49.484225+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a labeled graph, train a GNN, then separately record which layer, if used for prediction, actually labels each node correctly; if those oracle-best layers match MetSelect's choices no more often than chance, the claim that MetSelect finds the optimal layer is false.","supporting_citations":[],"review_version":1}