{"id":"14cb49c5-2851-46f0-bbc5-402c6c45445c","arxiv_id":"2411.08758","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper reports state-of-the-art node classification on five of six tested datasets using multi-scale products of a directed adjacency matrix, but the claimed scale invariance is not rigorously demonstrated.","lead":"This paper introduces scaled ego-graphs, which replace a single directed edge with longer sequences of directed edges, and claims node classification is invariant to this rescaling. It then builds ScaleNet, a graph network that combines several such scales and reports top accuracy on most of its benchmark datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Definition 7's invariance equality is not tested: Table 1 compares separately trained models per scale, and f is unquantified, so the paper's central explanation is unsupported.","rationale":"The paper separates into two parts: (1) the conceptual claim that node classification exhibits scale invariance, formalized in Definition 7 and used to explain DiGCN/SymDiGCN's success and to motivate combining scaled graphs; and (2) ScaleNet, an empirical architecture that combines multi-scale adjacency products. Part 2 is reasonably supported: Table 4 shows ScaleNet competitive or state-of-the-art, Wilcoxon signed-rank tests are reported, the weight-1 ablation (1iG versus DiG) is a useful and reproducible finding, and code is provided. Part 1 is the load-bearing weakness. Definition 7 quantifies over k but not over f, so the equality is not a well-defined graph property without specifying f. Section 3.2's Table 1 reports one accuracy per scaled graph from separate training runs, which can only demonstrate that good classifiers exist at each scale, not that the same classifier is invariant. The parenthetical 'after removing shared edges' still uses separately trained models. The text says 'confirming scale invariance' without a proof or a same-classifier check, despite Appendix D claiming proofs of all novel claims are included. On the heterophilic datasets, the table shows A^T A and A^T A^T accuracies close to the no-input baseline while AA and AA^T are much higher; this directional pattern is consistent with per-scale performance differences and makes the invariance reading especially doubtful. A direct same-weights test would settle whether Eq. 5 holds. Until then, the central explanatory claim is asserted rather than established, so the reader's REJECT is warranted. The empirical ScaleNet results could be salvaged by re-scoping the claim to performance invariance across scales, but that is a substantial revision, not a minor correction. The secondary inconsistency between the abstract's 'seven datasets' and Appendix A.1's 'six datasets' (with PubMed listed in Table 6 but absent from the main tables) reinforces the need for a careful revision but is not the primary reason for rejection.","tokens_in":21204,"tokens_out":5672,"duration_ms":54509,"concrete_test":"Train a single GNN (e.g., the encoder used for ScaleNet's A-branch) on the original adjacency matrix A for each dataset and split, freeze it, and evaluate the same weights on the scaled adjacency matrices A A^T, A^T A, A A, and A^T A^T, both with and without generated self-loops removed, using the same node features. Compute per-node prediction agreement between the original graph and each scaled graph on the test nodes, with mean and standard deviation over the available splits. Definition 7 predicts near-total agreement and comparable accuracy; if agreement instead mirrors Table 1's directional pattern (high for AA and AA^T, low for A^T A and A^T A^T on Chameleon/Squirrel), the equality fails and the claim must be weakened. Also rerun Table 1 with all 10/20 splits and error bars to rule out single-split artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, Definition 7 (Eq. 5), asserts f(G_v) = f(G^k(v)) for all k ≥ 1, but f is never quantified. As written, this is not a property of the graph unless f is fixed—e.g., an optimal classifier, a specific trained model, or an explicit architecture family. Section 3.2 and Table 1 do not test the equality: each column trains a separate GNN on a different scaled adjacency matrix, on a single split. Equal or comparable accuracies only show that some classifier can do well at each scale, not that the same classifier labels original and scaled ego-graphs identically. The parenthetical columns that remove shared edges still use separately trained models. Appendix D's reproducibility checklist states that proofs of all novel claims are included, but no proof of Definition 7 appears; instead, Table 1 is followed by 'confirming scale invariance.' The issue is load-bearing because the explanation of inception models' success and the motivation for combining scaled graphs rest on Eq. 5. On heterophilic graphs, the table actually shows A^T A and A^T A^T near the 'no input' baseline while AA and AA^T are high; that pattern is naturally read as per-scale performance differences, not as same-classifier invariance. The ScaleNet architecture could remain useful even if the claim were weakened to 'classification performance is roughly preserved across scales,' but the paper's central conceptual claim is asserted rather than established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the concept of \"scaled ego-graphs\" for directed graphs, obtained by ordered products of the adjacency matrix A and its transpose A^T. It claims that node classification is scale-invariant (Definition 7, Eq. 5), argues that this invariance explains the success of digraph inception models such as DiGCN and SymDiGCN, and proposes ScaleNet, a multi-scale architecture that combines features from these scaled graphs. The paper reports experiments on seven datasets (four homophilic, two heterophilic, plus an imbalanced setting), claiming state-of-the-art or tied-best accuracy on all datasets, and also proposes simplified inception variants with unit edge weights that outperform the original weighted versions. The code and detailed hyperparameter settings are provided.","tokens_in":21451,"tokens_out":4863,"duration_ms":42450,"significance":"If the scale-invariance claim were rigorously established, it would provide a principled explanation for the empirical success of multi-scale digraph models and motivate a unified treatment of homophilic and heterophilic graphs. The practical findings—that replacing DiGCN's expensive edge weights with constant weights improves or matches performance, and that ScaleNet achieves strong results across both graph types—are valuable and potentially publishable. The paper includes a reproducibility-conscious appendix with code, hyperparameter grids, and Wilcoxon signed-rank tests for the main comparisons. However, the central conceptual claim of scale invariance is not adequately supported by the evidence, which limits the significance of the theoretical framing.","major_comments":[{"comment":"The invariance equality f(G_v) = f(G^k(v)) is not tested. Table 1 reports accuracies of separately trained GNNs on different scaled adjacency matrices, each on a single split; this does not show that the same classifier f labels the original and scaled ego-graphs identically. As written, Eq. (5) is not a well-defined property of the graph unless f is specified (e.g., a fixed trained model, an optimal classifier, or an explicit architecture family). The statement in Section 3.2 that \"higher-scale graphs consistently achieve performance comparable to their lower-scale counterparts, confirming scale invariance\" is not a valid inference from those comparisons. Please either specify f and test same-classifier invariance (for example, train on one scale and evaluate on another, or compare the predictions of a single model on original versus scaled ego-graphs), or reformulate the claim as \"classification performance is roughly preserved across scales\" and adjust the motivation accordingly.","section":"Definition 7, Eq. (5) and Section 3.2"},{"comment":"The heterophilic results in Table 1 show strong scale dependence that contradicts the invariance claim. For Chameleon, A achieves 78 while AT achieves 30 (near the no-input baseline of 22), and AAT/AA achieve 68/70 while AT A/AT AT achieve 29/30. This is the opposite of f(G_v) = f(G^k(v)) for all k and all scaled variants; at best it shows that some scaled graphs preserve the discriminative information of A while others discard it. Please address this directly, either by testing Eq. (5) at the node level or by explicitly restricting the claimed invariance to the scales that preserve information, with a rationale for why those scales are the relevant ones.","section":"Table 1, heterophilic rows"},{"comment":"The formula AGG-Bα(M,N,X) = (1+α)α AGG(M,X) + (1+α)(1−α) AGG(N,X) does not match the following sentence. For α=0, the formula gives weight 0 on M and weight 1 on N (i.e., it uses only N), whereas the text states \"α=0 uses only M\"; for α=1, the formula gives weight 2 on M and 0 on N (i.e., only M), whereas the text states \"α=1 uses only N.\" Please correct either the formula or the text, and clarify how the reported α values in Table 4 (including the special modes α=2 and α=3 defined in Eqs. (7)-(8)) map to the formula, since the current inconsistency makes the method ambiguous.","section":"Section 4.2, Eq. (6)"},{"comment":"The checklist states that \"Proofs of all novel claims are included\" and that \"All theoretical claims are demonstrated empirically to hold,\" but no proof or direct verification of Definition 7 appears anywhere in the manuscript. The only evidence is the Table 1 accuracy comparison, which, as noted above, does not establish Eq. (5). Please either supply a proof for a clearly specified f, or correct the checklist to reflect that the scale-invariance claim is a hypothesis supported only indirectly by accuracy comparisons rather than a proven theoretical claim.","section":"Appendix D, Reproducibility Checklist"}],"minor_comments":[{"comment":"Figure 2 and Figure 4 appear to be the same figure with slightly different captions; please unify them or differentiate their content to avoid redundancy.","section":"Figures 2 and 4"},{"comment":"The word \"satisficing\" in the dataset checklist should be \"satisfying\" (or \"satisfactory\").","section":"Appendix D"},{"comment":"The footnote \"When accuracy is below 45%, only one split is used\" means that some entries (e.g., Dir-GNN on CiteSeer) are single-split results; please state this in the main text and clearly mark single-split entries so that the table does not mix single-split and multi-split statistics without indication.","section":"Table 3 footnote"},{"comment":"The final column labeled \"None\" is described as \"all zero input\"; consider renaming it to \"Zero input\" for clarity, and consider reporting multiple splits or confidence intervals for the single-split accuracy columns, since the invariance claim rests entirely on these numbers.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The empirical results for ScaleNet and the simplified inception models are interesting and may well be publishable after a major revision. The main concern is that the paper's central conceptual contribution—graph scale invariance as defined in Eq. (5)—is not supported by the current evidence and is even contradicted by the heterophilic rows of Table 1. I believe this is fixable by substantially reworking the claim and adding appropriate experiments, but it requires more than cosmetic changes. The reproducibility checklist overstates the theoretical support, and the inconsistency around Eq. (6) should be corrected as well. I would not recommend rejection if the authors are willing to revise the framing and provide the missing support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe central conceptual claim in this paper does not survive scrutiny, but the empirical work underneath is real and worth a referee's time. Scale invariance as stated in Definition 7—f(G_v) = f(G^k(v)) for the same classifier f—is asserted, not established. Table 1 compares separately trained models on single splits; that only shows that some classifier can do well at each scale, not that a single classifier is invariant. The stress-test note is on target: Section 3.2's 'confirming scale invariance' is a non sequitur. The paper's own reproducibility checklist says proofs of all novel claims are included, yet no proof of Definition 7 appears. This matters because the motivation for fusing scaled graphs rests on that equality.\n\nWhat is genuinely new: the weight-1 ablation. Replacing DiGCN's expensive edge weights with a constant weight of 1 improves accuracy on several datasets, including Telegram, Chameleon, and Squirrel, and is simpler and faster. That is a clean, reproducible empirical finding, and the code is available. The flexible bidirectional aggregation (per-dataset alpha, beta, gamma) is also new, and the tuning patterns are sensible—homophilic graphs benefit from self-loops and bidirectional aggregation, heterophilic graphs prefer removing self-loops and dropping opposite-direction scaled edges. The Wilcoxon signed-rank tests on 30 splits are more careful than typical in this subfield.\n\nSoft spots, in proportion: the abstract says seven datasets while the experiments use six—a simple but embarrassing inconsistency. The grid search over many per-dataset hyperparameters risks overfitting, though the significance tests mitigate that. And the invariance framing, as a formal property, is unsupported. It could be reframed as 'classification performance is roughly preserved across scales,' which is what Table 1 actually shows.\n\nWho is this for? Anyone working on directed GNNs or heterophilic graphs will get value from the architecture and the weight-1 result. The paper deserves peer review, but the referee should focus on the invariance claim and demand either a direct test of Definition 7 or an explicit weakening of the claim. I would not desk-reject it.","headline":"Solid empirical architecture with an unsupported central invariance claim; worth reviewing but needs major reframing.","tokens_in":22029,"tokens_out":3791,"would_cite":false,"duration_ms":31596,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Node classification in directed graphs is scale-invariant across adjacency-matrix products, the paper claims, and a model built on this invariance, ScaleNet, tops five of seven benchmarks.","keywords":["scale invariance","directed graphs","ego-graphs","node classification","homophily","heterophily","graph neural networks","multi-scale aggregation"],"falsifier":"Train a single GNN on the original adjacency matrix $A$ for Chameleon or Squirrel, then apply it without any retraining to the scaled matrices $AAT$, $AA$, $A^T A$, and $A^T A^T$; if accuracy on any scale drops toward the no-input baseline (roughly 22%–38%), the invariance $f(G_v)=f(G_k(v))$ is false for that dataset, regardless of the comparable separately trained numbers in Table 1.","tokens_in":20956,"feed_emoji":"🕸️","tokens_out":7513,"duration_ms":59023,"temperature":0.7,"pith_summary":"Inside a GNN, the object being classified is the ego-graph of the center node. The paper extends image-style scale invariance to directed graphs by defining scaled ego-graphs, built from ordered sequences of directed edges encoded as products of the adjacency matrix $A$ and its transpose $A^T$. It claims and empirically supports that these scaled versions preserve node labels: for any $k\\ge 1$, $f(G_v)=f(G_k(v))$. If this scale invariance holds, the products of adjacency matrices can be fused without destroying label information, which explains why DiGCN and SymDiGCN succeed on homophilic graphs and why they fail on heterophilic ones. The paper then introduces ScaleNet, which flexibly combines scaled graphs with a directional weight parameter and achieves state-of-the-art accuracy on five of seven datasets while matching the top model on the other two.","feed_headline":"Scale invariance holds in directed graph classification","feed_subtitle":"A unified model, ScaleNet, fuses multi-scale edges to top five of seven node-classification benchmarks.","key_machinery":"The central object is the scaled adjacency matrix set $\\mathcal{A}^k = \\{a_1 a_2 \\cdots a_k \\mid a_i \\in \\{A, A^T\\}\\}$, whose matrices encode $k$th-scale edges (ordered sequences of directed in- and out-hops); a scaled ego-graph is the ego-graph reachable through one such scaled-edge type. Definition 7, $f(G_v) = f(G_k(v))$, is the load-bearing identity: it is what licenses fusing multiple scaled graphs without destroying label information. The architecture that exploits it is the bidirectional aggregation block $AGG\\text{-}B_\\alpha(M,N,X) = (1+\\alpha)\\alpha\\, AGG(M,X) + (1+\\alpha)(1-\\alpha)\\, AGG(N,X)$, whose parameter $\\alpha$ selects, balances, unions, or intersects opposite-direction scaled matrices, combined layer-wise by a COMB function (Jumping Knowledge or addition) and optionally with self-loops, batch norm, and nonlinearities.","core_discovery":"The paper's central claim is Definition 7: for any $k \\ge 1$, $f(G_v) = f(G_k(v))$, meaning the classification of a node does not change when its ego-graph is replaced by a $k$th-scale ego-graph formed from ordered sequences of in- and out-edge hops. Table 1 shows, on seven benchmark datasets, that higher-scale graphs reach accuracies comparable to their lower-scale counterparts, even after removing edges shared with the lower scale; the paper reads this as confirming scale invariance. It further finds that on homophilic graphs both directions $A$ and $A^T$ carry signal, while on heterophilic graphs $A$-based scales preserve performance but $A^T$-based scales degrade because many nodes have no in-neighbors. From this it concludes that the success of digraph inception models rests on scale-invariant multi-scale fusion, that their expensive edge-weight computation is unnecessary (constant weight 1 or even random weights match or beat it), and that a direction-aware fusion of scaled graphs, ScaleNet, unifies homophilic and heterophilic node classification.","pith_inferences":["The paper's Table 1 trains a separate classifier per scale, so it does not directly test whether one fixed classifier labels original and scaled ego-graphs identically; verifying Definition 7 would require training once on $A$ and evaluating on the scaled matrices without retraining.","The consistent $A$-versus-$A^T$ asymmetry on heterophilic graphs suggests the grid-searched direction parameters $\\alpha, \\beta, \\gamma$ could be replaced by a simple data statistic, such as the share of nodes with zero in-neighbors.","If constant-weight scaled edges keep beating random-walk-weighted ones, the label signal in higher-order scales may be purely topological; other proximity-based GNNs could similarly drop their spectral-weight machinery.","If scale invariance is confirmed for a fixed classifier, scaled ego-graphs become a principled, non-random data-augmentation scheme for node classification, with potential benefits for robustness to distribution shift."],"forward_implications":["Scale invariance makes multi-scale adjacency products safe to combine, explaining the empirical success of DiGCN and SymDiGCN on homophilic graphs.","The expensive eigenvalue- and normalization-based edge weights in those methods are unnecessary; constant unit weights give equal or better accuracy, and random weights can also outperform the learned ones.","On heterophilic graphs scale invariance is directional: $A$, $AA$, and $AAT$ preserve information, while $A^T$, $A^T A$, and $A^T A^T$ inject noise and zeros; excluding the bad direction recovers performance.","A single tunable architecture, ScaleNet, can thereby match or beat specialized models on both graph types, reaching state-of-the-art accuracy on five of seven datasets and matching on the remaining two.","Constant-weight scaled edges avoid the out-of-memory failures of SymDiGCN and DiGCN on larger datasets, improving scalability."],"supporting_citations":[{"why":"Defines DiGCN(ib), the digraph inception model whose success the paper attributes to scale invariance and whose edge-weight computation it replaces with constant unit weights.","marker":"Tong et al. 2020a"},{"why":"Defines SymDiGCN, the second inception baseline, whose incidence-normalization weights the paper shows to be unnecessary.","marker":"Tong et al. 2020b"},{"why":"Introduces Dir-GNN, the strongest heterophilic baseline and the source of the bidirectional in/out aggregation scheme ScaleNet generalizes.","marker":"Rossi et al. 2024"},{"why":"Supplies the GCN message-passing layer used as the AGG function inside ScaleNet's aggregation block.","marker":"Kipf and Welling 2016"},{"why":"Supplies the Jumping Knowledge mechanism used as COMB1/COMB2 for layer-wise fusion of scaled features.","marker":"Xu et al. 2018"},{"why":"Provides the $\\alpha$-depth ego-graph definition that scaled ego-graphs extend.","marker":"Alvarez-Gonzalez, Kaltenbrunner, and Gómez 2023"},{"why":"Gives the generalization-error argument for why enforcing invariance in classifiers is desirable, motivating the paper's premise.","marker":"Sokolic et al. 2017"},{"why":"Introduces MagNet, a Hermitian baseline, and provides the Telegram dataset splits used in experiments.","marker":"Zhang et al. 2021"}],"fun_headline_variants":["ScaleNet: scale-invariant learning boosts directed graph classification","Scale invariance in directed graphs: ScaleNet tops five of seven benchmarks","ScaleNet: multi-scale directed edges unify homophilic and heterophilic graphs","Scale invariance holds for directed graphs: ScaleNet leads node classification","ScaleNet: scale-invariant learning tops directed graph benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's evidence for Definition 7 is that separately trained GNNs reach similar accuracy on different scaled graphs; it assumes this implies one and the same classifier would label original and scaled ego-graphs identically, but Table 1 never shows that.","fun_headline_variants_meta":{"raw":{"variants":["ScaleNet: scale-invariant learning boosts directed graph classification","Scale invariance in directed graphs: ScaleNet tops five of seven benchmarks","ScaleNet: multi-scale directed edges unify homophilic and heterophilic graphs","Scale invariance holds for directed graphs: ScaleNet leads node classification","ScaleNet: scale-invariant learning tops directed graph benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001321,"raw_usage":{"total_tokens":5428,"prompt_tokens":1041,"completion_tokens":4387,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":4299}},"tokens_in":657,"tokens_out":4387,"duration_ms":29834,"temperature":1.0,"reasoning_tokens":4299,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:23:10.742883+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a single GNN on the original adjacency matrix $A$ for Chameleon or Squirrel, then apply it without any retraining to the scaled matrices $AAT$, $AA$, $A^T A$, and $A^T A^T$; if accuracy on any scale drops toward the no-input baseline (roughly 22%–38%), the invariance $f(G_v)=f(G_k(v))$ is false for that dataset, regardless of the comparable separately trained numbers in Table 1.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Dir-GNN, the strongest heterophilic baseline and the source of the bidirectional in/out aggregation scheme ScaleNet generalizes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the $\\alpha$-depth ego-graph definition that scaled ego-graphs extend."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the generalization-error argument for why enforcing invariance in classifiers is desirable, motivating the paper's premise."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces MagNet, a Hermitian baseline, and provides the Telegram dataset splits used in experiments."}],"review_version":1}