{"id":"11859207-2bcb-49f2-bb60-1bfa3b449a35","arxiv_id":"2501.03464","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A graph neural network that mixes k-nearest-neighbor and fuzzy C-means cluster features outperforms transformer baselines on AudioSet, FSD50K, and ESC-50.","lead":"Researchers built an audio classification model that combines nearest neighbor links with fuzzy cluster information inside a graph neural network. They report it beats transformer baselines on three public audio benchmarks while using fewer parameters, and works well without ImageNet pretraining.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed advantage over AST is not established because LHGNN's results were produced with the PSLA training pipeline while AST's benchmark numbers are quoted from its original paper; pipeline, not architecture, may cause the gap.","rationale":"The reader identified the same load-bearing weakness: baseline numbers are quoted from their original papers rather than re-trained under identical conditions. My read agrees, and I found no other concern that is more load-bearing. The parameter count claim (31M vs 87M) is credible, and the graph kernel is a plausible inductive bias, but the central assertion of superior performance rests entirely on uncontrolled comparisons. The no-pretraining gaps are large, but they are likely inflated because AST from scratch was not given the PSLA training pipeline; modern augmentations substantially improve transformer training from scratch. The AudioSet margin of 0.007 mAP could easily reverse with repeated runs. The proposed concrete test—re-training AST under the exact LHGNN recipe—would settle whether the advantage is architectural or pipeline-driven. Since the reader already assigned CONDITIONAL, I do not move the verdict; I only reinforce that this concern is decisive for the paper's main claim.","tokens_in":6763,"tokens_out":5552,"duration_ms":50074,"concrete_test":"Retrain AST (same 87M architecture) from scratch on AudioSet and FSD50K using the exact LHGNN recipe: same log-mel features, same mixup alpha=0.5, same SpecAugment masks (time 192, freq 48), same AdamW settings, same 30-epoch schedule, and same checkpoint averaging. If the re-trained AST reaches approximately 0.44 mAP on AudioSet or 0.57 on FSD50K, the claimed 'distinct advantage without pretraining' is a pipeline artifact rather than evidence for the graph architecture. Additionally, run both models with 3–5 seeds and report confidence intervals; if the AudioSet gap is within noise, the headline outperformance claim should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that LHGNN outperforms transformer-based models with fewer parameters and especially without pretraining. Tables I–III compare LHGNN against AST numbers quoted from the original AST paper, but LHGNN is trained with the PSLA pipeline (mixup, SpecAugment, checkpoint averaging, 30 epochs, tuned k/K/P) while AST's published numbers come from its own pipeline. The text explicitly states that the training pipeline follows [20] and that scores were computed with checkpoint averaging. The no-pretraining comparison is the most fragile: AST from scratch (0.366 mAP on AudioSet, 0.396 on FSD50K) was trained without these modern augmentations, and a transformer trained from scratch with the same pipeline could plausibly score much higher. On AudioSet the pretrained gap is only 0.007 mAP (0.466 vs 0.459), which is within the range of seed noise; no error bars or significance tests are reported for AudioSet or FSD50K. Thus the paper's core claim conflates architecture with training recipe.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces LHGNN, a pyramid graph neural network for audio classification and tagging. Each LHG block updates tokens using two complementary relation types: k-NN local neighborhoods and Fuzzy C-Means cluster centers, fusing them with a max-relative graph convolution followed by a ConvFFN. The model is evaluated on AudioSet, FSD50K, and ESC50. The authors report that LHGNN surpasses AST accuracies while using fewer parameters, particularly without ImageNet pretraining, and they provide ablations on FSD50K supporting the combined kernel and fuzzy clustering.","tokens_in":7011,"tokens_out":4705,"duration_ms":44976,"significance":"If the comparison were controlled, the result would be a useful data point: a 31M-parameter GNN-style model competitive with an 87M-parameter transformer, with better from-scratch scores on AudioSet and FSD50K. The ablations in Tables IV and V are clearly presented and support the design choices of the combined kernel and Fuzzy C-Means clustering. However, the current evidence is not sufficient to support the headline claim because baseline scores are quoted from prior papers rather than re-trained under the same pipeline, and the main margins are small with no error bars. The paper does not provide code or pretrained models, limiting reproducibility, though this is not the central issue.","major_comments":[{"comment":"The central comparison to AST is confounded by training pipeline. LHGNN is trained with the PSLA pipeline (mixup, SpecAugment, checkpoint averaging) as stated in Section III-A.1, while AST's numbers are quoted from its original paper and use its own pipeline. On AudioSet the gap is 0.007 mAP (0.466 vs 0.459), and on FSD50K it is 0.016 (0.59 vs 0.574); these margins are small relative to typical seed variance. To support the claim that LHGNN outperforms transformer-based models, the authors should retrain AST (and ideally PSLA) under the identical training recipe, and report means and standard deviations over multiple seeds.","section":"III-A, Tables I and II"},{"comment":"The hyperparameters k and K are selected for best results and changed when ImageNet pretraining is used (k=25/K=10 without, k=9/K=5 with), but the manuscript does not describe a validation protocol or sensitivity analysis. Since the final comparisons hinge on these choices, and since k, K, and P are in effect free parameters, the paper should either fix them a priori or report robustness across ranges.","section":"II-C"},{"comment":"The claim of a 'distinct advantage in scenarios lacking ImageNet pretraining' is not supported on ESC50: LHGNN is only evaluated with ImageNet pretraining (96.2), and the no-pretraining baseline ERANN reaches 96.1, which the text itself acknowledges as 'equally well'. Please either add an LHGNN-from-scratch ESC50 result or soften the claim to be specific to AudioSet and FSD50K.","section":"III-C.2"}],"minor_comments":[{"comment":"Calling the operation in Eq. (3) a 'graph convolution' is misleading because no learned weights appear inside the max-relative aggregation; the only learnable projection is h(·) after concatenation. Please clarify or rename the operation.","section":"II-B1, Eq. (3)"},{"comment":"The statement that LHGNN without pretraining is 'comparable to AST with ImageNet pretraining' is based on a 0.001 mAP difference (0.573 vs 0.574), which is within noise; please add variance estimates or rephrase the claim.","section":"III-B"},{"comment":"The caption says 'different kernels', but the three rows are different input features to the same kernel; 'different input components' would be more accurate.","section":"IV, Table IV caption"},{"comment":"The k-NN step does not specify the distance metric; please state whether Euclidean distance is used.","section":"II-B1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is reasonably written and the architecture is understandable, but the evaluation needs a controlled baseline comparison before the main claim can be accepted. The ESC50 inconsistency with the abstract should be resolved. A retrained AST baseline under the PSLA pipeline would substantially strengthen the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. The core idea is simple and sensible: replace pairwise self-attention with a graph update that concatenates kNN-local differences and Fuzzy C-Means cluster differences, then runs an MLP. That specific fused kernel (Eq. 3) is not in the prior ViG or ATGNN work, and Table IV shows each branch alone gives 0.531/0.501 mAP on FSD50K while the fusion gives 0.573. So there is a real architecture contribution, and the ablations are clean. The no-pretraining experiments are the most interesting: LHGNN from scratch gets 0.442 on AudioSet vs AST's 0.366, and 0.573 on FSD50K vs AST's 0.396. If those held under matched conditions, that would be a meaningful result for low-resource audio.\n\nThe soft spot is exactly what the stress-test flags: LHGNN is trained with the PSLA pipeline (mixup, SpecAugment, checkpoint averaging, tuned hyperparameters), while the AST numbers are quoted from the original AST paper. The paper says AST/PSLA/LHGNN scores are checkpoint-averaged, but that doesn't make the training recipes equal. The pretrained gap on AudioSet is 0.466 vs 0.459 mAP, within seed noise; no error bars or significance tests are given. The from-scratch gaps are larger, but AST's published from-scratch numbers came from its own pipeline, not PSLA. So the paper's central claim that LHGNN outperforms transformers with fewer parameters is plausible but not established. The circularity burden is low: the evaluation is on public test sets, and hyperparameter tuning is standard. I don't see any invented entities or hidden fitting.\n\nWho is this for? People building audio taggers without ImageNet pretraining, and anyone who cares about graph-based alternatives to attention. A serious referee should see it, because the architecture is coherent and the no-pretraining result is worth pinning down. But the manuscript needs either a matched-pipeline re-comparison, error bars, or an honest softening of the headline claim. I'd send it to review, expecting major revision focused on evaluation comparability.","headline":"A sensible fused local/higher-order graph kernel for audio tagging, but the headline 'beats AST' claim is undercut by comparing LHGNN trained with the PSLA recipe against AST numbers from its own pipeline.","tokens_in":7541,"tokens_out":2068,"would_cite":true,"duration_ms":19899,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph network with local and cluster edges outperforms Transformer audio systems on three benchmarks.","keywords":["audio classification","audio tagging","graph neural networks","local-higher order graph convolution","Fuzzy C-Means clustering","k-nearest neighbor graph","AudioSet","FSD50K"],"falsifier":"Retrain AST and LHGNN on AudioSet with the identical training pipeline and compare mAP with confidence intervals; if AST matches or exceeds LHGNN's 0.466, the paper's outperformance claim fails.","tokens_in":6550,"feed_emoji":"🎧","tokens_out":9355,"duration_ms":78656,"temperature":0.7,"pith_summary":"Transformers model audio events through pairwise attention, which the paper argues leaves higher-order relations between patches underused. LHGNN instead treats spectrogram patches as graph nodes and updates each node from two sources at once: its k nearest neighbours, capturing local structure, and the Fuzzy C-Means centroids it most belongs to, capturing cluster-level relations. Across AudioSet, FSD50K, and ESC50, the reported results beat the Transformer-based AST on every benchmark while using 31M parameters versus AST's 87M. The gap is largest when no ImageNet pretraining is used, so LHGNN is presented as a more practical choice when large pretraining corpora are unavailable. If these results are correct, the practical consequence is that self-attention is not the only competitive way to relate audio structure; a graph kernel with explicit local and cluster edges can do it with a smaller model.","feed_headline":"Graph network outperforms audio Transformers on three benchmarks","feed_subtitle":"On AudioSet it scores 0.466 mAP versus 0.459, uses 31M parameters, and does not need ImageNet pretraining","key_machinery":"The load-bearing object is the Local-Higher Order Graph Convolution inside each LHG block. Given a flattened spectrogram feature map $X$, each node $x_i$ computes its k-NN set $S_i$ and, via Fuzzy C-Means clustering, membership scores to $P$ centroids; after updating the centroids, the $K$ centroids with highest membership form the set $L_i$. The node is updated as $x''_i = \\sigma(x_i \\oplus \\max(S_i - x_i) \\oplus \\max(L_i - x_i))$, where $\\sigma$ is an MLP with GELU non-linearity, and the result is projected back to the original dimension and added residually. The max-relative operations compare each node with its neighbours and with its cluster centroids along every feature dimension, and the concatenation fuses local and higher-order evidence into one vector; the paper shows this fusion is what lifts performance.","core_discovery":"LHGNN's central discovery is that fusing complementary local and higher-order graph relations lets a comparatively small graph network match or exceed transformer performance on audio tagging and classification. On AudioSet, LHGNN reaches 0.466 mAP with 31M parameters against AST's 0.459 with 87M; without ImageNet pretraining the gap widens in its favour (0.442 vs 0.366). On FSD50K it reaches 0.59 mAP with pretraining and 0.573 without, versus 0.574 for pretrained AST, while on ESC50 it reaches 96.2% accuracy versus 95.6% for AST. The paper attributes the gain to its proposed graph convolution, which concatenates the node with max-relative differences to local neighbours and to the top Fuzzy C-Means centroids, and its ablations show that each branch alone is weaker than the combination.","pith_inferences":["A natural untested extension is to apply the local-plus-cluster kernel to other spectrogram tasks such as music tagging or speech emotion recognition, where ImageNet pretraining is often unavailable and the reported pretraining-free gains would matter most.","The ablation's density-based clustering score (0.574) edges out Fuzzy C-Means (0.573), so a computationally efficient variant of density-based clustering might give a further gain; the paper does not test that.","Because LHGNN needs 30 epochs on AudioSet versus 5 for pretrained AST, parameter efficiency does not automatically mean compute efficiency; a fair comparison would also include total training FLOPs and wall-clock time."],"forward_implications":["If the reported AudioSet numbers hold, LHGNN beats AST by 0.466 to 0.459 mAP while using 31M parameters instead of 87M, making it a parameter-efficient alternative for weakly labelled sound event tagging.","Without ImageNet pretraining, LHGNN keeps 0.442 mAP on AudioSet and 0.573 on FSD50K, versus 0.366 and 0.396 for AST, so the architecture's advantage is largest precisely when pretrained backbones are unavailable.","The FSD50K ablation shows the combined kernel (0.573 mAP) outperforms the local-only kernel (0.531) and the cluster-only kernel (0.501), indicating that fusing both relation types is necessary for the reported gain.","With ImageNet pretraining LHGNN reaches top scores on FSD50K (0.59 mAP) and ESC50 (96.2% accuracy), so the model remains competitive in data-rich settings rather than only in data-scarce ones."],"supporting_citations":[{"why":"Supplies the AST Transformer baseline whose AudioSet, FSD50K, and ESC50 scores LHGNN is compared against.","marker":"[5]"},{"why":"The paper cites this as the source of the Fuzzy C-Means clustering and hypergraph-style projection behind the higher-order branch.","marker":"[8]"},{"why":"Supplies the pyramid architecture and the max-relative graph convolution that LHGNN adapts into its local-higher order kernel.","marker":"[10]"},{"why":"Supplies the ConvFFN block with depthwise convolution used after graph convolution to preserve local information.","marker":"[14]"},{"why":"Provides the AudioSet dataset and its original baseline mAP of 0.314.","marker":"[17]"},{"why":"Provides the FSD50K dataset and its baseline result.","marker":"[18]"},{"why":"Provides the ESC50 dataset and the five-fold evaluation protocol used for classification accuracy.","marker":"[19]"},{"why":"Supplies the training pipeline, including mixup, spectrogram masking, and checkpoint averaging, that LHGNN follows.","marker":"[20]"}],"fun_headline_variants":["Small graph network beats audio Transformers on three benchmarks","No ImageNet needed: LHGNN outperforms audio Transformers","Fuzzy C-Means boosts graph net past audio Transformers","AudioSet mAP 0.466: graph net beats AST with fewer parameters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on comparing LHGNN's numbers with published Transformer baselines that were trained under different pipelines, with no error bars or significance tests, so if those baselines were re-run under LHGNN's exact training protocol the reported margins could vanish or reverse.","fun_headline_variants_meta":{"raw":{"variants":["Small graph network beats audio Transformers on three benchmarks","No ImageNet needed: LHGNN outperforms audio Transformers","Fuzzy C-Means boosts graph net past audio Transformers","AudioSet mAP 0.466: graph net beats AST with fewer parameters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000744,"raw_usage":{"total_tokens":3285,"prompt_tokens":878,"completion_tokens":2407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":2333}},"tokens_in":494,"tokens_out":2407,"duration_ms":16173,"temperature":1.0,"reasoning_tokens":2333,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:51:57.128764+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain AST and LHGNN on AudioSet with the identical training pipeline and compare mAP with confidence intervals; if AST matches or exceeds LHGNN's 0.466, the paper's outperformance claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the FSD50K dataset and its baseline result."},{"cited_title":"Vision gnn: An image is worth graph of nodes,","cited_arxiv_id":null,"evidence_quote":"Supplies the pyramid architecture and the max-relative graph convolution that LHGNN adapts into its local-higher order kernel."},{"cited_title":"Orthogonal transformer,","cited_arxiv_id":null,"evidence_quote":"Supplies the ConvFFN block with depthwise convolution used after graph convolution to preserve local information."},{"cited_title":"Audioset,","cited_arxiv_id":null,"evidence_quote":"Provides the AudioSet dataset and its original baseline mAP of 0.314."},{"cited_title":"Esc-50: Dataset for environmental sound classification,","cited_arxiv_id":null,"evidence_quote":"Provides the ESC50 dataset and the five-fold evaluation protocol used for classification accuracy."},{"cited_title":"PSLA: Improving audio tagging with pretraining, sampling, labeling, and aggregation,","cited_arxiv_id":null,"evidence_quote":"Supplies the training pipeline, including mixup, spectrogram masking, and checkpoint averaging, that LHGNN follows."}],"review_version":1}