{"id":"9a3a2c72-ee15-4714-975f-28d4d0a53dae","arxiv_id":"2504.21366","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"DGFNet combines bottleneck gated fusion with a transformer decoder for audio-visual source separation and reports modest SDR/SIR/SAR gains on MUSIC and MUSIC-21.","lead":"This paper adds a learned gating fusion module to an existing audio-visual separation model, letting the network weight how much it relies on visual versus audio features at each input. On two music separation benchmarks it reports small gains in separation quality over the iQuery baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline mismatch between Table 1 (iQuery 10.63) and Table 3 (Baseline 10.95) makes the claimed 0.62 dB SDR gain uninterpretable; without seed-level variance the fusion module is not isolated.","rationale":"The reader's weakest assumption is exactly the inconsistency between Table 1 (iQuery 10.63) and Table 3 (Baseline 10.95), and I agree that this is the load-bearing point. The paper's contribution is modest and plausible—a gated convex combination of audio and audio-visual features at the bottleneck—and the method is described without internal contradictions. The central claim, however, is empirical, and the empirical support is thin: a 0.2–0.3 dB difference, no error bars, no seed count, no significance test, no separate ablation of the audio attention module, and no released code. The mismatch between the two baseline numbers means the reader cannot tell whether the reported 0.62 dB gain is against a fair same-environment baseline or against a literature number from a different evaluation setup. This does not warrant rejection, because the flaw is addressable and the method may well work; it does warrant keeping the verdict CONDITIONAL pending the controlled comparison and code release.","tokens_in":12387,"tokens_out":5241,"duration_ms":51338,"concrete_test":"Run one controlled experiment on MUSIC with identical data split and preprocessing: train (i) iQuery, (ii) iQuery + DGFM, (iii) iQuery + audio attention, and (iv) full DGFNet, each with at least 3 random seeds; report per-track mean SDR, standard deviation, and paired 95% confidence intervals. If the iQuery condition reproduces at ~10.95 rather than 10.63, the headline gain shrinks to 0.30 dB; if the paired DGFM-vs-iQuery confidence interval includes 0, dynamic gating is not established as the cause. Releasing code and seeds is required to make this check reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that DGFM improves separation by 0.62 dB SDR over iQuery on MUSIC and 0.36 dB on MUSIC-21—depends on a baseline that is not consistently defined across the paper. Table 1 reports iQuery at 10.63 dB, while Table 3 lists the no-fusion Baseline at 10.95 dB. Using the Table 3 baseline, the full DGFNet gain is 0.30 dB and the DGFM-only gain is 0.22 dB, not 0.62 dB. The paper gives no seed count, variance, or significance test, and releases no code, so differences of this size are within plausible run-to-run noise. The audio attention module is also not ablated separately, so even the smaller gain cannot be attributed to dynamic gating alone. On MUSIC-21 no fusion ablation is reported at all. The architecture is a straightforward extension of iQuery and the method description is internally consistent; the weakness is in the empirical support, not in the derivation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DGFNet, an end-to-end audio-visual source separation model built on the iQuery architecture. The main additions are a Dynamic Gating Fusion Module (DGFM) that combines intermediate audio features with object-level visual features at the U-Net bottleneck using a learned sigmoid gate, and an audio attention module inserted after each upsampling layer of the U-Net decoder. The authors report SDR gains of 0.62 dB over iQuery on MUSIC and 0.36 dB on MUSIC-21, with corresponding SIR and SAR improvements, and present an ablation study of fusion variants plus an analysis of the learned gating weight distribution. The paper claims to be the first to combine bottleneck feature fusion with decoder-side interaction for this task.","tokens_in":12544,"tokens_out":3243,"duration_ms":35210,"significance":"If the reported gains are reproducible, DGFM is a simple and plausible improvement over iQuery: the gating mechanism is a natural way to let the model down-weight the visual modality when it is uninformative, and the audio attention module is a reasonable enhancement to the U-Net decoder. The paper also provides qualitative spectrogram comparisons and a distributional analysis of the gating weights, which help explain the mechanism. However, the empirical support is currently not strong enough to establish the central claim. The baseline SDR for iQuery is 10.63 dB in Table 1 but 10.95 dB in Table 3, no variance or significance testing is reported, no code is released, and the ablation does not isolate the audio attention module from DGFM. These issues make the headline 0.62 dB gain uninterpretable as evidence for the proposed components. The contribution is therefore significant only conditionally on the experiments being made consistent and statistically reliable.","major_comments":[{"comment":"The baseline is inconsistent across the two tables. Table 1 reports iQuery at 10.63 dB SDR on MUSIC, while Table 3 lists the \"Baseline\" at 10.95 dB SDR on the same dataset. Recomputing the gains from Table 3, DGFNet improves over the no-fusion baseline by only 0.30 dB, and the DGFM-only variant improves by 0.22 dB, not the 0.62 dB claimed in Section 4.2.1. The authors must explain why the two baselines differ, specify the exact training/evaluation protocol for each row, and report all comparisons under identical conditions. Without a consistent baseline, the improvement cannot be attributed to the fusion module.","section":"Tables 1 and 3, Sections 4.2.1 and 4.2.3"},{"comment":"No statistical reliability is established. The paper reports no number of training runs, no standard deviation or error bars, and no significance tests for any metric. The claimed improvements are on the order of 0.22 to 0.68 dB, which can fall within run-to-run variation for this type of model. The abstract's statement that the method achieves \"significant performance improvements\" is therefore not supported by the evidence presented. The authors should report results across multiple seeds with means and variances, and ideally a paired significance test against the reproduced iQuery baseline.","section":"Section 4.2, Tables 1 and 2"},{"comment":"The ablation does not isolate the contribution of the audio attention module. The row \"DGFNet(Ours)\" includes both DGFM and the audio attention module, while \"+DGFM\" includes only the fusion module. Consequently the difference between \"+DGFM\" and \"DGFNet(Ours)\" reflects the audio attention module, but this component is never ablated separately against the baseline. Moreover, no fusion ablation is reported on MUSIC-21 at all. The authors should provide separate ablations for DGFM and audio attention on both datasets before claiming that dynamic gating is the key component.","section":"Table 3, Section 4.2.3"}],"minor_comments":[{"comment":"The dimensions in the DGFM description are underspecified: object features F_O are introduced as R^{C_O}, while intermediate audio features F_mid are R^{C_A x F_S x T_S}. The paper says the two are fused by pixel-wise multiplication along the channel dimension, but it does not state how the object features are projected or broadcast to the audio feature map. This should be spelled out for reproducibility.","section":"Section 3.3, Figure 4"},{"comment":"The row labels are ambiguous. \"Baseline\" is described as \"the model without fusion at the bottleneck layer,\" which should coincide with the iQuery baseline from Table 1, yet the numbers differ. The caption should state explicitly which model components are present in each row, including the audio attention module.","section":"Table 3"},{"comment":"The weight distribution analysis is descriptive only. To support the claim that gating adaptively balances modalities, the authors should correlate the learned sigma values with separation quality or with visual-audio correspondence for individual test samples.","section":"Section 4.2.4, Figures 6 and 7"},{"comment":"The text says that some baseline numbers are taken from [2] and that iQuery was run in the authors' environment with \"the same training settings,\" but it is not stated which other methods were reproduced locally and which were copied. A clear statement of reproduced versus cited results is needed for fair comparison.","section":"Section 4.1.3"},{"comment":"The claim \"we are the first to propose the combination of bottleneck feature fusion with decoder interaction decoding strategy\" is not substantiated by a literature search or a positioning discussion against methods that use cross-modal attention or gating in related tasks. The statement should be softened or supported with evidence.","section":"Section 1, Contributions"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of ICMR and the proposed architecture is a reasonable incremental extension of iQuery. The main barrier is empirical: the baseline discrepancy between Tables 1 and 3, the absence of any uncertainty quantification, and the incomplete ablation prevent the central claim from being assessed. I would ask the authors to resolve these issues and add reproducibility details before acceptance; I see no circularity problem in the method itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my read on DGFNet. The new content is narrow but real: a dynamic gating fusion placed at the bottleneck of iQuery's decoder-only architecture, plus a lightly modified EMA attention module in the audio U-Net. That specific combination hasn't been tested before, so there's a small novelty. The paper is clearly written, the method description is internally consistent, and the qualitative results and gating-weight distribution plot are nice. The citation pattern is fair—they engage with iQuery and the relevant prior AV-separation work.\n\nThe soft spots are load-bearing for the headline empirical claim. Table 1 reports the iQuery baseline at 10.63 dB SDR on MUSIC; Table 3 reports the no-fusion baseline at 10.95 dB. Since both are supposed to be the same underlying model, the mismatch is unexplained. Using the Table 3 baseline, the full-model gain is 0.30 dB and the fusion-only gain is 0.22 dB, not the claimed 0.62. With no seed count, no variance, no significance test, and no released code, those differences are within plausible run-to-run noise. The audio attention module is not ablated separately either, so even the smaller gain cannot be cleanly attributed to the gating fusion. And on MUSIC-21 there is no fusion ablation at all. The 'first to combine bottleneck fusion with decoder interaction' claim is also overstated, since gating fusion is common elsewhere; applying it at this specific point is an incremental step.\n\nThe core idea is not disproven. The architecture is reasonable, the training setup follows iQuery, and the direction of the results is consistent with intuition. But as written, the empirical support is not tight enough to validate a real improvement. The authors can fix this: rerun all baselines under identical conditions, report per-seed variance, isolate the attention module in the ablation, and release code. With those changes this could be a useful incremental contribution for the AV-separation community.\n\nMy recommendation: send it to peer review, because the idea deserves a careful look and the issues are fixable. I would not cite it in its current form as evidence of an improvement.","headline":"Plausible incremental architecture, but the ablation table contradicts the main baseline, making the headline gain uninterpretable; send to review with a request for variance and code.","tokens_in":13091,"tokens_out":2974,"would_cite":false,"duration_ms":28765,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learnable gating weight at the encoder bottleneck lets audio-visual source separation choose per sample how much vision to trust, gaining 0.62 dB SDR over the iQuery baseline on MUSIC.","keywords":["audio-visual source separation","dynamic gating fusion","gating mechanism","audio attention module","MUSIC dataset","MUSIC-21 dataset","encoder-decoder architecture","cocktail party problem"],"falsifier":"Reproduce iQuery with the identical training recipe (same STFT, object detector, optimizer, and test split) and compare SDR. If the faithful baseline reaches 10.95 dB—the value listed in the ablation table—rather than the 10.63 dB in the main table, DGFNet's advantage shrinks from 0.62 to about 0.30 dB, and the claimed margin depends on an unexplained setup difference rather than the gating module alone.","tokens_in":12145,"feed_emoji":"🎧","tokens_out":7781,"duration_ms":70098,"temperature":0.7,"pith_summary":"The paper tries to establish a better way to combine audio and visual information for audio-visual source separation. It argues that both common fusion designs—early bottleneck fusion and decoder-only interaction—lose or confuse modality-specific information, and proposes DGFNet, which inserts a learned gating coefficient at the encoder bottleneck so the model can decide per input how much visual evidence should shape the audio representation. If the claim is right, the gating module is a general fix: it improves separation quality over the iQuery baseline by 0.62 dB SDR on MUSIC and 0.36 dB SDR on MUSIC-21, and it behaves as intended, with the gating weight concentrated near 0.5 but able to shift for ambiguous or clear visuals.","feed_headline":"Gating fusion lifts audio-visual separation by 0.62 dB","feed_subtitle":"DGFNet dynamically weighs visual and audio cues per input, gaining 0.36 dB on the larger MUSIC-21 set.","key_machinery":"The load-bearing mechanism is the Dynamic Gating Fusion Module (DGFM): a data-dependent convex combination of a fused audio-visual feature and the unmodified audio feature at the encoder bottleneck. A sigmoid gating weight $\\sigma$ is computed from $1\\times1$ convolutional projections of both candidates, and the final fused feature is $\\sigma F_{av} + (1-\\sigma)F_{mid}^a$. This replaces the fixed channel-wise concatenation or pixel-wise multiplication used by earlier systems, letting the network modulate the modality contribution per sample. The paper also adds an audio attention module adapted from the Efficient Multi-scale Attention module to the decoder upsampling path, and feeds the fused features into a query-based Audio-Visual Transformer decoder.","core_discovery":"The central claim is that a Dynamic Gating Fusion Module (DGFM) placed at the bottleneck of a U-Net audio encoder improves audio-visual source separation by adaptively reweighting audio and visual features. The fused audio-visual feature $F_{av}$ is computed by pixel-wise multiplication of object features and intermediate audio features; both $F_{av}$ and the audio feature $F_{mid}^a$ pass through $1\\times1$ convolutions, their outputs are summed, and a sigmoid produces the gating coefficient $\\sigma$. The module replaces the audio feature with $\\sigma F_{av} + (1-\\sigma)F_{mid}^a$, so the model can lean on vision when it is informative and fall back on audio when it is not. The paper reports that DGFNet outperforms the iQuery baseline on both datasets, and its ablation shows that removing only the gating module reduces SDR, supporting the module as the source of the gain.","pith_inferences":["The gating coefficient could be conditioned on an explicit reliability estimate, such as object-detection confidence or estimated signal-to-noise ratio, making the adaptation interpretable and testable; the paper learns it implicitly.","One could compare the distribution of $\\sigma$ across instrument categories to check whether visually salient instruments receive systematically higher visual weights than occluded or small ones.","The same convex gating formula is a natural fit for other audio-visual tasks with time-varying modality reliability, such as active source separation or audio-visual navigation."],"forward_implications":["Other encoder-decoder audio-visual separation models can adopt the same bottleneck gating without altering their decoder, since DGFM only replaces the intermediate audio feature.","The gain transfers across datasets with different instrument vocabularies (11 vs 21 classes), so the mechanism is not overfit to a single distribution.","Because the learned $\\sigma$ stays near 0.5 on average but shifts at the extremes, the dynamic part mainly matters for hard or ambiguous visual inputs; a fixed equal-weight fusion would lose those cases.","Injecting vision at the bottleneck rather than only in the decoder means the generated separation mask is informed by visual evidence earlier, which should improve separation when the visual object is detectable."],"supporting_citations":[{"why":"supplies the iQuery baseline, the query-based Audio-Visual Transformer decoder, and the comparison numbers DGFNet extends","marker":"[2]"},{"why":"provides the object-detection visual features and the U-Net style audio feature extraction that DGFNet builds on","marker":"[10]"},{"why":"introduces the mix-and-separate framework, the MUSIC dataset, and the ground-truth mask training objective","marker":"[43]"},{"why":"is the Efficient Multi-scale Attention module from which the audio attention module is adapted","marker":"[22]"},{"why":"is the Faster R-CNN object detector used to extract object-level visual features","marker":"[27]"},{"why":"provides the MUSIC-21 dataset and motion-feature context used in the larger benchmark","marker":"[42]"},{"why":"supplies the pre-trained I3D motion encoder used to compute motion features","marker":"[5]"}],"fun_headline_variants":["Dynamic gating fuses audio and vision for cleaner separation","Adaptive gating blends audio and visual cues for better separation","Audio-visual separation improved by dynamic gating fusion","Gating module adaptively reweights audio and visual features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim assumes the iQuery baseline was reproduced faithfully and that the main-table and ablation-table numbers are comparable; if those conditions differ, the reported gains cannot be credited to the gating module alone.","fun_headline_variants_meta":{"raw":{"variants":["Dynamic gating fuses audio and vision for cleaner separation","Adaptive gating blends audio and visual cues for better separation","Audio-visual separation improved by dynamic gating fusion","Gating module adaptively reweights audio and visual features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00099,"raw_usage":{"total_tokens":4194,"prompt_tokens":939,"completion_tokens":3255,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":3188}},"tokens_in":555,"tokens_out":3255,"duration_ms":20752,"temperature":1.0,"reasoning_tokens":3188,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:04:19.980026+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce iQuery with the identical training recipe (same STFT, object detector, optimizer, and test split) and compare SDR. If the faithful baseline reaches 10.95 dB—the value listed in the ablation table—rather than the 10.63 dB in the main table, DGFNet's advantage shrinks from 0.62 to about 0.30 dB, and the claimed margin depends on an unexplained setup difference rather than the gating module alone.","supporting_citations":[],"review_version":1}