{"id":"2e04cf4f-46e9-45fe-9a1a-9fe987db8913","arxiv_id":"2505.11889","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"On DCASE 2023 Task 4, CRNN+BEATs remains the strongest single-model setup, BEATs+WavLM fusion improves event-F1 slightly, and the new nSEBBs post-processing matches cSEBBs accuracy at about 6 percent of its runtime.","lead":"This paper benchmarks five self-supervised audio models for sound event detection and tests ways to combine them in a CRNN classifier. It also introduces a faster adaptive post-processing method that adjusts event boundaries, a practical concern for deploying sound event detection on limited hardware.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No seed-level variance is reported despite §4.3's three-seed repetition, so the small PSDS1/event-F1 gains and the 'comparable' nSEBBs accuracy are not established.","rationale":"Reader's CONDITIONAL verdict is appropriate. I examined the mapping-generalization concern first, since Eq. (3) and Tables 1–2 are hand-specified; that remains a secondary worry. However, the most load-bearing issue is more basic: the experiments were repeated three times per §4.3 yet no variance appears anywhere in the manuscript. The decisive comparisons are small in absolute terms, and in SED three-seed variation can be comparable to the reported deltas. The paper's own nSEBBs-vs-cSEBBs numbers (e.g., 0.499 vs 0.500 in Table 8) illustrate how close the comparisons are. This is not an accusation of any kind; it is a missing piece of evidence that the authors can supply from the runs they already performed. A revised version reporting mean±std for the key rows, plus the mapping sensitivity check, would make the claims verifiable. Therefore the reader's conditional verdict should stand.","tokens_in":18008,"tokens_out":18061,"duration_ms":171648,"concrete_test":"Obtain from the authors the per-seed values behind Table 5 (CRNN+BEATs and CRNN+BEATs+WavLMbase concat, event-F1) and Table 7 (all standalone SSL models under baseline, cSEBBs, nSEBBs, PSDS1), then compute mean±std over the three seeds promised in §4.3. If the nSEBBs-vs-cSEBBs differences or the 2.9-point event-F1 gain are within one standard deviation, the headline claims are unsupported; if they exceed roughly two standard deviations, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing issue is the absence of seed-level uncertainty in the tables that carry the paper's two headline results. Section 4.3 states that every experiment was repeated with three different random seeds, but all of Tables 3–8 report single numbers. The fusion claim rests on CRNN+BEATs+WavLMbase (concat) improving event-F1 from 55.7% to 58.6% (Table 5), and the post-processing claim rests, for example, on CRNN+HuBERTbase PSDS1 moving from 0.352 to 0.393 under nSEBBs (Table 7). These are small effects in a metric where three-seed variation in SED can easily reach several PSDS1 points. The same tables report nSEBBs-vs-cSEBBs differences as small as 0.001–0.016 PSDS1 (Tables 7 and 8), so the 'comparable accuracy' claim depends on distinguishing a 0.001 difference from noise. Without per-seed means and standard deviations, the reader cannot determine whether the reported improvements are signal or seed noise. This is more immediate than the mapping-generalization concern: the effect sizes need variance support even on the DCASE validation set.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates five self-supervised learning (SSL) models (BEATs, HuBERT, wav2vec 2.0, WavLM, Dasheng) as feature extractors for sound event detection (SED) on the DCASE 2023 Task 4 development set, using a CRNN+BEATs baseline. It proposes three SSL fusion strategies: individual SSL embedding integration, dual-modal fusion (BEATs plus one other model), and full aggregation of all SSL embeddings. It further introduces nSEBBs, an adaptive post-processing method that replaces cSEBBs' grid-searched thresholds with a hand-specified mapping from per-class statistics (posterior contrast ratio and average event duration) to the SEBBs parameters. The main reported results are that dual-modal fusion with WavLM yields the highest event-F1 (58.6% vs. 55.7% for CRNN+BEATs, Table 5), and nSEBBs achieves PSDS1 comparable to cSEBBs at roughly 1/14 the runtime (8 s vs. 130 s, §5.4), with the largest PSDS1 gain over the no-post-processing baseline being about 0.04 (e.g., CRNN+HuBERTbase from 0.352 to 0.393, Table 7).","tokens_in":18328,"tokens_out":5992,"duration_ms":56134,"significance":"If the results hold, the paper offers a useful systematic benchmark of SSL representations for SED and a computationally cheaper post-processing alternative that retains most of the benefit of cSEBBs. The per-class analysis in Table 6 and the explicit algorithmic descriptions are strengths. The paper is honest about the limited gains from fusion and the degradation from full aggregation. However, the empirical claims rest on single-point estimates from three-seed runs, and the adaptive mapping is presented without sensitivity or transfer analysis, so the generality of the speed-accuracy trade-off is not yet established. With added uncertainty quantification and mapping validation, this could be a solid contribution to the SED community.","major_comments":[{"comment":"The paper states in §4.3 that 'Each experiment was repeated three times with different random seeds to ensure reproducibility,' yet all tables report a single number per configuration. This is load-bearing for both headline claims: the fusion claim in Table 5 (CRNN+BEATs+WavLMbase concat event-F1 55.7→58.6) and the post-processing claim in Table 7 (CRNN+HuBERTbase PSDS1 0.352→0.393) involve small differences that could easily be within seed noise in SED. The 'nSEBBs is comparable to cSEBBs' assertion is especially fragile, as Table 7 reports differences as small as 0.001–0.016 PSDS1 (e.g., 0.393 vs. 0.390 for HuBERT; 0.504 vs. 0.520 for CRNN+BEATs). Without per-seed values, means, and standard deviations (or significance tests), the central quantitative claims are not established. Please report the three-seed statistics for at least Tables 4, 5, 7, and 8, and discuss whether the observed differences are consistent across seeds.","section":"§4.3, Tables 3–8"},{"comment":"The adaptive parameter mapping is presented as a principled replacement for cSEBBs' grid search, but the constants in Eq. (3) and Tables 1–2 (θ_base ∈ {2.0, 2.4, 2.8, 3.2}, lstep ∈ {0.384, 0.512, 0.640, 0.800}, θ_abs ∈ {0.12, 0.18, 0.24, 0.30}, and the piecewise slopes 0.1/0.3) are introduced without any sensitivity analysis, ablation, or cross-dataset check. Since the runtime advantage of nSEBBs (8 s vs. 130 s, §5.4) comes precisely from replacing the search with this fixed mapping, the mapping's generality is load-bearing for the paper's central efficiency claim. If these constants were chosen by inspecting the DCASE 2023 validation set, the reported nSEBBs numbers are not independent predictions. Please provide (a) a sensitivity analysis showing how PSDS1 varies with each constant, (b) the distribution of (PCR, avgED) over the validation set to justify the bin boundaries, and (c) ideally, a cross-dataset evaluation (e.g., DCASE 2024 Task 4) to test transfer.","section":"§3.3, Eq. (3), Tables 1–2"},{"comment":"The claim that nSEBBs 'improving PSDS1 by up to 4%' is ambiguous because the comparison baseline is not stated. In Table 7, the improvements over the no-post-processing baseline are up to 0.041 points (CRNN+HuBERTbase: 0.352→0.393), which is about 4 percentage points, but cSEBBs achieves 0.390 on the same row. For CRNN+BEATs, nSEBBs (0.504) is actually worse than cSEBBs (0.520). The abstract and conclusion should clarify that the improvement is over the raw baseline, not over cSEBBs, and should quantify the nSEBBs-vs-cSEBBs gap (which is often within 0.01–0.02 PSDS1, tying into the missing uncertainty estimates in the first major comment).","section":"Abstract, §5.4, Table 7"}],"minor_comments":[{"comment":"The variable names are inconsistent: Algorithm 2 returns 'PCR' as the first statistic, but Eq. (3) uses 'pcrc'; the text also refers to 'PCR (Percentage of Coverage Rate)' in one place while earlier defining it as 'Posterior Contrast Ratio'. Please unify terminology and define the acronym at first use.","section":"§3.3, Algorithm 2 / Eq. (3)"},{"comment":"Table 6 lists 'CRNN+BEATs+WavLMbase' without specifying whether the additive or concatenative fusion variant is used, even though Table 5 shows different results for the two fusion modes. Please state the fusion method used for the class-wise results.","section":"Table 6"},{"comment":"The 'cSEBBsD' variant is said to 'result from the concurrent application of both thresholds,' but no description is given of how θ_abs and θ_rel interact or how the cSEBBsD thresholds were chosen. A brief algorithmic note or a reference to the same mapping as nSEBBsD would clarify the comparison.","section":"§5.4, Table 7"},{"comment":"The definition of 'nl← Percentile(sc[t], 10)' is unclear: is this the 10th percentile of the score values for that class? If so, for scores that are mostly near zero (as is typical for sigmoid outputs), this quantity can be extremely small, making the PCR unstable. Please state the percentile computation and discuss numerical stability.","section":"§3.3, Algorithm 2"},{"comment":"The paper contains several grammatical and typographical errors (e.g., 'all these performance difference may due to the differ in their model structures', 'The system employs three principal operational parameters: 1) ... 2) ...', and the inconsistent use of 'nSEBBsD' with a subscript). A careful language edit is needed.","section":"General"},{"comment":"The second bullet claims 'a novel normalization-based post-processing method,' but the algorithm does not explicitly normalize the scores; it computes mean and variance and applies a mask. Please clarify what 'normalized' refers to (e.g., the use of statistical moments to derive parameters) to avoid overclaiming.","section":"§1, Contribution list"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and addresses a relevant topic. The main issues are the missing seed-level uncertainty and the unvalidated hand-specified adaptive mapping, both of which are fixable with additional experiments and analysis. I also note that the code is only available 'upon reasonable request,' which limits reproducibility; a public release would strengthen the paper. The novelty is incremental (a modified SEBBs post-processor and a fusion comparison), but the benchmark value is real if the statistical support is added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, narrow paper. It benchmarks five SSL front-ends (BEATs, HuBERT, wav2vec 2.0, WavLM, Dasheng) on DCASE 2023 Task 4 SED, tests add vs concat fusion, and proposes nSEBBs, an adaptive variant of Ebbers et al.'s cSEBBs post-processor. The benchmark tables are clearly organized and will be referenced by people picking an SSL encoder for SED. The runtime claim is the most interesting thing: nSEBBs takes 8 s versus 130 s for grid-searched cSEBBs, roughly a 15x speed-up, with roughly equal PSDS1.\n\nThe fusion results are honestly reported: most dual-model fusions don't beat the CRNN+BEATs baseline on PSDS1, and full fusion degrades everything. The one clear win is event-F1 for BEATs+WavLMbase concat (58.6% vs 55.7% baseline). Good that they include a per-class table and discussion of why speech-pretrained embeddings help only certain classes.\n\nMain problems:\n\nFirst, no seed-level variance anywhere, even though Section 4.3 says each experiment ran three seeds. Tables 7 and 8 show differences of 0.001-0.016 PSDS1 between nSEBBs and cSEBBs. Those are within the noise a single seed can produce. You can't claim 'comparable' or 'slightly better' without the standard deviation. This is the load-bearing gap.\n\nSecond, the abstract says 'improving PSDS1 by up to 4%' - that's true only when comparing nSEBBs to the raw baseline (e.g., HuBERTbase 0.352 to 0.393), not to cSEBBs (0.390). And the conclusion says nSEBBs 'significantly outperforms traditional static thresholding approaches' - if that means median filtering, fine, but the comparison in the paper is to cSEBBs, where nSEBBs is at best a wash. That sentence overstates the evidence.\n\nThird, the adaptive mapping in Eq. 3 and Tables 1-2 looks hand-picked. No sensitivity analysis, no ablation showing those specific breakpoints matter, no second dataset. It could transfer poorly. This is a moderate concern, not fatal, but it should be flagged in revision.\n\nAlso, code is 'available from the first author upon reasonable request,' which usually means not available. For a benchmark, public code would help.\n\nBottom line: I'd recommend peer review - the paper deserves a referee - but I'd expect a revision that reports per-seed means and standard deviations, rephrases the accuracy claims, and ideally releases code. If the authors can't show the variance, the nSEBBs accuracy story should be downgraded to 'efficiency-equivalent.'\n\nFor a reading group, it's a maybe - good if you're working on SED/DCASE style tasks, less relevant otherwise. I'd cite it as a reference for SSL selection in SED, but only alongside a caveat about the variance issue. Serious thinker: yes; the overstatements look like enthusiasm, not deception.","headline":"A useful SED benchmark with an adaptive post-processing trick; the accuracy edge over cSEBBs is small and statistically unsupported.","tokens_in":18832,"tokens_out":4865,"would_cite":true,"duration_ms":42911,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On DCASE 2023, one SSL pairing lifts F1 and an adaptive post-processor matches grid search at 1/14 the runtime.","keywords":["sound event detection","self-supervised learning","BEATs","HuBERT","WavLM","SEBBs","nSEBBs","PSDS"],"falsifier":"Run the same CRNN+BEATs system with nSEBBs on a different strongly labeled SED benchmark, such as DCASE 2024 Task 4 or a held-out split of DESED, without retuning the PCR/duration-to-parameter mapping; if nSEBBs no longer matches or beats cSEBBs in PSDS1, the adaptive mapping is overfit to the 2023 validation set.","tokens_in":17824,"feed_emoji":"🎧","tokens_out":7728,"duration_ms":64831,"temperature":0.7,"pith_summary":"The paper tries to settle two practical questions in sound event detection: which self-supervised models are worth using, and how to post-process their outputs without expensive parameter searches. It reports that BEATs is the best single SSL feature extractor for a CRNN detector, and that fusing BEATs with WavLM embeddings gives a modest but consistent boost to event-level F1, while fusing all five SSL models hurts every metric. Its new post-processor, nSEBBs, replaces grid search with per-class statistics and matches the grid-searched baseline on PSDS1 while cutting runtime from about 130 s to 8 s. If these results hold, a practitioner can pick one strong SSL backbone, consider a single complementary pairing, and skip the parameter search.","feed_headline":"Adaptive post-processing keeps SED accuracy at 1/14 the runtime","feed_subtitle":"On DCASE 2023 Task 4, nSEBBs matches cSEBBs' PSDS1 while cutting post-processing cost from 130 s to 8 s.","key_machinery":"The load-bearing mechanism is nSEBBs, a normalized variant of the sound event bounding boxes post-processor. For each event class it derives two statistics from the frame-level scores: the posterior contrast ratio (PCR), a log-ratio of high-confidence to low-confidence posterior mass, and the average event duration (avgED), read off a high-score mask. A fixed mapping table converts those statistics into the step-filter length and the relative merging threshold, replacing the three-parameter grid search of cSEBBs. The underlying boundary detection still uses cSEBBs' delta-score change points and segment merging. The fusion results rest on a simpler mechanism: frame-level SSL embeddings from different encoders are temporally aligned and combined by element-wise addition or concatenation before the CRNN classifier.","core_discovery":"On the DCASE 2023 Task 4 validation set, the paper establishes three claims. First, among individual SSL embeddings, CRNN+BEATs is clearly the best, reaching PSDS1 0.497 and event-F1 55.7%, while speech-pretrained models (wav2vec2.0, HuBERT, WavLM) stay near the MT-CRNN baseline. Second, dual-modal fusion helps only in specific pairings: concatenating BEATs with WavLMbase raises event-F1 to 58.6% and inter-F1 to 80.2%, but every dual or full fusion keeps PSDS1 below the single-BEATs baseline, and full fusion drops to PSDS1 0.448. Third, the adaptive nSEBBs post-processor matches or slightly exceeds the grid-searched cSEBBs on PSDS1 for standalone SSL models (e.g., HuBERTbase 0.393 vs 0.390) and for dual fusion (WavLMbase 0.500 vs 0.499), while running in about 8 s instead of 130 s.","pith_inferences":["The authors' mapping from PCR and avgED to parameters is presented without an ablation; a natural check is to learn that mapping from the validation set rather than fixing it, and to test on a later DCASE task to see whether the constants transfer.","The WavLM complementarity suggests that pretraining-domain diversity (speech vs audio) is the useful axis rather than audio-domain pretraining per se; fusing BEATs with another non-speech model trained on a distinct corpus would test whether the F1 gain is due to WavLM specifically.","Because nSEBBs needs only per-class score statistics, the same recipe could be lifted to other temporal detection tasks with variable event durations, such as anomaly or bioacoustic event detection; the paper does not test this.","Full-fusion collapse hints that element-wise addition of many SSL streams saturates or misaligns; an attention-weighted or learned combination might avoid the drop, but that is an extrapolation from the paper's evidence."],"forward_implications":["CRNN+BEATs is a strong default: no tested SSL addition improves its PSDS1, so for boundary-focused evaluation the single backbone is enough.","A practitioner who wants higher event-level F1 should concatenate BEATs with WavLMbase, which raises event-F1 from 55.7% to 58.6% and inter-F1 from 79.0% to 80.2%.","nSEBBs removes the cost of grid search: per-class PCR and avgED pick the same quality parameters in about 8 s versus 130 s, so post-processing can run on-device.","Dual-threshold versions (nSEBBsD and cSEBBsD) consistently lose PSDS1 to single-threshold versions, so adding an absolute threshold is not a free improvement.","Aggregating all five SSL representations is counterproductive, dropping PSDS1 to 0.448 and event-F1 to 53.7%, which argues for selective fusion rather than model soup."],"supporting_citations":[{"why":"Introduces sound event bounding boxes (SEBBs), the post-processor that nSEBBs adapts and compares against.","marker":"(Ebbers et al. 2024)"},{"why":"Supplies the BEATs and WavLM pre-trained encoders used as the paper's feature extractors and fusion components.","marker":"(Chen et al. 2022)"},{"why":"Provides HuBERT, one of the standalone SSL models benchmarked and one where nSEBBs exceeds cSEBBs.","marker":"(Hsu et al. 2021)"},{"why":"Provides wav2vec 2.0, another standalone SSL encoder in the comparison.","marker":"(Baevski et al. 2020)"},{"why":"Provides Dasheng, the general-audio SSL encoder whose fusion results are part of the comparison.","marker":"(Dinkel et al. 2024)"},{"why":"Defines the DESED dataset and DCASE Task 4 setup on which all experiments are evaluated.","marker":"(Turpault et al. 2019)"},{"why":"Defines the PSDS1/PSDS2 metrics that carry the paper's main quantitative claims.","marker":"(Bilen et al. 2020)"}],"fun_headline_variants":["SSL fusion done right: BEATs+WavLM boosts event-F1","nSEBBs: 16x faster post-processing, same SED accuracy","For SED, BEATs beats all; BEATs+WavLM for dual fusion","Adaptive post-processing cuts SED runtime 16x","Complementary SSL fusion: only some pairs win"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The hand-built tables converting each class's confidence and duration statistics into post-processing parameters were tuned on the DCASE 2023 validation set and are assumed to transfer to other recordings and datasets.","fun_headline_variants_meta":{"raw":{"variants":["SSL fusion done right: BEATs+WavLM boosts event-F1","nSEBBs: 16x faster post-processing, same SED accuracy","For SED, BEATs beats all; BEATs+WavLM for dual fusion","Adaptive post-processing cuts SED runtime 16x","Complementary SSL fusion: only some pairs win"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1528,"prompt_tokens":983,"completion_tokens":545,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":447}},"tokens_in":599,"tokens_out":545,"duration_ms":5221,"temperature":1.0,"reasoning_tokens":447,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:45:22.945680+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same CRNN+BEATs system with nSEBBs on a different strongly labeled SED benchmark, such as DCASE 2024 Task 4 or a held-out split of DESED, without retuning the PCR/duration-to-parameter mapping; if nSEBBs no longer matches or beats cSEBBs in PSDS1, the adaptive mapping is overfit to the 2023 validation set.","supporting_citations":[{"cited_title":", Germain , F.G","cited_arxiv_id":null,"evidence_quote":"Introduces sound event bounding boxes (SEBBs), the post-processor that nSEBBs adapts and compares against."},{"cited_title":", Zhou , Y","cited_arxiv_id":null,"evidence_quote":"Provides wav2vec 2.0, another standalone SSL encoder in the comparison."},{"cited_title":", Yan , Z","cited_arxiv_id":null,"evidence_quote":"Provides Dasheng, the general-audio SSL encoder whose fusion results are part of the comparison."},{"cited_title":", Serizel , R","cited_arxiv_id":null,"evidence_quote":"Defines the DESED dataset and DCASE Task 4 setup on which all experiments are evaluated."}],"review_version":1}