{"id":"0da7ad64-1d64-47df-9d5a-5e924b9c6e03","arxiv_id":"2507.19062","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"OmniGSE combines a continuous feature enhancement stage with a RootLM/BranchLM token generation stage to improve general speech enhancement on mixed distortions.","lead":"This paper presents OmniGSE, a two-stage speech enhancement system that first cleans continuous audio features with a lightweight transformer, then uses a hierarchical language model to regenerate high-quality discrete audio tokens. It reports improved quality over prior systems on benchmarks covering noise, reverberation, bandwidth limits, clipping, and packet loss, especially when several distortions occur together.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Private fine-tuning data confounds the benchmark comparisons; the claimed general superiority over prior models is not established without a controlled retraining of baselines on the same data.","rationale":"The reader's weakest assumption—evaluation fairness—is also the most load-bearing condition for the paper's central claim. The abstract and conclusion assert superiority, but the comparison is not controlled: OmniGSE is fine-tuned on a private high-fidelity dataset that baselines did not receive, and the training distortion mix is custom. The paper's ablations (Table 5) show both stages help, which supports internal consistency, and the two-stage idea is plausible. However, internal ablations do not establish external superiority. The reported wins are often small and sometimes reverse across metrics (e.g., Table 2 SBS/SIM), and no significance testing is provided. A controlled retraining check would settle whether the framework or the data is responsible. I therefore keep the reader's CONDITIONAL verdict; the paper should not be rejected, but the 'surpasses' claim should be treated as unverified until the confound is removed.","tokens_in":14334,"tokens_out":6342,"duration_ms":62081,"concrete_test":"Retrain (or fine-tune) the strongest open-source baselines, e.g., AnyEnhance and MaskSR, on the same private high-fidelity corpus with the exact distortion simulation of Section 3.1/4.1, using comparable compute, then rerun Tables 1–4. If the performance gap narrows to within metric noise (e.g., OVRL differences of 0.001–0.01), the claimed architectural superiority is not established; if the gap persists, the concern is resolved in the authors' favor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—'OmniGSE surpasses existing models across multiple benchmarks'—requires that the reported gains be attributable to the proposed two-stage architecture rather than to an unfair training advantage. Section 4.1 states that the authors 'fine-tuned our model on a private high-fidelity speech dataset.' The discriminative and generative baselines in Tables 1–4 (AnyEnhance, MaskSR, LLaSE-G1, etc.) were not given this data, nor is there any indication they were trained with the same distortion pipeline of Section 3.1 (noise 100%, reverb 50%, other distortions equally likely). Because the test sets are public and no confidence intervals are reported, the numerical advantages in Tables 1–3—some as small as 0.001 in OVRL (Table 2, SR) or below the baseline on other metrics (e.g., SBS and SIM in Table 2)—cannot be separated from data or simulation-recipe effects. The disclosed private fine-tuning is the least controlled variable in the comparison. If the private data or the hand-tuned distortion mix drives the gap, the 'general' claim and the 'particularly compound distortions' claim in the abstract do not follow from the experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OmniGSE, a two-stage general speech enhancement framework that combines a continuous feature enhancement stage (channel-split NAC-RoFormer) with a discrete token generation stage based on a hierarchical language model (RootLM plus BranchLMs). The authors evaluate on DNS2020 denoising/dereverberation, Voicefixer SR/GSR, and Interspeech 2022 PLC benchmarks, and they report ablations of the two stages, the NAC-RoFormer, the hierarchical LM, encoder fine-tuning, and teacher forcing. The central claim is that this cross-domain collaborative design surpasses existing single-paradigm discriminative and generative methods, with particular strength in compound distortion scenarios.","tokens_in":14505,"tokens_out":5227,"duration_ms":52820,"significance":"The architectural idea is coherent: a discriminative pre-enhancement stage feeds high-SNR continuous features to a generative LM-based token predictor, and the hierarchical RootLM/BranchLM design explicitly models RVQ inter-codebook dependencies. The ablations in Table 5 provide useful evidence that each component contributes to the final performance. The breadth of the benchmark coverage (denoising, dereverberation, super-resolution, restoration, packet loss) is a strength. However, the central comparative claim is currently not fully supported because the evaluation uses a private fine-tuning dataset unavailable to the baselines, reports only point estimates without confidence intervals, and the listening tests omit the most relevant generative baselines. If these evaluation gaps are addressed, the framework would be a solid contribution to general speech enhancement.","major_comments":[{"comment":"The comparison is confounded by the private fine-tuning data. Section 4.1 states that the authors 'fine-tuned our model on a private high-fidelity speech dataset,' but the baselines in Tables 1-4 were not given this data. Additionally, the training distortion recipe in Sec. 3.1 (noise 100%, reverb 50%, other distortions equally likely) may be aligned with the test-set conditions, and the baselines were not trained with this exact recipe. Therefore the reported gains cannot be attributed solely to the proposed architecture. I request a controlled experiment: (i) train OmniGSE without the private data; (ii) fine-tune at least one representative generative baseline (e.g., MaskSR or AnyEnhance) on the same private data and distortion pipeline; and (iii) report the performance difference to isolate the contribution of the private data.","section":"Sec. 4.1, Tables 1-4"},{"comment":"No confidence intervals or significance tests are reported, and the claimed superiority is not consistent across all metrics. For example, in Table 2 OmniGSEfb has lower SBS (0.930 vs 0.941) and SIM (0.935 vs 0.943) than AnyEnhance, and in Table 3 OmniGSEfb has lower NISQA (4.293 vs 4.335) than MaskSR. Some OVRL differences in Table 1 are as small as 0.026 (3.444 vs 3.418). Without variance estimates or statistical tests, the statement in the abstract that OmniGSE 'surpasses existing models across multiple benchmarks' is overstated, and the specific claim of excelling in compound distortions is not uniformly supported by the GSR results.","section":"Tables 1-3"},{"comment":"The subjective listening tests compare OmniGSE only against discriminative baselines (FullSubNet, VoiceFixer, TF-GridNet) and omit the strongest generative competitors (MaskSR, AnyEnhance, LLaSE-G1) that are the direct point of comparison for the generative stage. Figure 3 and Figure 4 also lack details on the number of listeners, the number of stimuli, and any statistical analysis. Please extend the listening test to include at least the strongest generative baselines and report listener counts and significance testing.","section":"Figures 3-4"}],"minor_comments":[{"comment":"The word 'knowlege' should be 'knowledge' in the sentence about discrete codebooks encapsulating rich prior knowledge.","section":"Sec. 2.2"},{"comment":"The description of the private high-fidelity dataset is too vague; please report its size, duration, and whether it was used only for fine-tuning after the main training or as part of the main training mixture, since this materially affects the comparison.","section":"Sec. 4.1"},{"comment":"The full-band and wideband models contain roughly 0.97B and 1.23B parameters, respectively, yet Sec. 2.2 claims reduced computational cost. Please report inference time (e.g., RTF) and memory usage to substantiate the efficiency claim.","section":"Sec. 4.2"},{"comment":"The x-axis label 'SNR' is ambiguous; please define whether this is signal-domain SNR or a feature-space measure, and additionally report the final objective scores for each conditioning feature rather than only the feature SNR.","section":"Figure 5"},{"comment":"Some baseline entries have missing values (e.g., VoiceFixer SBS and SIM, SELM and GenSE NISQA/SBS/SIM); please indicate whether these are not reported or not applicable, and consider supplementing the missing metrics for a complete comparison.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The private fine-tuning data is the most serious issue: it creates an uncontrolled advantage for the proposed method over all baselines, which is especially problematic for a systems paper claiming general superiority. The authors should be asked to either release the private data (unlikely) or perform controlled ablation experiments that isolate its effect. The paper is otherwise within scope for an ACM MM audience, though the speech-enhancement focus is more traditional signal processing than multimedia; the evaluation breadth helps justify fit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look. The new bit is the two-stage design: a lightweight channel-split NAC-RoFormer cleans the continuous features from a neural audio codec, then a hierarchical LM (shared RootLM plus per-codebook BranchLMs) generates discrete tokens. That specific combination is not in the cited prior work. GenSE does two-stage semantic-to-acoustic generation without a continuous pre-enhancement stage; Li et al. does feature enhancement without the hierarchical LM. The architecture is plausible and the ablations are informative: removing either stage hurts on the expected tasks, and replacing the hierarchical LM with a single shared LM makes things worse than having no second stage at all. That is real evidence the design choices matter.\n\nThe soft spots are all in the evaluation. Fine-tuning on a private high-fidelity dataset, while the baselines were not given that data, is a genuine confound. The paper reports no confidence intervals or significance tests, so a 0.001 OVRL gain on the SR test is not distinguishable from noise. On several metrics OmniGSE is actually below AnyEnhance or MaskSR (e.g., SBS and SIM in Table 2, NISQA in Table 3). The subjective tests compare only against FullSubNet, VoiceFixer, and TF-GridNet, leaving out the strongest generative baselines, which is a strange omission when the selling point is generative restoration. No code or audio samples either.\n\nNone of this is fatal. The architecture is sound, the ablation study is commendably thorough, and the paper makes an honest attempt to cover denoising, dereverberation, super-resolution, restoration, and packet loss. The overclaim is in the abstract's \"surpasses,\" but with better controls the two-stage idea could hold up. The private-data issue is fixable in principle: retrain at least the strongest baseline on the same data, or report gains with and without the private fine-tuning.\n\nThe right audience is researchers working on codec-based generative enhancement. This paper deserves peer review, but with a request for either controlled baseline retraining or significance testing, plus release of code and the fine-tuning protocol. I would not cite it in its current form, but I would read a revised version.","headline":"Two-stage continuous-then-discrete GSE with a RootLM/BranchLM codec hierarchy is a real architectural contribution, but private fine-tuning data and missing significance tests keep the 'surpasses' claim from being proven.","tokens_in":15079,"tokens_out":1484,"would_cite":false,"duration_ms":17413,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage speech enhancer that first cleans continuous features, then generates discrete tokens, claims to beat single-paradigm models.","keywords":["general speech enhancement","hierarchical language models","neural audio codec","residual vector quantization","compound distortions","speech restoration","packet loss concealment","cross-domain collaborative optimization"],"falsifier":"Train OmniGSE on public data only and evaluate on a held-out compound-distortion set whose distortion recipe differs from the training recipe; if its advantage over single-paradigm baselines disappears, the central claim of general superiority is not supported.","tokens_in":14119,"feed_emoji":"🎙️","tokens_out":6961,"duration_ms":62746,"temperature":0.7,"pith_summary":"The paper introduces OmniGSE, a general speech enhancement system aimed at speech degraded by several distortions at once: background noise, reverberation, limited bandwidth, clipping, and packet loss. Its central claim is that no single paradigm is enough, because discriminative models are precise on regression tasks like denoising while generative models are flexible on reconstruction tasks like declipping, so the two should be combined. OmniGSE does this in two stages: it first enhances the continuous features produced by a neural audio codec encoder, then uses a hierarchical language model to generate the codec's discrete tokens. The paper reports that this architecture beats both discriminative and generative baselines across multiple benchmark test sets, with the largest advantage on compound distortions. A sympathetic reader would care because real-world recordings usually suffer several degradation types together, and most existing methods are built for only one.","feed_headline":"Two-stage model bests single-paradigm speech enhancers","feed_subtitle":"It first cleans continuous speech features, then lets a hierarchical token model restore what is missing.","key_machinery":"The load-bearing object is the hierarchical language model built on the codec's residual vector quantization: a RootLM models shared acoustic content, while per-level BranchLMs predict each codebook level conditioned on the RootLM and on the previous level, which is what lets the system regenerate missing spectral or temporal content while keeping acoustic consistency. The supporting mechanism is the channel-split NAC-RoFormer, which lowers the computational cost of attending to 1024-dimensional codec features by grouping channels and alternating temporal and cross-group attention.","core_discovery":"Stage I uses a channel-split network with dual-path rotary-position attention to map the codec encoder's features for distorted speech toward the clean-speech features produced by a teacher codec encoder, and the codec encoder itself is fine-tuned on distorted input at the same time. Stage II takes those enhanced pre-quantized features as conditioning and autoregressively predicts the residual vector quantization tokens of clean speech. The hierarchical language model consists of one RootLM, which captures acoustic features shared across codebook levels, and a separate BranchLM per level, which captures the progressive relationship from one codebook level to the next; the levels are trained with teacher forcing under a cross-entropy loss. The discovery, stated on the paper's terms, is that this continuous-to-discrete collaboration resolves the precision-versus-flexibility tradeoff, and that the hierarchical LM reduces inter-level prediction conflicts that a single shared LM would introduce.","pith_inferences":["Editorial inference: the RootLM/BranchLM split is a general recipe for any residual-vector-quantization codec, so it could transfer to other neural codecs and to low-bitrate speech synthesis.","Editorial inference: the architecture implies that discriminative feature cleaning can act as a universal conditioning front-end for generative audio models, not just for speech enhancement.","Editorial inference: the paper does not report how sensitive the results are to the hand-chosen distortion probabilities; varying them per deployment is a direct test of the framework's real-world generality."],"forward_implications":["A single OmniGSE model could replace separate denoising, dereverberation, bandwidth extension, declipping, and packet-loss concealment modules.","Because the first stage improves the conditioning features, the second stage can focus on regenerating missing content instead of suppressing noise, so the two stages compound.","The paper's ablation says using separate BranchLMs avoids the pattern conflicts that a single multi-level LM produces.","The framework reports its restoration quality without extra self-supervised semantic features, which lowers conditioning cost."],"supporting_citations":[{"why":"Supplies the pre-trained neural audio codec whose encoder, discrete codebooks, and decoder the whole two-stage pipeline is built around.","marker":"[15]"},{"why":"Introduces the idea of enhancing pre-quantized neural-codec embeddings, which the first stage extends.","marker":"[16]"},{"why":"Provides the full-band restoration test sets and one of the unified baselines the paper must beat.","marker":"[21]"},{"why":"Defines the masked-language-model restoration approach that is the strongest generative comparison for full-band speech.","marker":"[18]"},{"why":"Is the two-stage language-model speech enhancement baseline the paper's architecture is positioned against.","marker":"[40]"},{"why":"Is a prompt-guided unified generative enhancement baseline evaluated on the same restoration benchmarks.","marker":"[46]"},{"why":"Supplies the rotary position embedding used in the channel-split dual-path attention of the first stage.","marker":"[30]"},{"why":"An alternative hierarchical codec-language-model structure used in the ablation to justify separate BranchLMs.","marker":"[4]"}],"fun_headline_variants":["Continuous-to-discrete collaboration tames compound distortions in speech","Hierarchical language models beat single-paradigm speech enhancers","OmniGSE: two-stage pipeline for robust multi-distortion speech","From continuous to discrete: hierarchical LMs handle compound noise","Clean features first, then hierarchical token generation for speech"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim assumes the performance gains come from the two-stage architecture itself, not from the private high-fidelity fine-tuning data or from the hand-chosen distortion mix aligning with the test sets.","fun_headline_variants_meta":{"raw":{"variants":["Continuous-to-discrete collaboration tames compound distortions in speech","Hierarchical language models beat single-paradigm speech enhancers","OmniGSE: two-stage pipeline for robust multi-distortion speech","From continuous to discrete: hierarchical LMs handle compound noise","Clean features first, then hierarchical token generation for speech"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00059,"raw_usage":{"total_tokens":2762,"prompt_tokens":935,"completion_tokens":1827,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":1743}},"tokens_in":551,"tokens_out":1827,"duration_ms":17863,"temperature":1.0,"reasoning_tokens":1743,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:01:00.759485+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train OmniGSE on public data only and evaluate on a held-out compound-distortion set whose distortion recipe differs from the training recipe; if its advantage over single-paradigm baselines disappears, the central claim of general superiority is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained neural audio codec whose encoder, discrete codebooks, and decoder the whole two-stage pipeline is built around."},{"cited_title":"Speech Enhancement Using Continuous Embeddings of Neural Audio Codec","cited_arxiv_id":"2502.16240","evidence_quote":"Introduces the idea of enhancing pre-quantized neural-codec embeddings, which the first stage extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the full-band restoration test sets and one of the unified baselines the paper must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"An alternative hierarchical codec-language-model structure used in the ablation to justify separate BranchLMs."}],"review_version":2}