{"id":"1728a52c-be6b-4fe4-87c4-12c31aaf2bbd","arxiv_id":"2608.07781","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An inference-time module uses ASR alignments and per-segment noisy/enhanced mixing to reduce over-suppression in speech enhancement outputs.","lead":"This paper describes a plug-in that fixes over-suppression in speech enhancement: it uses speech recognition timestamps to cut audio into word-sized pieces, then mixes noisy and enhanced audio per piece to maximize quality scores. Why read: it promises better intelligibility and downstream ASR performance without retraining existing enhancers.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 2's robustness term R is unspecified: 'Whisper embedding L2 similarity' lacks a reference; clean-reference use would break the inference-time claim.","rationale":"The reader's weakest_assumption (convex hull) is a legitimate theoretical boundary, but it is not the most load-bearing for the central claim. The method's fallback criterion (§2.3) explicitly compares against noisy and enhanced, so even if the convex hull cannot represent the clean signal exactly, the optimizer can still choose a better endpoint; the empirical gains suggest this is often sufficient. A more direct threat is the undefined robustness term in Eq. 2. The paper repeatedly emphasizes inference-time operation without clean references, yet it never specifies what the 'Whisper embedding L2 similarity' is computed against. Given the evaluation metrics include clean-reference distances (§3.2), a reader cannot rule out clean-reference leakage in the objective. This ambiguity also affects the interpretation of the scores: if R uses the noisy input as reference, the method is essentially a soft revert to noisy, which would explain STOI gains but not necessarily PESQ gains. I also note the identical metric values across different datasets in Table 1 (e.g., VCTK DEMAND 'enh' and 'ours' columns match URGENT 2024 exactly), which strongly suggests a table compilation error; the reader already flagged this, and it further undermines confidence in the numeric claims. Neither issue changes the appropriate verdict from CONDITIONAL: the paper needs clarification of R and corrected tables before acceptance.","tokens_in":8687,"tokens_out":11193,"duration_ms":99783,"concrete_test":"Obtain the authors' code or ask for the exact definition of R. On a single dataset (e.g., URGENT 2024), rerun the λ optimization twice: once with R as the negative L2 distance between Whisper embeddings of s_corr and the clean reference, and once with R as the negative L2 distance between s_corr and the noisy input (or another reference-free proxy). Compare PESQ/STOI and SCOREQ. If the reported gains in Table 1 reproduce only with the clean-reference R, the method is not inference-time and the central claim is refuted; if they reproduce with a reference-free R, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 defines L(λ)=γ Q(s_corr)+δ R(s_corr) with R described only as 'Whisper embedding L2 similarity'. No reference signal is specified. In the experimental setup (§3.1), Whisper-base is used 'to compute a L2 distance-based robustness objective', but the second operand is not stated. If R compares the corrected waveform to a clean reference (as done for the SpkVer evaluation in §3.2), the method requires ground-truth clean speech during inference, contradicting the abstract's claim of operating 'entirely in the inference stage without additional training' and §1's 'without retraining'. If R instead compares to the noisy input, the paper does not say so, and the optimization would pull corrected output toward the noisy signal, potentially negating noise reduction. All reported gains in Tables 1–3 are contingent on this unspecified term, so the central claim is not reproducible as written.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an inference-time \"rethink-and-refine\" correction module for speech enhancement. Given a noisy signal and the output of a pretrained SE model, the method uses ASR-based word/phoneme alignment to segment both signals, then optimizes per-segment convex mixing weights so that the reconstructed waveform maximizes a composite objective combining a non-intrusive quality score Q and a speech-preservation robustness score R. The authors claim that this plug-in module improves PESQ, STOI, and downstream ASR/speaker-verification metrics over the baseline enhancer across three SE models and several datasets, while requiring no retraining. A subjective listening study on a subset of URGENT 2024 is also reported. The central idea is clear and the experiments cover a reasonable range of conditions, but the definition of the robustness term R is incomplete, and one of the headline evaluation metrics (SCOREQ) is also the optimization objective, which weakens the evidence as currently presented.","tokens_in":8864,"tokens_out":4947,"duration_ms":46671,"significance":"If the claimed gains are real and reproducible, the module would be a practically attractive post-processor: it treats the base SE model as a black box, requires no training, and could be attached to arbitrary enhancers to reduce over-suppression artifacts. The paper also evaluates downstream WER, speaker embedding distance, jitter/shimmer, and a small subjective test, which is broader than typical SE evaluations. The main limitation is that the empirical claim is currently conditional on an underspecified objective function: the R term in Eq. (2) is not defined as a computable function, and the SCOREQ metric appears both inside the optimized objective and as a reported result, creating a selection effect. The paper's value will be clear once the objective is fully specified and the independent metrics are separated from the optimized ones.","major_comments":[{"comment":"The robustness term R(·) is not defined precisely enough for the experiments to be reproducible. The text says only that R measures \"Whisper embedding L2 similarity,\" and Section 3.1 repeats that Whisper-base is used \"to compute a L2 distance-based robustness objective,\" but the reference signal for the L2 distance is never stated. If the reference is the clean speech, the method requires ground-truth clean audio at inference time, contradicting the abstract's claim that it operates \"entirely in the inference stage\" and Section 1's claim of \"without retraining.\" If the reference is the noisy input, the optimizer is rewarded for pulling corrected segments toward the noisy signal, which would counteract noise reduction. If the reference is the enhanced signal itself, the term becomes vacuous. Because all reported gains in Tables 1-3 are produced by optimizing Eq. (2), the central experimental claim is not reproducible as written.","section":"Section 2.2, Eq. (2)"},{"comment":"There is a circularity between the optimization objective and the evaluation protocol. SCOREQ appears as the Q(·) term in Eq. (2), and SCOREQ MOS is also reported as a headline metric in Tables 1 and 2. The SCOREQ improvements are therefore partly selected by the optimizer rather than independently measured; they should not be cited as evidence of improved perceptual quality. In addition, the R term uses Whisper-base embeddings while WER is computed with Whisper-base, so the downstream WER gains are not fully independent of the objective either. The manuscript should report the optimized metric separately from independently evaluated metrics, or should replace SCOREQ in the reported tables with a held-out non-intrusive metric that does not appear in Eq. (2).","section":"Section 2.2 and Section 3.2, Tables 1-2"},{"comment":"The stated contribution of \"consistent gains in intrusive metrics\" is not fully supported by the tables. On VCTK-DEMAND in Table 1, PESQ drops from 3.559 (enhanced) to 3.415 (corrected), and in Table 2, SGMSE AB-Aes CE drops from 4.084 to 4.050 while PQ drops from 5.716 to 5.554. The text acknowledges \"most cases\" for some metrics, but the abstract and contribution (2) claim consistent gains. Moreover, the tables are labeled \"mean±std\" but only point estimates are shown, so the reader cannot assess whether the improvements are statistically meaningful; many of the reported differences are likely within one standard deviation. Please either qualify the central claim to match the actual per-metric outcomes or provide per-utterance statistical tests with standard deviations and significance levels.","section":"Section 4.1, Table 1 and Section 4.2, Table 2"},{"comment":"The repair mechanism is bounded by the convex hull of the noisy and enhanced segments. If the enhancer introduces phase, spectral, or temporal distortions that are not present in either input, Eq. (1) cannot undo them, and the optimizer can only choose how much of each input to keep. The paper should explicitly acknowledge this limitation and discuss what kinds of over-suppression artifacts are actually correctable by this interpolation. This does not invalidate the approach, but it narrows the claim that the module can be applied to \"any off-the-shelf SE model\" for arbitrary corruption types.","section":"Section 2.1, Eq. (1)"}],"minor_comments":[{"comment":"The figure uses θ and α for the mixing coefficients while Section 2.2 uses λ; the notation should be unified for clarity.","section":"Fig. 1 and Section 2.2"},{"comment":"The figure shows a minimization of -γQ - δR while Eq. (2) defines maximization of γQ + δR; the sign convention should be made consistent in both places.","section":"Fig. 1 and Eq. (2)"},{"comment":"The table headers say \"mean±std\" but the displayed values contain no standard deviations; either include the standard deviations or change the header labels.","section":"Tables 1-3"},{"comment":"The enhanced WER of 226.4% is much larger than the noisy WER of 93.3%; since WER can exceed 100% when insertions dominate, the paper should explain this behavior and clarify whether the correction module mainly reduces insertions.","section":"Table 3, URGENT 2025 row"},{"comment":"The enhanced row reports jitter as 0.84×10⁻² while the clean row reports 2.36×10⁻²; this is surprising and should be verified, since enhanced speech is expected to have jitter closer to, or larger than, clean speech.","section":"Table 3, URGENT 2024 row"},{"comment":"Only mean ratings are given for the listening test; the paper should report the number of utterances and annotators per condition and a more standard inter-rater reliability measure, since the reported 54.77% agreement is difficult to interpret.","section":"Section 3.2, subjective listening test"},{"comment":"The method optimizes K mixing weights per utterance and evaluates the full reconstructed waveform multiple times, but no runtime or complexity analysis is provided. A brief wall-clock comparison to the base SE model would help assess practical usability.","section":"Section 5"},{"comment":"The prior global-mixing methods [6,7] are discussed qualitatively but no direct comparison to them is presented; adding such a comparison would strengthen the claim that per-segment optimization is beneficial.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be a workshop-style paper, and the main technical concern is the underspecified R term in Eq. (2). The authors should be asked to state explicitly what the Whisper embedding L2 distance is computed against, and to separate optimized metrics from independent evaluation metrics. If the R term turns out to use a clean reference, the inference-time claim would collapse and the paper would need substantial rewriting. If the authors can clarify this and re-run the independent metric analysis, the work could be a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea here is genuinely worth a look: instead of the usual global or utterance-level mixing between noisy and enhanced speech, the authors segment both signals with an ASR aligner and optimize a per-segment interpolation weight at test time, guided by a non-intrusive quality score (SCOREQ) and a speech-preservation term. They also add a reconciliation step that tries two segmentations and a fallback that keeps the original noisy or enhanced signal if it scores better. That is a real, practical extension of the prior output-level correction work, and the experiments cover a decent spread: five datasets, three SE models, plus a small listening test. The writing is clear and the method is easy to understand.\n\nWhat worries me is the robustness objective in Eq. (2). The paper says only “Whisper embedding L2 similarity” and never specifies the second operand. If it compares against a clean reference, the method needs ground-truth speech at inference, which contradicts the abstract’s claim of operating entirely in the inference stage. If it compares against the noisy input, the optimizer is being pulled back toward noise. Either way, every gain in Tables 1–3 is contingent on a term that is not reproducible as written. That is the load-bearing flaw.\n\nThere are also signs of sloppy table preparation. VCTK-DEMAND rows in Table 1 share AB-Aes PQ and SCOREQ values with URGENT 2024 rows, which looks like a copy-paste error. The captions say “mean±std” but no standard deviations appear anywhere. A WER of 226.4% for the enhanced condition in URGENT 2025 is possible (insertions) but needs an explanation. And the abstract says “consistent improvements” while Table 1 shows VCTK PESQ dropping from 3.559 to 3.415 and Table 2 shows SGMSE CE/PQ dropping. The paper concedes this later, but the abstract overclaims.\n\nThe overlap between the optimization objective and evaluation is also real: SCOREQ selects the weights and is then reported as a headline metric, and Whisper embeddings drive R while Whisper ASR computes WER. The independent metrics (PESQ, STOI) mostly do improve, so I would not call the central claim false, only not fully established.\n\nThe convex-interpolation assumption is a limitation but not a fatal one; it is reasonable for attenuation-type over-suppression, even if it cannot fix phase or spectral distortions outside the convex hull of the two inputs.\n\nBottom line: the method is promising and the experiments are broad enough to deserve serious referee time, but the missing specification of R and the table errors mean the paper is not acceptable in its current form. I would send it to peer review with a request for major revision, and I would not cite or build on the numbers until R is pinned down and the tables are corrected.","headline":"Useful plug-in idea for fixing over-suppression at inference time, but the robustness objective R is unspecified and several tables look wrong; deserves a major-revision round, not acceptance as-is.","tokens_in":9427,"tokens_out":3191,"would_cite":false,"duration_ms":29630,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Over-suppression in speech enhancement can be corrected at inference time by per-segment remixing of the noisy and enhanced signals, guided by ASR alignments and optimized against a composite quality-and-preservation objective, with no…","keywords":["speech enhancement","over-suppression","inference-time correction","convex interpolation","non-intrusive quality assessment","ASR-guided segmentation","rethink-and-refine"],"falsifier":"Run the module on a deliberately damaged utterance in which an enhancer over-suppresses a phoneme by time-reversing that segment's waveform; a time-reversed segment and the noisy segment span only their own samples, so no $\\lambda \\in [0,1]$ can reproduce the time-forward clean segment, and if PESQ/STOI of the corrected output does not beat the better of noisy and enhanced on that segment, the convex-hull repair assumption is falsified. A lighter check: inject a pure tone into a segment at a frequency absent from both inputs and verify that optimal interpolation leaves the artifact in the output.","tokens_in":8453,"feed_emoji":"🎙️","tokens_out":18963,"duration_ms":145295,"temperature":0.7,"pith_summary":"Speech enhancement models sometimes over-suppress: they remove genuine speech cues together with the noise, and the output can be less intelligible and less useful for downstream tasks than the unprocessed recording. This paper claims that this failure can be repaired after the fact, without retraining the enhancer, by treating the noisy input and the enhanced output as two views of the same utterance and locally blending them back together. An ASR model aligns both signals into word- or phoneme-level intervals, and for each interval the module picks a mixing weight that maximizes a composite objective built from a non-intrusive quality predictor and a speech-preservation score. Because only the damaged regions are remixed toward the noisy original, the denoising benefit is kept where the enhancer worked. Across three enhancers and four datasets the paper reports higher PESQ, STOI, and subjective ratings, and lower ASR word error rates, which is what makes the approach matter: existing enhancers can be wrapped by the module and immediately produce better and more usable speech.","feed_headline":"Remixing noisy and enhanced audio fixes over-suppression","feed_subtitle":"Per-segment mixing, guided by quality and speech-preservation scores, lifts PESQ and STOI and cuts ASR word errors.","key_machinery":"The load-bearing object is the per-segment convex interpolation of Equation 1, $$\\hat{s}^{\\mathrm{corr}}_i(t,\\lambda_i) = \\lambda_i \\hat{s}_i(t) + (1-\\lambda_i)x_i(t)$$ with $\\lambda_i \\in [0,1]$, which turns over-suppression repair into a per-segment blending problem. ASR-guided segmentation (Charsiu for phoneme alignments, Whisper-timestamped with DTW for word alignments) supplies the paired intervals; the weights are chosen by gradient descent on the composite objective $L(\\lambda) = \\gamma Q(s^{\\mathrm{corr}}) + \\delta R(s^{\\mathrm{corr}})$, with $Q$ the SCOREQ non-intrusive quality predictor and $R$ the Whisper-embedding L2 similarity. Reconciliation runs the optimization on both the noisy-derived and enhanced-derived segment boundaries and keeps the higher-scoring reconstruction, and a final fallback retains the original noisy or enhanced signal if either scores higher. This machinery is what allows a black-box enhancer to be treated as a first-pass generator whose unreliable regions can be re-estimated, making a training-free correction stage possible.","core_discovery":"The central claim is that the destructive error of over-suppression is local, detectable, and correctable from observable signals alone. For each ASR-aligned segment there exists a weight $\\lambda_i \\in [0,1]$ such that replacing the enhanced segment with the convex combination $\\lambda_i \\hat{s}_i(t) + (1-\\lambda_i)x_i(t)$ improves perceptual quality relative to the enhancer's output while preserving speech content. The module never modifies the enhancer; it optimizes the whole vector of segment weights against a composite objective, compares the corrected waveform, the enhanced waveform, and the noisy waveform under the same objective, and keeps the highest-scoring one. Across CMGAN, SEMamba, and SGMSE+ on URGENT 2024/2025, VCTK-DEMAND, and MSP-PODCAST/Audioset, the paper reports consistent gains in PESQ, STOI, SCOREQ MOS, and aesthetic scores, plus reductions in ASR word error rate and speaker-embedding distance, and a listening test rating corrected audio above the enhanced baseline.","pith_inferences":["The convex-hull repair bound means the method's ceiling is set by the noisy and enhanced inputs: if an enhancer introduces distortion not representable as a blend of the two (phase flips, pitch shifts, inserted hallucinated tones), no $\\lambda$ can reconstruct the clean speech; a richer repair family such as spectral masking or generative inpainting would be needed.","The robustness term $R$ is Whisper-embedding similarity, which ties correction to what ASR attends to; swapping $R$ for an emotion- or speaker-oriented embedding would likely reshape which segments get repaired, meaning the module is a general scaffolding whose behavior is set by the chosen objective.","The per-segment weights are optimized by gradient descent at test time, so latency scales with the number of segments and optimizer steps; a learned predictor that maps segment features to mixing weights could amortize the correction for real-time use, though the paper leaves efficiency to future work.","Reconciliation currently chooses between two segmentation views (noisy-derived and enhanced-derived); extending to several aligners or a minimum-cost boundary merge might reduce sensitivity to segmentation errors, an avenue the paper does not explore."],"forward_implications":["Pretrained speech enhancers can be upgraded at inference time: wrapping any existing model with the module adds a second pass that repairs over-suppressed segments, so deployments do not need retraining or new paired data.","Downstream speech tasks benefit directly: corrected audio lowers ASR word error rates and speaker-verification distances relative to enhanced audio, so transcription and speaker-identification systems receive more faithful inputs.","Because the optimizer works per segment, well-denoised regions are left mostly enhanced while only unreliable intervals are blended back toward the noisy signal, avoiding the uniform noise reintroduction of global mixing approaches.","The fallback rule (keep the better of noisy, enhanced, or corrected under the objective) makes the correction conservative: in principle it should never degrade the chosen baseline on that objective.","The framework separates enhancement from correction, mirroring the rethink-and-refine pattern from language-model decoding, which suggests speech-processing pipelines can be built with independent first-pass and repair stages."],"supporting_citations":[{"why":"CMGAN, one of the three pretrained enhancers wrapped by the module; the cross-model generality claim rests on repeated gains over its output.","marker":"[1]"},{"why":"SEMamba, the primary enhancer for the dataset-wise evaluation; most reported PESQ/STOI and downstream gains are measured on its output.","marker":"[2]"},{"why":"SGMSE+, the third enhancer tested, showing the correction is not tied to one architecture.","marker":"[3]"},{"why":"The global-interpolation baseline this work generalizes; it uses a single fixed mixing rule for the whole signal rather than per-segment weights.","marker":"[6]"},{"why":"The SNR-adapting utterance-level interpolation baseline whose MSP-PODCAST/Audioset evaluation setting is reused and whose global coefficient is replaced by per-segment optimization.","marker":"[7]"},{"why":"Charsiu, the alignment model that supplies phoneme-level timestamps for segmenting noisy and enhanced speech.","marker":"[14]"},{"why":"Whisper, used both to produce word-level timestamps and to compute the embedding L2 similarity that defines the speech-preservation objective R.","marker":"[15]"},{"why":"The dynamic time warping alignment used by Whisper-timestamped to obtain word-level segment boundaries.","marker":"[16]"},{"why":"SCOREQ, the non-intrusive quality predictor used as Q in the optimized composite objective.","marker":"[17]"}],"fun_headline_variants":["Over-suppression? Remix noisy and enhanced segments at inference","No-retrain remix cures over-suppressed speech","Inference-time rethink-and-refine lifts speech quality","Fixing over-suppression by remixing with noisy audio","Speech over-suppression? Remix and refine at inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The repair is only as good as the two signals being blended: every over-suppressed segment is assumed to be fixable by some straight-line mix of the enhanced segment and the original noisy segment, so any enhancer error that neither input contains (phase reversals, pitch shifts, inserted artifacts) falls outside what the module can restore.","fun_headline_variants_meta":{"raw":{"variants":["Over-suppression? Remix noisy and enhanced segments at inference","No-retrain remix cures over-suppressed speech","Inference-time rethink-and-refine lifts speech quality","Fixing over-suppression by remixing with noisy audio","Speech over-suppression? Remix and refine at inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000825,"raw_usage":{"total_tokens":3599,"prompt_tokens":926,"completion_tokens":2673,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":2592}},"tokens_in":542,"tokens_out":2673,"duration_ms":18357,"temperature":1.0,"reasoning_tokens":2592,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:12:56.812815+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the module on a deliberately damaged utterance in which an enhancer over-suppresses a phoneme by time-reversing that segment's waveform; a time-reversed segment and the noisy segment span only their own samples, so no $\\lambda \\in [0,1]$ can reproduce the time-forward clean segment, and if PESQ/STOI of the corrected output does not beat the better of noisy and enhanced on that segment, the convex-hull repair assumption is falsified. A lighter check: inject a pure tone into a segment at a frequency absent from both inputs and verify that optimal interpolation leaves the artifact in the output.","supporting_citations":[{"cited_title":"Reflexion: Lan- guage agents with verbal reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"SCOREQ, the non-intrusive quality predictor used as Q in the optimized composite objective."},{"cited_title":"Mitigating Over-Suppression in Speech Enhancement via Inference-Time Rethink-and-Refine Correction Module","cited_arxiv_id":"2608.07781","evidence_quote":"CMGAN, one of the three pretrained enhancers wrapped by the module; the cross-model generality claim rests on repeated gains over its output."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SEMamba, the primary enhancer for the dataset-wise evaluation; most reported PESQ/STOI and downstream gains are measured on its output."},{"cited_title":"Models and datasets We evaluate our approach using three state-of-the-art SE models, including CMGAN [1], SEMamba [2], and SGMSE [3]","cited_arxiv_id":null,"evidence_quote":"SGMSE+, the third enhancer tested, showing the correction is not tied to one architecture."},{"cited_title":"CM- GAN: Conformer-based metric GAN for speech en- hancement,","cited_arxiv_id":null,"evidence_quote":"The global-interpolation baseline this work generalizes; it uses a single fixed mixing rule for the whole signal rather than per-segment weights."},{"cited_title":"An investigation of incorporating mamba for speech enhancement,","cited_arxiv_id":null,"evidence_quote":"The SNR-adapting utterance-level interpolation baseline whose MSP-PODCAST/Audioset evaluation setting is reused and whose global coefficient is replaced by per-segment optimization."},{"cited_title":"An algorithm for intelligibility prediction of time- frequency weighted noisy speech,","cited_arxiv_id":null,"evidence_quote":"Charsiu, the alignment model that supplies phoneme-level timestamps for segmenting noisy and enhanced speech."},{"cited_title":"MetricGAN- OKD: Multi-metric optimization of MetricGAN via on- line knowledge distillation for speech enhancement,","cited_arxiv_id":null,"evidence_quote":"Whisper, used both to produce word-level timestamps and to compute the embedding L2 similarity that defines the speech-preservation objective R."},{"cited_title":"Multi-CMGAN+/+: Leveraging multi- objective speech quality metric prediction for speech en- hancement,","cited_arxiv_id":null,"evidence_quote":"The dynamic time warping alignment used by Whisper-timestamped to obtain word-level segment boundaries."}],"review_version":1}