{"id":"eba23adf-394b-444a-9717-74953127a4ce","arxiv_id":"2505.17070","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A streaming transducer ASR system that combines an encoder-driven voice activity detector, an end-of-word token, and a delay penalty improves endpointing F1 and lowers WER on Switchboard compared with blank-based endpointing.","lead":"This paper adds a second speech-detector network and an end-of-word token to a streaming transducer ASR system so the system can notice when a speaker has stopped without cutting off the last word. The methods improve endpoint detection F1 on the Switchboard corpus and keep word error rate near the no-endpointing level.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"EOW rule's WER gain may be a latency effect: nominal 200 ms trailing silence does not equal endpoint latency when the endpoint must wait for EOW, and the EOW model's 'oracle WER' is not in Table 3.","rationale":"The paper's own Figure 5 establishes that WER saturates as latency increases. Endpointing is a latency-WER tradeoff, so any comparison that claims improved WER at '200 ms trailing silence' must hold actual end-to-end endpoint latency fixed. The EOW rule, by construction, can add latency beyond the TS threshold when it waits for EOW. This is not a minor reporting gap; it determines whether the 21.4% WER is evidence for better endpointing or simply a less aggressive endpoint. The missing no-EP WER for the EOW model is part of the same claim: 'oracle WER for this model (21.4%)' needs its own baseline. The reader's ASpIRE-teacher concern is real but secondary; it affects the VAD branch's absolute quality, whereas the latency/oracle-baseline issue attacks the headline WER result directly. Verdict remains conditional: if the latency-matched rerun and no-EP baseline confirm the result, the paper's empirical case is much stronger; if not, the central claim is unsupported. I therefore keep the reader's CONDITIONAL verdict, with the additional required condition of a latency-matched, oracle-baseline comparison.","tokens_in":7835,"tokens_out":12790,"duration_ms":122208,"concrete_test":"On the 19-call test set, log actual endpoint-decision times for: EOW rule at delta=200 ms, encNET+TS at delta=200 ms, and blank-based EP at delta=200 ms; compute each true endpoint's (decision_time - reference_speech_end_time) and report median/quantiles. Then run encNET+TS with delta adjusted so its median latency equals the EOW rule's median latency (e.g., delta=450 ms if that is EOW's median) and recompute precision/recall/WER. Also decode the EOW-trained model with no endpointing and report that WER. If matched-latency WERs are within 0.5% absolute, the EOW WER gain is a latency effect; if EOW stays ~1.5% better and the no-EP WER equals 21.4%, the central claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.4 reports EOW-based endpointing with WER 21.4%, called 'oracle WER for this model,' at a trailing-silence (TS) setting of 200 ms. But an EOW endpoint is only finalized when the EOW token is emitted (Section 2.3, cases C1/C2). If the joiner emits EOW after the 200 ms silence threshold, actual endpoint latency is longer than 200 ms. The paper never reports the endpoint-latency distribution for the EOW rule at the 200 ms setting; Figure 5 plots WER only for 400/600/800 ms latency and shows all systems approach oracle by 600 ms. If EOW's effective latency is closer to 400-600 ms, its low WER is the expected latency-WER tradeoff, not an endpointing improvement. Additionally, Table 3 lacks a 'no endpointing' row for the EOW-trained model; the only oracle WER given (21.92%, Table 2) is for the vanilla model. The claim that EOW achieves 21.4% oracle WER therefore cannot be checked from the reported data. The central 'improves WER significantly' conclusion thus rests on an uncontrolled comparison.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses endpointing in streaming transducer-based ASR for conversational speech. It proposes two components: an auxiliary VAD network that uses either mel features (melNET) or encoder embeddings (encNET) to trigger endpoints from trailing silence, and an end-of-word (EOW) token trained with a delay penalty to promote prompt emission of word-final tokens. Three endpointing strategies are compared: trailing-silence only (TS), EOW-only, and TS+EOW. Experiments on Switchboard (train/validation/test: 2353/132/19 calls, with 3.1 h of test audio) report endpoint precision/recall/F1 and WER at 200 ms trailing silence, plus latency-WER curves for 400-800 ms. The main reported results are that encNET gives EER 0.105 versus 0.182 for melNET, that encNET TS endpointing raises F1 from 46.6 to 78.6 and lowers WER from 24.4% to 23.0%, and that the EOW rule achieves 21.4% WER, which the text calls 'oracle WER for this model'.","tokens_in":8033,"tokens_out":5367,"duration_ms":49528,"significance":"If the claims hold, the paper gives a practical and inexpensive way to improve endpointing in streaming conversational ASR: reusing encoder embeddings for a lightweight VAD branch, and adding an EOW token with a delay penalty. Strengths of the study are that it evaluates on spontaneous Switchboard speech rather than read speech, performs call-level decoding, uses the sed_eval toolkit with a 200 ms tolerance, and includes a blank-based baseline and an oracle for comparison. The DET-curve comparison between encNET and melNET is informative. However, the central EOW result is under-supported: the oracle WER for the EOW-trained model is not reported, the 200 ms operating point is missing from the latency plot, and the small test set has no significance tests. These issues are fixable but currently weaken the main conclusion.","major_comments":[{"comment":"The statement that EOW-based endpointing 'achiev[es] oracle WER for this model (21.4%)' cannot be verified from the reported data. Table 2 gives the no-endpointing oracle WER for the vanilla model as 21.92%, but no oracle WER for the EOW-trained model is reported anywhere, and Table 3 has no 'no endpointing' row for that model. The authors should either measure and report the EOW model's oracle WER explicitly, or soften the claim to a direct comparison with the other endpointing strategies.","section":"Section 3.4 / Table 3"},{"comment":"The WER comparison for the EOW rule at the 200 ms trailing-silence setting is confounded by latency. As described in Section 2.3 for cases C1 and C2, an endpoint under the EOW rule is finalized only when the joiner emits EOW; if EOW emission occurs after the 200 ms silence threshold, the actual endpoint latency is longer than 200 ms. Figure 5 plots WER only for 400, 600, and 800 ms and shows all systems approaching the oracle by 600 ms, so the 21.4% WER could reflect a longer effective latency rather than better endpointing. The authors should report the actual endpoint-latency distribution for each strategy at the 200 ms setting and compare WER at matched latency.","section":"Section 3.4 / Figure 5 and Section 2.3"},{"comment":"The statistical basis is thin: the test set contains only 19 calls (3.1 h), and no confidence intervals or significance tests are reported for the precision/recall/F1/WER differences. Because several of the headline differences are small (e.g., 21.4% vs. 21.7%, or 23.0% vs. 21.92%), the authors should add bootstrap confidence intervals or significance tests and explicitly state that the delay-penalty coefficient, the EER operating point, and the trailing-silence thresholds were selected on the validation set only.","section":"Section 3.1 / Tables 1-3"}],"minor_comments":[{"comment":"The text says 'improving the WER (from 22.4% to 22.9%)', but Table 2 shows the blank-based vanilla baseline WER is 24.4%; this appears to be a typo and should be corrected to 24.4%.","section":"Section 3.3"},{"comment":"The phrase 'with out endpointing' should read 'without endpointing'.","section":"Section 3.3"},{"comment":"The definition of the reference endpoint events used in the sed_eval alignment is not stated; the authors should specify how reference timestamps were derived from the call-level transcripts, since precision/recall/F1 depend directly on this definition.","section":"Section 3.1"},{"comment":"The figure omits the 200 ms operating point that is central to the discussion in Sections 3.3 and 3.4; adding this point (or explaining why it is not shown) would make the latency-WER trade-off much clearer.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The strongest claim, that the EOW rule reaches oracle WER, depends on an oracle number that is not in the paper. I would ask the authors to supply the no-endpointing WER for the EOW model and to disclose the actual endpoint latency at the 200 ms setting. The paper is otherwise an incremental but potentially useful engineering contribution, and with those additions it could be acceptable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nWhat you should know: this is a legitimate engineering increment, but the paper's strongest sentence — that EOW endpointing achieves oracle WER — is not supported by its own tables. The stress-test note is right: Table 3 has no 'no endpointing' row for the EOW-trained model, so 21.4% cannot be compared against a proper oracle for that model, and the WER gain could be mostly a latency effect, since an EOW endpoint waits for the token to be emitted rather than stopping at the nominal 200 ms of trailing silence.\n\nThe real contributions are modest but real. Using the Zipformer encoder embeddings as input to a small feed-forward VAD (encNET) beats a mel-spectrogram VAD (EER 0.105 vs 0.182) and raises endpointing F1 from 46.6 to 78.6 against a blank-based baseline. The EOW token plus delay penalty is a sensible pairing to avoid word fragments when stopping aggressively, and the evaluation on Switchboard spontaneous speech with call-level decoding is more realistic than the read-speech benchmarks common in this subfield. The direction of the improvements is consistent across Table 2 and Figure 5.\n\nThe soft spots are proportionate to the claims. The test set is 19 calls / 3.1 hours — no confidence intervals, no significance tests, and the delay penalty coefficient is tuned without an explicit statement that the test set was shielded. The paper also never compares directly against the closest prior methods, [16] and [17], which are both LSTM-based endpointing and EOQ-style tokens. And the conclusion overclaims: 'without any degradation from oracle WER' presumes a comparison that is not in the tables.\n\nWho gets value: engineers building streaming ASR endpointing, particularly for conversational products. The combination of encoder-based VAD with an EOW rule is worth knowing about, but the magnitude of the benefit is uncertain until the latency confound is unpacked. A serious referee would have a clear job: ask for the holdout tuning boundary, error bars, a direct oracle row for the EOW model, and a re-analysis that separates endpoint quality from the extra latency the EOW rule might impose.\n\nMy recommendation: send it to peer review. The problem is real, the approach is coherent, and the flaws are fixable without changing the architecture. But the manuscript as written should not pass unchanged; the WER claim needs to be either supported or withdrawn.\n\nBest,","headline":"A useful engineering increment whose headline WER claim is not actually supported by the reported data — the 'oracle' comparison is missing and the EOW rule's latency is uncontrolled.","tokens_in":8697,"tokens_out":2551,"would_cite":false,"duration_ms":23905,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an end-of-word token, trained with a delay penalty and synchronized with a voice-activity detector, lets a streaming transducer ASR endpoint conversational speech at its no-endpointing oracle word error rate.","keywords":["endpointing","streaming ASR","RNN-T","voice activity detection","end-of-word token","latency","Switchboard","Zipformer"],"falsifier":"A single ablation study would settle it: train four variants of the system--encNET trained with teacher labels versus hand-corrected labels, and with versus without the EOW token--and compare endpoint precision and WER. If the gains disappear when teacher labels are replaced by hand-corrected labels, the VAD teacher is the source; if they disappear when EOW is removed, the token is the source.","tokens_in":7571,"feed_emoji":"🎤","tokens_out":10718,"duration_ms":100816,"temperature":0.7,"pith_summary":"This paper tries to solve a user-facing problem in streaming speech recognition: deciding exactly when a speaker has finished, without cutting off the last word and without waiting too long. For transducer-based models, endpointing usually relies on runs of blank tokens, but delayed token emission makes blank runs unreliable. The paper claims that two modifications fix this: an auxiliary voice-activity detection network that reads the ASR encoder's internal representation, and an end-of-word token added to every word during training with a delay penalty. On conversational telephone speech, the end-of-word endpoint rule reaches a word error rate of 21.4%, the same as the model's no-endpointing oracle (the lower bound when no endpoint is ever triggered), while endpoint precision rises from 47.7% to 81.4% compared with blank-based endpointing. A reader should care because endpointing errors are what make voice agents interrupt users or return truncated transcripts, and these results suggest those errors can be cut sharply at low latency.","feed_headline":"Word-end token lets streaming ASR hit oracle WER","feed_subtitle":"On conversational speech, the rule lifts endpoint precision from 48% to 81% at the same word error rate as no endpointing.","key_machinery":"The central machinery is a pair of synchronized signals for the endpoint decision. The first is encNET, a two-hidden-layer feed-forward network that takes the Zipformer encoder's frame embeddings and emits a speech/non-speech label; the duration of contiguous non-speech frames from this network triggers a candidate endpoint. The second is an end-of-word (EOW) token appended to every word in the training transcripts, trained with a delay penalty so the transducer emits EOW promptly at word endings. The endpoint is finalized only when the most recent output from the transducer's joiner is EOW, which prevents the VAD's early silence detection from cutting a word in half while the delayed token is still on its way. The delay penalty removes the blank-token lag that causes false, missed, and delayed endpoints in the baseline.","core_discovery":"The central discovery, as the paper states it, is that an end-of-word token is the load-bearing element for high-precision endpointing in a delay-penalized transducer. The EOW token is appended to each word in the training text, so the transducer's output layer must emit a special symbol at the true word boundary. When the endpoint logic waits for that token instead of counting blank frames, false endpoints caused by delayed non-blank emission disappear, and WER on the Switchboard test set improves to 21.4%--exactly the oracle WER of the same model with no endpointing. Adding the VAD condition (trailing silence) alongside EOW gives the best F1 score of 73.0 at a small WER cost (21.7%). The paper also reports that a VAD trained on encoder embeddings (encNET) has an equal error rate of 0.105 versus 0.182 for a VAD trained on acoustic features, showing that the ASR encoder carries reliable speech/non-speech information.","pith_inferences":["A natural next test is to drop the external teacher entirely and train encNET from the ASR model's own blank/non-blank emission history; if that self-supervised VAD matches teacher-trained encNET, the endpointing pipeline becomes fully self-contained.","Because EOW forces a lexical boundary at word end, the same token could serve as a cheap word-boundary signal for word-level timestamps, segmentation, or turn-taking prediction in streaming systems, not just endpointing.","The precision/recall trade-off suggests an assistant-oriented deployment would use EOW for the primary endpoint and accept lower recall, recovering missed endpoints with a downstream backchannel or turn-taking model; the paper does not test this combination.","Languages with long or morphologically complex words may not align acoustic word boundaries with the EOW training token as cleanly as English, so the 21.4% result on Switchboard should be re-measured on such languages before generalizing."],"forward_implications":["With EOW gating, a 200 ms trailing-silence threshold no longer forces a WER penalty: EOW-based endpointing reaches oracle WER at that aggressive setting, whereas blank-based and TS-only endpointing degrade.","The VAD branch can be trained from the ASR encoder's own embeddings, so endpoint detection comes at little extra compute and shares the ASR model's acoustic context.","Combining TS and EOW rules gives production flexibility: TS+EOW produces the best F1 (73.0) at slight WER cost (21.7%), useful when avoiding missed endpoints matters more than WER.","The latency-WER curve saturates around 600 ms of trailing silence for all systems, so EOW shifts the usable operating point to much lower latency without transcription loss.","Reaching oracle WER under endpointing means endpoint decisions can be made without sacrificing ASR accuracy on spontaneous conversational speech."],"supporting_citations":[{"why":"supplies the delay penalty used in training so the end-of-word token is emitted without the transducer's usual lag.","marker":"[11]"},{"why":"cited as the source of the external VAD teacher labels used to train encNET for speech/non-speech classification.","marker":"[25]"},{"why":"provides the conversational telephone speech corpus, partitioned into the train, validation, and test sets used in all experiments.","marker":"[19]"},{"why":"supplies the test set of Switchboard calls on which endpoint precision, recall, and WER are measured.","marker":"[23]"},{"why":"defines the Zipformer encoder whose frame embeddings feed encNET and serve as the ASR backbone.","marker":"[7]"},{"why":"describes the blank-token aggregation baseline for endpointing that the paper compares against.","marker":"[22]"},{"why":"supplies the event-alignment precision and recall metrics used to score endpoint detections.","marker":"[24]"}],"fun_headline_variants":["EOW token achieves 81% endpoint precision at oracle WER","Word-end token fixes delayed emission, improves endpoint precision","Streaming ASR endpointing: EOW token hits oracle WER","EOW token + VAD yields best F1 in streaming ASR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach relies on the external VAD teacher's speech/non-speech labels being a trustworthy reference for conversational audio; if those labels are biased, encNET inherits the bias and the reported endpoint gains may reflect the teacher rather than the ASR encoder embeddings.","fun_headline_variants_meta":{"raw":{"variants":["EOW token achieves 81% endpoint precision at oracle WER","Word-end token fixes delayed emission, improves endpoint precision","Streaming ASR endpointing: EOW token hits oracle WER","EOW token + VAD yields best F1 in streaming ASR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000744,"raw_usage":{"total_tokens":3310,"prompt_tokens":927,"completion_tokens":2383,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":2309}},"tokens_in":543,"tokens_out":2383,"duration_ms":17963,"temperature":1.0,"reasoning_tokens":2309,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:16:49.989263+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A single ablation study would settle it: train four variants of the system--encNET trained with teacher labels versus hand-corrected labels, and with versus without the EOW token--and compare endpoint precision and WER. If the gains disappear when teacher labels are replaced by hand-corrected labels, the VAD teacher is the source; if they disappear when EOW is removed, the token is the source.","supporting_citations":[{"cited_title":"Zipformer: A faster and better encoder for automatic speech recognition,","cited_arxiv_id":null,"evidence_quote":"supplies the delay penalty used in training so the end-of-word token is emitted without the transducer's usual lag."},{"cited_title":"Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusion","cited_arxiv_id":"2401.14717","evidence_quote":"cited as the source of the external VAD teacher labels used to train encNET for speech/non-speech classification."},{"cited_title":"Endpoint detection for streaming end- to-end multi-talker ASR,","cited_arxiv_id":null,"evidence_quote":"provides the conversational telephone speech corpus, partitioned into the train, validation, and test sets used in all experiments."},{"cited_title":"Switchboard: Telephone speech corpus for research and development,","cited_arxiv_id":null,"evidence_quote":"supplies the test set of Switchboard calls on which endpoint precision, recall, and WER are measured."},{"cited_title":"Is the speaker done yet? Faster and more accurate end-of-utterance detection using prosody,","cited_arxiv_id":null,"evidence_quote":"supplies the event-alignment precision and recall metrics used to score endpoint detections."}],"review_version":1}