{"id":"7fe6830e-5ec8-4ada-ab6e-eac806ba28ee","arxiv_id":"2607.06831","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Gradient saliency of teacher-forced token log-probabilities, decoded by dynamic programming, yields usable word alignments for every ASR family including speech LLMs.","lead":"A training-free gradient saliency method produces word-level speech-to-text alignments for any differentiable ASR model, including speech LLMs. It is usable everywhere and beats weak native aligners (especially streaming models), though it costs one backward pass per token.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The paper's strongest claim is an empirical existence result: one training-free gradient procedure yields usable word alignments across CTC, transducers, AED and speech LLMs, and is competitive or superior precisely when the model's own signal is weak. That claim is backed by a broad, fair comparison (sixteen models, two corpora, native baselines, ablations on scoring, topology, depth and tokenization). The saliency-to-boundary step is heuristic rather than derived, yet the paper quantifies its sensitivity and shows robustness; this is standard practice for attribution-based methods and does not undermine the reported numbers. Public code is promised, further lowering correctness risk. Consequently the reader's ACCEPT / high-confidence verdict stands; no load-bearing concern requires a change.","tokens_in":16604,"tokens_out":449,"duration_ms":5756,"concrete_test":"Reproduce Table I Buckeye WBE for the three streaming models and Canary-Qwen using the public code with the paper's default (ρ=0.5, energy blank λ=2, word-level topology); if any of those four gradient rows exceeds its native/attention counterpart by more than 20 ms, the \"better where native is weak\" clause would need re-examination.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption (that L2-norm input saliency of teacher-forced token log-prob is clean enough for energy-weighted DP to recover word boundaries) is ordinary for the field and is already stress-tested by the paper itself. Tables V–VII show that p-norm choice, energy weighting ρ, blank schemes, and multi-pass attribution variants move WBE by only a few ms; the same decoder improves Whisper's own DTW (Table VIII); and the method remains usable in hypothesis mode (Table II). The central claim is carefully hedged (\"usable,\" \"usually somewhat behind a strong native,\" \"better where native is weak\") and is directly supported by the sixteen-model comparison on TIMIT and Buckeye. No hidden circularity, untested regime, or internal inconsistency appears that would overturn the result.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes a training-free gradient-based forced alignment that applies to any differentiable ASR model. For each teacher-forced token it computes the gradient of the token log-probability with respect to the input, reduces it to a per-frame saliency (log L2-norm), optionally energy-weights the scores, and recovers word boundaries by a single time-synchronous Viterbi pass over a word-level blank topology. The method is shown to work for CTC (via prefix scores), transducers (RNN-T and TDT), attention-based encoder-decoders, and speech LLMs. Evaluation covers sixteen models on TIMIT and Buckeye, comparing gradient alignment against each model’s own native posterior or attention-based alignment (and against MFA as a classical reference). The central empirical claim is that the gradient yields a usable alignment for every family, is usually a little behind a strong native aligner, but is better precisely where the native signal is weak (streaming models, Canary-Qwen), and that its main cost is one backward pass per token.","tokens_in":16790,"tokens_out":678,"duration_ms":6443,"significance":"If the result holds, the paper supplies a single, model-agnostic, training-free procedure that produces word timings for every current ASR family, including speech LLMs that otherwise lack a built-in aligner. The evaluation is unusually thorough (sixteen models, two corpora, reference and hypothesis modes, extensive ablations on reduction norms, blank schemes, energy weighting, encoder depth, tokenization, and compute cost) and the claims are carefully hedged. Public source code is promised, which strengthens reproducibility. The work is therefore a useful reference result and analysis tool even if practitioners continue to prefer cheaper native aligners when they are strong.","major_comments":[],"minor_comments":[{"comment":"Table I caption and surrounding text should state more explicitly that the starred Whisper-large-v3 row uses an intermediate encoder depth chosen on development data; a casual reader may otherwise treat it as the default input-gradient result.","section":null},{"comment":"Section II, Eq. (1)–(4): a short remark on numerical stability of the log-norm and of the energy-aware blank (especially when σ_t is near zero) would help implementers.","section":null},{"comment":"Figure 1 is dense; adding a brief legend for the colour scale of the saliency matrices would improve readability.","section":null},{"comment":"The compute-cost table (Table XII) reports only a subset of models; a one-sentence note that the RTF ordering is expected to be similar for the remaining families would be useful.","section":null},{"comment":"A few typographical inconsistencies remain (e.g., “V oxtral”, “Schl ¨uter”); a final pass would clean them.","section":null}],"recommendation":"accept","confidential_remarks":"The manuscript is already in good shape for a conference or journal track that values careful empirical analysis. I see no load-bearing technical flaw. The main practical limitation (per-token backward cost) is openly acknowledged by the authors and does not undermine the scientific claim."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean, useful engineering result rather than a conceptual breakthrough. The authors take the gradient of each teacher-forced token log-prob w.r.t. the input, reduce it to per-frame saliency, and decode word boundaries with a single DP pass that generalizes Whisper’s DTW. They show it works for every family they try—including CTC/transducer prefix scores and speech-LLM self-attention—and that it is better precisely where the native signal is weak (streaming models, Canary-Qwen).\n\nWhat is actually new is the systematic extension and the fair sixteen-model comparison on TIMIT and Buckeye (reference and hypothesis mode). Prior work already used input gradients for AED and MT; the increments here are the prefix-score treatment, the improved DP decoder (Table VIII shows it improves Whisper’s own DTW), the self-attention baseline for speech LLMs, and the honest ablations on norms, energy weighting, blank schemes, encoder depth and tokenization. Claims are carefully hedged (“usable,” “usually somewhat behind a strong native,” “better where native is weak”). Public code is promised. Math and citation pattern look solid; no circularity.\n\nSoft spots are real but proportionate. The method is expensive (one backward per token). Character targets help the autoregressive models but collapse for CTC/transducers. Intermediate encoder depths sometimes beat the input grid, so the “input-grid precision” claim is not absolute. The L2-saliency assumption is ordinary for the field and already stress-tested by their own tables; multi-pass attribution does not help. None of this overturns the central empirical claim.\n\nThis is for people who need forced alignment on modern ASR stacks, especially speech LLMs or streaming models where native timings are poor. It deserves a serious referee. I would accept it for peer review and would cite the tables when I next need a model-agnostic aligner baseline.","headline":"Solid, carefully scoped empirical paper: one training-free gradient procedure works as a forced aligner across CTC, transducers, AED and speech LLMs, with thorough evaluation and honest hedging.","tokens_in":17411,"tokens_out":480,"would_cite":true,"duration_ms":5216,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A single gradient-of-token-probability signal yields usable word boundaries for every differentiable ASR model, including speech LLMs.","keywords":["speech-to-text alignment","forced alignment","gradient-based attribution","automatic speech recognition","speech language models","CTC","transducers","attention-based encoder-decoder"],"falsifier":"On a held-out spontaneous-speech set, for any of the streaming models or Canary-Qwen, replace the gradient scores with random or constant saliency of the same shape and show that the dynamic program still recovers word-boundary errors within a few tens of milliseconds of the reported figures; if it does, the claimed saliency content is not doing the work.","tokens_in":17491,"feed_emoji":"⏱️","tokens_out":700,"duration_ms":7379,"temperature":0.7,"pith_summary":"Speech-to-text alignment tells you when each word in a transcript begins and ends in the audio. Classic models give you that timing by construction; modern attention models and speech language models usually do not, and people read timings off attention weights on a coarse encoder grid. This paper shows that you can instead take the gradient of each teacher-forced token log-probability with respect to the raw input, turn the gradient into a per-frame saliency matrix, and recover word boundaries with one dynamic-programming pass. The procedure needs no training, no extra heads, and no model changes. On sixteen models spanning CTC, transducers, attention-based encoder-decoders and speech LLMs, evaluated on both read and spontaneous speech, the gradient alignment is usable everywhere, trails a strong native aligner only modestly, and beats the native signal precisely where that signal is weak (streaming models and one speech LLM). Its main cost is one backward pass per token.","feed_headline":"One gradient signal aligns every ASR model family","feed_subtitle":"Token-probability gradients give usable word timings even for speech LLMs and streaming models","key_machinery":"Input-gradient saliency matrix: for each transcript token, take the log L2-norm of the gradient of its teacher-forced log-probability w.r.t. the input frames, optionally energy-weight it, apply a log-softmax over time, then decode the token-by-frame matrix with a word-level blank topology via time-synchronous Viterbi. This single signal replaces model-specific posteriors or attention heads and lives on the fine input grid rather than the coarse encoder grid.","core_discovery":"A training-free procedure—gradient of each teacher-forced token log-probability with respect to the input, reduced to per-frame saliency and decoded by a single dynamic-programming pass—produces a usable word-boundary alignment for every differentiable ASR family tested (CTC, transducers, AED, speech LLMs) and is better than the model’s own native or attention alignment precisely where that native signal is weak.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Token gradients give usable word timings for every ASR family","One backward pass aligns CTC to speech LLMs on the input grid","Gradient saliency yields word boundaries across all ASR models","Training-free gradients beat weak native alignments on streaming ASR","Per-token input gradients decode word timings for any differentiable ASR"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That the magnitude of the input gradient of a single token’s log-probability is already a clean enough local saliency signal that a simple energy-weighted dynamic program can recover accurate word boundaries.","fun_headline_variants_meta":{"raw":{"variants":["Token gradients give usable word timings for every ASR family","One backward pass aligns CTC to speech LLMs on the input grid","Gradient saliency yields word boundaries across all ASR models","Training-free gradients beat weak native alignments on streaming ASR","Per-token input gradients decode word timings for any differentiable ASR"]},"model":"grok-4.5","effort":"low","cost_usd":0.003422,"raw_usage":{"total_tokens":1183,"prompt_tokens":823,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":34220000,"prompt_tokens_details":{"text_tokens":823,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":295,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":823,"tokens_out":65,"duration_ms":71056,"temperature":1.0,"reasoning_tokens":295,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T20:23:41.762739+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a held-out spontaneous-speech set, for any of the streaming models or Canary-Qwen, replace the gradient scores with random or constant saliency of the same shape and show that the dynamic program still recovers word-boundary errors within a few tens of milliseconds of the reported figures; if it does, the claimed saliency content is not doing the work.","supporting_citations":[],"review_version":1}