{"id":"25a5ebb4-7cca-47bd-a77c-8015ae704312","arxiv_id":"2505.16182","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Discrete speech tokens trained on a speaker's first language improve ASR accuracy on that speaker's accented English, reproducing the interlanguage speech intelligibility benefit in machines.","lead":"The paper shows that ASR recognizes foreign-accented English better when the speech tokens are trained on the speaker's native language, mimicking a known human perception effect. The result hints that accent-robust recognition could be built using only native speech data, without any accented recordings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"L1-matched WER gains in Tables 2 and 5 are mostly under 2 WER and come from single runs with no seeds or significance tests; the ISIB conclusion may rest on noise, and the mismatched-benefit pattern suggests a generic non-English-tokenizer robustness effect rather than a shared-L1 perceptual…","rationale":"We read the paper as a deliberate, small-scale empirical demonstration that a discrete-token ASR pipeline can reproduce the ISIB pattern when the tokenizer's training language equals the speaker's L1. The design is clean: within each block, k-means training language is the manipulated variable, and the matched cases in Table 5 do show the expected ordering. The QE/MTER analysis is a thoughtful attempt to characterize cluster behavior without phoneme labels. However, the central result is quantitatively fragile. The distinguishing matched effect is on the order of 0.6-2.4 WER, and no run-to-run variance is reported. With ESPnet CTC/attention ASR on small out-of-domain sets like ERJ, single-run WER differences of about 1 WER are plausible across seeds, so the reported matched advantage could be noise. The mismatched-benefit pattern in Table 5 also strengthens an alternative explanation: since Chinese and Spanish tokenizers improve WER for all foreign accents, part of the benefit is a generic property of non-English centroids rather than a L1-specific effect. This does not invalidate the paper, but it means the conclusion that discrete tokens reproduce ISIB is underdetermined. A controlled analysis with seed variance and entropy-matching would settle the issue. We therefore keep the reader's conditional verdict, since the core result needs stronger evidence before the perceptual-simulation claim can be accepted.","tokens_in":9938,"tokens_out":8848,"duration_ms":78649,"concrete_test":"Rerun all Table 5 conditions (en/jp/ch/sp km on LibriSpeech960 ASR) with at least 5 independent seeds and report mean WER with 95% bootstrap confidence intervals over utterances. In parallel, compute each tokenizer's effective cluster entropy on each test set and on LibriSpeech. If the L1-matched advantage disappears when tokenizers are matched for effective entropy, or if the matched margin falls inside the seed-noise interval, the ISIB-specific conclusion is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires the L1-matched tokenizer to be the best among tested tokenizers for each accent, but the decisive margins are small. In Table 5, Chinese-accented English is 28.6 WER with Chinese km versus 29.3 with Spanish km (delta = 0.7) and Spanish-accented English is 21.8 with Spanish km versus 22.4 with Chinese km (delta = 0.6). In Table 2, the jp-over-en advantage on Japanese-accented English is 2.4-4.7 WER, yet on native English the same comparison ranges from -1.1 to +0.2, suggesting that single-run WER differences of this size are not obviously beyond training noise. No seeds, checkpoint-selection variability, or utterance-level confidence intervals are reported. Furthermore, Table 5 shows that Chinese and Spanish km improve WER over English km for every non-native English test set, including Arabic, Hindi, Korean, and Vietnamese, whose L1s are absent from the tokenizer set. A large part of the observed pattern is therefore a generic 'non-English tokenizer is more robust to accented English' effect, which could arise from coarser quantization (e.g., Japanese km conflating /r/ and /l/) rather than from simulating a native-X listener. Without controlling for effective cluster usage or entropy, the specific attribution of the matched improvement to ISIB is not secured.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript reports an analytical study of discrete-token ASR for foreign-accented English. The authors train k-means tokenizers on native English, Japanese, Chinese, or Spanish speech and feed the resulting discrete units to an English ASR system trained on LibriSpeech. They report that for Japanese-accented English (ERJ), a Japanese-trained tokenizer outperforms an English-trained tokenizer, and in a multi-accent experiment (L2-ARCTIC), the tokenizer trained on the speaker's native language gives the best WER among the four tokenizers. They interpret this as evidence for an interlanguage speech intelligibility benefit in machine perception and as support for the hypothesis that SSL discrete tokens simulate human speech perception, with the stated applicative goal of accent-robust ASR using only native speech data.","tokens_in":10233,"tokens_out":4176,"duration_ms":34466,"significance":"If the central pattern is robust, the paper would be a useful first demonstration that tokenizer training language can shift ASR robustness for non-native speech without accented training data, and it would provide a concrete, falsifiable prediction derived from the discrete-token-as-perception hypothesis. Strengths include the systematic variation of cluster size, SSL layer, and tokenizer language; the inclusion of both a matched-L1 condition (Japanese/English) and a multi-accent extension; and the use of publicly available corpora. However, the decisive comparisons rest on small single-run WER differences, and the design does not currently separate L1-specific matching from a generic robustness advantage of non-English tokenizers. The significance of the paper depends on closing that gap.","major_comments":[{"comment":"The matched-ISIB claim rests on WER gaps of 0.6–0.7 points for Chinese- and Spanish-accented English (28.6 vs 29.3 and 21.8 vs 22.4), with no confidence intervals, significance tests, or multiple seeds reported. Given that Table 2 shows en–jp differences on native English of –1.1 to +0.2, differences of this size are within plausible training noise, so the paper's conclusion that recognition accuracy was highest for every matched case is not statistically supported.","section":"Section 3.5, Table 5"},{"comment":"The mismatched columns show that Chinese and Spanish tokenizers improve WER over the English tokenizer not only for Chinese- and Spanish-accented English but also for Arabic, Hindi, Korean, and Vietnamese accented English—languages whose L1s are absent from the tokenizer set. This is evidence of a generic non-English-tokenizer robustness effect rather than L1-specific perception. The matched diagonal could be a byproduct of this general effect plus small fluctuations, so the ISIB attribution requires a control such as matching tokenizers on effective cluster usage or entropy, or comparing with a non-linguistic quantization baseline.","section":"Section 3.5, Table 5"},{"comment":"The load-bearing premise is that a k-means tokenizer trained on native X speech is a faithful computational model of a native-X listener's perception. The manuscript adopts this premise from [10] but provides no independent evidence for it; the experiments only show that certain tokenizer languages yield lower WER. To avoid circularity, the authors should either test the perception link directly (e.g., by predicting human intelligibility judgments or by showing that the effect is specific to L1-matched phonology rather than to any out-of-domain tokenizer) or explicitly frame the result as a technical ISIB analog without claiming to validate the perceptual hypothesis.","section":"Section 2.2, Figure 2"},{"comment":"The cluster-quality metrics do not consistently support the proposed mechanism. QE is lower for Japanese-trained k-means only for JE w10 at the 9th layer, and MTER is lower for Japanese-trained k-means only at cluster sizes 100 and 500 for JE w10, not at cluster size 2000 where the main ASR comparison is made. Thus the claim that L1-matched tokenization gives more adequate units for accented speech is only partially corroborated and does not explain the ASR result at the operating cluster size.","section":"Section 3.4, Table 4"}],"minor_comments":[{"comment":"There are typos: 'exracted' should be 'extracted' in Section 2.1, and 'expeted' should be 'expected' in the Figure 1 caption.","section":"Section 2.1, Figure 1"},{"comment":"The recognition examples 'app liciated' and 'app reciated' appear to have spacing issues; please format the examples with monospaced or aligned text for clarity.","section":"Table 3"},{"comment":"The evaluation on L2-ARCTIC is not described in the same detail as ERJ; please specify which speakers and sentences were used and the size of each accent-specific test subset.","section":"Section 3.5"},{"comment":"The paper does not state how many random seeds were used for k-means initialization or ASR training; at minimum, report the seed or the k-means initialization procedure.","section":"Section 3.1"},{"comment":"The reference list is appropriate, but [10] is the authors' own prior work; please clarify in the text which aspects of the perception model are established there and which are newly tested here.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper is within scope for an ASR-focused venue, but for a journal the statistical support and the confound between L1-matching and generic non-English robustness need to be addressed. I also note that the central hypothesis relies heavily on the authors' own prior work [10]; the editorial process should ensure that the novelty and validation of that premise are not overclaimed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Onda et al. report that discrete-token ASR shows the interlanguage speech intelligibility benefit: for Japanese-, Chinese-, and Spanish-accented English, training the k-means tokenizer on the speaker's L1 gives the best WER among tokenizer languages, including the spoken language. That is a genuinely new observation, and the experimental design is clean — swap only the tokenizer training language, keep everything else fixed. The pattern reproduces across two corpora and several L1s, and the QE/MTER metrics give partial independent support. I think the phenomenon is real, but the paper's strongest claims are not supported by the evidence as reported.\n\nThe soft spots are the usual small-scale ones, and they matter here. The decisive margins are tiny: in Table 5, Chinese-matched beats Spanish-mismatched by 0.7 WER, Spanish-matched beats Chinese-matched by 0.6. In Table 2, the matched advantage ranges from 2.4 to 4.7 WER on Japanese-accented English, but on native English the identical en-vs-jp comparison swings between -1.1 and +0.2, which tells you the single-run noise floor is comparable to the effect you are citing. There are no confidence intervals, no significance tests, no seeds, and no release of code or checkpoints. That alone makes the central claim provisional.\n\nThe bigger interpretive problem is the confound the stress-test note identifies. In Table 5, Chinese and Spanish km beat English km on every non-native English test set, including Arabic, Hindi, Korean, and Vietnamese — where the L1 is not in the tokenizer set at all. So most of the benefit of non-English tokenizers is generic robustness, not shared-L1 perception. The paper does not control for cluster entropy or effective usage, so the 'matched' advantage could be the residual after a generic effect rather than evidence for ISIB. The perceptual-simulation premise — that k-means on L1 speech simulates an L1 listener's perception — is inherited from the authors' own prior work and is asserted rather than tested. Section 3.2's conclusion is fine as a description of Table 1, but the Conclusions sentence that says 'best among tested languages, including the spoken language' is accurate only in a narrow sense; the mismatched results weaken the perceptual reading.\n\nWho is this for? People working on accent-robust ASR with limited accented data, and anyone interested in whether discrete units carry perceptual information. The paper deserves a serious referee — it is a reasonable conference submission — but the referee should demand significance testing, multi-seed runs, and an entropy or effective-vocabulary control before the ISIB claim is accepted. I would not desk-reject it.","headline":"Decisive margins are small and confounded by a generic non-English-tokenizer effect, but the core observation is new and worth referee time.","tokens_in":10807,"tokens_out":2249,"would_cite":false,"duration_ms":18932,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training the discrete tokenizer on the speaker's native language, rather than on the target language, improves ASR word error rates on accented English, reproducing a human perceptual advantage.","keywords":["discrete tokens","self-supervised learning","foreign accent","interlanguage speech intelligibility benefit","automatic speech recognition","k-means clustering","accent robustness","native speech only"],"falsifier":"A concrete falsifier would be to train a tokenizer on native speech of the same L1 but with the language-specific phonemes artificially neutralized through resynthesis, and check whether the matched-L1 WER gain disappears; if the gain persists, the effect is generic centroid robustness rather than a simulated native phonological filter.","tokens_in":9737,"feed_emoji":"🎙️","tokens_out":8263,"duration_ms":63403,"temperature":0.7,"pith_summary":"This paper tries to establish that a machine speech recognizer can reproduce the interlanguage speech intelligibility benefit (ISIB), the human tendency to understand accented speech better when the listener shares the speaker's native language. The lever is the discrete tokenizer used to convert speech into a token sequence: train its k-means centroids on native speech of the speaker's language, and the recognizer becomes more accurate for that accent even though the ASR itself is trained only on native English. Experiments with Japanese-, Chinese-, and Spanish-accented English show that the matched-L1 tokenizer yields the best word error rate among the tested tokenizer languages, including English. If the claim is right, accent-robust ASR can be built for language pairs where no accented recordings exist, using only native speech from both languages.","feed_headline":"Match the tokenizer's language to the speaker's accent to cut ASR errors","feed_subtitle":"The tokenizer language, not the target language, drives accented-English recognition, and only native speech data is needed.","key_machinery":"The central object is the discrete tokenizer: k-means clustering applied to hidden-layer features of a HuBERT model, producing an integer sequence from each utterance. Changing the language used to train the k-means centroids changes the set of phonetic categories the tokens can express, and the paper treats this as a model of a native listener's phonological filter. The ASR model is kept fixed and trained on native English, so any performance shift caused by changing the tokenizer language is attributed to the perceptual filter imposed before recognition.","core_discovery":"The central claim is that discrete tokens extracted from a self-supervised speech model carry a listener-like native-language filter, so that tokenizing X-accented English with a tokenizer trained on native X speech improves downstream recognition in the same way ISIB improves human perception. On LibriSpeech960 with 2000 clusters, Japanese-trained tokenization lowered the word error rate on Japanese-accented English by 2.4 to 4.7 points absolute compared with English-trained tokenization, while the English-trained tokenizer remained best for native English. The same matched-L1 advantage appeared for Chinese- and Spanish-accented English on the L2-ARCTIC corpus, and mismatched cases showed Chinese or Spanish tokenizers outperforming the English tokenizer on all non-native English test sets. The paper reads these results as evidence that discrete tokens simulate human speech perception and that ISIB has a computational analogue that can be exploited with only native speech data.","pith_inferences":["An extension the paper leaves implicit is per-speaker tokenizer selection: if the speaker's native language is known at inference time, the tokenizer could be switched on the fly without retraining the ASR.","The paper's accent-similarity explanation for mismatched ISIB could be turned into a predictive rule: pick the tokenizer language that maximizes acoustic similarity to the speaker's accent, estimated from an accent-classification model.","Since the SSL pretraining language was fixed to English in the multi-accent experiments, a natural next test is whether matching both the SSL model and the tokenizer to the speaker's L1 gives a larger gain than matching the tokenizer alone."],"forward_implications":["If the central claim holds, accent-robust ASR for an X-accented Y pair can be built without any accented recordings by training the ASR on native Y speech and the tokenizer on native X speech.","Because mismatched ISIB also occurs, a high-resource language whose accent resembles the speaker's can stand in when the speaker's native language is low-resource or unknown.","The matched-L1 tokenizer advantage strengthens the hypothesis that SSL discrete tokens encode a listener-like perceptual filter.","Cluster size matters: the ISIB gain appears only with sufficiently large codebooks (500 or 2000), suggesting that small codebooks erase the perceptual detail the effect depends on.","The approach changes only the tokenizer, leaving the ASR architecture and training data untouched, so it can be combined with other accent-robustness techniques."],"supporting_citations":[{"why":"Proposed that discrete tokens trained on a different language can simulate foreign accentuation using only native speech, the idea this paper extends to ASR.","marker":"[10]"},{"why":"Defines the interlanguage speech intelligibility benefit, the perceptual phenomenon the paper tries to reproduce in a machine.","marker":"[11]"},{"why":"Provides HuBERT, the self-supervised model whose hidden features are clustered into discrete tokens.","marker":"[15]"},{"why":"Introduced discrete-token-based ASR, the recognition architecture this paper varies by tokenizer language.","marker":"[19]"},{"why":"Supplies LibriSpeech, the native English data used for ASR training and for English k-means clustering.","marker":"[26]"},{"why":"Contributes the ERJ corpus, the Japanese-accented English evaluation set with accent-strength annotations.","marker":"[28]"},{"why":"Supplies L2-ARCTIC, the multi-accent English corpus used to test matched and mismatched ISIB beyond Japanese.","marker":"[37]"}],"fun_headline_variants":["Match tokenizer language to accent for robust ASR","Native-only data trains accent-robust discrete-token ASR","Discrete tokens replicate human accent perception for ASR","Tokenizer's L1 drives accented-English ASR performance","ISIB in machines: tokenizer matching boosts accent ASR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that k-means clusters trained on native speech of a language faithfully simulate how a native speaker of that language perceives speech, so a tokenizer trained on language X is a valid stand-in for an X-speaking listener.","fun_headline_variants_meta":{"raw":{"variants":["Match tokenizer language to accent for robust ASR","Native-only data trains accent-robust discrete-token ASR","Discrete tokens replicate human accent perception for ASR","Tokenizer's L1 drives accented-English ASR performance","ISIB in machines: tokenizer matching boosts accent ASR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1286,"prompt_tokens":913,"completion_tokens":373,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":291}},"tokens_in":529,"tokens_out":373,"duration_ms":3729,"temperature":1.0,"reasoning_tokens":291,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:05:34.886562+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier would be to train a tokenizer on native speech of the same L1 but with the language-specific phonemes artificially neutralized through resynthesis, and check whether the matched-L1 WER gain disappears; if the gain persists, the effect is generic centroid robustness rather than a simulated native phonological filter.","supporting_citations":[{"cited_title":"Foreign accent, comprehensi- bility, and intelligibility in the speech of second language learn- ers,","cited_arxiv_id":null,"evidence_quote":"Proposed that discrete tokens trained on a different language can simulate foreign accentuation using only native speech, the idea this paper extends to ASR."},{"cited_title":"Intelligible, comprehensible, non-native models in ESL/EFL pronunciation teaching,","cited_arxiv_id":null,"evidence_quote":"Defines the interlanguage speech intelligibility benefit, the perceptual phenomenon the paper tries to reproduce in a machine."},{"cited_title":"HuBERT: Self-supervised speech repre- sentation learning by masked prediction of hidden units,","cited_arxiv_id":null,"evidence_quote":"Provides HuBERT, the self-supervised model whose hidden features are clustered into discrete tokens."},{"cited_title":"The interlanguage speech intelligi- bility benefit,","cited_arxiv_id":null,"evidence_quote":"Introduced discrete-token-based ASR, the recognition architecture this paper varies by tokenizer language."},{"cited_title":"Second language speech learning: Theory, findings, and problems,","cited_arxiv_id":null,"evidence_quote":"Supplies LibriSpeech, the native English data used for ASR training and for English k-means clustering."},{"cited_title":"Ex- ploration of efficient end-to-end ASR using discretized input from self-supervised learning,","cited_arxiv_id":null,"evidence_quote":"Contributes the ERJ corpus, the Japanese-accented English evaluation set with accent-strength annotations."},{"cited_title":"Investigation of robust- ness of hubert features from different layers to domain, accent and language variations,","cited_arxiv_id":null,"evidence_quote":"Supplies L2-ARCTIC, the multi-accent English corpus used to test matched and mismatched ISIB beyond Japanese."}],"review_version":1}