{"id":"4b6a4004-518e-4eb3-92df-e4d58c3e55f2","arxiv_id":"2412.15622","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"TouchASP trains a single elastic mixture-of-experts ASR model on 1M hours of partly pseudo-labeled audio and reports SpeechIO CER dropping from 4.98% to 2.45% while adding multi-task perception.","lead":"This technical report describes TouchASP, a speech system combining a new elastic mixture-of-experts model with a million hours of weakly supervised web audio. The authors claim the model trains once, deploys at several sizes, and handles recognition plus language, age, gender, emotion, and sound event tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central eMoE claim rests on an unstated group-sampling schedule and a single run; the N=3/7/15 results may reflect schedule tuning rather than a robust property of the method.","rationale":"The reader's weakest assumption correctly identifies the unstated group-sampling distribution and the lack of per-group training-adequacy analysis as the key risk. My analysis agrees: the Table 4 evidence is a single run with no reported sampling schedule, and the central claim that a single eMoE can match multiple dedicated DeepSeekMoE models requires that all nested groups, especially the small N=3 group, receive sufficient training. Without the schedule or per-group gradient statistics, one cannot tell whether the result follows from the method or from an particular sampling choice. I also note the N=15 result is slightly worse than the dedicated baseline, slightly weakening the claim even under the reported schedule, but this is a secondary point. The reader's CONDITIONAL verdict is appropriate: the concern is concrete and addressable by releasing the sampling schedule and multi-seed experiments, but it does not by itself invalidate the method. Therefore, no change to the reader's verdict is warranted. The proposed test would settle the concern by directly probing the impact of the sampling schedule on the pruned model's accuracy.","tokens_in":11932,"tokens_out":4862,"duration_ms":41953,"concrete_test":"Obtain the training configuration used for Table 4 and compute the number of steps each group (N=3, N=7, N=15) was actually sampled. Then re-run the eMoE training with three different seeds and two alternate sampling schedules (e.g., double and halve the N=3 sampling probability, keeping total step count fixed), and report CER for N=3 and N=15 on the same SpeechIO test sets. If the N=3 CER varies by more than 0.2 absolute CER across schedules, the result is schedule-dependent and the central claim is not robust; if it stays within 0.1 CER, the training-adequacy concern is largely resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that a single eMoE training run matches separately trained DeepSeekMoE models at N=3, N=7, and N=15 (Section 4.2, Table 4) depends on the group-sampling schedule introduced in Section 3.1. The paper states that \"we set different sampling probabilities for different groups, giving smaller probabilities to the previous groups\" but never reports the actual probabilities, the number of steps each group was sampled, or the per-expert gradient counts induced by the nested groups. Without this information, Table 4's single run cannot distinguish a robust property of nested-group training from a hand-tuned schedule that happens to keep the N=3 router well-trained. The concern is load-bearing because if the N=3 (or N=7) group receives only a small fraction of updates, the pruned model's accuracy is not guaranteed by the method; it is an artifact of the schedule. Additionally, the same table shows eMoE at N=15 (CER 3.68) is actually worse than the dedicated DeepSeekMoE-1B (CER 3.62), so the \"comparable\" claim is already loose at the full-size end. The absence of per-group training statistics, seed analysis, or error bars leaves the central \"train once, deploy elastically\" promise supported by only one uncharacterized configuration.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"TouchASP proposes three contributions: (1) an elastic mixture-of-experts (eMoE) architecture that nests expert groups and prunes them at inference time to support different deployment sizes from a single training run; (2) an unsupervised web-data pipeline that yields roughly 650k hours of usable speech data from 1.26M raw hours, combined with other sources into a 1M-hour training set; and (3) a multi-task 'Automatic Speech Perception' (ASP) framework that adds language, dialect, age, gender, emotion, and sound-event prediction to ASR. The paper reports that eMoE trained once at 1B scale matches separately trained DeepSeekMoE models at N=3 and N=15 experts (Table 4), improves SpeechIO CER from 4.98% to 2.45% (Table 5), and shows competitive or strong results on multilingual, dialect, LID, gender/age, SER, and SED benchmarks (Tables 7-9). The work is framed as a technical report aimed at practical, elastic deployment of speech perception models.","tokens_in":12227,"tokens_out":4308,"duration_ms":36798,"significance":"If the central eMoE claim holds, the paper offers a practically valuable result: a single training run could replace multiple differently sized models, reducing training cost and deployment complexity. The data pipeline, with its explicit retention rate and external SpeechIO evaluation, is a useful reference for large-scale weakly supervised speech data construction. The multi-task ASP formulation is also a reasonable extension of the ASR-only paradigm, and the use of external human-annotated benchmarks (SpeechIO, CommonVoice, MELD, ESC-50) is a strength. However, the current evidence for the eMoE claim rests on a single uncharacterized training configuration, and several evaluation protocols contain confounds or test-set overlap with training data, so the significance is conditional on these issues being resolved with additional experiments and transparent reporting.","major_comments":[{"comment":"The central claim that a single eMoE training run matches separately trained DeepSeekMoE models at N=3, N=7, and N=15 is supported only by a single run with no variance or seed analysis, and the group-sampling schedule described in Section 3.1 is never specified. The paper says 'we set different sampling probabilities for different groups' but does not report these probabilities, the resulting per-group step counts, or the per-expert gradient counts induced by the nested groups. Since smaller groups are deliberately sampled less often, the N=3 result (CER 3.95 vs. DeepSeekMoE-350M 3.99) could reflect a hand-tuned schedule rather than a robust property of nested-group training. Additionally, the full-size comparison is already loose: eMoE at N=15 (CER 3.68) is worse than the dedicated DeepSeekMoE-1B (CER 3.62). To make the 'train once, deploy elastically' claim load-bearing, the authors should report the sampling distribution, per-group training statistics, and results across multiple seeds or at least provide error bars.","section":"Section 4.2, Table 4"},{"comment":"The data-scaling experiment compares 160K, 500K, and 1M hours of data by training for 100K steps in each condition. This holds the number of optimizer steps constant but changes the number of epochs per dataset, so the CER improvement (3.8 to 3.09) conflates data volume with training repetition and total unique tokens seen. If the batch size is fixed, the 160K-hour model sees each utterance more times than the 1M-hour model, making the comparison a compute-matched rather than a data-matched scaling study. The authors should either match total samples seen, report the number of epochs, or control for the effective training steps to isolate the effect of data volume.","section":"Section 4.1, Table 2"},{"comment":"The SER evaluation on MELD uses a test set that appears in the training data: Table 6 lists 'meld' among the SER training sources, and Table 9 reports TouchASP accuracy on the meld test set. Similarly, the gender test set (Aishell1, kws, kaggle) and age test set (kws, kaggle) overlap with the sources listed in Table 6 as training data (Aishell1, kaggle, wake-up data with gender/age labels). As a result, the reported accuracies are not directly comparable to baselines that did not train on these evaluation sets, and the claims of 'excellent recognition capability' for gender, age, and emotion are not supported by held-out evaluation. The authors should re-evaluate on datasets or splits that are disjoint from training, or clearly state the overlap and discount the comparisons accordingly.","section":"Section 4.3, Table 9 and Table 6"},{"comment":"The text states that 'TouchASP significantly outperforms its counterparts, in most test sets with the exception of Librispeech,' but Table 7 does not support this. On wenetspeech test_net, TouchASP (5.52) is worse than eMoE-1B (5.32) and Seed-ASR (4.66); on test_meeting, TouchASP (5.94) is worse than eMoE-1B (5.42); and on Librispeech test_other, TouchASP (4.38) is worse than eMoE-1B (4.28). The only clear gains are on Cantonese and dialect sets, where several baselines have missing entries (dashes). The narrative should be rephrased to reflect the actual pattern: TouchASP improves dialect recognition while regressing on Mandarin and English relative to its own eMoE-1B base, which the paper does acknowledge later but the earlier claim is overstated.","section":"Section 4.3, Table 7 and accompanying text"}],"minor_comments":[{"comment":"The group indexing is inconsistent: the text defines groups g1 through gZ, but later refers to 'group 0' and 'g0' without defining it. Please clarify whether the smallest group is g0 or g1, and adjust the notation consistently.","section":"Section 3.1"},{"comment":"There is an off-by-one ambiguity in the group definitions: 'g1 = (e0, e1, ..., eG)' suggests G+1 experts, while the text says the smallest group contains G experts. Also, '2^{Z-1} * G = M' should state whether M is the total number of non-shared experts or includes shared experts, since earlier M = S + N is defined.","section":"Section 3.1"},{"comment":"The entry 'N=sample[3, 7, 15]' for eMoE training is unclear: it should specify whether sampling is over the group indices, the number of active experts, or the set of nested group sizes, and how K=3 interacts with the smallest group of size 3.","section":"Section 4.2, Table 4"},{"comment":"The metric name is inconsistent: Section 4.1 and Table 2 use 'CER', but the text says 'evaluated using WER' and later 'weighted average WER' for SpeechIO. Please standardize the terminology to avoid confusing readers.","section":"Section 4.1 and Table 2"},{"comment":"Several entries in Table 7 are missing (dashes) for Seed-ASR and eMoE-1B on the Cantonese and dialect test sets, which prevents a direct comparison of TouchASP against those baselines on the exact same data. Reporting the evaluation coverage or adding the missing numbers would strengthen the comparison.","section":"Section 4.3, Table 7"},{"comment":"The filtering rule 'filter out data with WER greater than 10 and PER greater than 5' is ambiguous: it is not clear whether data are removed when both conditions hold or when either condition holds, nor how WER/PER are computed between the two ASR outputs when the reference is absent.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a technical report with a potentially valuable elasticity idea, but the experimental evidence for its central claim is currently too thin: the eMoE comparison is a single run with an unreported sampling schedule, the data-scaling experiment is confounded by training steps, and several benchmark evaluations have train/test overlap. These issues are fixable with additional experiments, held-out evaluation, and more transparent reporting, so I recommend major revision rather than rejection. The authors should also be encouraged to clarify the relationship between this report and the WeNet/U2++ line of work, as several baselines come from their own previous papers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The eMoE mechanism is genuinely new, and the data pipeline is solid, but the load-bearing 'train once, deploy elastically' claim rests on one run and an unreported sampling schedule—so treat Table 4 as a feasibility demonstration, not a settled result. The core idea: nested groups of experts with per-group routers, trained by dynamically sampling groups, so you can prune to N=3, 7, or 15 experts at inference. That is a real combination of slimmable networks and DeepSeekMoE, and it is the strongest part of the paper. The data pipeline (WhisperX + Paraformer, WER/PER filtering) is less novel but simple and effective; the reported CER improvement from 3.8 to 3.09 on SpeechIO going from 160K to 1M hours is plausible, and the 650K-hour filtered corpus is a useful resource.\n\nThe soft spot is exactly where you'd expect: Table 4. The paper mentions smaller sampling probabilities for early groups and larger for later groups, but never reports the actual probabilities, the per-group step counts, or per-expert gradient statistics. Without those, a single run cannot separate a robust property of nested-group training from a schedule hand-tuned to keep the N=3 group accurate. It does not help that eMoE at N=15 (CER 3.68) is actually worse than the dedicated DeepSeekMoE-1B (3.62), so 'comparable' is already loose at the full-size end. No error bars or seeds either.\n\nTwo smaller issues. The data scaling experiment trains all models for 100K steps, so the comparison conflates data volume with data composition; it is not a clean scaling curve. And the general perception section overstates its case: 'most test sets' means it wins on the dialect corpora, but it loses on Librispeech and WenetSpeech against Seed-ASR, and the emotion/SED numbers trail the cited SOTA—acknowledged in the limitations but not examined.\n\nOverall, this deserves a serious referee, not a desk reject. The eMoE idea is worth engaging with, and the data pipeline is reproducible in principle. Before publication the authors should report the sampling schedule, add per-group update counts, run a few seeds, and temper the perception claims. For your own work, cite it as prior art for elastic MoE if you're in that area, but don't build on the exact numbers.","headline":"Genuinely new eMoE idea and a solid data pipeline, but the 'train once, deploy elastically' claim is riding on a single run and an unreported sampling schedule.","tokens_in":12770,"tokens_out":4933,"would_cite":true,"duration_ms":38947,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One trained speech model, pruned at inference, matches per-size models.","keywords":["elastic mixture of experts","automatic speech recognition","automatic speech perception","weak supervision","data pipeline","multilingual ASR","sound event detection","edge deployment"],"falsifier":"Train eMoE and a dedicated small MoE on the same data, log how often each nested group is selected, and compare CER on a held-out SpeechIO-style set for the smallest group size: if the smallest eMoE configuration underperforms the dedicated small model by more than the paper's observed margin of about 0.04 percentage points on 500K hours, the elasticity mechanism is not supported. A cheaper check is to inspect the training curve of the smallest group's router loss and see whether it converges to the same value as the full router.","tokens_in":11740,"feed_emoji":"🎧","tokens_out":11321,"duration_ms":80358,"temperature":0.7,"pith_summary":"The paper's central claim is that a speech perception model can be trained once and then served at many different parameter sizes without retraining. It introduces eMoE, a mixture-of-experts architecture where experts are arranged in nested groups and a group is selected randomly during training; at inference, choosing a smaller group removes parameters and computation. The same system also uses a one-million-hour weakly labelled corpus built by a two-transcriber consistency filter, and a multi-task decoder that outputs transcription, language, gender, age, emotion, and sound-event labels. On the SpeechIO Mandarin benchmarks, the single eMoE model reaches 2.45% character error rate at full size and 2.67% at the smallest pruned size, down from 4.98% for the earlier dense baseline. If the elasticity result holds, large speech models become deployable on edge devices at no extra training cost.","feed_headline":"One trained speech model, pruned at inference, matches per-size models","feed_subtitle":"Nested expert groups make one speech model swap between edge and cloud sizes without retraining, cutting SpeechIO CER from 4.98% to 2.45%.","key_machinery":"The load-bearing mechanism is a nested power-of-two expert-grouping scheme. In each MoE layer, the $N$ independent experts are arranged so that group $g_1$ has $G$ experts, group $g_2$ has $2G$, and in general $g_i$ has $2^{i-1}G$, up to the full set of $M$ experts, with shared experts always present. Each group gets its own router, and training samples a group with probabilities that favour larger groups, so the larger configurations receive enough updates. Because each group is nested inside the next, parameters active in a small configuration are always part of every larger configuration; at inference, an application merely selects a group, giving an instantaneous accuracy-versus-parameter tradeoff. A second mechanism is the data-validation pipeline: two independently produced transcriptions of each web-audio clip are compared, and clips with word-error rate above 10 or phone-error rate above 5 are discarded, yielding 650K usable hours from 1.26M raw hours.","core_discovery":"The paper claims that one eMoE model trained once produces inference configurations that are comparable to separately trained models of each configuration. Concretely, with 500K hours of data, a single model with $N=3$, $N=7$, or $N=15$ routed experts achieves 3.95%, 3.71%, and 3.68% CER respectively on SpeechIO, against 3.99% for a dedicated smaller model and 3.62% for a dedicated larger model at the two endpoints. With one million hours of data the same trained weights give 2.67% CER at the smallest configuration and 2.45% at the full configuration, and the multi-task extension retains competitive performance on multilingual recognition, language, gender, age, emotion, and sound-event tasks. The paper presents this as evidence that 'train once, deploy elastically' is a viable alternative to training a separate model per device tier.","pith_inferences":["The paper does not report the group sampling distribution or the number of updates each nested router receives; if a re-implementation samples small groups too rarely, the smallest deployment size would underperform a dedicated small model, so tracking per-group update counts is a testable design rule.","The elasticity claim is demonstrated with a single model run on one benchmark family; evaluating eMoE across languages, domains, and multiple seeds, and comparing every intermediate size against dedicated models, would strengthen it into a general recipe.","The 51.6% retention rate of the data pipeline implies that the discarded half of the raw audio may still contain learnable signal, and a self-training or noise-tolerant objective trained on all raw audio could potentially beat the cleaned-only result.","Independent routers for each nested group may make small configurations easier to train, since a router only ranks experts inside its own group; this points to a design where routing heads are explicitly conditioned on the target deployment size."],"forward_implications":["A single TouchASP model can serve edge, mobile, and cloud deployments by switching expert groups, with no fine-tuning between sizes.","Scaling the weakly labelled web corpus from 160K to 1M hours lowers SpeechIO CER from 3.8% to 3.09% under the same architecture, so the two-transcriber consistency filter preserves enough quality to make data scale beneficial.","The multi-task token format lets one decoder handle transcription, language identification, gender and age classification, emotion recognition, and sound-event detection, removing the need for separate task-specific models.","At full size the model is competitive with strong published and commercial speech systems on the SpeechIO leaderboard, reaching 2.45% CER on that benchmark."],"supporting_citations":[{"why":"Supplies the fine-grained expert segmentation and shared-expert isolation that eMoE reorganizes into nested groups.","marker":"[4]"},{"why":"Provides the train-once, adjust-width-at-inference idea that motivates pruning expert groups instead of retraining.","marker":"[24]"},{"why":"Is the VAD/ASR engine used in the first pass of the data creation pipeline.","marker":"[25]"},{"why":"Is the copilot ASR engine whose independent transcription is compared to filter low-quality audio.","marker":"[26]"},{"why":"Is the MoE architecture used to measure the effect of data scaling and provides the baseline for the pipeline experiments.","marker":"[10]"},{"why":"Is the multi-task weak-supervision model whose task-token decoding format inspires the general perception framework.","marker":"[12]"},{"why":"Is a large-scale speech recognition model that TouchASP compares against in the multilingual evaluation.","marker":"[20]"},{"why":"Is the multi-capability speech foundation model that TouchASP compares against for perception tasks.","marker":"[21]"}],"fun_headline_variants":["Train once, deploy any size: TouchASP elastic speech AI","One model, every device: speech perception without retraining","Elastic eMoE: one model, infinite deployment sizes","TouchASP: one training, any scale, half the CER","Elastic speech model: scales from edge to cloud after one training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim collapses if a nested expert group sampled only occasionally during training does not learn to route and predict as well as a dedicated model trained exclusively at that size; the paper's evidence for this is a single comparison run, and the group sampling probabilities are not reported.","fun_headline_variants_meta":{"raw":{"variants":["Train once, deploy any size: TouchASP elastic speech AI","One model, every device: speech perception without retraining","Elastic eMoE: one model, infinite deployment sizes","TouchASP: one training, any scale, half the CER","Elastic speech model: scales from edge to cloud after one training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000953,"raw_usage":{"total_tokens":4063,"prompt_tokens":939,"completion_tokens":3124,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":3038}},"tokens_in":555,"tokens_out":3124,"duration_ms":20950,"temperature":1.0,"reasoning_tokens":3038,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:14:38.805045+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train eMoE and a dedicated small MoE on the same data, log how often each nested group is selected, and compare CER on a held-out SpeechIO-style set for the smallest group size: if the smallest eMoE configuration underperforms the dedicated small model by more than the paper's observed margin of about 0.04 percentage points on 500K hours, the elasticity mechanism is not supported. A cheaper check is to inspect the training curve of the smallest group's router loss and see whether it converges to the same value as the full router.","supporting_citations":[{"cited_title":"Whisperx: Time-accurate speech transcription of long-form audio","cited_arxiv_id":null,"evidence_quote":"Is the VAD/ASR engine used in the first pass of the data creation pipeline."}],"review_version":1}