{"id":"e3177616-4650-4685-bf62-569a26877eb7","arxiv_id":"2607.06054","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":3,"one_line_summary":"A Taiwan-specific tokenizer, language model, and bridge to a reused acoustic stack cut code-switching TTS CER from 11.45% to 4.81%, with 65.6% listener preference.","lead":"This paper builds a Taiwanese-Mandarin text-to-speech system by training a Taiwan-specific tokenizer, a Traditional-Chinese language model, and a bridge to a reused acoustic stack. It cuts character error rate from 11.45% to 4.81% on code-switching speech, showing that localizing the text frontend matters more than retraining audio components.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The 25.2% frontend-isolated CER gap hinges on Breeze-ASR-25 being an unbiased evaluator across frontends, but this is assumed rather than verified, and the human study does not cover the critical fine-tuned-baseline comparison.","rationale":"The reader correctly identified the most load-bearing concern: the ASR proxy metric's fairness across frontends is assumed but not verified, and the human study does not cover the fine-tuned baseline comparison that isolates the frontend effect. I agree this is the softest spot in the argument. The concern is not that the results are wrong — the controlled ablation design is sound, the tokenizer comparison is thorough, and the paper is transparent about limitations — but that the specific 25.2% number, which is the paper's most precise claim about frontend contribution, depends on an evaluator whose neutrality across the two compared systems is untested. The paper's own discussion (Section IX) acknowledges the test set is purpose-built for code-switching stress, which is honest, but the headline claim does not qualify the 25.2% as specific to this test set and this ASR. The CONDITIONAL verdict is appropriate: the claim is plausible and well-motivated, but one concrete verification step (a second ASR or human transcription of the fine-tuned baseline comparison) would substantially strengthen or weaken it. I do not think the concern rises to REJECT because the overall experimental design is reasonable, the ablation logic is correct, and the failure cases are honestly reported. But the gap between 'reasonable design' and 'verified claim' is exactly what the CONDITIONAL verdict captures.","tokens_in":14488,"tokens_out":5540,"duration_ms":426294,"concrete_test":"Run a second, independent ASR model (e.g., Whisper-large-v3 or a non-Taiwanese Mandarin ASR) on the same synthesized speech from the fine-tuned baseline and BlueMagpie-TTS across the 1,000-sentence test set. Recompute CER for both systems under this second ASR. If the relative CER gap between the fine-tuned baseline and BlueMagpie-TTS shrinks to below ~10% (from the reported 25.2%), Breeze-ASR-25 is differentially favoring BlueMagpie-TTS outputs and the frontend-isolated improvement is inflated. If the gap holds within a few percentage points, the frontend effect is robust to evaluator choice. As a cheaper alternative, have 2–3 human transcribers transcribe 100 randomly sampled sentences from each system and compute human CER; compare the human-measured gap to the ASR-measured 25.2%.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — that the 25.2% CER reduction from 6.43% to 4.81% comes from the Taiwan-localized frontend, not from voice adaptation — rests entirely on the synthesize-then-recognize loop using Breeze-ASR-25 (Section VII-A). The paper defends fairness by noting the same ASR is used for all systems. But same-ASR does not guarantee unbiased comparison if the ASR has differential accuracy across acoustic regimes. Breeze-ASR-25 is itself a Taiwan-localized model trained on Taiwanese Mandarin and code-switching speech [29]. If BlueMagpie-TTS produces speech whose accent, prosody, or code-switching timing more closely matches Breeze-ASR-25's training distribution, the ASR may transcribe BlueMagpie-TTS outputs more accurately than baseline outputs even if the actual intelligibility difference is smaller. This is differential evaluator bias, not a constant offset that same-ASR controls for. The human preference study (Section VII-D) partially addresses this, but it only compares BlueMagpie-TTS against the zero-shot base (11.45% CER), not against the fine-tuned baseline (6.43% CER) that isolates the frontend effect. So the 25.2% claim — the most specific and load-bearing number in the paper — has no human validation and relies on an unverified assumption about ASR neutrality. The paper itself acknowledges 'the ASR makes its own errors' but does not test whether those errors are distributed symmetrically across the two systems whose CER difference is the headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper presents BlueMagpie-TTS, a Taiwanese-accent code-switching TTS system built from three components: (1) PangolinTokenizer, a byte-level BPE tokenizer trained on Taiwan-context data; (2) Barbet, a 1.09B-parameter Traditional-Chinese language model serving as the text-semantic frontend; and (3) a reused VoxCPM2 acoustic stack connected via a learned bridge module. The central empirical claim is that replacing the text-semantic frontend with the Taiwan-localized stack, while keeping the acoustic stack fixed, lowers CER from 11.45% (zero-shot base) to 4.81%, and remains 25.2% below a fine-tuned baseline (6.43%) that shares the same acoustic stack and target-voice fine-tuning. The paper also reports tokenizer efficiency across eight baselines on a new benchmark (PangolinBench), standalone LM quality on TAIDE-14, and a human preference study showing 65.6% majority-vote preference for BlueMagpie-TTS over the zero-shot base.","tokens_in":15360,"tokens_out":1374,"duration_ms":224563,"significance":"The paper addresses a real gap: off-the-shelf TTS systems are poorly adapted to Taiwanese Mandarin and intra-sentential Chinese-English code-switching. The controlled comparison design—zero-shot base, fine-tuned base, and BlueMagpie-TTS sharing the same acoustic stack and fine-tuning—is well constructed and isolates the frontend effect more cleanly than a simple before-after comparison. The tokenizer benchmark with explicit pass criteria and the deterministic streaming BPE trainer are practical contributions. The human preference study with ten listeners and a sign test provides convergent evidence alongside the ASR-based metric. The recipe of reusing a pretrained acoustic stack and training only a bridge plus short fine-tuning is cost-effective and reproducible. The main concern is whether the load-bearing 25.2% claim is fully validated given the evaluation methodology.","major_comments":[{"comment":"§VII-A, §VII-C: The 25.2% frontend-isolated CER reduction (6.43% → 4.81%) rests on the synthesize-then-recognize loop using Breeze-ASR-25, which is itself a Taiwan-localized model. The paper states that using the same ASR for all systems keeps the comparison fair, but same-ASR does not guarantee unbiased comparison if the ASR has differential accuracy across acoustic regimes. If BlueMagpie-TTS produces speech whose accent, prosody, or code-switching timing more closely matches Breeze-ASR-25's training distribution, the ASR may transcribe BlueMagpie-TTS outputs more accurately than baseline outputs even if the actual intelligibility difference is smaller. The human preference study (§VII-D) partially addresses evaluator bias, but it only compares BlueMagpie-TTS against the zero-shot base (11.45% CER), not against the fine-tuned baseline (6.43% CER) that isolates the frontend effect. Thus,","section":null},{"comment":"§VII-B: The 1,000-sentence test set is internal and not publicly available. The paper acknowledges this ('so the numbers support relative comparison'), but the central CER claim would be substantially strengthened by either releasing the test set, reporting results on an external benchmark, or at minimum providing summary statistics (sentence length distribution, code-switching rate, English span types) that allow readers to assess generalizability. Without this, it is unclear whether the 25.2% gap would hold on other Taiwan-localized test sets. This is load-bearing because the headline result is specific to this test set.","section":null}],"minor_comments":[{"comment":"Abstract (standalone): says 'eight tokenizers' in one place and 'nine tokenizers' in another. The body and Table I show eight baselines plus PangolinTokenizer (nine total). The abstract should say 'eight' consistently when referring to baselines, or 'nine' when including PangolinTokenizer.","section":null},{"comment":"§IV-E, Eq. (1): The vocabulary efficiency score is described as 'an empirical formula... not a derived quantity.' The formula (1/(tokens per character)) / (vocabulary size / 100000) is dimensionally a rate scaled by an arbitrary constant. A brief justification for the 100000 scaling or a reference to prior work using this metric would help readers interpret the scores in Table II.","section":null},{"comment":"Table I: The caption says 'BEST IN BOLD, SECOND-BEST UNDERLINED' but the formatting of bold and underline is not visible in the text rendering. Please verify these are correctly marked in the final format.","section":null},{"comment":"§V-B, Table V: The model has 28 decoder layers with a 4-layer cycle pattern (global, sliding-window, Mamba). 28 is not divisible by 4. Please clarify how the pattern wraps or whether the last cycle is truncated.","section":null},{"comment":"§VI-A: The bridge module projects Barbet's 1536-dimensional hidden states into the acoustic stack's input space, but the target dimension is not specified. Please state the acoustic stack's expected input dimension so readers can assess the bridge's capacity.","section":null},{"comment":"§VII-E: The xRT (inverse real-time factor) is reported as median 4.75 and maximum 5.29, but the hardware is not specified. Please state the GPU or accelerator used, as this affects reproducibility of the speed claim.","section":null},{"comment":"§VIII: The paper acknowledges two limitations (tokenizer-frontend coupling, short test sentences not exercising 262K context). These are appropriate disclosures. Consider also noting the single-speaker evaluation as a limitation, since all target-voice fine-tuning uses one voice.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The reader's concern about ASR evaluator bias is the right question to ask, and the paper would benefit from addressing it. However, I assess this as a minor revision rather than major because: (1) the human preference study provides convergent evidence for the overall system quality, even though it does not cover the fine-tuned baseline comparison; (2) the controlled experimental design (same acoustic stack, same fine-tuning, same ASR, same generation parameters) is sound and the paper is transparent about the proxy nature of the metric; (3) the concern can be addressed by adding discussion and ideally a supplementary human comparison, which is within scope of a revision. The paper does not claim the 25.2% gap is a universal constant; it reports it on a specific test set with clear framing. The tokenizer and LM contributions are independently verifiable and solid."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for the careful reading and the constructive assessment. The two major comments both concern the strength of evidence for the 25.2% frontend-isolated CER reduction: (1) potential ASR evaluator bias from using Breeze-ASR-25, and (2) the internal, unreleased test set. We address each below and commit to concrete revisions for both.","responses":[{"response":"The referee raises a legitimate methodological concern. We agree that using the same ASR for all systems controls for absolute ASR difficulty but does not, by itself, rule out differential accuracy across acoustic regimes. If Breeze-ASR-25 is systematically better at transcribing Taiwanese-accented speech than other accents, and if BlueMagpie-TTS produces more Taiwanese-accented output, the ASR could favor our system for reasons unrelated to intelligibility. We acknowledge this as a genuine limitation of the synthesize-then-recognize methodology. We will take two steps in the revision. First, we will add an explicit discussion of this confound in Section VII-A, stating that same-ASR controls for absolute difficulty but not for differential accuracy, and noting the direction of the potential bias. Second, we will extend the human preference study to include the fine-tuned baseline (6.43% CER) as a third comparison arm, so that the frontend-isolated gap is validated by human listeners and not only by the ASR loop. We note that the existing human preference study already shows a strong effect (65.6% preference, p < 10^{-30}) for the zero-shot comparison, and the qualitative error analysis in Section VII-E (the base tends to drop or blur English spans, which the ASR then mis-recognizes) provides a mechanism that is not purely accent-matching: the base system fails to render English spans at all, which no ASR could transcribe correctly regardless of accent bias. Still, the referee's point stands for the 25.2% claim specifically, and the human study extension is the right fix.","revision_made":"yes","referee_comment":"§VII-A, §VII-C: The 25.2% frontend-isolated CER reduction rests on the synthesize-then-recognize loop using Breeze-ASR-25, a Taiwan-localized model. Same-ASR does not guarantee unbiased comparison if the ASR has differential accuracy across acoustic regimes. If BlueMagpie-TTS produces speech whose accent, prosody, or code-switching timing more closely matches Breeze-ASR-25's training distribution, the ASR may transcribe BlueMagpie-TTS outputs more accurately than baseline outputs even if the actual intelligibility difference is smaller. The human preference study only compares BlueMagpie-TTS against the zero-shot base, not against the fine-tuned baseline that isolates the frontend effect."},{"response":"This is a fair point. The headline result is specific to this test set, and without summary statistics or public availability, readers cannot assess generalizability. We will address this in two ways. First, we will add summary statistics for the test set in Section VII-B: sentence length distribution (mean, median, range), code-switching rate (fraction of sentences with at least one English span), and the breakdown of English span types (full words, abbreviations/acronyms, numbers/units, proper nouns), which we already categorize but do not currently report as counts. Second, we will release the test set publicly upon acceptance, subject to the licensing constraints of the source materials. We cannot report results on an existing external benchmark because, to our knowledge, there is no publicly available Taiwanese-Mandarin code-switching TTS benchmark with reference transcriptions. If the referee is aware of one, we would welcome the pointer. We will state the release plan explicitly in the revised manuscript.","revision_made":"yes","referee_comment":"§VII-B: The 1,000-sentence test set is internal and not publicly available. The central CER claim would be strengthened by releasing the test set, reporting on an external benchmark, or providing summary statistics (sentence length distribution, code-switching rate, English span types). Without this, it is unclear whether the 25.2% gap would hold on other Taiwan-localized test sets."}],"tokens_in":14333,"tokens_out":910,"duration_ms":139620,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The main thing to know: this paper shows that for Taiwanese-accented code-switching TTS, localizing the text frontend (tokenizer + LM) accounts for most of the quality gain, while the acoustic stack can be reused frozen. That is a useful practical finding for anyone adapting TTS to a new locale without retraining acoustic models from scratch. The controlled ablation in Table IX is the core evidence: zero-shot base at 11.45% CER, fine-tuned base at 6.43%, BlueMagpie-TTS at 4.81%, with the fine-tuned baseline sharing the same acoustic stack and target-voice data. The 25.2% frontend-isolated reduction is the headline number and it is defensible as a frontend effect. The tokenizer comparison (Table I) is thorough — eight baselines, clear Pareto view, 100% round-trip accuracy. Barbet's standalone quality on TAIDE-14 is reported honestly with bits-per-byte normalization across tokenizers. The bridge module design (zero-initialized SwiGLU, two-stage training) is clean engineering. The paper also reports failure cases plainly, which I appreciate. The stress-test concern about ASR bias is real but probably overstated. Breeze-ASR-25 is Taiwan-localized, so if it systematically favored BlueMagpie-TTS outputs, that would be a problem. But the paper reports specific failure cases where the ASR mis-transcribes BlueMagpie-TTS outputs (e.g., 'LLM' → 'LOL and'), which suggests the ASR is not just rubber-stamping the system. The human preference study covers the zero-shot base, not the fine-tuned baseline — that gap is legitimate and should be addressed in revision. But the CER gap between fine-tuned base and BlueMagpie-TTS is 1.62 percentage points on 1,000 sentences, which is large enough that ASR bias alone is unlikely to explain it. The bigger soft spots: no code or data released, the test set is internal and non-public, and the tokenizer and frontend contributions are coupled (the paper acknowledges this). The abstract says 'nine tokenizers' in one place and 'eight' in another — minor but should be fixed. This paper is for speech synthesis researchers and practitioners working on locale adaptation, especially for code-switching scenarios. It deserves a serious referee. The central claim holds up, the ablation is well-designed, and the recipe is concrete enough to be useful. I would accept for peer review and ask for: test set release, human study extended to the fine-tuned baseline, and the abstract inconsistency fixed.","headline":"Controlled ablation shows localizing the text frontend (tokenizer + LM) yields most of the TTS quality gain for Taiwanese-accented code-switching speech, while the acoustic stack can be reused frozen — a practical finding for adapting TTS to under-served locales.","tokens_in":15580,"tokens_out":636,"would_cite":true,"duration_ms":97190,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Taiwan-localized text frontend cuts TTS errors 58% in code-switching speech","keywords":[],"falsifier":"The frontend-dominance claim would be weakened if, when the same Taiwan-localized tokenizer and language model are attached to a different acoustic stack (not VoxCPM2), the CER reduction fails to replicate—suggesting the gain depends on an interaction between Barbet and VoxCPM2 specifically rather than being a general frontend effect. The claim would also be challenged if an independent ASR evaluator (not Breeze-ASR-25) produced a substantially smaller CER gap between BlueMagpie-TTS and the fine-tuned baseline, indicating the measured gap partly reflects ASR bias rather than true intelligible","tokens_in":14742,"feed_emoji":"🗣️","tokens_out":1357,"duration_ms":120975,"temperature":0.7,"pith_summary":"The paper argues that the dominant failure mode in off-the-shelf TTS for Taiwanese Mandarin—wrong accent, fragmented text, and garbled Chinese-English code-switching—stems not from the acoustic rendering engine but from the text side: the tokenizer and language model frontend that plan what to say and how to say it. To test this claim, the authors build a three-part stack. First, PangolinTokenizer, a byte-level BPE tokenizer trained on Taiwan-context data, achieves the lowest token rate (0.485 tokens/character) among nine compared tokenizers while using the smallest vocabulary. Second, Barbet, a billion-parameter Traditional-Chinese language model trained on that tokenizer, serves as the text-semantic frontend and ranks first among comparable public models on a 14-task evaluation. Third, BlueMagpie-TTS attaches Barbet to the pretrained acoustic stack of VoxCPM2 through a learned bridge module, keeping the acoustic stack fixed. On a 1,000-sentence Taiwan-localized test set, this stack lowers character error rate from 11.45% to 4.81%. Crucially, the authors isolate the frontend's contribution by comparing against a baseline that fine-tunes the same acoustic stack to the same target voice: BlueMagpie-TTS stays 25.2% below that baseline, and since the two systems share the same acoustic stack and the same voice fine-tuning, differing only in the frontend and its bridge, the remaining gain is attributable to the Taiwan-localized text side. A blind listening study with ten native speakers on 500 sentences confirms the intelligibility result: 65.6% of majority votes prefer BlueMagpie-TTS.","feed_headline":"Taiwan-localized text frontend cuts TTS errors 58%","feed_subtitle":"Swapping just the tokenizer and language model—not the acoustic engine—fixes code-switching speech. The gain survives voice fine-tuning, so","key_machinery":"The bridge module is the central mechanism that makes the frontend-swap architecture work. It is a lightweight component—an RMSNorm, a linear projection, and one residual SwiGLU block with zero-initialized output—that maps the hidden states of Barbet (1,536-dimensional, in its own representation space) into the input space expected by the VoxCPM2 acoustic stack. The bridge is trained in two stages: first, both pretrained parts are frozen and the bridge learns to align the two spaces via hidden-space distillation; then the full assembled model is jointly fine-tuned on target-voice speech. Because the bridge is the only component learned from scratch, the recipe is cheap to apply to new target","core_discovery":"The central finding is that when a strong acoustic stack is reused, a substantial portion of accent and code-switching quality in TTS is determined by the text-semantic frontend—the tokenizer and language model that plan the utterance—not by the acoustic renderer. By replacing only the frontend with a Taiwan-localized tokenizer (PangolinTokenizer) and language model (Barbet), while keeping the acoustic stack fixed and controlling for voice fine-tuning, the system achieves a 25.2% further CER reduction beyond voice adaptation alone. This means the switch-point quality at Chinese-English boundaries is largely a frontend planning problem, not an acoustic rendering problem.","pith_inferences":["The paper does not separate the tokenizer's contribution from the language model's contribution to the CER reduction, since Barbet is trained on PangolinTokenizer and the two are coupled. An ablation that swaps only the tokenizer or only the language model would clarify whether token efficiency or language-model specialization drives the gain.","The synthesize-then-recognize evaluation uses Breeze-ASR-25, a Taiwanese-Mandarin ASR model. If this ASR was trained on data with tokenization patterns similar to PangolinTokenizer's output, it could systematically favor BlueMagpie-TTS, inflating the CER gap. The human preference study mitigates this but only compares against the zero-shot base, not the fine-tuned baseline that isolates the fronte","The 1,000-sentence test set is internal and locally focused by design, so the reported CER numbers test worst-case code-switching scenarios rather than average-case performance. The absolute numbers may not generalize to broader usage distributions."],"forward_implications":["If the frontend-dominance claim generalizes, TTS systems for other under-represented language varieties could be localized by training a tokenizer and language model on cheap text data, then bridging to an existing acoustic stack, avoiding the cost of training or retraining acoustic models from scratch.","The token efficiency of the frontend propagates end-to-end: shorter token sequences mean lower inference cost and less KV-cache pressure, which matters for real-time and long-form synthesis.","The finding that short English abbreviations inside Mandarin matrices remain the hardest case suggests that the remaining code-switching failures are a data coverage problem in the frontend, not an architectural limitation of the acoustic stack.","The controlled comparison design—same acoustic stack, same voice fine-tuning, differing only in frontend—provides a template for isolating frontend contributions in other modular TTS architectures."],"fun_headline_variants":["Taiwan-trained frontend halves TTS errors without touching the acoustic model","Code-switching TTS failures traced to the text frontend, not the acoustic renderer","Swapping tokenizer and language model alone cuts TTS CER from 11.45% to 4.81%","Fixed acoustic stack, new Taiwan frontend: 63.9% relative WER reduction","Frontend planning drives Chinese-English switch-point quality in TTS"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The evaluation assumes that the ASR model used to score intelligibility (Breeze-ASR-25) is an unbiased judge across all TTS systems being compared. If this ASR model has its own bias toward text patterns produced by the Taiwan-localized frontend, it could systematically favor BlueMagpie-TTS outputs over the baseline, inflating the measured CER gap. The paper acknowledges the ASR makes errors and uses the same ASR for all systems, but does not verify that the ASR's error rate,","fun_headline_variants_meta":{"raw":{"variants":["Taiwan-trained frontend halves TTS errors without touching the acoustic model","Code-switching TTS failures traced to the text frontend, not the acoustic renderer","Swapping tokenizer and language model alone cuts TTS CER from 11.45% to 4.81%","Fixed acoustic stack, new Taiwan frontend: 63.9% relative WER reduction","Frontend planning drives Chinese-English switch-point quality in TTS"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":727,"prompt_tokens":620,"completion_tokens":107,"prompt_tokens_details":null},"tokens_in":620,"tokens_out":107,"duration_ms":42800,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T17:50:38.431038+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"The frontend-dominance claim would be weakened if, when the same Taiwan-localized tokenizer and language model are attached to a different acoustic stack (not VoxCPM2), the CER reduction fails to replicate—suggesting the gain depends on an interaction between Barbet and VoxCPM2 specifically rather than being a general frontend effect. The claim would also be challenged if an independent ASR evaluator (not Breeze-ASR-25) produced a substantially smaller CER gap between BlueMagpie-TTS and the fine-tuned baseline, indicating the measured gap partly reflects ASR bias rather than true intelligible","supporting_citations":[],"review_version":1}