Pith. sign in

REVIEW 4 cited by

Building a Taiwanese Mandarin Spoken Language Model: A First Attempt

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.07111 v2 pith:PRRR5JVU submitted 2024-11-11 cs.CL cs.SDeess.AS

Chih-Kai Yang , Yu-Kuan Fu , Chen-An Li , Yi-Cheng Lin , Yu-Xiang Lin , Wei-Chih Chen , Ho Lam Chung , Chun-Yi Kuan
show 13 more authors
Wei-Ping Huang Ke-Han Lu Tzu-Quan Lin Hsiu-Hsuan Wang En-Pei Hu Chan-Jan Hsu Liang-Hsuan Tseng I-Hsiang Chiu Ulin Sanga Xuanjun Chen Po-chun Hsu Shu-wen Yang Hung-yi Lee
This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords interactionmandarinmodelspokentaiwaneseattemptconversationaldialogues
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This technical report presents our initial attempt to build a spoken large language model (LLM) for Taiwanese Mandarin, specifically tailored to enable real-time, speech-to-speech interaction in multi-turn conversations. Our end-to-end model incorporates a decoder-only transformer architecture and aims to achieve seamless interaction while preserving the conversational flow, including full-duplex capabilities allowing simultaneous speaking and listening. The paper also details the training process, including data preparation with synthesized dialogues and adjustments for real-time interaction. We also developed a platform to evaluate conversational fluency and response coherence in multi-turn dialogues. We hope the release of the report can contribute to the future development of spoken LLMs in Taiwanese Mandarin.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models

    cs.SD 2026-07 conditional novelty 7.0 of 10

    Label-free real-vs-noise scoring of audio-encoder neurons, followed by sparse amplification, substantially improves LALM perception of non-semantic speech attributes without retraining.

  2. BreezyVoice: Adapting TTS for Taiwanese Mandarin with Enhanced Polyphone Disambiguation -- Challenges and Insights

    cs.CL 2025-01 conditional novelty 6.0 of 10

    BreezyVoice adapts CosyVoice to Taiwanese Mandarin with g2pW-based phonetic augmentation and a two-stage iconic-unit voice cloning pipeline, improving pronunciation accuracy and cloning robustness.

  3. Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Models

    cs.CL 2025-01 conditional novelty 6.0 of 10

    The authors release MNSC, the largest standardized multitask spoken Singlish corpus, and SingAudioLLM, a multimodal model that sets strong baselines on ASR, spoken QA, dialogue summarization, and paralinguistic QA.

  4. Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Models

    cs.CL 2025-05 conditional novelty 4.0 of 10

    In a three-stage end-to-end spoken language model, experience replay (mixing old data into later training) was the most effective mitigation against catastrophic forgetting, greatly outperforming model merging and LoR...

Pith tools