Pith. sign in

REVIEW 2 major objections 5 minor 25 references

MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read At 850 bps, a single-layer streaming codec with masked Gaussian noise injection matches or beats larger audio codecs on reconstruction and downstream tasks.

desk verdict Useful system paper undercut by test-set-selected mask ratios; the proof of Proposition 1 does not match the implemented masking. read the letter →

arxiv 2506.00385 v1 pith:D55XTGYV submitted 2025-05-31 cs.SD cs.AIcs.LGeess.AS

Yakun Song , Jiawei Chen , Xiaobin Zhuang , Chenpeng Du , Ziyang Ma , Jian Wu , Jian Cong , Dongya Jia
show 4 more authors
Zhuo Chen Yuping Wang Yuxuan Wang Xie Chen
This is my paper
classification cs.SDcs.AIcs.LGeess.AS
keywords neuralaudiocodecGaussiannoiseinjectionsingle-layervectorquantizationstreamingTransformerdiscretespeechtokenszero-shotTTSZipfdistributioncodebookcollapse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MagiCodec, a single-layer streaming Transformer audio codec that runs at 850 bits per second and 50 tokens per second. Its central claim is that replacing a fraction of input frames with Gaussian noise during training, together with a three-stage training pipeline, makes the quantized tokens easier for language models to use while keeping reconstruction quality high. The authors report that MagiCodec outperforms established codecs on word error rate, perceptual quality, speaker similarity, zero-shot TTS, phone recognition, emotion classification, and non-verbal detection, and that its token frequencies follow the Zipf-like power-law shape of natural-language tokens. If the claim holds, audio tokenizers do not need external semantic labels or multilayer residual quantizers to get both faithful reconstruction and good generation.

What carries the argument

The load-bearing mechanism is Gaussian frame-replacement noise injection: each frame is kept with probability $1-p$ and replaced with i.i.d. noise $\epsilon \sim \mathcal{N}(0,\sigma^2 I)$ with probability $p$. Proposition 1 is the identity that turns this into a frequency-domain low-pass regularizer with attenuation factor $(1-p)+p e^{-\sigma^2\|\omega\|^2/2}$. The other machinery is the staged training recipe: a quantization-free autoencoder stage with $\ell^2$ latent regularization, a frozen-encoder quantizer stage using straight-through estimation, a SimVQ-style linear reparameterization, and a final GAN vocoder stage with multi-period and multi-scale STFT discriminators. Together these components support a single vector-quantization layer with a codebook of 131072 entries at 50 tokens per second.

What would settle it

Train the identical three-stage pipeline with the same masking schedule but replace the Gaussian replacement with a fixed constant frame or with pure additive noise while holding every other hyperparameter fixed; if the reconstruction and downstream gains persist, the specific Gaussian-replacement mechanism is not what carries the result. Separately, compute $\mathbb{E}[f(\tilde{x})]$ for a quadratic $f$ under the actual operation and compare with the convolution in Proposition 1; a mismatch would falsify the stated Fourier derivation.

Watch

Extended reading notes

Core claim

MagiCodec's central discovery is that masking input frames with isotropic Gaussian noise, rather than adding noise or using semantic supervision, turns a vector-quantized codec into a better tokenizer for generative modeling. The paper derives an expression for the expected output of a network under this injection: $\mathbb{E}[f(\tilde{x})] = [(1-p)+p e^{-\sigma^2\|\omega\|^2/2}]\hat f(\omega)$, which multiplies each Fourier coefficient by a frequency-dependent factor that decays exponentially with frequency, so high-frequency content is attenuated while low-frequency structure is preserved. With a 131072-entry single codebook and a left-context streaming Transformer encoder, the resulting tokens at 50 Hz are reported to improve intelligibility, perceived quality, speaker similarity, and downstream zero-shot TTS, phoneme recognition, emotion recognition, and non-verbal detection compared with baselines including multi-layer codecs. The training is split into autoencoder, quantizer, and vocoder stages, with the encoder frozen during quantization to avoid codebook collapse. The paper also claims the token distributions approximate Zipf's law, especially for higher-order n-grams, suggesting the codes carry language-like semantic hierarchy.

Load-bearing premise

The paper's explanation of why frame replacement helps assumes the injected noise is additive and independent of the input, but under the real operation $\tilde{x} = (1-m)\odot x + m\odot \epsilon$ the effective perturbation $m\odot(\epsilon-x)$ depends on $x$, so the low-pass-filter proof does not follow as written.

Editorial extensions

If this is right

  • At 850 bps, a codec with one codebook layer can carry enough content, speaker, and paralinguistic information to drive credible zero-shot TTS, phoneme recognition, emotion classification, and non-verbal detection.
  • Masking up to 20–30% of frames during training improves both reconstruction and downstream scores, suggesting that moderate input corruption is a viable alternative to external semantic labels.
  • A 50 Hz token rate is the sweet spot: 25 Hz loses phonetic detail and 100 Hz produces longer sequences that hurt autoregressive generation, so token-rate choice interacts with downstream modelability.
  • If the token-frequency claim transfers, audio tokens with Zipf-like distributions should be easier for next-token language models to learn than flatter token histograms.
  • Because the encoder and quantizer are frozen while only the decoder receives GAN training, the discrete codes stay stable under adversarial finetuning, which the paper credits for avoiding codebook collapse.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's evidence, the ablations change the mask ratio but do not isolate Gaussian noise injection from the staged training recipe; a direct test would train the same pipeline with 0% masking versus injection at fixed variance, and the current evidence does not yet separate these two contributions.
  • Beyond the paper's proof, if Proposition 1's derivation gap is real (the effective perturbation depends on the input), the frequency-domain explanation is not established even if the empirical gains hold; a cleaner derivation would need to characterize the input-dependent distribution or treat the injection as a stochastic masking operator.
  • Beyond the paper's scope, because training was limited to 16 kHz English speech, the same recipe may need retuning for music, higher sample rates, or noisy conditions; the paper's own limitation section flags this untested territory.
  • Beyond the paper's visualization, the Zipf-like token distribution is demonstrated visually rather than by fitted power-law exponents; quantifying slopes and comparing them statistically across codecs would make the language-compatibility claim testable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. MagiCodec is a single-layer streaming Transformer audio codec that combines masked Gaussian noise injection with a three-stage training pipeline (autoencoder, quantizer, vocoder) to improve both reconstruction fidelity and downstream modelability of the discrete tokens. The paper claims state-of-the-art results across reconstruction metrics (WER, PER, STOI, PESQ, ViSQOL, UTMOS, SPK-SIM) and downstream tasks (zero-shot TTS, phoneme-level ASR, emotion recognition, non-verbal detection), and supports the design with a frequency-domain proposition about noise injection. Ablations vary the mask ratio and token rate, and the token distribution is compared with natural-language Zipf statistics. Code and pretrained models are promised at the provided repository.

Significance. If the empirical claims hold after independent verification, this is a practically important result: a single-layer 850 bps streaming codec that outperforms larger or multi-layer baselines on both reconstruction and generation would give an efficient tokenizer for audio language models, and the Zipf-like token distribution suggests good compatibility with LM-style training. The paper is commendable for evaluating reconstruction and downstream tasks side by side, for including ablations of the central design choices, and for making code and models available. The main caveats are that the theoretical justification for the noise-injection mechanism is not rigorously established, and that the reported state-of-the-art margins may be inflated by selecting the mask ratio per benchmark on the test sets.

major comments (2)
  1. [Appendix A.1, Proposition 1] The proof does not establish the stated claim for the implemented operation. For the actual input corruption x̃=(1−m)⊙x+m⊙ε, the proof rewrites x̃=x+ε′ with ε′=m⊙(ε−x), but ε′ depends on x and is not independent of x; conditionally on m=1 it has mean −x, so it is not zero-mean Gaussian noise independent of the signal. Consequently the step E[f(x+ε′)]=(1−p)f(x)+p(kσ∗f)(x) does not follow from the Gaussian density used in the preceding lines, and the Fourier attenuation factor (1−p)+p exp(−σ²‖ω‖²/2) is not derived for replacement noise. This leaves the central theoretical motivation in Section 3.2 unsupported. Please either prove the statement for the actual replacement operation with explicit conditioning on the mask and signal, or present the frequency-domain argument explicitly as a heuristic rather than as a proposition.
  2. [Sections 4.2 and 4.3, Tables 2-5 versus Tables 6-7] The main results appear to select the mask ratio per benchmark on the same test sets used for evaluation. Table 2's reconstruction WER/PER (3.16/1.63) match the 20% mask row in Table 6, while Table 3's TTS WER of 3.30 and Table 5's emotion ACC/F1 of 0.70/0.70 match the 30% mask row in Table 7. All of these metrics are reported on the same LibriSpeech test-clean, ESD test, and VocalSound test sets, and no validation split or repeated selection procedure is described. Choosing the mask ratio to maximize each benchmark on the test data biases the central claim that MagiCodec 'surpasses state-of-the-art codecs in both reconstruction quality and downstream tasks.' Please fix a single mask ratio using a held-out validation set, report all main results for that fixed configuration, and state whether the baseline comparisons are evaluated under the identical protocol.
minor comments (5)
  1. [Section 4.1.4] The subsection on emotion and nonverbal detection is duplicated verbatim (the same text appears twice); please remove one copy.
  2. [Section 4.3 and Figure 2] The text and figure caption refer to 'MagicCodec' in several places; use the consistent spelling 'MagiCodec'.
  3. [Tables 2 and 3] Table 3 marks BigCodec as non-streaming (×), while Table 2 marks BigCodec† as streaming (✓); please clarify whether the BigCodec results in Table 3 come from the original BigCodec or the streaming BigCodec-S variant and explain the different flags.
  4. [Tables 2-4] Reconstruction and TTS results are reported as single values without confidence intervals or significance tests, even though Table 5 includes standard deviations; please state the number of runs and random seeds for all quantitative results, or at least for the headline comparisons.
  5. [Appendix A.1, Eq. (2)] The proof uses bf, bδ, and bkσ without defining the Fourier convention or the domain of the transform; please add explicit definitions and unify the notation for the Fourier transform of a function.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reconstruction, downstream, and Zipf claims are empirical and self-contained; the unsound Proposition 1 and per-benchmark mask-ratio selection are correctness and validity concerns, not derivations that reduce to their inputs.

full rationale

MagiCodec's central claims are empirical comparisons computed against external, independently trained baselines (EnCodec, DAC, WavTokenizer, BigCodec, TS3Codec, etc.), so the SOTA assertions do not derive from the method's own definitions. The mask-ratio ablations in Tables 6-7 do show that the main tables' best reconstruction numbers (e.g., WER 3.16) use the mask=20% row, while the TTS and emotion numbers use the mask=30% row, meaning the reported 'MagiCodec' is not a single fixed configuration and the hyperparameter appears selected on the same test sets used for the headline comparison. This is a genuine validity/bias concern, but it is not circularity under the defined patterns: no equation or fitted parameter is renamed as a prediction, and the metric itself is not an input to the model's training objective. I also examined Proposition 1 and Appendix A.1. The proof treats the replacement noise x_tilde = (1-m) x + m epsilon as though epsilon' = m(epsilon - x) were independent of x and Gaussian, so the convolution and low-pass factor do not follow for the actual replacement operation; that makes the theoretical justification mathematically unsupported. An incorrect proof, however, is not a circular derivation: the empirical results do not depend on the proposition. The self-citations present (e.g., Seed-TTS, WavLM) are used as background or tooling, not as load-bearing evidence for the codec's claimed superiority, and no 'uniqueness theorem' or prior-work ansatz is invoked to force the design. The Zipf observation is descriptive and based on the trained token distributions, not assumed as a construction premise. Therefore no step in the claimed derivation chain is equivalent, by construction or by citation, to its own input, and the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central design relies on a handful of hand-chosen or unreported hyperparameters: the mask ratio (set to 0.2 from ablation), the Gaussian noise scale σ (not reported), and the loss weights (not reported). It also depends on standard Fourier and convolution facts, on the empirical spectral-bias assumption from the literature, and on the unproved coding hypothesis that replacement noise acts as a low-frequency regularizer. No new physical entities are introduced.

free parameters (3)
  • Mask ratio p = 0.20 (main; ablated 0, 0.1, 0.2, 0.3)
    Probability of replacing an input frame with Gaussian noise. It is not derived; the main model uses 20% because it yields the best WER/PER in the ablation on the evaluation set.
  • Gaussian noise standard deviation σ = not reported
    The noise scale appears in Proposition 1 and in the training procedure, but no value or schedule is given anywhere in the paper, leaving a hidden hyperparameter that affects the proposed regularization.
  • Loss weights λ_mel, λ_e, λ_q, λ_adv, λ_feat = not reported
    The total loss equations in Section 3.3 introduce these weights, but their values are not listed, so the exact training objective is underspecified.
assumptions (4)
  • standard math The Fourier transform of a Gaussian kernel is hat(k)_σ(ω) = e^(-σ²||ω||²/2) and the convolution theorem holds.
    Used in Appendix A.1 to derive the attenuation factor in Proposition 1.
  • standard math Linear operations commute with expectations for Fourier-transformable functions.
    The proof of Proposition 1 relies on E[f(x+ε)] = (k_σ * f)(x) for additive Gaussian noise.
  • domain assumption Neural networks have spectral bias, learning low frequencies first and overfitting high-frequency noise.
    Section 3.2.1 relies on Rahaman et al. (2019) to justify why attenuating high-frequency content improves modelability and reconstruction.
  • ad hoc to paper Training with replacement noise strengthens low-frequency semantic structure and improves downstream language-model compatibility.
    This is the core design hypothesis. The proposition is claimed to support it, but the proof in Appendix A.1 actually models additive noise, not frame replacement, so the assumption is not established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation." pith.science (2026). https://pith.science/paper/D55XTGYV

@misc{pith2026250600385,
  author       = {Pith},
  title        = {Pith review of: MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D55XTGYV}},
  note         = {Machine review of arXiv:2506.00385}
}
abstract

Neural audio codecs have made significant strides in efficiently mapping raw audio waveforms into discrete token representations, which are foundational for contemporary audio generative models. However, most existing codecs are optimized primarily for reconstruction quality, often at the expense of the downstream modelability of the encoded tokens. Motivated by the need to overcome this bottleneck, we introduce $\textbf{MagiCodec}$, a novel single-layer, streaming Transformer-based audio codec. MagiCodec is designed with a multistage training pipeline that incorporates Gaussian noise injection and latent regularization, explicitly targeting the enhancement of semantic expressiveness in the generated codes while preserving high reconstruction fidelity. We analytically derive the effect of noise injection in the frequency domain, demonstrating its efficacy in attenuating high-frequency components and fostering robust tokenization. Extensive experimental evaluations show that MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like distributions, as observed in natural languages, thereby improving compatibility with language-model-based generative architectures. The code and pre-trained models are available at https://github.com/Ereboas/MagiCodec.

Figures

Figures reproduced from arXiv: 2506.00385 by the authors.

Figure 1
Figure 1. The pipeline of the proposed MAGICODEC. its close adherence to the Zipf distribution observed in natural language, which facilitates downstream model training. 2 Related Work Neural audio codecs aim to encode continuous audio signals into discrete latent representations, which is a form of discrete audio tokenization. In recent years, neural audio codecs have become a research focus to achieve high-quality audio rec… view at source ↗
Figure 2
Figure 2. Visualization of the latent space using tSNE and 10 random classes in ESC-50 dataset. [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Plots of normalized token log-frequency versus normalized log-rank for various codec [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 1 canonical work pages

  1. [1]

    GPT-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [3]

    Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

    arXiv

  3. [6]

    High fidelity neural audio compression.arXiv preprint arXiv:2210.13438,

    Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression.arXiv preprint arXiv:2210.13438,

    arXiv

  4. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

    URLhttps://arxiv.org/abs/2410.00037. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. InProc. NAACL, pages 4171–4186,

    arXiv

  5. [10]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024a

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024a. Haohe Liu, Xuenan Xu, Yi Yuan, Mengyue Wu, Wenwu Wang, and Mark D Plumbley. SemantiCodec: An ultra low bitrate semantic audio codec for general sound.IEEE J...

    arXiv

  6. [12]

    Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville

    URL https://arxiv.org/ abs/2212.04356. Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. InInternational conference on machine learning, pages 5301–5310. PMLR,

    arXiv

  7. [14]

    V ocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis.arXiv preprint arXiv:2306.00814,

    Hubert Siuzdak. V ocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis.arXiv preprint arXiv:2306.00814,

    arXiv

  8. [15]

    SNAC: Multi-scale neural audio codec

    15 Hubert Siuzdak, Florian Grötschla, and Luca A Lanzendörfer. SNAC: Multi-scale neural audio codec. arXiv preprint arXiv:2410.14411,

    arXiv

Show all 25 references
  1. [16]

    Improving the diffusability of autoencoders.arXiv preprint arXiv:2502.14831,

    Ivan Skorokhodov, Sharath Girish, Benran Hu, Willi Menapace, Yanyu Li, Rameen Abdal, Sergey Tulyakov, and Aliaksandr Siarohin. Improving the diffusability of autoencoders.arXiv preprint arXiv:2502.14831,

    arXiv

  2. [19]

    BigCodec: Pushing the limits of low-bitrate neural speech codec.arXiv preprint arXiv:2409.05377,

    Detai Xin, Xu Tan, Shinnosuke Takamichi, and Hiroshi Saruwatari. BigCodec: Pushing the limits of low-bitrate neural speech codec.arXiv preprint arXiv:2409.05377,

    arXiv

  3. [20]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115,

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report.arXiv preprint arXiv:2412.15115,

    arXiv

  4. [21]

    Reconstruction vs

    Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models.arXiv preprint arXiv:2501.01423,

    arXiv

  5. [23]

    Representation collapsing problems in vector quantization.arXiv preprint arXiv:2411.16550,

    Wenhao Zhao, Qiran Zou, Rushi Shah, and Dianbo Liu. Representation collapsing problems in vector quantization.arXiv preprint arXiv:2411.16550,

    arXiv

  6. [24]

    FreeCodec: A disentangled neural speech codec with fewer tokens.arXiv preprint arXiv:2412.01053,

    Youqiang Zheng, Weiping Tu, Yueteng Kang, Jie Chen, Yike Zhang, Li Xiao, Yuhong Yang, and Long Ma. FreeCodec: A disentangled neural speech codec with fewer tokens.arXiv preprint arXiv:2412.01053,

    arXiv

  7. [25]

    Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99%.arXiv preprint arXiv:2406.11837, 2024a

    Lei Zhu, Fangyun Wei, Yanye Lu, and Dong Chen. Scaling the codebook size of vqgan to 100,000 with a utilization rate of 99%.arXiv preprint arXiv:2406.11837, 2024a. Yongxin Zhu, Bocheng Li, Yifei Xin, and Linli Xu. Addressing representation collapse in vector quantized models w...

    2010 arXiv

  8. [1995]

    Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al

    doi: 10.1162/neco.1995.7.1.108. Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, et al. AudioLM: a language modeling approach to audio generation.IEEE/ACM tran...

    1995 doi

  9. [2001]

    Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari

    doi: 10.1109/ICASSP.2001.941023. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. UTMOS: Utokyo-sarulab system for voiceMOS challenge 2022.arXiv preprint arXiv:2204.02152,

    2001

  10. [2010]

    Aaron Van Den Oord, Oriol Vinyals, et al

    doi: 10.1109/ ICASSP.2010.5495701. Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information processing systems, 30,

    2010

  11. [2015]

    Karol J. Piczak. ESC: Dataset for Environmental Sound Classification. InProceedings of the 23rd Annual ACM Conference on Multimedia, pages 1015–1018. ACM Press. ISBN 978-1-4503-3459-4. doi: 10.1145/2733373.2806390. URL http://dl.acm.org/citation.cfm?doid=2733373. 2806390. Alec...

  12. [2017]

    Ts3-codec: Transformer-based simple streaming single codec.arXiv preprint arXiv:2411.18803,

    Haibin Wu, Naoyuki Kanda, Sefik Emre Eskimez, and Jinyu Li. Ts3-codec: Transformer-based simple streaming single codec.arXiv preprint arXiv:2411.18803,

    arXiv

  13. [2020]

    Analyzing the language of visual tokens.arXiv preprint arXiv:2411.05001,

    David M Chan, Rodolfo Corona, Joonyong Park, Cheol Jun Cho, Yutong Bai, and Trevor Darrell. Analyzing the language of visual tokens.arXiv preprint arXiv:2411.05001,

    arXiv

  14. [2021]

    SpeechTokenizer: Unified speech tokenizer for speech large language models.arXiv preprint arXiv:2308.16692,

    Xin Zhang, Dong Zhang, Shimin Li, Yaqian Zhou, and Xipeng Qiu. SpeechTokenizer: Unified speech tokenizer for speech large language models.arXiv preprint arXiv:2308.16692,

    arXiv

  15. [2022]

    Repcodec: A speech representation codec for speech tokenization.arXiv preprint arXiv:2309.00169,

    14 Zhichao Huang, Chutong Meng, and Tom Ko. Repcodec: A speech representation codec for speech tokenization.arXiv preprint arXiv:2309.00169,

    arXiv

  16. [2023]

    WavTokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling.arXiv preprint arXiv:2408.16532,

    Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, et al. WavTokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling.arXiv preprint arXiv:2408.16532,

    arXiv

  17. [2024]

    Seed-TTS: A family of high-quality versatile speech generation models.arXiv preprint arXiv:2406.02430,

    Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, et al. Seed-TTS: A family of high-quality versatile speech generation models.arXiv preprint arXiv:2406.02430,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.