Pith. sign in

REVIEW 3 major objections 5 minor 49 references

EG-VAE: A Unified Framework for Electric Guitar Tone Transfer and Removal

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EG-VAE claims that a single variational autoencoder, factorizing a wet electric guitar recording into frame-level content and global tone, can perform both tone transfer and tone removal better than task-specific baselines.

desk verdict A genuinely novel unified tone transfer/removal framework with convincing internal ablations, but the plugin-only evaluation makes the general guitar claim unproven; still deserving of peer review. read the letter →

arxiv 2608.05513 v1 pith:FGP6JWTO submitted 2026-08-06 eess.AS

Yen-Tung Yeh , Yun-Ning (Amy) Hung , Yi-Hsuan Yang This is my paper · ORCID
classification eess.AS
keywords electricguitartonetransferremovaldryDIrecoverycontent-tonedisentanglementvariationalautoencodermaskingunseengeneralizationaudioeffectsmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EG-VAE claims that electric guitar tone transfer and tone removal are two directions of the same operation and can be solved by one model. The model factorizes a wet recording into a frame-level content embedding (what is played) and a global tone embedding (the signal chain), then realizes transfer by swapping tone embeddings and removal by a tone-masking forward pass. The paper reports that this single unified model beats task-specific baselines on both tasks, cutting the Mel distance of tone transfer from 1.53 to 0.86 on seen tones while also improving removal from 1.21 to 1.10. A second training stage with variational sampling and audio-effects augmentation is what makes unseen reference tones work. If true, this means removal and transfer can share one learned representation instead of separate task-specific systems.

What carries the argument

The central mechanism is tone masking, which operates through conditional layer normalization (CLN), a layer that injects a global attribute via scale and shift parameters. Tone masking overrides those parameters to the identity ($\gamma=1$, $\beta=0$), so the content embedding alone reaches the decoder; supervised against the dry signal, this masked pass becomes the tone-removal procedure at inference and simultaneously penalizes tone leakage into content. The supporting machinery includes an information bottleneck on the content embedding, content-tone perturbation, pitch and preset supervision, posterior variance-parameterised Gaussian dropout to prevent tone-latent collapse, and a second training stage with variational sampling and audio-effects augmentation that smooths the tone space for unseen tones.

What would settle it

Record wet/dry pairs from physical amplifier-and-pedal rigs or from many non-plugin renderers and compare EG-VAE against the same baselines; if its Mel-distance and listening-test advantages shrink or disappear outside the two-plugin distribution, the generalization claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that a wet guitar recording can be split into a frame-level content embedding and a global tone embedding by a variational autoencoder, and that the same split serves both tasks. Tone transfer recombines one recording's content with another's tone embedding. Tone removal is the same masked forward pass used during training, where the conditional layer normalization's modulation is overridden to $\gamma=1,\beta=0$, leaving the content embedding to reconstruct the dry signal alone; this single operation both defines removal and strengthens disentanglement by forcing tone information out of the content path. The paper reports that EG-VAE reduces the Mel distance of transfer by 44% over the strongest baseline on seen tones (0.86 vs 1.53) and improves removal over the best specialized baseline (1.10 vs 1.21), with subjective ratings approaching the ground-truth anchor, and that a two-stage training procedure with variational tone sampling and effects augmentation preserves quality on unseen tones.

Load-bearing premise

The evaluation assumes that wet recordings rendered by two commercial guitar plugins are representative of real electric-guitar tones, so that a model trained and tested on these renderings will transfer to real amplifiers, pedals, and recording chains.

Editorial extensions

If this is right

  • Tone removal stops being a separate module: the same masked forward pass used in training outputs the dry signal at inference, so a single checkpoint serves both tasks.
  • Because transfer operates on a representation already stripped of the input's tone, it sidesteps the error accumulation that a two-stage remove-then-model pipeline suffers.
  • Unseen reference tones remain usable: variational sampling plus effects augmentation makes the tone space smooth enough that held-out chains decode coherently, with perceptual path length dropping from 2.00 to 0.90.
  • Minimizing spectral distance is not enough for removal quality: one baseline with competitive spectral scores was rated lowest in audio quality, while EG-VAE led both audio quality and dryness in the listening test.
  • The ablation results indicate the disentanglement mechanisms are complementary; removing content-tone perturbation hurts all three modes most, and removing tone masking also degrades reconstruction and transfer, not just removal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The plugin-based evaluation bounds the claim: real-world rigs add room acoustics, microphones, and physical amplifier behavior absent from the two commercial plugin chains, so the seen/unseen gains should be re-measured on recordings of physical chains.
  • Tone masking is a general recipe for conditional-normalization architectures: any attribute injected through scale/shift parameters can be removed by the identity modulation ($\gamma=1$, $\beta=0$), turning the masked pass into a self-supervised disentanglement objective for other attributes or instruments.
  • The paper's own conclusion flags delay as an open ambiguity: a delayed repetition is an effect acoustically but resembles performed content, so the current content/tone assignment has no consistent cue; resolving this could extend the framework to spatial and time-based effects.
  • Perceptual path length on unseen tones could serve as a tuning signal for audio style-transfer models generally, since it measures exactly the smoothness that makes zero-shot transfer work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EG-VAE, a unified variational autoencoder for electric guitar tone transfer (EGTT) and tone removal (EGTR). A wet recording is factorized into a frame-level content embedding and a global tone embedding; transfer recombines a source's content with a reference's tone, while removal is performed by a 'tone masking' operation that overrides the conditional layer normalization to decode from content alone. A second training stage adds variational sampling and Wiener-Hammerstein audio-effects augmentation to improve handling of unseen tones. The model is evaluated on a custom dataset rendered from EGDB dry signals through two Neural DSP plugins, compared with task-specific baselines using spectral distances and a 14-participant listening test, and analyzed with ablations and a perceptual-path-length smoothness measure.

Significance. If the results hold beyond the rendering distribution, the work is a meaningful step in guitar tone modeling: it unifies two previously separate tasks in a single representation, introduces a simple tone-masking mechanism that serves both training and inference, and reports large objective and subjective gains over task-specific baselines. The paper is clearly written, includes a thorough ablation of each proposed component, evaluates tone-space smoothness, and provides demonstration audio. The central novelty—tone masking as a dual-purpose disentanglement/removal objective, together with the two-stage training for unseen-tone transfer—is conceptually attractive and well motivated. The main open question is external validity, because all training and test data are plugin renderings and no real-world wet/dry recordings are used.

major comments (3)
  1. [Section IV-A and Abstract] The entire empirical evaluation is based on renderings of EGDB dry signals through two Neural DSP plugins (Archetype: Cory Wong X for seen tones, Morgan Amps Suite for unseen tones); no real-world wet/dry recordings or third-party data are used anywhere. The 'unseen' condition is thus a held-out plugin of the same brand and rendering paradigm, not a real signal chain. Since the abstract and introduction claim that EG-VAE solves EGTT and EGTR generally, this is a load-bearing limitation: the learned content–tone factorization and masking-based removal may not transfer to chains involving analog pedals, tube amplifiers, cabinet/microphone coloration, or recording/mixing effects. The authors should either temper the generalization claim to 'plugin-rendered tones' or, preferably, evaluate on a modest set of real wet/dry guitar recordings (or a third-party dataset) to test whether the reported gains persist.
  2. [Section IV-C, Tables I and II] The baselines are retrained with the optimization objective standardized to the multi-scale mel-spectrogram loss used for EG-VAE, and Distortion Recovery's HiFi-GAN vocoder is replaced with BigVGAN. For methods originally designed with different losses (e.g., HDemucs hybrid spectrogram/waveform loss, DPTNet SI-SNR), this standardization may arbitrarily disadvantage them, and no ablation shows the standardized loss is suitable for each architecture. In addition, the 'w/ EGTR' baselines are trained on clean dry signals but at inference receive EG-VAE's estimated dry signal, a train/test distribution shift that could handicap them; the paper should justify this setup or fine-tune the baselines on estimated dry inputs. Without these controls, the magnitude of the reported advantage over baselines is uncertain.
  3. [Section I and Table II] The introduction states that EG-VAE improves over Distortion Recovery '1.10 vs 1.21 Mel on seen tones, matching on unseen.' Table II shows unseen Mel 1.19 vs 1.22 (EG-VAE better) and unseen STFT 1.87 vs 1.86 (Distortion Recovery better). This discrepancy should be corrected, and the abstract's 'outperforms task-specific baselines on transfer and removal' should be qualified to note that on unseen-tone STFT for removal the two models are essentially tied.
minor comments (5)
  1. [Section IV-D and Tables III/IV] The captions of Tables III and IV do not state which evaluation split they report; Table IV's numbers match the unseen-tone results in Tables I and II, but the text never says so explicitly. Please state in the captions and the ablation section that Table III is on seen tones and Table IV is on unseen tones.
  2. [Section IV-D] The objective metrics are averaged over 2,000 ten-second test examples, but the test partitions are described in hours (142 and 29 hours); please clarify how the 2,000 examples are sampled from these partitions.
  3. [Section V-B] The subjective ratings are reported with standard deviations but no statistical significance testing; with 14 participants, a paired test (e.g., Wilcoxon signed-rank) would help confirm whether the AQ and Dryness differences between EG-VAE and the baselines are reliable.
  4. [Section III-D] The claim that tone masking 'directly' produces the dry signal is somewhat strong: it is only through training with L_mask that the decoder learns to map the no-modulation regime to the dry target, so the inference operation is a learned readout rather than an exact inverse. Consider softening the wording in Eq. (11) and the surrounding text.
  5. [Section I] The sentence 'owing to the highly nonlinear distortion and the wide diversity of effects ... its signal chain' is missing a preposition and is ungrammatical; please revise.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: EG-VAE's claims are supervised, held-out empirical results against independent baselines, not derivations from their own outputs.

full rationale

EG-VAE is a supervised representation-learning system, and its central claims are empirical: it is trained on paired wet/dry plugin renderings and evaluated on held-out DI content and held-out tones against independently implemented baselines. The EGTR readout (the masked forward pass, x_EGTR = D_mask(z), Section III-D) is exactly the operation trained by Eq. (12) with L_mask = L_audio(D_mask(z), x_dry); this is standard supervised evaluation, not a fitted parameter being renamed as a prediction, and the test partitions do not overlap in DI content or tone. The EGTT recombination in Eq. (6) is likewise directly trained through the conversion mode, and its improvement is an empirical result, not a constructional equivalence. The 'w/ EGTR' baseline configuration injects EG-VAE's own removal output into the baselines, which strengthens the baselines rather than inflating EG-VAE's comparison, so it is not circular. The self-citations present (EGDB/EGDB-PG as the DI data source, the amplifier-modeling baseline, and the Wiener-Hammerstein augmentation reference) are data sources, baselines, or standard modeling choices; none is a load-bearing proof, and no uniqueness theorem is imported from the authors' prior work. The plugin-only evaluation is a genuine external-validity limitation, but it is not a circularity: the model's performance is measured against held-out renderings, and the baselines face the same data distribution. Overall, no step in the paper's derivation chain reduces by construction to its own inputs; the score reflects only non-load-bearing self-citation, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The paper introduces no new physical agents or data-generating processes, so the invented_entities ledger is almost empty: tone masking and perturbation are operations, not entities. The central load-bearing items are the hand-chosen hyperparameters and the domain assumptions about factorizability, decoder behavior at the masked operating point, and representativeness of the plugin-rendered dataset. The backbone architecture and its losses are pulled from prior work (DAC-VAE), and the dataset from EGDB plus proprietary plugins, so these count as external assumptions rather than contributions.

free parameters (5)
  • Loss weights (lambda_mel, lambda_fm, lambda_adv, lambda_c_KL, lambda_s_KL, lambda_pitch, lambda_preset) = 15, 2, 1, 1e-4, 1e-4, 1, 5
    These hyperparameters are set by hand and the paper does not report a sensitivity study; the central claims of disentanglement and transfer quality depend on their values, and at least some were likely chosen based on validation performance.
  • Training mode sampling probabilities = [0.25, 0.45, 0.30] for self-reconstruction, conversion, masked reconstruction
    The balance between the three training modes is chosen by hand and directly determines how the model learns the disentanglement and the masked forward pass; no sensitivity analysis is reported.
  • Content bottleneck dimension = 128 (projected from 1024)
    The capacity of the content latent is a design choice essential for forcing tone information into the global embedding; it is a hand-chosen structural hyperparameter, although its effect is partially explored through the KL weight.
  • Tone embedding dimensionality = 64
    The dimensionality of the tone space is set by hand and affects how smoothly the second-stage space can be shaped and how much information it can retain.
  • Stage step counts and KL warmup = 150k stage 1, 200k stage 2, 10k warmup
    Training duration and warmup schedule are picked by hand; the paper states the lowest validation loss determines the checkpoint, so these are indirectly tuned.
assumptions (5)
  • domain assumption The mapping from dry to wet (the tone) is roughly independent of the played content in the proposed representation, so that content and tone embeddings can be recombined across recordings without catastrophic artifacts.
    This is the central inductive bias of the paper, stated in Section III-A and enforced through multiple objectives; if this factorization does not hold for real guitar signals, both EGTT and EGTR degrade. The model is explicitly trained to encourage it, so it is a design assumption rather than an external fact.
  • domain assumption LayerNormalization(z_t) followed by decoding produces a meaningful dry-signal output for any content embedding, i.e., the no-modulation regime is a valid operating point for the decoder.
    Used in Section III-D for tone masking; this is guaranteed only to the extent that the training objective grounds the masked pass to dry signals, and it may fail for content embeddings far from the training distribution.
  • domain assumption The decoder response varies smoothly with respect to the tone embedding, so that interpolation and sampling in tone space produce coherent audio.
    Assumed in Section III-E and measured via PPL in Section V-D; the paper provides evidence for this within its own model, but it is a structural assumption about the learned manifold rather than a mathematical guarantee.
  • domain assumption The EGDB dry recordings and the chosen Neural DSP plugin renderings form a representative distribution of electric guitar performance and tones.
    The dataset construction in Section IV-A assumes this; the paper's external validity rests on it, and the authors do not test on real wet recordings or third-party renderings.
  • domain assumption The DAC-VAE encoder/decoder and adversarial training setup provide sufficient reconstruction fidelity for the disentanglement objectives to be effective.
    The paper adopts the DAC-VAE architecture and its losses without independent justification; if the backbone were too weak, the content embedding would be unable to carry the dry signal under masking, so this is a load-bearing background choice.
invented entities (2)
  • Tone masking operation (no-modulation regime, gamma=1, beta=0 in CLN) independent evidence
    purpose: Defines the EGTR procedure at inference while acting as a disentanglement objective during training.
    It is not an entity but an operation; its falsifiable handle is the EGTR performance against external baselines. It is anchored to the shared CLN operation and produces a specific testable output (the dry DI estimate).
  • Content-tone perturbation training mode
    purpose: Enforces that content and tone embeddings are invariant to the factor they should not encode.
    A training recipe rather than an entity; its effect is demonstrated through ablation, not through an independent external prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EG-VAE: A Unified Framework for Electric Guitar Tone Transfer and Removal." pith.science (2026). https://pith.science/paper/FGP6JWTO

@misc{pith2026260805513,
  author       = {Pith},
  title        = {Pith review of: EG-VAE: A Unified Framework for Electric Guitar Tone Transfer and Removal},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FGP6JWTO}},
  note         = {Machine review of arXiv:2608.05513}
}
read the original abstract

Electric guitar tone transfer (EGTT) and tone removal (EGTR) are two fundamental tasks in guitar tone modeling: EGTT replaces a recording's tone with that of a reference, while EGTR recovers the dry direct-input (DI) signal from a wet, processed recording. Despite their highly related nature, prior work has addressed them independently, and both works have yet to achieve satisfactory results. In this paper, we propose EG-VAE, a unified framework that jointly models EGTT and EGTR by disentangling frame-level content and global tone representations from wet recordings with a variational autoencoder. EGTT is achieved by recombining a source's content with a reference's tone, while EGTR is attained by a novel tone masking objective that enforces content-tone disentanglement during training and realizes the removal procedure at inference. To improve transfer to tones unseen in training, a second training stage shapes a smooth tone space through variational sampling and audio-effects augmentation. Experimental results from both objective and subjective evaluations demonstrate that EG-VAE outperforms task-specific baselines on transfer and removal. Demos are available at https://guitar-tone-demo.vercel.app/.

Figures

Figures reproduced from arXiv: 2608.05513 by the authors.

Figure 1
Figure 1. The two tone-modeling tasks studied in this work. A dry direct-input [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of EG-VAE. A shared encoder maps the input waveform to features [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Tone conditioning versus tone masking, sharing the same CLN [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Tone removal on a representative distortion-heavy tone. Top: wet input [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: t-SNE visualization of the tone embedding [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 43 canonical work pages

  1. [1]

    Towards automatic transcription of polyphonic electric guitar music: A new dataset and a multi-loss transformer model,

    Y .-H. Chen, W.-Y . Hsiao, T.-K. Hsieh, J.-S. R. Jang, and Y .-H. Yang, “Towards automatic transcription of polyphonic electric guitar music: A new dataset and a multi-loss transformer model,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP). IEEE, 2022

    work page 2022

  2. [2]

    Towards Generalizability to Tone and Content Variations in the Transcription of Amplifier Rendered Electric Guitar Audio

    Y .-H. Chen, Y .-C. Cheng, Y .-T. Yeh, J.-T. Wu, J.-S. R. Jang, and Y .-H. Yang, “Towards generalizability to tone and content variations in the transcription of amplifier rendered electric guitar audio,”arXiv preprint arXiv:2504.07406, 2025

    work page Pith review arXiv 2025

  3. [3]

    High resolution guitar transcription via domain adaptation,

    X. Riley, D. Edwards, and S. Dixon, “High resolution guitar transcription via domain adaptation,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024

    work page 2024

  4. [4]

    Automatic detection of audio effects in guitar and bass recordings,

    M. Stein, J. Abeßer, C. Dittmar, and G. Schuller, “Automatic detection of audio effects in guitar and bass recordings,” inAudio Engineering Society Convention 128, 2010

    work page 2010

  5. [5]

    Synthtab: Leveraging synthesized data for guitar tablature transcription,

    Y . Zang, Y . Zhong, F. Cwitkowitz, and Z. Duan, “Synthtab: Leveraging synthesized data for guitar tablature transcription,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024

    work page 2024

  6. [6]

    Guitarflow: Realistic electric guitar synthesis from tablatures via flow matching and style transfer,

    J. Loth, P. Sarmento, M. Sandler, and M. Barthet, “Guitarflow: Realistic electric guitar synthesis from tablatures via flow matching and style transfer,”arXiv preprint arXiv:2510.21872, 2025

    arXiv 2025

  7. [7]

    Block-oriented gray box modeling of guitar amplifiers,

    F. Eichas, S. M ¨oller, and U. Z ¨olzer, “Block-oriented gray box modeling of guitar amplifiers,” inProc. International Conference on Digital Audio Effects (DAFx), 2017

    work page 2017

  8. [8]

    Real-time black-box modelling with recurrent neural networks,

    A. Wright, E.-P. Damsk ¨agg, and V . V ¨alim¨aki, “Real-time black-box modelling with recurrent neural networks,” inProc. International Con- ference on Digital Audio Effects (DAFx), 2019. 12

    work page 2019

Show all 49 references
  1. [9]

    Neural modeling of phaser and flanging effects,

    A. Wright and V . Valimaki, “Neural modeling of phaser and flanging effects,”Journal of the Audio Engineering Society, 2021

    2021

  2. [10]

    Ddsp guitar amp: Interpretable guitar amplifier modeling,

    Y .-T. Yeh, Y .-H. Chen, Y .-C. Cheng, J.-T. Wu, J.-J. Fu, Y .-F. Yeh, and Y .-H. Yang, “Ddsp guitar amp: Interpretable guitar amplifier modeling,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025

    2025

  3. [11]

    Improving inference-time optimisation for vocal effects style transfer with a gaussian prior,

    C.-Y . Yu, M. A. Mart ´ınez-Ram´ırez, J. Koo, W.-H. Liao, Y . Mitsufuji, and G. Fazekas, “Improving inference-time optimisation for vocal effects style transfer with a gaussian prior,” inProc. WASPAA, 2025

    2025

  4. [12]

    ST-ITO: Controlling audio effects for style transfer with inference-time optimization,

    C. J. Steinmetz, S. Singh, M. Comunit `a, I. Ibnyahya, S. Yuan, E. Bene- tos, and J. D. Reiss, “ST-ITO: Controlling audio effects for style transfer with inference-time optimization,” inProc. International Society for Music Information Retrieval (ISMIR), 2024

    2024

  5. [13]

    Ito-master: Inference-time optimization for audio effects modeling of music mastering processors,

    J. Koo, M. A. Mart ´ınez-Ram´ırez, W.-H. Liao, G. Fabbro, M. Mancusi, and Y . Mitsufuji, “Ito-master: Inference-time optimization for audio effects modeling of music mastering processors,” inProc. International Society for Music Information Retrieval (ISMIR), 2025

    2025

  6. [14]

    Style transfer of audio effects with differentiable signal processing,

    C. J. Steinmetz, N. J. Bryan, and J. D. Reiss, “Style transfer of audio effects with differentiable signal processing,”Journal of the Audio Engineering Society, 2022

    2022

  7. [15]

    Towards zero-shot amplifier modeling: One-to-many amplifier modeling via tone embedding control,

    Y .-H. Chen, Y .-T. Yeh, Y .-C. Cheng, J.-T. Wu, Y .-H. Ho, J.-S. R. Jang, and Y .-H. Yang, “Towards zero-shot amplifier modeling: One-to-many amplifier modeling via tone embedding control,” inProc. International Society for Music Information Retrieval (ISMIR), 2024

    2024

  8. [16]

    Distortion audio effects: Learning how to recover the clean signal,

    J. Imort, G. Fabbro, M. A. M. Ram ´ırez, S. Uhlich, Y . Koyama, and Y . Mitsufuji, “Distortion audio effects: Learning how to recover the clean signal,” inProc. International Society for Music Information Retrieval (ISMIR), 2022

    2022

  9. [17]

    Distortion recovery: A two-stage method for guitar effect removal,

    Y .-S. Lee, Y .-P. Peng, J.-T. Wu, M. Cheng, L. Su, and Y .-H. Yang, “Distortion recovery: A two-stage method for guitar effect removal,” in Proc. International Conference on Digital Audio Effects (DAFx), 2024

    2024

  10. [18]

    General purpose audio effect removal,

    M. Rice, C. J. Steinmetz, G. Fazekas, and J. D. Reiss, “General purpose audio effect removal,” inProc. WASPAA, 2023

    2023

  11. [19]

    Posterior variance-parameterised gaussian dropout: Improving disentangled sequential autoencoders for zero-shot voice conversion,

    Y .-J. Luo and S. Dixon, “Posterior variance-parameterised gaussian dropout: Improving disentangled sequential autoencoders for zero-shot voice conversion,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024

    2024

  12. [20]

    Automated physical modeling of nonlinear audio circuits for real-time audio effects,

    D. T. Yeh, J. S. Abel, and J. O. Smith, “Automated physical modeling of nonlinear audio circuits for real-time audio effects,”IEEE Trans. Audio, Speech, Lang. Process., 2010

    2010

  13. [21]

    A review of digital techniques for modeling vacuum-tube guitar amplifiers,

    J. Pakarinen and D. T. Yeh, “A review of digital techniques for modeling vacuum-tube guitar amplifiers,”Computer Music Journal, 2009

    2009

  14. [22]

    DDSP: Differentiable digital signal processing,

    J. Engel, L. Hantrakul, C. Gu, and A. Roberts, “DDSP: Differentiable digital signal processing,” inProc. International Conference on Learning Representations (ICLR), 2020

    2020

  15. [23]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,

    J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” inProc. NeurIPS, 2020

    2020

  16. [24]

    Autovc: Zero-shot voice style transfer with only autoencoder loss,

    K. Qian, Y . Zhang, S. Chang, X. Yang, and M. Hasegawa-Johnson, “Autovc: Zero-shot voice style transfer with only autoencoder loss,” in Proc. ICML, 2019

    2019

  17. [25]

    Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, Y . Liu, Y . Leng, K. Song, S. Tanget al., “Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models,” inProc. ICML, 2024

    2024

  18. [26]

    Vevo: Controllable zero-shot voice imitation with self-supervised disentanglement,

    X. Zhang, X. Zhang, K. Peng, Z. Tang, V . Manohar, Y . Liu, J. Hwang, D. Li, Y . Wang, J. Chanet al., “Vevo: Controllable zero-shot voice imitation with self-supervised disentanglement,” inProc. International Conference on Learning Representations (ICLR), 2025

    2025

  19. [27]

    Vevo2: A unified and controllable framework for speech and singing voice generation,

    X. Zhang, J. Zhang, Y . Wang, C. Wang, Y . Chen, D. Jia, Z. Chen, and Z. Wu, “Vevo2: A unified and controllable framework for speech and singing voice generation,”arXiv preprint arXiv:2508.16332, 2025

    2025

  20. [28]

    Diffusion timbre transfer via mutual information guided inpainting,

    C. H. Lee, J. Nistal, S. Lattner, M. Pasini, and G. Fazekas, “Diffusion timbre transfer via mutual information guided inpainting,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2026

    2026

  21. [29]

    Wavetransfer: A flexible end-to-end multi-instrument timbre transfer with diffusion,

    T. Baoueb, X. Bie, H. Janati, and G. Richard, “Wavetransfer: A flexible end-to-end multi-instrument timbre transfer with diffusion,” inIEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP), 2024

    2024

  22. [30]

    Disentangled sequential autoencoder,

    Y . Li and S. Mandt, “Disentangled sequential autoencoder,” inProc. ICML, 2018

    2018

  23. [31]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,”arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  24. [32]

    beta-vae: Learning basic visual concepts with a constrained variational framework

    I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework.” inProc. International Conference on Learning Representations (ICLR), 2017

    2017

  25. [33]

    Movie gen: A cast of media foundation models,

    A. Polyak, A. Zohar, A. Brown, A. Tjandra, A. Sinha, A. Lee, A. Vyas, B. Shi, C.-Y . Ma, C.-Y . Chuanget al., “Movie gen: A cast of media foundation models,”arXiv preprint arXiv:2410.13720, 2024

    2024 arXiv

  26. [34]

    High- fidelity audio compression with improved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High- fidelity audio compression with improved rvqgan,” inProc. NeurIPS, 2023

    2023

  27. [35]

    Adaspeech: Adaptive text to speech for custom voice,

    M. Chen, X. Tan, B. Li, Y . Liu, T. Qin, S. Zhao, and T.-Y . Liu, “Adaspeech: Adaptive text to speech for custom voice,” inProc. In- ternational Conference on Learning Representations (ICLR), 2021

    2021

  28. [36]

    Learning disentangled rep- resentations of timbre and pitch for musical instrument sounds using gaussian mixture variational autoencoders,

    Y .-J. Luo, K. Agres, and D. Herremans, “Learning disentangled rep- resentations of timbre and pitch for musical instrument sounds using gaussian mixture variational autoencoders,” inProc. International Soci- ety for Music Information Retrieval (ISMIR), 2019

    2019

  29. [37]

    Neural networks fail to learn periodic functions and how to fix it,

    L. Ziyin, T. Hartwig, and M. Ueda, “Neural networks fail to learn periodic functions and how to fix it,”Advances in Neural Information Processing Systems, 2020

    2020

  30. [38]

    Layer normalization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,”arXiv preprint arXiv:1607.06450, 2016

    2016 arXiv

  31. [39]

    Neural analysis and synthesis: Reconstructing speech from self-supervised representations,

    H.-S. Choi, J. Lee, W. Kim, J. Lee, H. Heo, and K. Lee, “Neural analysis and synthesis: Reconstructing speech from self-supervised representations,” inProc. NeurIPS, 2021

    2021

  32. [40]

    Arbitrary style transfer in real-time with adaptive instance normalization,

    X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” inProc. IEEE International Confer- ence on Computer Vision, 2017, pp. 1501–1510

    2017

  33. [41]

    Synthcloner: Synthesizer-style audio transfer via factorized codec with adsr envelope control,

    J.-Y . Liu, T.-C. Hsu, Y .-T. Yeh, L. Su, and Y .-H. Yang, “Synthcloner: Synthesizer-style audio transfer via factorized codec with adsr envelope control,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2026

    2026

  34. [42]

    Speaker generation,

    D. Stanton, M. Shannon, S. Mariooryad, R. Skerry-Ryan, E. Battenberg, T. Bagby, and D. Kao, “Speaker generation,” inProc. International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2022

    2022

  35. [43]

    Hierarchical generative modeling for controllable speech synthesis,

    W.-N. Hsu, Y . Zhang, R. J. Weiss, H. Zen, Y . Wu, Y . Wang, Y . Cao, Y . Jia, Z. Chen, J. Shenet al., “Hierarchical generative modeling for controllable speech synthesis,” inProc. International Conference on Learning Representations (ICLR), 2018

    2018

  36. [44]

    Hybrid spectrogram and waveform source separation,

    A. D ´efossez, “Hybrid spectrogram and waveform source separation,” arXiv preprint arXiv:2111.03600, 2021

    2021 arXiv

  37. [45]

    Phase-aware speech enhancement with deep complex U-Net,

    H.-S. Choi, J.-H. Kim, J. Huh, A. Kim, J.-W. Ha, and K. Lee, “Phase-aware speech enhancement with deep complex U-Net,” inProc. International Conference on Learning Representations (ICLR), 2018

    2018

  38. [46]

    Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,

    J. Chen, Q. Mao, and D. Liu, “Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,” in Proc. Interspeech, 2020

    2020

  39. [47]

    Bigvgan: A universal neural vocoder with large-scale training,

    S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “Bigvgan: A universal neural vocoder with large-scale training,” inProc. International Conference on Learning Representations (ICLR), 2022

    2022

  40. [48]

    A style-based generator architecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019

    2019

  41. [49]

    Visualizing data using t-SNE

    L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE.” Journal of machine learning research, 2008

    2008

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.