REVIEW 14 cited by
Mustango: Toward Controllable Text-to-Music Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The quality of the text-to-music models has reached new heights due to recent advancements in diffusion models. The controllability of various musical aspects, however, has barely been explored. In this paper, we propose Mustango: a music-domain-knowledge-inspired text-to-music system based on diffusion. Mustango aims to control the generated music, not only with general text captions, but with more rich captions that can include specific instructions related to chords, beats, tempo, and key. At the core of Mustango is MuNet, a Music-Domain-Knowledge-Informed UNet guidance module that steers the generated music to include the music-specific conditions, which we predict from the text prompt, as well as the general text embedding, during the reverse diffusion process. To overcome the limited availability of open datasets of music with text captions, we propose a novel data augmentation method that includes altering the harmonic, rhythmic, and dynamic aspects of music audio and using state-of-the-art Music Information Retrieval methods to extract the music features which will then be appended to the existing descriptions in text format. We release the resulting MusicBench dataset which contains over 52K instances and includes music-theory-based descriptions in the caption text. Through extensive experiments, we show that the quality of the music generated by Mustango is state-of-the-art, and the controllability through music-specific text prompts greatly outperforms other models such as MusicGen and AudioLDM2.
Forward citations
Cited by 14 Pith papers
-
Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Model
Diff-Symbo generates long, text-controlled symbolic music by autoregressively extending 8-bar latent diffusion segments conditioned on the previous segment's latent.
-
Unified Audio Intelligence Without Regressing on Text Intelligence
A unified 30B MoE audio-text LLM achieves state-of-the-art audio understanding, generation, and speech tasks while preserving text reasoning comparable to its text-only backbone.
-
AImoclips: A Benchmark for Evaluating Emotion Conveyance in Text-to-Music Generation
AImoclips is a new open benchmark showing that text-to-music systems convey high-arousal emotions better than low-arousal ones and that all models converge toward emotionally neutral music.
-
JAM: A Tiny Flow-based Song Generator with Fine-grained Controllability and Aesthetic Alignment
JAM is a 530M-parameter flow-matching song generator that adds word- and phoneme-level timing control and duration control, achieving strong lyric fidelity and musicality scores when ground-truth timings are provided.
-
Controllable Video-to-Music Generation with Multiple Time-Varying Conditions
A two-stage video-to-music model with four time-varying controls (rhythm, melody, intensity, emotion) claims better controllability and alignment than prior V2M systems.
-
DiffRhythm+: Controllable and Flexible Full-Length Song Generation with Preference Optimization
DiffRhythm+ improves full-length lyric-to-song generation via balanced data scaling, MuLan-based multimodal style control, and DPO fine-tuning guided by automated aesthetic scorers.
-
MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners
A lightweight adapter that adds rotary position embeddings to decoupled cross-attention enables efficient time-varying style control and audio inpainting/outpainting for text-to-music diffusion Transformers.
-
Video-Guided Text-to-Music Generation Using Public Domain Movie Collections
OSSL is the first self-hosted, mood-annotated video-music dataset, and a video adapter on MusicGen-Medium improves film music generation over text-only baselines.
-
DanceChat: Large Language Model-Guided Music-to-Dance Generation
An LLM-generated text choreography, fused with music and beat features, guides a diffusion model to produce more diverse and physically plausible dance motion, with a multi-modal alignment loss intended to bridge musi...
-
AudioGenie: A Training-Free Multi-Agent Framework for Diverse Multimodality-to-Multiaudio Generation
A training-free multi-agent framework that decomposes multimodal inputs into audio events, selects specialized generators, and self-corrects outputs to produce multiple audio types.
-
A Survey on Evaluation Metrics for Music Generation
A taxonomy and critical review of evaluation metrics for music generation, identifying gaps such as weak correlation with human perception and lack of standardization.
-
Segment Transformer: AI-Generated Music Detection via Music Structural Analysis
A two-stage transformer framework classifies AI-generated music from short clips and beat-segmented full tracks, reporting 99.9% accuracy on SONICS without releasing code or ablations.
-
TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization
TinyMusician distills MusicGen and applies hand-picked mixed-precision quantization to make a 1.04 GB on-device music generator, but the headline '93% quality, 55% smaller' claims conflict with the paper's own tables.
-
Genre Controlled Music Generation via Activation Steering
Activation steering with linear probe weights on MusicGen's residual stream shifts generated music between genres at inference time, outperforming text prompting in CLAP and listener preference but with incomplete reporting.
Discussion (0). Continue with ORCID to comment.