REVIEW 32 cited by
Muse: Text-To-Image Generation via Masked Generative Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io
Forward citations
Cited by 32 Pith papers
-
Dimension-Free Convergence of Discrete Diffusion Models: Adjoint Equations Induce the Right Space
Using adjoint equations, the paper derives S-independent IPM convergence bounds for masked and uniform discrete diffusion, controlled by score-matching losses.
-
MentalThink: Shaping Thoughts in Mental SVG World
MLLMs that generate and render SVG sketches as multi-turn intermediate reasoning steps reach 55.1% on VSIBench and 76.0% on MindCube, far above the Qwen2.5-VL-7B backbone.
-
UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Models
UDM-GRPO is the first RL integration for uniform discrete diffusion models, using final clean samples as actions and forward-process trajectory reconstruction to raise GenEval accuracy from 69% to 96% and OCR accuracy...
-
MVGBench: Comprehensive Benchmark for Multi-view Generation Models
MVGBench evaluates multi-view generators through self-consistency of 3D reconstructions and uses this protocol to rank 12 models and build a better one.
-
Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
Sparse-LaViDa speeds up masked diffusion LLMs by truncating masked tokens, using register tokens and a step-causal attention mask, reaching ~2-2.8x speedup over LaViDa-O with comparable generation quality.
-
RubricRL: Simple Generalizable Rewards for Text-to-Image Generation
Using an LLM to generate prompt-specific visual rubrics and grade each criterion independently gives a more interpretable reward that improves text-to-image model alignment beyond composite and learned scalar rewards.
-
IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction
IAR2 achieves state-of-the-art ImageNet 256×256 image generation (FID 1.50 with rejection sampling) by splitting visual tokens into semantic and detail codes and predicting them hierarchically with a local-context-awa...
-
Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
Lavida-O introduces an elastic mixture-of-transformers architecture that brings high-resolution text-to-image generation, object grounding, and image editing into a single masked diffusion model, using planning and se...
-
Layout-Conditioned Autoregressive Text-to-Image Generation via Structured Masking
SMARLI achieves strong layout control in autoregressive text-to-image generation via structured attention masks and GRPO post-training with a CLIP-based layout reward.
-
Learning neuro-symbolic convergent term rewriting systems
Two modular neuro-symbolic systems learn to simplify formulas by imitating term rewriting steps, and the new FastNRS variant generalizes to deeper formulas while being far faster than the original.
-
MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing
A masking-augmented diffusion objective plus pause-token inference scaling modestly improves instruction adherence and source preservation for OmniGen-based image editing.
-
CI-VID: A Coherent Interleaved Text-Video Dataset
CI-VID provides 341,550 interleaved text-video sequences with individual and transition captions and shows initial evidence that fine-tuning on them improves coherent multi-scene video generation.
-
Is Visual in-Context Learning for Compositional Medical Tasks within Reach?
Training on synthetic compositional task sequences with sequence-level masking lets a transformer-based in-context learner follow multi-step medical imaging instructions on held-out images, but well below codebook upp...
-
Learning golf swing signatures from a single wrist-worn inertial sensor
Training on synthetic wrist-sensor data derived from video pose estimates, the system reconstructs full-body golf kinematics with 5.3 cm joint error and detects swing events with 82.1% accuracy on a new 1,046-swing pr...
-
Noise Consistency Regularization for Improved Subject-Driven Image Synthesis
Adding consistency-to-pretrained and multiplicative-noise consistency losses to fine-tuning improves subject identity and background diversity over DreamBooth on a 30-subject benchmark.
-
HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation
HMAR is an image generator that builds each resolution scale from the previous scale and refines it with masked prediction, matching or improving ImageNet FID/IS versus VAR with faster training and inference.
-
Native-Resolution Image Synthesis
A single diffusion transformer trained on native-resolution ImageNet achieves state-of-the-art FID at 256 and 512, and extrapolates to 1024 and 1536 with moderate degradation.
-
Semantics-Aware Human Motion Generation from Audio Instructions
An end-to-end audio-conditioned model generates 3D human motion from spoken instructions, with performance close to text-conditioned baselines on synthetic speech datasets.
-
MARch\'e: Fast Masked Autoregressive Image Generation with Cache-Aware Attention
MARche accelerates masked autoregressive image generation by caching stable token projections and refreshing only attention-selected tokens, reaching up to 1.72x speedup with some loss in FID.
-
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
A diffusion-based vision-language model matches several autoregressive baselines on multimodal benchmarks while enabling controllable generation and faster decoding at reduced quality.
-
SpecMaskFoley: Steering Pretrained Spectral Masked Generative Transformer Toward Synchronized Video-to-audio Synthesis via ControlNet
A ControlNet branch plus a frequency-aware feature aligner lets a pretrained masked generative TTA model produce video-synchronized foley, beating several from-scratch models on VGGSound.
-
Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model
Forecasting with a non-causal encoder-only model improves fixed-horizon accuracy when the model is asked to output extra future tokens, an effect the authors call delayed chain-of-thought.
-
HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting
HDT forecasts multivariate time series by generating a discrete coarse trend of the future, then generating finer target tokens conditioned on that predicted trend, outperforming prior methods on five datasets.
-
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths
UniMoD prunes tokens with task-specific routers in unified multimodal transformers, cutting training FLOPs by 15-40% while roughly maintaining benchmark performance.
-
The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation
An agentic dialogue-to-video pipeline (ScripterAgent, DirectorAgent, CriticAgent) claims to improve long-horizon cinematic coherence, but its supporting evaluation is partly self-referential.
-
Room Impulse Response Generation Conditioned on Acoustic Parameters
MaskGIT conditioned on acoustic parameters, operating on Descript Audio Codec tokens, generates room impulse responses that outperform StoRIR and FastRIR in objective and MUSHRA evaluations.
-
DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer
DC-AR generates 512x512 images in 12 masked autoregressive steps plus 20 diffusion refinement steps, using a 32x compressed 2D tokenizer, and reports gFID 5.49 on MJHQ-30K.
-
MapBERT: Bitwise Masked Modeling for Real-Time Semantic Mapping Generation
MapBERT uses a lookup-free BitVAE and a BERT-style masked transformer with object-aware masking to generate complete indoor semantic maps from partial observations in real time.
-
IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling
On AudioCaps, IMPACT reports the best Fréchet Distance and Fréchet Audio Distance among the compared systems while generating audio faster than diffusion baselines.
-
E-MD3C: Taming Masked Diffusion Transformers for Efficient Zero-Shot Object Customization
A masked diffusion transformer with a compact condition collector beats the heavier AnyDoor baseline on VITON-HD quality metrics while using a quarter of the parameters and less compute.
-
Zero-Residual Concept Erasure via Progressive Alignment in Text-to-Image Model
A closed-form concept erasure method that enforces zero alignment residual in the optimization objective and applies updates progressively across layers to better preserve generation quality.
-
FBI: Learning Dexterous In-hand Manipulation with Dynamic Visuotactile Shortcut Policy
The abstract claims a new visuotactile robot manipulation policy (FBI) that outperforms baselines, but the manuscript body is an unrelated paper on text-to-image synthesis, so the claimed result is absent.
Discussion (0). Continue with ORCID to comment.