REVIEW 21 cited by
Autoregressive Image Generation without Vector Quantization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Conventional wisdom holds that autoregressive models for image generation are typically accompanied by vector-quantized tokens. We observe that while a discrete-valued space can facilitate representing a categorical distribution, it is not a necessity for autoregressive modeling. In this work, we propose to model the per-token probability distribution using a diffusion procedure, which allows us to apply autoregressive models in a continuous-valued space. Rather than using categorical cross-entropy loss, we define a Diffusion Loss function to model the per-token probability. This approach eliminates the need for discrete-valued tokenizers. We evaluate its effectiveness across a wide range of cases, including standard autoregressive models and generalized masked autoregressive (MAR) variants. By removing vector quantization, our image generator achieves strong results while enjoying the speed advantage of sequence modeling. We hope this work will motivate the use of autoregressive generation in other continuous-valued domains and applications. Code is available at: https://github.com/LTH14/mar.
Forward citations
Cited by 21 Pith papers
-
MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation
Distilling frozen Motion-JEPA features into a compact 32-D latent whose geometry is coupled to the decoder lets a standard non-autoregressive flow-matching DiT reach state-of-the-art text-to-motion quality on HumanML3...
-
World Modeling with Probabilistic Structure Integration
A single probabilistic video model extracts optical flow, depth, and segments via counterfactual prompts, then integrates those structures as new token types to improve its own video predictions.
-
Hita: Holistic Tokenizer for Autoregressive Image Generation
Hita's holistic-to-local tokenization lets vanilla autoregressive image models generate global tokens first, improving FID, convergence, and enabling zero-shot style transfer and inpainting.
-
Token Radius Attention for Efficient Video Generation
Video diffusion transformers can run ~1.5-2x faster with competitive quality by converting each query's attention entropy into a spatially decayed retention radius instead of dense attention.
-
Beyond Token-Level Cross-Entropy: Fr\'echet Distributional Post-Training for Autoregressive Image Generation
FD-loss post-training with detached rollout replay and a probability-level straight-through estimator improves FID and FD_r6 across eight ImageNet configurations.
-
SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation
SRC-Flow compresses RAE features via a Semantic Representation Compressor into a low-dimensional space, enabling normalizing flows to reach gFID 1.65 on ImageNet 256x256 and 2.07 on 512x512 while retaining exact likelihoods.
-
ELT: Elastic Looped Transformers for Visual Generation
Weight-shared looped transformers trained with intra-loop self-distillation match MaskGIT-class FID/FVD at roughly 4x fewer parameters and support any-time inference across loop counts.
-
InertialAR: Autoregressive 3D Molecule Generation with Inertial Frames
An autoregressive transformer with inertial-frame tokenization and geometric rotary positional encoding reports state-of-the-art validity and stability on QM9, GEOM-Drugs, and B3LYP, plus strong functional-group-condi...
-
Scalable Training for Vector-Quantized Networks with 100% Codebook Utilization
A training-only ViT-based projector, VQBridge, combined with learning annealing, achieves full codebook utilization in vector-quantized networks at large codebook sizes, improving reconstruction and autoregressive ima...
-
Missing Fine Details in Images: Last Seen in High Frequencies
A wavelet-based VAE that trains low- and high-frequency branches separately improves image reconstruction and diffusion generation.
-
Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing
VARIN uses a Location-aware Argmax Inversion pseudo-inverse of Gumbel-max sampling to extract editable discrete noises, enabling training-free prompt-guided editing for visual autoregressive models.
-
ROVI: A VLM-LLM Re-Captioned Dataset for Open-Vocabulary Instance-Grounded Text-to-Image Generation
ROVI's pre-detection VLM-LLM re-captioning yields richer open-vocabulary box labels on 1M curated images, and a GLIGEN model trained on ROVI improves instance grounding, prompt fidelity, and aesthetic quality in the p...
-
Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction
A causal audio language model with continuous-valued tokens and masked next-token prediction matches diffusion-based text-to-audio quality with smaller, streamable models.
-
CaO$_2$: Rectifying Inconsistencies in Diffusion-Based Dataset Distillation
CaO2 selects confident diffusion-generated samples and optimizes their latents against the denoising objective, achieving state-of-the-art distilled-dataset accuracy on ImageNet subsets.
-
StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling
StreamMel interleaves text tokens with continuous mel frames in one autoregressive model, reaching state-of-the-art streaming latency with quality comparable to offline zero-shot TTS on LibriSpeech.
-
STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
A latent-space transformer autoregressive flow with one deep block plus shallow refiners, tuned noise injection, and score-based guidance reaches competitive FID in high-resolution image synthesis, the first at this s...
-
CLEAR: Continuous Latent Autoregressive Modeling for High-quality and Low-latency Speech Synthesis
CLEAR is a zero-shot TTS model that autoregressively predicts compact continuous audio latents with a per-token rectified flow head, reaching 1.88% WER on LibriSpeech Subset-B with an RTF of 0.29 and a 96 ms streaming delay.
-
Parameterized Diffusion Optimization enabled Autoregressive Ordinal Regression for Diabetic Retinopathy Grading
AOR-DR decomposes DR severity grading into conditional binary steps modeled by a diffusion decoder and reports higher accuracy and macro-F1 than six ordinal regression methods on four datasets.
-
Real-Time 3D Vision-Language Embedding Mapping
Combining local embedding masking with confidence-weighted 3D integration yields, the paper claims, a real-time metric-accurate 3D map of vision-language embeddings for language-guided object localization.
-
SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping
SkipVAR selects, per sample, between step skipping and unconditional branch replacement using handcrafted frequency features and a trained logistic regression, to accelerate visual autoregressive generation.
-
MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation
An empirical testbed shows that allocating more transformer layers to autoregressive conditioning improves FID under low inference budgets, while block size should scale with image resolution.
Discussion (0). Sign in to comment.