REVIEW 19 cited by
Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
The Open-MAGVIT2 project produces an open-source replication of Google's MAGVIT-v2 tokenizer, a tokenizer with a super-large codebook (i.e., $2^{18}$ codes), and achieves the state-of-the-art reconstruction performance on ImageNet and UCF benchmarks. We also provide a tokenizer pre-trained on large-scale data, significantly outperforming Cosmos on zero-shot benchmarks (1.93 vs. 0.78 rFID on ImageNet original resolution). Furthermore, we explore its application in plain auto-regressive models to validate scalability properties, producing a family of auto-regressive image generation models ranging from 300M to 1.5B. To assist auto-regressive models in predicting with a super-large vocabulary, we factorize it into two sub-vocabulary of different sizes by asymmetric token factorization, and further introduce ``next sub-token prediction'' to enhance sub-token interaction for better generation quality. We release all models and codes to foster innovation and creativity in the field of auto-regressive visual generation.
Forward citations
Cited by 19 Pith papers
-
Generative Refinement Networks for Visual Synthesis
Hierarchical Binary Quantization plus global refinement AR yields 0.56 rFID reconstruction and 1.81 gFID class-conditional generation on ImageNet, with competitive T2I/T2V at 2B scale.
-
Hita: Holistic Tokenizer for Autoregressive Image Generation
Hita's holistic-to-local tokenization lets vanilla autoregressive image models generate global tokens first, improving FID, convergence, and enabling zero-shot style transfer and inpainting.
-
Interspatial Attention for Efficient 4D Human Video Generation
A new symmetric 3D-to-2D attention mechanism with relative positional encodings, plus a motion-tuned video VAE, improves controllable 4D human video generation.
-
Revisiting Autoregressive Models for Generative Image Classification
Order-marginalized any-order AR models (RandAR) outperform diffusion generative classifiers on ImageNet and OOD sets and match strong SSL models at far lower cost.
-
Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning
PointRSP recursively partitions a point cloud's KNN graph into a non-balanced binary tree and autoregressively generates cluster structure and residual features, improving 1-NNA on ShapeNet.
-
UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
An MLLM-conditioned next-scale VAR decoder handles 15+ unified visual generation tasks with competitive quality and substantially lower latency than diffusion baselines.
-
IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction
IAR2 achieves state-of-the-art ImageNet 256×256 image generation (FID 1.50 with rejection sampling) by splitting visual tokens into semantic and detail codes and predicting them hierarchically with a local-context-awa...
-
Scalable Training for Vector-Quantized Networks with 100% Codebook Utilization
A training-only ViT-based projector, VQBridge, combined with learning annealing, achieves full codebook utilization in vector-quantized networks at large codebook sizes, improving reconstruction and autoregressive ima...
-
SpectralAR: Spectral Autoregressive Visual Generation
An autoregressive image generator that tokenizes images in the DCT frequency domain into nested 1D spectral sequences and generates them coarse-to-fine, reaching 3.02 gFID with 64 tokens on ImageNet-1K.
-
Humanoid World Models: Open World Foundation Models for Humanoid Robotics
Masked-transformers trained on humanoid video forecast future frames with better FID than flow-matching models, and parameter sharing cut model size 33-53% with minimal quality loss.
-
TokBench: Evaluating Your Visual Tokenizer before Visual Generation
TokBench measures text recognition accuracy and face similarity on reconstructed images and videos across 16 tokenizers, showing small text and faces are poorly preserved and often missed by traditional metrics.
-
Training-Free Watermarking for Autoregressive Image Generation
IndexMark watermarks images from autoregressive models by replacing similar codebook tokens with green tokens, then detecting the green-token rate after reconstruction.
-
GVC-RT: Towards Real-Time Generative Video Compression at Ultra-Low Bitrates
A real-time generative video codec that keeps the perceptual benefits of generative compression at ultra-low bitrates while running fast enough for live 1080p use.
-
Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation
An audio-visual language model that adds full-face visual features to a pre-trained expressive speech model improves emotion recognition and expressive speech generation by a few F1 points over speech-only on syntheti...
-
Near-Field Variable-Width Beam Coverage and Codebook Design for XL-RIS
An XL-RIS near-field algorithm generates variable-width beams that cover arbitrarily shaped regions and feeds joint multi-XL-RIS codebooks, claiming higher spectral efficiency and lower outage in simulation.
-
VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers
A convolutional residual VQ-VAE action tokenizer trained on over 100x more data than prior work improves OpenVLA success rates and inference speed on several manipulation tasks.
-
CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation
CycleVAR adapts a pretrained visual autoregressive model to unpaired image translation using softmax-relaxed quantization and source-token prefixes, achieving FID scores competitive with CycleGAN-Turbo.
-
Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression
ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.
-
MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
Splitting quantization across multiple small sub-codebooks with nested masking raises VQ-VAE reconstruction fidelity, giving MGVQ rFID 0.49 and PSNR 24.70 on ImageNet at 16 times downsampling.
Discussion (0). Sign in to comment.