Pith. sign in

REVIEW 17 cited by

Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.13863 v1 pith:FPJNFMHJ submitted 2024-10-17 cs.CV cs.LG

Lijie Fan , Tianhong Li , Siyang Qin , Yuanzhen Li , Chen Sun , Michael Rubinstein , Deqing Sun , Kaiming He
show 1 more author
Yonglong Tian
This is my paper
classification cs.CVcs.LG
keywords modelstokenscontinuousgenevalscalingautoregressivefluidscore
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Scaling up autoregressive models in vision has not proven as beneficial as in large language models. In this work, we investigate this scaling problem in the context of text-to-image generation, focusing on two critical factors: whether models use discrete or continuous tokens, and whether tokens are generated in a random or fixed raster order using BERT- or GPT-like transformer architectures. Our empirical results show that, while all models scale effectively in terms of validation loss, their evaluation performance -- measured by FID, GenEval score, and visual quality -- follows different trends. Models based on continuous tokens achieve significantly better visual quality than those using discrete tokens. Furthermore, the generation order and attention mechanisms significantly affect the GenEval score: random-order models achieve notably better GenEval scores compared to raster-order models. Inspired by these findings, we train Fluid, a random-order autoregressive model on continuous tokens. Fluid 10.5B model achieves a new state-of-the-art zero-shot FID of 6.16 on MS-COCO 30K, and 0.69 overall score on the GenEval benchmark. We hope our findings and results will encourage future efforts to further bridge the scaling gap between vision and language models.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RAE-NWM: Navigation World Model in Dense Visual Representation Space

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Navigation world models trained in dense DINOv2 space with flow-matching CDiT-DH and time-gated action injection improve structural stability and planning over VAE baselines.

  2. HD-Prot: A Protein Language Model for Joint Sequence-Structure Modeling with Continuous Structure Tokens

    cs.CE 2025-12 conditional novelty 6.0 of 10

    HD-Prot shows that a protein language model can jointly generate sequences and structures using continuous structure tokens instead of quantized tokens, reaching competitive performance on four protein design tasks.

  3. MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A semantically aligned latent space plus multi-token cross-attention conditioning sets a new state of the art in text-to-human-motion generation on HumanML3D.

  4. A Unified Low-level Foundation Model for Enhancing Pathology Image Quality

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A prompt-guided diffusion model pretrained on 190 million pathology patches outperforms task-specific models across most restoration and virtual staining benchmarks.

  5. HPSv3: Towards Wide-Spectrum Human Preference Score

    cs.CV 2025-08 conditional novelty 6.0 of 10

    HPSv3, trained on the new 1.08M-pair HPDv3 dataset, reaches 76.9% pairwise preference accuracy on its own test set and Spearman 0.94 against human model rankings, and is used to iteratively refine generated images (CoHP).

  6. PixNerd: Pixel Neural Field Diffusion

    cs.CV 2025-07 conditional novelty 6.0 of 10

    PixNerd is a single-stage pixel-space diffusion transformer that uses predicted neural field weights to decode large patches, reaching 2.15 FID on ImageNet 256 without a VAE.

  7. Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesis

    cs.CV 2025-07 conditional novelty 6.0 of 10

    DisCon treats discrete image tokens as conditioning signals rather than targets, letting a continuous autoregressive model refine details and reach gFID 1.38 on ImageNet-256.

  8. Transition Matching: Scalable and Flexible Generative Modeling

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Transition Matching unifies flow matching and continuous autoregressive generation as discrete-time Markov processes, with three variants that improve text-to-image quality and speed.

  9. Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    1D binary image latents reduce a 1024x1024 image to 128 discrete tokens and support text-to-image generation with diffusion and autoregressive models.

  10. VideoMAR: Autoregressive Video Generatio with Continuous Tokens

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A decoder-only autoregressive video model with continuous tokens, frame-wise causal attention, and a next-frame diffusion loss reports a higher VBench-I2V score than Cosmos I2V with a much smaller model and dataset.

  11. STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A latent-space transformer autoregressive flow with one deep block plus shallow refiners, tuned noise injection, and score-based guidance reaches competitive FID in high-resolution image synthesis, the first at this s...

  12. Plug-and-Play Context Feature Reuse for Efficient Masked Generation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    ReCAP interleaves full model evaluations with lightweight steps that reuse cached context features, delivering up to 2.4x faster masked generation with minimal FID loss.

  13. When Models Learn to Ask Why: Adaptive Causal Reasoning for Trustworthy Medical Vision-Language Models

    cs.AI 2026-03 conditional novelty 5.0 of 10

    MedCausalX adds adaptive causal reflection tokens and trajectory-level RL on a new CRMed dataset to cut medical VLM hallucinations and raise diagnostic consistency.

  14. DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer

    cs.CV 2025-07 conditional novelty 5.0 of 10

    DC-AR generates 512x512 images in 12 masked autoregressive steps plus 20 diffusion refinement steps, using a 32x compressed 2D tokenizer, and reports gFID 5.49 on MJHQ-30K.

  15. IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling

    eess.AS 2025-05 conditional novelty 5.0 of 10

    On AudioCaps, IMPACT reports the best Fréchet Distance and Fréchet Audio Distance among the compared systems while generating audio faster than diffusion baselines.

  16. Hierarchical Masked Autoregressive Models with Low-Resolution Token Pivots

    cs.CV 2025-05 conditional novelty 5.0 of 10

    Hi-MAR adds a low-resolution token prediction phase and a diffusion transformer head to masked autoregressive image generation, improving FID and cutting autoregressive steps.

  17. StyleAR: Customizing Multimodal Autoregressive Model for Style-Aligned Text-to-Image Generation

    cs.CV 2025-05 reject novelty 4.0 of 10

    StyleAR enables autoregressive image generation models to do style-aligned text-to-image generation using only binary text-image data, via self-reconstruction training and style-enhanced tokens.

Pith tools