Pith. sign in

REVIEW 7 cited by

An Image is Worth 32 Tokens for Reconstruction and Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.07550 v1 pith:CFOQFUAZ submitted 2024-06-11 cs.CV

Qihang Yu , Mark Weber , Xueqing Deng , Xiaohui Shen , Daniel Cremers , Liang-Chieh Chen This is my paper
classification cs.CV
keywords titokimageimageslatenttokensgenerationgfidbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advancements in generative models have highlighted the crucial role of image tokenization in the efficient synthesis of high-resolution images. Tokenization, which transforms images into latent representations, reduces computational demands compared to directly processing pixels and enhances the effectiveness and efficiency of the generation process. Prior methods, such as VQGAN, typically utilize 2D latent grids with fixed downsampling factors. However, these 2D tokenizations face challenges in managing the inherent redundancies present in images, where adjacent regions frequently display similarities. To overcome this issue, we introduce Transformer-based 1-Dimensional Tokenizer (TiTok), an innovative approach that tokenizes images into 1D latent sequences. TiTok provides a more compact latent representation, yielding substantially more efficient and effective representations than conventional techniques. For example, a 256 x 256 x 3 image can be reduced to just 32 discrete tokens, a significant reduction from the 256 or 1024 tokens obtained by prior methods. Despite its compact nature, TiTok achieves competitive performance to state-of-the-art approaches. Specifically, using the same generator framework, TiTok attains 1.97 gFID, outperforming MaskGIT baseline significantly by 4.21 at ImageNet 256 x 256 benchmark. The advantages of TiTok become even more significant when it comes to higher resolution. At ImageNet 512 x 512 benchmark, TiTok not only outperforms state-of-the-art diffusion model DiT-XL/2 (gFID 2.74 vs. 3.04), but also reduces the image tokens by 64x, leading to 410x faster generation process. Our best-performing variant can significantly surpasses DiT-XL/2 (gFID 2.13 vs. 3.04) while still generating high-quality samples 74x faster.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Single-pass Adaptive Image Tokenization for Minimum Program Search

    cs.CV 2025-07 conditional novelty 7.0 of 10

    KARL conditions a tokenizer on a target reconstruction loss and learns halting probabilities that produce an adaptive token count in a single forward pass.

  2. Hita: Holistic Tokenizer for Autoregressive Image Generation

    cs.CV 2025-07 conditional novelty 7.0 of 10

    Hita's holistic-to-local tokenization lets vanilla autoregressive image models generate global tokens first, improving FID, convergence, and enabling zero-shot style transfer and inpainting.

  3. Segment This Thing: Foveated Tokenization for Efficient Point-Prompted Segmentation

    cs.CV 2025-06 conditional novelty 7.0 of 10

    A point-prompted segmentation model gains efficiency by foveated tokenization, cutting tokens from 4096 to 172 while staying competitive on mIoU benchmarks.

  4. Scalable Training for Vector-Quantized Networks with 100% Codebook Utilization

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-only ViT-based projector, VQBridge, combined with learning annealing, achieves full codebook utilization in vector-quantized networks at large codebook sizes, improving reconstruction and autoregressive ima...

  5. Is Visual in-Context Learning for Compositional Medical Tasks within Reach?

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Training on synthetic compositional task sequences with sequence-level masking lets a transformer-based in-context learner follow multi-step medical imaging instructions on held-out images, but well below codebook upp...

  6. Instella-T2I: Pushing the Limits of 1D Discrete Latent Space Image Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    1D binary image latents reduce a 1024x1024 image to 128 discrete tokens and support text-to-image generation with diffusion and autoregressive models.

  7. CoDiCodec: Unifying Continuous and Discrete Compressed Representations of Audio

    cs.SD 2025-09 conditional novelty 5.0 of 10

    CoDiCodec unifies continuous and discrete audio compression in one consistency-trained autoencoder, using FSQ-dropout to serve both continuous ~11 Hz embeddings and 2.38 kbps discrete tokens.

Pith tools