Pith. sign in

REVIEW 4 major objections 5 minor 73 references

TokBench: Evaluating Your Visual Tokenizer before Visual Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new benchmark, TokBench, measures whether visual tokenizers preserve readable text and recognizable faces, and shows that standard metrics like PSNR, SSIM, and LPIPS fail to reflect these critical losses.

desk verdict TokBench is a genuinely useful, honest benchmark for measuring text/face preservation in tokenizers and VAEs, with a transparent protocol and public data; the main caveat is that its OCR/face-recognition proxies are never validated against human perception. read the letter →

arxiv 2505.18142 v2 pith:E56A5G26 submitted 2025-05-23 cs.CV cs.DB

Junfeng Wu , Dongliang Luo , Weizhi Zhao , Zhihao Xie , Yuanhao Wang , Junyi Li , Xudong Xie , Yuliang Liu
show 1 more author
Xiang Bai
This is my paper
classification cs.CVcs.DB
keywords visualtokenizervariationalautoencodertextrecognitionfacereconstructionqualitybenchmarklatentcompressionvideogeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that visual tokenizers and autoencoders, which compress images and videos into latent representations, lose fine-grained detail needed for text and faces, and that this loss sets a ceiling on the quality of any generative model built on those latents. To make this loss measurable, the authors introduce TokBench, a curated collection of text- and face-rich images and videos, and evaluate reconstruction with task-aware metrics: text recognition accuracy (T-ACC), normalized edit distance (T-NED), and face similarity (F-Sim). The paper finds that these metrics track human judgments on text legibility and identity preservation, while traditional pixel- and feature-level metrics like PSNR, SSIM, and LPIPS often contradict them. The result is a lightweight benchmark that can be run before selecting or training a tokenizer for generation.

What carries the argument

The central machinery is a two-part evaluation protocol. First, a curated dataset of 12,398 images and 403 video clips (51,590 frames) rich in text and faces, with instances categorized by relative scale into small, medium, and large difficulty levels. Second, a lightweight pipeline that crops reconstructed regions and feeds them into a pretrained OCR model to compute T-ACC and T-NED, and a pretrained face recognition model to compute cosine-similarity F-Sim. The scale-based difficulty rating, which defines a lower bound for representable characters and faces, is what makes the benchmark sensitive to the small-object failures that global metrics miss.

What would settle it

A human perception study would settle it: if human raters judge reconstructed text as readable while T-ACC stays low, or judge a face as the same identity while F-Sim stays low, across a controlled set of distortions, then the metrics do not track the intended quality. A concrete version is to compare T-ACC and F-Sim against human ratings on a varied set of reconstructed images with different tokenizers and scale levels, and check whether the metrics' rankings align with human rankings.

Watch

Extended reading notes

Core claim

The central claim is that reconstruction quality of visual tokenizers is best assessed by how well they preserve semantically critical fine-grained content, especially text and faces, rather than by global pixel or feature statistics. Using established OCR and face recognition models, the paper defines T-ACC and T-NED for text readability and F-Sim for identity preservation, and demonstrates that these metrics separate tokenizers in ways that PSNR, SSIM, LPIPS, and FID do not reliably do. The experiments show that at small scales even the best continuous tokenizers fall short of a simple resize upper bound, and high-compression discrete tokenizers perform substantially worse, with the gaps widening at higher resolutions. The paper extends the framework to video tokenizers and finds similar patterns in text and face reconstruction, reinforcing the conclusion that standard evaluation metrics are not sufficient for these critical contents.

Load-bearing premise

The benchmark assumes the pretrained OCR and face recognition models give an accurate, unbiased measure of what humans perceive as readable text and preserved identity; if those models are insensitive to certain artifacts or biased toward specific fonts, scales, or color shifts, the scores would misrepresent reconstruction quality.

Editorial extensions

If this is right

  • A tokenizer that scores low on TokBench cannot support high-fidelity generation of text-containing images, such as posters, documents, or UI screenshots, regardless of its FID or PSNR.
  • Because the evaluation runs in minutes with modest memory, it can be inserted into tokenizer training loops as a diagnostic, potentially guiding architecture choices toward better fine-grained reconstruction.
  • The observation that continuous VAEs outperform discrete tokenizers on small text and faces suggests that compression ratio and quantization strategy directly trade off against readable text and recognizable identities, informing future tokenizer design.
  • For video, the frame-by-frame metrics let developers test whether temporal compression further degrades text and faces beyond what spatial compression alone would predict.
  • The benchmark's rank ordering of tokenizers provides a sanity-check upper bound: no tokenizer should be trusted to generate legible text if it cannot beat simple resizing on these metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to use T-ACC and F-Sim as training objectives or reward signals, either through a loss that penalizes tokenizers for failing OCR and face recognition, or by employing these models as discriminators in adversarial training; the paper does not propose this, but the metrics are designed to support such use.
  • The scale-based analysis suggests that other small, dense-textured content, such as barcodes, logos, biometric features, or medical imagery, may be similarly misrepresented by global metrics, so the same methodology could be adapted to those domains.
  • Since the paper relies on OCR and face recognition models as proxies for human perception, a validation step would be a human study comparing subjective legibility and identity judgments with T-ACC and F-Sim; the paper's own limitation section acknowledges that contrast or saturation distortions are not captured, which hints that the proxies are incomplete.
  • The finding that discrete tokenizers collapse on small text at 256 resolution implies that autoregressive visual generation models using such tokenizers will inherit poor text rendering unless they add explicit conditioning or upsampling, a consequence the paper states implicitly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TokBench, a benchmark for evaluating visual tokenizers and VAEs on reconstruction quality for two content types that are visually critical but often poorly captured by existing metrics: text and faces. The authors curate a dataset of 12,398 images and 403 video clips from existing text-spotting, document, and face datasets, and define scale-based difficulty levels. Reconstruction quality is measured with an OCR model (PARSeq) for text, yielding T-ACC and T-NED, and a face recognition model (insightface) for faces, yielding F-Sim. They evaluate a range of discrete and continuous tokenizers at 256, 512, and 1024 resolutions, plus video tokenizers at 256 and 480, and compare against a Resize upper bound. The central claims are that existing pixel- and feature-based metrics (PSNR, SSIM, LPIPS, FID) fail to reflect reconstruction quality for text and faces, and that the proposed task-aware metrics provide an effective complement.

Significance. If valid, TokBench addresses a real gap: fine-grained text and face content is disproportionately important for downstream generation quality, yet standard metrics are insensitive to artifacts that destroy text readability or identity. The benchmark is comprehensive (many tokenizers, multiple resolutions, image and video), lightweight (about 2GB memory and 4 minutes for images), and ships a dataset and evaluation code. The inclusion of a Resize upper bound is good practice. However, the load-bearing claim that T-ACC/T-NED and F-Sim reflect human-perceived reconstruction quality is supported only by qualitative examples and a few contradictory table entries, not by a human study or a quantitative comparison against human judgments. Given that OCR and face-recognition models are deliberately robust to degradations that are visible to humans, the lack of perceptual validation is a substantive limitation. The benchmark is nevertheless a useful diagnostic tool for machine-readability and embedding-level face similarity, and the paper's broader empirical survey of tokenizer performance is valuable.

major comments (4)
  1. [Sections 3.2 and 5] The central claim that T-ACC/T-NED and F-Sim 'effectively reflect' reconstruction quality, while traditional metrics fail, is not validated against human perception. The evidence in Figure 1 is a few hand-picked examples, and the contradictory trends in Table 1 are not quantified. Since the benchmark is proposed as a perceptual proxy, I ask for a human-study validation: collect human ratings on a stratified subset of reconstructed patches (or paired comparisons) and report rank correlations between each proposed metric and human judgments, compared with PSNR/SSIM/LPIPS. Without this, the headline conclusion that the metrics reflect reconstruction quality remains an assumption.
  2. [Section 3.1 and Appendix A.2] The thresholds defining valid instances and difficulty levels are ad hoc: the 5x5 pixel minimum character size, the 25-pixel minimum face size, and the difficulty boundaries in Table 6 are chosen without sensitivity analysis. These thresholds determine which instances enter each difficulty bin, and therefore directly affect every reported T-ACC, T-NED, and F-Sim value. Please provide a sensitivity analysis showing that the relative ranking of tokenizers is stable with respect to these thresholds, or justify them with perceptual data.
  3. [Section 3.2] The choice of PARSeq and insightface as proxies assumes that these models' recognition decisions and embedding similarities align with human readability and identity preservation. These models are trained to be robust to blur, low resolution, lighting, and color variation, so a reconstruction that is visibly degraded may still receive a high F-Sim if the identity-defining structure is preserved. At minimum, characterize the sensitivity of both models to controlled degradations (Gaussian blur, saturation reduction, contrast change) and show that T-ACC and F-Sim respond monotonically to degradation levels that humans notice. The paper's own limitation section acknowledges that contrast/saturation distortions are not captured; the impact of these blind spots on the rank ordering of tokenizers should be quantified.
  4. [Section 4.2 and Table 1] The evidence that 'traditional metrics fail' is anecdotal rather than statistical. The paper cites specific pairwise comparisons (e.g., UniTok vs. VA-VAE/SDXL, VAR vs. Open-MAGVIT2) as contradictory, but does not quantify how often or how strongly the metrics disagree. I recommend a systematic analysis: across all tokenizers and resolutions, compute correlation coefficients (e.g., Spearman) between each traditional metric and the proposed T-ACC/T-NED/F-Sim, and also report the number of pairs whose ordering is reversed. This would turn the illustrative claim into a measurable one.
minor comments (5)
  1. [Section 3.1.1] There is a typo in the phrase 'to represent the the i-th text regions'; it should be 'the i-th text region'.
  2. [Equation (1)] The assumption that characters are uniformly distributed in the bounding box is stated in prose but should be stated explicitly as part of the equation's definition; its validity for multi-line or irregularly shaped text regions (e.g., curved text) is unclear and should be discussed.
  3. [Table 1] The column header 'Text(%)' is ambiguous; it appears to be the T-ACC value, but this should be labeled explicitly and consistently across all tables.
  4. [Abstract and Section 4.1] The claim 'requiring just 2GB memory and 4 minutes' refers to metric computation, not reconstruction; the abstract should clarify this to avoid misleading readers about the total evaluation cost.
  5. [Section 4.4] For video evaluation, frames are sampled at 1 FPS and averages are reported per frame; this procedure does not capture temporal artifacts such as flicker or texture shimmer, which are relevant for video tokenizers. The paper should state this limitation explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TokBench's metrics are computed by external OCR and face-recognition models and are not fitted to or derived from the paper's conclusions.

full rationale

TokBench's scoring chain is self-contained against external machinery. T-ACC and T-NED are computed by running the pretrained PARSeq OCR model (via docTR) on text crops from original and reconstructed images, and F-Sim is the cosine similarity of insightface embeddings of corresponding face crops. These definitions involve no parameter fitted to the benchmark data, no equation whose output is assumed as an input, and no prediction that is a renamed fit. The data-cleaning rule that instances must be correctly recognized by the OCR model on the original image is a test-set inclusion criterion, not a constraint that encodes tokenizer rankings; it only ensures the baseline is readable before reconstruction, and the 'Resize' row provides an empirical upper bound. The self-citations present in the paper (e.g., UniTok [29] is co-authored by a current author) are ordinary prior-art references to externally released tokenizers and do not carry the load-bearing argument. Figure 1 is illustrative rather than a derivation, and the Section 5 admission that contrast and saturation distortions are not captured is a stated limitation of the proxy metrics, not a circularity: it does not reduce T-ACC or F-Sim to the paper's conclusions. The possible invalidity of OCR and face-recognition models as substitutes for human perception is a construct-validity and robustness question, not a circularity question, and no quoted equation or citation chain makes the benchmark's scores equal to its own inputs.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The benchmark relies on external models and hand-set thresholds rather than new scientific entities. The manual thresholds are reasonable but should be documented as design choices.

free parameters (5)
  • minimum character pixel threshold = 5 x 5 pixels
    Assumed in Data Cleaning (Sec 3.1.1) as the minimum pixels to clearly represent a character; removes instances below this from evaluation.
  • minimum text scale = 0.005 (image), dynamic for video
    Texts with relative scale below 0.005 are removed in Sec 3.1.1; scales are computed via Eq. 1 with a uniform character distribution assumption.
  • minimum face scale = 0.05 for supplementary faces, 25 pixels for recognizability
    Faces below these scales are excluded (Sec 3.1.2 and Appendix A.2); the 25-pixel figure is set after visualization of the Resize upper bound.
  • face detection confidence threshold = 0.5
    Only detected faces with confidence above 0.5 are kept as supplementary GT (Sec 3.1.2).
  • difficulty rating boundaries = small/medium/large scale intervals in Table 6
    The categories used for small, medium, and large are chosen based on the scale lower bounds at each resolution; different resolutions get different boundaries.
assumptions (4)
  • domain assumption PARSeq OCR is an accurate judge of text readability on reconstructed images.
    The entire T-ACC and T-NED pipeline assumes that exact string match with PARSeq reflects perceptual text quality (Sec 3.2).
  • domain assumption Insightface face recognition cosine similarity measures identity preservation.
    F-Sim uses feature vectors from insightface; the paper assumes higher cosine similarity means better face reconstruction (Sec 3.2).
  • ad hoc to paper Character scale can be approximated as uniform distribution in the bounding box.
    Eq. 1 divides by character count N_c to define relative text scale; this is an approximation without empirical validation.
  • domain assumption Small-scale text and face reconstruction determines the upper bound of visual generation quality.
    The introduction claims information loss of compressed representations limits generation fidelity; the benchmark operationalizes this via text and faces.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TokBench: Evaluating Your Visual Tokenizer before Visual Generation." pith.science (2026). https://pith.science/paper/E56A5G26

@misc{pith2026250518142,
  author       = {Pith},
  title        = {Pith review of: TokBench: Evaluating Your Visual Tokenizer before Visual Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E56A5G26}},
  note         = {Machine review of arXiv:2505.18142}
}
read the original abstract

In this work, we reveal the limitations of visual tokenizers and VAEs in preserving fine-grained features, and propose a benchmark to evaluate reconstruction performance for two challenging visual contents: text and face. Visual tokenizers and VAEs have significantly advanced visual generation and multimodal modeling by providing more efficient compressed or quantized image representations. However, while helping production models reduce computational burdens, the information loss from image compression fundamentally limits the upper bound of visual generation quality. To evaluate this upper bound, we focus on assessing reconstructed text and facial features since they typically: 1) exist at smaller scales, 2) contain dense and rich textures, 3) are prone to collapse, and 4) are highly sensitive to human vision. We first collect and curate a diverse set of clear text and face images from existing datasets. Unlike approaches using VLM models, we employ established OCR and face recognition models for evaluation, ensuring accuracy while maintaining an exceptionally lightweight assessment process <span style="font-weight: bold; color: rgb(214, 21, 21);">requiring just 2GB memory and 4 minutes</span> to complete. Using our benchmark, we analyze text and face reconstruction quality across various scales for different image tokenizers and VAEs. Our results show modern visual tokenizers still struggle to preserve fine-grained features, especially at smaller scales. We further extend this evaluation framework to video, conducting comprehensive analysis of video tokenizers. Additionally, we demonstrate that traditional metrics fail to accurately reflect reconstruction performance for faces and text, while our proposed metrics serve as an effective complement.

Figures

Figures reproduced from arXiv: 2505.18142 by the authors.

Figure 1
Figure 1. Comparison of Different Metrics with Human Judgments. In each case, previous metrics (PSNR, SSIM, LPIPS) demonstrate discrepancies with human assessments, whereas our proposed face similarity and text accuracy effectively reflect the reconstruction quality. The reference image represents the original, while Patch 0 and Patch 1 show reconstruction results from different visual tokenizers. The same regions are cropped… view at source ↗
Figure 2
Figure 2. Statistics and Sample Diversity of TokBench-Image. TokBench features a balanced instance-scale distribution with particular emphasis on small-scale face and text instances, presenting significant challenges for existing visual reconstruction approaches. 3.1 Image Data Curation 3.1.1 Text Data Curation Data Collection We first collect text images from eight existing open-source datasets for diversity. Specifically, t… view at source ↗
Figure 3
Figure 3. Overview of the evaluation process of TokBench. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison between reconstructed images (right) and original images (left) under different [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: T-ACC and F-Sim metrics across reconstruction resolutions versus target scales. Smaller scales present greater challenges, and even the best-performing VAE show gap for improve￾ment when compared to the “resize” upper bound. VA-VAE/SDXL and VAR [51] vs. Open-MAGVIT2 (p…
Figure 6
Figure 6. Figure 6: Visualization results of text and face reconstruction performance for different methods at [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Visualization results of text and face reconstruction performance for different methods at [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 29 canonical work pages

  1. [1]

    FlexTok: Resampling images into 1d token sequences of flexible length.arXiv 2025, 2025

    Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, O ˘guzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan. FlexTok: Resampling images into 1d token sequences of flexible length.arXiv 2025, 2025

    work page 2025

  2. [2]

    Scene text recognition with permuted autoregressive sequence models

    Darwin Bautista and Rowel Atienza. Scene text recognition with permuted autoregressive sequence models. InEuropean conference on computer vision, pages 178–196, 2022

    work page 2022

  3. [3]

    Total-text: toward orientation robustness in scene text detection.Int

    Chee-Kheng Chng, Chee Seng Chan, and Cheng-Lin Liu. Total-text: toward orientation robustness in scene text detection.Int. J. Document Anal. Recognit., 23(1):31–52, 2020

    work page 2020

  4. [4]

    Diagnosing and enhancing vae models.arXiv preprint arXiv:1903.05789, 2019

    Bin Dai and David Wipf. Diagnosing and enhancing vae models.arXiv preprint arXiv:1903.05789, 2019

    arXiv 1903

  5. [5]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255, 2009

    work page 2009

  6. [6]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. InInternational Conference on Machine Learning, 2024

    2024

  7. [7]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021

    2021

  8. [8]

    Data filtering networks.arXiv preprint arXiv:2309.17425, 2023

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks.arXiv preprint arXiv:2309.17425, 2023

    arXiv 2023

Show all 73 references
  1. [9]

    Mmbench-video: A long-form multi-shot benchmark for holistic video understanding

    Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench-video: A long-form multi-shot benchmark for holistic video understanding. Advances in Neural Information Processing Systems, 37:89098–89124, 2024

    2024

  2. [10]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.arXiv preprint arXiv:2405.21075, 2024

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.arXiv preprint arXiv:2405.21075, 2024

    2024 arXiv

  3. [11]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  4. [12]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in neural information processing systems, volume 33, pages 6840–6851, 2020

    2020

  5. [13]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. InInternational Conference on Learning Representations, 2023

    2023

  6. [14]

    Labeled faces in the wild: A database forstudying face recognition in unconstrained environments

    Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. InWorkshop on faces in’Real-Life’Images: detection, alignment, and recognition, 2008

    2008

  7. [15]

    Icdar2019 competition on scanned receipt ocr and information extraction

    Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and CV Jawahar. Icdar2019 competition on scanned receipt ocr and information extraction. In International Conference on Document Analysis and Recognition, pages 1516–1520, 2019

    2019

  8. [16]

    insightface.https://github.com/deepinsight/insightface, 2024

    insightface team. insightface.https://github.com/deepinsight/insightface, 2024

    2024

  9. [17]

    Icdar 2015 competition on robust reading

    Dimosthenis Karatzas, Lluis Gomez-Bigorda, Anguelos Nicolaou, Suman Ghosh, Andrew Bagdanov, Masakazu Iwamura, Jiri Matas, Lukas Neumann, Vijay Ramaseshan Chandrasekhar, Shijian Lu, et al. Icdar 2015 competition on robust reading. InInternational Conference on Document Analysis...

    2015

  10. [18]

    Icdar 2013 robust reading competition

    Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez i Big- orda, Sergi Robles Mestre, Joan Mas, David Fernandez Mota, Jon Almazan Almazan, and Lluis Pere De Las Heras. Icdar 2013 robust reading competition. InInternational Conference on Document...

    2013

  11. [19]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. InInternational Conference on Learning Representations, 2014

    2014

  12. [20]

    Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization

    Martin Koestinger, Paul Wohlhart, Peter M Roth, and Horst Bischof. Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. InIEEE international conference on computer vision workshops (ICCV workshops), pages 2144–2151, 2011

    2011

  13. [21]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  14. [22]

    Flux.https://github.com/black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/black-forest-labs/flux, 2024

    2024

  15. [23]

    Mvbench: A comprehensive multi-modal video understanding benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195–22206, 2024

    2024

  16. [24]

    Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    2024 arXiv

  17. [25]

    World model on million-length video and language with ringattention.arXiv preprint arXiv:2402.08268, 2024

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with ringattention.arXiv preprint arXiv:2402.08268, 2024

    2024 arXiv

  18. [26]

    Abcnet: Real-time scene text spotting with adaptive bezier-curve network

    Yuliang Liu, Hao Chen, Chunhua Shen, Tong He, Lianwen Jin, and Liangwei Wang. Abcnet: Real-time scene text spotting with adaptive bezier-curve network. Inproceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9809–9818, 2020

    2020

  19. [27]

    Curved scene text detection via transverse and longitudinal sequence connection.Pattern Recognition, 90:337–345, 2019

    Yuliang Liu, Lianwen Jin, Shuaitao Zhang, Canjie Luo, and Sheng Zhang. Curved scene text detection via transverse and longitudinal sequence connection.Pattern Recognition, 90:337–345, 2019

    2019

  20. [28]

    Open- magvit2: An open-source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open- magvit2: An open-source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    2024 arXiv

  21. [29]

    Unitok: A unified tokenizer for visual generation and understanding.arXiv preprint arXiv:2502.20321, 2025

    Chuofan Ma, Yi Jiang, Junfeng Wu, Jihan Yang, Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A unified tokenizer for visual generation and understanding.arXiv preprint arXiv:2502.20321, 2025

    2025

  22. [30]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model.arXiv preprint arXiv:2502.10248, 2025

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-t2v technical report: The practice, challenges, and future of video foundation model.arXiv preprint arXiv:2502.10248, 2025

    2025 arXiv

  23. [31]

    Hdr-vdp-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions.ACM Transactions on graphics (TOG), 30(4):1–14, 2011

    Rafat Mantiuk, Kil Joong Kim, Allan G Rempel, and Wolfgang Heidrich. Hdr-vdp-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions.ACM Transactions on graphics (TOG), 30(4):1–14, 2011

    2011

  24. [32]

    Infographicvqa

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawa- har. Infographicvqa. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1697–1706, 2022

    2022

  25. [33]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2200–2209, 2021. 14

    2021

  26. [34]

    Finite scalar quantization: Vq-vae made simple.arXiv preprint arXiv:2309.15505, 2023

    Fabian Mentzer, David Minnen, Eirikur Agustsson, and Michael Tschannen. Finite scalar quantization: Vq-vae made simple.arXiv preprint arXiv:2309.15505, 2023

    2023 arXiv

  27. [35]

    doctr: Document text recognition.https://github.com/mindee/doctr, 2021

    Mindee. doctr: Document text recognition.https://github.com/mindee/doctr, 2021

    2021

  28. [36]

    Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification-rrc-mlt

    Nibal Nayef, Fei Yin, Imen Bizid, Hyunsoo Choi, Yuan Feng, Dimosthenis Karatzas, Zhenbo Luo, Umapada Pal, Christophe Rigaud, Joseph Chazalon, et al. Icdar2017 robust reading challenge on multi-lingual scene text detection and script identification-rrc-mlt. InInternational Conf...

    2017

  29. [37]

    Cosmos-tokenizer

    NVIDIA. Cosmos-tokenizer. https://research.nvidia.com/labs/dir/ cosmos-tokenizer/, 2024

    2024

  30. [38]

    Cord: a consolidated receipt dataset for post-ocr parsing

    Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. Cord: a consolidated receipt dataset for post-ocr parsing. InWorkshop on Document Intelligence at NeurIPS 2019, 2019

    2019

  31. [39]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  32. [40]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  33. [41]

    Tokenflow: Unified image tokenizer for multimodal understanding and generation.arXiv preprint arXiv:2412.03069, 2024

    Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation.arXiv preprint arXiv:2412.03069, 2024

    2024 arXiv

  34. [42]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational Conference on Machine Learning, pages 8821–8831, 2021

    2021

  35. [43]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  36. [44]

    300 faces in-the-wild challenge: Database and results.Image and vision computing, 47:3–18, 2016

    Christos Sagonas, Epameinondas Antonakos, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: Database and results.Image and vision computing, 47:3–18, 2016

    2016

  37. [45]

    Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    2016

  38. [46]

    Patel, Rama Chellappa, and David W

    Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M. Patel, Rama Chellappa, and David W. Jacobs. Frontal to profile face verification in the wild. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1–9, 2016

    2016

  39. [47]

    Tex- tocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text

    Amanpreet Singh, Guan Pang, Mandy Toh, Jing Huang, Wojciech Galuba, and Tal Hassner. Tex- tocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8802–8812, 2021

    2021

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021

    2021

  41. [49]

    Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  42. [50]

    Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024. 15

    2024 arXiv

  43. [51]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    2024

  44. [52]

    Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    2017

  45. [53]

    Coco-text: Dataset and benchmark for text detection and recognition in natural images.arXiv preprint arXiv:1601.07140, 2016

    Andreas Veit, Tomas Matera, Lukas Neumann, Jiri Matas, and Serge Belongie. Coco-text: Dataset and benchmark for text detection and recognition in natural images.arXiv preprint arXiv:1601.07140, 2016

    2016 arXiv

  46. [54]

    Omnitok- enizer: A joint image-video tokenizer for visual generation

    Junke Wang, Yi Jiang, Zehuan Yuan, Binyue Peng, Zuxuan Wu, and Yu-Gang Jiang. Omnitok- enizer: A joint image-video tokenizer for visual generation. InAdvances in Neural Information Processing Systems, volume 37, pages 28281–28295, 2024

    2024

  47. [55]

    Bridging continuous and discrete tokens for autoregressive visual generation.arXiv preprint arXiv:2503.16430, 2025

    Yuqing Wang, Zhijie Lin, Yao Teng, Yuanzhi Zhu, Shuhuai Ren, Jiashi Feng, and Xihui Liu. Bridging continuous and discrete tokens for autoregressive visual generation.arXiv preprint arXiv:2503.16430, 2025

    2025 arXiv

  48. [56]

    Image quality assessment: from error visibility to structural similarity.IEEE Trans

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Trans. Image Process., 13(4):600–612, 2004

    2004

  49. [57]

    Maskbit: Embedding-free image generation via bit tokens.Transactions on Machine Learning Research, 2024

    Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang- Chieh Chen. Maskbit: Embedding-free image generation via bit tokens.Transactions on Machine Learning Research, 2024

    2024

  50. [58]

    Liquid: Language models are scalable multi-modal generators.arXiv preprint arXiv:2412.04332, 2024

    Junfeng Wu, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable multi-modal generators.arXiv preprint arXiv:2412.04332, 2024

    2024 arXiv

  51. [59]

    Look at boundary: A boundary-aware face alignment algorithm

    Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, and Qiang Zhou. Look at boundary: A boundary-aware face alignment algorithm. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2129–2138, 2018

    2018

  52. [60]

    Dstext v2: A comprehensive video text spotting dataset for dense and small text.Pattern Recognition, 149:110177, 2024

    Weijia Wu, Yiming Zhang, Yefei He, Luoming Zhang, Zhenyu Lou, Hong Zhou, and Xiang Bai. Dstext v2: A comprehensive video text spotting dataset for dense and small text.Pattern Recognition, 149:110177, 2024

    2024

  53. [61]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024

    arXiv

  54. [62]

    Layton: Latent consistency tokenizer for 1024-pixel image reconstruction and generation by 256 tokens

    Qingsong Xie, Zhao Zhang, Zhe Huang, Yanhao Zhang, Haonan Lu, and Zhenyu Yang. Layton: Latent consistency tokenizer for 1024-pixel image reconstruction and generation by 256 tokens. arXiv preprint arXiv:2503.08377, 2025

    2025 arXiv

  55. [63]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  56. [64]

    Reconstruction vs

    Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. generation: Taming opti- mization dilemma in latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  57. [65]

    Language model beats diffusion–tokenizer is key to visual generation.arXiv preprint arXiv:2310.05737, 2023

    Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion–tokenizer is key to visual generation.arXiv preprint arXiv:2310.05737, 2023

    arXiv

  58. [66]

    An image is worth 32 tokens for reconstruction and generation.Advances in Neural Information Processing Systems, 37:128940–128966, 2024

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation.Advances in Neural Information Processing Systems, 37:128940–128966, 2024

    2024

  59. [67]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. InInternational Conference on Learning Representations, 2025. 16

    2025

  60. [68]

    Fsim: A feature similarity index for image quality assessment.IEEE Trans

    Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. Fsim: A feature similarity index for image quality assessment.IEEE Trans. Image Process., 20(8):2378–2386, 2011

    2011

  61. [69]

    The unrea- sonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018

    2018

  62. [70]

    Icdar 2019 robust reading challenge on reading chinese text on signboard

    Rui Zhang, Yongsheng Zhou, Qianyi Jiang, Qi Song, Nan Li, Kai Zhou, Lei Wang, Dong Wang, Minghui Liao, Mingkun Yang, et al. Icdar 2019 robust reading challenge on reading chinese text on signboard. InInternational Conference on Document Analysis and Recognition, pages 1577–1581, 2019

    2019

  63. [71]

    Image and video tokenization with binary spherical quantization.arXiv preprint arXiv:2406.07548, 2024

    Yue Zhao, Yuanjun Xiong, and Philipp Krähenbühl. Image and video tokenization with binary spherical quantization.arXiv preprint arXiv:2406.07548, 2024

    2024 arXiv

  64. [72]

    Cross-age LFW: A database for studying cross-age face recognition in unconstrained environments.CoRR, abs/1708.08197, 2017

    Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age LFW: A database for studying cross-age face recognition in unconstrained environments.CoRR, abs/1708.08197, 2017

    2017 arXiv

  65. [73]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. A Evaluation Setting A.1 Tokenizer Selection In this secti...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.