Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

GenSpace: Benchmarking Spatially-Aware Image Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GenSpace claims that a 3D-reconstruction metric aligns with human spatial judgments on generated images far better than vision-language models do, and that current image generators cannot reliably control 3D layout.

desk verdict A useful spatial-generation benchmark whose central human-alignment claim is undercut by a biased score-mapping comparison; the benchmark itself deserves peer review with heavy revision. read the letter →

arxiv 2505.24870 v2 pith:IWVU7O3G submitted 2025-05-30 cs.CV

Zehan Wang , Jiayang Xu , Ziang Zhang , Tianyu Pang , Chao Du , Hengshuang Zhao , Zhou Zhao This is my paper
classification cs.CV
keywords spatialawarenessimagegenerationevaluationtext-to-imageinstruction-basedediting3Dscenereconstructionrelationmetricdepthreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GenSpace is a benchmark and evaluation method for spatial awareness in image generation. The paper's central claim is that current generative models, although visually fluent, do not reliably follow spatial instructions about pose, relation, or measurement, and that this failure is measurable only by reconstructing the 3D geometry of the generated image rather than by asking a vision-language model to look at it. The benchmark covers 1,800 text-to-image prompts and 1,800 instruction-editing pairs across nine sub-domains, and the proposed metric chains detection, segmentation, depth, orientation, and camera calibration into a 3D unprojection before scoring. On 900 human-labeled images the metric agrees with human judgments 76.22% of the time, versus 56.44% for the best general vision-language model, which is the evidence that the evaluation itself is trustworthy. If the paper is right, spatial-faithfulness scores should be treated as a distinct capability axis for image generators, one that current models are far from mastering.

What carries the argument

The central mechanism is the paper's Spatial Expertise Pipeline: it runs object detection and segmentation, metric depth estimation, orientation estimation, and camera calibration on a single generated image, unprojects the pixels into a canonical 3D point cloud, and compares the recovered object poses and locations with the target spatial state. Scoring maps three difference types into [0,100]: orientation difference (full score within 30 degrees, falling linearly to zero at 45 degrees), relation correctness (correct or not), and relative distance error (full score within 33%, falling to zero at 44%); sub-domains that require several conditions at once multiply the corresponding sub-scores. This machinery is what replaces holistic VLM judgment with direct geometric comparison, and it is what carries the paper's claim that spatial errors are detectable and quantifiable.

What would settle it

Re-score the same generated images after swapping the metric-depth or camera-calibration backbone: if per-model rankings change materially, the metric is tracking its estimators rather than spatial faithfulness. Separately, rerun the human-alignment study on a larger randomly sampled set with inter-annotator agreement reported; if the pipeline's agreement with human labels drops toward the VLM range, the headline 76% versus 56% comparison would not generalize.

Watch

Extended reading notes

Core claim

The paper claims that spatial awareness in image generation can be decomposed into three increasing levels - spatial pose (object and camera orientation), spatial relation (egocentric camera-view, allocentric object-centered, and view-independent intrinsic relations), and spatial measurement (object size, inter-object distance, camera distance) - and that the correct way to evaluate these is to reconstruct 3D scene geometry from the generated image and compare it with the requested spatial state. It introduces a metric that chains object detection and segmentation, metric depth estimation, orientation estimation, and camera calibration into a single 3D unprojection, then scores orientation differences, relation correctness, and relative distance errors on a 0-100 scale. Validated on 900 manually labeled samples, the metric attains 76.22% average agreement with human labels, while the strongest general vision-language model reaches 56.44%, supporting the paper's contention that VLM judgments systematically miss spatial errors. Applying the benchmark to open and closed generative models in both text-to-image and instruction-editing settings, the paper reports that egocentric relations are handled well, while allocentric transformations, intrinsic relations, and all quantitative measurement sub-domains are largely failed by every tested model.

Load-bearing premise

The load-bearing premise is that the off-the-shelf depth, orientation, and camera-calibration estimators recover accurate and unbiased 3D geometry from generated images, so the scores reflect the image's spatial content rather than estimator artifacts; the paper validates this on only 100 hand-picked, label-balanced images per subdomain and does not report inter-annotator agreement or uncertainty bounds.

Editorial extensions

If this is right

  • Vision-language-model scores under-report spatial error: the paper's pipeline agrees with human labels on 76.22% of scored samples, while the best general vision-language model reaches 56.44%, so VLM-based evaluation is not a reliable proxy for spatial fidelity.
  • Egocentric relations are nearly solved, but allocentric and intrinsic relations are not: models default to camera-centric reasoning and frequently reverse left/right when asked to adopt an object's perspective.
  • Quantitative spatial measurement is essentially absent from current generators: varying the requested meters in the prompt changes the generated distance or size almost not at all in the paper's condition analysis.
  • Unified multimodal generators do better than dedicated diffusion and editing models at similar overall human-preference levels, and closed-source systems outperform open-source ones on spatial tasks.
  • Specialized instruction-editing models trained mostly on appearance edits are nearly unable to carry out spatial edits such as rotating an object or moving the camera, indicating a training-data gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the pipeline scores orientation, relation, and distance separately, it could be used as a per-error feedback signal for fine-tuning generators, something a single holistic VLM score cannot provide.
  • The same 3D-unprojection evaluation is directly transferable to multi-view and video generation, where cross-frame consistency can be checked against the recovered geometry rather than judged from a single image.
  • If these results hold on a larger random sample, VLM-based leaderboard rankings should be treated as measures of visual appeal rather than spatial correctness; that is a testable consequence of the 20-point alignment gap.
  • The left/right reversal pattern on allocentric prompts suggests a viewer-default prior in generation; a targeted experiment would fine-tune a generator on object-centered spatial annotations and test whether allocentric accuracy rises without hurting egocentric performance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces GenSpace, a benchmark for spatially-aware image generation, with three dimensions (Pose, Relation, Measurement) and nine sub-domains, spanning text-to-image generation and instruction-based editing. It also presents an automated evaluation pipeline that reconstructs 3D scene geometry using visual foundation models (detection, segmentation, depth, orientation, and camera calibration) and scores spatial correctness via hand-defined thresholds. The central claim is that this pipeline aligns with human spatial perception better than general-purpose VLMs (76.22% vs 56.44% average agreement, Table 3). The authors benchmark several generative models and identify core limitations: object perspective understanding, egocentric-allocentric transformation, and metric measurement adherence.

Significance. If the human-alignment claim holds, the work is a useful resource for the community: a new benchmark and metric for spatial awareness, public code (GitHub), a systematic taxonomy, and an evaluation of 12+ generative models with extensive visualizations. The benchmark construction and prompt diversity are thoughtful, and the paper's finding that current models fail at allocentric reasoning and quantitative measurements is plausible. However, the main evidence for the metric's validity is a human study with methodological weaknesses, and the headline comparison with VLMs is biased by score discretization. The contribution is primarily empirical and would be strengthened by a more rigorous validation.

major comments (3)
  1. [Section 4 (Human Alignment of Metrics)] The comparison in Table 3 is biased by the asymmetric discretization of evaluator scores. The proposed pipeline's geometric scores are thresholded in Section 3.3, so any orientation error within 30° or distance error within 33% is scored 100, and relation answers are binary 0/100. In contrast, VLM scores are mapped with the rule 0→Incorrect, (0,100)→Partially Correct, 100→Correct. Because VLMs are instructed to assign scores from 0 to 100 and typically return graded values (e.g., 95) for images that humans label 'Correct', the fixed mapping systematically undercounts VLM agreement, while the pipeline's 100 is an interval, not a point mass. The reported 20-point gap may thus reflect the discretization rule rather than genuine alignment. Please report threshold-free agreement metrics (e.g., Spearman rank correlation or AUC) or per-method calibrated cut points, and show the score distributions for each evaluator.
  2. [Section 4 (Human Alignment of Metrics)] The human validation lacks statistical rigor. For each sub-domain, 100 images are labeled by three annotators, but no inter-annotator agreement (e.g., Fleiss' κ) is reported, and the resolution of disagreements is unspecified. The accuracies in Tables 3 and 7 are reported without confidence intervals or significance tests. With n=100, the standard error of an accuracy is roughly 5 percentage points, so several sub-domain differences (e.g., Intrinsic: 56 vs 48 in Table 3) are not clearly meaningful. The paper should provide agreement statistics, uncertainty quantification (e.g., bootstrapped CIs or McNemar's test for paired comparisons), and a description of how final labels were reached.
  3. [Section 3.3 (Evaluation Pipeline & Metric)] The pipeline's validity rests on components whose accuracy is not independently established. In particular, OrientAnything [56] is a model from the same research group (reference [56] shares the first author), and the human test set is self-constructed around the paper's own taxonomy. The paper should provide external validation of the pipeline components (e.g., orientation and relative depth accuracy on standard benchmarks) or explicitly discuss the risk of circularity and how the human study mitigates it. This is necessary because the pipeline's scores are the basis for all benchmark results, not just the human-alignment comparison.
minor comments (6)
  1. [Section 5.1] The text says 'we evaluate 8 models for text-to-image generation' but lists 9 models (six expertise and three unified). Please correct the count.
  2. [Table 1] The typo 'Ppposite' should be 'Opposite'.
  3. [Section 4] The prompt template used to query the VLM baselines is not provided; this is needed for reproducibility of the human-alignment comparison.
  4. [Conclusion, Limitation and Future Work] The limitation section is very brief and does not address the limitations of the evaluation pipeline (e.g., reliance on depth/orientation estimators) or the human study; consider expanding.
  5. [Figure 3] The x-axis for several sub-domains includes a category 'None'; please clarify in the caption whether this indicates detection failure or an unspecified condition.
  6. [Table 4] The header 'Ave. Rank Arena ELO' is ambiguous; the column after the nine sub-domain scores appears to be an average rank, not an average score. Please rename for clarity.

Circularity Check

1 steps flagged · score 6.0 of 10

The Table 3 human-alignment comparison is partially an artifact of the fixed score-to-label mapping: VLM continuous scores are collapsed so that any non-100 score counts as 'Partially Correct', while the pipeline's threshold-based scores are near-binary.

  1. self definitional [Section 4 (Human Alignment of Metrics), Table 3; Section 3.3 Step 3 Quantitative Score Mapping]
    "To align the fine-grained continuous scores with this categorical system for comparison, we map scores as follows: 0 to 'Incorrect,' (0, 100) to 'Partially Correct,' and 100 to 'Correct.' ... For orientation, differences within 30° receive a score of 100. ... For relation, Yes (correct) scores 100, and No (incorrect) scores 0. For distance, relative errors within 33% are scored as 100."

    The claimed 20-point advantage (76.22 vs 56.44) is computed after collapsing each evaluator's score with the same three-bin rule. The pipeline's Step-3 thresholds are designed to emit exactly 100 for any state inside hand-set bounds (≤30°, ≤33%) and 0 or a linearly decayed value outside, so its outputs are mostly 0 or 100. General VLMs asked to assign a 0-100 score rarely produce exactly 100; a score of 99, which semantically indicates correctness, is counted as 'Partially Correct' and thus as disagreement with a human 'Correct' label. The paper reports no per-method calibration, cut-point optimization, rank correlation, or AUC, so the relative ranking of evaluators is driven by the discontinuous mapping rather than by an independent measurement of human alignment.

full rationale

The benchmark taxonomy, prompt construction, and the model-by-model spatial scores (Tables 4-5) are self-contained and are not circular: they are rule-based measurements of generated images, and the model rankings are compared externally to Arena ELO. The use of OrientAnything, a same-author model, is a self-citation but not a load-bearing circular step, because OrientAnything is an independently trained visual foundation model with its own external evaluations. The central circularity is confined to the evaluator-comparison claim in Table 3. The pipeline's score thresholds and the VLM score-to-label mapping are both hand-selected inputs, and the comparison collapses the two in an asymmetric way: the pipeline's hard thresholds produce 100/0 scores, while VLM graded scores are mapped so that any value below 100 becomes 'Partially Correct'. Because the headline conclusion that the proposed pipeline is more human-aligned than VLMs rests on this comparison, the 20-point gap is partly an artifact of the mapping choice. The paper does not report threshold tuning on the human-labeled set, so I do not treat the thresholds themselves as fitted; the circularity is therefore partial rather than total. If the thresholds had been optimized on the same 900 human annotations, the score would be higher. Overall, the derivation is not equivalent to its inputs, but the key comparative claim is partially forced by the evaluation protocol.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the accuracy and fairness of the custom evaluation pipeline and the curated human test set; these are not externally benchmarked and depend on hand-selected thresholds and taxonomies.

free parameters (4)
  • orientation_score_threshold = 30 deg full score, 45 deg zero
    Hand-chosen in Sec 3.3 Step 3 to map orientation differences to scores; no perceptual or psychophysical justification.
  • distance_score_threshold = 33% relative error full score, 44% zero
    Hand-chosen in Sec 3.3 Step 3 to map distance differences to scores; no empirical basis.
  • object_size_prompt_values = N varies by object
    Values like 0.2x, 20cm, 50cm, 30cm chosen per object category without stated criteria; could affect results.
  • human_label_sample_balance = roughly balanced Correct/Partially/Incorrect
    Human alignment sample was curated to have roughly equal labels per subdomain, which may not reflect real generation error rates and can affect measured agreement.
assumptions (4)
  • domain assumption Monocular depth, orientation, and camera calibration models give metric scene geometry accurate enough for scoring.
    The pipeline in Sec 3.3 relies on Depth Anything, OrientAnything, WildCamera, and PerspectiveFields without uncertainty quantification or external validation.
  • ad hoc to paper The three dimensions (Pose, Relation, Measurement) and nine subdomains exhaustively cover spatial awareness in image generation.
    Taxonomy introduced in Sec 3.1 is not derived from prior work and may omit other spatial aspects.
  • domain assumption Human annotations (Correct/Partially/Incorrect) are reliable and consistent across annotators.
    Sec 4 describes three annotators but does not report inter-annotator agreement.
  • ad hoc to paper The VLM scoring mapping (0=Incorrect, (0,100)=Partially, 100=Correct) is a fair comparison.
    VLMs are scored on a continuous 0-100 scale but aggregated into three categories, which may penalize VLMs that give intermediate scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GenSpace: Benchmarking Spatially-Aware Image Generation." pith.science (2026). https://pith.science/paper/IWVU7O3G

@misc{pith2026250524870,
  author       = {Pith},
  title        = {Pith review of: GenSpace: Benchmarking Spatially-Aware Image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IWVU7O3G}},
  note         = {Machine review of arXiv:2505.24870}
}
read the original abstract

Humans can intuitively compose and arrange scenes in the 3D space for photography. However, can advanced AI image generators plan scenes with similar 3D spatial awareness when creating images from text or image prompts? We present GenSpace, a novel benchmark and evaluation pipeline to comprehensively assess the spatial awareness of current image generation models. Furthermore, standard evaluations using general Vision-Language Models (VLMs) frequently fail to capture the detailed spatial errors. To handle this challenge, we propose a specialized evaluation pipeline and metric, which reconstructs 3D scene geometry using multiple visual foundation models and provides a more accurate and human-aligned metric of spatial faithfulness. Our findings show that while AI models create visually appealing images and can follow general instructions, they struggle with specific 3D details like object placement, relationships, and measurements. We summarize three core limitations in the spatial perception of current state-of-the-art image generation models: 1) Object Perspective Understanding, 2) Egocentric-Allocentric Transformation and 3) Metric Measurement Adherence, highlighting possible directions for improving spatial intelligence in image generation.

Figures

Figures reproduced from arXiv: 2505.24870 by the authors.

Figure 1
Figure 1. Illustration of GenSpace’s three basic evaluation dimensions and nine corresponding sub-domains for text-to-image generation and instruction-based image editing. All results shown are generated by GPT-4o. Zoom in for best viewing. Abstract Humans can intuitively compose and arrange scenes in the 3D space for photog￾raphy. However, can advanced AI image generators plan scenes with similar 3D spatial awareness when cr… view at source ↗
Figure 2
Figure 2. Overview of our evaluation pipeline & metric. We use advanced visual foundation models [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Impact of varying spatial conditions on the spatial states of generated samples. The [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (25 more)
Figure 4
Figure 4. Figure 4: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Impact of varying spatial conditions on the spatial states of generated samples from [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]
Figure 16
Figure 16. Figure 16: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]
Figure 17
Figure 17. Figure 17: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p020_17.png]
Figure 18
Figure 18. Figure 18: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]
Figure 19
Figure 19. Figure 19: Visualization of Text-to-image Generation Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p021_19.png]
Figure 20
Figure 20. Figure 20: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p022_20.png]
Figure 21
Figure 21. Figure 21: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p023_21.png]
Figure 22
Figure 22. Figure 22: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p023_22.png]
Figure 23
Figure 23. Figure 23: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p024_23.png]
Figure 24
Figure 24. Figure 24: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p024_24.png]
Figure 25
Figure 25. Figure 25: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p025_25.png]
Figure 26
Figure 26. Figure 26: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p025_26.png]
Figure 27
Figure 27. Figure 27: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p026_27.png]
Figure 28
Figure 28. Figure 28: Visualization of Instruction-based Image Editing Benchmark on the subdomain [PITH_FULL_IMAGE:figures/full_fig_p026_28.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Do Image Editing Models Understand Lighting?

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    New 3DLP benchmark with real-world 1K HDR pairs shows state-of-the-art image editing models vary in physical lighting consistency, with best models close to reality but error-prone in low-light regions.

Reference graph

Works this paper leans on

69 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [56]

    Orient anything: Learning robust object orientation estimation from rendering 3d models.arXiv preprint arXiv:2412.18605, 2024

    Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. Orient anything: Learning robust object orientation estimation from rendering 3d models.arXiv preprint arXiv:2412.18605, 2024. 12

    arXiv 2024

  2. [1]

    Univ of California Press, 1972

    Rudolf Arnheim.Art and visual perception: A psychology of the creative eye. Univ of California Press, 1972

    1972

  3. [2]

    artificialanalysis, 2025

    artificialanalysis.ai. artificialanalysis, 2025. https://artificialanalysis.ai/text-to-image/ arena?tab=leaderboard

    work page 2025

  4. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  5. [4]

    The state of the art of spatial interfaces for 3d visualization

    Lonni Besançon, Anders Ynnerman, Daniel F Keefe, Lingyun Yu, and Tobias Isenberg. The state of the art of spatial interfaces for 3d visualization. InComputer Graphics Forum, volume 40, pages 293–326. Wiley Online Library, 2021

    work page 2021

  6. [5]

    Improving image generation with better captions.Computer Science

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  7. [6]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18392–18402, 2023

    2023

  8. [7]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024

    2024

Show all 69 references
  1. [8]

    Spatialrgpt: Grounded spatial reasoning in vision language models.arXiv preprint arXiv:2406.01584, 2024

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision language models.arXiv preprint arXiv:2406.01584, 2024

    2024 arXiv

  2. [9]

    Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    2025 arXiv

  3. [10]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InForty-first international conference on machine l...

    2024

  4. [11]

    Routledge, 2017

    Michael Freeman.The Photographer’s Eye Digitally Remastered 10th Anniversary Edition: Composition and Design for Better Digital Photos. Routledge, 2017

    2017

  5. [12]

    Commonsense-t2i challenge: Can text-to-image generation models understand commonsense?arXiv preprint arXiv:2406.07546, 2024

    Xingyu Fu, Muyu He, Yujie Lu, William Yang Wang, and Dan Roth. Commonsense-t2i challenge: Can text-to-image generation models understand commonsense?arXiv preprint arXiv:2406.07546, 2024

    2024 arXiv

  6. [13]

    Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346, 2025

    Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al. Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346, 2025

    2025 arXiv

  7. [14]

    Interaction strategies for effective augmented reality geo-visualization: Insights from spatial cognition.Human–Computer Interaction, 36(2): 107–149, 2021

    Aaron L Gardony, Shaina B Martis, Holly A Taylor, and Tad T Brunyé. Interaction strategies for effective augmented reality geo-visualization: Insights from spatial cognition.Human–Computer Interaction, 36(2): 107–149, 2021

    2021

  8. [15]

    Geneval: An object-focused framework for evaluating text-to-image alignment.Advances in Neural Information Processing Systems, 36:52132–52152, 2023

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment.Advances in Neural Information Processing Systems, 36:52132–52152, 2023

    2023

  9. [16]

    Gemini-2.0-flash, 2025

    Google. Gemini-2.0-flash, 2025. https://aistudio.google.com/prompts/new_chat?model= gemini-2.0-flash-exp

    2025

  10. [17]

    Gemini-2.5-pro, 2025.https://deepmind.google/technologies/gemini/pro/

    Google. Gemini-2.5-pro, 2025.https://deepmind.google/technologies/gemini/pro/

    2025

  11. [18]

    Prompt-to- prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022. 10

    2022 arXiv

  12. [19]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  13. [20]

    T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    2023

  14. [21]

    T2i-compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  15. [22]

    Composer: Creative and controllable image synthesis with composable conditions.arXiv preprint arXiv:2302.09778, 2023

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable image synthesis with composable conditions.arXiv preprint arXiv:2302.09778, 2023

    2023 arXiv

  16. [23]

    Perspective fields for single image camera calibration

    Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Blackburn-Matzen, Matthew Sticha, and David F Fouhey. Perspective fields for single image camera calibration. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17307–1...

    2023

  17. [24]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  18. [25]

    Pick-a- pic: An open dataset of user preferences for text-to-image generation.Advances in Neural Information Processing Systems, 36:36652–36663, 2023

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a- pic: An open dataset of user preferences for text-to-image generation.Advances in Neural Information Processing Systems, 36:36652–36663, 2023

    2023

  19. [26]

    Flux.https://github.com/black-forest-labs/flux, 2024

    Black Forest Labs. Flux.https://github.com/black-forest-labs/flux, 2024

    2024

  20. [27]

    Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.arXiv preprint arXiv:2405.08748, 2024

    Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, et al. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine-grained chinese understanding.arXiv preprint arXiv:2405.08748, 2024

    2024 arXiv

  21. [28]

    Evaluating text-to-visual generation with image-to-text generation

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. InEuropean Conference on Computer Vision, pages 366–384. Springer, 2024

    2024

  22. [29]

    Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  23. [30]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  24. [31]

    Improving long-text alignment for text-to-image diffusion models.arXiv preprint arXiv:2410.11817, 2024

    Luping Liu, Chao Du, Tianyu Pang, Zehan Wang, Chongxuan Li, and Dong Xu. Improving long-text alignment for text-to-image diffusion models.arXiv preprint arXiv:2410.11817, 2024

    2024 arXiv

  25. [32]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. InEuropean Conference on Computer Vision, pages 38–55. Springer, 2024

    2024

  26. [33]

    Step1x-edit: A practical framework for general image editing.arXiv preprint arXiv:2504.17761, 2025

    Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Step1x-edit: A practical framework for general image editing.arXiv preprint arXiv:2504.17761, 2025

    2025 arXiv

  27. [34]

    3dsrbench: A comprehensive 3d spatial reasoning benchmark.arXiv preprint arXiv:2412.07825, 2024

    Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark.arXiv preprint arXiv:2412.07825, 2024

    2024

  28. [35]

    Phybench: A physical commonsense benchmark for evaluating text-to-image models.arXiv preprint arXiv:2406.11802, 2024

    Fanqing Meng, Wenqi Shao, Lixin Luo, Yahong Wang, Yiran Chen, Quanfeng Lu, Yue Yang, Tianshuo Yang, Kaipeng Zhang, Yu Qiao, et al. Phybench: A physical commonsense benchmark for evaluating text-to-image models.arXiv preprint arXiv:2406.11802, 2024

    2024 arXiv

  29. [36]

    Hands-free interaction in immersive virtual reality: A systematic review.IEEE Transactions on Visualization and Computer Graphics, 27(5):2702–2713, 2021

    Pedro Monteiro, Guilherme Gonçalves, Hugo Coelho, Miguel Melo, and Maximino Bessa. Hands-free interaction in immersive virtual reality: A systematic review.IEEE Transactions on Visualization and Computer Graphics, 27(5):2702–2713, 2021

    2021

  30. [37]

    Wise: A world knowledge-informed semantic evaluation for text-to-image generation.arXiv preprint arXiv:2503.07265, 2025

    Yuwei Niu, Munan Ning, Mengren Zheng, Bin Lin, Peng Jin, Jiaqi Liao, Kunpeng Ning, Bin Zhu, and Li Yuan. Wise: A world knowledge-informed semantic evaluation for text-to-image generation.arXiv preprint arXiv:2503.07265, 2025. 11

    2025 arXiv

  31. [38]

    Gpt-4o, 2025.https://openai.com/index/introducing-4o-image-generation/

    OpenAI. Gpt-4o, 2025.https://openai.com/index/introducing-4o-image-generation/

    2025

  32. [39]

    Gpt-o3, 2025.https://openai.com/index/introducing-o3-and-o4-mini/

    OpenAI. Gpt-o3, 2025.https://openai.com/index/introducing-o3-and-o4-mini/

    2025

  33. [40]

    Diffusion handles enabling 3d edits for diffusion models by lifting activations to 3d

    Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, and Niloy J Mitra. Diffusion handles enabling 3d edits for diffusion models by lifting activations to 3d. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pag...

    2024

  34. [41]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    2023

  35. [42]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  36. [43]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  37. [44]

    Does spatial cognition emerge in frontier models?arXiv preprint arXiv:2410.06468, 2024

    Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, and Vladlen Koltun. Does spatial cognition emerge in frontier models?arXiv preprint arXiv:2410.06468, 2024

    2024 arXiv

  38. [45]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  39. [46]

    Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159, 2024

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159, 2024

    2024 arXiv

  40. [47]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  41. [48]

    Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans.Advances in neural information processing systems, 29, 2016

    2016

  42. [49]

    Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural informat...

    2022

  43. [50]

    Seededit: Align image re-generation to image editing.arXiv preprint arXiv:2411.06686, 2024

    Yichun Shi, Peng Wang, and Weilin Huang. Seededit: Align image re-generation to image editing.arXiv preprint arXiv:2411.06686, 2024

    2024 arXiv

  44. [51]

    Evaluating the generation of spatial relations in text and image generative models.arXiv preprint arXiv:2411.07664, 2024

    Shang Hong Sim, Clarence Lee, Alvin Tan, and Cheston Tan. Evaluating the generation of spatial relations in text and image generative models.arXiv preprint arXiv:2411.07664, 2024

    2024 arXiv

  45. [52]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  46. [53]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    2024

  47. [54]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and ...

    2024

  48. [55]

    Diffusion models are geometry critics: Single image 3d editing using pre-trained diffusion priors

    Ruicheng Wang, Jianfeng Xiang, Jiaolong Yang, and Xin Tong. Diffusion models are geometry critics: Single image 3d editing using pre-trained diffusion priors. InEuropean Conference on Computer Vision, pages 441–458. Springer, 2024

    2024

  49. [57]

    Omniedit: Building image editing generalist models through specialist supervision

    Cong Wei, Zheyang Xiong, Weiming Ren, Xeron Du, Ge Zhang, and Wenhu Chen. Omniedit: Building image editing generalist models through specialist supervision. InThe Thirteenth International Conference on Learning Representations, 2024

    2024

  50. [58]

    Thinking in space: How multimodal large language models see, remember, and recall spaces.arXiv preprint arXiv:2412.14171, 2024

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces.arXiv preprint arXiv:2412.14171, 2024

    2024 arXiv

  51. [59]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024

    2024

  52. [60]

    Depth anything v2.Advances in Neural Information Processing Systems, 37:21875–21911, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2.Advances in Neural Information Processing Systems, 37:21875–21911, 2024

    2024

  53. [61]

    Image sculpting: Precise object editing with 3d geometry control

    Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, and Saining Xie. Image sculpting: Precise object editing with 3d geometry control. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4241–4251, 2024

    2024

  54. [62]

    Gaze-supported 3d object manipulation in virtual reality

    Difeng Yu, Xueshi Lu, Rongkai Shi, Hai-Ning Liang, Tilman Dingler, Eduardo Velloso, and Jorge Goncalves. Gaze-supported 3d object manipulation in virtual reality. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–13, 2021

    2021

  55. [63]

    Routledge, 2017

    Richard D Zakia and John Suler.Perception and IMAGING: Photography as a way of Seeing. Routledge, 2017

    2017

  56. [64]

    Magicbrush: A manually annotated dataset for instruction-guided image editing.Advances in Neural Information Processing Systems, 36:31428–31449, 2023

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing.Advances in Neural Information Processing Systems, 36:31428–31449, 2023

    2023

  57. [65]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  58. [66]

    In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer.arXiv preprint arXiv:2504.20690, 2025

    Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang. In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer.arXiv preprint arXiv:2504.20690, 2025

    2025 arXiv

  59. [67]

    Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities

    Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities. arXiv preprint arXiv:2410.17385, 2024

    arXiv

  60. [68]

    Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Systems, 36:11127–11150, 2023

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Systems, 36:11127–11150, 2023

    2023

  61. [69]

    right/left view

    Shengjie Zhu, Abhinav Kumar, Masa Hu, and Xiaoming Liu. Tame a wild camera: in-the-wild monocular camera calibration.Advances in Neural Information Processing Systems, 36:45137–45149, 2023. 13 A Detailed scoring criteria for each sub-domain. In Tab.6, we show the demand for sp...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.