Pith. sign in

REVIEW 3 major objections 7 minor 59 references

Rethink Sparse Signals for Pose-guided Text-to-image Generation

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse keypoint skeletons can be made as precise as depth maps for pose-guided image generation.

desk verdict Solid empirical paper: sparse-pose ControlNet with learnable keypoint embeddings and keypoint-token attention supervision shows real gains, but the headline claim of matching dense signals rests on a pose estimator trained on the same data. read the letter →

arxiv 2506.20983 v1 pith:PKCEJDMO submitted 2025-06-26 cs.CV

Wenjie Xuan , Jing Zhang , Juhua Liu , Bo Du , Dacheng Tao This is my paper
classification cs.CV
keywords sparseposesignalspose-guidedtext-to-imagegenerationOpenControlNetlearnablekeypointembeddingsconceptlearningcross-attentionheatmapdiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that sparse pose signals—the simple keypoint-and-skeleton stick figures of OpenPose—have been prematurely abandoned in favor of dense guides such as depth maps for pose-controlled text-to-image generation. It proposes SP-Ctrl, a ControlNet-style adapter that makes sparse keypoints precise enough to rival dense signals by turning keypoint colors into learnable embeddings and by adding per-keypoint text tokens whose cross-attention maps are trained to land on the keypoints. On the AP-10K animal benchmark the method raises pose mAP from 44.25 (ControlNet with OpenPose) to 55.63, approaching depth-based ControlNet at 57.07, and on Human-Art it reaches 51.11 versus 45.26 for ControlNet. The payoff is that users keep the editability, cross-species transfer, and shape diversity of sparse skeletons without giving up alignment.

What carries the argument

The machinery is the combination of a Spatial-Pose Representation (SPR) and a Keypoint Concept Learning (KCL) strategy inside a ControlNet. SPR replaces the fixed RGB blob colors of OpenPose with learnable per-keypoint embeddings produced by a small MLP, so each of the 17 keypoints gets a distinctive vector optimized for the diffusion loss, while skeleton edges use an all-ones embedding. KCL injects new textual tokens for each visible keypoint into the prompt and, during training, forces the cross-attention maps of those tokens at the 3rd transformer block and timesteps 250–500 to match Gaussian heatmaps centered at the keypoint locations. The heatmap loss is what makes the keypoint tokens spatially grounded, and the learnable embeddings are what make the pose input expressive.

What would settle it

Re-run the AP-10K and Human-Art comparisons using a pose estimator never trained on those datasets, or have human annotators mark keypoints on a sample of generated images, and check whether SP-Ctrl keeps its reported lead over sparse-pose ControlNet and its small gap to depth-conditioned ControlNet.

Watch

Extended reading notes

Core claim

The paper's central claim is that sparse OpenPose-style keypoints are sufficient for precise pose-guided text-to-image generation, provided the pose signal is re-encoded and the model is taught what each keypoint means. Two bottlenecks are identified: OpenPose's RGB rendering is designed for visualization and carries little learnable signal, and point-like keypoints are hard for the diffusion model to attend to. SP-Ctrl addresses both: a Spatial-Pose Representation learns a distinct embedding per keypoint through an MLP optimized by the denoising objective, and Keypoint Concept Learning adds new text tokens such as <left-eye> to the prompt and minimizes an L2 loss between the cross-attention maps of those tokens and keypoint-heatmap targets during training. With these two additions, the model matches the pose alignment of depth-conditioned ControlNet while keeping the flexibility of sparse signals, as demonstrated on animal and human benchmarks.

Load-bearing premise

The reported pose mAP is computed with ViTPose++ pretrained on the same AP-10K and Human-Art datasets used for training, and the paper provides no error bars or independent human evaluation, so the headline gains assume the estimator rewards true keypoint alignment rather than spurious patterns in the generated images.

Editorial extensions

If this is right

  • Sparse-pose ControlNet trained this way can approach or match the pose alignment of depth-conditioned ControlNet (55.63 versus 57.07 mAP on AP-10K) without requiring depth maps.
  • Because the condition is still a sparse skeleton, users can edit, create, and transfer poses manually or across species, which dense signals constrain.
  • The method does not need pretrained pose estimators during training, unlike HumanSD or GRPose, which simplifies the training pipeline.
  • The slight CLIP-Score decrease (23.86 versus 24.77 on AP-10K) is attributed to the new keypoint tokens being absent from the CLIP vocabulary, an evaluation discrepancy rather than a text-alignment failure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cross-attention heatmap constraint could generalize to other sparse geometric guidance, such as edges or object landmarks, wherever a spatial target map can be defined.
  • An independent pose estimator or a human keypoint-rating study would settle whether the mAP gains reflect true anatomical alignment or exploitation of ViTPose++ trained on the same benchmarks.
  • Because the keypoint embeddings are learned from random vectors, the results suggest the diffusion objective alone can imbue purely spatial tokens with semantic identity, a property worth testing in other conditioning tasks.
  • The remaining >25% mAP gap to real images noted in the appendix points to pose diversity rather than representation as the next bottleneck; augmenting training with synthetic complex poses is a concrete next test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes SP-Ctrl, a ControlNet-based method for pose-guided text-to-image generation using sparse OpenPose signals. It introduces two main components: (1) a Spatial-Pose Representation (SPR) that replaces fixed RGB keypoint embeddings with learnable embeddings produced by an MLP, and (2) a Keypoint Concept Learning (KCL) strategy that adds new textual tokens for each keypoint and applies a heatmap loss (Eq. 3) on cross-attention maps to align them with ground-truth keypoint heatmaps. Experiments on AP-10K and Human-Art report pose mAP gains over ControlNet (11.38 points on AP-10K, 5.85 on Human-Art) and pose mAP close to depth-based ControlNet on AP-10K (55.63 vs 57.07). The paper also demonstrates qualitative benefits in image diversity, cross-species generalization, and pose editing, and includes extensive ablations of the proposed modules and hyperparameters.

Significance. If the results are reliable, the paper makes a strong empirical case that sparse pose signals can be competitive with dense signals (e.g., depth) for pose-guided generation, while retaining advantages in editability and cross-species generalization. The two proposed mechanisms—learnable spatial-pose embeddings and keypoint concept learning—are novel, well-motivated by cross-attention analysis, and thoroughly ablated. The paper also ships a clear architecture description and qualitative comparisons. However, the central 'matches dense signals' claim rests on a single pose-estimator metric that may be biased, and no error bars are provided; the significance is therefore provisional until the evaluation concern is addressed.

major comments (3)
  1. [Section 4.1, Metrics paragraph] The pose mAP is computed with ViTPose++-H pretrained on AP-10K and Human-Art, the same datasets used to train SP-Ctrl. Since the proposed KCL loss (Eq. 3) explicitly trains cross-attention maps to match ground-truth keypoint heatmaps, the generator may learn to place locally detectable keypoint-like appearance cues that this specific estimator recognizes, potentially inflating the measured pose alignment. This does not make the training loss circular, but it makes the evaluation metric a non-neutral ruler for the central claim of matching dense signals. The authors should validate with an independent pose estimator not trained on these datasets (e.g., a detector trained on COCO or a different architecture), a human perceptual study, or at least report pose mAP from a second estimator and error bars over multiple runs.
  2. [Abstract and Section 4.5, Table 3] The abstract's claim that SP-Ctrl 'even matches the performance of dense signal-based methods' is overstated. On AP-10K, Table 3 shows a 1.44 mAP gap to depth-based ControlNet (55.63 vs 57.07), and on Human-Art no dense baseline is reported. The wording should be revised to 'approaches' or 'is competitive with,' and the residual gap should be acknowledged quantitatively. The text in Section 4.5 says 'almost matching,' which is more accurate, but the abstract and Figure 1 use 'matches' and 'gap' inconsistently; Figure 1's labels 'Gap=12.82%' and '11.38%' are not directly traceable to Table 3.
  3. [Section 4.2 and Section 4.4] No error bars or statistical significance tests are reported for any metric. The authors generate three images per prompt (Section 4.2), but all numbers in Tables 1–3 and Figures 6–7 are point estimates. Several ablation differences are small (e.g., 0.86% in Fig. 6(b), and 'competitive' across embedding dimensions in Fig. 6(c)), and the headline 11.38-point gain over ControlNet could also be subject to variance. The authors should report standard deviations across at least three seeds or bootstrap confidence intervals for the key comparisons (Ours vs. ControlNet, Ours vs. Depth).
minor comments (7)
  1. [Section 4.3 and Appendix C, CLIP-Score analysis] The CLIP-Score evaluation removes the newly introduced keypoint tokens from the prompt during evaluation, while generation uses them. This is explained in Section 4.3 and Fig. S17, but the comparison with baseline methods, which use standard prompts, is not apples-to-apples. It would be informative to also report CLIP-Score for all methods using prompts without keypoint tokens, and to explicitly state the CLIP-Score drop (e.g., 0.68 in Section 4.4) as a limitation of the method.
  2. [Figure 1] The bars labeled 'Mask (ControlNet)', 'Depth (ControlNet)', 'Sparse Pose (ControlNet)', and 'Sparse Pose (Ours)' are inconsistent with Table 3, which reports mask mAP 26.74, depth 57.07, sparse 44.25, and ours 55.63. The figure's 'Gap=12.82%' and '11.38%' labels are unclear; specify which gaps are shown and how they are computed.
  3. [Section 3.1, Eq. (1)] The distinction between 'randomly initialized vectors E0', 'fixed E0', and 'learnable E0' is confusing. Clarify that E0 is the input to the spatial-pose embedding module G, and that in the default setting E0 is fixed and only G's parameters φ are learned.
  4. [Section 3.2] The decision to detach the gradient from the noisy image query Q when computing Lht is mentioned but not ablated. If this choice is important for training stability, include an ablation; otherwise, remove the detail or justify it briefly.
  5. [Appendix E] Appendix E states that a significant gap in pose accuracy (greater than 25%) remains between synthesized and real images. This limitation should be acknowledged in the main paper's conclusion or abstract to calibrate reader expectations.
  6. [Table 1] In Table 1, the 'Real Image' row for Human-Art has an empty FID cell. Use an em dash and add a note explaining why FID is not computed or not applicable for real images.
  7. [General notation] The table headers use 'Pose mAP↑', 'FID↓', 'CLIP-Score↑' in some tables and 'Pose mAP', 'FID', 'CLIP-Score' in others; ensure consistent arrow notation across Tables 1–3 and the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training objectives and evaluation are disclosed, and the method's gains are not derived by construction from a fitted parameter.

full rationale

Walking the derivation chain, the paper's two claimed components are both trained with explicit supervision on ground-truth pose information. The spatial-pose representation (SPR) optimizes an MLP module G(·) with the denoising objective in Eq. (6), so the learnable keypoint embeddings are not fitted to the evaluation metric. Keypoint Concept Learning (KCL) adds tokens and supervises their cross-attention maps against ground-truth keypoint heatmaps via Eq. (3), which is direct supervision rather than a self-referential prediction. No equation in the paper defines the output in terms of the evaluation quantity, and no fitted parameter is renamed as a prediction. The pose mAP metric is computed with ViTPose++ pretrained on AP-10K and Human-Art, which is disclosed in Section 4.1; this is a legitimate measurement concern because the estimator shares the annotation distribution used for training, but it is not a derivation-level circularity: the model is not trained to optimize ViTPose++ outputs, and the same metric is applied uniformly to baselines. The Appendix E limitation (a >25% pose gap versus real images) is an honest external check and does not make the central claim circular. References to prior works by overlapping authors (AP-10K, ViTPose++) are used as dataset and metric, not as an unverified load-bearing theorem. The CLIP-Score evaluation discrepancy from removing newly introduced tokens is explicitly explained and does not conceal a circular step. Overall, the central claims are supported by independently defined training losses and external comparisons, so no circularity is found.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the validity of the pose-estimator metric, the stability of cross-attention as a spatial grounding mechanism, and a set of validation-tuned hyperparameters (loss weight, time-step range, attention block). The learned embeddings and ControlNet weights are trained, not free parameters, but the design choices above are selected based on validation performance.

free parameters (4)
  • heatmap loss weight eta = 0.1
    Set by validation search (Appendix Fig. S16) to balance Lldm and Lht; impacts both pose mAP and CLIP-Score.
  • time step range for heatmap loss = 250-500
    Selected via ablation (Fig. 7b) as most informative for keypoint concept learning; other ranges hurt performance.
  • transformer block for heatmap loss = 3rd block (DownBlock.2)
    Selected via ablation (Fig. 7c); using this block alone matches all-block performance.
  • keypoint embedding dimension C' = 3
    Chosen to match OpenPose channels; ablations show that other dimensions give competitive results, so this is a design choice rather than a critical fitted parameter.
assumptions (3)
  • domain assumption Stable Diffusion v1.5 and the ControlNet adapter behave as described, and the ControlNet architecture can be trained from scratch with SD encoder initialization.
    The method builds on these base models; their behavior is taken as given (Section 3.3).
  • domain assumption Cross-attention maps in the 3rd transformer block of the denoising U-Net encode spatial layout of keypoint tokens, and constraining them with a heatmap improves pose alignment.
    Supported by DAAM and prompt-to-prompt studies, but assumed here as a generalizable mechanism.
  • domain assumption The OKS-based mAP from ViTPose++ is a valid measure of pose alignment for generated images.
    Used as the primary metric; its biases are not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethink Sparse Signals for Pose-guided Text-to-image Generation." pith.science (2026). https://pith.science/paper/PKCEJDMO

@misc{pith2026250620983,
  author       = {Pith},
  title        = {Pith review of: Rethink Sparse Signals for Pose-guided Text-to-image Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKCEJDMO}},
  note         = {Machine review of arXiv:2506.20983}
}
read the original abstract

Recent works favored dense signals (e.g., depth, DensePose), as an alternative to sparse signals (e.g., OpenPose), to provide detailed spatial guidance for pose-guided text-to-image generation. However, dense representations raised new challenges, including editing difficulties and potential inconsistencies with textual prompts. This fact motivates us to revisit sparse signals for pose guidance, owing to their simplicity and shape-agnostic nature, which remains underexplored. This paper proposes a novel Spatial-Pose ControlNet(SP-Ctrl), equipping sparse signals with robust controllability for pose-guided image generation. Specifically, we extend OpenPose to a learnable spatial representation, making keypoint embeddings discriminative and expressive. Additionally, we introduce keypoint concept learning, which encourages keypoint tokens to attend to the spatial positions of each keypoint, thus improving pose alignment. Experiments on animal- and human-centric image generation tasks demonstrate that our method outperforms recent spatially controllable T2I generation approaches under sparse-pose guidance and even matches the performance of dense signal-based methods. Moreover, SP-Ctrl shows promising capabilities in diverse and cross-species generation through sparse signals. Codes will be available at https://github.com/DREAMXFAR/SP-Ctrl.

Figures

Figures reproduced from arXiv: 2506.20983 by the authors.

Figure 1
Figure 1. Performance of dense (i.e., depth) and sparse signals for pose-guided text-to-image generation. Though sparse pose signals are simple, they are supposed to suffer from poor pose control than dense signals. Our work improves the pose control of sparse signals, achieving competitive pose alignment with dense signals while enjoying image diversity and cross-species generalization capability. Abstract Recent works favor… view at source ↗
Figure 2
Figure 2. The overall architecture of our spatial-pose ControlNet ( [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of OpenPose and our spatial pose. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Visualized cross-attention maps of ControlNet at time [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Visualized comparison of our SP-Ctrl and other popular methods. See Appendix Fig. S8 and Fig. S9 for more examples. OpenPose Keypoint Positions Skeleton Links Pose vs. Fixed RGB Embeddings Skeleton (𝑥𝑥0, 𝑦𝑦0, v0) 𝒌𝒌𝟎𝟎 ··· ··· (𝒌𝒌𝟏𝟏, 𝒌𝒌𝟐𝟐) (𝒌𝒌𝟏𝟏, 𝒌𝒌𝟓𝟓) (𝒌𝒌𝟐𝟐, 𝒌𝒌𝟑𝟑) ··· …
Figure 6
Figure 6. Figure 6: Discussions on the spatial-pose representation. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Discussions on the keypoint concept learning strategy. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 11
Figure 11. Figure 11: Examples of shape diversity, where our method syn [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 9
Figure 9. Figure 9: Visualized cross-attention maps of SP-Ctrl at time step 399, where the net keypoint tokens {⟨ki⟩}N i=1 attend to spatial positions of keypoints. Full results refer to Appendix Fig. S7. A clear phot of giraffe on the mugshot. This is a clear photo of leopard in the zoo.…
Figure 13
Figure 13. Figure 13: Examples of synthesized images with edited sparse [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 48 canonical work pages

  1. [1]

    Spatext: Spatio-textual representation for con- trollable image generation

    Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin. Spatext: Spatio-textual representation for con- trollable image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18370–18380, 2023. 2, 3, 6

    work page 2023

  2. [2]

    Ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. Ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 7

    arXiv 2022

  3. [3]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 8

    arXiv 2023

  4. [4]

    Person image synthesis via de- noising diffusion model

    Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Jorma Laaksonen, Mubarak Shah, and Fahad Shahbaz Khan. Person image synthesis via de- noising diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5968–5976, 2023. 2

    work page 2023

  5. [5]

    Openpose: Realtime multi-person 2d pose estimation using part affinity fields

    Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43(1):172–186,

    work page

  6. [6]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffu- sion

    Di Chang, Yichun Shi, Quankai Gao, Hongyi Xu, Jessica Fu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mohammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffu- sion. In 41st International Conference on Machine Learning,

    work page

  7. [7]

    Up- gpt: Universal diffusion model for person image generation, editing and pose transfer

    Soon Yau Cheong, Armin Mustafa, and Andrew Gilbert. Up- gpt: Universal diffusion model for person image generation, editing and pose transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4173– 4182, 2023. 2

    work page 2023

  8. [8]

    Openmmlab pose estimation tool- box and benchmark

    MMPose Contributors. Openmmlab pose estimation tool- box and benchmark. https://github.com/open- mmlab/mmpose, 2020. 5, 12

    work page 2020

Show all 59 references
  1. [9]

    Make-a-scene: Scene- based text-to-image generation with human priors

    Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scene: Scene- based text-to-image generation with human priors. In Eu- ropean Conference on Computer Vision , pages 89–106. Springer, 2022. 2

    2022

  2. [10]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 2, 12

    2022 arXiv

  3. [11]

    Controllable person image synthesis with pose-constrained latent diffusion

    Xiao Han, Xiatian Zhu, Jiankang Deng, Yi-Zhe Song, and Tao Xiang. Controllable person image synthesis with pose-constrained latent diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22768–22777, 2023. 2

    2023

  4. [12]

    Prompt-to-prompt image editing with cross-attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompt image editing with cross-attention control. In The 11th Interna- tional Conference on Learning Representations, 2023. 4

    2023

  5. [13]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. In EMNLP, 2021. 5

    2021

  6. [14]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 1, 2

    2024

  7. [15]

    Composer: Creative and controllable im- age synthesis with composable conditions

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: Creative and controllable im- age synthesis with composable conditions. In Proceedings of the 40th International Conference on Machine Learning , pages 13753–13773, 2023. 2

    2023

  8. [16]

    Stable-pose: Leveraging transformers for pose-guided text-to-image generation

    Wang Jiajun, Morteza Ghahremani, Li Yitong, Bj ¨orn Om- mer, and Christian Wachinger. Stable-pose: Leveraging transformers for pose-guided text-to-image generation. Ad- vances in Neural Information Processing Systems , 2024. 1, 2

    2024

  9. [17]

    Spac-net: Synthetic pose- aware animal controlnet for enhanced pose estimation.arXiv preprint arXiv:2305.17845, 2023

    Le Jiang and Sarah Ostadabbas. Spac-net: Synthetic pose- aware animal controlnet for enhanced pose estimation.arXiv preprint arXiv:2305.17845, 2023. 1

    2023 arXiv

  10. [18]

    Skip-and-play: Depth-driven pose-preserved image generation for any objects

    Kyungmin Jo and Jaegul Choo. Skip-and-play: Depth-driven pose-preserved image generation for any objects. arXiv preprint arXiv:2409.02653, 2024. 1

    2024 arXiv

  11. [19]

    Human-art: A versatile human-centric dataset bridg- ing natural and artificial scenes

    Xuan Ju, Ailing Zeng, Jianan Wang, Qiang Xu, and Lei Zhang. Human-art: A versatile human-centric dataset bridg- ing natural and artificial scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 5, 12

    2023

  12. [20]

    Humansd: A native skeleton-guided diffusion model for human image generation

    Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd: A native skeleton-guided diffusion model for human image generation. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15988–15998, 2023. 1, 2, 5

    2023

  13. [21]

    Dreampose: Fashion image-to-video synthesis via stable diffusion

    Johanna Karras, Aleksander Holynski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffusion. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 22623–22633. IEEE, 2023. 1, 2

    2023

  14. [22]

    Segment anything in high quality

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In Advances in Neural Information Processing Systems, 2023. 8

    2023

  15. [23]

    Dense text-to-image generation with attention modulation

    Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. Dense text-to-image generation with attention modulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7701–7711, 2023. 4

    2023

  16. [24]

    Dispose: Disen- tangling pose guidance for controllable human image anima- tion

    Hongxiang Li, Yaowei Li, Yuhang Yang, Junjie Cao, Zhi- hong Zhu, Xuxin Cheng, and Long Chen. Dispose: Disen- tangling pose guidance for controllable human image anima- tion. arXiv preprint arXiv:2412.09349, 2024. 1

    2024 arXiv

  17. [25]

    Controlnet++: Improving conditional controls with efficient consistency feedback

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaon- ing Wang, Xuefeng Xiao, and Chen Chen. Controlnet++: Improving conditional controls with efficient consistency feedback. In European Conference on Computer Vision , pages 129–147. Springer, 2024. 1, 2

    2024

  18. [26]

    Ecnet: Effective controllable text-to-image diffusion models

    Sicheng Li, Keqiang Sun, Zhixin Lai, Xiaoshi Wu, Feng Qiu, Haoran Xie, Kazunori Miyata, and Hongsheng Li. Ecnet: Effective controllable text-to-image diffusion models. arXiv preprint arXiv:2403.18417, 2024. 1, 2

    2024 arXiv

  19. [27]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22511–22521, 2023. 2

    2023

  20. [28]

    Controllable text-to-3d generation via surface-aligned gaus- sian splatting

    Zhiqi Li, Yiming Chen, Lingzhe Zhao, and Peidong Liu. Controllable text-to-3d generation via surface-aligned gaus- sian splatting. In International Conference on 3D Vision ,

  21. [29]

    Ctrl-x: Controlling structure and appear- ance for text-to-image generation without guidance

    Kuan Heng Lin, Sicheng Mo, Ben Klingher, Fangzhou Mu, and Bolei Zhou. Ctrl-x: Controlling structure and appear- ance for text-to-image generation without guidance. In Ad- vances in Neural Information Processing Systems, 2024. 2

    2024

  22. [30]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision , pages 38–55. Springe...

    2024

  23. [31]

    Hyperhuman: Hyper-realistic human generation with latent structural diffusion

    Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu, Ziwei Liu, and Sergey Tulyakov. Hyperhuman: Hyper-realistic human generation with latent structural diffusion. In The 12th International Conference on Learning Representations, 2024. 1, 2

    2024

  24. [32]

    Smartcontrol: Enhancing controlnet for handling rough visual conditions

    Xiaoyu Liu, Yuxiang Wei, Ming Liu, Xianhui Lin, Peiran Ren, Xuansong Xie, and Wangmeng Zuo. Smartcontrol: Enhancing controlnet for handling rough visual conditions. In European Conference on Computer Vision , pages 1–17. Springer, 2024. 1

    2024

  25. [33]

    Controllable person image synthesis with attribute-decomposed gan

    Yifang Men, Yiming Mao, Yuning Jiang, Wei-Ying Ma, and Zhouhui Lian. Controllable person image synthesis with attribute-decomposed gan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,

  26. [34]

    Freecontrol: Training-free spatial control of any text-to-image diffusion model with any condition

    Sicheng Mo, Fangzhou Mu, Kuan Heng Lin, Yanli Liu, Bochen Guan, Yin Li, and Bolei Zhou. Freecontrol: Training-free spatial control of any text-to-image diffusion model with any condition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pa...

    2024

  27. [35]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 20...

    2024

  28. [36]

    Consolidating attention features for multi-view image editing

    Or Patashnik, Rinon Gal, Daniel Cohen-Or, Jun-Yan Zhu, and Fernando De la Torre. Consolidating attention features for multi-view image editing. InSIGGRAPH Asia 2024 Con- ference Papers, pages 1–12, 2024. 1

    2024

  29. [37]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 1

    2024 arXiv

  30. [38]

    Learn, imagine and create: Text-to-image generation from prior knowledge

    Tingting Qiao, Jing Zhang, Duanqing Xu, and Dacheng Tao. Learn, imagine and create: Text-to-image generation from prior knowledge. Advances in Neural Information Process- ing Systems, 32, 2019. 1

    2019

  31. [39]

    Mirrorgan: Learning text-to-image generation by re- description

    Tingting Qiao, Jing Zhang, Duanqing Xu, and Dacheng Tao. Mirrorgan: Learning text-to-image generation by re- description. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1505–1514,

  32. [40]

    Unicontrol: A unified diffu- sion model for controllable visual generation in the wild

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al. Unicontrol: A unified diffu- sion model for controllable visual generation in the wild. In Proceedings of the 37th International Conference ...

    2023

  33. [41]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 2, 4

    2022

  34. [42]

    Benchmarking and error diagnosis in multi-instance pose estimation

    Matteo Ruggero Ronchi and Pietro Perona. Benchmarking and error diagnosis in multi-instance pose estimation. InPro- ceedings of the IEEE international conference on computer vision, pages 369–378, 2017. 5

    2017

  35. [43]

    Stable Diffusion v1-5, 2022

    Runway. Stable Diffusion v1-5, 2022. 5

    2022

  36. [44]

    Advancing pose-guided image synthesis with pro- gressive conditional diffusion models

    Fei Shen, Hu Ye, Jun Zhang, Cong Wang, Xiao Han, and Yang Wei. Advancing pose-guided image synthesis with pro- gressive conditional diffusion models. In The 12th Interna- tional Conference on Learning Representations, 2024. 2

    2024

  37. [45]

    Deep high-resolution representation learning for human pose es- timation

    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose es- timation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 4

    2019

  38. [46]

    What the DAAM: Interpreting stable diffu- sion using cross attention

    Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. What the DAAM: Interpreting stable diffu- sion using cross attention. In Proceedings of the 61st Annual Meeting of the Association for Computational ...

    2023

  39. [47]

    Visualizing data using t-sne

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9 (86):2579–2605, 2008. 7, 8

    2008

  40. [48]

    Animatezoo: Zero-shot video generation of cross-species animation via subject alignment

    Yuanfeng Xu, Yuhao Chen, Zhongzhan Huang, Zijian He, Guangrun Wang, Philip Torr, and Liang Lin. Animatezoo: Zero-shot video generation of cross-species animation via subject alignment. arXiv preprint arXiv:2404.04946, 2024. 1, 2

    2024 arXiv

  41. [49]

    Vit- pose++: Vision transformer for generic body pose estima- tion

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vit- pose++: Vision transformer for generic body pose estima- tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2):1212–1230, 2024. 5

    2024

  42. [50]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  43. [51]

    When controlnet meets inexplicit masks: A case study of controlnet on its contour- following ability

    Wenjie Xuan, Yufei Xu, Shanshan Zhao, Chaoyue Wang, Juhua Liu, Bo Du, and Dacheng Tao. When controlnet meets inexplicit masks: A case study of controlnet on its contour- following ability. In Proceedings of the 32nd ACM Interna- tional Conference on Multimedia , pages 6979–698...

    2024

  44. [52]

    Grpose: Learning graph relations for human image generation with pose priors

    Xiangchen Yin, Donglin Di, Lei Fan, Hao Li, Chen Wei, Xi- aofei Gou, Yang Song, Xiao Sun, and Xun Yang. Grpose: Learning graph relations for human image generation with pose priors. Proceedings of the AAAI Conference on Artifi- cial Intelligence, 2025. 1, 2

    2025

  45. [53]

    Ap-10k: A benchmark for animal pose estima- tion in the wild

    Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. Ap-10k: A benchmark for animal pose estima- tion in the wild. In 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2021. 2, 5, 12

    2021

  46. [54]

    Pise: Person image synthesis and editing with decoupled gan

    Jinsong Zhang, Kun Li, Yu-Kun Lai, and Jingyu Yang. Pise: Person image synthesis and editing with decoupled gan. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 7982–7990, 2021. 2

    2021

  47. [55]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1, 2, 4, 13

    2023

  48. [56]

    Uni-controlnet: All-in-one control to text-to-image diffusion models

    Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan-Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Sys- tems, 36, 2024. 1, 2

    2024

  49. [57]

    Unipc: A unified predictor-corrector framework for fast sampling of diffusion models

    Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 2023. 5

    2023

  50. [58]

    Cross attention based style distribution for controllable person image synthesis

    Xinyue Zhou, Mingyu Yin, Xinyuan Chen, Li Sun, Changxin Gao, and Qingli Li. Cross attention based style distribution for controllable person image synthesis. In European Con- ference on Computer Vision, pages 161–178. Springer, 2022. 2

    2022

  51. [59]

    Champ: Controllable and consistent human image an- imation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image an- imation with 3d parametric guidance. In European Confer- ence on Computer Vision , pages 145–162. Springer, 2024. 1 ...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.