Pith. sign in

REVIEW 3 major objections 6 minor 55 references

BridgeIV: Bridging Customized Image and Video Generation through Test-Time Autoregressive Identity Propagation

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that autoregressive structure-and-texture propagation plus test-time reward optimization gives customized text-to-video generation subject-consistency gains of 7.8 CLIP-I and 13.1 DINO over the baseline.

desk verdict The STPM propagation scheme is a genuine new technique and the DINO gains look real, but the CLIP-I headline is partly an artifact of optimizing the same metric at test time. read the letter →

arxiv 2505.06985 v1 pith:2SWQXL7M submitted 2025-05-11 cs.CV

Panwen Hu , Jiehui Huang , Qiang Sun , Xiaodan Liang This is my paper · ORCID
classification cs.CV
keywords customizedtext-to-videogenerationsubjectidentityconsistencyattentionmappropagationmatchingflowtest-timerewardoptimizationlatentdiffusionmodelsstructureandtexture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that customized text-to-video generation can preserve a reference subject's identity across frames without retraining a video model on image data. It takes a customized text-to-image model, attaches a pretrained temporal motion module, and interleaves a structure-and-texture propagation module into the denoising loop. The module warps the subject's cross-attention map (structure) and self-attention values (texture) from a generated reference frame into every video frame, using a matching flow computed from intermediate diffusion features. A test-time reward optimization then nudges the latents toward higher similarity with the reference images. If correct, this closes much of the consistency gap that currently separates customized image generation from customized video generation.

What carries the argument

The central mechanism is autoregressive structure-and-texture propagation driven by a semantic matching flow, which is a displacement field that warps the subject token's cross-attention map (structure) and the self-attention value features (texture) from the reference frame to each next frame, gated by a soft foreground mask derived from the attention map. Test-time reward optimization then adjusts the latents by gradient ascent on latent-domain and pixel-domain similarity rewards. The whole loop runs inside the denoising U-Net's decoder features, so no extra training of the video model is needed beyond the initial text-to-image customization.

What would settle it

Compare the matching flow extracted at the noisiest denoising steps against ground-truth correspondences on a synthetic video with known rigid motion; if the flows are near-random at those steps and ablating them leaves the final consistency scores unchanged, the propagation module is not doing the claimed warping.

Watch

Extended reading notes

Core claim

The paper claims that the modality gap between a customized text-to-image model and a video model can be bridged by explicitly propagating the subject's structure and texture autoregressively. Structure is captured by the cross-attention map of the special subject token, texture by the value matrix of self-attention; semantic matching flows, computed from features of the second-to-last decoder layer of the denoising U-Net, warp these quantities from the reference frame into each subsequent frame. A soft foreground mask derived from the attention map restricts propagation to the subject. After generation, test-time reward optimization refines the latents using segmentation-masked cosine similarity in latent space and CLIP similarity in pixel space. On a public benchmark, each component contributes, with the full method improving CLIP-I by 7.8 and DINO by 13.1 over the baseline and outperforming zero-shot, two-stage, and previous tuning-based customized video methods.

Load-bearing premise

The load-bearing premise is that the matching flow computed from noisy intermediate diffusion features is accurate enough to warp the subject's structure and texture between frames, even early in denoising when latents are nearly pure noise.

Editorial extensions

If this is right

  • If the claim holds, a customized text-to-image model can be turned into a consistent video generator without collecting customized video training data or training a video adapter.
  • Structure propagation alone yields gains of 4.6 in CLIP-I and 10.5 in DINO over the baseline, making it the dominant component; texture propagation and reward optimization then close the remaining fine-detail gap.
  • Because propagation is autoregressive and mask-gated, background and motion dynamics from the base motion module are largely preserved while the subject identity is stabilized.
  • The reported gains over two-stage and zero-shot baselines imply that tuning-based identity representation plus test-time refinement is currently a stronger recipe for uncommon-subject consistency than training a general-purpose extractor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper does not pursue is applying the same attention-warping propagation to long-form video; since propagation is autoregressive, errors may accumulate over many frames, so a natural test is whether periodic re-anchoring to the reference image prevents drift.
  • The mask derived from the subject-token attention map is a soft prior, so for heavily occluded or tiny subjects the mask may be unreliable; a testable extension would replace it with a detection-based mask.
  • Test-time reward optimization is pointed at latent and CLIP rewards, but the same loop could be aimed at face-identity embeddings for human subjects, which the paper lists as a current limitation.
  • The cost of computing dense matching flows at every denoising step is not analyzed; subsampling flows every few steps or computing them at a single resolution would be a practical extension if the consistency gains survive.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes BridgeIV, a three-stage framework for customized text-to-video (CT2V) generation. In the customization stage, a text-to-image model is personalized via textual inversion and DreamBooth. In the second stage, a Structure and Texture Propagation Module (STPM) extracts subject structure from cross-attention maps and subject texture from self-attention value features of the customized T2I model, then warps and autoregressively propagates these features into each video frame during denoising using a matching flow computed from mid-level U-Net features. In the third stage, a Test-Time Reward Optimization (TTRO) method refines the noisy latents by gradient ascent on a latent-domain reward and a pixel-domain CLIP similarity reward. The method is evaluated on the Vico benchmark against two-stage, zero-shot, and tuning-based baselines, reporting a CLIP-I of 80.0 and a DINO score of 62.6, which are claimed as state-of-the-art.

Significance. If the reported gains hold, the STPM idea is a meaningful contribution: it leverages well-understood attention structures (cross-attention for layout, self-attention for texture) and transfers them to video through a learned correspondence flow, avoiding full video fine-tuning. The introduction of test-time reward optimization to CT2V is also a timely and potentially reusable idea. The DINO-based improvements are not directly optimized by the TTRO reward and therefore carry some independent evidence for the STPM claims. However, the central headline improvement of +7.8 in CLIP-I is partially self-confirming because the TTRO pixel reward directly maximizes the same CLIP embedding similarity that defines the evaluation metric; this must be addressed before the quantitative claims can be accepted at face value.

major comments (3)
  1. [§4.2, §5.1, Table 1] The TTRO pixel-domain reward ℜ_pixel in Section 4.2 is defined as the average CLIP cosine similarity between masked generated frames and masked reference images, while the evaluation metric CLIP-I in Section 5.1 is defined as the average CLIP cosine similarity between generated frames and reference subject images. The paper does not state that a different CLIP checkpoint or a different vision encoder is used for TTRO than for evaluation. Consequently, the +7.8 CLIP-I improvement over the baseline in Table 1 and the +1.1 CLIP-I gain attributed to the Reward component in Table 2 partly reflect direct gradient ascent on the evaluation metric rather than an independent measure of capability. The DINO gains are not directly optimized and thus support the system more credibly, but the headline CLIP-I claim, as reported, is circular. Please re-evaluate with an independent embedding model (e.g., DINOv2 or a different CLIP backbone), or explicitly justify that the TTRO CLIP checkpoint and evaluation CLIP are distinct and report the sensitivity to that choice.
  2. [Table 1, Table 2] No error bars, standard deviations, or significance tests are reported for any quantitative result. Given that the benchmark contains only 16 subjects and 24 prompts, and the margin over the best zero-shot method is only 2.2 CLIP-I points (77.8 vs. 80.0) and 2.7 DINO points (59.9 vs. 62.6), these differences may be within subject-level noise. Please report per-subject means and variances, and perform paired significance tests (e.g., Wilcoxon signed-rank) between the proposed method and the top baselines, and between ablation conditions.
  3. [§4.1 (Matching Flow), Table 2] The matching flow is the load-bearing component of STPM: structural and texture information is warped using correspondences computed from noisy features of the second-to-last decoder layer at every denoising step. The paper provides no ablation or analysis of this design choice. In particular, there is no sensitivity study on (i) which decoder layer provides the features, (ii) which denoising timesteps should use flow-based warping versus direct identity propagation, and (iii) the hyperparameters of TTRO such as the gradient scale λ and the number of reward iterations. Since the consistency gain depends on the accuracy of these correspondences early in the reverse process, an ablation that varies the feature layer and an evaluation of warping accuracy (e.g., via foreground mask overlap as in Figure 4) would substantially strengthen the central derivation. Please add such ablations or, at a minimum, clearly specify how all these choices were fixed and justify them empirically.
minor comments (6)
  1. [§4.1, Eq. (3)] The recursion in Eq. (3) is unclear: for f > 1 it defines ᴴ*_f = Warp(ᴴ*_f, o_{F_f → F_{f+1}}), which is forward-looking and references o_{F_J → F_{J+1}} for the final frame. Please write the recursion over f = 1..J explicitly, or correct the flow index (e.g., o_{F_{f-1} → F_f}) and state how the last frame is handled.
  2. [§4, §4.2] In the Method overview and in Section 4.2, the method is once referred to as "Test-Time Reward Optimization (TTRP)" and later as TTRO; please use a single acronym consistently.
  3. [§5.1] The DINO metric cites reference [50], "DINO: DETR with improved denoising anchor boxes," which is the detection method, while the text describes using ViT-S/16 DINO, the self-supervised vision transformer from Caron et al. Please cite the correct DINO paper and ensure the reference list entry matches.
  4. [Eq. (1)] The training objective in Eq. (1) uses ∥·∥ without a subscript and without a squared norm; standard LDM uses ∥·∥_2^2. Please fix for clarity.
  5. [§4.1] The text contains a typo, "Specifially"; please correct.
  6. [§5.3, Table 2] The ablation table omits the TPM-only condition (TPM without SPM). Showing this row would help isolate the marginal effect of texture propagation, which is currently only presented in combination with SPM.

Circularity Check

1 steps flagged · score 6.0 of 10

TTRO's pixel reward is a masked version of the CLIP-I metric, so the Reward-row CLIP-I gain is partly self-confirming; STPM and DINO results remain independent.

  1. fitted input called prediction [Section 4.2 (Test-time Reward Optimization, pixel reward Rpixl) and Section 5.1 (Evaluation Metrics, CLIP-I); Table 2 ablation row with Reward.]
    "the reward function in pixel domain is formulated as: Rpixl = 1 JN X f X n Clip(Ff ⊙ ˆMf, In ⊙ ˆMn). ... CLIP-I assesses visual similarity by calculating the average cosine similarity between the CLIP image embeddings of all generated frames and the embeddings of the original subject images."

    The TTRO pixel reward is a foreground-masked CLIP cosine similarity between generated frames and reference images; CLIP-I is the (unmasked) CLIP cosine similarity between generated frames and reference images. Both are attributed to the same off-the-shelf CLIP model [27], and the paper never states that a different CLIP checkpoint or feature space is used for the metric. Test-time reward optimization is therefore gradient ascent on a quantity that directly raises the later evaluation metric. The +1.1 CLIP-I gain credited to Reward in Table 2 is largely an optimization artifact: it measures convergence to the optimized objective rather than independent identity fidelity.

full rationale

The central STPM mechanism is not circular: structure is propagated using cross-attention maps of the special token and matching flows computed from decoder features, and texture is propagated through self-attention values; neither is fit to CLIP-I or DINO, and the reported SPM/TPM CLIP-I and DINO gains are externally measured. The customization stage relies on standard Textual Inversion and DreamBooth, which are not defined in terms of the evaluation metrics. The identified circularity is confined to the TTRO/CLIP-I relationship: the pixel reward explicitly optimizes CLIP similarity to the reference images, while CLIP-I is the paper's headline identity metric. Because the reward is a masked near-variant of the metric and the same model [27] is cited for both, the Reward-row CLIP-I improvement of +1.1 is partly forced by construction. The DINO gain (+13.1 total, +1.5 from Reward) and the qualitative comparisons are not directly optimized, so the overall framework retains independent support; the score reflects partial, not total, circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central method introduces no new entities beyond the framework. The load-bearing choices are a handful of unstated hyperparameters (TTRO lambda and iteration count, layer selections) and several assumptions inherited from attention-based editing and correspondence literature. The most fragile is the reliability of matching flows on noisy features, which is assumed rather than demonstrated.

free parameters (5)
  • TTRO gradient scale lambda = not reported
    Section 4.2 updates latents as z_hat = z_hat + lambda * grad(R_lat + R_pixel), but the value of lambda is never given. This is a hand-chosen hyperparameter the central TTRO result depends on.
  • TTRO iteration count = not reported
    The method is described as iterative but the number of refinement steps is not specified. Without it, the reported CLIP-I and DINO gains cannot be reproduced.
  • Matching flow feature layer = second-to-last decoder layer
    Section 4.1 selects this layer as a compromise between spatial resolution and semantic richness, an ad hoc modeling choice that determines flow accuracy.
  • Attended cross-attention layers = not specified
    The paper omits layer notation 'for simplicity' and does not state which layers receive the warped attention maps; the final result likely depends on this choice.
  • Customization training hyperparameters = not reported
    Textual inversion and DreamBooth steps, learning rates, and regularization weight are absent, so the quality of the customized base model is a hidden variable.
assumptions (5)
  • domain assumption Cross-attention maps of the special token encode the subject's spatial structure
    Section 3.2 and Section 4.1 rely on this to extract structural information, citing Prompt-to-Prompt and eDiff-I.
  • domain assumption Self-attention values control appearance attributes such as color and texture
    Section 3.2 uses this to justify texture propagation, citing prior personalization and editing work.
  • domain assumption Argmax cosine-similarity matching on noisy diffusion features yields correct semantic correspondence
    Section 4.1 assumes this throughout matching flow computation; only visualizations, not quantitative validation, are provided.
  • domain assumption Grounding-SAM segmentation on decoded frames produces accurate foreground masks
    TTRO's reward in Section 4.2 depends on these masks; segmentation failures would misguide the gradient and degrade the result.
  • domain assumption AnimateDiff's motion module is compatible with the customized T2I model
    The pipeline inflates the personalized Stable Diffusion with AnimateDiff; the paper itself notes this direct combination causes the distortions it aims to fix, so the assumption is that the propagation module can correct them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BridgeIV: Bridging Customized Image and Video Generation through Test-Time Autoregressive Identity Propagation." pith.science (2026). https://pith.science/paper/2SWQXL7M

@misc{pith2026250506985,
  author       = {Pith},
  title        = {Pith review of: BridgeIV: Bridging Customized Image and Video Generation through Test-Time Autoregressive Identity Propagation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2SWQXL7M}},
  note         = {Machine review of arXiv:2505.06985}
}
read the original abstract

Both zero-shot and tuning-based customized text-to-image (CT2I) generation have made significant progress for storytelling content creation. In contrast, research on customized text-to-video (CT2V) generation remains relatively limited. Existing zero-shot CT2V methods suffer from poor generalization, while another line of work directly combining tuning-based T2I models with temporal motion modules often leads to the loss of structural and texture information. To bridge this gap, we propose an autoregressive structure and texture propagation module (STPM), which extracts key structural and texture features from the reference subject and injects them autoregressively into each video frame to enhance consistency. Additionally, we introduce a test-time reward optimization (TTRO) method to further refine fine-grained details. Quantitative and qualitative experiments validate the effectiveness of STPM and TTRO, demonstrating improvements of 7.8 and 13.1 in CLIP-I and DINO consistency metrics over the baseline, respectively.

Figures

Figures reproduced from arXiv: 2505.06985 by the authors.

Figure 1
Figure 1. The first row presents the generated customized images, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The processes of the structure and texture injection stage (the second stage) and the latent enhancement stage (the third stage). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The visualization of normalized attention maps of the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The visualization of the warping results with the match [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The comparisons with different state-of-the-art CT2V methods, including two-stages methods, VideoStudio [ [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The visualization of the generated videos with different [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 22 canonical work pages

  1. [1]

    ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Ji- aming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image dif- fusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 3, 5

    arXiv 2022

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 1, 2, 3

    arXiv 2023

  3. [3]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 22563–22575, 2023. 1

    work page 2023

  4. [4]

    Still-moving: Customized video generation without customized video data

    Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, Omer Tov, Michael Rubinstein, Lior Wolf, Tali Dekel, Tomer Michaeli, and Inbar Mosseri. Still-moving: Customized video generation without customized video data. ACM Transactions on Graphics (TOG) , 43(6):1–11, 2024. 1, 2, 4

    work page 2024

  5. [5]

    Video- dreamer: Customized multi-subject text-to-video generation with disen-mix finetuning

    Hong Chen, Xin Wang, Guanning Zeng, Yipeng Zhang, Yuwei Zhou, Feilin Han, and Wenwu Zhu. Video- dreamer: Customized multi-subject text-to-video generation with disen-mix finetuning. arXiv preprint arXiv:2311.00990,

    arXiv

  6. [6]

    Sitcom-crafter: A plot-driven human motion generation sys- tem in 3d scenes

    Jianqi Chen, Panwen Hu, Xiaojun Chang, Zhenwei Shi, Michael Christian Kampffmeyer, and Xiaodan Liang. Sitcom-crafter: A plot-driven human motion generation sys- tem in 3d scenes. arXiv preprint arXiv:2410.10790 , 2024. 1

    arXiv 2024

  7. [7]

    Photoverse: Tuning-free image customization with text-to-image diffusion models

    Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, et al. Photoverse: Tuning-free image customization with text-to-image diffusion models. arXiv preprint arXiv:2309.05793, 2023. 2, 3

    arXiv 2023

  8. [8]

    Cats: Cost ag- gregation transformers for visual correspondence

    Seokju Cho, Sunghwan Hong, Sangryul Jeon, Yunsung Lee, Kwanghoon Sohn, and Seungryong Kim. Cats: Cost ag- gregation transformers for visual correspondence. Advances in Neural Information Processing Systems , 34:9011–9023,

Show all 55 references
  1. [9]

    An image is worth one word: Personalizing text-to- image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patash- nik, Amit H Bermano, Gal Chechik, and Daniel Cohen- Or. An image is worth one word: Personalizing text-to- image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. 1, 2, 3, 4

    2022 arXiv

  2. [10]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3

    2023 arXiv

  3. [11]

    Svdiff: Compact param- eter space for diffusion fine-tuning

    Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact param- eter space for diffusion fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7323–7334, 2023. 2

    2023

  4. [12]

    Vico: Plug-and-play visual condition for personalized text-to-image generation

    Shaozhe Hao, Kai Han, Shihao Zhao, and Kwan-Yee K Wong. Vico: Plug-and-play visual condition for personalized text-to-image generation. arXiv preprint arXiv:2306.00971,

    arXiv

  5. [13]

    Id-animator: Zero-shot identity-preserving human video generation

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id-animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275, 2024. 1, 2, 3

    2024 arXiv

  6. [14]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221 ,

    arXiv

  7. [15]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 3

    2022 arXiv

  8. [16]

    Neural matching fields: Implicit representation of matching fields for visual correspondence

    Sunghwan Hong, Jisu Nam, Seokju Cho, Susung Hong, San- gryul Jeon, Dongbo Min, and Seungryong Kim. Neural matching fields: Implicit representation of matching fields for visual correspondence. Advances in Neural Information Processing Systems, 35:13512–13526, 2022. 4, 5

    2022

  9. [17]

    The hadamard product

    Roger A Horn. The hadamard product. In Proc. symp. appl. math, pages 87–169, 1990. 6

    1990

  10. [18]

    A reinforcement learning-based automatic video editing method using pre-trained vision-language model

    Panwen Hu, Nan Xiao, Feifei Li, Yongquan Chen, and Rui Huang. A reinforcement learning-based automatic video editing method using pre-trained vision-language model. In Proceedings of the 31st ACM International Conference on Multimedia, pages 6441–6450, 2023. 1

    2023

  11. [19]

    Storyagent: Cus- tomized storytelling video generation via multi-agent collab- oration

    Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun Chang, and Xiaodan Liang. Storyagent: Cus- tomized storytelling video generation via multi-agent collab- oration. arXiv preprint arXiv:2411.04925, 2024. 1, 2

    2024 arXiv

  12. [20]

    Videobooth: Diffusion-based video generation with image prompts

    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. Videobooth: Diffusion-based video generation with image prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6689– 6700, 202...

    2024

  13. [21]

    Personalvideo: High id-fidelity video customization without dynamic and semantic degradation

    Hengjia Li, Haonan Qiu, Shiwei Zhang, Xiang Wang, Yu- jie Wei, Zekun Li, Yingya Zhang, Boxi Wu, and Deng Cai. Personalvideo: High id-fidelity video customization without dynamic and semantic degradation. arXiv preprint arXiv:2411.17048, 2024. 2

    2024 arXiv

  14. [22]

    Cones: Concept neurons in diffusion models for customized generation

    Zhiheng Liu, Ruili Feng, Kai Zhu, Yifei Zhang, Kecheng Zheng, Yu Liu, Deli Zhao, Jingren Zhou, and Yang Cao. Cones: Concept neurons in diffusion models for customized generation. arXiv preprint arXiv:2303.05125, 2023. 2

    2023 arXiv

  15. [23]

    Videostudio: Generating consistent-content and multi-scene videos

    Fuchen Long, Zhaofan Qiu, Ting Yao, and Tao Mei. Videostudio: Generating consistent-content and multi-scene videos. In European Conference on Computer Vision, pages 468–485. Springer, 2024. 2, 6, 7 9

    2024

  16. [24]

    Magic-me: Identity-specific video customized diffu- sion

    Ze Ma, Daquan Zhou, Chun-Hsiao Yeh, Xue-She Wang, Xi- uyu Li, Huanrui Yang, Zhen Dong, Kurt Keutzer, and Jiashi Feng. Magic-me: Identity-specific video customized diffu- sion. arXiv preprint arXiv:2402.09368, 2024. 2, 3, 6, 8

    2024 arXiv

  17. [25]

    Dreammatcher: appearance matching self-attention for semantically-consistent text-to- image personalization

    Jisu Nam, Heesu Kim, DongJae Lee, Siyoon Jin, Seungry- ong Kim, and Seunggyu Chang. Dreammatcher: appearance matching self-attention for semantically-consistent text-to- image personalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  18. [26]

    Attndreambooth: To- wards text-aligned personalized text-to-image generation

    Lianyu Pang, Jian Yin, Baoquan Zhao, Feize Wu, Fu Lee Wang, Qing Li, and Xudong Mao. Attndreambooth: To- wards text-aligned personalized text-to-image generation. Advances in Neural Information Processing Systems , 37: 39869–39900, 2024. 2

    2024

  19. [27]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  20. [28]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks,

  21. [29]

    Consisti2v: Enhanc- ing visual consistency for image-to-video generation

    Weiming Ren, Huan Yang, Ge Zhang, Cong Wei, Xinrun Du, Wenhao Huang, and Wenhu Chen. Consisti2v: Enhanc- ing visual consistency for image-to-video generation. arXiv preprint arXiv:2402.04324, 2024. 2

    2024 arXiv

  22. [30]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 3

    2021

  23. [31]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2250...

    2023

  24. [32]

    Customvideox: 3d reference attention driven dynamic adaptation for zero-shot customized video diffusion transformers

    D She, Mushui Liu, Jingxuan Pang, Jin Wang, Zhen Yang, Wanggui He, Guanghao Zhang, Yi Wang, Qihan Huang, Haobin Tang, et al. Customvideox: 3d reference attention driven dynamic adaptation for zero-shot customized video diffusion transformers. arXiv preprint arXiv:2502.06527 ,

    arXiv

  25. [33]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3, 6

    2010 arXiv

  26. [34]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Pro- cessing Systems, 36:1363–1389, 2023. 5

    2023

  27. [35]

    Key-locked rank one editing for text-to-image personaliza- tion

    Yoad Tewel, Rinon Gal, Gal Chechik, and Yuval Atzmon. Key-locked rank one editing for text-to-image personaliza- tion. In ACM SIGGRAPH 2023 conference proceedings , pages 1–11, 2023. 3

    2023

  28. [36]

    Pdc-net+: Enhanced probabilistic dense corre- spondence network

    Prune Truong, Martin Danelljan, Radu Timofte, and Luc Van Gool. Pdc-net+: Enhanced probabilistic dense corre- spondence network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):10247–10266, 2023. 4, 5

    2023

  29. [37]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1921–1930, 2023. 3

    1921

  30. [38]

    Dreamvideo: High-fidelity image-to- video generation with image retention and text guidance

    Cong Wang, Jiaxi Gu, Panwen Hu, Songcen Xu, Hang Xu, and Xiaodan Liang. Dreamvideo: High-fidelity image-to- video generation with image retention and text guidance. arXiv preprint arXiv:2312.03018, 2023. 1

    2023 arXiv

  31. [39]

    Easycontrol: Transfer controlnet to video diffusion for controllable generation and interpolation

    Cong Wang, Jiaxi Gu, Panwen Hu, Haoyu Zhao, Yuan- fan Guo, Jianhua Han, Hang Xu, and Xiaodan Liang. Easycontrol: Transfer controlnet to video diffusion for controllable generation and interpolation. arXiv preprint arXiv:2408.13005, 2024. 1

    2024 arXiv

  32. [40]

    Uniadapter: All-in-one control for flexible video generation

    Cong Wang, Panwen Hu, Haoyu Zhao, Yuanfan Guo, Jiaxi Gu, Xiao Dong, Jianhua Han, Hang Xu, and Xiaodan Liang. Uniadapter: All-in-one control for flexible video generation. IEEE Transactions on Circuits and Systems for Video Tech- nology, 2025. 1

    2025

  33. [41]

    Ms-diffusion: Multi-subject zero-shot im- age personalization with layout guidance

    Xiaowei Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. Ms-diffusion: Multi-subject zero-shot im- age personalization with layout guidance. arXiv preprint arXiv:2406.07209, 2024. 6

    2024 arXiv

  34. [42]

    Lavie: High-quality video generation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video generation with cascaded latent diffusion models. International Journal of Computer Vision, pages 1–20, 2024. 1

    2024

  35. [43]

    Customvideo: Customizing text-to- video generation with multiple subjects

    Zhao Wang, Aoxue Li, Lingting Zhu, Yong Guo, Qi Dou, and Zhenguo Li. Customvideo: Customizing text-to- video generation with multiple subjects. arXiv preprint arXiv:2401.09962, 2024. 2, 6

    2024

  36. [44]

    Humanvid: Demystifying training data for camera-controllable human image animation

    Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Youqing Fang, Yuwei Guo, Wenran Liu, Jing Tan, Kai Chen, Tianfan Xue, Bo Dai, et al. Humanvid: Demystifying training data for camera-controllable human image animation. Advances in Neural Information Processing Systems , 37:20111–20131,

  37. [45]

    Dreamvideo: Composing your dream videos with customized subject and motion

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhi- heng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hong- ming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  38. [46]

    Cus- tomcrafter: Customized video generation with preserving motion and concept composition abilities

    Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guang- cong Zheng, Zhongang Qi, Ying Shan, and Xi Li. Cus- tomcrafter: Customized video generation with preserving motion and concept composition abilities. arXiv preprint arXiv:2408.13239, 2024. 2

    2024 arXiv

  39. [47]

    Fastcomposer: Tuning-free multi- subject image generation with localized attention

    Guangxuan Xiao, Tianwei Yin, William T Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention. Interna- tional Journal of Computer Vision, pages 1–20, 2024. 1 10

    2024

  40. [48]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In Eu- ropean Conference on Computer Vision , pages 399–417. Springer, 202...

    2024

  41. [49]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 1

    2024 arXiv

  42. [50]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 7

    2022 arXiv

  43. [51]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems , 36:45533–45547,

  44. [52]

    Ssr-encoder: Encoding selective subject representation for subject-driven generation

    Yuxuan Zhang, Yiren Song, Jiaming Liu, Rui Wang, Jinpeng Yu, Hao Tang, Huaxia Li, Xu Tang, Yao Hu, Han Pan, et al. Ssr-encoder: Encoding selective subject representation for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  45. [53]

    Tora: Trajectory-oriented diffusion transformer for video genera- tion

    Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video genera- tion. arXiv preprint arXiv:2407.21705, 2024. 1

    2024 arXiv

  46. [54]

    Cami2v: Camera-controlled image-to-video dif- fusion model

    Guangcong Zheng, Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Xi Li. Cami2v: Camera-controlled image-to-video dif- fusion model. arXiv preprint arXiv:2410.15957, 2024. 1

    2024 arXiv

  47. [55]

    Vlogger: Make your dream a vlog

    Shaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, and Yali Wang. Vlogger: Make your dream a vlog. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8806–8817, 2024. 2 11

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.