The reviewed record of science sign in
Pith

arxiv: 2606.19958 · v1 · pith:XWJ5DOJJ · submitted 2026-06-18 · cs.CV

SketchKeyAnime: Reference-anchored Sparse Key-Sketch Animation Synthesis

Meixi Li , Xianlin Zhang , Yue Zhang , Xueming Li This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-26 18:41 UTCgrok-4.3pith:XWJ5DOJJrecord.jsonopen to challenge →

classification cs.CV
keywords sketch-guided animationvideo diffusionsparse key sketchesreference image conditioningtemporal coherenceanimation synthesisdual-branch conditioningadaptive loss
0
0 comments X

The pith

A video diffusion model generates controllable animations from one reference image and sparse key sketches.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a video diffusion framework can produce structurally controllable, appearance-consistent, and temporally coherent animations when given only a single reference RGB image plus a handful of temporally indexed key sketches. It does so by encoding local geometric constraints from the sketches together with semantic-temporal context from the reference, then fusing them through a specialized attention mechanism while applying stronger supervision on the key frames. This matters to a reader because most existing animation and video methods require dense frame conditions, complete sketch sequences, or RGB boundary frames, which raises the manual effort needed for production. If the claim holds, animation workflows could shift toward lower-cost inputs while still preserving fidelity to the provided sketches and overall coherence across time.

Core claim

Given a single reference RGB image and a few temporally indexed key sketches, SketchKeyAnime introduces a dual-branch conditioning mechanism to encode local geometric constraints alongside semantic-temporal context. It leverages Sketch Cross Attention to fuse reference image and sketch conditions with learnable gating, and incorporates an Adaptive Weighted Loss to strengthen supervision on key-sketch frames and line-art regions. Experimental results on the Aesthetic subset of Sakuga-42M show that the approach consistently outperforms representative animation interpolation and sketch-guided generation baselines, reducing EDMD by 31.9% and FVD by 9.5% versus the best-performing baseline while

What carries the argument

Dual-branch conditioning mechanism that encodes geometric constraints from sketches and semantic-temporal context from the reference, fused via Sketch Cross Attention with learnable gating and trained under an Adaptive Weighted Loss.

If this is right

  • Animation production can proceed with far fewer drawn frames while retaining control over structure and appearance.
  • Sketch fidelity and temporal smoothness improve relative to methods that rely on dense conditions or RGB boundaries.
  • The same reference image can anchor multiple different sketch sequences without loss of character consistency.
  • Supervision focused on key frames and line-art regions yields measurable gains in both quantitative metrics and visual coherence.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same sparse-conditioning strategy could be tested on non-animation video tasks such as motion transfer or scene editing where only a few guide frames are available.
  • Adaptive weighting on structurally important regions might transfer to other diffusion models that need to respect partial line drawings or edge maps.
  • Interactive tools could let artists add or edit a few key sketches and immediately see updated full sequences without regenerating from scratch.

Load-bearing premise

The chosen test set and baselines give a fair measure of how well the dual-branch mechanism and adaptive loss handle sparse key-sketch inputs without results being driven by dataset-specific tuning.

What would settle it

A follow-up experiment that replaces the dual-branch conditioning and adaptive loss with a single-branch baseline while keeping all other factors fixed, then measures whether the reported reductions in EDMD and FVD disappear on the same inputs.

Figures

Figures reproduced from arXiv: 2606.19958 by Meixi Li, Xianlin Zhang, Xueming Li, Yue Zhang.

Figure 1
Figure 1. Figure 1: Overview of the proposed reference-anchored sparse key-sketch animation synthesis task. Given a single RGB reference image [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the SketchKeyAnime architecture. The framework takes a reference image and sparse temporally indexed key [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Semantic-Temporal Sketch Context Encoder. Given sparse key sketches with their temporal indices, a frozen CLIP image encoder [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison with SparseCtrl, ToonCrafter, AMT, and EISAI under sparse key-sketch conditions. For baselines that [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison with SparseCtrl, ToonCrafter, AMT, and EISAI under sparse key-sketch conditions. For baselines that [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative ablation comparison of SketchKeyAnime components under sparse key-sketch conditions. The compared variants [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
read the original abstract

Traditional animation production relies heavily on manual drawing and iterative refinement, particularly for key-pose design, in-betweening, and character coloring. While existing animation and video generation methods have made notable progress, they typically depend on RGB boundary frames, dense frame-wise conditions, or complete sketch sequences, limiting their applicability under low-cost input conditions. We present SketchKeyAnime, a video diffusion framework for generating structurally controllable, appearance-consistent, and temporally coherent animations from sparse key-sketch inputs. Given a single reference RGB image and a few temporally indexed key sketches, SketchKeyAnime introduces a dual-branch conditioning mechanism to encode local geometric constraints alongside semantic-temporal context. It leverages Sketch Cross Attention to fuse reference image and sketch conditions with learnable gating, and incorporates an Adaptive Weighted Loss to strengthen supervision on key-sketch frames and line-art regions. Experimental results on the Aesthetic subset of Sakuga-42M show that our approach consistently outperforms representative animation interpolation and sketch-guided generation baselines. Compared to the best-performing baseline, SketchKeyAnime reduces EDMD by 31.9\% and FVD by 9.5\%, demonstrating superior sketch fidelity and temporal coherence, while achieving the best overall performance across most quantitative metrics. These results validate the proposed framework and highlight its potential for low-cost, highly controllable animation creation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper presents SketchKeyAnime, a video diffusion framework for synthesizing structurally controllable, appearance-consistent, and temporally coherent animations from a single reference RGB image plus a few temporally indexed key sketches. It introduces a dual-branch conditioning mechanism, Sketch Cross Attention with learnable gating to fuse reference and sketch conditions, and an Adaptive Weighted Loss that strengthens supervision on key-sketch frames and line-art regions. On the Aesthetic subset of Sakuga-42M, the method outperforms representative animation interpolation and sketch-guided baselines, reducing EDMD by 31.9% and FVD by 9.5% relative to the best baseline while achieving the best overall performance on most metrics.

Significance. If the reported gains prove robust, the work addresses a practical gap in low-cost animation pipelines by enabling high-quality output from sparse inputs rather than dense RGB or sketch sequences. The quantitative improvements on a large dataset (Sakuga-42M) provide concrete evidence of progress in controllable video generation; the dual-branch design and adaptive loss are plausible mechanisms for balancing geometric fidelity with temporal coherence.

major comments (3)
  1. [Abstract] Abstract (experimental results paragraph): The selection criteria and construction details for the 'Aesthetic subset' of Sakuga-42M are not provided, so it is impossible to determine whether the 31.9% EDMD and 9.5% FVD reductions reflect genuine advances in the sparse key-sketch regime or are artifacts of subset curation (e.g., favoring high-contrast line art). This directly affects the central claim that the dual-branch mechanism and adaptive loss deliver generalizable improvements.
  2. [Abstract / §4] Abstract (experimental results paragraph) and §4 (Experiments): No ablation studies isolate the contribution of Sketch Cross Attention (including its learnable gating weights) or the Adaptive Weighted Loss (including per-region weights) to key-frame fidelity or temporal coherence. Without these, the performance deltas cannot be attributed to the proposed components rather than dataset-specific tuning or baseline re-implementation choices.
  3. [Abstract] Abstract (experimental results paragraph): The paper compares against 'representative animation interpolation and sketch-guided generation baselines' but provides no evidence that these baselines were re-tuned or adapted for the exact sparse temporal indexing used in the evaluation; if the baselines were not optimized for the same input sparsity, the reported margins may not fairly test the sparse-input regime.
minor comments (2)
  1. [Abstract] The abstract states the method 'consistently outperforms' baselines but does not report error bars, statistical significance tests, or the number of runs, which would strengthen the quantitative claims.
  2. [Method] Notation for the learnable gating weights in Sketch Cross Attention and the per-region weights in the Adaptive Weighted Loss should be introduced with explicit equations in the method section to improve reproducibility.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the presentation of our results. We address each major comment point by point below and indicate planned revisions to the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract (experimental results paragraph): The selection criteria and construction details for the 'Aesthetic subset' of Sakuga-42M are not provided, so it is impossible to determine whether the 31.9% EDMD and 9.5% FVD reductions reflect genuine advances in the sparse key-sketch regime or are artifacts of subset curation (e.g., favoring high-contrast line art). This directly affects the central claim that the dual-branch mechanism and adaptive loss deliver generalizable improvements.

    Authors: We agree that explicit details on subset construction are necessary for assessing generalizability. The Aesthetic subset was derived from Sakuga-42M by applying aesthetic quality filters and line-art visibility criteria to focus on representative animation cases. In the revised manuscript, we will add a new paragraph in §4 (Experiments) describing the exact selection process, filtering thresholds, and statistics of the subset to address this concern. revision: yes

  2. Referee: [Abstract / §4] Abstract (experimental results paragraph) and §4 (Experiments): No ablation studies isolate the contribution of Sketch Cross Attention (including its learnable gating weights) or the Adaptive Weighted Loss (including per-region weights) to key-frame fidelity or temporal coherence. Without these, the performance deltas cannot be attributed to the proposed components rather than dataset-specific tuning or baseline re-implementation choices.

    Authors: The referee correctly notes the absence of component-specific ablations. While the manuscript motivates the design choices through the overall framework, we acknowledge that isolating the contributions of Sketch Cross Attention (with gating) and the Adaptive Weighted Loss would strengthen the claims. We will add these ablation studies to the revised §4, reporting their impact on EDMD and FVD, either in the main paper or as supplementary material. revision: yes

  3. Referee: [Abstract] Abstract (experimental results paragraph): The paper compares against 'representative animation interpolation and sketch-guided generation baselines' but provides no evidence that these baselines were re-tuned or adapted for the exact sparse temporal indexing used in the evaluation; if the baselines were not optimized for the same input sparsity, the reported margins may not fairly test the sparse-input regime.

    Authors: We confirm that the baselines were adapted to accept the same single reference image plus sparse key sketches by adjusting their input conditioning pipelines accordingly. However, the manuscript does not detail these adaptations. In the revision, we will expand the baseline descriptions in §4 to explicitly document the modifications made for the sparse temporal indexing, ensuring transparency in the comparison setup. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical performance claims rest on external baselines and dataset splits, not self-referential definitions or fitted predictions.

full rationale

The paper introduces a dual-branch video diffusion architecture with Sketch Cross Attention and Adaptive Weighted Loss, then reports quantitative improvements (EDMD, FVD) on the Aesthetic subset of Sakuga-42M versus external baselines. No derivation chain, uniqueness theorem, or first-principles result is claimed; the central assertions are empirical comparisons that do not reduce to the model's own fitted parameters or self-citations by construction. The method is self-contained against the stated benchmarks.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard assumptions of conditional video diffusion models plus the effectiveness of the two newly introduced mechanisms; no independent verification of those mechanisms is supplied in the abstract.

free parameters (2)
  • learnable gating weights in Sketch Cross Attention
    Parameters that control fusion of reference image and sketch features; optimized during training.
  • per-region weights in Adaptive Weighted Loss
    Scalars that increase supervision on key-sketch frames and line-art regions; chosen or learned to emphasize those areas.
axioms (1)
  • domain assumption Conditional video diffusion models can encode both local geometric constraints and semantic-temporal context from mixed image-sketch inputs.
    Core modeling choice invoked to justify the dual-branch design.

pith-pipeline@v0.9.1-grok · 5765 in / 1233 out tokens · 42044 ms · 2026-06-26T18:41:44.951734+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 3 canonical work pages · 1 internal anchor

  1. [1]

    Flipsketch: Flipping static drawings to text-guided sketch animations

    Hmrishav Bandyopadhyay and Yi-Zhe Song. Flipsketch: Flipping static drawings to text-guided sketch animations. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 28394–28404, June 2025. 3

    2025

  2. [2]

    Depth-aware video frame interpolation

    Wenbo Bao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, and Ming-Hsuan Yang. Depth-aware video frame interpolation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3703–3712, 2019. 3

    2019

  3. [3]

    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling la- tent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2, 4, 7

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  4. [4]

    Learning to generate line drawings that convey geometry and semantics

    Caroline Chan, Fr ´edo Durand, and Phillip Isola. Learning to generate line drawings that convey geometry and semantics. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7915–7925, June 2022. 7

    2022

  5. [5]

    Improving the per- ceptual quality of 2d animation interpolation

    Shuhong Chen and Matthias Zwicker. Improving the per- ceptual quality of 2d animation interpolation. InComputer Vision – ECCV 2022, Lecture Notes in Computer Science, pages 271–287. Springer, 2022. 2, 3, 8

    2022

  6. [6]

    SEINE: Short-to-long video diffu- sion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. SEINE: Short-to-long video diffu- sion model for generative transition and prediction. InIn- ternational Conference on Learning Representations (ICLR). OpenReview.net, 2024. 3

    2024

  7. [7]

    LDMVFI: Video frame interpolation with latent diffusion models

    Duolikun Danier, Fan Zhang, and David Bull. LDMVFI: Video frame interpolation with latent diffusion models. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 38, pages 1472–1480. AAAI Press, 2024. 3

    2024

  8. [8]

    Sparsectrl: Adding sparse con- trols to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse con- trols to text-to-video diffusion models. InComputer Vision – ECCV 2024, Lecture Notes in Computer Science, pages 330–348. Springer, 2024. 2, 8

    2024

  9. [9]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. InInter- national Conference on Learning Representations (ICLR). OpenReview.net, 2024. 2

    2024

  10. [10]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. InAdvances in Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc., 2017. 8

    2017

  11. [11]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InAdvances in Neural Informa- tion Processing Systems (NeurIPS), volume 33, pages 6840–

  12. [12]

    Curran Associates, Inc., 2020. 2, 4

    2020

  13. [13]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video dif- fusion models. InAdvances in Neural Information Process- ing Systems (NeurIPS), volume 35, pages 8633–8646. Cur- ran Associates, Inc., 2022. 2, 4

    2022

  14. [14]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8153–8163, 2024. 3

    2024

  15. [15]

    LVCD: Reference-based lineart video colorization with diffusion models.ACM Transactions on Graphics, 43(6), November

    Zhitong Huang, Mohan Zhang, and Jing Liao. LVCD: Reference-based lineart video colorization with diffusion models.ACM Transactions on Graphics, 43(6), November

  16. [16]

    Real-time intermediate flow estimation for video frame interpolation

    Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. InComputer Vision – ECCV 2022, Lecture Notes in Computer Science, pages 624–642. Springer, 2022. 3

    2022

  17. [17]

    Video in- terpolation with diffusion models

    Siddhant Jain, Daniel Watson, Eric Tabellion, Aleksander Ho?ynski, Ben Poole, and Janne Kontkanen. Video in- terpolation with diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7341–7351, June 2024. 3

    2024

  18. [18]

    Super slomo: High quality estimation of multiple intermediate frames for video interpolation

    Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9000–9008, 2018. 3

    2018

  19. [19]

    Vidsketch: Hand-drawn sketch-driven video gener- ation with diffusion control.Neural Networks, 196:108465,

    Lifan Jiang, Shuang Chen, Boxi Wu, Deng Cai, and Jiahui Zhang. Vidsketch: Hand-drawn sketch-driven video gener- ation with diffusion control.Neural Networks, 196:108465,

  20. [20]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15954–15964, 2023. 2

    2023

  21. [21]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding varia- tional bayes. InInternational Conference on Learning Rep- resentations (ICLR), 2014. 4

    2014

  22. [22]

    IFRNet: Intermediate feature refine network for effi- cient frame interpolation

    Lingtong Kong, Boyuan Jiang, Donghao Luo, Wenqing Chu, Xiaoming Huang, Ying Tai, Chengjie Wang, and Jie Yang. IFRNet: Intermediate feature refine network for effi- cient frame interpolation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1969–1978, 2022. 3

    1969

  23. [23]

    AdaCoF: Adaptive collaboration of flows for video frame interpolation

    Hyeongmin Lee, Taeoh Kim, Tae-Young Chung, Daehyun Pak, Yukyung Ban, and Sangyoun Lee. AdaCoF: Adaptive collaboration of flows for video frame interpolation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5316–5325, 2020. 3

    2020

  24. [24]

    Amt: All-pairs multi-field transforms for efficient frame interpolation

    Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun- Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- 12 tern Recognition (CVPR), pages 9801–9810, June 2023. 2, 3, 8

    2023

  25. [25]

    Ctrl- adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl- adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model. InInternational Conference on Learning Representations (ICLR). OpenRe- view.net, 2025. 3

    2025

  26. [26]

    Sketchvideo: Sketch-based video generation and editing

    Feng-Lin Liu, Hongbo Fu, Xintao Wang, Weicai Ye, Pengfei Wan, Di Zhang, and Lin Gao. Sketchvideo: Sketch-based video generation and editing. InProceedings of the Com- puter Vision and Pattern Recognition Conference (CVPR), pages 23379–23390, June 2025. 2, 3

    2025

  27. [27]

    Manganinja: Line art colorization with precise reference following

    Zhiheng Liu, Ka Leong Cheng, Xi Chen, Jie Xiao, Hao Ouyang, Kai Zhu, Yu Liu, Yujun Shen, Qifeng Chen, and Ping Luo. Manganinja: Line art colorization with precise reference following. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 5666– 5677, June 2025. 8

    2025

  28. [28]

    Sketch- between: Video-to-video synthesis for sprite animation via sketches

    Dagmar Lukka Loftsd ´ottir and Matthew Guzdial. Sketch- between: Video-to-video synthesis for sprite animation via sketches. InProceedings of the 17th International Confer- ence on the Foundations of Digital Games, pages 32:1–32:7. ACM, 2022. 3

    2022

  29. [29]

    Anidoc: Animation creation made easier

    Yihao Meng, Hao Ouyang, Hanlin Wang, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Zhiheng Liu, Yujun Shen, and Huamin Qu. Anidoc: Animation creation made easier. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 18187–18197, June 2025. 3

    2025

  30. [30]

    Video frame inter- polation via adaptive separable convolution

    Simon Niklaus, Long Mai, and Feng Liu. Video frame inter- polation via adaptive separable convolution. InProceedings of the IEEE International Conference on Computer Vision (ICCV), pages 261–270, 2017. 3

    2017

  31. [31]

    Sakuga-42m dataset: Scaling up cartoon re- search.arXiv preprint arXiv:2405.07425, 2024

    Zhenglin Pan. Sakuga-42m dataset: Scaling up cartoon re- search.arXiv preprint arXiv:2405.07425, 2024. 7

    work page arXiv 2024

  32. [32]

    PySceneDetect.https : //github.com/Breakthrough/PySceneDetect,

    PySceneDetect Authors. PySceneDetect.https : //github.com/Breakthrough/PySceneDetect,

  33. [33]

    Accessed: October 1, 2023. 7

    2023

  34. [34]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InProceedings of the 38th International Conference on Machine Learning (ICML), Proceedings of Machin...

    2021

  35. [35]

    Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless

    Fitsum A. Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless. FILM: Frame interpolation for large motion. InComputer Vision – ECCV 2022, Lecture Notes in Computer Science, pages 250–266. Springer, 2022. 3

    2022

  36. [36]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, June 2022. 2, 4

    2022

  37. [37]

    Reference-based deep line art video colorization.IEEE Transactions on Visualization and Com- puter Graphics, 29(6):2965–2979, 2023

    Min Shi, Jia-Qi Zhang, Shu-Yu Chen, Lin Gao, Yu-Kun Lai, and Fang-Lue Zhang. Reference-based deep line art video colorization.IEEE Transactions on Visualization and Com- puter Graphics, 29(6):2965–2979, 2023. 3

    2023

  38. [38]

    Deep ani- mation video interpolation in the wild

    Li Siyao, Shiyu Zhao, Weijiang Yu, Wenxiu Sun, Dimitris Metaxas, Chen Change Loy, and Ziwei Liu. Deep ani- mation video interpolation in the wild. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6587–6595, June 2021. 2, 3

    2021

  39. [39]

    De- noising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. De- noising diffusion implicit models. InInternational Confer- ence on Learning Representations (ICLR). OpenReview.net,

  40. [40]

    FVD: A new metric for video generation

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Rapha¨el Marinier, Marcin Michalski, and Sylvain Gelly. FVD: A new metric for video generation. InICLR Workshop on Deep Generative Models for Highly Structured Data. OpenReview.net, 2019. 8

    2019

  41. [41]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neu- ral Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc., 2017. 5

    2017

  42. [42]

    MCVD: Masked conditional video diffusion for prediction, generation, and interpolation

    Vikram V oleti, Alexia Jolicoeur-Martineau, and Chris Pal. MCVD: Masked conditional video diffusion for prediction, generation, and interpolation. InAdvances in Neural Infor- mation Processing Systems (NeurIPS), volume 35. Curran Associates, Inc., 2022. 3

    2022

  43. [43]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. InAdvances in Neural Informa- tion Processing Systems (NeurIPS), volume 36. Curran As- sociates, Inc., 2023. 2

    2023

  44. [44]

    Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Kemelmacher-Shlizerman, Aleksander Holynski, and Steven M. Seitz. Generative inbetweening: Adapting image-to-video models for keyframe interpolation. In International Conference on Learning Representations (ICLR). OpenReview.net, 2025. 2, 3

    2025

  45. [45]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity.IEEE Transactions on Image Pro- cessing, 13(4):600–612, 2004. 8

    2004

  46. [46]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Pa- pers. Association for Computing Machinery, 2024. 3

    2024

  47. [47]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7623–7633, 2023. 2

    2023

  48. [48]

    Tooncrafter: Generative cartoon interpolation.ACM Transactions on Graphics, 43(6), November 2024

    Jinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xin- tao Wang, Ying Shan, and Tien-Tsin Wong. Tooncrafter: Generative cartoon interpolation.ACM Transactions on Graphics, 43(6), November 2024. 2, 3, 7 13

    2024

  49. [49]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Hao Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. InComputer Vision – ECCV 2024, Lecture Notes in Computer Science, pages 399–417. Springer, 2024. 3

    2024

  50. [50]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1481–1490, 2024. 3

    2024

  51. [51]

    Sketch me a video.arXiv preprint arXiv:2110.04710, 2021

    Haichao Zhang, Gang Yu, Tao Chen, and Guozhong Luo. Sketch me a video.arXiv preprint arXiv:2110.04710, 2021. 3

    work page arXiv 2021

  52. [52]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, October 2023. 2

    2023

  53. [53]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), June 2018. 8

    2018

  54. [54]

    Thin-plate spline-based interpolation for animation line inbetweening

    Tianyi Zhu, Wei Shang, and Dongwei Ren. Thin-plate spline-based interpolation for animation line inbetweening. InProceedings of the AAAI Conference on Artificial Intelli- gence, volume 39, pages 10995–11003. AAAI Press, 2025. 2, 3 14

    2025