Pith. sign in

REVIEW 2 major objections 5 minor 56 references

EF-VI: Enhancing End-Frame Injection for Video Inbetweening

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A lightweight module, EF-Net, injects temporally adaptive end-frame features into a transformer-based image-to-video diffusion model, strengthening the end-frame constraint for video inbetweening without reversing the temporal axis.

desk verdict Good adapter, believable gains, but the end-frame mechanism is not cleanly isolated; one targeted ablation would settle it. read the letter →

arxiv 2505.21205 v2 pith:LCKOSK3P submitted 2025-05-27 cs.CV

Liuhan Chen , Xiaodong Cun , Xiaoyu Li , Xianyi He , Shenghai Yuan , Jie Chen , Ying Shan , Li Yuan This is my paper
classification cs.CV
keywords videoinbetweeningkeyframeinterpolationdiffusionmodelsimage-to-videoend-frameinjectiontransformer-basedgenerationcontrollableEF-Net
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video inbetweening asks a model to fill in frames between a start and an end frame. The paper claims that the end frame is the under-used half of that pair: fine-tuning a large image-to-video diffusion model to take both frames leaves the end frame with weaker control, while the alternative of denoising the video backwards disrupts the model's learned video representation. To fix this, EF-VI keeps the start-frame path of the base model and adds a lightweight module, EF-Net, that encodes only the end frame and expands it into temporally adaptive, frame-wise features injected into the transformer blocks of the diffusion model. The result, the paper argues, is a stronger and more symmetric end-frame constraint without temporal reversal, and it reports consistent gains over seven baselines on DAVIS, Pexels, and cartoon benchmarks.

What carries the argument

EF-Net, a lightweight injection module. It takes the end frame $c_e$, patchifies it into tokens, passes them through $M=4$ transformer blocks (DiT blocks from CogVideoX), predicts token-wise temporal coefficients with a linear projection, expands the features $f$ times by an outer product to obtain frame-wise features, concatenates them with the patchified noised latent $z_t$, and passes the result through an MLP. These features are added to the outputs of the first $M$ blocks of the base I2V-DM, so the end frame explicitly influences the entire video's inner features while the model's temporal-forward representation stays intact.

What would settle it

Take a test pair from DAVIS, run EF-VI, then rerun it with the end frame replaced by an unrelated image (for example, a different scene) while keeping the text prompt fixed. If the middle-frame LPIPS to the original end frame barely changes, the end-frame injection is not doing the causal work the paper attributes to it; if swapping the end frame changes the generated middle frames substantially, the constraint is real.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the end-frame condition in I2V-diffusion-based inbetweening can be strengthened without reversing the temporal axis. EF-VI processes the start frame exactly as the base model does, and injects an extra path for the end frame: EF-Net encodes the end frame, turns it into frame-wise feature sets via a linear projection and outer product, conditions them on the noised latent by concatenation and an MLP, and adds them to the output of the first four DiT blocks. The paper's quantitative case is Eq. 9, $z_{t-1} = D_\theta(z_t; t, J(c_s), J(c_e), \text{EF-Net}(c_e))$, with Table 1 reporting LPIPS 0.2096, FID 22.30, FVD 382.03, and VBench 0.8240 on DAVIS, beating both direct fine-tuning and bidirectional sampling variants built on the same CogVideoX-5B-I2V base.

Load-bearing premise

The paper's central diagnostic is that average LPIPS distance from generated middle frames to each boundary frame measures how strongly each boundary constrains the generation; if those distances mostly reflect content similarity or the base model's prior instead, the claimed asymmetry between start and end control is not established.

Editorial extensions

If this is right

  • Direct fine-tuning of transformer-based I2V-DMs for inbetweening leaves the end frame under-constrained; EF-VI's injection path is what closes that gap, so alternative injection architectures should be measured against this baseline.
  • Temporal reversal (bidirectional sampling) is not a safe way to add an end-frame condition to transformer-based I2V-DMs with causal 3D VAEs, since flipping the latent falls outside the VAE's representation space.
  • The end-frame feature is cheap: EF-Net adds only about three seconds to a 50-step CogVideoX generation, versus roughly double the time for bidirectional sampling.
  • Training is efficient: 6,000 iterations on four A800 GPUs with a total batch size of 4 suffices to beat baselines trained with ten times the overhead.
  • The method transfers to cartoon video without retraining, producing the best scores on a 100-clip cartoon benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because EF-Net only touches the first four blocks and reads the noised latent, the same injection pattern should transfer to other transformer-based I2V-DMs (for example, Wan2.1-based inbetweening) without re-deriving the causal-3D-VAE fix; this is untested in the paper.
  • The LPIPS diagnostic assumes boundary proximity equals constraint strength. A sharper test would swap the end frame for a semantically unrelated image and check whether generated middle frames still track it; if they do, the improvement may reflect a smoothing prior rather than a true end-frame condition.
  • The outer-product expansion into frame-wise features creates a natural extension to multi-keyframe inbetweening: feed several keyframes through EF-Net and let the temporal coefficients blend them, which the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. EF-VI is a video inbetweening method built on the transformer-based image-to-video diffusion model CogVideoX-5B-I2V. The paper argues that existing I2V-DM-based approaches either inject the end frame too weakly (direct fine-tuning) or disrupt the input representation (temporally bidirectional sampling). To address this, the authors introduce EF-Net, a lightweight module that expands the end frame into temporally adaptive frame-wise features and injects them into early transformer blocks of the denoiser. The reported experiments on DAVIS, Pexels, and cartoon datasets show consistent improvements over seven baselines on LPIPS, FID, FVD, and VBench, together with a 32-participant user study and a small inference-time overhead relative to the base model.

Significance. If the empirical results are reproducible, this is a practically useful contribution: it provides an efficient way to adapt a powerful transformer-based I2V-DM to keyframe interpolation without doubling inference cost, and it supplies broad evidence including two real-world test sets, a cartoon generalization test, per-dimension VBench scores, a user study, and training/inference efficiency comparisons. The supplementary materials include code and data examples, which strengthens reproducibility. The main weakness is that the proposed mechanism—that EF-Net specifically strengthens the end-frame constraint—is not cleanly isolated from the module's additional noised-latent input, and the LPIPS-based diagnostic used to motivate the method is confounded by generation quality. These issues are fixable and do not undermine the empirical comparison itself.

major comments (2)
  1. [3.3, Eq. (12), Table 3] The central mechanism claim is not isolated by the ablations. The final injected feature is F^j = MLP(Concat(ˆF^j, Patchify(z_t))) in Eq. (12), so F^j depends on both the end-frame tokens and the noised latent z_t. The 'w/o EF-Net' row removes the entire module, and 'EF-Net (w/o z_t)' removes only the z_t branch; there is no ablation that removes the end-frame branch while keeping Patchify(z_t), nor a control that feeds a wrong end frame into EF-Net. The gain of Ours over w/o EF-Net in Table 3 could therefore be driven entirely by a learned shortcut from the noisy latent to early transformer blocks, rather than by end-frame information. Please add a ce-masked or wrong-end-frame control: keep Patchify(z_t) but suppress or replace the end-frame input to EF-Net. If the output changes substantially, the end-frame pathway is functional; if not, the paper's stated mechanism is not supported.
  2. [3.2, Fig. 2] The motivation that direct fine-tuning has a weak end-frame constraint, and that EF-VI makes the constraint more symmetric, relies on the average LPIPS distance between generated middle frames and the two boundary frames. This proxy is not validated as a measure of constraint strength: LPIPS is sensitive to overall sharpness, texture, and content similarity, and a lower-quality model can appear 'less constrained' by both boundaries. Moreover, the start frame is an exact input and the base model is pretrained to condition on it, so the asymmetry in Fig. 2(a) is expected even without a genuinely weaker end-frame constraint. Please add a control that holds generation quality approximately fixed, or report a boundary-fidelity measure that is less confounded by global quality, before using Fig. 2 as evidence for the proposed mechanism.
minor comments (5)
  1. [3.2, after Eq. (8)] The sentence 'As shown in Fig. 1 (b), the LPIPS values ...' refers to LPIPS curves, but Fig. 1(b) is a qualitative frame comparison; this should be Fig. 2(b).
  2. [Appendix A, reference [17]] The claim that CogVideoX-5B-I2V uses a causal 3D VAE is supported by citing [17], which is the authors' own WF-VAE paper rather than the CogVideoX VAE source. The supplementary's Fig. 6 is direct evidence for the reversal failure, so please cite the actual CogVideoX/VAE documentation or rely on Fig. 6 to avoid a self-citation gap.
  3. [4.3, Table 3] The description of 'EF-Net (w/o z_t)' is incomplete: when Patchify(z_t) is removed from Eq. (12), what is the input to the MLP, and are the token-wise temporal coefficients still produced in the same way? Please specify the exact architecture of this variant.
  4. [3.3, Eq. (13)] The 'lightweight' claim is supported only by wall-clock inference time; please report the added parameter count and FLOPs of EF-Net relative to the 5B base model.
  5. [4.2, Table 1] All quantitative results appear to come from a single evaluation run; given the stochastic nature of diffusion sampling and the small metric gaps between some baselines (e.g., VBench 0.8240 vs 0.8214), please report results over multiple seeds or at least state the inference seed.

Circularity Check

2 steps flagged · score 2.0 of 10

No derivation reduces to its inputs; only a minor self-citation and a same-metric validation loop, so the empirical claims stand.

  1. other [Sec. 3.2, Fig. 2 (motivation and validation)]
    "We quantify this by calculating the average LPIPS [49] (a lower value indicating greater similarity) between the generated middle frames and the two boundary frames on 119 test pairs from the DA VIS dataset [27]. ... As shown in Fig. 2 (c), compared to the other two types of methods, the LPIPS curves of our EF-VI are closer to those of the GT. This proves that our method can achieve a more symmetric constraint of the start and end frames."

    The same unvalidated proxy, average LPIPS between the generated middle frames and the two boundary frames, is used both to establish the premise that direct fine-tuning has a weak end-frame constraint and to conclude that EF-VI fixes that weakness. Because the proxy is never independently validated as a measure of constraint strength, and because Eq. 12 feeds z_t into the EF-Net features so the end-frame pathway is not isolated, the validation loop supports the paper's interpretation rather than independently establishing the mechanism. This is a mild evidentiary circularity, not a mathematical reduction, and it does not force the reported headline metrics.

  2. self citation load bearing [Sec. 3.2 and Appendix A; reference [17]]
    "recent and more powerful transformer-based I2V-DMs utilize causal 3D-V AEs [17] for temporal compression of video frames. In this case, zt has a strong temporal-forward causal relationship, and Flip(zt) falls outside the representation space of 3D-V AEs, detailed in Appendix A."

    The premise that flipping the temporally compressed latent disrupts the input representation is attributed to [17], WF-VAE, which shares authors with the present paper. However, the paper also provides its own visual reconstruction evidence in Appendix A and a CogVX-BD quantitative result in Table 1, so the self-citation is not the sole or load-bearing support. It is a minor self-referential citation rather than a derivation that collapses into the authors' prior work.

full rationale

The paper's central contribution is an architectural and empirical one: EF-Net injects frame-wise features into CogVideoX-5B-I2V, and the method is evaluated on held-out DAVIS and Pexels sets. Equations (1)-(13) are definitions and forward-pass computations; no equation is shown to equal another by construction, and no fitted parameter is renamed as a prediction. The reported LPIPS, FID, FVD, and VBench numbers are external benchmark results, not outputs of a fitted model evaluated on its own training targets. The two flagged issues are mild. First, the same average-LPIPS boundary-distance statistic is used both to motivate the weak end-frame constraint and to validate the claimed symmetric constraint, and because EF-Net's injected features also depend on z_t through Eq. 12, the end-frame attribution is not fully isolated; this is a validity/confound concern rather than a circular reduction. Second, the causal-3D-VAE claim that temporal reversal disrupts the input representation cites the authors' own WF-VAE paper, but the appendix and the CogVX-BD experiments provide independent supporting evidence. Neither issue makes the central result equivalent to its inputs, so the appropriate score is low.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

EF-Net is a trainable neural module, not an invented physical entity; no new particles, forces, dimensions, or conserved quantities are introduced. The central claim rests on design choices and empirical assumptions rather than on new ontological commitments.

free parameters (4)
  • Number of EF-Net transformer blocks M = 4
    Hand-selected and much smaller than the 42 base blocks; the paper does not ablate M, so this design choice is not shown to be optimal.
  • EF-Net feature scale w = 1.0
    Ablated in Table 4; w=1.0 is best on LPIPS, FID, and FVD, but w=1.5 and w=2.0 slightly improve VBench, so the chosen value is a trade-off.
  • Training iterations, batch size, learning rate = 6000 / 4 / 3e-5
    Set to fit the compute budget; no sensitivity analysis is reported.
  • Inference steps = 50
    Matches the officially recommended setting for CogVideoX-5B-I2V and is not ablated.
assumptions (4)
  • domain assumption LPIPS distance between generated middle frames and boundary frames is a valid proxy for the strength of the end-frame constraint
    Section 3.2 and Fig. 2 use LPIPS to identify the weak end-frame constraint and to demonstrate improved symmetry after training; no external validation of this proxy is provided.
  • domain assumption CogVideoX-5B-I2V's causal 3D-VAE makes temporally reversed latents fall outside the input representation space, so bidirectional sampling is harmful
    Appendix A supports this with reconstruction experiments in Fig. 6, but the paper assumes these artifacts are caused by the VAE representation rather than by the sampling or decoding setup.
  • domain assumption The pre-trained I2V-DM's generative prior transfers to the inbetweening task and to the iStock training distribution
    The method relies on fine-tuning a 5B model with only 6,000 iterations, which presumes the base model already knows most of the dynamics needed.
  • standard math Standard diffusion training and inference math
    Equations (1)-(3) use a standard noise schedule, denoising objective, and sampling update from CogVideoX; these are not re-derived.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EF-VI: Enhancing End-Frame Injection for Video Inbetweening." pith.science (2026). https://pith.science/paper/LCKOSK3P

@misc{pith2026250521205,
  author       = {Pith},
  title        = {Pith review of: EF-VI: Enhancing End-Frame Injection for Video Inbetweening},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LCKOSK3P}},
  note         = {Machine review of arXiv:2505.21205}
}
read the original abstract

Video inbetweening aims to synthesize intermediate video sequences conditioned on the given start and end frames. Current state-of-the-art methods primarily extend large-scale pre-trained Image-to-Video Diffusion Models (I2V-DMs) by incorporating the end-frame condition via direct fine-tuning or temporally bidirectional sampling. However, the former results in a weak end-frame constraint, while the latter inevitably disrupts the input representation of video frames, leading to suboptimal performance. To improve the end-frame constraint while avoiding disruption of the input representation, we propose a novel video inbetweening framework specific to recent and more powerful transformer-based I2V-DMs, termed EF-VI. It efficiently strengthens the end-frame constraint by utilizing an enhanced injection. This is based on our proposed well-designed lightweight module, termed EF-Net, which encodes only the end frame and expands it into temporally adaptive frame-wise features injected into the I2V-DM. Extensive experiments demonstrate the superiority of our EF-VI compared with other baselines.

Figures

Figures reproduced from arXiv: 2505.21205 by the authors.

Figure 1
Figure 1. Comparison between current Image-to-Video Diffusion Models (I2V-DM)-based methods and our EF-VI. All the methods are based on the same I2V-DM, i.e., CogVideoX-5B-I2V [44]. According to the results, our method can produce more harmonious transitions in the challenging scenario. (a) Ground truth. (b) I2V results (conditioned on the start frame only) of the base model. (c) Video inbetweening results of the directly-fin… view at source ↗
Figure 2
Figure 2. LPIPS curves of current I2V-DM-based methods, our EF-VI, and the Ground Truth (GT). For each method and the GT, we calculate the average LPIPS [49] (a lower value indicating greater similarity) between the generated intermediate frames and the two boundary frames on 119 test pairs from the DAVIS dataset [27]. (a) Directly-Finetuning-Based Method and GT. (b) Temporally￾Bidirectional-Based Method and GT. (c) Our EF-VI… view at source ↗
Figure 3
Figure 3. Our proposed framework EF-VI and its core module EF-Net. (a) Our EF-VI efficiently enhances end-frame constraints by handling the start frame as before and applying an improved injection to the end frame. (b) This new injection mechanism is based on a lightweight module, termed EF-Net, which efficiently encodes the end frame, expands it into temporally adaptive frame-wise features, and injects them into the I2V-DM. … view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Visual comparison between our EF-VI and some state-of-the-art methods. Our EF-VI significantly improves the visual quality of the generated intermediate content across diverse scenarios. text prompts, for a fair comparison, we utilize a popular vision language model, Q…
Figure 5
Figure 5. Figure 5: Results of the user study. Each pie chart illustrates the proportion of videos output by each method that are selected by participants in a specific evaluation dimension. Although these metrics do not fully align with human in￾tuitive perception, they still prove the s…
Figure 6
Figure 6. Figure 6: Comparison of using different VAEs to reconstruct videos from the temporally-reversed latent representation. (a) Orig￾inal video. (b) Temporally-reversed video. (c) Video reconstructed using SVD VAE from the temporally-reversed latent representation. (d) Video reconstr…
Figure 7
Figure 7. Figure 7: Visual comparison between different types of text prompts. Utilizing VLM to predict or refine text prompts can enhance the performance of our EF-VI. Prompt Types LPIPS↓ FID↓ FVD↓ VBench↑ D-P 0.2266 27.26 567.54 0.8129 U-P 0.2168 23.63 477.87 0.8169 VLM-P 0.2096 22.30 3…
Figure 8
Figure 8. Figure 8: Some challenging examples of our EF-VI for video inbetweening. Due to the enhanced injection, our EF-VI can generate harmonious intermediate content in complex scenarios involving large and complicated motions of vehicles, people, animals, and cartoon characters, demon…
Figure 9
Figure 9. Figure 9: Visual results of our EF-VI for cartoon video inbetweening. The generated high-quality cartoon videos demonstrate a strong generalization ability of our method. ness of our method. E. Details of User Study For our user study, the full text of the instructions given to …
Figure 10
Figure 10. Figure 10: Visual comparison between our EF-VI and some advanced methods on cartoon video inbetweening. Our EF-VI achieves better visual quality when generalized to cartoon data. each group, the start and end frames are identical (predeter￾mined images), while the intermediate c…
Figure 11
Figure 11. Figure 11: Visual comparison of Wan2.1-FLF2V-14B [37], CogVX-FT [6], and our EF-VI. Wan2.1-FLF2V-14B improves the quality of generated videos at the cost of much more computation. Methods Video Size Step Time (s) Model Size Wan2.1-FLF2V-14B 81 × 1280 × 720 50 3829.85 14B CogVX-F…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 31 canonical work pages

  1. [1]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Jun- yang Lin. Qwen2.5-vl technical repor...

    arXiv 2025

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 , 2023. 2, 3, 5, 1

    arXiv 2023

  3. [3]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  4. [4]

    Ldmvfi: Video frame interpolation with latent diffusion models

    Duolikun Danier, Fan Zhang, and David Bull. Ldmvfi: Video frame interpolation with latent diffusion models. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 1472–1480, 2024. 2

    work page 2024

  5. [5]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7346–7356, 2023. 3

    2023

  6. [6]

    Cogvideox-interpolation

    Zhengcong Fei. Cogvideox-interpolation. https : / / github.com/feizc/CogvideX- Interpolation ,

    work page

  7. [7]

    Explo- rative inbetweening of time and space

    Haiwen Feng, Zheng Ding, Zhihao Xia, Simon Niklaus, Vic- toria Abrevaya, Michael J Black, and Xuaner Zhang. Explo- rative inbetweening of time and space. In European Con- ference on Computer Vision, pages 378–395. Springer, 2024. 3

    work page 2024

  8. [8]

    Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models.arXiv preprint arXiv:2503.10592, 2025

    Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models.arXiv preprint arXiv:2503.10592, 2025. 3

    arXiv 2025

Show all 56 references
  1. [9]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 5

    2017

  2. [10]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  3. [11]

    Real-time intermediate flow estimation for video frame interpolation

    Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. In European Conference on Com- puter Vision, pages 624–642. Springer, 2022. 2

    2022

  4. [12]

    Vbench: Comprehensive bench- mark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive bench- mark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2024

  5. [13]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  6. [14]

    Auto-encoding vari- ational {Bayes}

    Diederik P Kingma and Max Welling. Auto-encoding vari- ational {Bayes}. In Int. Conf. on Learning Representations,

  7. [15]

    Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 2

    2024

  8. [16]

    Amt: All-pairs multi-field transforms for efficient frame interpolation

    Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun- Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9801–9810, 2023. 2

    2023

  9. [17]

    Wf-vae: Enhancing video vae by wavelet-driven energy flow for latent video diffusion model

    Zongjian Li, Bin Lin, Yang Ye, Liuhan Chen, Xinhua Cheng, Shenghai Yuan, and Li Yuan. Wf-vae: Enhancing video vae by wavelet-driven energy flow for latent video diffusion model. arXiv preprint arXiv:2411.17459, 2024. 4

    2024 arXiv

  10. [18]

    Ctrl- adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl- adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model. InThe Thirteenth In- ternational Conference on Learning Representations , 2025. 2

    2025

  11. [19]

    Sparse global matching for video frame interpolation with large motion

    Chunxu Liu, Guozhen Zhang, Rui Zhao, and Limin Wang. Sparse global matching for video frame interpolation with large motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19125– 19134, 2024. 2

    2024

  12. [20]

    Re- conx: Reconstruct any scene from sparse views with video diffusion model

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Re- conx: Reconstruct any scene from sparse views with video diffusion model. arXiv preprint arXiv:2408.16767, 2024. 3

    2024 arXiv

  13. [21]

    Video frame interpolation via direct synthesis with the event- based reference

    Yuhan Liu, Yongjian Deng, Hao Chen, and Zhen Yang. Video frame interpolation via direct synthesis with the event- based reference. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 8477–8487, 2024. 2

    2024

  14. [22]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 1

    2024 arXiv

  15. [23]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, pages 4296–4304, 2024. 5

    2024

  16. [24]

    Softmax splatting for video frame interpolation

    Simon Niklaus and Feng Liu. Softmax splatting for video frame interpolation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 5437–5446, 2020. 2

    2020

  17. [25]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 4195–4205,

  18. [26]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 2, 8

    2024 arXiv

  19. [27]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 3, 4, 5

    2017 arXiv

  20. [28]

    Film: Frame inter- polation for large motion

    Fitsum Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless. Film: Frame inter- polation for large motion. In European Conference on Com- puter Vision, pages 250–266. Springer, 2022. 2, 5

    2022

  21. [29]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  22. [30]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Confer- ence on Learning Representations, 2022. 3

    2022

  23. [31]

    Bridging the gap: Sketch-aware interpola- tion network for high-quality animation sketch inbetween- ing

    Jiaming Shen, Kun Hu, Wei Bao, Chang Wen Chen, and Zhiyong Wang. Bridging the gap: Sketch-aware interpola- tion network for high-quality animation sketch inbetween- ing. In Proceedings of the 32nd ACM International Confer- ence on Multimedia, pages 10287–10295, 2024. 8

    2024

  24. [32]

    Deep geometrized cartoon line inbe- tweening

    Li Siyao, Tianpei Gu, Weiye Xiao, Henghui Ding, Ziwei Liu, and Chen Change Loy. Deep geometrized cartoon line inbe- tweening. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7291–7300, 2023. 8

    2023

  25. [33]

    Motionbridge: Dynamic video inbetweening with flexible controls

    Maham Tanveer, Yang Zhou, Simon Niklaus, Ali Mahdavi Amiri, Hao Zhang, Krishna Kumar Singh, and Nanxuan Zhao. Motionbridge: Dynamic video inbetweening with flexible controls. arXiv preprint arXiv:2412.13190 , 2024. 2, 3

    2024 arXiv

  26. [34]

    Fvd: A new metric for video generation

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Rapha¨el Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. ICLR Workshop,

  27. [35]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 3, 1

    2017

  28. [36]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 2

    2017

  29. [37]

    Wan: Open and advanced large-scale video gen- erative models

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video gen- erative models. arXiv preprint arXiv:2503.20314, 2025. 2, 8, 1, 6

    2025 arXiv

  30. [38]

    Framer: Interactive frame interpolation

    Wen Wang, Qiuyu Wang, Kecheng Zheng, Hao Ouyang, Zhekai Chen, Biao Gong, Hao Chen, Yujun Shen, and Chun- hua Shen. Framer: Interactive frame interpolation. arXiv preprint arXiv:2410.18978, 2024. 2, 3

    2024 arXiv

  31. [39]

    Generative inbetweening: Adapting image-to-video models for keyframe interpolation

    Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Kemelmacher-Shlizerman, Aleksander Holynski, and Steven M Seitz. Generative inbetweening: Adapting image-to-video models for keyframe interpolation. In The Thirteenth International Conference on Learning Representations, 2025. 2, 3, 5, 1

    2025

  32. [40]

    Perception-oriented video frame interpolation via asymmetric blending

    Guangyang Wu, Xin Tao, Changlin Li, Wenyi Wang, Xiao- hong Liu, and Qingqing Zheng. Perception-oriented video frame interpolation via asymmetric blending. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2753–2762, 2024. 2

    2024

  33. [41]

    Tooncrafter: Generative cartoon interpolation

    Jinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xin- tao Wang, Ying Shan, and Tien-Tsin Wong. Tooncrafter: Generative cartoon interpolation. ACM Transactions on Graphics (TOG), 43(6):1–11, 2024. 2

    2024

  34. [42]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In Eu- ropean Conference on Computer Vision , pages 399–417. Springer, 2024. 2

    2024

  35. [43]

    Vibidsam- pler: Enhancing video interpolation using bidirectional dif- fusion sampler

    Serin Yang, Taesung Kwon, and Jong Chul Ye. Vibidsam- pler: Enhancing video interpolation using bidirectional dif- fusion sampler. In The Thirteenth International Conference on Learning Representations, 2025. 2, 5, 1

    2025

  36. [44]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. In ICLR, 2025. 1, 2, 3, 5

    2025

  37. [45]

    Language model beats diffusion-tokenizer is key to visual generation

    Lijun Yu, Jos ´e Lezama, Nitesh B Gundavarapu, Luca Ver- sari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G Hauptmann, et al. Language model beats diffusion-tokenizer is key to visual generation. In 12th International Conference on Learning Represe...

    2024

  38. [46]

    Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation

    Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. Extracting motion and ap- pearance via inter-frame attention for efficient video frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5...

    2023

  39. [47]

    Motion-aware generative frame in- terpolation

    Guozhen Zhang, Yuhan Zhu, Yutao Cui, Xiaotong Zhao, Kai Ma, and Limin Wang. Motion-aware generative frame in- terpolation. arXiv preprint arXiv:2501.03699 , 2025. 3, 5, 1

    2025 arXiv

  40. [48]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 5, 8

    2023

  41. [49]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 3, 4, 5

    2018

  42. [50]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077, 2023. 2

    2023 arXiv

  43. [51]

    Easycontrol: Adding efficient and flexible control for diffusion transformer

    Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer. arXiv preprint arXiv:2503.07027, 2025. 3

    2025 arXiv

  44. [52]

    Tora: Trajectory-oriented diffusion transformer for video genera- tion

    Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video genera- tion. arXiv preprint arXiv:2407.21705, 2024. 3

    2024 arXiv

  45. [53]

    Eden: Enhanced diffusion for high-quality large-motion video frame interpo- lation

    Zihao Zhang, Haoran Chen, Haoyu Zhao, Guansong Lu, Yanwei Fu, Hang Xu, and Zuxuan Wu. Eden: Enhanced diffusion for high-quality large-motion video frame interpo- lation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2025. 2

    2025

  46. [54]

    Clearer frames, anytime: Re- solving velocity ambiguity in video frame interpolation

    Zhihang Zhong, Gurunandan Krishnan, Xiao Sun, Yu Qiao, Sizhuo Ma, and Jian Wang. Clearer frames, anytime: Re- solving velocity ambiguity in video frame interpolation. In European Conference on Computer Vision, pages 346–363. Springer, 2024. 8

    2024

  47. [55]

    Generative inbetweening through frame- wise conditions-driven video generation

    Tianyi Zhu, Dongwei Ren, Qilong Wang, Xiaohe Wu, and Wangmeng Zuo. Generative inbetweening through frame- wise conditions-driven video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 3, 5, 1

    2025

  48. [56]

    This is the first and last frame of a video clip. Describe this video in one continuous para- graph

    Tianyi Zhu, Wei Shang, and Dongwei Ren. Thin-plate spline-based interpolation for animation line inbetweening. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 10995–11003, 2025. 2 EF-VI: Enhancing End-Frame Injection for Video Inbetweening Supplementa...

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.