Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Semantic Frame Interpolation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Semantic Frame Interpolation proposes a single text-guided model that generates any number of intermediate frames between two given images, backed by a 300K-clip dataset and a benchmark.

desk verdict The dataset and benchmark are worth a look, but the model's endpoint fidelity contradicts its central claim of precise control. read the letter →

arxiv 2507.05173 v1 pith:FYF23376 submitted 2025-07-07 cs.CV

Yijia Hong , Jiangning Zhang , Ran Yi , Yuji Wang , Weijian Cao , Xiaobin Hu , Zhucun Xue , Yabiao Wang
show 2 more authors
Chengjie Wang Lizhuang Ma
This is my paper · ORCID
classification cs.CV
keywords semanticframeinterpolationvideodiffusionmodelsMixture-of-LoRAtext-guidedgenerationframe-to-frameSFI-300KSFIBench
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Semantic Frame Interpolation (SFI) as a new task: given a first frame, a last frame, a text prompt, and a requested number N of intermediate frames, a model should generate N frames that are visually continuous and semantically aligned with the prompt. To solve it, the paper introduces SemFi, a model built on a 14-billion-parameter image-to-video diffusion model, with a Mixture-of-LoRA module that activates one of six frame-count-specialized adapters plus a shared universal adapter so the same model can handle both 5-frame interpolations and 81-frame transitions. It also contributes SFI-300K, a 300,000-clip dataset with captions and endpoint pairs filtered for meaningful differences, and SFIBench, a benchmark measuring video fidelity, frame fidelity, semantic fidelity, and video quality. The reported experiments claim SemFi outperforms the image-to-video backbone and existing interpolation baselines, especially at low frame counts, and is more stable across all tested frame scales.

What carries the argument

Mixture-of-LoRA (MoL), defined as a set of low-rank adapters with one universal LoRA that is always active and six expert LoRAs, one per target frame count in S={5,9,17,33,65,81}; at inference the model activates the expert minimizing |N−s|. The second mechanism is dual-endpoint conditioning: the initial and final frames occupy the first and last slots of the guidance tensor, the binary mask marks only those two slots as preserved, and the two CLIP image embeddings are summed before cross-attention. Together they let a frozen 14B image-to-video diffusion model adapt its temporal behavior to the requested length while keeping text-conditioning intact.

What would settle it

Run SemFi on any SFIBench sample with known first and last frames and compute pixel-level PSNR/SSIM between the first generated frame and the given first frame, and between the last generated frame and the given last frame; if those scores are not near-perfect, as the paper's Table 2 already suggests, the claim that the model preserves its input endpoints fails.

Watch

Extended reading notes

Core claim

SemFi's central claim is that a single model can perform semantic frame interpolation at arbitrary frame counts with text control, by treating it as a conditional generation problem p(I1:N | If, Il, T, N) that generalizes both short-range video frame interpolation and long-range frame-to-frame generation. The model conditions on both endpoints: the first and last frames are placed at the two ends of the guidance tensor with a binary mask marking them as preserved, and their CLIP embeddings are summed for cross-attention. Frame-count adaptivity comes from a Mixture-of-LoRA module with one universal adapter and six expert adapters for frame counts 5, 9, 17, 33, 65, and 81; for an unseen count, the nearest expert by absolute difference is selected. The paper supports the claim with SFI-300K and SFIBench, reporting that SemFi improves overall LPIPS/FID and video-quality metrics over the Wan backbone, avoids the backbone's collapse at 5 frames, and shows lower variance across frame counts. The authors explicitly note that the 81-frame training cap and the real-world composition of the dataset bound the current scope.

Load-bearing premise

The load-bearing premise is that the mask conditioning actually makes the generated sequence begin with the given first frame and end with the given last frame, and the paper's own frame-fidelity numbers (PSNR 21.96 dB, SSIM 0.66) indicate the generated endpoints are far from those inputs.

Editorial extensions

If this is right

  • A single SemFi model covers the full range from 5-frame interpolation to 81-frame generation, replacing separate short-range and long-range models.
  • Text prompts can change the semantic content of the transition even when the first and last frames are fixed, enabling prompt-controlled scene transitions and effects blending.
  • SFI-300K and SFIBench give the community a shared dataset and standardized metrics for comparing future SFI methods.
  • Because expert selection is based only on absolute distance in frame count, the method generalizes to arbitrary requested N without retraining.
  • The ablation results indicate that multi-frame training data and the MoL module are both needed to avoid scale-content mismatch, such as generating short-range content across a long sequence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's endpoint-adherence numbers (PSNR 21.96 dB, SSIM 0.66 on frame fidelity) are lower than the Wan baseline's, which suggests that the mask conditioning does not fully force the generated video to start and end at the exact input frames; a reader should verify endpoint preservation before relying on the 'precise control' claim.
  • The nearest-expert rule maps every requested count to one of six discrete experts; a natural extension is to blend or interpolate between neighboring experts for counts between 5 and 81, which the paper does not test.
  • The dataset's CLIP and flow filtering likely biases SFI-300K toward moderate motion and real-world content, so text-controlled interpolation on synthetic or highly dynamic footage is an open question the paper acknowledges.
  • A practical extension of SFI, unmentioned by the authors, is endpoint-plus-prompt storyboarding: using the same model to generate alternative narrative transitions between the same two keyframes, which the qualitative results hint at but do not quantify.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper introduces Semantic Frame Interpolation (SFI), a task in which a model is given first and last frames, a text prompt, and a requested number of intermediate frames, and must generate the intermediate sequence. The authors propose SemFi, built on Wan2.1, which uses a Mixture-of-LoRA module with one universal and six frame-count-specific experts to adapt generation across different clip lengths. They also construct SFI-300K, a dataset of 300,000 video clips with captions, and SFIBench, an evaluation protocol measuring video fidelity, frame fidelity, semantic fidelity, and video quality. The paper claims that SemFi achieves better performance on arbitrary frame counts while maintaining precise control, and that it is the first unified model for both short- and long-range generative in-betweening with text control.

Significance. The paper's resources are potentially valuable: SFI-300K is a large-scale dataset for a newly proposed task, and SFIBench provides a multi-dimensional evaluation protocol that goes beyond simple fidelity. The Mixture-of-LoRA module is a sensible approach to handling multiple frame scales within a single model, and the paper reports careful quantitative comparisons against strong baselines including Wan, GI, and FCVG. However, the central claim of arbitrary frame counts with precise control is not supported by the evidence: the model fails to preserve the conditioned endpoints (Frame Fidelity PSNR 21.96 dB vs. 26.87 dB for Wan, SSIM 0.6576 vs. 0.7878, Table 2), the evaluation covers only the six trained scales in S, and the N notation is inconsistent between the formulation and the architecture.

major comments (4)
  1. [3.3.2, Table 2] The Frame Fidelity results directly contradict the task definition in Eq. (1) and the endpoint-conditioning mechanism described in Section 3.2. SemFi achieves PSNR 21.96 dB and SSIM 0.6576 against the conditioned first/last frames, whereas the Wan backbone achieves 26.87 dB and 0.7878. Since SFI requires the generated sequence to begin with the given If and end with the given Il, the reported fidelity means that SemFi does not produce an interpolation of the input endpoints and the claim of 'precise control' is quantitatively falsified. Table A1 provides no per-scale Frame Fidelity breakdown, so it remains unknown whether endpoint failure is systematic or scale-specific.
  2. [3.1, 3.2] The definition of N is inconsistent. Section 3.1 defines N as the number of intermediate frames, so the output is {I1,...,IN} and the total clip length would be N+2. Section 3.2 states that the last frame is placed as the last element of the temporal sequence and intermediate frames occupy positions 1 to N-2, implying that N is the total number of frames including the two endpoints. The same symbol N is used in Eq. (2) for expert selection and in Eq. (3) for dataset clips. This ambiguity affects how the outputs are interpreted in Table A1 (e.g., '5 frames' could mean 5 intermediate frames or 5 total frames) and how Frame Fidelity is computed.
  3. [3.2, 3.3.2, Table A1] The claim of supporting 'arbitrary frame counts' is not evidenced. The expert LoRA scales S = {5, 9, 17, 33, 65, 81} are the same scales used for dataset construction (Eq. 3) and for the benchmark test set (Section 3.3.2), and Table A1 evaluates only these six scales. No experiment uses a frame count outside S, so the nearest-expert generalization rule in Eq. (2) is never tested. The paper therefore demonstrates performance on the training scales only, not on arbitrary N.
  4. [3.3, 4.2] The benchmark construction and evaluation protocol have several reproducibility gaps that affect the central comparison. (i) The CLIP/flow filtering thresholds in Section 3.3.1 are given only qualitatively ('high and low thresholds') with no numeric values. (ii) Eq. (3) uses indices f/2 - s/2 and f/2 + s/2, which are non-integers whenever f and s are odd (all s in S are odd); the intended floor/ceil operation is not specified. (iii) The baselines GI and FCVG are VFI methods that do not natively accept text prompts, yet Semantic Fidelity is computed against text; the paper does not describe how prompts were supplied to these baselines. (iv) Since the test clips are cut from videos that also contribute training clips (only the test clips are excluded, Section 4.1), potential near-duplicate leakage between training and test is not discussed.
minor comments (4)
  1. [Table 2 caption] The phrase 'best, second-best, and third-best results' is not discernible in the plain-text version of the table; please ensure the highlighting is consistently applied and described in the caption.
  2. [3.3.1] The sentence 'we first filter out videos Vfiltered with a frame rate (FPS) less than or equal to 30' is ambiguous; it should state explicitly whether such videos are removed or retained, and the subsequent condition on total frame count should be stated with clear inequalities.
  3. [3.3.1, Table 1] The dataset is described as containing 300,000 video clips, but Table 1 lists 'TotalVideos' as 300,000; please reconcile the vocabulary (videos vs. clips) throughout.
  4. [Section 4.2] The claim that 'SemFi demonstrates overall superiority over its backbone model Wan as well as GI' is not fully supported by Table 2, where FCVG still achieves better Video Fidelity (LPIPS 0.2122, FID 61.00) and better Frame Fidelity (PSNR 24.12, SSIM 0.6942) than SemFi.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation found: SemFi is an empirical system paper whose main claims rely on external benchmarks, with a mild self-referential design choice (shared frame-count set S) that limits the generalization claim rather than making it circular.

full rationale

The paper's derivation chain is not circular. The SFI task in Sec. 3.1 is a conditional-generation formulation (Eq. 1), and SemFi's Mixture-of-LoRA and dual-frame conditioning are architectural proposals evaluated against external metrics (PSNR, SSIM, LPIPS, VBench, ViCLIP); no fitted parameter is renamed as a prediction, and no result is forced by the paper's own definitions. The only self-referential element is that the same hand-chosen frame-count set S = {5, 9, 17, 33, 65, 81} defines the expert LoRA scales (Sec. 3.2), the dataset clip segmentation (Eq. 3), and the benchmark test scales (Sec. 3.3.2). This means the claims of 'arbitrary frame counts' and 'graceful generalization to unseen interpolation requirements' are untested for N outside S, which is a generalization and validity caveat, not circular reasoning. Similarly, Table 2's Frame Fidelity results (SemFi 21.96 dB PSNR and 0.6576 SSIM versus Wan's 26.87 dB and 0.7878) contradict the 'precise control' emphasis, but this is an external measurement that falsifies a premise; it is not a construction that makes the claim true by definition. The Limitations section acknowledges the 81-frame training cap but does not address the endpoint-fidelity gap; that omission is a completeness concern, not circularity. Because the core evaluation is self-contained and benchmarked independently, the appropriate circularity score is low.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claims rest primarily on domain assumptions about the Wan2.1 backbone, the mask-based endpoint conditioning, and the generalization of the nearest-expert router. The model is an empirical system, so there are no mathematical axioms; the fragile premises are engineering assumptions, one of which is contradicted by the reported frame fidelity.

free parameters (3)
  • Expert frame-count set S = {5, 9, 17, 33, 65, 81}
    Hand-chosen to cover short, medium, and long generation; this set defines both the dataset clip scales (Eq. 3) and the LoRA expert scales (Eq. 2), so it is the central design choice the model and evaluation depend on.
  • Mixture-of-LoRA composition = 1 universal + 6 experts, rank 16
    The number of experts and the LoRA rank are chosen without sensitivity analysis; they determine parameter count and routing behavior.
  • CLIP/flow filtering thresholds = not reported
    The dataset curation (Sec. 3.3.1) removes videos outside unspecified 'high and low thresholds' for CLIP score and flow score; these thresholds directly shape the training distribution and benchmark.
assumptions (5)
  • domain assumption Wan2.1 I2V is an effective prior for semantic video interpolation
    SemFi is built entirely by adapting Wan2.1's frozen weights (Sec. 3.2); any failure of the base model's temporal prior is inherited.
  • domain assumption Setting both endpoint masks to 1 enforces that the generated video starts and ends with the input frames
    Sec. 3.2 assumes mask-based conditioning preserves endpoints, but Tab. 2's Frame Fidelity shows PSNR 21.96, contradicting this.
  • domain assumption The nearest-expert router (Eq. 2) generalizes to frame counts outside S
    For off-grid N the model picks the closest expert, but no experiment tests any N outside the six training scales.
  • domain assumption Qwen2.5-VL-32B captions are accurate and high-density
    SFI-300K captions are generated by this model with a 'carefully designed prompt' not included; caption quality is assumed.
  • domain assumption Centered clipping around the video center produces valid interpolation targets
    Eq. 3 cuts clips symmetrically around the middle; this assumes the central segment is representative of a good interpolation between its endpoints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semantic Frame Interpolation." pith.science (2026). https://pith.science/paper/FYF23376

@misc{pith2026250705173,
  author       = {Pith},
  title        = {Pith review of: Semantic Frame Interpolation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FYF23376}},
  note         = {Machine review of arXiv:2507.05173}
}
read the original abstract

Generating intermediate video content of varying lengths based on given first and last frames, along with text prompt information, offers significant research and application potential. However, traditional frame interpolation tasks primarily focus on scenarios with a small number of frames, no text control, and minimal differences between the first and last frames. Recent community developers have utilized large video models represented by Wan to endow frame-to-frame capabilities. However, these models can only generate a fixed number of frames and often fail to produce satisfactory results for certain frame lengths, while this setting lacks a clear official definition and a well-established benchmark. In this paper, we first propose a new practical Semantic Frame Interpolation (SFI) task from the perspective of academic definition, which covers the above two settings and supports inference at multiple frame rates. To achieve this goal, we propose a novel SemFi model building upon Wan2.1, which incorporates a Mixture-of-LoRA module to ensure the generation of high-consistency content that aligns with control conditions across various frame length limitations. Furthermore, we propose SFI-300K, the first general-purpose dataset and benchmark specifically designed for SFI. To support this, we collect and process data from the perspective of SFI, carefully designing evaluation metrics and methods to assess the model's performance across multiple dimensions, encompassing image and video, and various aspects, including consistency and diversity. Through extensive experiments on SFI-300K, we demonstrate that our method is particularly well-suited to meet the requirements of the SFI task.

Figures

Figures reproduced from arXiv: 2507.05173 by the authors.

Figure 1
Figure 1. Left: (a) Conventional video frame interpolation approaches (illustrated here with an SVD [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. SemFi model architecture. SemFi employs a Mixture-of-LoRA module to dynamically activate the most suitable LoRA parameters for the target frame count, enabling high-quality semantic frame interpolation across diverse frame generation requirements. is reformulated by setting the initial frame If and the last frame Il as the first and last elements of the temporal sequence respectively, with zero-padded intermediate f… view at source ↗
Figure 3
Figure 3. Dataset Analysis. (a) SFI-300K demonstrates substantial diversity across video categories; [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison. Quantitative Comparison. The quantitative evaluation results on SFIBench are presented in Tab. 2. SemFi demonstrates overall superiority over its backbone model Wan as well as GI. Its advantages in semantic fidelity and video quality further hig…
Figure 5
Figure 5. Figure 5: Ablation on training data. Generated samples for 5-frame (top) and 9-frame (bottom) cases. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Ablation on MoL. Both examples show 81-frame generation results. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation

    cs.CV 2026-04 conditional novelty 5.0 of 10

    Adding bidirectional cycle-consistent training with learnable direction tokens improves long-video interpolation quality without added inference cost.

Reference graph

Works this paper leans on

51 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    aesthetic - predictor

    LAION AI. aesthetic - predictor. https://github.com/LAION-AI/ aesthetic-predictor, 2022. 7

    work page 2022

  2. [2]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. a...

    arXiv 2025

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  4. [4]

    Kernel-based frame interpolation for spatio-temporally adaptive rendering

    Karlis Martins Briedis, Abdelaziz Djelouah, Raphaël Ortiz, Mark Meyer, Markus Gross, and Christopher Schroers. Kernel-based frame interpolation for spatio-temporally adaptive rendering. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–11, 2023. 2, 3

    work page 2023

  5. [5]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators. 2024. URL https://openai. com/research/video-generation-models-as-world-simulators, 3:1, 2024. 3

    work page 2024

  6. [6]

    Multiple video frame interpolation via enhanced deformable separable convolution

    Xianhang Cheng and Zhenzhong Chen. Multiple video frame interpolation via enhanced deformable separable convolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):7029–7045, 2021. 2, 3

    work page 2021

  7. [7]

    Channel attention is all you need for video frame interpolation

    Myungsub Choi, Heewon Kim, Bohyung Han, Ning Xu, and Kyoung Mu Lee. Channel attention is all you need for video frame interpolation. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 10663–10671, 2020. 3

    work page 2020

  8. [8]

    Ldmvfi: Video frame interpolation with latent diffusion models

    Duolikun Danier, Fan Zhang, and David Bull. Ldmvfi: Video frame interpolation with latent diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1472–1480, 2024. 3

    work page 2024

Show all 51 references
  1. [9]

    Video frame interpo- lation with stereo event and intensity cameras

    Chao Ding, Mingyuan Lin, Haijian Zhang, Jianzhuang Liu, and Lei Yu. Video frame interpo- lation with stereo event and intensity cameras. IEEE Transactions on Multimedia, 2024. 2, 3

    2024

  2. [10]

    Video frame interpolation: A comprehensive survey

    Jiong Dong, Kaoru Ota, and Mianxiong Dong. Video frame interpolation: A comprehensive survey. ACM Transactions on Multimedia Computing, Communications and Applications , 19(2s):1–31, 2023. 3, 4

    2023

  3. [11]

    Explorative inbetweening of time and space

    Haiwen Feng, Zheng Ding, Zhihao Xia, Simon Niklaus, Victoria Abrevaya, Michael J Black, and Xuaner Zhang. Explorative inbetweening of time and space. In European Conference on Computer Vision, pages 378–395. Springer, 2024. 3

    2024

  4. [12]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3

    2023 arXiv

  5. [13]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 3

    2022

  6. [14]

    Motion-aware latent diffusion models for video frame interpolation

    Zhilin Huang, Yijie Yu, Ling Yang, Chujun Qin, Bing Zheng, Xiawu Zheng, Zikun Zhou, Yaowei Wang, and Wenming Yang. Motion-aware latent diffusion models for video frame interpolation. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 1043–1052, 2024. 3

    2024

  7. [15]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  8. [16]

    High-resolution frame interpolation with patch-based cascaded diffusion

    Junhwa Hur, Charles Herrmann, Saurabh Saxena, Janne Kontkanen, Wei-Sheng Lai, Yichang Shih, Michael Rubinstein, David J Fleet, and Deqing Sun. High-resolution frame interpolation with patch-based cascaded diffusion. In Proceedings of the AAAI Conference on Artificial Intellige...

    2025

  9. [17]

    Video interpolation with diffusion models

    Siddhant Jain, Daniel Watson, Eric Tabellion, Ben Poole, Janne Kontkanen, et al. Video interpolation with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7341–7351, 2024. 3

    2024

  10. [18]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148–5157, 2021. 7

    2021

  11. [19]

    Benchmarking video frame interpolation

    Simon Kiefhaber, Simon Niklaus, Feng Liu, and Simone Schaub-Meyer. Benchmarking video frame interpolation. arXiv preprint arXiv:2403.17128, 2024. 3, 4

    2024 arXiv

  12. [20]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 3

    2024 arXiv

  13. [21]

    Amt: All-pairs multi-field transforms for efficient frame interpolation

    Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun-Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9801–9810, 2023. 7

    2023

  14. [22]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131, 2024. 6

    2024 arXiv

  15. [23]

    Video frame interpolation with transformer

    Liying Lu, Ruizheng Wu, Huaijia Lin, Jiangbo Lu, and Jiaya Jia. Video frame interpolation with transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3532–3542, 2022. 3

    2022

  16. [24]

    Kling AI. (n.d.). Kling ai: Next-generative ai creative studio, 2024. https://www.klingai. com/global/, Last accessed on 2025-05-07. 2, 3, 4

    2024

  17. [25]

    Softmax splatting for video frame interpolation

    Simon Niklaus and Feng Liu. Softmax splatting for video frame interpolation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5437–5446,

  18. [26]

    Video frame interpolation via adaptive convolution

    Simon Niklaus, Long Mai, and Feng Liu. Video frame interpolation via adaptive convolution. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 670–679, 2017. 3

    2017

  19. [27]

    Video frame interpolation via adaptive separable convolution

    Simon Niklaus, Long Mai, and Feng Liu. Video frame interpolation via adaptive separable convolution. In Proceedings of the IEEE international conference on computer vision, pages 261–270, 2017. 3

    2017

  20. [28]

    Bmbc: Bilateral motion estimation with bilateral cost volume for video interpolation

    Junheum Park, Keunsoo Ko, Chul Lee, and Chang-Su Kim. Bmbc: Bilateral motion estimation with bilateral cost volume for video interpolation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16, pages 109–125. Springe...

    2020

  21. [29]

    Video frame interpolation using real-time intermediate flow estimation

    B Prasanna, S Niranjan, et al. Video frame interpolation using real-time intermediate flow estimation. In 2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS), volume 1, pages 1–5. IEEE, 2024. 2, 3

    2024

  22. [30]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  23. [31]

    Film: Frame interpolation for large motion

    Fitsum Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless. Film: Frame interpolation for large motion. In European Conference on Computer Vision (ECCV), 2022. 2, 7 11

    2022

  24. [32]

    Bim-vfi: directional motion field-guided frame interpolation for video with non-uniform motions

    Wonyong Seo, Jihyong Oh, and Munchurl Kim. Bim-vfi: directional motion field-guided frame interpolation for video with non-uniform motions. arXiv preprint arXiv:2412.11365, 2024. 3

    2024 arXiv

  25. [33]

    Video frame interpolation transformer

    Zhihao Shi, Xiangyu Xu, Xiaohong Liu, Jun Chen, and Ming-Hsuan Yang. Video frame interpolation transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17482–17491, 2022. 3

    2022

  26. [34]

    Forward and backward warping for optical flow-based frame interpolation

    Joi Shimizu, Heming Sun, and Jiro Katto. Forward and backward warping for optical flow-based frame interpolation. In 2022 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pages 082–086. IEEE, 2022. 3

    2022

  27. [35]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 6

    2012 arXiv

  28. [36]

    Adapool: Exponential adaptive pooling for information- retaining downsampling

    Alexandros Stergiou and Ronald Poppe. Adapool: Exponential adaptive pooling for information- retaining downsampling. IEEE Transactions on Image Processing, 32:251–266, 2022. 6

    2022

  29. [37]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020. 6, 7

    2020

  30. [38]

    Video frame interpolation based on deformable kernel region

    Haoyue Tian, Pan Gao, and Xiaojiang Peng. Video frame interpolation based on deformable kernel region. In IJCAI, pages 1349–1355, 2022. 3

    2022

  31. [39]

    Wan: Open and advanced large-scale video generative models

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 2, 3, 4, 7, 14

    2025 arXiv

  32. [40]

    Koala-36m: A large-scale video dataset improving con- sistency between fine-grained conditions and video content

    Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, et al. Koala-36m: A large-scale video dataset improving con- sistency between fine-grained conditions and video content. arXiv preprint arXiv:2410.08260,

    arXiv

  33. [41]

    Generative inbetweening: Adapting image-to-video models for keyframe interpolation

    Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Kemelmacher-Shlizerman, Aleksander Holyn- ski, and Steven M Seitz. Generative inbetweening: Adapting image-to-video models for keyframe interpolation. arXiv preprint arXiv:2408.15239, 2024. 2, 6, 7

    2024 arXiv

  34. [42]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinyuan Chen, Yaohui Wang, Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, and Yu Qiao. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023. 7

    2023 arXiv

  35. [43]

    Internvideo: General video foundation models via generative and discriminative learning

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, Sen Xing, Guo Chen, Junting Pan, Jiashuo Yu, Yali Wang, Limin Wang, and Yu Qiao. Internvideo: General video foundation models via generative and discriminative lear...

    2022 arXiv

  36. [44]

    Zooming slow-mo: Fast and accurate one-stage space-time video super-resolution

    Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P Allebach, and Chenliang Xu. Zooming slow-mo: Fast and accurate one-stage space-time video super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3370–3379, 2020. 3

    2020

  37. [45]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In European Conference on Computer Vision , pages 399–417. Springer, 2024. 3

    2024

  38. [46]

    Video enhancement with task-oriented flow

    Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow. International Journal of Computer Vision, 127:1106–1125, 2019. 6

    2019

  39. [47]

    Diffusion models: A comprehensive survey of methods and applications

    Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys, 56(4):1–39, 2023. 3 12

    2023

  40. [48]

    Vibidsampler: Enhancing video interpolation using bidirectional diffusion sampler

    Serin Yang, Taesung Kwon, and Jong Chul Ye. Vibidsampler: Enhancing video interpolation using bidirectional diffusion sampler. arXiv preprint arXiv:2410.05651, 2024. 2, 3

    2024 arXiv

  41. [49]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 3

    2024 arXiv

  42. [50]

    Extracting motion and appearance via inter-frame attention for efficient video frame interpola- tion

    Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. Extracting motion and appearance via inter-frame attention for efficient video frame interpola- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 56...

    2023

  43. [51]

    TF", “MS

    Tianyi Zhu, Dongwei Ren, Qilong Wang, Xiaohe Wu, and Wangmeng Zuo. Genera- tive inbetweening through frame-wise conditions-driven video generation. arXiv preprint arXiv:2412.11755, 2024. 2, 3, 7 13 A Additional Experiments First Frame Text prompt: A girl slowly picks up a bouq...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.