Pith. sign in

REVIEW 3 major objections 5 minor 68 references

EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EventDiff shows that video frame interpolation benefits when a diffusion model denoises in latent space rather than estimating optical flow.

desk verdict A genuinely new latent diffusion framework for event-based VFI with strong ablations; headline gains are mostly credible, but author-finetuned baselines need scrutiny and the closest event-diffusion prior is missing from the tables. read the letter →

arxiv 2505.08235 v1 pith:WZE7XCN2 submitted 2025-05-13 cs.CV

Hanle Zheng , Xujie Han , Zegang Peng , Shangbin Zhang , Guangxun Du , Zhuo Zou , Xilin Wang , Jibin Wu
show 2 more authors
Hao Guo Lei Deng
This is my paper · ORCID
classification cs.CV
keywords event-basedvideoframeinterpolationlatentdiffusionmodeleventcameraspatio-temporalcrossattentionhybridautoencoderevent-enhancedvisualgenerationmotiondeblurring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EventDiff claims that the best way to use event-camera data for video frame interpolation is to skip explicit motion estimation altogether: an autoencoder fuses the two boundary frames with their event streams into a compact latent representation, and a diffusion process denoises the interpolated frame's code directly in that space. The paper argues this removes the handcrafted optical-flow and warping bottlenecks that limit previous event-based methods, while event data supplies the temporal detail that frame-only diffusion models lack. On Vimeo90K-Triplet, GoPro, and SNU-FILM, and on the real-world HQ-EVFI, BS-ERGB, and HQF datasets, it reports state-of-the-art PSNR and SSIM, with up to 1.98 dB over the best event-based method and 5.72 dB over the best diffusion-based method, at 4.24x faster inference thanks to a 5-step denoising schedule.

What carries the argument

The load-bearing mechanism is the Event-Frame Hybrid AutoEncoder (HAE), an autoencoder that compresses ground-truth frames into a small latent code ($H/2^n \times W/2^n$) and builds a multi-scale hybrid feature pyramid from the boundary frame-event pairs; its decoder reconstructs the interpolated frame from the latent code plus that pyramid. Inside each downsampling block, the Spatial-Temporal Cross Attention (STCA) module fuses the two modalities by factorizing full spatio-temporal cross-attention into a spatial cross-attention (SCA) and a temporal cross-attention (TCA), reducing cost from $O(T\hat{H}\hat{W}\cdot\hat{H}\hat{W})$ to $O(((T/p^2+\hat{H}\hat{W})\cdot\hat{H}\hat{W})$. Stage 1 pretrains the HAE with an $L_1$ reconstruction loss plus a vector-quantization regularization; Stage 2 trains the diffusion U-Net to minimize the $L_1$ distance between the final denoised embedding and the ground-truth embedding, jointly fine-tuning the HAE decoder so that the whole pipeline works with only $T=5$ diffusion steps.

What would settle it

Rerun the comparisons with the released pretrained weights of TimeLens, CBMNet-L, and TLXNet+, or with their original training protocols, on the same simulated and real event streams; if the PSNR or SSIM margin over EventDiff drops below the reported 1.98 dB or reverses on any dataset, the central state-of-the-art claim is falsified. A second check is to supply optical flow explicitly as an additional condition to EventDiff: if that version matches or beats the direct-denoising version, the paper's qualitative claim that direct denoising avoids flow errors is undermined.

Watch

Extended reading notes

Core claim

The central claim is that a single end-to-end latent diffusion framework, EventDiff, can outperform dedicated flow-estimation pipelines on event-based frame interpolation while being substantially faster. The key evidence is that the Event-Frame Hybrid AutoEncoder produces a hybrid feature pyramid plus a ground-truth latent code, and the diffusion U-Net reconstructs that code from the boundary condition embeddings; because the decoder is jointly fine-tuned in the second training stage, the model tolerates residual diffusion error and needs only five denoising steps. The paper reports that this design reaches 41.15 dB PSNR on Vimeo90K-Triplet in its f3 variant and 38.14 dB in f4, surpassing the previous event-based best by 1.98 dB, and that it generalizes across difficulty levels on SNU-FILM and to event-based motion deblurring on GoPro with competitive results.

Load-bearing premise

The reported advantage over prior event-based methods depends on the authors' own finetuning of those models on their simulated event data, so any mismatch between that finetuning protocol or event distribution and the original training could shrink the measured gap.

Editorial extensions

If this is right

  • If the reported results hold, diffusion-based VFI can be both more accurate and faster than flow-warping event methods, reversing the usual trade-off where generative models win only on perceptual quality.
  • The 5-step denoising schedule, about 79 ms per 448x256 frame on an RTX 4090, makes latent-diffusion interpolation practical for near-real-time use.
  • Threading event information through a hybrid autoencoder gives gains across difficulty levels from easy to extreme motion, suggesting that a unified latent code can carry both detail and large-displacement information.
  • The same HAE-plus-diffusion recipe transfers to event-based motion deblurring with only task-specific input and output changes, supporting the paper's claim of a general event-enhanced generation framework.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a natural stress test is to freeze the pretrained HAE from Stage 1 and train only the diffusion stage on unseen event-camera domains, since the paper's own ablations show Stage-2 joint optimization is what absorbs residual latent error.
  • Editorial inference: the f3/f4 trade-off suggests that an adaptive or per-pixel choice of latent resolution could beat either fixed setting, a possibility the paper does not explore.
  • Editorial inference: because performance saturates at four to five denoising steps, distilling the diffusion process to one step or learning a deterministic mapping from condition embeddings to the target code is the obvious next efficiency experiment, though the paper only reports five steps.
  • Editorial inference: the comparison against baselines depends on simulated events, so a transfer test using real event streams from a different sensor would clarify whether the margin holds under the sim-to-real gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EventDiff, a latent-diffusion framework for event-based video frame interpolation. An Event-Frame Hybrid AutoEncoder (HAE) with a Spatial-Temporal Cross Attention (STCA) module encodes the two boundary frame-event pairs into hybrid pyramid features and projects the ground-truth frame into a compact embedding; a U-Net denoiser then reconstructs that embedding from the boundary conditions, and the HAE decoder synthesizes the interpolated frame. Training is two-stage: the HAE is first pretrained with ground-truth supervision, then jointly optimized with the diffusion model using a loss that combines the L1 error between the final denoised embedding and the ground-truth embedding and the L1 error between the reconstructed frame and the ground-truth frame. Experiments are reported on Vimeo90K-Triplet, GoPro, SNU-FILM, and three real-world event datasets, with claims of state-of-the-art PSNR/SSIM, a 1.98 dB gain over the best event-based method on Vimeo90K-Triplet, a 5.72 dB gain over diffusion-based methods, and 4.24x faster inference with only 5 denoising steps. The paper also reports an extension to event-based motion deblurring on GoPro.

Significance. If the reported results are reproducible, EventDiff would be a practically significant contribution: it combines event data with a latent diffusion process in a way that achieves high distortion metrics (PSNR/SSIM), not just perceptual quality, while reducing the inference cost to five steps. The paper's ablations are a genuine strength: Tables IV-VI and Fig. 8 provide informative analyses of embedding spatial size, fusion strategy, training schemes, and step count, and the method description is detailed enough to reimplement. The extensibility experiment to motion deblurring, while not SOTA, is a useful demonstration of generality. However, the central benchmark claims are only as strong as the comparison protocol; the author-finetuned baselines and underspecified event-simulation details are the main risks to the paper's headline numbers.

major comments (3)
  1. [§IV.A.3 and Tables I/III] The paper states that TimeLens, CBMNet-L, and TLXNet+ are finetuned by the authors on their own simulated data using the implementation of [14], and the Table I footnote repeats this. Because TLXNet+ is the strongest event-based competitor in several reported columns (GoPro 7-skips, SNU-FILM extreme, BS-ERGB), any systematic mismatch between the authors' finetuning pipeline and the original training protocol could lower those baseline numbers and inflate EventDiff's margins. This is load-bearing for the 'superior performance in SNU-FILM' claim and for the real-world SOTA claims in Table III. I note that the specific 1.98 dB Vimeo90K-Triplet gain is computed against DSEVFI, which is not in the finetuned set, so that particular number is less affected; nevertheless, the manuscript should, per table and per setting, state which entries use official weights, report the original published numbers alongside the finetuned ones, and provide the full finetuning protocol (event simulator parameters, temporal binning, epochs, learning rates, and the code used for finetuning).
  2. [§IV.A.1, §IV.A.3, and Eq. (3)] The event simulation and evaluation protocol is underspecified. Section IV.A.1 says only that 'synthetic event data simulated using ESIM [50]', and Eq. (3) uses T temporal bins, but the experimental settings never give T, the ESIM contrast thresholds, noise settings, or the random seeds. Event-based methods in Tables I and III are known to be sensitive to event representation. The paper should also clarify whether the non-finetuned entries (e.g., DSEVFI) are taken from the original publications or re-evaluated on the authors' event streams; if re-evaluated, the exact preprocessing must be given. Without these details, the comparisons are not reproducible and different test-event distributions could be mixed across methods.
  3. [§III.B, §III.C, Eqs. (8)-(12)] There is an internal inconsistency in the diffusion formulation. Eq. (8) defines q(z_{t'}|z_{t'-1}) as a Gaussian centered on sqrt(1-beta_t') z_gt, which is not a valid Markovian forward step and is inconsistent with the closed-form sampling distribution in Eq. (9). The conditioning should be on z_{t'-1}, or Eq. (8) should be removed and Eq. (9) presented as the forward process. In addition, the paper replaces the standard DDPM noise-prediction loss of Eq. (11) with the direct final-embedding loss of Eq. (12) and uses T=5 steps from random Gaussian noise at inference; the authors should justify why this objective is sufficient to learn the reverse process and clarify whether the model is still performing diffusion at test time rather than a deterministic conditional generation. A small comparison between the Eq. (11) objective and the Eq. (12) objective under the same T would directly address this concern.
minor comments (5)
  1. [Throughout] The dataset name is inconsistently spelled as 'SNU-FLIM' (e.g., §IV.A.1) and 'SNU-FILM' (e.g., tables), and Fig. 6's caption writes 'SUN-FILM'. Please standardize.
  2. [§IV.D.3 and Table VI] The text says 'EventDiff-V0 to EventDiff-V7' and then describes V1 through V6, but Table VI lists only V0-V6. Either add V7 or remove the reference.
  3. [Fig. 2 and §III.A] The text says 'In the first stage (the right of Fig. 2)', but the figure labels Stage 1 on the left and Stage 2 on the right. Please correct the reference.
  4. [Table V] Table V reports parameter counts of 5.2-6.0 M for fusion strategies, while Table IV lists Our-f3 as 18.7 M; the relationship between the two numbers should be stated explicitly (e.g., encoder-only vs. total model).
  5. [Throughout] There are small text errors: 'Timelens' vs. 'TimeLens', 'Vimeo90k' vs. 'Vimeo90K', and 'preformance' in the Fig. 1 caption. A copyedit pass would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are externally benchmarked supervised-learning results, and the author-finetuned baselines are a comparison-protocol concern rather than a circular derivation.

full rationale

EventDiff is a trained interpolation system whose central claims are benchmark PSNR/SSIM/LPIPS numbers on external synthetic and real-world datasets (Vimeo90K-Triplet, GoPro, SNU-FILM, HQ-EVFI, BS-ERGB, HQF) against external baselines. The diffusion loss in Eq. (12) and the joint loss in Eq. (14) directly minimize L1 distances between the denoised embedding/output and the ground-truth embedding/frame; this is standard supervised regression, not a derivation in which the target is an input by construction. The HAE borrows VQGAN/VQFIGAN/LDMVFI building blocks, but the event-fusion STCA module and the two-stage training strategy are stated as design choices, not as results forced by a self-citation or by a uniqueness theorem. The only notable evaluation caveat is that the TimeLens, CBMNet-L and TLXNet+ numbers in Table I come from author-finetuned models on simulated events, as stated in Section IV.A.3 and the Table I footnote; this is a fairness and verifiability concern about baseline comparability, not a circularity, because those baseline numbers are not used to define EventDiff's objective, architecture, or derived claims. No load-bearing step reduces to its own input, and no self-citation chain is invoked. Therefore no circularity is present.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The system introduces architectural components (HAE, STCA, 5-step denoising) and hyperparameters, which are either tuned on the target benchmark or adopted from prior diffusion work.

free parameters (4)
  • T (number of diffusion steps) = 5
    The paper chooses T=5 by evaluating PSNR versus step count (Fig. 8a); this is a hyperparameter tuned on the Vimeo90K-Triplet validation/test, not derived from first principles.
  • Number of downsample blocks (f3 vs f4) = 3 or 4
    The latent spatial size is chosen by comparing f2, f3, f4, f5 on Vimeo90K-Triplet (Table IV). This is an architecture hyperparameter tuned on the target benchmark.
  • Variance schedule beta_t = linear from 1e-5 to 0.1
    Chosen as standard practice in diffusion models; no search or derivation is reported, but it is a free hyperparameter.
  • Learning rates and training epochs = 1e-5 / 1e-6, 90 epochs
    Standard hyperparameters chosen for training; not derived.
assumptions (3)
  • domain assumption The latent embedding z captures enough information that denoising it with 5 steps and decoding yields high-quality frames.
    This is the core design assumption of the two-stage HAE+diffusion framework; it is validated empirically by Tables I, IV, and Fig. 8, but not proven.
  • domain assumption Events simulated by ESIM on Vimeo90K/GoPro transfer to real event data and to the finetuned baselines.
    The paper trains on synthetic events and finetunes on real-world datasets; the sim-to-real gap is a known concern in event vision and is only partially mitigated by the real-world results.
  • domain assumption The comparison baselines (TimeLens, CBMNet-L, TLXNet+) are fairly finetuned and evaluated.
    The paper states these are finetuned on its own simulated data and implementation from [14]; this assumption directly affects the claimed gains.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation." pith.science (2026). https://pith.science/paper/WZE7XCN2

@misc{pith2026250508235,
  author       = {Pith},
  title        = {Pith review of: EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZE7XCN2}},
  note         = {Machine review of arXiv:2505.08235}
}
read the original abstract

Video Frame Interpolation (VFI) is a fundamental yet challenging task in computer vision, particularly under conditions involving large motion, occlusion, and lighting variation. Recent advancements in event cameras have opened up new opportunities for addressing these challenges. While existing event-based VFI methods have succeeded in recovering large and complex motions by leveraging handcrafted intermediate representations such as optical flow, these designs often compromise high-fidelity image reconstruction under subtle motion scenarios due to their reliance on explicit motion modeling. Meanwhile, diffusion models provide a promising alternative for VFI by reconstructing frames through a denoising process, eliminating the need for explicit motion estimation or warping operations. In this work, we propose EventDiff, a unified and efficient event-based diffusion model framework for VFI. EventDiff features a novel Event-Frame Hybrid AutoEncoder (HAE) equipped with a lightweight Spatial-Temporal Cross Attention (STCA) module that effectively fuses dynamic event streams with static frames. Unlike previous event-based VFI methods, EventDiff performs interpolation directly in the latent space via a denoising diffusion process, making it more robust across diverse and challenging VFI scenarios. Through a two-stage training strategy that first pretrains the HAE and then jointly optimizes it with the diffusion model, our method achieves state-of-the-art performance across multiple synthetic and real-world event VFI datasets. The proposed method outperforms existing state-of-the-art event-based VFI methods by up to 1.98dB in PSNR on Vimeo90K-Triplet and shows superior performance in SNU-FILM tasks with multiple difficulty levels. Compared to the emerging diffusion-based VFI approach, our method achieves up to 5.72dB PSNR gain on Vimeo90K-Triplet and 4.24X faster inference.

Figures

Figures reproduced from arXiv: 2505.08235 by the authors.

Figure 1
Figure 1. EventDiff compared with previous Event-based VFI methods. (a) The traditional event-based VFI methods, such as Timelens [11] et al., use multi-component architectures with separate warping, synthesis and refinement modules. These methods rely on intermediate products like optical flow serving as manually designed auxiliary elements for fusing frames and events, which make bottlenecks and lead to suboptimal results. … view at source ↗
Figure 2
Figure 2. Illustration of the EventDiff framework. The framework consists of two training stages. In Stage 1 (left), we train the Event-Frame Hybrid AutoEncoder (HAE) using ground-truth inputs, enabling it to capture hybrid pyramid features and project the ground-truth into the latent space. In Stage 2 (right), the ground-truth inputs are removed and a diffusion model is employed to reconstruct the ground-truth embedding with… view at source ↗
Figure 3
Figure 3. Illustration of the Event-Frame Hybrid AutoEncoder (HAE) architecture. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Illustration of (a) Temporal Cross Attention (TCA) and (b) Spatial [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the U-net with a condition encoder for diffusion model. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visual comparison of VFI methods on the SUN-FILM dataset with hard-level (the first two rows) and extreme-level (the last two rows) conditions. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visual comparison of VFI methods on HQ-EVFI dataset. We select two sub-regions to zoom-in for better details. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Analysis of the iterative diffusion and denoising processes of the [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 44 canonical work pages

  1. [40]

    Motion-aware latent diffusion models for video frame interpolation,

    Z. Huang, Y . Yu, L. Yang, C. Qin, B. Zheng, X. Zheng, Z. Zhou, Y . Wang, and W. Yang, “Motion-aware latent diffusion models for video frame interpolation,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 1043–1052

    work page 2024

  2. [14]

    Timelens-xl: Real-time event-based video frame interpolation with large motion,

    Y . Ma, S. Guo, Y . Chen, T. Xue, and J. Gu, “Timelens-xl: Real-time event-based video frame interpolation with large motion,” in European Conference on Computer Vision . Springer, 2024, pp. 178–194

    work page 2024

  3. [50]

    Video to events: Recycling video datasets for event cameras,

    D. Gehrig, M. Gehrig, J. Hidalgo-Carri ´o, and D. Scaramuzza, “Video to events: Recycling video datasets for event cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3586–3595

    2020

  4. [1]

    Many-to-many splatting for efficient video frame interpolation,

    P. Hu, S. Niklaus, S. Sclaroff, and K. Saenko, “Many-to-many splatting for efficient video frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 3553–3562

    work page 2022

  5. [2]

    Real-time intermediate flow estimation for video frame interpolation,

    Z. Huang, T. Zhang, W. Heng, B. Shi, and S. Zhou, “Real-time intermediate flow estimation for video frame interpolation,” in European Conference on Computer Vision . Springer, 2022, pp. 624–642

    work page 2022

  6. [3]

    Super slomo: High quality estimation of multiple intermediate frames for video interpolation,

    H. Jiang, D. Sun, V . Jampani, M.-H. Yang, E. Learned-Miller, and J. Kautz, “Super slomo: High quality estimation of multiple intermediate frames for video interpolation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 9000–9008

    work page 2018

  7. [4]

    En- hanced bi-directional motion estimation for video frame interpolation,

    X. Jin, L. Wu, G. Shen, Y . Chen, J. Chen, J. Koo, and C.-h. Hahm, “En- hanced bi-directional motion estimation for video frame interpolation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 5049–5057

    work page 2023

  8. [5]

    Bmbc: Bilateral motion esti- mation with bilateral cost volume for video interpolation,

    J. Park, K. Ko, C. Lee, and C.-S. Kim, “Bmbc: Bilateral motion esti- mation with bilateral cost volume for video interpolation,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16 . Springer, 2020, pp. 109–125

    work page 2020

Show all 68 references
  1. [6]

    Asymmetric bilateral motion estima- tion for video frame interpolation,

    J. Park, C. Lee, and C.-S. Kim, “Asymmetric bilateral motion estima- tion for video frame interpolation,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 14 539–14 548

    2021

  2. [7]

    Ladder: An efficient framework for video frame interpolation,

    T. Shen, D. Li, Z. Gao, L. Tian, and E. Barsoum, “Ladder: An efficient framework for video frame interpolation,” arXiv preprint arXiv:2404.11108, 2024

    2024 arXiv

  3. [8]

    Video frame interpolation via adaptive convolution,

    S. Niklaus, L. Mai, and F. Liu, “Video frame interpolation via adaptive convolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 670–679

    2017

  4. [9]

    Revisiting adaptive convolutions for video frame interpolation,

    S. Niklaus, L. Mai, and O. Wang, “Revisiting adaptive convolutions for video frame interpolation,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision , 2021, pp. 1099–1109

    2021

  5. [10]

    A low power, fully event-based gesture recognition system,

    A. Amir, B. Taba, D. Berg, T. Melano, J. McKinstry, C. Di Nolfo, T. Nayak, A. Andreopoulos, G. Garreau, M. Mendoza et al. , “A low power, fully event-based gesture recognition system,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp....

    2017

  6. [11]

    Time lens: Event-based video frame interpolation,

    S. Tulyakov, D. Gehrig, S. Georgoulis, J. Erbach, M. Gehrig, Y . Li, and D. Scaramuzza, “Time lens: Event-based video frame interpolation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 155–16 164

    2021

  7. [12]

    Time lens++: Event-based frame interpolation with parametric non-linear flow and multi-scale fusion,

    S. Tulyakov, A. Bochicchio, D. Gehrig, S. Georgoulis, Y . Li, and D. Scaramuzza, “Time lens++: Event-based frame interpolation with parametric non-linear flow and multi-scale fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, p...

    2022

  8. [13]

    Event-based video frame interpolation with cross-modal asymmetric bidirectional motion fields,

    T. Kim, Y . Chae, H.-K. Jang, and K.-J. Yoon, “Event-based video frame interpolation with cross-modal asymmetric bidirectional motion fields,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18 032–18 042

    2023

  9. [15]

    Video frame interpolation via direct synthesis with the event-based reference,

    Y . Liu, Y . Deng, H. Chen, and Z. Yang, “Video frame interpolation via direct synthesis with the event-based reference,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 8477–8487

    2024

  10. [16]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502 , 2020

    2010 arXiv

  11. [17]

    Score-based generative modeling through stochastic differ- ential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,” arXiv preprint arXiv:2011.13456 , 2020

    2011 arXiv

  12. [18]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,” Advances in Neural Information Processing Systems, vol. 35, pp. 8633–8646, 2022

    2022

  13. [19]

    Make-a-video: Text-to-video generation without text-video data,

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni et al. , “Make-a-video: Text-to-video generation without text-video data,” arXiv preprint arXiv:2209.14792 , 2022

    2022 arXiv

  14. [20]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation,

    S. Yin, C. Wu, H. Yang, J. Wang, X. Wang, M. Ni, Z. Yang, L. Li, S. Liu, F. Yang et al., “Nuwa-xl: Diffusion over diffusion for extremely long video generation,” arXiv preprint arXiv:2303.12346 , 2023

    2023 arXiv

  15. [21]

    Dynamicrafter: Animating open-domain im- ages with video diffusion priors,

    J. Xing, M. Xia, Y . Zhang, H. Chen, W. Yu, H. Liu, G. Liu, X. Wang, Y . Shan, and T.-T. Wong, “Dynamicrafter: Animating open-domain im- ages with video diffusion priors,” in European Conference on Computer Vision. Springer, 2024, pp. 399–417

    2024

  16. [22]

    Mcvd-masked conditional video diffusion for prediction, generation, and interpolation,

    V . V oleti, A. Jolicoeur-Martineau, and C. Pal, “Mcvd-masked conditional video diffusion for prediction, generation, and interpolation,” Advances in neural information processing systems , vol. 35, pp. 23 371–23 385, 2022

    2022

  17. [23]

    Ldmvfi: Video frame interpolation with latent diffusion models,

    D. Danier, F. Zhang, and D. Bull, “Ldmvfi: Video frame interpolation with latent diffusion models,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 2, 2024, pp. 1472–1480

    2024

  18. [24]

    Frame interpolation with con- secutive brownian bridge diffusion,

    Z. Lyu, M. Li, J. Jiao, and C. Chen, “Frame interpolation with con- secutive brownian bridge diffusion,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 3449–3458

    2024

  19. [25]

    Video frame interpolation via deformable separable convolution,

    X. Cheng and Z. Chen, “Video frame interpolation via deformable separable convolution,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, 2020, pp. 10 607–10 614

    2020

  20. [26]

    Multiple video frame interpolation via enhanced deformable separable convolution,

    ——, “Multiple video frame interpolation via enhanced deformable separable convolution,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 7029–7045, 2021

    2021

  21. [27]

    Pdwn: Pyramid deformable warping network for video interpolation,

    Z. Chen, R. Wang, H. Liu, and Y . Wang, “Pdwn: Pyramid deformable warping network for video interpolation,” IEEE Open Journal of Signal Processing, vol. 2, pp. 413–424, 2021

    2021

  22. [28]

    Memc-net: Motion estimation and motion compensation driven neural network for video interpolation and enhancement,

    W. Bao, W.-S. Lai, X. Zhang, Z. Gao, and M.-H. Yang, “Memc-net: Motion estimation and motion compensation driven neural network for video interpolation and enhancement,” IEEE transactions on pattern analysis and machine intelligence , vol. 43, no. 3, pp. 933–948, 2019

    2019

  23. [29]

    St-mfnet: A spatio-temporal multi- flow network for frame interpolation,

    D. Danier, F. Zhang, and D. Bull, “St-mfnet: A spatio-temporal multi- flow network for frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 3521–3531

    2022

  24. [30]

    Video frame interpolation with transformer,

    L. Lu, R. Wu, H. Lin, J. Lu, and J. Jia, “Video frame interpolation with transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 3532–3542

    2022

  25. [31]

    Extracting motion and appearance via inter-frame attention for efficient video frame interpolation,

    G. Zhang, Y . Zhu, H. Wang, Y . Chen, G. Wu, and L. Wang, “Extracting motion and appearance via inter-frame attention for efficient video frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 5682–5692

    2023

  26. [32]

    Amt: All-pairs multi-field transforms for efficient frame interpolation,

    Z. Li, Z.-L. Zhu, L.-H. Han, Q. Hou, C.-L. Guo, and M.-M. Cheng, “Amt: All-pairs multi-field transforms for efficient frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9801–9810

    2023

  27. [33]

    Frame interpolation transformer and uncertainty guid- ance,

    M. Plack, K. M. Briedis, A. Djelouah, M. B. Hullin, M. Gross, and C. Schroers, “Frame interpolation transformer and uncertainty guid- ance,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9811–9821

    2023

  28. [34]

    Efi-net: Video frame interpolation from fusion of events and frames,

    G. Paikin, Y . Ater, R. Shaul, and E. Soloveichik, “Efi-net: Video frame interpolation from fusion of events and frames,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 1291–1301

    2021

  29. [35]

    Superfast: 200× video frame interpolation via event camera,

    Y . Gao, S. Li, Y . Li, Y . Guo, and Q. Dai, “Superfast: 200× video frame interpolation via event camera,” IEEE transactions on pattern analysis and machine intelligence , vol. 45, no. 6, pp. 7764–7780, 2022

    2022

  30. [36]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  31. [37]

    Video interpolation with diffusion models,

    S. Jain, D. Watson, E. Tabellion, B. Poole, J. Kontkanen et al., “Video interpolation with diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 7341–7351. 15

    2024

  32. [38]

    Framer: Interactive frame interpolation,

    W. Wang, Q. Wang, K. Zheng, H. Ouyang, Z. Chen, B. Gong, H. Chen, Y . Shen, and C. Shen, “Framer: Interactive frame interpolation,” arXiv preprint arXiv:2410.18978, 2024

    2024 arXiv

  33. [39]

    Dreammover: Leveraging the prior of diffusion models for image interpolation with large motion,

    L. Shen, T. Liu, H. Sun, X. Ye, B. Li, J. Zhang, and Z. Cao, “Dreammover: Leveraging the prior of diffusion models for image interpolation with large motion,” in European Conference on Computer Vision. Springer, 2024, pp. 336–353

    2024

  34. [41]

    Mobilenets: Efficient convo- lutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convo- lutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  35. [42]

    Maxvit: Multi-axis vision transformer,

    Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y . Li, “Maxvit: Multi-axis vision transformer,” in European conference on computer vision. Springer, 2022, pp. 459–479

    2022

  36. [43]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyals et al. , “Neural discrete representation learning,” Advances in neural information processing systems , vol. 30, 2017

    2017

  37. [44]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  38. [45]

    Image-to-image translation with conditional adversarial networks,

    P. Isola, J.-Y . Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125– 1134

    2017

  39. [46]

    Diffi2i: efficient diffusion model for image-to-image translation,

    B. Xia, Y . Zhang, S. Wang, Y . Wang, X. Wu, Y . Tian, W. Yang, R. Timotfe, and L. Van Gool, “Diffi2i: efficient diffusion model for image-to-image translation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  40. [47]

    Video enhance- ment with task-oriented flow,

    T. Xue, B. Chen, J. Wu, D. Wei, and W. T. Freeman, “Video enhance- ment with task-oriented flow,”International Journal of Computer Vision, vol. 127, pp. 1106–1125, 2019

    2019

  41. [48]

    Deep multi-scale convolutional neural network for dynamic scene deblurring,

    S. Nah, T. Hyun Kim, and K. Mu Lee, “Deep multi-scale convolutional neural network for dynamic scene deblurring,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 3883–3891

    2017

  42. [49]

    Channel attention is all you need for video frame interpolation,

    M. Choi, H. Kim, B. Han, N. Xu, and K. M. Lee, “Channel attention is all you need for video frame interpolation,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 07, 2020, pp. 10 663–10 671

    2020

  43. [51]

    Reducing the sim-to-real gap for event cameras,

    T. Stoffregen, C. Scheerlinck, D. Scaramuzza, T. Drummond, N. Barnes, L. Kleeman, and R. Mahony, “Reducing the sim-to-real gap for event cameras,” in Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVII 16 . Springer, ...

    2020

  44. [52]

    Ifrnet: Intermediate feature refine network for efficient frame interpolation,

    L. Kong, B. Jiang, D. Luo, W. Chu, X. Huang, Y . Tai, C. Wang, and J. Yang, “Ifrnet: Intermediate feature refine network for efficient frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 1969–1978

    2022

  45. [53]

    A unified pyramid recurrent network for video frame interpolation,

    X. Jin, L. Wu, J. Chen, Y . Chen, J. Koo, and C.-h. Hahm, “A unified pyramid recurrent network for video frame interpolation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1578–1587

    2023

  46. [54]

    Video interpolation by event-driven anisotropic adjustment of optical flow,

    S. Wu, K. You, W. He, C. Yang, Y . Tian, Y . Wang, Z. Zhang, and J. Liao, “Video interpolation by event-driven anisotropic adjustment of optical flow,” in European Conference on Computer Vision . Springer, 2022, pp. 267–283

    2022

  47. [55]

    Event-based fusion for motion deblurring with cross- modal attention,

    L. Sun, C. Sakaridis, J. Liang, Q. Jiang, K. Yang, P. Sun, Y . Ye, K. Wang, and L. V . Gool, “Event-based fusion for motion deblurring with cross- modal attention,” in European conference on computer vision. Springer, 2022, pp. 412–428

    2022

  48. [56]

    Scale-recurrent network for deep image deblurring,

    X. Tao, H. Gao, X. Shen, J. Wang, and J. Jia, “Scale-recurrent network for deep image deblurring,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 8174–8182

    2018

  49. [57]

    Hinet: Half instance normalization network for image restoration,

    L. Chen, X. Lu, J. Zhang, X. Chu, and C. Chen, “Hinet: Half instance normalization network for image restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 182–192

    2021

  50. [58]

    Learning degradation representations for image deblurring,

    D. Li, Y . Zhang, K. C. Cheung, X. Wang, H. Qin, and H. Li, “Learning degradation representations for image deblurring,” in European confer- ence on computer vision . Springer, 2022, pp. 736–753

    2022

  51. [59]

    Simple baselines for image restoration,

    L. Chen, X. Chu, X. Zhang, and J. Sun, “Simple baselines for image restoration,” in European conference on computer vision . Springer, 2022, pp. 17–33

    2022

  52. [60]

    Self-supervised non- uniform kernel estimation with flow-based motion prior for blind image deblurring,

    Z. Fang, F. Wu, W. Dong, X. Li, J. Wu, and G. Shi, “Self-supervised non- uniform kernel estimation with flow-based motion prior for blind image deblurring,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 18 105–18 114

    2023

  53. [61]

    Bringing events into video deblurring with non-consecutively blurry frames,

    W. Shang, D. Ren, D. Zou, J. S. Ren, P. Luo, and W. Zuo, “Bringing events into video deblurring with non-consecutively blurry frames,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 4531–4540

    2021

  54. [62]

    Learning to deblur and generate high frame rate video with an event camera,

    C. Haoyu, T. Minggui, S. Boxin, W. YIzhou, and H. Tiejun, “Learning to deblur and generate high frame rate video with an event camera,” arXiv preprint arXiv:2003.00847 , 2020

    2003 arXiv

  55. [63]

    Motion Aware Event Representation-Driven Image Deblurring,

    Z. Sun, X. Fu, L. Huang, A. Liu, and Z.-J. Zha, “Motion Aware Event Representation-Driven Image Deblurring,” in European Conference on Computer Vision. Springer, 2024, pp. 418–435

    2024

  56. [64]

    Motion deblurring via spatial- temporal collaboration of frames and events,

    W. Yang, J. Wu, J. Ma, L. Li, and G. Shi, “Motion deblurring via spatial- temporal collaboration of frames and events,” inProceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 7, 2024, pp. 6531– 6539

    2024

  57. [65]

    Diffir: Efficient diffusion model for image restoration,

    B. Xia, Y . Zhang, S. Wang, Y . Wang, X. Wu, Y . Tian, W. Yang, and L. Van Gool, “Diffir: Efficient diffusion model for image restoration,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 13 095–13 105

    2023

  58. [66]

    Multiscale structure guided diffusion for image deblurring,

    M. Ren, M. Delbracio, H. Talebi, G. Gerig, and P. Milanfar, “Multiscale structure guided diffusion for image deblurring,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 10 721–10 733

    2023

  59. [67]

    Hierarchical integration diffusion model for realistic image deblurring,

    Z. Chen, Y . Zhang, D. Liu, J. Gu, L. Kong, X. Yuan et al., “Hierarchical integration diffusion model for realistic image deblurring,” Advances in neural information processing systems , vol. 36, 2024

    2024

  60. [68]

    Efficient image deblurring networks based on diffusion models,

    K. Chen and Y . Liu, “Efficient image deblurring networks based on diffusion models,” arXiv preprint arXiv:2401.05907 , 2024. Hanle Zheng received the B.E. degree in Measure- ment and Control Technology and Instruments from the Department of Precision Instrument, Tsinghua Univ...

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.