{"id":"f3773699-9029-4ac9-a890-b90045e8a84b","arxiv_id":"2505.08235","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"EventDiff reports state-of-the-art event-based video frame interpolation by running a latent diffusion denoising process on an event-frame hybrid autoencoder, with only 5 inference steps.","lead":"EventDiff is a new video frame interpolation system that uses event cameras and a diffusion model to create missing frames, and the paper reports large accuracy gains over existing methods on standard benchmarks. It is worth reading because it combines two currently hot techniques, event cameras and diffusion models, and claims both higher quality and much faster speed than prior diffusion-based approaches.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline gains over event-based SOTA rest on author-finetuned baselines; official-weight evaluation is needed to verify the 1.98 dB claim.","rationale":"The reader's weakest assumption and my stress-test pass converge on the same point: the empirical foundation of the paper's headline claim is the finetuned baseline comparison in Section IV.A.3 and the Table I footnote. I looked for other possible load-bearing concerns, but none are stronger. The architecture is clearly described, the two-stage training is coherent, the ablations in Tables IV, V, and VI support the design choices, and the reduction to five diffusion steps is substantiated by Figure 8. The method's novelty—latent-space diffusion conditioned on hybrid event-frame features—is plausible, and the reported efficiency gain is internally consistent with the step-count reduction. The absence of code is a reproducibility limitation but not a correctness flaw. The missing quantitative comparison with Huang et al. [40] is a completeness gap, but it does not directly undermine the stated central claim as much as the author-finetuned baselines do. My concern is a condition: the numbers may be valid, but the paper has not yet demonstrated that the baselines were given a fair and consistent protocol. Since the authors themselves disclose the finetuning procedure, the issue is addressable through an official-weight evaluation or a detailed protocol comparison. Therefore, the CONDITIONAL verdict remains appropriate, and I do not see grounds to move to ACCEPT or REJECT.","tokens_in":23909,"tokens_out":2021,"duration_ms":21997,"concrete_test":"Obtain the official pretrained checkpoints of TimeLens, CBMNet-L, and TLXNet+ (or retrain them using their original event-simulation pipelines and training recipes), evaluate them on Vimeo90K-Triplet, GoPro, and SNU-FILM with the same metrics as EventDiff, and recompute the reported margins. If the official or originally trained baselines match the author-finetuned numbers within noise and the 1.98 dB margin persists, the concern is resolved. If the baselines improve materially and the margin shrinks or reverses, the SOTA claim must be qualified and the finetuning protocol reported in detail.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—up to 1.98 dB PSNR over the best event-based VFI method on Vimeo90K-Triplet and SOTA across SNU-FILM difficulty levels—is calibrated against TimeLens, CBMNet-L, and TLXNet+ numbers that the authors produced by finetuning those models on their own ESIM-simulated events using the implementation of [14]. This is stated explicitly in Section IV.A.3 and in the Table I footnote. The paper acknowledges that official pretrained weights, training code, and dataset generation processes are not fully available for these models, so the comparison is not against the originally reported baselines. If the finetuning protocol or simulated event distribution differs from what those methods were designed and trained for—for example in temporal binning, noise level, or event polarity statistics—the baseline PSNR/SSIM values could be depressed, inflating EventDiff's margin. This matters because the headline 'best event-based method' comparison in both Table I and Table III includes at least one author-finetuned entry. The qualitative claim that direct denoising avoids flow-estimation errors is also not directly measured, but the load-bearing issue is the comparison baseline: a systematic protocol mismatch could reduce or reverse the reported gains over event-based SOTA. This is a condition on the central claim rather than an internal inconsistency.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EventDiff, a latent-diffusion framework for event-based video frame interpolation. An Event-Frame Hybrid AutoEncoder (HAE) with a Spatial-Temporal Cross Attention (STCA) module encodes the two boundary frame-event pairs into hybrid pyramid features and projects the ground-truth frame into a compact embedding; a U-Net denoiser then reconstructs that embedding from the boundary conditions, and the HAE decoder synthesizes the interpolated frame. Training is two-stage: the HAE is first pretrained with ground-truth supervision, then jointly optimized with the diffusion model using a loss that combines the L1 error between the final denoised embedding and the ground-truth embedding and the L1 error between the reconstructed frame and the ground-truth frame. Experiments are reported on Vimeo90K-Triplet, GoPro, SNU-FILM, and three real-world event datasets, with claims of state-of-the-art PSNR/SSIM, a 1.98 dB gain over the best event-based method on Vimeo90K-Triplet, a 5.72 dB gain over diffusion-based methods, and 4.24x faster inference with only 5 denoising steps. The paper also reports an extension to event-based motion deblurring on GoPro.","tokens_in":24243,"tokens_out":8530,"duration_ms":84196,"significance":"If the reported results are reproducible, EventDiff would be a practically significant contribution: it combines event data with a latent diffusion process in a way that achieves high distortion metrics (PSNR/SSIM), not just perceptual quality, while reducing the inference cost to five steps. The paper's ablations are a genuine strength: Tables IV-VI and Fig. 8 provide informative analyses of embedding spatial size, fusion strategy, training schemes, and step count, and the method description is detailed enough to reimplement. The extensibility experiment to motion deblurring, while not SOTA, is a useful demonstration of generality. However, the central benchmark claims are only as strong as the comparison protocol; the author-finetuned baselines and underspecified event-simulation details are the main risks to the paper's headline numbers.","major_comments":[{"comment":"The paper states that TimeLens, CBMNet-L, and TLXNet+ are finetuned by the authors on their own simulated data using the implementation of [14], and the Table I footnote repeats this. Because TLXNet+ is the strongest event-based competitor in several reported columns (GoPro 7-skips, SNU-FILM extreme, BS-ERGB), any systematic mismatch between the authors' finetuning pipeline and the original training protocol could lower those baseline numbers and inflate EventDiff's margins. This is load-bearing for the 'superior performance in SNU-FILM' claim and for the real-world SOTA claims in Table III. I note that the specific 1.98 dB Vimeo90K-Triplet gain is computed against DSEVFI, which is not in the finetuned set, so that particular number is less affected; nevertheless, the manuscript should, per table and per setting, state which entries use official weights, report the original published numbers alongside the finetuned ones, and provide the full finetuning protocol (event simulator parameters, temporal binning, epochs, learning rates, and the code used for finetuning).","section":"§IV.A.3 and Tables I/III"},{"comment":"The event simulation and evaluation protocol is underspecified. Section IV.A.1 says only that 'synthetic event data simulated using ESIM [50]', and Eq. (3) uses T temporal bins, but the experimental settings never give T, the ESIM contrast thresholds, noise settings, or the random seeds. Event-based methods in Tables I and III are known to be sensitive to event representation. The paper should also clarify whether the non-finetuned entries (e.g., DSEVFI) are taken from the original publications or re-evaluated on the authors' event streams; if re-evaluated, the exact preprocessing must be given. Without these details, the comparisons are not reproducible and different test-event distributions could be mixed across methods.","section":"§IV.A.1, §IV.A.3, and Eq. (3)"},{"comment":"There is an internal inconsistency in the diffusion formulation. Eq. (8) defines q(z_{t'}|z_{t'-1}) as a Gaussian centered on sqrt(1-beta_t') z_gt, which is not a valid Markovian forward step and is inconsistent with the closed-form sampling distribution in Eq. (9). The conditioning should be on z_{t'-1}, or Eq. (8) should be removed and Eq. (9) presented as the forward process. In addition, the paper replaces the standard DDPM noise-prediction loss of Eq. (11) with the direct final-embedding loss of Eq. (12) and uses T=5 steps from random Gaussian noise at inference; the authors should justify why this objective is sufficient to learn the reverse process and clarify whether the model is still performing diffusion at test time rather than a deterministic conditional generation. A small comparison between the Eq. (11) objective and the Eq. (12) objective under the same T would directly address this concern.","section":"§III.B, §III.C, Eqs. (8)-(12)"}],"minor_comments":[{"comment":"The dataset name is inconsistently spelled as 'SNU-FLIM' (e.g., §IV.A.1) and 'SNU-FILM' (e.g., tables), and Fig. 6's caption writes 'SUN-FILM'. Please standardize.","section":"Throughout"},{"comment":"The text says 'EventDiff-V0 to EventDiff-V7' and then describes V1 through V6, but Table VI lists only V0-V6. Either add V7 or remove the reference.","section":"§IV.D.3 and Table VI"},{"comment":"The text says 'In the first stage (the right of Fig. 2)', but the figure labels Stage 1 on the left and Stage 2 on the right. Please correct the reference.","section":"Fig. 2 and §III.A"},{"comment":"Table V reports parameter counts of 5.2-6.0 M for fusion strategies, while Table IV lists Our-f3 as 18.7 M; the relationship between the two numbers should be stated explicitly (e.g., encoder-only vs. total model).","section":"Table V"},{"comment":"There are small text errors: 'Timelens' vs. 'TimeLens', 'Vimeo90k' vs. 'Vimeo90K', and 'preformance' in the Fig. 1 caption. A copyedit pass would improve readability.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The main risk is empirical rather than conceptual: the headline SOTA claims on SNU-FILM and real-world datasets rest partly on author-finetuned baselines. If the authors can provide original published baseline numbers, a precise finetuning protocol, and a clear statement of which entries were re-evaluated, the paper would be substantially stronger. The diffusion-formulation inconsistency in Eqs. (8)-(12) is fixable with rewriting and a small experiment. No concerns about novelty or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"EventDiff is worth a serious look. It is a genuinely new combination: latent diffusion for event-based VFI, with an event-frame hybrid autoencoder, factorized spatial-temporal cross-attention, and a direct L1 loss on the final denoised embedding rather than per-step noise prediction. The five-step sampling with joint fine-tuning of the decoder is a pragmatic design that appears to work, and the efficiency numbers are consistent with that design.\n\nThe paper is mostly well-executed. The architecture is described at reimplementation level. The ablations (embedding size, fusion method, training scheme, step count) are the ones you'd want to see, and they support the design choices. The deblurring extension is a small appendix-style experiment, but it shows the framework generalizes. I believe the core contribution is real.\n\nThe soft spots are about comparison fairness, not internal correctness. TimeLens, CBMNet-L, and TLXNet+ are finetuned by the authors on their own simulated event data because official weights or training recipes are unavailable. That is disclosed in Section IV.A.3 and the Table I footnote, which is good. But it means the margins over those baselines in GoPro, SNU-FILM, and the real-world tables could be inflated by simulator differences. One nuance: the 1.98 dB headline on Vimeo90K-Triplet is against DSEVFI, which is not one of the finetuned models, so the stress-test note that says 'headline gains rest on author-finetuned baselines' is not quite right for that number. The broader point still holds for many other entries. The fix is straightforward: compare to official weights where available, report the simulator and finetuning protocol in detail, and include variance across training runs.\n\nThe other real gap is the missing quantitative comparison to Huang et al. [40], the closest event+diffusion prior. It is cited in related work but never appears in the tables. For a paper claiming SOTA among event-diffusion methods, that omission needs to be fixed. Minor issues: no code/data release, no variance estimates, and the 'avoids flow errors' claim is qualitative rather than measured.\n\nNet: this is a solid, novel empirical contribution. It deserves serious peer review. I would accept it conditionally, with the baseline fairness analysis and the [40] comparison as required revisions, and I'd encourage the authors to release code and data.","headline":"A genuinely new latent diffusion framework for event-based VFI with strong ablations; headline gains are mostly credible, but author-finetuned baselines need scrutiny and the closest event-diffusion prior is missing from the tables.","tokens_in":24736,"tokens_out":3395,"would_cite":true,"duration_ms":30541,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EventDiff shows that video frame interpolation benefits when a diffusion model denoises in latent space rather than estimating optical flow.","keywords":["event-based video frame interpolation","latent diffusion model","event camera","video frame interpolation","spatio-temporal cross attention","hybrid autoencoder","event-enhanced visual generation","motion deblurring"],"falsifier":"Rerun the comparisons with the released pretrained weights of TimeLens, CBMNet-L, and TLXNet+, or with their original training protocols, on the same simulated and real event streams; if the PSNR or SSIM margin over EventDiff drops below the reported 1.98 dB or reverses on any dataset, the central state-of-the-art claim is falsified. A second check is to supply optical flow explicitly as an additional condition to EventDiff: if that version matches or beats the direct-denoising version, the paper's qualitative claim that direct denoising avoids flow errors is undermined.","tokens_in":23663,"feed_emoji":"⚡","tokens_out":6959,"duration_ms":62315,"temperature":0.7,"pith_summary":"EventDiff claims that the best way to use event-camera data for video frame interpolation is to skip explicit motion estimation altogether: an autoencoder fuses the two boundary frames with their event streams into a compact latent representation, and a diffusion process denoises the interpolated frame's code directly in that space. The paper argues this removes the handcrafted optical-flow and warping bottlenecks that limit previous event-based methods, while event data supplies the temporal detail that frame-only diffusion models lack. On Vimeo90K-Triplet, GoPro, and SNU-FILM, and on the real-world HQ-EVFI, BS-ERGB, and HQF datasets, it reports state-of-the-art PSNR and SSIM, with up to 1.98 dB over the best event-based method and 5.72 dB over the best diffusion-based method, at 4.24x faster inference thanks to a 5-step denoising schedule.","feed_headline":"Latent diffusion tops event-video interpolation by 1.98 dB","feed_subtitle":"Five denoising steps in a compressed space beat flow-based event pipelines and run 4.24x faster.","key_machinery":"The load-bearing mechanism is the Event-Frame Hybrid AutoEncoder (HAE), an autoencoder that compresses ground-truth frames into a small latent code ($H/2^n \\times W/2^n$) and builds a multi-scale hybrid feature pyramid from the boundary frame-event pairs; its decoder reconstructs the interpolated frame from the latent code plus that pyramid. Inside each downsampling block, the Spatial-Temporal Cross Attention (STCA) module fuses the two modalities by factorizing full spatio-temporal cross-attention into a spatial cross-attention (SCA) and a temporal cross-attention (TCA), reducing cost from $O(T\\hat{H}\\hat{W}\\cdot\\hat{H}\\hat{W})$ to $O(((T/p^2+\\hat{H}\\hat{W})\\cdot\\hat{H}\\hat{W})$. Stage 1 pretrains the HAE with an $L_1$ reconstruction loss plus a vector-quantization regularization; Stage 2 trains the diffusion U-Net to minimize the $L_1$ distance between the final denoised embedding and the ground-truth embedding, jointly fine-tuning the HAE decoder so that the whole pipeline works with only $T=5$ diffusion steps.","core_discovery":"The central claim is that a single end-to-end latent diffusion framework, EventDiff, can outperform dedicated flow-estimation pipelines on event-based frame interpolation while being substantially faster. The key evidence is that the Event-Frame Hybrid AutoEncoder produces a hybrid feature pyramid plus a ground-truth latent code, and the diffusion U-Net reconstructs that code from the boundary condition embeddings; because the decoder is jointly fine-tuned in the second training stage, the model tolerates residual diffusion error and needs only five denoising steps. The paper reports that this design reaches 41.15 dB PSNR on Vimeo90K-Triplet in its f3 variant and 38.14 dB in f4, surpassing the previous event-based best by 1.98 dB, and that it generalizes across difficulty levels on SNU-FILM and to event-based motion deblurring on GoPro with competitive results.","pith_inferences":["Editorial inference: a natural stress test is to freeze the pretrained HAE from Stage 1 and train only the diffusion stage on unseen event-camera domains, since the paper's own ablations show Stage-2 joint optimization is what absorbs residual latent error.","Editorial inference: the f3/f4 trade-off suggests that an adaptive or per-pixel choice of latent resolution could beat either fixed setting, a possibility the paper does not explore.","Editorial inference: because performance saturates at four to five denoising steps, distilling the diffusion process to one step or learning a deterministic mapping from condition embeddings to the target code is the obvious next efficiency experiment, though the paper only reports five steps.","Editorial inference: the comparison against baselines depends on simulated events, so a transfer test using real event streams from a different sensor would clarify whether the margin holds under the sim-to-real gap."],"forward_implications":["If the reported results hold, diffusion-based VFI can be both more accurate and faster than flow-warping event methods, reversing the usual trade-off where generative models win only on perceptual quality.","The 5-step denoising schedule, about 79 ms per 448x256 frame on an RTX 4090, makes latent-diffusion interpolation practical for near-real-time use.","Threading event information through a hybrid autoencoder gives gains across difficulty levels from easy to extreme motion, suggesting that a unified latent code can carry both detail and large-displacement information.","The same HAE-plus-diffusion recipe transfers to event-based motion deblurring with only task-specific input and output changes, supporting the paper's claim of a general event-enhanced generation framework."],"supporting_citations":[{"why":"Provides the latent-diffusion VFI baseline (LDMVFI) and the VQFIGAN autoencoder architecture that EventDiff adapts into the HAE.","marker":"[23]"},{"why":"Defines TLXNet+, the state-of-the-art event-based method whose reported performance EventDiff exceeds, and whose implementation is used to finetune the comparison baselines.","marker":"[14]"},{"why":"CBMNet-L, one of the strong cross-modal event-based baselines that EventDiff compares against and that the authors finetune on their simulated events.","marker":"[13]"},{"why":"TimeLens introduces the temporal event voxel representation and is the classic event-based VFI method EventDiff contrasts with.","marker":"[11]"},{"why":"DiffI2I supplies the training objective of directly minimizing the error of the final denoised embedding rather than per-step noise.","marker":"[46]"},{"why":"ESIM generates the synthetic event streams used to train and evaluate the method and to finetune the baselines.","marker":"[50]"},{"why":"The latent diffusion and VQGAN formulation grounds the two-stage latent-space design.","marker":"[36]"},{"why":"EFNet is the event-based deblurring method whose GoPro setup and event representation EventDiff reuses for its deblurring extension.","marker":"[55]"}],"fun_headline_variants":["EventDiff: Latent diffusion beats flow methods by 1.98 dB","Five-step latent diffusion outruns flow for event VFI","EventDiff: 1.98 dB gain, 4.24x faster interpolation","Unified latent diffusion wins event video frame interpolation","EventDiff: Diffusion in latent space, 1.98 dB better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage over prior event-based methods depends on the authors' own finetuning of those models on their simulated event data, so any mismatch between that finetuning protocol or event distribution and the original training could shrink the measured gap.","fun_headline_variants_meta":{"raw":{"variants":["EventDiff: Latent diffusion beats flow methods by 1.98 dB","Five-step latent diffusion outruns flow for event VFI","EventDiff: 1.98 dB gain, 4.24x faster interpolation","Unified latent diffusion wins event video frame interpolation","EventDiff: Diffusion in latent space, 1.98 dB better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001115,"raw_usage":{"total_tokens":4706,"prompt_tokens":1070,"completion_tokens":3636,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":3543}},"tokens_in":686,"tokens_out":3636,"duration_ms":22599,"temperature":1.0,"reasoning_tokens":3543,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:58:39.825296+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the comparisons with the released pretrained weights of TimeLens, CBMNet-L, and TLXNet+, or with their original training protocols, on the same simulated and real event streams; if the PSNR or SSIM margin over EventDiff drops below the reported 1.98 dB or reverses on any dataset, the central state-of-the-art claim is falsified. A second check is to supply optical flow explicitly as an additional condition to EventDiff: if that version matches or beats the direct-denoising version, the paper's qualitative claim that direct denoising avoids flow errors is undermined.","supporting_citations":[{"cited_title":"Ldmvfi: Video frame interpolation with latent diffusion models,","cited_arxiv_id":null,"evidence_quote":"Provides the latent-diffusion VFI baseline (LDMVFI) and the VQFIGAN autoencoder architecture that EventDiff adapts into the HAE."},{"cited_title":"Timelens-xl: Real-time event-based video frame interpolation with large motion,","cited_arxiv_id":null,"evidence_quote":"Defines TLXNet+, the state-of-the-art event-based method whose reported performance EventDiff exceeds, and whose implementation is used to finetune the comparison baselines."},{"cited_title":"Event-based video frame interpolation with cross-modal asymmetric bidirectional motion fields,","cited_arxiv_id":null,"evidence_quote":"CBMNet-L, one of the strong cross-modal event-based baselines that EventDiff compares against and that the authors finetune on their simulated events."},{"cited_title":"Diffi2i: efficient diffusion model for image-to-image translation,","cited_arxiv_id":null,"evidence_quote":"DiffI2I supplies the training objective of directly minimizing the error of the final denoised embedding rather than per-step noise."}],"review_version":1}