Pith. sign in

REVIEW 3 major objections 5 minor 17 references

EraserDiT: Fast Video Inpainting with Diffusion Transformer Model

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EraserDiT claims a diffusion transformer can erase large objects from 2K video in about a minute while beating prior methods on all reported metrics.

desk verdict Solid engineering on a video DiT with a real speed win; the SOTA claim rests on an under-specified evaluation resolution that needs to be fixed before the numbers can be trusted. read the letter →

arxiv 2506.12853 v2 pith:PSLRHO2J submitted 2025-06-15 cs.CV

Jie Liu , Zheng Hui This is my paper
classification cs.CV
keywords videoinpaintingobjectremovaldiffusiontransformerLTX-Videotemporalconsistencyhigh-resolutioncircularpositionshift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video object removal, erasing a person, car, or subtitle and filling the gap with plausible background, becomes much harder when the masked region is large and the camera or subject moves. EraserDiT claims that a Diffusion Transformer (DiT), the architecture family behind modern text-to-video generators, can handle these cases at 1080p-class resolution and do it fast: about 65 seconds for a 97-frame 2K clip on one H800 GPU. The method fine-tunes only the video transformer of the LTX-Video model on roughly 60,000 synthetic mask videos, adds a Circular Position-Shift inference strategy to keep long sequences temporally consistent, and generates text prompts automatically with a vision-language model. The paper reports both a quality gain and a practical speed advantage over current diffusion-based erasers, which take minutes per clip.

What carries the argument

The load-bearing mechanism is the video Diffusion Transformer, instantiated by taking LTX-Video's architecture, 3D full attention over latent tokens plus a 3D causal VAE that compresses video by 32×32×8 in space-time, and fine-tuning only the denoising transformer on the concatenation of masked-video latents, downsampled binary masks, and text embeddings. A second mechanism, the Circular Position-Shift (CPS) strategy, handles sequences longer than the 81-frame training window: it reflect-pads the video, joins the ends into a circular sequence, and slides a fixed-size window across it while shifting the window start by a cumulative offset at each denoising step, so every selected subsequence is a physically coherent clip and no frame boundary is repeatedly reconstructed from the same context. Because CPS nearly doubles inference cost, the authors distill the conditional and unconditional model into a single classifier-free-guidance student to restore the original runtime. The automated prompt generator and the synthetic mask dataset supply the conditioning and supervision that make the transformer learn removal rather than mere regeneration.

What would settle it

Take a fixed set of DAVIS and HQVI videos and their masks, run DiffEraser, MiniMax-Remover, and EraserDiT all at 2160×1200 and again all at 960×528, and recompute PSNR, SSIM, LPIPS, and VFID; if the PSNR advantage over DiffEraser shrinks below roughly 1 dB or the VFID ordering changes, the reported superiority is a resolution artifact instead of a method effect.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a video Diffusion Transformer with 3D full attention, driven by a 3D causal VAE with 32×32×8 spatiotemporal downsampling, can serve as a strong video object remover. The authors report that EraserDiT outperforms DiffEraser and MiniMax-Remover on every metric in Table 1: on HQVI, PSNR rises from 27.41 to 30.78 and VFID drops from 0.0174 to 0.0072; on DAVIS, PSNR rises from 27.63 to 31.10. They also report a speed advantage on the same hardware: 65 seconds for a 2160×1200, 97-frame video, versus 7 minutes 20 seconds for MiniMax-Remover and 73 seconds for DiffEraser at its maximum supported 960×528 resolution. The central mechanism is training only the denoising transformer on masked-video latents concatenated with downsampled masks, while freezing the VAE and text encoder, and then using a Circular Position-Shift sliding window during inference so that frames beyond the training length remain coherent. The paper also contributes a synthetic mask dataset and a VLM-based prompt generator that make the method usable in practice.

Load-bearing premise

The claim of superior quality assumes that the Table 1 numbers were computed at comparable resolutions, but DiffEraser was run at 960×528 while EraserDiT was run at 2160×1200, and the paper does not state the resolution used for each metric row.

Editorial extensions

If this is right

  • Large-mask, high-motion video erasure becomes a one-minute interactive operation at 2K resolution on a single H800 GPU, instead of a multi-minute batch job.
  • Temporal flicker in long completions is reduced by the Circular Position-Shift windowing, making the output usable for footage longer than the training window.
  • The VLM-generated prompt pipeline lets users erase objects after only a bounding-box click, with no manual text description.
  • The DiT recipe transfers the gains of text-to-video foundation models to inpainting, suggesting future video generators can carry an inpainting head with little extra training.
  • On DAVIS and HQVI, the reported PSNR, SSIM, LPIPS, and VFID all improve over DiffEraser and MiniMax-Remover, while per-clip time drops from about 7 minutes to about 1 minute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same masked-latent-conditioning recipe could be lifted onto other DiT video generators: only the denoising transformer is fine-tuned, so the VAE, text encoder, and CPS windowing would transfer with retuning of window length and shift offset.
  • CPS nearly doubles the inference workload, so the distilled student model is doing essential work; for clips much longer than 97 frames or for real-time editing, one could cut denoising steps or apply CPS to selected blocks and measure the flicker trade-off.
  • Because the synthetic masks are dominated by people and animals, the advertised coverage of subtitles and vehicles is an open empirical question; a small labelled evaluation set for those categories would settle it.
  • The reported runtime uses no acceleration method, so combining CPS and CFG distillation with standard step-distillation would likely push 2K erasure below a minute per clip.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EraserDiT, a video object-removal and inpainting method built on the LTX-Video DiT backbone. It introduces a Circular Position-Shift (CPS) inference strategy for long-sequence temporal consistency, an automated VLM-based prompt generation pipeline, and a large synthetic training set of roughly 60,000 mask videos. Experiments compare against DiffEraser and MiniMaxRemover on DAVIS and HQVI, reporting consistent improvements in PSNR, SSIM, LPIPS, and VFID, with a claimed inference time of 65 seconds for a 2160x1200, 97-frame video on one H800 GPU.

Significance. If the reported numbers hold under a fair evaluation protocol, EraserDiT would represent a substantial advance in high-resolution video object removal, both in quality and speed, and the CPS mechanism would be a useful inference-time tool for temporal consistency in long videos. The paper's strengths include a concrete architectural recipe, a large synthetic mask dataset, and an explicit timing benchmark. However, the central empirical claim currently rests on Table 1, whose resolution protocol is under-specified, and the CPS ablation is qualitative only. Because the method itself is defensible and the issues are addressable through additional experiments and clarifications, the work merits a major revision rather than rejection.

major comments (3)
  1. [Performance Evaluation, Quantitative Results (Table 1)] The headline claim of consistent superiority over DiffEraser and MiniMaxRemover rests entirely on Table 1, but the evaluation resolution for the metric rows is not stated. The table note reports timing for a 2160x1200, 97-frame video and states that DiffEraser was tested at 960x528, but it does not state whether PSNR, SSIM, LPIPS, and VFID for each method were computed at native output resolutions, at a common resolution, or after upsampling. Lower-resolution outputs are not directly comparable to higher-resolution outputs under these metrics, and the large reported gaps (e.g., 31.10 vs 27.63 PSNR on DAVIS) could be substantially influenced by resolution mismatch. Please specify the exact resolution at which each metric was computed, and provide an additional comparison with all methods evaluated at a common resolution (or with outputs matched before metric computation).
  2. [Methodology, Inference pipeline (Algorithm 1, Circular Position-Shift)] The CPS strategy is a central contribution, but Algorithm 1 does not specify how the outputs of overlapping sliding windows are combined. When the shift offset alpha is nonzero, consecutive denoising windows overlap, and the pseudocode appears to overwrite previously denoised latent entries (zcurr idx t-1 = ED(...)) without any averaging, blending, or boundary handling. Please clarify the merge rule for overlapping windows and justify that the procedure is well-defined and does not introduce discontinuities. In addition, the ablation for CPS is entirely qualitative: the text cites supplementary videos but provides no quantitative temporal-consistency metric (e.g., warp error, temporal LPIPS, or frame-to-frame consistency). Without such evidence, the claim that CPS 'significantly improves' temporal coherence is not yet supported.
  3. [Performance Evaluation, Quantitative Results] The protocol for generating DAVIS masks is under-specified: the paper says only 'we generate binary masks' without describing the mask source, the number of videos/masks, the mask sizes, or whether identical masks were used for all compared methods. If masks differ across methods or if masks are not held-out from training data, the comparison is not controlled. Please describe the mask-generation procedure and confirm that DiffEraser, MiniMaxRemover, and EraserDiT were evaluated on exactly the same masked inputs.
minor comments (5)
  1. [Abstract] The abstract reports a resolution of 2160 x 2100 for the timing experiment, while Table 1 and the Performance Evaluation section state 2160 x 1200; please correct this inconsistency.
  2. [Training Details, Eq. (1)] The sentence defining Focal Area loss contains a typo: 'the dilated mask pf M input j' should read 'the dilated mask of M input j'.
  3. [Throughout] The baseline name is spelled inconsistently as both 'DiffEraser' and 'DiffuEraser' (the reference list uses 'DiffuEraser'), and similarly 'MiniMax-Remover' and 'MinimaxRemover' appear in different places; please standardize.
  4. [Methodology, Algorithm 1] The pseudocode uses 'right pad(indices, alpha_sigma)' and a 'warp' padding mode without defining either; please specify the padding operation and the meaning of 'warp' so the algorithm is reproducible.
  5. [Conclusion / Reproducibility] The paper does not state whether code, trained checkpoints, or the synthetic mask dataset will be released; given that the method fine-tunes a large foundation model, a public release or at least a detailed training configuration would significantly strengthen reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method's claims come from standard training on external video data and evaluation against standard metrics on DAVIS and HQVI; no prediction is defined in terms of a fitted input.

full rationale

The paper's derivation chain is self-contained relative to its training and evaluation setup. The method finetunes a frozen LTX-Video backbone on synthetic paired data (background videos plus SAM-extracted masks) with an L2 loss and a focal-area loss; these are ordinary training objectives, not quantities defined in terms of the final evaluation metrics. The reported PSNR, SSIM, LPIPS, and VFID values are computed on external benchmarks (DAVIS and HQVI) against their ground-truth frames, so the central performance claim is not equivalent to any fitted parameter or self-referential definition. The Circular Position-Shift inference strategy is a proposed algorithmic modification, and its reported benefit is supported by an ablation comparison, even though that comparison is only described qualitatively for temporal consistency; weak evidence is not circularity. The only notable concern is an underspecified evaluation protocol: Table 1 reports DiffEraser at 960x528 while EraserDiT is timed at 2160x1200, and the paper does not state the resolution at which each method's PSNR, SSIM, LPIPS, and VFID were computed. If the metric resolutions differ, the quantitative advantage could be a resolution artifact. However, that is a correctness or evaluation-protocol risk, not a circularity step: the numbers are not constructed from the inputs or from the method's own equations. There are no self-citations in the sense relevant to circularity, no uniqueness theorem imported from the authors' own prior work, and no ansatz smuggled in via citation that does the load-bearing work. The paper's training masks come from external data and standard tools, and its prompt-generation step is a usability component rather than a device that forces the evaluation outcome. Therefore, the appropriate finding is no significant circularity with a score of 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the quality of a pretrained video generator (LTX-Video), on the transferability of synthetic mask training, and on the correctness of the causal VAE mask intersection and the circular sequence construction. The paper tunes several hyperparameters (alpha, guidance scale, denoising steps, loss weights) without sensitivity analysis. No new physical entities are introduced.

free parameters (4)
  • position-shift offset alpha
    In each denoising timestep the sliding window start is shifted by alpha; the paper does not report how alpha is chosen or whether it is tuned.
  • CFG distillation guidance scale = 3.0
    The student model is conditioned on a fixed guidance scale of 3.0 and trained on it; no sensitivity analysis is provided.
  • Denoising steps = 40
    Inference uses 40 denoising steps; no ablation over step counts for the quality-speed trade-off is reported.
  • Focal area loss coefficient = 1 + Dmask
    The loss weighting by dilated mask is hand-designed; no comparison to unweighted training on the final model is shown.
assumptions (5)
  • domain assumption LTX-Video is a suitable backbone for video inpainting because inpainting requires lower generative capacity than text-to-video.
    Methodology section: "The strong reference nature of video completion tasks means that the generative capability requirement is lower compared to pure text-to-video tasks. Therefore, we chose LTX-Video as our backbone model."
  • domain assumption The 3D VAE encoder's temporal downsampling justifies intersecting every eight contiguous mask frames into one mask frame.
    Methodology section: "The masks M input j of eight temporally contiguous mask frames Mj are intersected to form a single mask frame, except for the first mask frame." If the intersection is wrong, the masked latent conditions on over-masked or under-masked regions.
  • domain assumption Reflecting and concatenating the sequence produces a circular sequence in which any subsequence is physically coherent.
    Algorithm 1 and Figure 4: "In a circular sequence, any selected subsequence is a physically coherent video segment." This assumption fails at reflection points where the temporal direction reverses.
  • domain assumption Synthetic masks pasted onto Pexels background videos generalize to real object-removal scenarios.
    Training Data section: 600k Pexels backgrounds and automatically extracted masks are used; no real paired data is used, so transferability is assumed.
  • domain assumption The automatic VLM prompt accurately describes the scene as if the target object were absent.
    Inference pipeline: a VLM such as Qwen2.5-VL generates prompts; the accuracy of this unverified step is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EraserDiT: Fast Video Inpainting with Diffusion Transformer Model." pith.science (2026). https://pith.science/paper/PSLRHO2J

@misc{pith2026250612853,
  author       = {Pith},
  title        = {Pith review of: EraserDiT: Fast Video Inpainting with Diffusion Transformer Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PSLRHO2J}},
  note         = {Machine review of arXiv:2506.12853}
}
abstract

Video object removal and inpainting are critical tasks in the fields of computer vision and multimedia processing, aimed at restoring missing or corrupted regions in video sequences. Traditional methods predominantly rely on flow-based propagation and spatio-temporal Transformers, but these approaches face limitations in effectively leveraging long-term temporal features and ensuring temporal consistency in the completion results, particularly when dealing with large masks. Consequently, performance on extensive masked areas remains suboptimal. To address these challenges, this paper introduces a novel video inpainting approach leveraging the Diffusion Transformer (DiT). DiT synergistically combines the advantages of diffusion models and transformer architectures to maintain long-term temporal consistency while ensuring high-quality inpainting results. We propose a Circular Position-Shift strategy to further enhance long-term temporal consistency during the inference stage. Additionally, the proposed method interactively removes specified objects, and generates corresponding prompts. In terms of processing speed, it takes only 65 seconds (testing on one NVIDIA H800 GPU) to complete a video with a resolution of $2160 \times 2100$ with 97 frames without any acceleration method. Experimental results indicate that the proposed method demonstrates superior performance in content fidelity, texture restoration, and temporal consistency. Project page:https://jieliu95.github.io/EraserDiT_demo/

Figures

Figures reproduced from arXiv: 2506.12853 by the authors.

Figure 1
Figure 1. The visual results of DiffEraser, MinimaxRe [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The training pipeline of the proposed method for video object removal. In this method, the pipeline requires three [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The inference pipeline of the proposed method for video object removal. As illustrated in the figure, in the process of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The Circular Position-Shift strategy involves re [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: The results on the DAVIS dataset (Perazzi et al. 2016) are presented as follows: the first column shows the input [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The results on the DAVIS (Perazzi et al. 2016) and HQVI (Cho et al. 2025) dataset are presented as follows: the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 3 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  3. [3]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024

    2024

  4. [4]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023

    arXiv 2023

  5. [5]

    Ltx-video: Realtime video latent diffusion

    Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103, 2024

    arXiv 2024

  6. [6]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024

    arXiv 2024

  7. [7]

    Diffueraser: A diffusion model for video inpainting

    Xiaowen Li, Haolan Xue, Peiran Ren, and Liefeng Bo. Diffueraser: A diffusion model for video inpainting. arXiv preprint arXiv:2501.10018, 2025

    arXiv 2025

  8. [8]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    arXiv 2022

Show all 17 references
  1. [9]

    Fuseformer: Fusing fine-grained information in transformers for video inpainting

    Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, and Hongsheng Li. Fuseformer: Fusing fine-grained information in transformers for video inpainting. In Proceedings of the IEEE/CVF international conference on computer vision, pag...

    2021

  2. [10]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, page...

    2016

  3. [11]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140): 0 1--67, 2020

    2020

  4. [12]

    Sam 2: Segment anything in images and videos, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2024

  5. [13]

    Grounded sam: Assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159, 2024

    2024 arXiv

  6. [14]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  7. [15]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  8. [16]

    Learning joint spatial-temporal transformations for video inpainting

    Yanhong Zeng, Jianlong Fu, and Hongyang Chao. Learning joint spatial-temporal transformations for video inpainting. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVI 16, pages 528--543. Springer, 2020

    2020

  9. [17]

    Propainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin CK Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10477--10486, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.