Se\~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists

Bin Liang; Bojia Zi; Kam-Fai Wong; Marco Chen; Penghui Ruan; Rong Xiao; Shaozhe Hao; Shihao Zhao; Xianbiao Qi; Youze Huang

REVIEW 5 major objections 5 minor 5 cited by

Se\~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-million-pair instruction dataset, produced by four specialist editors and filtered with a learned quality classifier, trains a first-frame-guided ControlNet editor that outperforms prior video editing methods on frame consistency…

desk verdict A genuinely new, large-scale real-video editing dataset with solid engineering, but the filtering quality and evaluation are less rigorous than the claims require. read the letter →

arxiv 2502.06734 v3 pith:3SHYXTGI submitted 2025-02-10 cs.CV

Bojia Zi , Penghui Ruan , Marco Chen , Xianbiao Qi , Shaozhe Hao , Shihao Zhao , Youze Huang , Bin Liang

show 2 more authors

Rong Xiao Kam-Fai Wong

This is my paper · ORCID

classification cs.CV

keywords videoeditingdatasetinstruction-basedend-to-enddiffusionmodelsdatafilteringinpaintingControlNettext-videoalignment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Señorita-2M sets out to close the data gap that holds back end-to-end video editing: models trained on (source video, edited video, instruction) triples edit fast, but until now no large set of clean training pairs existed. The paper builds roughly two million such triples by running four specialized editing models over real Internet videos, rewriting edit prompts into natural instructions with a language model, and discarding failures through a learned quality classifier plus CLIP-based thresholds for text alignment and amount of change. A first-frame-guided ControlNet editor trained on this dataset reports lower warp error, higher text alignment, and better temporal consistency than TokenFlow, Flatten, AnyV2V, and InsV2V, and wins 53.17% of user preferences. The ablation shows that Señorita-2M pairs improve scores over InsV2V pairs even when the sample count is held at 60K, so the dataset's quality, not just its size, is the point.

What carries the argument

The load-bearing object is the dataset construction and filtering pipeline rather than any single network. Candidate edits are generated by four specialists built on a common video diffusion base; a language model converts object names and style prompts into clear instructions; and a three-stage filter keeps only pairs that look edited (ensemble MLP classifiers on CLIP frame features, threshold 0.6), match their instruction (CLIP text–video similarity, thresholds 0.2 to 0.22 depending on task), and differ enough from the source (vision-similarity threshold 0.95). On the training side, the architecture that carries the result is a ControlNet-conditioned video diffusion transformer in which the edited first frame is concatenated into the main branch, letting a single edit anchor the whole clip. The remover also relies on a deliberate training trick: 90% of its masks come from unrelated videos, which breaks the usual correlation between mask shape and content and lets classifier-free guidance erase the target object.

What would settle it

Take a random sample of pairs from each of the 18 task categories, run the same quality classifier and CLIP thresholds, and have annotators judge success and failure; if the annotators' failure rate is far above the pipeline's implied rate, or if many pairs judged successful show no visible change, the filtering claim fails. A simpler direct check is to train the identical editor on a 120K pre-filter subset versus a 120K post-filter subset; if metrics do not improve, the filter is not doing the work.

Watch

Extended reading notes

Core claim

The central claim is that specialist-generated, heavily filtered training pairs can make end-to-end instruction-based video editing the strongest option, not merely a fast one. Four expert editors—a global stylizer, a local stylizer, a text-guided inpainter, and a remover—are trained on a common video diffusion base and applied to 388,909 crawled videos, producing candidates across 18 tasks including style transfer, object swap, removal, addition, inpainting, outpainting, grounding, and conditional generation. The candidates pass through a cascade that removes low-quality edits (quality classifier, threshold 0.6), text-misaligned edits (CLIP similarity thresholds 0.2 and 0.22), and near-identical pairs (similarity above 0.95). Training the final editor with a first-frame-guided ControlNet on the surviving pairs yields Ewarp 9.42, CLIPScore 0.2895, temporal consistency 0.9775, and 53.17% user preference against the four baselines, and the equal-sample ablation attributes a clear part of that gain to the dataset itself.

Load-bearing premise

The whole dataset's 'high quality' label rests on a failure detector trained from 5,000 hand-labeled videos and on three similarity thresholds chosen without a separate validation set; if that detector and those thresholds do not transfer to all 18 tasks and all two million pairs, the dataset quality claim is not established.

Editorial extensions

If this is right

An end-to-end editor trained on Señorita-2M inherits the fast single-pass inference of supervised methods while beating inversion-based baselines on frame consistency and text alignment.
Holding training samples at 60K, Señorita-2M still improves CLIPScore and temporal consistency over InsV2V data, so the dataset can benefit researchers who cannot reproduce the four expert generators.
The best architecture found, first-frame-guided ControlNet, points to edited-key-frame conditioning as the productive design for learning video edits from pairs.
The 18 task types, including grounding and conditional generation, make the dataset a plausible starting point for a single multi-task video editor rather than separate per-task models.
Releasing the dataset and trained models, as the paper says it will, would let others reproduce the reported numbers and train editors on new base models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The quality label would benefit from an external audit: randomly sample pairs from all 18 tasks and compare human failure judgments with the quality classifier's decisions, something the paper does not report.
The user-preference figure is reported without participant counts or protocol details in the main text, so it is best read as a directional signal until the study design is available.
A direct way to isolate the filter's contribution would be to train the same editor on random pre-filter pairs versus post-filter pairs at equal size; the metric gap would quantify how much of the gain comes from filtering rather than from the specialist generators.
The remover's unrelated-mask trick is a transferable idea: any inpainting-style system that should erase rather than regenerate can break the mask–content correlation the same way.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Señorita-2M is the first genuinely large instruction-based video editing dataset built from real videos, and that is the main story. Roughly two million pairs across 18 tasks, with four trained specialist models doing the editing and a three-stage filter cleaning the result. If the release matches the description, this becomes a standard training resource for end-to-end video editors, similar to what InstructPix2Pix and UltraEdit did for images. That is the thing to know before reading.

What is actually new: no existing open dataset comes close on scale and task diversity. VIVID-10M is not released and is local-only; InsV2V is synthetic and small. The four expert models—global stylizer, local stylizer, inpainter, remover—are trained on CogVideoX with a fair amount of care, and the appendix gives real implementation detail. The ablation against InsV2V with matched 60K training samples, and the architecture comparison, are the right kind of evidence. The filtering pipeline—quality classifier, CLIP text-alignment, CLIP visual-similarity—is a sensible design.

Now the soft spots. The quality filtering rests on a small validation set: roughly 120 successful and 60 failed videos, with hand-set thresholds for different tasks, and no per-task precision/recall or threshold sensitivity analysis. The paper itself notes that object removal gets no text-alignment filter and that thresholds differ by task; that is honest, but it also means the 'high-quality' label for all 18 tasks is not actually established. The CLIP circularity is real but partial: CLIP is used in curation and in the CLIPScore metric, so the text-alignment gains in Table 2 are partly inherited. The user study helps, but participant and video counts are missing. There are no error bars anywhere, and the DAVIS benchmark with randomly generated prompts is not a standard evaluation. Data and models are not released.

None of this kills the resource. The ablation comparing Señorita-2M against InsV2V on the same 60K budget shows gains in temporal consistency and CLIPScore, and the user preference numbers point the same way. The central claim—a large real-video dataset helps train better end-to-end editors—holds up. It is just weaker than the paper's tone suggests.

Bottom line: this deserves peer review, not desk rejection. A serious referee should ask for the release and for per-task filtering validation, or at least threshold sensitivity, plus basic statistics on the user study. I would read it carefully if I were building video editors and bring it to a reading group, but I would not cite it as a released resource until the data is actually out.

Referee Report

5 major / 5 minor

Summary. The paper introduces Señorita-2M, a dataset of approximately two million instruction-based video editing pairs built from 388,909 real videos crawled from Pexels. The construction pipeline trains four expert models (global stylizer, local stylizer, inpainter, remover) on a WebVid-10M-derived annotation set, uses these experts plus vision tools (SAM2, depth/HED/Canny detectors) to generate edited pairs, and applies a cascade of quality, text-alignment, and visual-similarity filters. The authors also train several end-to-end editing architectures on the dataset and report that their best model outperforms prior methods on Ewarp, CLIPScore, temporal consistency, and user preference (Table 2). An ablation (Table 3) shows that training on Señorita-2M improves over training on InsV2V with the same sample budget.

Significance. If the quality claims are substantiated, this is a useful contribution to the video-editing community: it is among the first large-scale, instruction-based video editing datasets with both local and global edit types, and the expert-model design and filtering pipeline are a serious attempt at curating training data for end-to-end editors. The paper also reports a systematic comparison of editing architectures (Table 4), and the authors commit to open-sourcing the dataset and models, which would enable reproducibility and follow-up work. However, the central claim that Señorita-2M is 'high-quality' is currently supported mainly by the authors' own filtering pipeline, whose validation is thin, and by evaluations that lack statistical rigor. A careful revision that adds per-task filter validation, an independent text-alignment metric, and uncertainty measures would substantially strengthen the contribution.

major comments (5)

[§3.2.4 / Appendix D.1] The quality filter is the load-bearing component for the 'high-quality' claim, but its validation is reported only as 5,000 annotated videos (~1,000 failures) and a validation set of 120 successful and 60 failed samples, with no per-task precision, recall, or retention rates. Because the 18 tasks have very different failure modes (object removal has no text-alignment filter, local stylization compares masked regions, global stylization compares long style prompts), a single ensemble classifier with a fixed 0.6 threshold cannot be assumed to transfer uniformly. The paper should report per-task filter performance (precision/recall on a held-out stratified sample), the number of pairs retained per task, and a threshold sensitivity analysis. Without this, the 'high-quality' label for all 2M pairs is not established.
[§4.3.1 and §D.2] The text-alignment filter in Appendix D.2 uses CLIP text-video similarity to accept edited pairs, and the main evaluation in §4.3.1 uses CLIPScore to measure text-video alignment. Consequently, the reported text-alignment improvement (Table 2, CLIPScore 0.2895 vs. 0.2723) may be partly inherited from the curation criterion rather than reflecting a genuinely better editor. To make the claim convincing, the authors should evaluate text alignment with an independent metric (e.g., a different vision-language model or a human judgment study on the DAVIS outputs) or at least show that the improvement persists when the evaluation prompts are outside the distribution used for filtering.
[Table 2 and §4.3.2] The quantitative comparison reports no error bars, no confidence intervals, and no significance tests. The differences in Ewarp and CLIPScore between methods are small in absolute terms (e.g., CLIPScore 0.2895 vs. 0.2723), and the user study is described only by a single preference percentage (53.17%) with no information about the number of participants, the number of videos rated, or the rating protocol. Without this information, the claim of state-of-the-art performance is not statistically supported. The authors should provide variance estimates (e.g., bootstrap intervals) across DAVIS videos and full details of the user study.
[§3.2.2 and Table 1] The paper claims 18 editing tasks and approximately 2M pairs, but it does not report the number of pairs per task after filtering. Since tasks such as inpainting/outpainting contribute roughly 60,000 pairs (Appendix C.4.4) and conditional generation tasks (depth, HED, etc.) are likely much larger, the distribution is highly uneven. If some tasks have very few retained pairs after filtering, the dataset's usefulness for 'general' video editing is unclear. Reporting per-task counts before and after filtering is necessary to assess coverage.
[§B.4 (Table 7) and §3.2.4] The expert models' quantitative comparisons (Tables 5–8) are computed on datasets that appear to be the same or similar to the expert training/evaluation data, and they are not accompanied by any error bars either. For instance, Table 7 reports that InsV2V has the lowest Ewarp on object swap, which the authors explain as a failure mode (no actual swap), but this interpretation should be validated by a human study or by reporting per-sample statistics. More generally, the lack of uncertainty quantification in all experimental tables makes it difficult to judge whether the reported differences are meaningful.

minor comments (5)

[General] There are numerous typos and formatting errors, e.g., 'blodfaced' in Tables 2 and 8, the accented title 'Se\~norita' appearing inconsistently in the text, and several broken bibliography entries (e.g., '?;' in the first paragraph of the introduction and the Pexels URL formatting).
[§3.2.4 / §D.1] The text says 'Different thresholds were applied for different tasks' in the quality filter, but §4.1 only reports a single threshold of 0.6 and mentions a lower threshold for object addition. The threshold values for each task should be listed explicitly in one place.
[§4.3.1] The evaluation uses 'randomly generated editing prompts' on DAVIS, but it is unclear how these prompts were generated and whether they are appropriate for all compared methods, especially inversion-based methods that may require specific prompt formats. A description of prompt generation and validation is missing.
[Appendix D.2] The text says 'we set thresholds of 0.2 and 0.22, respectively' for object swap and local stylization, but the preceding sentence says 'we set a lower threshold of 0.2' for global stylization and later §4.1 says global stylization and object addition use 0.2. Please clarify the exact thresholds for each task in the main text to avoid confusion.
[§4.2] The training details for the editing model report two stages (first stage at 336×592, second stage at 448×768), but the second-stage data and any potential domain shift are not described. A brief note on the resolution adaptation would help reproducibility.

Circularity Check

1 steps flagged · score 4.0 of 10

CLIP text-alignment filter overlaps with the CLIPScore evaluation metric, but the central dataset claim is independently supported by the controlled ablation and user study.

fitted input called prediction [Sec. 3.2.4/D.2 (text-alignment filter) and Sec. 4.3.1 (CLIPScore evaluation)]
"To address this issue, we utilize CLIP to measure the similarity between edited videos and their corresponding text prompts. / For object swap and local stylization, edits are applied within a masked region, whereas text-video similarity is computed across the entire video, leading to a lower similarity score. To account for this discrepancy, we set thresholds of 0.2 and 0.22, respectively. / We evaluate the stability of the edited videos using Ewarp and Temporal Consistency, while the CLIPScore is used to assess the text-video alignment."

The curation filter keeps exactly the training pairs whose CLIP(text, edited-video) similarity exceeds hand-set thresholds (0.2/0.22), and the evaluation metric CLIPScore is the same CLIP(text, video) cosine similarity. A model trained on pairs pre-selected by this filter is therefore rewarded at evaluation on the same criterion used to curate its training set, so part of the reported text-alignment advantage is inherited from curation rather than independently measured. The circularity is partial because DAVIS evaluation prompts and videos are unseen, and the Ewarp, Temporal Consistency, and user-preference results are external to this filter.

full rationale

No fully circular derivation chain was found. The dataset is constructed by four expert models and then filtered by a trained quality classifier, CLIP text-video similarity, and CLIP original-vs-edited similarity; none of these steps logically entails the final claims by itself. The one concrete overlap is that the text-alignment filter (Sec. D.2) uses CLIP(text, edited video) and the headline text-alignment metric (Sec. 4.3.1) is CLIPScore, i.e., the same function. This creates a partial selection-bias circularity on that single metric. However, the central claim that Señorita-2M trains better editors is independently supported by the controlled ablation in Table 3 (same CogVideoX-5B architecture and matched sample counts for InsV2V vs Señorita-2M), by Ewarp and Temporal Consistency gains, and by a user preference study (53.17% vs 19.40%). Self-citations to CoCoCo and Uni-ControlNet are architectural or negative results and are not load-bearing. The small manual validation set for the quality classifier and the hand-set thresholds are a generalization-risk weakness, not a circularity. The score of 4 reflects partial metric inheritance on CLIPScore rather than a forced derivation of the central claim.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The dataset construction and evaluation rely on several assumptions that are not independently verified. The most important are that CLIP similarity is a valid measure of editing quality, that the DAVIS benchmark generalizes to the dataset's 18 tasks, that the authors' expert models produce reliable ground truth, and that the small quality classifier generalizes to the full 2M pairs. These are domain assumptions or ad hoc choices rather than standard mathematical axioms.

free parameters (5)

Quality filter threshold = 0.6 (lower for object addition)
Samples below this confidence score are dropped; the threshold is chosen by hand without sensitivity analysis (Sec D.1).
Text-alignment CLIP thresholds = 0.2 for global stylization and object addition, 0.22 for object swap and local stylization
Pairs below these CLIP text-video similarities are removed; thresholds vary per task and are not justified (Sec D.2).
Visual similarity threshold = 0.95
Pairs with CLIP similarity between source and edited video above 0.95 are considered unchanged and removed (Sec D.3).
Remover mask ratio = 90% unrelated masks, 10% precise masks
Training data mix for the remover; chosen to break mask-shape correlation (Sec B.5).
Classifier-free guidance scales = Global stylizer 4, local stylizer 6, inpainter 6, remover 2
Inference hyperparameters for expert models; no ablation reported.

assumptions (4)

domain assumption CLIP-based text-video similarity is a valid proxy for instruction alignment and editing quality
Used both to filter the dataset (Sec 3.2.4) and to evaluate the final model (CLIPScore, Sec 4.3.1).
domain assumption The DAVIS dataset with randomly generated editing prompts is a representative benchmark for general video editing
All quantitative comparisons are run on DAVIS (Sec 4.3.1); no evidence that prompts or videos cover the 18 dataset tasks.
ad hoc to paper The authors' four expert models produce edits of sufficient quality to serve as training targets
The dataset targets are generated by these models (Sec 3.2); if experts fail systematically, errors propagate into the dataset.
ad hoc to paper The quality classifier trained on 5,000 annotated videos generalizes to all 2M generated pairs
The classifier is trained on a small sample and applied to the full dataset (Sec D.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Se\~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists." pith.science (2026). https://pith.science/paper/3SHYXTGI

@misc{pith2026250206734,
  author       = {Pith},
  title        = {Pith review of: Se\~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3SHYXTGI}},
  note         = {Machine review of arXiv:2502.06734}
}

read the original abstract

Recent advancements in video generation have spurred the development of video editing techniques, which can be divided into inversion-based and end-to-end methods. However, current video editing methods still suffer from several challenges. Inversion-based methods, though training-free and flexible, are time-consuming during inference, struggle with fine-grained editing instructions, and produce artifacts and jitter. On the other hand, end-to-end methods, which rely on edited video pairs for training, offer faster inference speeds but often produce poor editing results due to a lack of high-quality training video pairs. In this paper, to close the gap in end-to-end methods, we introduce Se\~norita-2M, a high-quality video editing dataset. Se\~norita-2M consists of approximately 2 millions of video editing pairs. It is built by crafting four high-quality, specialized video editing models, each crafted and trained by our team to achieve state-of-the-art editing results. We also propose a filtering pipeline to eliminate poorly edited video pairs. Furthermore, we explore common video editing architectures to identify the most effective structure based on current pre-trained generative model. Extensive experiments show that our dataset can help to yield remarkably high-quality video editing results. More details are available at https://senorita-2m-dataset.github.io.

Figures

Figures reproduced from arXiv: 2502.06734 by the authors.

**Figure 1.** The visual results given by editing models trained on our Señorita-2M. Best viewed with Acrobat Reader. Click the images to play the animation clips. ods. Omni-Edit (Wei et al., 2024) improved UltraEdit by employing more experts to generate higher-quality datasets, resulting in better-performing models. The field of video editing differs significantly from image editing, with most video editing techniques being inve… view at source ↗

**Figure 2.** Top: The data construction pipeline of the Señorita-2M dataset. Bottom: The filtering pipeline of Señorita-2M. Further details are provided in the Appendix C.3 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Visualization of our Señorita-2M. Best viewed with Acrobat Reader. Click the images to play the animation clips. outpainting. More details can be found in the Appendix C.4. Object Swap. Object swap uses FLUX-Fill and our trained inpainter. To begin with, the LLaMA-3 (Dubey et al., 2024) suggests a replacement object, which is then swapped in the first frame by FLUX-Fill. The inpainter generates the remaining frames … view at source ↗

Figures from the paper (12 more)

**Figure 4.** Figure 4: Overview of Señorita-2M dataset: a statistical analysis. and hed, and processed it through our global stylizer to generate the rest frames. Finally, an LLM was used to convert the style prompts into actionable instructions for further content optimization. Object Groun…

**Figure 5.** Figure 5: Editing results compared between different editing methods. B. The Design of Video Editing Experts B.1. The Construction of Expert Training Dataset As shown in [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

**Figure 6.** Figure 6: The construction pipeline of annotated training dataset for experts training. commas and used as input prompts for Grounded-SAM2 (Liu et al., 2023a; Ravi et al., 2024). This process generates phrase names and corresponding object mask sequences within the video. These …

**Figure 7.** Figure 7: Top: The training pipeline of our global stylizer. Bottom: The data construction pipeline for Señorita-2M using our global stylizer. features from the control branch (ControlNet) into the main branch (base model) through zero convolution. The control branch consists of…

**Figure 8.** Figure 8: The visual results of our global stylization. The video on the left depicts the original video, while the video on the right displays the edited videos. Best viewed with Acrobat Reader. Click the images to play the animation clips. During inference, we append the requi…

**Figure 9.** Figure 9: Top: The training pipeline of our local stylizer. Bottom: The data construction pipeline for Señorita-2M using our local stylizer [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

**Figure 10.** Figure 10: The visual results of our local stylizer. The video on the left depicts the original video, while the video on the right displays the edited videos. Best viewed with Acrobat Reader. Click the images to play the animation clips. outperforms all baselines with the highe…

**Figure 11.** Figure 11: Top: The training pipeline of our inpainter. Bottom: The data construction pipeline for Señorita-2M using our inpainter. Although many studies have explored text-guided video inpainting, such as AVID (Zhang et al., 2023b) and COCOCO (Zhang et al., 2023b), most of thes…

**Figure 12.** Figure 12: The visual results of our inpainter. The video on the left depicts the original video, while the video on the right displays the edited videos. Best viewed with Acrobat Reader. Click the images to play the animation clips. initialized with the parameters of CogVideoX-…

**Figure 13.** Figure 13: The framework of our remover and sub-dataset construction pipeline. B.5. The Construction of Remover Traditional video inpainter, such as Propainter (Zhou et al., 2023), uses optical flow to guide the completion. However, these methods show weaker performance than dif…

**Figure 14.** Figure 14: The visual results of our object remover. The video on the left depicts the original video, while the video on the right displays the edited videos. Best viewed with Acrobat Reader. Click the images to play the animation clips. removal, since it has a high percentage …

**Figure 15.** Figure 15: Editing results compared between different editing methods. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

CoT-Edit: Let CoT Guide Instruction Video Editing
cs.CV 2026-08 conditional novelty 6.0 of 10

CoT-Edit achieves state-of-the-art instruction-based video editing by generating bounding boxes and enriched instructions with a CoT-enhanced multimodal planner, which guide mask-based diffusion editing.
ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing
cs.CV 2026-07 conditional novelty 6.0 of 10

Test-time tuning of video diffusion models collapses generation toward the source video; ElasticTTT counters this with noisy targets, contrastive source-prompt guidance, and asynchronous region-wise noise scheduling, ...
Under One Sun: Multi-Object Generative Perception of Materials and Illumination
cs.CV 2026-03 conditional novelty 6.0 of 10

Factorizing video editing into semantic-token anchoring and motion-restoration pre-training produces strong zero-shot and SOTA open-source instruction-guided video edits without heavy external structural priors.
O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing
cs.CV 2025-09 conditional novelty 6.0 of 10

A video editor trained on randomly distorted objects, then steered by adaptive noise at inference, is claimed to surpass dedicated and unified editors across eight tasks with far less training.
MiniMax-Remover: Taming Bad Noise Helps Video Object Removal
cs.CV 2025-05 conditional novelty 6.0 of 10

A two-stage video object remover that removes text conditioning and uses minimax adversarial noise to achieve high-quality removal in 6 sampling steps without classifier-free guidance.

Reference graph

Works this paper leans on

35 extracted references · 13 canonical work pages · cited by 5 Pith papers

[3]

H., Chechik, G., and Cohen-Or, D

Gal, R., Alaluf, Y ., Atzmon, Y ., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618,

arXiv
[4]

Tokenflow: Consistent diffusion features for consistent video editing

Geyer, M., Bar-Tal, O., Bagon, S., and Dekel, T. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373,

arXiv
[5]

Sparsectrl: Adding sparse controls to text-to-video diffusion models, 2023a

Guo, Y ., Yang, C., Rao, A., Agrawala, M., Lin, D., and Dai, B. Sparsectrl: Adding sparse controls to text-to-video diffusion models, 2023a. Guo, Y ., Yang, C., Rao, A., Wang, Y ., Qiao, Y ., Lin, D., and Dai, B. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023b. Hertz, A...

arXiv
[6]

commas and used as input prompts for Grounded-SAM2 (Liu et al., 2023a; Ravi et al., 2024)

The construction pipeline of annotated training dataset for experts training. commas and used as input prompts for Grounded-SAM2 (Liu et al., 2023a; Ravi et al., 2024). This process generates phrase names and corresponding object mask sequences within the video. These annotations are used to train the inpainter, remover, and local stylizer models. Moreove...

work page 2024
[7]

Vivid-10m: A dataset and baseline for versatile and interactive video local editing

Hu, J., Zhong, T., Wang, X., Jiang, B., Tian, X., Yang, F., Wan, P., and Zhang, D. Vivid-10m: A dataset and baseline for versatile and interactive video local editing. arXiv preprint arXiv:2411.15260,

arXiv
[8]

Hq-edit: A high-quality dataset for instruction-based image editing.arXiv preprint arXiv:2404.09990,

Hui, M., Yang, S., Zhao, B., Shi, Y ., Wang, H., Wang, P., Zhou, Y ., and Xie, C. Hq-edit: A high-quality dataset for instruction-based image editing.arXiv preprint arXiv:2404.09990,

arXiv
[10]

Video diffusion models are strong video inpainter

Lee, M., Cho, S., Shin, C., Lee, J., Yang, S., and Lee, S. Video diffusion models are strong video inpainter. arXiv preprint arXiv:2408.11402,

arXiv
[11]

Stablev2v: Stablizing shape consistency in video-to-video editing

Liu, C., Li, R., Zhang, K., Lan, Y ., and Liu, D. Stablev2v: Stablizing shape consistency in video-to-video editing. arXiv preprint arXiv:2411.11045, 2024a. Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al. Grounding dino: Marry- ing dino with grounded pre-training for open-set object detection. arXiv prepr...

arXiv

Show all 35 references

[12]

The video on the left depicts the original video, while the video on the right displays the edited videos

The visual results of our inpainter. The video on the left depicts the original video, while the video on the right displays the edited videos. Best viewed with Acrobat Reader. Click the images to play the animation clips. initialized with the parameters of CogVideoX-5B-I2V . ...

2017
[13]

Sdedit: Guided image synthesis and edit- ing with stochastic differential equations

Meng, C., He, Y ., Song, Y ., Song, J., Wu, J., Zhu, J.-Y ., and Ermon, S. Sdedit: Guided image synthesis and edit- ing with stochastic differential equations. arXiv preprint arXiv:2108.01073,

arXiv
[14]

Revideo: Remake a video with motion and content control

Mou, C., Cao, M., Wang, X., Zhang, Z., Shan, Y ., and Zhang, J. Revideo: Remake a video with motion and content control. arXiv preprint arXiv:2405.13865,

arXiv
[15]

Zero-shot image-to-image translation

Parmar, G., Kumar Singh, K., Zhang, R., Li, Y ., Lu, J., and Zhu, J.-Y . Zero-shot image-to-image translation. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1– 11,

2023
[16]

The 2017 davis chal- lenge on video object segmentation

Pont-Tuset, J., Perazzi, F., Caelles, S., Arbeláez, P., Sorkine- Hornung, A., and Van Gool, L. The 2017 davis chal- lenge on video object segmentation. arXiv preprint arXiv:1704.00675,

2017 arXiv
[17]

T., and Mildenhall, B

Poole, B., Jain, A., Barron, J. T., and Mildenhall, B. Dream- fusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988,

arXiv
[18]

Fatezero: Fusing attentions for zero-shot text- based video editing

Qi, C., Cun, X., Zhang, Y ., Lei, C., Wang, X., Shan, Y ., and Chen, Q. Fatezero: Fusing attentions for zero-shot text- based video editing. arXiv preprint arXiv:2303.09535,

arXiv
[21]

Ronneberger, O., Fischer, P., and Brox, T

URL https:// arxiv.org/abs/2408.00714. Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Confer- ence, Munich, Germany, Oct...

2015 arXiv
[22]

Plug- and-play diffusion features for text-driven image-to-image translation

Tumanyan, N., Geyer, M., Bagon, S., and Dekel, T. Plug- and-play diffusion features for text-driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1921– 1930,

1921
[23]

Omniedit: Building image editing general- ist models through specialist supervision

Wei, C., Xiong, Z., Ren, W., Du, X., Zhang, G., and Chen, W. Omniedit: Building image editing general- ist models through specialist supervision. arXiv preprint arXiv:2411.07199,

arXiv
[24]

Zhang, K., Mo, L., Chen, W., Sun, H., and Su, Y

URL https://arxiv.org/abs/ 2405.18406. Zhang, K., Mo, L., Chen, W., Sun, H., and Su, Y . Mag- icbrush: A manually annotated dataset for instruction- guided image editing. Advances in Neural Information Processing Systems, 36, 2024a. Zhang, L., Rao, A., and Agrawala, M. Adding ...
[25]

Cococo: Improving text- guided video inpainting for better consistency, controlla- bility and compatibility.arXiv preprint arXiv:2403.12035,

Zi, B., Zhao, S., Qi, X., Wang, J., Shi, Y ., Chen, Q., Liang, B., Wong, K.-F., and Zhang, L. Cococo: Improving text- guided video inpainting for better consistency, controlla- bility and compatibility.arXiv preprint arXiv:2403.12035,

arXiv
[27]

This limitation prevents us from applying techniques such as ControlNet to repaint a video effectively

and HunyuanVideo (Kong et al., 2025), lack sufficient ability to generate videos that accurately follow style information. This limitation prevents us from applying techniques such as ControlNet to repaint a video effectively. To address this, we shift our focus to image-based...

2025
[28]

Table 5 shows that our expert model outperforms all baselines, achieving the lowest Ewarp (9.02), highest CLIPScore (0.3145), and best Temporal Consistency (0.9781)

The model processes a video within 2 minutes on an Nvidia RTX 4090, at a resolution of 336×592, producing 33 frames. Table 5 shows that our expert model outperforms all baselines, achieving the lowest Ewarp (9.02), highest CLIPScore (0.3145), and best Temporal Consistency (0.9...

2024
[29]

The best results are boldfaced

Quantitative Comparison on Local Stylization. The best results are boldfaced. Methods Ewarp(10−3) (↓) CLIPScore ( ↑) Temp-Cons ( ↑) PSNR ( ↑) SSIM ( ↑) LPIPS ( ↓) MSE ( ↓) Tokenflow 16.60 0.2876 0.9810 18.79 0.8555 0.1483 987.90 Flatten 17.18 0.2923 0.9751 18.64 0.8605 0.1463 ...

2017
[30]

Bottom: The data construction pipeline for Señorita-2M using our inpainter

Top: The training pipeline of our inpainter. Bottom: The data construction pipeline for Señorita-2M using our inpainter. Although many studies have explored text-guided video inpainting, such as A VID (Zhang et al., 2023b) and COCOCO (Zhang et al., 2023b), most of these method...

2024
[32]

The framework of our remover and sub-dataset construction pipeline. B.5. The Construction of Remover Traditional video inpainter, such as Propainter (Zhou et al., 2023), uses optical flow to guide the completion. However, these methods show weaker performance than diffusion mo...

2023
[33]

For object recognition, we utilize CogVLM-video-llama3-chat (Hong et al.,

to generate video captions while adhering to the length restrictions of CLIP (Ramesh et al., 2021). For object recognition, we utilize CogVLM-video-llama3-chat (Hong et al.,

2021
[34]

We set the maximum token length to 120 and use six frames per video

with INT8 precision for efficient inference on Nvidia RTX 4090 GPUs. We set the maximum token length to 120 and use six frames per video. The videos and detected object names are then fed into Grounded-SAM2. Specifically, we employ the SWinB_CogCoor model from Grounding-DINO (...

2024
[35]

Detect" or

to generate instructions, by giving some examples. We use the original videos as the source videos, while the edited videos as the target videos, along with the instructions to build a (source, target, instruction) video editing triplet. C.3.2. O BJECT GROUNDING . We provide v...

1984
[592]

There be

For the object removal, the original videos are used as source videos, while the edited videos are used as the target videos. For the object addition, the edited videos are used as source videos, while the original videos are used as the target videos. We use LLM to generate i...

1984
[2017]

Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models

Lu, C., Zhou, Y ., Bao, F., Chen, J., Li, C., and Zhu, J. Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095,

arXiv
[2021]

Hierarchical text-conditional image generation with clip latents

Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3,

arXiv
[2022]

Cogvlm2: Vi- sual language models for image and video understanding

Hong, W., Wang, W., Ding, M., Yu, W., Lv, Q., Wang, Y ., Cheng, Y ., Huang, S., Ji, J., Xue, Z., et al. Cogvlm2: Vi- sual language models for image and video understanding. arXiv preprint arXiv:2408.16500,

arXiv
[2023]

Videocrafter2: Overcoming data limitations for high-quality video diffusion models

Chen, H., Zhang, Y ., Cun, X., Xia, M., Wang, X., Weng, C., and Shan, Y . Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024a. Chen, H., Zhang, Y ., Cun,...

arXiv
[2024]

Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al

URL https://openreview.net/forum? id=IoKRezZMxF. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

arXiv
[2025]

Ku, M., Wei, C., Ren, W., Yang, H., and Chen, W

URL https://arxiv.org/abs/2412.03603. Ku, M., Wei, C., Ren, W., Yang, H., and Chen, W. Anyv2v: A plug-and-play framework for any video-to-video edit- ing tasks. arXiv preprint arXiv:2403.14468,

arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [3]

H., Chechik, G., and Cohen-Or, D

Gal, R., Alaluf, Y ., Atzmon, Y ., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618,

arXiv

[2] [4]

Tokenflow: Consistent diffusion features for consistent video editing

Geyer, M., Bar-Tal, O., Bagon, S., and Dekel, T. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373,

arXiv

[3] [5]

Sparsectrl: Adding sparse controls to text-to-video diffusion models, 2023a

Guo, Y ., Yang, C., Rao, A., Agrawala, M., Lin, D., and Dai, B. Sparsectrl: Adding sparse controls to text-to-video diffusion models, 2023a. Guo, Y ., Yang, C., Rao, A., Wang, Y ., Qiao, Y ., Lin, D., and Dai, B. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023b. Hertz, A...

arXiv

[4] [6]

commas and used as input prompts for Grounded-SAM2 (Liu et al., 2023a; Ravi et al., 2024)

The construction pipeline of annotated training dataset for experts training. commas and used as input prompts for Grounded-SAM2 (Liu et al., 2023a; Ravi et al., 2024). This process generates phrase names and corresponding object mask sequences within the video. These annotations are used to train the inpainter, remover, and local stylizer models. Moreove...

work page 2024

[5] [7]

Vivid-10m: A dataset and baseline for versatile and interactive video local editing

Hu, J., Zhong, T., Wang, X., Jiang, B., Tian, X., Yang, F., Wan, P., and Zhang, D. Vivid-10m: A dataset and baseline for versatile and interactive video local editing. arXiv preprint arXiv:2411.15260,

arXiv

[6] [8]

Hq-edit: A high-quality dataset for instruction-based image editing.arXiv preprint arXiv:2404.09990,

Hui, M., Yang, S., Zhao, B., Shi, Y ., Wang, H., Wang, P., Zhou, Y ., and Xie, C. Hq-edit: A high-quality dataset for instruction-based image editing.arXiv preprint arXiv:2404.09990,

arXiv

[7] [10]

Video diffusion models are strong video inpainter

Lee, M., Cho, S., Shin, C., Lee, J., Yang, S., and Lee, S. Video diffusion models are strong video inpainter. arXiv preprint arXiv:2408.11402,

arXiv

[8] [11]

Stablev2v: Stablizing shape consistency in video-to-video editing

Liu, C., Li, R., Zhang, K., Lan, Y ., and Liu, D. Stablev2v: Stablizing shape consistency in video-to-video editing. arXiv preprint arXiv:2411.11045, 2024a. Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al. Grounding dino: Marry- ing dino with grounded pre-training for open-set object detection. arXiv prepr...

arXiv