Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that subject-to-video generation needs its own evaluation and training infrastructure, and provides a 180-prompt benchmark with three human-aligned metrics plus a million-scale dataset whose synthetic and cross-video…

desk verdict Useful benchmark and dataset for S2V evaluation, but the Nexus Data claim is undermined by an internal contradiction and a missing table, so treat that part as unproven until revised. read the letter →

arxiv 2505.20292 v4 pith:XUI2Z6Z3 submitted 2025-05-26 cs.CV cs.AI

Shenghai Yuan , Xianyi He , Yufan Deng , Yang Ye , Jinfa Huang , Bin Lin , Jiebo Luo , Li Yuan This is my paper
classification cs.CVcs.AI
keywords subject-to-videogenerationS2Vbenchmarkmillion-scalevideodatasetNexusScoreNaturalGmeidentitypreservationData
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to give subject-to-video generation the same kind of infrastructure that text-to-video already has: a purpose-built benchmark and a large training set. It claims OpenS2V-Eval is the first comprehensive S2V benchmark, adding subject naturalness to the usual consistency and text-relevance axes, with three automatic metrics that align with human preference. It also claims OpenS2V-5M is the first open-source million-scale S2V dataset, whose Nexus Data exposes models to subject views different from the training frame to discourage copying. If these claims hold, S2V models can be selected and trained by criteria that match what humans notice, rather than by global text-to-video scores.

What carries the argument

The load-bearing machinery is Nexus Data, a class of training pairs that decouple the reference image from the exact frame viewpoint. GPT-Frame Pairs use a commercial image-generation model to synthesize a complete subject image from the video's first frame plus keywords, yielding a novel view with consistent identity. Cross-Frame Pairs use a multimodal retrieval model's similarity score to link the same subject appearing in different clips split from one long video. On the benchmark side, NexusScore, NaturalScore, and GmeScore are the quantitative instruments that detect whether a model actually preserves the subject, appears physically natural, and follows the prompt.

What would settle it

Human audit of a random sample of Nexus Data pairs: if the two images in a GPT-Frame or Cross-Frame pair do not show the same identity, the label-noise assumption fails. For the metrics, check whether a deliberately copy-pasted video (the reference image pasted into every frame) gets a high NexusScore and a low NaturalScore; the split is only meaningful if those two scores diverge.

Watch

Extended reading notes

Core claim

The paper's discovery is that subject-to-video evaluation should separate consistency from naturalness, and that both can be quantified automatically. NexusScore computes similarity between a cropped reference image and the detected subject region in each video frame, avoiding background noise; NaturalScore averages a five-point realism judgment from a large vision-language model over sampled frames; GmeScore measures text alignment with a multimodal retrieval model that handles long prompts. Applied to 180 prompts in seven categories across 18 models, the metrics reveal that closed-source models lead in fidelity while open-source models show copy-paste and generalization failures. On the data side, the paper constructs subject-text-video triples from segmented video frames and supplements them with Nexus Data—pairs connecting the same subject across clips of a long video, and pairs synthesizing complete subject images from first frames—to break the shortcut where models simply copy the reference view. Fine-tuning a text-to-video model on 300k samples from OpenS2V-5M converts it into an S2V model, with larger gains when Nexus Data is included.

Load-bearing premise

The construction of Nexus Data assumes both that the commercial image generator preserves identity when synthesizing a new view and that the retrieval-model similarity between clips from one long video identifies the same subject; the paper reports no identity-preservation check or manual verification for these pairs.

Editorial extensions

If this is right

  • OpenS2V-Eval gives the field a common six-axis report card—consistency, naturalness, text relevance, face similarity, aesthetics, and motion—so model comparisons stop being cherry-picked.
  • A model that scores high on consistency but low on naturalness is now identifiable as copy-paste behavior, exactly the failure existing benchmarks reward.
  • Training on OpenS2V-5M, including Nexus Data, converts a text-to-video model into a subject-to-video model even with a partial training run.
  • The 5.1M regular triples plus 0.35M Nexus pairs give open-source S2V training data at a scale comparable to mainstream video datasets.
  • Because GmeScore handles long prompts, the benchmark suits current DiT-based video models that prefer detailed captions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As an editorial extension: if the cross-video association is reliable, the same clustering could produce paired data for tracking a subject through an entire long video, enabling narrative-length consistency rather than clip-level consistency.
  • A testable extension of the benchmark is to apply NexusScore and NaturalScore to edited or artistically stylized videos, where 'naturalness' should be interpreted relative to style; the current metrics may need a style-conditional variant.
  • If the data-construction assumption fails, the release of video metadata lets anyone audit identity consistency in Nexus Data without redistributing raw video, which is a practical way to measure label noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces OpenS2V-Nexus, a combined benchmark and dataset for subject-to-video (S2V) generation. OpenS2V-Eval contains 180 prompts in seven categories, together with three new automatic metrics: NexusScore (subject consistency), NaturalScore (subject naturalness), and GmeScore (text relevance). The authors evaluate 18 S2V models on this benchmark and report a human-preference study. OpenS2V-5M is a proposed 5.4M-clip dataset with 5.1M regular subject-text-video triples and 0.35M 'Nexus Data' built from GPT-Image-1 multi-view synthesis and cross-video associations. A training experiment with a ConsisID-style model is used to argue that the dataset, and especially Nexus Data, converts a T2V model into an S2V model and addresses three core S2V challenges: poor generalization, copy-paste, and inadequate human fidelity.

Significance. If the claims hold, the work addresses a genuine gap: existing S2V benchmarks are mostly coarse adaptations of VBench, and no large-scale open S2V dataset exists. The proposed metrics separately target subject consistency, naturalness, and text relevance, which is a useful decomposition. The paper also provides a substantial resource (benchmark, dataset, code, evaluation of 18 models) that could accelerate S2V research. However, the central dataset claim currently rests on an incomplete and internally inconsistent experiment, and the identity-preservation premise of Nexus Data is asserted without measurement. The approximately 75% human correlation of the proposed metrics, acknowledged in Appendix E.1, further limits how strongly the thresholds in the data pipeline can be validated. These issues need correction before the resource can be fully relied upon.

major comments (4)
  1. [Section 5.2, Table 4, Figure 9(b)] The claim that Nexus Data 'plays a crucial role' is not supported by the reported quantitative results. Table 4 shows Ours‡ (which Figure 9 identifies as the model without Nexus Data) achieving a higher Total Score than Ours† (with Nexus Data): 52.97% versus 51.67%, with Ours‡ also higher on every reported sub-score. If the labels are as stated, this directly contradicts the main-text claim; if the labels are reversed, then the figure caption and the table must be corrected consistently. In addition, the text references 'Table 7' for results, but no Table 7 exists in the manuscript. Please present the complete results for the with-Nexus and without-Nexus comparison in a single, consistent, and clearly labeled table, and reconcile the numbers with the qualitative statement.
  2. [Section 4.1, Eq. (5), Eq. (6)] The Nexus Data construction assumes that GPT-Image-1 preserves subject identity when generating Igen and that the GME similarity threshold in Eq. (6) correctly identifies the same subject across different clips of a long video. No quantitative identity-preservation check is reported for GPT-Frame Pairs, and no precision or error analysis is given for Cross-Frame Pairs. Appendix C.3 offers only qualitative examples. If a nontrivial fraction of these pairs contain different subjects or hallucinated attributes, the 0.35M Nexus samples would inject label noise exactly in the direction the dataset is meant to fix. Please add a quantitative validation on a random subset, e.g., human-verified identity matching rates for both pair types, with the chosen thresholds and estimated error bars.
  3. [Section 3.3, Eq. (3), Section 4.1, Eq. (6)] There is a circularity risk: the same retrieval model Mretrieve (GME) is used to construct Cross-Frame Pairs (Eq. 6), to compute NexusScore (Eq. 3), and to compute GmeScore. Consequently, improvements measured on models trained with Nexus Data may partly reflect alignment with the construction model's representation rather than a general gain in subject-to-video generation. This is a correctness-risk concern, not a claim of intentional bias. Please report how the proposed metrics correlate with human preference separately for models trained with and without Nexus Data, and discuss whether the ~75% correlation acknowledged in Appendix E.1 is sufficient to validate the thresholds used in Eq. (6).
  4. [Eq. (3) and Appendix D.4] NexusScore averages over T', the number of frames in which an object is detected, rather than the total number of frames T. As written, a video in which only one frame passes the detection and similarity thresholds can receive a perfect score, making the metric vulnerable to gaming and partially explaining why high NexusScores can coexist with the 'consistency fade' phenomenon the authors themselves document in Appendix B.4. Please report the distribution of T'/T for the evaluated models and consider incorporating a penalty or a minimum coverage requirement.
minor comments (6)
  1. [Appendix B.2] In the comparison of naturalness metrics, the text says 'None of these methods perform as effectively as the NexusScore we propose, which is based on GPT-4o'; this should refer to NaturalScore, not NexusScore.
  2. [Section 4.2 and Appendix C.2] The count '0.35M Nexus Data' is ambiguous: it could mean 0.35M pairs or 0.35M clustering centers. Please clarify the exact number of Cross-Frame and GPT-Frame pairs released.
  3. [Table 4] The Human-Domain evaluation table omits NexusScore, even though the benchmark defines it as a primary subject-consistency metric and Figure 9(a) validates it. Please indicate why NexusScore is not reported for this task, or include it.
  4. [Section 3.1] The description of test-pair collection says 50 and 24 subject-text pairs are taken from ConsisID and A2 Bench, and additional samples come from DreamBench and web videos; the arithmetic leading to the total of 180 samples across the seven categories is not stated. A breakdown per category would clarify the composition.
  5. [References] The reference list contains apparent duplicates: [100] and [101] are the same paper ('Towards a better metric for text-to-video generation'). Please merge and renumber.
  6. [Title and Abstract] The manuscript title uses 'OpenS2V-Nexus' while the abstract and body use 'OPEN S2V-N EXUS' in some places; unify the formatting.

Circularity Check

1 steps flagged · score 5.0 of 10

Shared GME retrieval model defines both Nexus Data identity labels and the benchmark's subject-consistency metric, making the dataset validation partly self-referential.

  1. self definitional [Section 3.3 (NexusScore, Eq. 3); Section 4.1 (Cross-Frame Pairs, Eq. 6)]
    "Finally, the similarity between Ci,t and Ri is evaluated in the image feature space, yielding: SNexus = 1/(I × T′) Σ_i Σ_t Mretrieve(Ci,t, Ri), where ci,t > α and si,t > β (3) ... The similarity between subjects across these clips is computed using a multimodal retrieval model [119] Mretrieval ... S(Cij, Ckl) = sim(Mretrieval(Cij), Mretrieval(Ckl)) (6)"

    The same multimodal retrieval embedding (GME, [119]) is used both to define which different video clips contain the 'same subject' when constructing Cross-Frame Pairs (Eq. 6) and to compute the subject-consistency metric NexusScore (Eq. 3). Thus the claim that training on Nexus Data improves subject consistency is supported by a metric whose notion of identity was used to generate the very labels being tested: high-scoring pairs are, by construction, pairs that GME considers similar. No human verification of the cross-video associations is reported (Appendix C.3 gives only qualitative examples), and Appendix E.1 concedes that the metrics show only approximately 75% correlation with human preferences, so the shared-embedding bias is not fully broken.

full rationale

The paper's main benchmark contribution is largely self-contained: OpenS2V-Eval is assembled from externally sourced images and prompts, and the three proposed metrics are validated against a human preference study (60 videos, 173 participants), with the acknowledged ~75% correlation in Appendix E.1. The dataset contribution, however, contains a partially self-referential step: same-subject labels for Cross-Frame Pairs are generated by thresholding GME similarity (Eq. 6), and the same GME model is then used as NexusScore (Eq. 3) to claim that the dataset improves subject consistency. Because the identity labels and the evaluation metric share the same embedding, the dataset validation cannot fully separate genuine identity preservation from optimization of GME-similarity; the missing Table 7, which would quantify the 'crucial role' of Nexus Data, further leaves the central dataset claim uncheckable. No load-bearing self-citation chain or uniqueness argument appears, and the GPT-Frame Pairs' identity-preservation assumption is an unverified premise rather than a circular reduction. Overall, the benchmark is independent enough to avoid a high score, but the dataset's central validation is partially circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several unverified assumptions about closed models (GPT-4o, GPT-Image-1, GME) and on hand-chosen thresholds and weights. No new physical entities are posited. The free parameters and assumptions are mostly domain assumptions about the reliability of the tools used, not fitting to the target result, but they do affect the scores and rankings.

free parameters (5)
  • NexusScore thresholds alpha and beta = Not specified
    Eq. 3 uses predefined thresholds alpha and beta to filter detection confidence and entity similarity; values are not given and affect the score.
  • Total Score weights = Nexus 0.20, Natural 0.24, Gme 0.12, FaceSim 0.20, Aesthetic 0.12, Motion 0.12 (open-domain); different for human-domain
    Weights in Eq. 7 are chosen by hand and change the model ranking.
  • Normalization ranges for NexusScore, AestheticScore, MotionScore = [0,0.05], [0,1], [4,7] respectively
    Ranges are derived from empirical distributions of the generated videos, not from theory, which can favor models that concentrate in the observed range.
  • Frame sampling count T=32 = 32
    All metrics except MotionScore sample 32 frames; changing this may alter scores.
  • NaturalScore GPT-4o run count = 3
    NaturalScore averages three GPT-4o runs; the choice is not justified beyond robustness.
assumptions (5)
  • domain assumption GPT-4o is a reliable proxy for human judgment of naturalness and physical plausibility in generated videos.
    NaturalScore relies entirely on GPT-4o scores with a five-point rubric; the paper shows limited evidence comparing GPT-4o to humans (about 75% correlation).
  • domain assumption GME multimodal retrieval similarity is a valid measure of subject identity and text relevance.
    GmeScore and cross-video pairing in Eq. 6 use GME as the ground truth for identity and relevance without independent cross-checks.
  • domain assumption YOLO-World detections correctly locate all target subjects in every video frame.
    NexusScore uses YOLO-World bboxes as a gate; missed or erroneous detections silently drop frames from the score.
  • domain assumption The 60-video, 173-participant study is representative enough to validate six metrics.
    No power analysis or confidence intervals are reported, and the appendix notes only about 75% correlation.
  • domain assumption GPT-Image-1 preserves subject identity while generating novel views.
    GPT-Frame Pairs are created by prompting GPT-Image-1; no identity-preservation check is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation." pith.science (2026). https://pith.science/paper/XUI2Z6Z3

@misc{pith2026250520292,
  author       = {Pith},
  title        = {Pith review of: OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XUI2Z6Z3}},
  note         = {Machine review of arXiv:2505.20292}
}
read the original abstract

Subject-to-Video (S2V) generation aims to create videos that faithfully incorporate reference content, providing enhanced flexibility in the production of videos. To establish the infrastructure for S2V generation, we propose OpenS2V-Nexus, consisting of (i) OpenS2V-Eval, a fine-grained benchmark, and (ii) OpenS2V-5M, a million-scale dataset. In contrast to existing S2V benchmarks inherited from VBench that focus on global and coarse-grained assessment of generated videos, OpenS2V-Eval focuses on the model's ability to generate subject-consistent videos with natural subject appearance and identity fidelity. For these purposes, OpenS2V-Eval introduces 180 prompts from seven major categories of S2V, which incorporate both real and synthetic test data. Furthermore, to accurately align human preferences with S2V benchmarks, we propose three automatic metrics, NexusScore, NaturalScore and GmeScore, to separately quantify subject consistency, naturalness, and text relevance in generated videos. Building on this, we conduct a comprehensive evaluation of 18 representative S2V models, highlighting their strengths and weaknesses across different content. Moreover, we create the first open-source large-scale S2V generation dataset OpenS2V-5M, which consists of five million high-quality 720P subject-text-video triples. Specifically, we ensure subject-information diversity in our dataset by (1) segmenting subjects and building pairing information via cross-video associations and (2) prompting GPT-Image-1 on raw frames to synthesize multi-view representations. Through OpenS2V-Nexus, we deliver a robust infrastructure to accelerate future S2V generation research.

Figures

Figures reproduced from arXiv: 2505.20292 by the authors.

Figure 1
Figure 1. Example of Seven Categories from OpenS2V-Eval. These categories fully encompass the subject-to-video tasks, allowing comprehensive evaluation. Videos are generated by Kling [43] [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The Pipeline of Constructing OpenS2V-Eval. (Left) Our benchmark includes not only real subject images but also synthetic images constructed through GPT-Image-1 [1], allowing for a more comprehensive evaluation. (Right) The metrics are tailored for subject-to-video generation, evaluating not only S2V characteristics (e.g., consistency) but also basic video elements (e.g., motion). three core issues of subject-to-vide… view at source ↗
Figure 3
Figure 3. Statistics in OpenS2V-Eval. The benchmark covers diverse categories and prompt words, with subject images displaying high aesthetics, thus enabling a thorough evaluation. 3.3 New Automatic Metrics As previously mentioned, existing S2V benchmarks are usually adapted from T2V rather than being specifically tailored. For subject-to-video, it is crucial to evaluate not only global aspects such as visual quality and moti… view at source ↗
Figures from the paper (21 more)
Figure 4
Figure 4. Figure 4: The Pipeline of Constructing OpenS2V-5M. First, we filter low-quality videos based on scores such as aesthetics and motion, then utilize GroundingDino [56] and SAM2.1 [76] to extract subject images and get Regular Data. Subsequently, we create Nexus Data through cross-…
Figure 5
Figure 5. Figure 5: Comparison between Regular Data and Nexus Data. The latter is of higher quality. video. The similarity between subjects across these clips is computed using a multimodal retrieval model [119] Mretrieval, which computes the similarity score S(Cij , Ckl) for any pair of …
Figure 6
Figure 6. Figure 6: Qualitative Comparison among Different Methods for the Open-Domain Subject-to￾Video task. Existing methods handle non-human entities better than human identities, and perform better with single subject compared to multiple subjects. The video features a man sitting in …
Figure 7
Figure 7. Figure 7: Qualitative Comparison among Different Methods for the Human-Domain Subject-to￾Video task. They are unable to generate consistent side profiles and suffer from copy-paste issues. and human perception. It is evident that the three proposed metrics—Nexus Score, NaturalSc…
Figure 8
Figure 8. Figure 8: Qualitative Comparison among Different Methods for the Single-Domain Subject-to￾Video task. Existing models perform better on single-subject than multi-subject tasks. Ours † 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% NexusScore NaturalScore GmeScore FaceSim AestheticScore …
Figure 9
Figure 9. Figure 9: (a) Alignment between Automatic Metrics and Human Perception. The proposed metrics are comparable to other metrics [17, 6, 16] in terms of human preference. (2) Validation of ConsisID-Nexu-5M with † and without ‡ Nexus Data. Training are based on ConsisID [113]. natura…
Figure 10
Figure 10. Figure 10: Comparison with Existing Metircs for Subject Consistency and Text Relevance. The proposed automatic metricsalign more closely with human preferences compared to the commonly used DINO-I [116], CLIP-I [73], and CLIP-T [73] in existing S2V methods [40, 55, 37, 21]. Nexu…
Figure 11
Figure 11. Figure 11: Comparison with Existing Methods for Subject Naturalness. Existing AIGC anomaly detection models and multimodal models are both prone to misidentifying generated content as real. introduced. A notable example is IP-Adapter [110], which leverages large datasets to trai…
Figure 12
Figure 12. Figure 12: Visual Reference for Varying Scores of Different Metircs. It is evident that the proposed NexusScore, NaturalScore, and GmeScore are highly correlated with human perception. evaluation akin to human evaluation to gather user preferences for DINO-I, CLIP-I, and CLIP-T.…
Figure 13
Figure 13. Figure 13: Example of Common Issues faced by current Subject-to-Video Generation Models. These videos are generated by Kling [43] and SkyReels-A2 [21] for demonstration purposes only. (a) Open-Domain Evaluation (b) Human-Domain Evaluation (c) Single-Domain Evaluation [PITH_FULL…
Figure 14
Figure 14. Figure 14: Visualization of all the Quantitative Results in OpenS2V-Eval. While part of this issue can be attributed to human perception being more sensitive to facial changes, the primary cause lies in the models’ insufficient capabilities. This is also one of the reasons why h…
Figure 15
Figure 15. Figure 15: Statistics in OpenS2V-5M. The dataset includes a diverse range of categories, clip durations and caption lengths, with most of videos being in high quality (e.g., resolution, aesthetic) [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: More Showcases Generated by Ours‡. C.4 Samples of Collected Data [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: Samples from the OpenS2V-5M dataset. The dataset consists of subject-text-video triples, which exhibit more physical knowledge than existing large-scale T2V dataset [12, 91]. D.2 Details of Evaluation Models As most S2V models [113, 123, 18, 55, 21, 40] do not support…
Figure 18
Figure 18. Figure 18: Distribution of NexusScore, AestheticsScore and MotionScore. D.5 Additional Details of Metircs Normalization OpenS2V-Eval evaluates six key dimensions: subject consistency, subject naturalness, text relevance, face similarity, visual quality, and motion amplitude. Due…
Figure 19
Figure 19. Figure 19: Visualization of the Questionnaire for User Study. Given an image caption, please retrieve the entity words that indicate background, subject, and visually separable objects. [Definition of background] The background spaces that appear in most of the image area. [Defi…
Figure 20
Figure 20. Figure 20: Visualization of Different Input Text Prompts. participant to mitigate cognitive bias. ④ We implement a sliding verification upon submission to ensure that all questionnaires are completed manually, thereby preventing automated (bot) responses. ⑤ We exclude any questi…
Figure 21
Figure 21. Figure 21: More Showcases in OpenS2V-Eval for Open-Domain Subject-to-Video Generation. Kling1.6 Phantom 1.3B SkyReels -A2 P14B VACE P1.3B The video begins with a close-up of a vintage television resting on a classic wooden stand in a cozy living room. The camera zooms in to high…
Figure 22
Figure 22. Figure 22: More Showcases in OpenS2V-Eval for Single-Domain Subject-to-Video Generation. 14 [PITH_FULL_IMAGE:figures/full_fig_p032_22.png]
Figure 23
Figure 23. Figure 23: More Showcases in OpenS2V-Eval for Human-Domain Subject-to-Video Generation. 15 [PITH_FULL_IMAGE:figures/full_fig_p033_23.png]
Figure 24
Figure 24. Figure 24: More Showcases in OpenS2V-Eval for Human-Domain Subject-to-Video Generation. 16 [PITH_FULL_IMAGE:figures/full_fig_p034_24.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RefCaptioner: Multi-Reference Image-Grounded Video Captioning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Mixed-data SFT plus Hierarchical Coverage-Discounted GRPO yields open-source SOTA multi-reference image-grounded video captions on MRVBench without hurting general captioning.

  2. Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Aura combines VLM meta-queries, T5-teacher alignment, subject-aware RoPE shifts, memory tokens, and a large AIGC-curated dataset to claim SOTA multi-element subject-to-video generation under OpenS2V-Eval Total score.

  3. Rethinking Position Embedding as a Context Controller for Multi-Reference and Multi-Shot Video Generation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    SideInfo-RoPE encodes reference-identity agreement as an extra rotary axis, disambiguating similar characters in multi-reference multi-shot video generation while keeping full semantic attention.

  4. Vera: Identity-Faithful Human Subject-to-Video Generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Vera improves identity consistency in human subject-to-video generation using cross-clip identity-aligned data, face-weighted masked loss, and layer-aware reference attention.

  5. Identity-Preserving Text-to-Video Generation via Training-Free Prompt, Image, and Guidance Enhancement

    cs.CV 2025-09 conditional novelty 5.0 of 10

    A training-free prompt, image, and guidance enhancement framework improves face consistency and video quality for identity-preserving text-to-video generation, winning the ACM Multimedia 2025 IPVG challenge.

Reference graph

Works this paper leans on

130 extracted references · 1 canonical work pages · cited by 5 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Detecting ai-generated images using vision transformers: A robust approach for safeguarding visual media integrity

    Nouar AlDahoul and Yasir Zaki. Detecting ai-generated images using vision transformers: A robust approach for safeguarding visual media integrity. Available at SSRN, 2024

    2024

  3. [3]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. a...

    arXiv 2025

  4. [4]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1728–1738, 2021

    2021

  5. [5]

    Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. arXiv preprint arXiv:2405.04233, 2024

    arXiv 2024

  6. [6]

    Gary Bradski, Adrian Kaehler, et al. Opencv. Dr. Dobb’s journal of software tools, 3(2), 2000

    2000

  7. [7]

    Ditctrl: Exploring attention control in multi-modal diffusion trans- former for tuning-free multi-prompt longer video generation.arXiv preprint arXiv:2412.18597, 2024

    Minghong Cai, Xiaodong Cun, Xiaoyu Li, Wenze Liu, Zhaoyang Zhang, Yong Zhang, Ying Shan, and Xiangyu Yue. Ditctrl: Exploring attention control in multi-modal diffusion trans- former for tuning-free multi-prompt longer video generation.arXiv preprint arXiv:2412.18597, 2024

    arXiv 2024

  8. [9]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion

    Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mohammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. arXiv preprint arXiv:2311.12052, 2023

    arXiv 2023

Show all 130 references
  1. [10]

    Photoverse: Tuning-free image customization with text-to-image diffusion models

    Li Chen, Mengyi Zhao, Yiheng Liu, Mingxu Ding, Yangyang Song, Shizun Wang, Xu Wang, Hao Yang, Jing Liu, Kang Du, et al. Photoverse: Tuning-free image customization with text-to-image diffusion models. arXiv preprint arXiv:2309.05793, 2023

    2023 arXiv

  2. [11]

    Od-vae: An omni-dimensional video compressor for improving latent video diffusion model

    Liuhan Chen, Zongjian Li, Bin Lin, Bin Zhu, Qian Wang, Shenghai Yuan, Xing Zhou, Xinhua Cheng, and Li Yuan. Od-vae: An omni-dimensional video compressor for improving latent video diffusion model. arXiv preprint arXiv:2409.01199, 2024

    2024 arXiv

  3. [12]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. arXiv preprint arXiv:2402.19479, 2024

    2024 arXiv

  4. [13]

    Multi- subject open-set personalization in video generation

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aberman, Jun-Yan Zhu, Ming-Hsuan Yang, and Sergey Tulyakov. Multi- subject open-set personalization in video generation. arXiv preprint arXiv:2501.06187, 2025

    2025 arXiv

  5. [14]

    Unireal: Universal image generation and editing via learning real-world dynamics

    Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, et al. Unireal: Universal image generation and editing via learning real-world dynamics. arXiv preprint arXiv:2412.07774, 2024

    2024 arXiv

  6. [15]

    Yolo- world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo- world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 16901–16911, 2024

    2024

  7. [16]

    improved-aesthetic-predictor

    christophschuhmann. improved-aesthetic-predictor. improved-aesthetic-predictor Lab, 2024. 11

    2024

  8. [17]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In CVPR, pages 4690–4699, 2019

    2019

  9. [18]

    Cinema: Coherent multi-subject video generation via mllm-based guidance

    Yufan Deng, Xun Guo, Yizhi Wang, Jacob Zhiyuan Fang, Angtian Wang, Shenghai Yuan, Yiding Yang, Bo Liu, Haibin Huang, and Chongyang Ma. Cinema: Coherent multi-subject video generation via mllm-based guidance. arXiv preprint arXiv:2503.10391, 2025

    2025 arXiv

  10. [19]

    Worldscore: A unified evaluation benchmark for world generation

    Haoyi Duan, Hong-Xing Yu, Sirui Chen, Li Fei-Fei, and Jiajun Wu. Worldscore: A unified evaluation benchmark for world generation. arXiv preprint arXiv:2504.00983, 2025

    2025

  11. [20]

    Vchitect-2.0: Parallel transformer for scaling up video diffusion models

    Weichen Fan, Chenyang Si, Junhao Song, Zhenyu Yang, Yinan He, Long Zhuo, Ziqi Huang, Ziyue Dong, Jingwen He, Dongwei Pan, et al. Vchitect-2.0: Parallel transformer for scaling up video diffusion models. arXiv preprint arXiv:2501.08453, 2025

    2025 arXiv

  12. [21]

    Skyreels-a2: Compose anything in video diffusion trans- formers

    Zhengcong Fei, Debang Li, Di Qiu, Jiahua Wang, Yikun Dou, Rui Wang, Jingtao Xu, Mingyuan Fan, Guibin Chen, Yang Li, et al. Skyreels-a2: Compose anything in video diffusion trans- formers. arXiv preprint arXiv:2504.02436, 2025

    2025 arXiv

  13. [22]

    Ingredients: Blending custom photos with video diffusion transformers

    Zhengcong Fei, Debang Li, Di Qiu, Changqian Yu, and Mingyuan Fan. Ingredients: Blending custom photos with video diffusion transformers. arXiv preprint arXiv:2501.01790, 2025

    2025 arXiv

  14. [23]

    Ae-nerf: Augmenting event-based neural radiance fields for non-ideal conditions and larger scene

    Chaoran Feng, Wangbo Yu, Xinhua Cheng, Zhenyu Tang, Junwu Zhang, Li Yuan, and Yonghong Tian. Ae-nerf: Augmenting event-based neural radiance fields for non-ideal conditions and larger scene. arXiv preprint arXiv:2501.02807, 2025

    2025 arXiv

  15. [24]

    An image is worth one word: Personalizing text-to-image generation using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022

    2022 arXiv

  16. [25]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Yaohui Wang, Yu Qiao, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023

    2023 arXiv

  17. [26]

    Pulid: Pure and lightning id customization via contrastive alignment

    Zinan Guo, Yanze Wu, Zhuowei Chen, Lang Chen, and Qian He. Pulid: Pure and lightning id customization via contrastive alignment. arXiv preprint arXiv:2404.16022, 2024

    2024 arXiv

  18. [27]

    Uniportrait: A unified framework for identity- preserving single-and multi-human image personalization

    Junjie He, Yifeng Geng, and Liefeng Bo. Uniportrait: A unified framework for identity- preserving single-and multi-human image personalization. arXiv preprint arXiv:2408.05939, 2024

    2024 arXiv

  19. [28]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. arXiv preprint arXiv:2406.15252, 2024

    2024 arXiv

  20. [29]

    Id-animator: Zero-shot identity-preserving human video generation

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, Man Zhou, and Jie Zhang. Id-animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275, 2024

    2024 arXiv

  21. [30]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  22. [31]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024

    2024

  23. [32]

    Animate anyone 2: High-fidelity character image animation with environment affordance

    Li Hu, Guangyuan Wang, Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, and Liefeng Bo. Animate anyone 2: High-fidelity character image animation with environment affordance. arXiv preprint arXiv:2502.06145, 2025

    2025 arXiv

  24. [33]

    Hunyuancustom: A multimodal-driven architecture for customized video generation

    Teng Hu, Zhentao Yu, Zhengguang Zhou, Sen Liang, Yuan Zhou, Qin Lin, and Qinglin Lu. Hunyuancustom: A multimodal-driven architecture for customized video generation. arXiv preprint arXiv:2505.04512, 2025. 12

    2025 arXiv

  25. [34]

    Curricularface: adaptive curriculum learning loss for deep face recognition

    Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, Jilin Li, and Feiyue Huang. Curricularface: adaptive curriculum learning loss for deep face recognition. In CVPR, pages 5901–5910, 2020

    2020

  26. [35]

    Conceptmaster: Multi-concept video customization on diffusion transformer models without test-time tuning

    Yuzhou Huang, Ziyang Yuan, Quande Liu, Qiulin Wang, Xintao Wang, Ruimao Zhang, Pengfei Wan, Di Zhang, and Kun Gai. Conceptmaster: Multi-concept video customization on diffusion transformer models without test-time tuning. arXiv preprint arXiv:2501.04698, 2025

    2025 arXiv

  27. [36]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. arXiv preprint arXiv:2311.17982, 2023

    2023 arXiv

  28. [37]

    Vbench++: Comprehensive and versatile benchmark suite for video generative models

    Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, et al. Vbench++: Comprehensive and versatile benchmark suite for video generative models. arXiv preprint arXiv:2411.13503, 2024

    2024 arXiv

  29. [38]

    Infiniteyou: Flexible photo recrafting while preserving your identity

    Liming Jiang, Qing Yan, Yumin Jia, Zichuan Liu, Hao Kang, and Xin Lu. Infiniteyou: Flexible photo recrafting while preserving your identity. arXiv preprint arXiv:2503.16418, 2025

    2025 arXiv

  30. [39]

    Videobooth: Diffusion-based video generation with image prompts

    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. Videobooth: Diffusion-based video generation with image prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6689–6700, 2024

    2024

  31. [40]

    Vace: All-in-one video creation and editing

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. arXiv preprint arXiv:2503.07598, 2025

    2025 arXiv

  32. [41]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024

    2024 arXiv

  33. [42]

    Subjective-aligned dateset and metric for text-to-video quality assessment

    Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, and Ning Liu. Subjective-aligned dateset and metric for text-to-video quality assessment. arXiv preprint arXiv:2403.11956, 2024

    2024 arXiv

  34. [43]

    Kwai. Keling. Kwai, 2024

    2024

  35. [44]

    Pika-2.0 lab discord server

    Pika Lab. Pika-2.0 lab discord server. Pika Lab, 2024

    2024

  36. [45]

    Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation

    Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation. arXiv preprint arXiv:2412.00115, 2024

    2024 arXiv

  37. [46]

    Improving synthetic image detection towards generalization: An image transformation perspective

    Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Fuli Feng. Improving synthetic image detection towards generalization: An image transformation perspective. arXiv preprint arXiv:2408.06741, 2024

    2024 arXiv

  38. [47]

    Photomaker: Customizing realistic human photos via stacked id embedding

    Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming-Ming Cheng, and Ying Shan. Photomaker: Customizing realistic human photos via stacked id embedding. In CVPR, pages 8640–8650, 2024

    2024

  39. [48]

    Wf-vae: Enhancing video vae by wavelet-driven energy flow for latent video diffusion model

    Zongjian Li, Bin Lin, Yang Ye, Liuhan Chen, Xinhua Cheng, Shenghai Yuan, and Li Yuan. Wf-vae: Enhancing video vae by wavelet-driven energy flow for latent video diffusion model. arXiv preprint arXiv:2411.17459, 2024

    2024 arXiv

  40. [49]

    Movie weaver: Tuning-free multi-concept video personalization with anchored prompts

    Feng Liang, Haoyu Ma, Zecheng He, Tingbo Hou, Ji Hou, Kunpeng Li, Xiaoliang Dai, Felix Juefei-Xu, Samaneh Azadi, Animesh Sinha, et al. Movie weaver: Tuning-free multi-concept video personalization with anchored prompts. arXiv preprint arXiv:2502.07802, 2025

    2025 arXiv

  41. [50]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131, 2024

    2024 arXiv

  42. [51]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. EMNLP, 2024. 13

    2024

  43. [52]

    Stiv: Scalable text and image condi- tioned video generation

    Zongyu Lin, Wei Liu, Chen Chen, Jiasen Lu, Wenze Hu, Tsu-Jui Fu, Jesse Allardice, Zhengfeng Lai, Liangchen Song, Bowen Zhang, et al. Stiv: Scalable text and image condi- tioned video generation. arXiv preprint arXiv:2412.07730, 2024

    2024

  44. [53]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  45. [54]

    Lumina-video: Efficient and flexible video generation with multi-scale next-dit

    Dongyang Liu, Shicheng Li, Yutong Liu, Zhen Li, Kai Wang, Xinyue Li, Qi Qin, Yufei Liu, Yi Xin, Zhongyu Li, et al. Lumina-video: Efficient and flexible video generation with multi-scale next-dit. arXiv preprint arXiv:2502.06782, 2025

    2025 arXiv

  46. [55]

    Phantom: Subject-consistent video generation via cross-modal alignment

    Lijie Liu, Tianxiang Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, and Xinglong Wu. Phantom: Subject-consistent video generation via cross-modal alignment. arXiv preprint arXiv:2502.11079, 2025

    2025 arXiv

  47. [56]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023

    2023 arXiv

  48. [57]

    Evalcrafter: Benchmarking and evaluating large video generation models

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and evaluating large video generation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2024

  49. [58]

    Fetv: A benchmark for fine-grained evaluation of open-domain text-to-video genera- tion

    Yuanxin Liu, Lei Li, Shuhuai Ren, Rundong Gao, Shicheng Li, Sishuo Chen, Xu Sun, and Lu Hou. Fetv: A benchmark for fine-grained evaluation of open-domain text-to-video genera- tion. Advances in Neural Information Processing Systems, 36, 2024

    2024

  50. [59]

    Step-video-t2v technical report: The practice, challenges, and future of video foundation model

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-t2v technical report: The practice, challenges, and future of video foundation model. arXiv preprint arXiv:2502.10248, 2025

    2025 arXiv

  51. [60]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024

    2024 arXiv

  52. [61]

    Model reveals what to cache: Profiling-based feature reuse for video diffusion models

    Xuran Ma, Yexin Liu, Yaofu Liu, Xianfeng Wu, Mingzhe Zheng, Zihao Wang, Ser-Nam Lim, and Harry Yang. Model reveals what to cache: Profiling-based feature reuse for video diffusion models. arXiv preprint arXiv:2504.03140, 2025

    2025 arXiv

  53. [62]

    Follow your pose: Pose-guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4117–4125, 2024

    2024

  54. [63]

    Follow-your-click: Open-domain regional image animation via short prompts

    Yue Ma, Yingqing He, Hongfa Wang, Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung-Yeung Shum, Wei Liu, et al. Follow-your-click: Open-domain regional image animation via short prompts. arXiv preprint arXiv:2403.08268, 2024

    2024 arXiv

  55. [64]

    Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation

    Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, et al. Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation. arXiv preprint arXiv:2406.01900, 2024

    2024 arXiv

  56. [65]

    Magic-me: Identity-specific video customized diffusion

    Ze Ma, Daquan Zhou, Chun-Hsiao Yeh, Xue-She Wang, Xiuyu Li, Huanrui Yang, Zhen Dong, Kurt Keutzer, and Jiashi Feng. Magic-me: Identity-specific video customized diffusion. arXiv preprint arXiv:2402.09368, 2024

    2024 arXiv

  57. [66]

    Multi-task image classifier

    Abdellahi El Moustapha. Multi-task image classifier. https://huggingface.co/Abdu07/ multitask-model, 2025

    2025

  58. [67]

    Openvid-1m: A large-scale high-quality dataset for text-to-video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation. arXiv preprint arXiv:2407.02371, 2024. 14

    2024 arXiv

  59. [68]

    Nyuad ai generated images detector

    Yasir Zaki Nouar AlDahoul. Nyuad ai generated images detector

  60. [69]

    Dreamdance: Animating human images by enriching 3d geometry cues from 2d poses

    Yatian Pang, Bin Zhu, Bin Lin, Mingzhe Zheng, Francis EH Tay, Ser-Nam Lim, Harry Yang, and Li Yuan. Dreamdance: Animating human images by enriching 3d geometry cues from 2d poses. arXiv preprint arXiv:2412.00397, 2024

    2024 arXiv

  61. [70]

    Open-sora 2.0: Training a commercial-level video generation model in 200 k

    Xiangyu Peng, Zangwei Zheng, Chenhui Shen, Tom Young, Xinying Guo, Binluo Wang, Hang Xu, Hongxin Liu, Mingyan Jiang, Wenjun Li, et al. Open-sora 2.0: Training a commercial-level video generation model in 200 k. arXiv preprint arXiv:2503.09642, 2025

    2025 arXiv

  62. [71]

    Dreambench++: A human-aligned benchmark for personalized image generation

    Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. Dreambench++: A human-aligned benchmark for personalized image generation. arXiv preprint arXiv:2406.16855, 2024

    2024 arXiv

  63. [72]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  64. [73]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  65. [74]

    Hierarchical text-conditional image generation with clip latents, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents, 2022. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  66. [75]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, pages 8821–8831, 2021

    2021

  67. [76]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenh...

    2024 arXiv

  68. [77]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aber- man. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pages 22500–22510, 2023

    2023

  69. [78]

    Magi-1: Autoregressive video generation at scale, 2025

    Sand-AI. Magi-1: Autoregressive video generation at scale, 2025

    2025

  70. [79]

    Seaweed-7b: Cost-effective training of video generation foundation model

    Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7b: Cost-effective training of video generation foundation model. arXiv preprint arXiv:2504.08685, 2025

    2025 arXiv

  71. [80]

    Dragdiffusion: Harnessing diffusion models for interactive point-based image editing

    Yujun Shi, Chuhui Xue, Jun Hao Liew, Jiachun Pan, Hanshu Yan, Wenqing Zhang, Vincent YF Tan, and Song Bai. Dragdiffusion: Harnessing diffusion models for interactive point-based image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024

  72. [81]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792, 2022

    2022 arXiv

  73. [82]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012

    2012 arXiv

  74. [83]

    Animate-x: Universal character image animation with enhanced motion representation

    Shuai Tan, Biao Gong, Xiang Wang, Shiwei Zhang, Dandan Zheng, Ruobing Zheng, Kecheng Zheng, Jingdong Chen, and Ming Yang. Animate-x: Universal character image animation with enhanced motion representation. arXiv preprint arXiv:2410.10306, 2024. 15

    arXiv

  75. [84]

    Cycle3d: High-quality and consistent image-to-3d generation via generation- reconstruction cycle

    Zhenyu Tang, Junwu Zhang, Xinhua Cheng, Wangbo Yu, Chaoran Feng, Yatian Pang, Bin Lin, and Li Yuan. Cycle3d: High-quality and consistent image-to-3d generation via generation- reconstruction cycle. arXiv preprint arXiv:2407.19548, 2024

    2024 arXiv

  76. [85]

    Gemma 3 technical report

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  77. [86]

    Genmo Team. Mochi 1. https://github.com/genmoai/models, 2024

    2024

  78. [87]

    Hailuo Team. Hailuo. Hailuo Lab, 2024

    2024

  79. [88]

    Paddleocr

    PaddleOCR Team. Paddleocr. PaddleOCR Lab, 2024

    2024

  80. [89]

    Wan: Open and advanced large-scale video generative models

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  81. [90]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  82. [91]

    Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content

    Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, et al. Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content. arXiv preprint arXiv:2410.08260, 2024

    arXiv

  83. [92]

    Instantid: Zero-shot identity-preserving generation in seconds

    Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. Instantid: Zero-shot identity-preserving generation in seconds. arXiv preprint arXiv:2401.07519, 2024

    2024 arXiv

  84. [93]

    Vidprom: A million-scale real prompt-gallery dataset for text-to- video diffusion models

    Wenhao Wang and Yi Yang. Vidprom: A million-scale real prompt-gallery dataset for text-to- video diffusion models. arXiv preprint arXiv:2403.06098, 2024

    2024 arXiv

  85. [94]

    Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation

    Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation. arXiv preprint arXiv:2305.10874, 2023

    2023 arXiv

  86. [95]

    Internvid: A large-scale video-text dataset for multimodal understanding and generation

    Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023

    2023 arXiv

  87. [96]

    Is your world simulator a good story presenter? a consecutive events-based benchmark for future long video generation

    Yiping Wang, Xuehai He, Kuan Wang, Luyao Ma, Jianwei Yang, Shuohang Wang, Si- mon Shaolei Du, and Yelong Shen. Is your world simulator a good story presenter? a consecutive events-based benchmark for future long video generation. arXiv preprint arXiv:2412.16211, 2024

    2024 arXiv

  88. [97]

    Echovideo: Identity-preserving human video generation by multimodal feature fusion

    Jiangchuan Wei, Shiyue Yan, Wenfeng Lin, Boyuan Liu, Renjie Chen, and Mingyu Guo. Echovideo: Identity-preserving human video generation by multimodal feature fusion. arXiv preprint arXiv:2501.13452, 2025

    2025 arXiv

  89. [98]

    Dreamvideo: Composing your dream videos with customized subject and motion

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In CVPR, pages 6537–6549, 2024

    2024

  90. [99]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference on...

    2023

  91. [101]

    Towards a better metric for text-to-video generation

    Jay Zhangjie Wu, Guian Fang, Haoning Wu, Xintao Wang, Yixiao Ge, Xiaodong Cun, David Junhao Zhang, Jia-Wei Liu, Yuchao Gu, Rui Zhao, et al. Towards a better metric for text-to-video generation. arXiv preprint arXiv:2401.07781, 2024

    2024 arXiv

  92. [102]

    Motionbooth: Motion-aware customized text-to-video generation

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Motionbooth: Motion-aware customized text-to-video generation. arXiv preprint arXiv:2406.17758, 2024

    2024 arXiv

  93. [103]

    Videomaker: Zero-shot customized video generation with the inherent force of video diffusion models

    Tao Wu, Yong Zhang, Xiaodong Cun, Zhongang Qi, Junfu Pu, Huanzhang Dou, Guangcong Zheng, Ying Shan, and Xi Li. Videomaker: Zero-shot customized video generation with the inherent force of video diffusion models. arXiv preprint arXiv:2412.19645, 2024

    2024 arXiv

  94. [104]

    Omnigen: Unified image generation

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409.11340, 2024

    2024 arXiv

  95. [105]

    Easyanimate: A high-performance long video generation method based on transformer architecture

    Jiaqi Xu, Xinyi Zou, Kunzhe Huang, Yunkuo Chen, Bo Liu, MengLi Cheng, Xing Shi, and Jun Huang. Easyanimate: A high-performance long video generation method based on transformer architecture. arXiv preprint arXiv:2405.18991, 2024

    2024

  96. [106]

    MSR-VTT: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. MSR-VTT: A large video description dataset for bridging video and language. CVPR, pages 5288–5296, 2016

    2016

  97. [107]

    A sanity check for ai-generated image detection

    Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection. arXiv preprint arXiv:2406.19435, 2024

    2024 arXiv

  98. [108]

    Videogen-eval: Agent-based system for video generation evaluation

    Yuhang Yang, Ke Fan, Shangkun Sun, Hongxiang Li, Ailing Zeng, FeiLin Han, Wei Zhai, Wei Liu, Yang Cao, and Zheng-Jun Zha. Videogen-eval: Agent-based system for video generation evaluation. arXiv preprint arXiv:2503.23452, 2025

    2025 arXiv

  99. [109]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  100. [110]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  101. [111]

    From slow bidirectional to fast causal video generators

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast causal video generators. arXiv preprint arXiv:2412.07772, 2024

    2024

  102. [112]

    Evagaussians: Event stream assisted gaussian splatting from blurry images

    Wangbo Yu, Chaoran Feng, Jiye Tang, Xu Jia, Li Yuan, and Yonghong Tian. Evagaussians: Event stream assisted gaussian splatting from blurry images. arXiv preprint arXiv:2405.20224, 2024

    2024 arXiv

  103. [113]

    Identity-preserving text-to-video generation by frequency decomposition

    Shenghai Yuan, Jinfa Huang, Xianyi He, Yunyuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. Identity-preserving text-to-video generation by frequency decomposition. arXiv preprint arXiv:2411.17440, 2024

    2024 arXiv

  104. [114]

    Magictime: Time-lapse video generation models as metamorphic simulators

    Shenghai Yuan, Jinfa Huang, Yujun Shi, Yongqi Xu, Ruijie Zhu, Bin Lin, Xinhua Cheng, Li Yuan, and Jiebo Luo. Magictime: Time-lapse video generation models as metamorphic simulators. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  105. [115]

    Chronomagic-bench: A benchmark for meta- morphic evaluation of text-to-time-lapse video generation

    Shenghai Yuan, Jinfa Huang, Yongqi Xu, Yaoyang Liu, Shaofeng Zhang, Yujun Shi, Rui-Jie Zhu, Xinhua Cheng, Jiebo Luo, and Li Yuan. Chronomagic-bench: A benchmark for meta- morphic evaluation of text-to-time-lapse video generation. Advances in Neural Information Processing Syste...

    2024

  106. [116]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022

    2022 arXiv

  107. [117]

    Compress & align: Curating image-text data with human knowledge

    Lei Zhang, Fangxun Shu, Sucheng Ren, Bingchen Zhao, Hao Jiang, and Cihang Xie. Compress & align: Curating image-text data with human knowledge. arXiv preprint arXiv:2312.06726, 2023. 17

    2023 arXiv

  108. [118]

    Packing input frame context in next-frame prediction models for video generation

    Lvmin Zhang and Maneesh Agrawala. Packing input frame context in next-frame prediction models for video generation. arXiv preprint arXiv:2504.12626, 2025

    2025

  109. [119]

    Gme: Improving universal multimodal retrieval by multimodal llms

    Xin Zhang, Yanzhao Zhang, Wen Xie, Mingxin Li, Ziqi Dai, Dingkun Long, Pengjun Xie, Meishan Zhang, Wenjie Li, and Min Zhang. Gme: Improving universal multimodal retrieval by multimodal llms. arXiv preprint arXiv:2412.16855, 2024

    2024 arXiv

  110. [120]

    Fantasyid: Face knowledge enhanced id-preserving video generation

    Yunpeng Zhang, Qiang Wang, Fan Jiang, Yaqi Fan, Mu Xu, and Yonggang Qi. Fantasyid: Face knowledge enhanced id-preserving video generation. arXiv preprint arXiv:2502.13995, 2025

    2025 arXiv

  111. [121]

    Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, et al. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755, 2025

    2025 arXiv

  112. [122]

    Videogen-of-thought: A collaborative framework for multi-shot video generation

    Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma, Yexin Liu, Wenjie Shu, Yatian Pang, Feilong Tang, Qifeng Chen, Harry Yang, et al. Videogen-of-thought: A collaborative framework for multi-shot video generation. arXiv preprint arXiv:2412.02259, 2024

    2024

  113. [123]

    Concat-id: Towards universal identity-preserving video synthesis

    Yong Zhong, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Chongxuan Li. Concat-id: Towards universal identity-preserving video synthesis. arXiv preprint arXiv:2503.14151, 2025

    2025 arXiv

  114. [124]

    Allegro: Open the black box of commercial-level video generation model

    Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Allegro: Open the black box of commercial-level video generation model. arXiv preprint arXiv:2410.15458, 2024

    2024 arXiv

  115. [125]

    Celebv-hq: A large-scale video facial attributes dataset

    Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, and Chen Change Loy. Celebv-hq: A large-scale video facial attributes dataset. In European conference on computer vision, pages 650–667. Springer, 2022. 18 Paper Appendix for OpenS2V-Nexus: A Ultra-S...

    2022

  116. [126]

    All higher scores are better

    The results demonstrate that the proposed NexusScore and GmeScore offer greater accuracy in assessing subject consistency and text relevance compared to others. All higher scores are better. B.2 Comparison with Existing Metrics for Subject Naturalness To evaluate whether a gen...

  117. [127]

    While these alternatives do not surpass Kling [ 43], they still outperform open-source models

    and Vidu [5] may be preferred. While these alternatives do not surpass Kling [ 43], they still outperform open-source models. (2) For community developers, it is recommended to base S2V model development on Phantom [55] or V ACE [40], as it generates videos with relatively hig...

  118. [128]

    The entity word is a singular or plural noun without any quantifier or descriptive phrase

  119. [129]

    (e.g, 'red top' better than 'top', 'martial arts uniforms' better than 'uniforms')

    The entity word must be an exact subset of the caption, including its characters, words, and symbols. (e.g, 'red top' better than 'top', 'martial arts uniforms' better than 'uniforms')

  120. [130]

    Exclude any part of the body (e.g., 'hands', 'legs', 'feet', 'head')

  121. [131]

    Exclude abstract or non-physical concepts (e.g., 'facial expressions', 'gestures', 'stance')

  122. [132]

    Do not modify or interpret any part of the caption

    Exclude actions or descriptions (e.g., 'adjusting', 'imitating'). Do not modify or interpret any part of the caption. Here is an example, follow this JSON format to output the results: Caption: A woman in a mask and coat, with long brown hair, shows a small green-capped bottle...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.