REVIEW 23 cited by
VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Video generation has witnessed significant advancements, yet evaluating these models remains a challenge. A comprehensive evaluation benchmark for video generation is indispensable for two reasons: 1) Existing metrics do not fully align with human perceptions; 2) An ideal evaluation system should provide insights to inform future developments of video generation. To this end, we present VBench, a comprehensive benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions, each with tailored prompts and evaluation methods. VBench has several appealing properties: 1) Comprehensive Dimensions: VBench comprises 16 dimensions in video generation (e.g., subject identity inconsistency, motion smoothness, temporal flickering, and spatial relationship, etc). The evaluation metrics with fine-grained levels reveal individual models' strengths and weaknesses. 2) Human Alignment: We also provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception, for each evaluation dimension respectively. 3) Valuable Insights: We look into current models' ability across various evaluation dimensions, and various content types. We also investigate the gaps between video and image generation models. 4) Versatile Benchmarking: VBench++ supports evaluating text-to-video and image-to-video. We introduce a high-quality Image Suite with an adaptive aspect ratio to enable fair evaluations across different image-to-video generation settings. Beyond assessing technical quality, VBench++ evaluates the trustworthiness of video generative models, providing a more holistic view of model performance. 5) Full Open-Sourcing: We fully open-source VBench++ and continually add new video generation models to our leaderboard to drive forward the field of video generation.
Forward citations
Cited by 23 Pith papers
-
RDPO: Real Data Preference Optimization for Physics Consistency Video Generation
RDPO builds preference pairs by reverse-sampling real video latents with a pre-trained generator, then fine-tunes with Flow-DPO, improving physics consistency metrics on two video models.
-
OmniVR: Joint Video-Audio Conditional Generation for Restoring Degraded Historical Films
A single 22B-parameter model jointly restores degraded video and audio of historical films, beating prior separate restorers on visual, audio, and sync metrics.
-
WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models
WorldRoamBench is a new benchmark for interactive world models that evaluates four stability dimensions with custom metrics and finds no tested model performs reliably across all.
-
Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations
A two-stage framework bootstraps cross-category motion transfer by training on self-generated motion-equivalent video pairs, enabling direct reference-video-conditioned animation across morphology gaps.
-
Learning Explicit Physical Parameter Control and Benchmarking for Video Generation
Explicit instance-level physical parameter conditioning with routing attention improves physical-law consistency in image-to-video generation, as measured on the authors' new simulator-based benchmark.
-
Anti-Prompt: Image Protection against Text-Guided Image-to-Video Generation
Imperceptible image perturbations that suppress text-conditioned attention and boost visual-only pathways disrupt text-guided I2V generation more efficiently than prior I2V protection.
-
When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generation
Parallel, role-specialized prompt agents improve cultural relevance in text-to-video generation, with a new cross-cultural benchmark showing the largest gains for location cues.
-
InsertAnywhere: Geometrically Grounded and Optics-Aware Video Object Insertion
InsertAnywhere inserts a reference object into arbitrary videos by reconstructing 4D geometry to propagate a user-given placement across frames and fine-tuning video diffusion on ROSE++, a removal-to-insertion dataset...
-
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
A five-aspect, 24-metric benchmark, a 26K human-annotated dataset, and an AI evaluator show that today's driving world models cannot simultaneously look real, respect geometry, and behave safely.
-
InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problem
A training-free, near-zero-overhead inverse solver for novel-view video generation and inpainting that projects masks into continuous multi-channel latent masks and applies DDS with conjugate gradient in latent space.
-
CineScale: Free Lunch in High-Resolution Cinematic Visual Generation
CineScale extends pre-trained diffusion models to 8k image and 4k video generation with mostly tuning-free inference plus a small LoRA adaptation for video.
-
Pusa V1.0: Unlocking Temporal Control in Pretrained Video Diffusion Models via Vectorized Timestep Adaptation
A $500 LoRA fine-tune of Wan2.1-T2V with per-frame random timesteps matches Wan-I2V's benchmark quality and adds zero-shot start-end and video-extension capabilities.
-
TokensGen: Harnessing Condensed Tokens for Long Video Generation
TokensGen generates consistent long videos by representing each clip as condensed semantic tokens, generating all tokens jointly from text, and stitching clips with adaptive FIFO denoising.
-
DualX-VSR: Dual Axial Spatial$\times$Temporal Transformer for Real-World Video Super-Resolution without Motion Compensation
DualX-VSR uses vertical-temporal and horizontal-temporal attention in a ViViT-style transformer to perform real-world video super-resolution without motion compensation.
-
OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation
A benchmark and five-million-clip dataset for evaluating and training subject-to-video generation models, with three new metrics for subject consistency, naturalness, and text alignment.
-
Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation
A chunk-wise streaming video model with bounded multi-scale memory and streaming 4K upscaling reports real-time interactive long-form generation and top Arena preference/stability scores.
-
Resilience Revisited: A Multidimensional Framework Derived from Realistic Attack Scenarios
A new MDRI framework decomposes power system resilience into five dimensions with a calibrated multiplicative term, showing multi-vector attacks cause 5.6 times more degradation than linear sums plus 60.6% from coupling.
-
DreamSwapV: Mask-guided Subject Swapping for Any Customized Video Editing
DreamSwapV performs mask-guided, subject-agnostic subject swapping in videos using multiple conditions, an adaptive mask strategy, and a new benchmark.
-
Towards Holistic Visual Quality Assessment of AI-Generated Videos: A LLM-Based Multi-Dimensional Evaluation Model
AIGVEval combines BLIP, 3D Swin Transformer, and SlowFast features with a LoRA-tuned LLM to predict AI-generated video quality, hitting second place on the NTIRE 2025 Track 2 leaderboard.
-
RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control
Metric-scale depth alignment plus scene-constrained noise shaping improves camera controllability and video quality for image-to-video generation on RealEstate10K.
-
From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
Physical intelligence needs an embodied brain that reasons over interventions and emits capability requests, grounded by a physical harness and shared experience contracts rather than direct actuator policies.
-
FaceAnonyMixer: Cancelable Faces via Identity Consistent Latent Space Mixing
FaceAnonyMixer claims a cancelable face generation method that irreversibly mixes real latent codes with key-derived synthetic codes for privacy-preserving face recognition.
-
T2VWorldBench: A Benchmark for Evaluating World Knowledge in Text-to-Video Generation
A 1,200-prompt benchmark across six world-knowledge domains reports that ten state-of-the-art text-to-video models average below 0.70 on a 0 to 1 scale for producing videos consistent with real-world knowledge.
Discussion (0). Continue with ORCID to comment.