REVIEW 10 cited by
WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent rapid advancements in text-to-video (T2V) generation, such as SoRA and Kling, have shown great potential for building world simulators. However, current T2V models struggle to grasp abstract physical principles and generate videos that adhere to physical laws. This challenge arises primarily from a lack of clear guidance on physical information due to a significant gap between abstract physical principles and generation models. To this end, we introduce the World Simulator Assistant (WISA), an effective framework for decomposing and incorporating physical principles into T2V models. Specifically, WISA decomposes physical principles into textual physical descriptions, qualitative physical categories, and quantitative physical properties. To effectively embed these physical attributes into the generation process, WISA incorporates several key designs, including Mixture-of-Physical-Experts Attention (MoPA) and a Physical Classifier, enhancing the model's physics awareness. Furthermore, most existing datasets feature videos where physical phenomena are either weakly represented or entangled with multiple co-occurring processes, limiting their suitability as dedicated resources for learning explicit physical principles. We propose a novel video dataset, WISA-32K, collected based on qualitative physical categories. It consists of 32,000 videos, representing 17 physical laws across three domains of physics: dynamics, thermodynamics, and optics. Experimental results demonstrate that WISA can effectively enhance the compatibility of T2V models with real-world physical laws, achieving a considerable improvement on the VideoPhy benchmark. The visual exhibitions of WISA and WISA-32K are available in the https://360cvgroup.github.io/WISA/.
Forward citations
Cited by 10 Pith papers
-
RDPO: Real Data Preference Optimization for Physics Consistency Video Generation
RDPO builds preference pairs by reverse-sampling real video latents with a pre-trained generator, then fine-tunes with Flow-DPO, improving physics consistency metrics on two video models.
-
muSync-GS: Physics-Synchronized Driving Video Synthesis for Weather and Geometric Road Hazards
muSync-GS couples weather and road-shape edits in driving videos to a calibrated vehicle-dynamics model, so the synthesized ego motion and telemetry change with the same controls that drive the visual edits.
-
VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System
Using executable Blender code as an intermediate simulation draft improves physical consistency in text-to-video generation, lifting OmniWeaving from 0.475 to 0.558 on PhyGenBench and from 52.18% to 77.88% on VBench-2.0.
-
VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generation
An MLLM extracts transferable physical cues from a reference video and conditions a pretrained I2V generator so new scenes follow that physics without exhaustive prompts.
-
Learning Explicit Physical Parameter Control and Benchmarking for Video Generation
Explicit instance-level physical parameter conditioning with routing attention improves physical-law consistency in image-to-video generation, as measured on the authors' new simulator-based benchmark.
-
SHIFT: Motion Alignment in Video Diffusion Models with Adversarial Hybrid Fine-Tuning
Pixel-motion rewards plus adversarial hybrid fine-tuning (SHIFT) reverse dynamic-degree collapse in image-conditioned video diffusion models while preserving appearance.
-
VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models
VideoREPA adds a token-relation distillation loss that aligns a text-to-video diffusion model's internal features with VideoMAEv2, boosting physical commonsense scores on VideoPhy and VideoPhy2.
-
From Black Box to Transparency: Enhancing Automated Interpreting Assessment with Explainable AI in College Classrooms
An explainable model using BLEURT, CometKiwi, pause features, and Chinese phraseological diversity predicts human-rated quality dimensions in English-Chinese consecutive interpreting, with SHAP identifying the stronge...
-
Physics-Grounded Motion Forecasting via Equation Discovery for Trajectory-Guided Image-to-Video Generation
A retrieval-initialized symbolic regression method discovers equations of motion from video trajectories and uses them to guide image-to-video generation, improving physical alignment on classical mechanics scenes.
-
Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation
A hierarchical direct preference optimization with four alignment levels plus automated data selection improves physical plausibility of text-to-video models.
Discussion (0). Sign in to comment.