REVIEW 6 cited by
Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformer
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present a novel approach for generating 360-degree high-quality, spatio-temporally coherent human videos from a single image. Our framework combines the strengths of diffusion transformers for capturing global correlations across viewpoints and time, and CNNs for accurate condition injection. The core is a hierarchical 4D transformer architecture that factorizes self-attention across views, time steps, and spatial dimensions, enabling efficient modeling of the 4D space. Precise conditioning is achieved by injecting human identity, camera parameters, and temporal signals into the respective transformers. To train this model, we collect a multi-dimensional dataset spanning images, videos, multi-view data, and limited 4D footage, along with a tailored multi-dimensional training strategy. Our approach overcomes the limitations of previous methods based on generative adversarial networks or vanilla diffusion models, which struggle with complex motions, viewpoint changes, and generalization. Through extensive experiments, we demonstrate our method's ability to synthesize 360-degree realistic, coherent human motion videos, paving the way for advanced multimedia applications in areas such as virtual reality and animation.
Forward citations
Cited by 6 Pith papers
-
Interspatial Attention for Efficient 4D Human Video Generation
A new symmetric 3D-to-2D attention mechanism with relative positional encodings, plus a motion-tuned video VAE, improves controllable 4D human video generation.
-
CharacterShot: Controllable and Consistent 4D Character Animation
A new pipeline generates pose-controlled, view-consistent 4D character animations from one reference image and a 2D pose sequence, backed by a new 13,115-character dataset and benchmark.
-
DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds
DreamDance animates a single character artwork by reconstructing its background as a 3D Gaussian scene and then inpainting the animated character into the rendered video.
-
DevilSight: Augmenting Monocular Human Avatar Reconstruction through a Virtual Perspective
A monocular human avatar reconstruction method generates pseudo back-view videos with a fine-tuned diffusion model and uses them as extra training data for a 3D Gaussian avatar.
-
iDiT-HOI: Inpainting-based Hand Object Interaction Reenactment via Video Diffusion Transformer
A two-stage inpainting-based video diffusion transformer that reuses pretrained attention to reenact hand-object interactions with novel objects, reporting SOTA performance on Re-HOLD and a new in-the-wild dataset.
-
Human Motion Video Generation: A Survey
A comprehensive survey with a five-phase pipeline model for human motion video generation, covering over 200 papers and adding a new benchmark comparison of nine pose-guided methods.
Discussion (0). Continue with ORCID to comment.