REVIEW 6 cited by
4K4DGen: Panoramic 4D Generation at 4K Resolution
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
The blooming of virtual reality and augmented reality (VR/AR) technologies has driven an increasing demand for the creation of high-quality, immersive, and dynamic environments. However, existing generative techniques either focus solely on dynamic objects or perform outpainting from a single perspective image, failing to meet the requirements of VR/AR applications that need free-viewpoint, 360$^{\circ}$ virtual views where users can move in all directions. In this work, we tackle the challenging task of elevating a single panorama to an immersive 4D experience. For the first time, we demonstrate the capability to generate omnidirectional dynamic scenes with 360$^{\circ}$ views at 4K (4096 $\times$ 2048) resolution, thereby providing an immersive user experience. Our method introduces a pipeline that facilitates natural scene animations and optimizes a set of dynamic Gaussians using efficient splatting techniques for real-time exploration. To overcome the lack of scene-scale annotated 4D data and models, especially in panoramic formats, we propose a novel \textbf{Panoramic Denoiser} that adapts generic 2D diffusion priors to animate consistently in 360$^{\circ}$ images, transforming them into panoramic videos with dynamic scenes at targeted regions. Subsequently, we propose \textbf{Dynamic Panoramic Lifting} to elevate the panoramic video into a 4D immersive environment while preserving spatial and temporal consistency. By transferring prior knowledge from 2D models in the perspective domain to the panoramic domain and the 4D lifting with spatial appearance and geometry regularization, we achieve high-quality Panorama-to-4D generation at a resolution of 4K for the first time.
Forward citations
Cited by 6 Pith papers
-
What Makes for Text to 360-degree Panorama Generation with Stable Diffusion?
Freezing query and key LoRA weights and strengthening the output weight with a mixture of experts gives a memory-efficient, state-of-the-art text-to-360-degree-panorama generator.
-
Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation
Hallo4D uses vision-language models to detect and correct spatial and temporal mistakes in AI-generated 3D and 4D content, improving consistency without retraining the base generators.
-
OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenes
OmniX trains separate LoRA adapters on FLUX.1-dev so one framework handles panorama generation, intrinsic perception (depth, normals, albedo, roughness, metallic), and completion, then feeds the maps into PBR-ready 3D scenes.
-
Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images
The authors introduce OHF2024, a human-annotated database of AI-generated omnidirectional images, and BLIP2OIQA plus BLIP2OISal models that achieve the best reported scores on this database for multi-perspective quali...
-
TanDiT: Tangent-Plane Diffusion Transformer for High-Quality 360{\deg} Panorama Generation
TanDiT generates high-quality 360-degree panoramas by jointly generating grids of tangent-plane views with a single diffusion transformer and refining them with a pretrained model.
-
PanoWan: Lifting Diffusion Video Generation Models to 360{\deg} with Latitude/Longitude-aware Mechanisms
PanoWan adapts the Wan 2.1 text-to-video model to generate seamless 360-degree videos by remapping initial noise, rotating the latent grid during denoising, and padding the latent before VAE decoding, trained on a new...
Discussion (0). Continue with ORCID to comment.