Pith. sign in

REVIEW 5 major objections 4 minor 5 cited by

UCM unifies camera control and long-term memory in video world models by warping the positional encodings of reference and historical frames to target viewpoints, significantly outperforming prior methods on both.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:32 UTC pith:ZQAR5GPP

load-bearing objection UCM is a genuinely new combination of PE-Field warping, sparse dual-stream attention, and a point-cloud rendering trick for training; the mechanism is worth refereeing, but the empirical claims rest on unquantified depth/pose sensitivity and reimplemented, unreleased baselines. the 5 major comments →

arxiv 2602.22960 v2 pith:ZQAR5GPP submitted 2026-02-26 cs.CV

UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models

Tianxing Xu , Zixuan Wang , Guangyuan Wang , Li Hu , Zhongyi Zhang , Peng Zhang , Bang Zhang , Songhai Zhang This is my paper
classification cs.CV
keywords world modelvideo generationcamera controllong-term consistencypositional encoding warpingmemorydiffusion transformerpoint cloud rendering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that camera control and long-term memory in video-generation world models can be unified into a single mechanism: time-aware positional encoding warping. Instead of relying on implicit 3D priors learned from raw camera parameters or Plücker embeddings, UCM lifts historical frames to point clouds, projects them into target viewpoints, and rewrites each token's positional encoding to reflect its true 3D location. This gives every generated frame an explicit, token-level spatial link to reference and memory frames, yielding state-of-the-art camera controllability and long-term scene consistency. A dual-stream diffusion transformer and a point-cloud-based data curation strategy make the approach efficient and trainable on over 500K monocular web videos.

Core claim

The paper establishes that explicit, geometry-grounded token correspondence, injected by warping positional encodings from memory and reference frames into target viewpoints, is the right way to condition video generation for both camera control and long-term memory. UCM rewrites the 3D positional encoding of each clean token as [i, U, V] — the target frame index and the projected pixel coordinates of the token's 3D point — so the diffusion transformer sees exactly where each historical token should land. This replaces implicit conditioning (raw poses, Plücker embeddings) and outperforms both implicit methods and explicit point-cloud/TSDF conditioning on camera error, FVD, and view recall co

What carries the argument

Time-aware positional encoding warping (Eq. 5): depth maps and camera poses lift reference and memory frames into point clouds; each point is projected into the target camera to yield warped coordinates [i, U, V]; reference tokens are warped to every target viewpoint, while memory tokens are warped to their most relevant viewpoint. This supplies explicit spatio-temporal correspondence to the self-attention. The dual-stream diffusion transformer then applies a block-sparse attention mask: clean tokens attend only within their own frame, while noisy tokens attend to all noisy tokens plus only the clean tokens warped into their view, cutting the quadratic cost of memory conditioning.

Load-bearing premise

UCM assumes the per-frame depth maps and camera poses are accurate enough that projecting historical tokens into a target viewpoint places them at their true 3D locations; when depth or pose errors are large, the warped positional encodings misplace tokens and both camera control and memory consistency degrade.

What would settle it

Take a fixed synthetic scene with ground-truth depth, add controlled Gaussian noise to the depth maps used for warping while holding the rest of the pipeline fixed, and measure RotErr/TransErr and LPIPS. If metrics degrade steeply with noise, the mechanism's dependence on accurate geometry is confirmed; if they are flat, the model is learning to disregard bad warps. A second check: compare the warped coordinate maps to ground-truth correspondences; misalignment concentrated at object silhouettes and thin structures would pinpoint where the model is blind.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • World models for precise camera control and long-term consistency can be trained from ordinary monocular videos, since point-cloud rendering simulates scene revisits without multi-view or long-revisit datasets.
  • At 2.4 seconds per frame on an A100, interactive, camera-driven world simulation becomes practical for real-time-adjacent applications.
  • Explicit geometric correspondence via warped positional encodings is a viable and superior alternative to learned camera embeddings, with rotation error dropping to 1.01 degrees versus 2.22 for the next-best implicit method.
  • The block-sparse dual-stream design allows scaling to 20–40 memory frames with modest computational overhead, directly supporting longer-horizon scene consistency.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The warping mechanism could transfer to other conditional generation tasks where geometry is known — such as single-image novel view synthesis or scene editing — as a general 'geometric PE conditioning' recipe beyond world models.
  • The method's chief sensitivity is depth and pose quality; an uncertainty-aware version that downweights warped tokens with unreliable depth could make it robust in dynamic or textureless regions, a natural next step.
  • The heuristic of assigning each memory frame to its 'most relevant viewpoint' could be replaced by a learned assignment, which might improve memory injection for complex, non-repeating trajectories.
  • The data curation trick of rendering point clouds from random offsets to synthesize revisits is a cheap source of long-horizon supervision and may benefit other memory-based video generation models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes UCM, a framework for camera-controlled long-term video world modeling built on a diffusion transformer. The core idea is time-aware positional encoding warping (Sec. 4.1, Eq. 5): historical and reference frames are lifted to point clouds using estimated depth and camera poses, projected into target viewpoints, and their positional encodings are replaced with warped coordinates. This provides explicit token-level spatial correspondence for both camera control and memory injection. To keep the cost manageable, the authors design a dual-stream sparse-attention DiT block (Sec. 4.2) and a data-curation scheme that simulates scene revisiting from monocular videos (Sec. 4.3). The model is trained on 561K videos and evaluated on camera-control and long-term-consistency benchmarks, where it reports large gains over reimplemented baselines (Tables 1-2).

Significance. If the reported results are robust, UCM is a meaningful advance: it shows that explicit geometric warping of positional encodings can be integrated into a latent DiT video generator, unifying two previously separate capabilities (camera control and memory) without building full 3D scene representations. The design is clean and the ablation study is instructive. The paper also contributes a practical data-curation pipeline for long-horizon world models. However, the quantitative evaluation lacks error bars and significance testing, the camera-control metrics are estimated indirectly from predicted depth/pose, and the method's sensitivity to depth/pose noise is not analyzed. These gaps make the 'significantly outperforms' claim currently under-supported, though the underlying idea is plausible and worth pursuing.

major comments (5)
  1. [Sec. 5.2, Tables 1-2] No error bars, confidence intervals, or significance tests are reported for any of the quantitative results. Video generation is stochastic, and metrics such as FID/FVD are known to be noisy even with hundreds of videos. The claim of 'significantly outperforming' is therefore not statistically grounded. Please report means and standard deviations over at least three independent seeds and perform paired significance tests (e.g., bootstrap or Wilcoxon) for the key comparisons.
  2. [Sec. 5.1/5.2] Camera control metrics (RotErr, TransErr) are computed by extracting poses from generated videos using Depth Anything 3. This is an indirect evaluation: errors in the pose estimator on generated (possibly distorted) frames are conflated with actual camera-control errors. The paper should validate the metric on synthetic sequences with known camera trajectories (e.g., Tanks-and-Temples or synthetic MiraData sequences), or at least report the pose-estimation accuracy of Depth Anything 3 on the evaluation set. Without this, the reported RotErr/TransErr values are difficult to interpret.
  3. [Sec. 4.1, Eqs. 3-5; Sec. 5.1] The entire warping mechanism relies on per-frame depth maps and camera poses (STream3R at inference, Depth Anything 3 for training and evaluation). A depth or pose error directly misplaces the warped coordinates [U,V], and the block-sparse attention mask (Sec. 4.2) forces noisy tokens to attend only to clean tokens warped to the same camera view, so a mis-projected token actively injects content from the wrong 3D location. No sensitivity analysis is provided. Please add noise-injection experiments on clean data by perturbing depths and poses with calibrated noise, compare against oracle depth/pose, and show how RotErr/TransErr and visual metrics degrade. This is essential to support the claim that explicit geometry-grounded warping is superior to implicit conditioning.
  4. [Sec. 5.2, baseline reimplementation] All baselines (UCPE, C-a-M, VMem, VWM) are reimplemented by the authors on the same 1.3B-parameter foundation model and trained with UCM's data-curation strategy. While this is a controlled comparison, the 'state-of-the-art' claim is conditional on the quality of these reimplementations. Please state explicitly which of the original checkpoints/code were used (the text says weights are inaccessible for some), discuss any hyperparameter tuning, and phrase conclusions as 'under our reimplementation' where appropriate.
  5. [Sec. 5.3, Cycle Trajectory protocol] The cycle-trajectory evaluation compares newly generated frames to the model's own temporally symmetric generated frames, not to ground truth. This measures self-consistency and may inflate memory scores if the model learns to re-generate similar content rather than to track the true scene. The Memory Initialization protocol is direct, but the headline long-term memory results (Table 2, right block) rely on the cycle protocol. Please add an external consistency measure (e.g., feature matching with the initial reference frame or against a held-out ground-truth trajectory where available) and report those numbers alongside the self-consistency metrics.
minor comments (4)
  1. [Sec. 5.4, Table 3] The table format is confusing: the rows with checkmarks are not labeled clearly, especially the row '20 ✓' (is this with or without 'Dual' and 'Sparse'?). Please add a clearer legend or footnote describing what each checkmark denotes and what is absent in each ablation.
  2. [Sec. 4.1, Eq. 1] Definitions of x0 and x1 appear after the equation. Move them before the equation to avoid confusion, or state them inline in the equation's preamble.
  3. [Sec. 5.1, implementation details] The paper states that Depth Anything 3 is used for both training annotation and evaluation pose extraction, while STream3R is used at inference. Clarify why two different depth/pose estimators are used and whether the choice affects the warping accuracy or the evaluation fairness.
  4. [General] The abstract and conclusion repeatedly use the phrase 'significantly outperforms' without any statistical backing. Please either add the statistical evidence or soften the language to 'outperforms in our experiments'.

Circularity Check

1 steps flagged

No load-bearing circularity: the core PE-warping mechanism is a geometric construction from input depth/pose, but the Cycle Trajectory protocol partly evaluates the model against its own generated frames.

specific steps
  1. other [Sec. 5.3, Long-term Memory, Cycle Trajectory protocol]
    "Cycle Trajectory. Given the initial frame and a text prompt as conditions, we generate a long-term video that adheres to the cycle camera trajectory by making the camera return to the starting point along the same path in reverse order. For visual quality metrics, we evaluate whether newly generated frames match historical temporally symmetric generated frames."

    The reference for the view-recall metrics is the model's own temporally symmetric generated frames rather than an external ground-truth video. A model that drifts smoothly but consistently can still score well because the comparison baseline is itself. This is a mild evaluative circularity. It is also partial: the companion Memory Initialization protocol compares against ground-truth frames, so the central long-term-consistency claim retains independent grounding.

full rationale

UCM's central mechanism is Eq. 5: warped positional encodings W = [i, U, V] obtained by lifting input depth maps with input camera poses and projecting them into target viewpoints. This is an explicit geometric operation on the inputs, not a fitted parameter renamed as a prediction, and it does not by construction determine the evaluation metrics. Training uses 561k monocular videos and an external I2V base model, and evaluation is anchored to external benchmarks (RealEstate10K, Tanks & Temples, held-out Context-as-Memory). No load-bearing uniqueness theorem or ansatz is imported from the authors' own prior work; references to PE-Field, STream3R, Depth Anything 3, and Wan are external prior work. One shared-estimator caveat exists: Depth Anything 3 is used both to annotate training point clouds/poses and to extract camera poses for the camera-control metric, which could create a mild evaluation confound, but this is not a derivation-level circularity. The only protocol-level circularity is the Cycle Trajectory self-consistency metric. Because the Memory Initialization protocol directly compares against ground truth, the main claims remain independently grounded, so the overall circularity score is low.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The method relies on the accuracy of monocular depth/pose estimators, on a learned ability of a DiT to exploit warped positional encodings, and on point-cloud rendering being a faithful proxy for true revisits. These domain assumptions are the main cost of the contribution; no new physical or discrete entities are introduced.

free parameters (3)
  • Number of retrieved memory frames M = 20
    Hand-picked baseline after Tab. 3 ablation; memory quality generally improves with M up to 40, so 20 is a designer's trade-off, not derived.
  • Random camera offset Δc in data curation
    Sampled distribution for simulated revisits is not specified, but it directly shapes the training supervision.
  • Random temporal shift Δi
    Augmentation distance for warping rendered frames; exact distribution not specified.
axioms (4)
  • domain assumption Depth maps and camera poses estimated by STream3R / Depth Anything 3 are accurate enough for inverse projection (Eq. 3) and re-projection (Eq. 4) to establish correct spatial correspondence.
    The whole PE-warping mechanism inherits errors from these estimators; entered at Sec 4.1 Eq.3-4 and Sec 5.1.
  • domain assumption Warped positional encodings plus binary attention masks are a sufficient mechanism for a DiT to learn camera control and long-term memory from data.
    The method's effectiveness depends on the model being able to exploit the warped coordinates; this is trained, not derived.
  • domain assumption Point-cloud rendering with random camera offsets and temporal shifts simulates long-term scene revisiting well enough to train a generalizable world model.
    Data curation in Sec 4.3 relies on this; no analysis of how rendered holes or reconstruction artifacts affect transfer.
  • standard math Rectified flow objective (Eq. 1) and the pretrained Wan2.1-based I2V/VAE backbone provide a suitable base for learning these conditioning signals.
    Assumed from prior work, used in Sec 3 and Sec 5.1.

pith-pipeline@v1.3.0-alltime-deepseek · 14709 in / 14235 out tokens · 135359 ms · 2026-08-02T20:32:18.743490+00:00 · methodology

0 comments
read the original abstract

World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation.

Figures

Figures reproduced from arXiv: 2602.22960 by Bang Zhang, Guangyuan Wang, Li Hu, Peng Zhang, Songhai Zhang, Tianxing Xu, Zhongyi Zhang, Zixuan Wang.

Figure 1
Figure 1. Figure 1: Visual results of our proposed UCM. Given a reference image, a user-specified camera trajectory and a prompt, UCM enables camera-controlled, long-term consistent world generation via time-aware positional encoding warping. Abstract World models based on video generation demonstrate remarkable potential for simulating interactive environments but face persis￾tent difficulties in two key areas: maintaining l… view at source ↗
Figure 2
Figure 2. Figure 2: An overview of our proposed UCM. Given previously generated frames and a specific camera trajectory as input, UCM encodes the historical frames into clean tokens to condition the denoising of noisy tokens. For camera control and memory injection, the framework proposes time-aware positional encoding warping to establish spatio-temporal correspondence and an efficient dual-stream transformer architecture fo… view at source ↗
Figure 3
Figure 3. Figure 3: The architecture of UCM DiT-block. Each noisy token attends to all other noisy tokens and is guided by clean tokens via time-aware warped PEs, implemented through KV concatenation. For the clean tokens, each token attends only to other clean to￾kens within the same frame using original PEs. This block-sparse attention mask (here, with 𝑘𝑗 = 𝑗 for visualization) enables camera control and memory guidance wit… view at source ↗
Figure 4
Figure 4. Figure 4: Simulated revisiting from different viewpoints. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visual comparison of camera controllability. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visual comparison of long-term memory under two evaluation settings. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Supplementary visual comparison of long-term memory preservation. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Supplementary visual results of our proposed UCM. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Echo-Memory: A Controlled Study of Memory in Action World Models

    cs.CV 2026-06 unverdicted novelty 6.0

    A controlled study finds that block-wise state-space recurrence outperforms other memory designs for open-domain scene return in action-conditioned video models, and that standard replay metrics do not adequately meas...

  2. Geometry-Aware Implicit Memory for Video World Models

    cs.CV 2026-06 unverdicted novelty 6.0

    GIM-World adds a camera-queryable geometry distillation head and pruning rule to implicit memory in video world models, claiming better long-horizon geometric consistency on the MIND benchmark than explicit and implic...

  3. Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interaction

    cs.CV 2026-07 conditional novelty 5.0

    A distilled video model jointly controls multi-scale upper-body motion latents and two discrete contact states to generate real-time human–object interaction at 25 FPS.

  4. PE-Field 4D: Video Generation Models as Canvas

    cs.CV 2026-07 conditional novelty 5.0

    Warping reference tokens' positional encodings into the target view, with depth offsets and frame-level compression fixes, improves geometry-aware camera control in video diffusion transformers.

  5. SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer

    cs.CV 2026-05 unverdicted novelty 5.0

    SANA-WM is a 2.6B-parameter efficient world model that synthesizes minute-scale 720p videos with 6-DoF camera control, trained on 213K public clips in 15 days on 64 H100s and runnable on single GPUs at 36x higher thro...

Reference graph

Works this paper leans on

67 extracted references · 27 linked inside Pith · cited by 5 Pith papers

  1. [1]

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos J Storkey, Tim Pearce, and François Fleuret. 2024. Diffusion for world modeling: Visual details matter in atari.Advances in Neural Information Processing Systems37 (2024), 58757–58791

    2024

  2. [2]

    Genesis Authors. 2024. Genesis: A universal and generative physics engine for robotics and beyond, December 2024.URL https://github. com/Genesis-Embodied- AI/Genesis9 (2024)

    2024

  3. [3]

    Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B Lindell, and Sergey Tulyakov

  4. [4]

    Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al . 2025. Recammaster: Camera-controlled generative rendering from a single video.arXiv preprint arXiv:2503.11647(2025)

    Pith/arXiv arXiv 2025

  5. [5]

    Yunpeng Bai, Haoxiang Li, and Qixing Huang. 2025. Positional encoding field. arXiv preprint arXiv:2510.20385(2025)

    arXiv 2025

  6. [6]

    Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. 2025. Navigation world models. InProceedings of the Computer Vision and Pattern Recognition Conference. 15791–15801

    2025

  7. [7]

    Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Ming- ming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, et al. 2025. Go-with- the-flow: Motion-controllable video diffusion models using real-time warped noise. InProceedings of the Computer Vision and Pattern Recognition Conference. 13–23

    2025

  8. [8]

    Chenjie Cao, Jingkai Zhou, Shikai Li, Jingyun Liang, Chaohui Yu, Fan Wang, Xiangyang Xue, and Yanwei Fu. 2025. Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation. InProceedings of the SIGGRAPH Asia 2025 Conference Papers. 1–12

    2025

  9. [9]

    Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. 2024. Gamegen-x: Interactive open-world game video generation.arXiv preprint arXiv:2411.00769(2024)

    Pith/arXiv arXiv 2024

  10. [10]

    Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. 2024. Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems37 (2024), 24081–24125

    2024

  11. [11]

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. 2017. Scannet: Richly-annotated 3d reconstructions of indoor scenes. InProceedings of the IEEE conference on computer vision and pattern recognition. 5828–5839

    2017

  12. [12]

    Etched Decart, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen. 2024. Oasis: A universe in a transformer.URL: https://oasis-model. github. io(2024)

    2024

  13. [13]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. InForty- first international conference on machine learning

    2024

  14. [14]

    Wanquan Feng, Jiawei Liu, Pengqi Tu, Tianhao Qi, Mingzhen Sun, Tianxiang Ma, Songtao Zhao, Siyu Zhou, and Qian He. 2024. I2vcontrol-camera: Precise video camera control with adjustable motion strength.arXiv preprint arXiv:2411.06525 (2024)

    Pith/arXiv arXiv 2024

  15. [15]

    Jianxiong Gao, Zhaoxi Chen, Xian Liu, Junhao Zhuang, Chengming Xu, Jian- feng Feng, Yu Qiao, Yanwei Fu, Chenyang Si, and Ziwei Liu. 2025. LongVie 2: Multimodal Controllable Ultra-Long Video World Model.arXiv preprint arXiv:2512.13604(2025)

    arXiv 2025

  16. [16]

    Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. 2025. Seedance 1.0: Exploring the Boundaries of Video Generation Models.arXiv preprint arXiv:2506.09113(2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Yuchao Gu, Weijia Mao, and Mike Zheng Shou. 2025. Long-context autoregressive video modeling with next-frame prediction.arXiv preprint arXiv:2503.19325 (2025)

    Pith/arXiv arXiv 2025

  18. [18]

    Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Cheng Lin, Ziwei Liu, et al . 2025. Diffusion as shader: 3d-aware video diffusion for versatile video generation control. InProceedings of the Spe- cial Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers. 1–12

    2025

  19. [19]

    Xiangyu Guo, Zhanqian Wu, Kaixin Xiong, Ziyang Xu, Lijun Zhou, Gangwei Xu, Shaoqing Xu, Haiyang Sun, Bing Wang, Guang Chen, et al. 2025. Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistency.arXiv preprint arXiv:2506.07497(2025)

    Pith/arXiv arXiv 2025

  20. [20]

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. 2024. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101(2024)

    Pith/arXiv arXiv 2024

  21. [21]

    Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. 2025. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models.arXiv preprint arXiv:2503.10592(2025)

    Pith/arXiv arXiv 2025

  22. [22]

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems30 (2017)

    2017

  23. [23]

    Jonathan Ho and Tim Salimans. 2022. Classifier-Free Diffusion Guidance.CoRR abs/2207.12598 (2022). arXiv:2207.12598 doi:10.48550/ARXIV.2207.12598

    Pith/arXiv arXiv doi:10.48550/arxiv.2207.12598 2022

  24. [24]

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. 2025. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. arXiv preprint arXiv:2506.08009(2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xintao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. 2024. Miradata: A large-scale video dataset with long durations and structured captions.Advances in Neural Information Processing Systems37 (2024), 48955–48970

    2024

  26. [26]

    Anssi Kanervisto, Dave Bignell, Linda Yilin Wen, Martin Grayson, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Tabish Rashid, Tim Pearce, Yuhan Cao, et al . 2025. World and human action models towards gameplay ideation.Nature638, 8051 (2025), 656–663

    2025

  27. [27]

    Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114(2013)

    Pith/arXiv arXiv 2013

  28. [28]

    Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. 2017. Tanks and temples: Benchmarking large-scale scene reconstruction.ACM Transactions on Graphics (ToG)36, 4 (2017), 1–13

    2017

  29. [29]

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. 2024. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603 (2024)

    Pith/arXiv arXiv 2024

  30. [30]

    Yushi Lan, Yihang Luo, Fangzhou Hong, Shangchen Zhou, Honghua Chen, Zhaoyang Lyu, Shuai Yang, Bo Dai, Chen Change Loy, and Xingang Pan. 2025. Stream3r: Scalable sequential 3d reconstruction with causal transformer.arXiv preprint arXiv:2508.10893(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    Runjia Li, Philip Torr, Andrea Vedaldi, and Tomas Jakab. 2025. VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory.arXiv preprint arXiv:2506.18903(2025)

    Pith/arXiv arXiv 2025

  32. [32]

    Teng Li, Guangcong Zheng, Rui Jiang, Shuigen Zhan, Tao Wu, Yehao Lu, Yining Lin, Chuanyun Deng, Yepan Xiong, Min Chen, et al. 2025. Realcam-i2v: Real- world image-to-video generation with interactive complex camera control. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 28785– 28796

    2025

  33. [33]

    Haotong Lin, Sili Chen, Junhao Liew, Donny Y Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. 2025. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647(2025)

    Pith/arXiv arXiv 2025

  34. [34]

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al . 2024. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 22160–22169

    2024

  35. [35]

    Tianran Liu, Shengwen Zhao, and Nicholas Rhinehart. 2025. Towards founda- tional LiDAR world models with efficient latent flow matching.arXiv preprint arXiv:2506.23434(2025). Unifying Camera Control and Memory with Time-aware PE Warping for World Models

    arXiv 2025

  36. [36]

    Yifan Liu, Zhiyuan Min, Zhenwei Wang, Junta Wu, Tengfei Wang, Yixuan Yuan, Yawei Luo, and Chunchao Guo. 2025. Worldmirror: Universal 3d world recon- struction with any-prior prompting.arXiv preprint arXiv:2510.10726(2025)

    arXiv 2025

  37. [37]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101(2017)

    Pith/arXiv arXiv 2017

  38. [38]

    Baorui Ma, Huachen Gao, Haoge Deng, Zhengxiong Luo, Tiejun Huang, Lulu Tang, and Xinlong Wang. 2025. You see it, you got it: Learning 3d creation on pose-free videos at scale. InProceedings of the Computer Vision and Pattern Recognition Conference. 2016–2029

    2025

  39. [39]

    J Parker-Holder, P Ball, J Bruce, V Dasagi, K Holsheimer, C Kaplanis, A Mo- ufarek, G Scully, J Shar, J Shi, et al . 2024. Genie 2: A large-scale foundation world model.URL: https://deepmind. google/discover/blog/genie-2-a-large-scale- foundation-world-model(2024)

    2024

  40. [40]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transform- ers. InProceedings of the IEEE/CVF international conference on computer vision. 4195–4205

    2023

  41. [41]

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. 2021. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference on computer vision. 10912– 10922

    2021

  42. [42]

    Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. 2025. History-guided video diffusion.arXiv preprint arXiv:2502.06764(2025)

    Pith/arXiv arXiv 2025

  43. [43]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing568 (2024), 127063

    2024

  44. [44]

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. 2018. Towards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717(2018)

    Pith/arXiv arXiv 2018

  45. [45]

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. 2024. Diffusion models are real-time game engines.arXiv preprint arXiv:2408.14837(2024)

    Pith/arXiv arXiv 2024

  46. [46]

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. 2025. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314(2025)

    Pith/arXiv arXiv 2025

  47. [47]

    Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Yi Zhang, Chang Zeng, Yanxi Zhou, et al. 2025. Spatialvid: A large-scale video dataset with spatial annotations.arXiv preprint arXiv:2509.09676(2025)

    arXiv 2025

  48. [48]

    Zhou Wang and Alan C Bovik. 2002. A universal image quality index.IEEE signal processing letters9, 3 (2002), 81–84

    2002

  49. [49]

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing13, 4 (2004), 600–612

    2004

  50. [50]

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. 2024. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers. 1–11

    2024

  51. [51]

    Tong Wu, Shuai Yang, Ryan Po, Yinghao Xu, Ziwei Liu, Dahua Lin, and Gordon Wetzstein. 2025. Video World Models with Long-term Spatial Memory.arXiv preprint arXiv:2506.05284(2025)

    Pith/arXiv arXiv 2025

  52. [52]

    Zeqi Xiao, Yushi Lan, Yifan Zhou, Wenqi Ouyang, Shuai Yang, Yanhong Zeng, and Xingang Pan. 2025. Worldmem: Long-term consistent world simulation with memory.arXiv preprint arXiv:2504.12369(2025)

    arXiv 2025

  53. [53]

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al . 2024. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072(2024)

    Pith/arXiv arXiv 2024

  54. [54]

    Meng You, Zhiyu Zhu, Hui Liu, and Junhui Hou. 2024. Nvs-solver: Video diffusion model as zero-shot novel view synthesizer.arXiv preprint arXiv:2405.15364(2024)

    Pith/arXiv arXiv 2024

  55. [55]

    Jiwen Yu, Jianhong Bai, Yiran Qin, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. 2025. Context as memory: Scene-consistent interactive long video generation with memory retrieval.arXiv preprint arXiv:2506.03141 (2025)

    Pith/arXiv arXiv 2025

  56. [56]

    Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan. 2025. Trajectorycrafter: Redi- recting camera trajectory for monocular videos via diffusion models.arXiv preprint arXiv:2503.05638(2025)

    Pith/arXiv arXiv 2025

  57. [57]

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. 2024. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048(2024)

    Pith/arXiv arXiv 2024

  58. [58]

    Funkhouser

    Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas A. Funkhouser. 2017. 3DMatch: Learning Local Geometric Descrip- tors from RGB-D Reconstructions. In2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 199–208. doi:10.1109/CVPR.2017.29

    doi:10.1109/cvpr.2017.29 2017

  59. [59]

    Shangjin Zhai, Zhichao Ye, Jialin Liu, Weijian Xie, Jiaqi Hu, Zhen Peng, Hua Xue, Danpeng Chen, Xiaomeng Wang, Lei Yang, et al. 2025. Stargen: A spatiotem- poral autoregression framework with video diffusion model for scalable and controllable scene generation. InProceedings of the Computer Vision and Pattern Recognition Conference. 26822–26833

    2025

  60. [60]

    Cheng Zhang, Boying Li, Meng Wei, Yan-Pei Cao, Camilo Cruz Gambardella, Dinh Phung, and Jianfei Cai. 2025. Unified Camera Positional Encoding for Controlled Video Generation.arXiv preprint arXiv:2512.07237(2025)

    arXiv 2025

  61. [61]

    Lvmin Zhang and Maneesh Agrawala. 2025. Packing input frame context in next- frame prediction models for video generation.arXiv preprint arXiv:2504.12626 (2025)

    arXiv 2025

  62. [62]

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang

  63. [63]

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely

  64. [64]

    Zheng Zhu, Xiaofeng Wang, Wangbo Zhao, Chen Min, Bohan Li, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, et al. 2024. Is sora a world simulator? a comprehensive survey on general world models and beyond.arXiv preprint arXiv:2405.03520(2024). Tian-Xing et al. Memory Initialization Cycle Trajectory UCM(Ours)GT C-a-MVMemVWM UCM(Ours) C-a-MVMemVWM ...

    arXiv 2024

  65. [66]

    arXiv preprint arXiv:1805.09817(2018)

    Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817(2018)

    Pith/arXiv arXiv 2018

  66. [2018]

    InProceedings of the IEEE conference on computer vision and pattern recognition

    The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition. 586–595

  67. [2025]

    InProceedings of the Computer Vision and Pattern Recognition Conference

    Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. InProceedings of the Computer Vision and Pattern Recognition Conference. 22875–22889