{"id":"0e964a02-1484-4d93-b1de-4a19a2d0b7a9","arxiv_id":"2506.01103","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A 4D autoregressive world model that conditions future RGB frame prediction on its own predicted depth and camera pose, reducing drift in long-horizon video generation.","lead":"DeepVerse is a video generation model that predicts future frames together with their 3D geometry, using these geometric estimates to reduce errors in long videos. It shows that adding depth and camera-pose information to an autoregressive video model improves consistency over hundreds of frames.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Self-generated geometry is fed back at every inference step but its accuracy is never measured; if predicted depth/raymap errors compound, the claimed drift reduction could be an artifact of teacher-forced training.","rationale":"The reader's weakest assumption is the right one to probe. DeepVerse's contribution is a feedback loop that carries geometry, and the clearest risk is that the loop is never stress-tested: inference feeds back the model's own predictions, while training appears to use clean synthetic 4D states. This is a specific exposure-bias risk for the claimed drift reduction, not a vague concern about geometry quality. The existing evidence is not negligible — Table 1 shows consistent improvements with depth, the synthetic data pipeline provides precise geometry supervision, and the architecture details are concrete — but the differences are small, there are no error bars, and no intrinsic geometry metric is reported. A substitution experiment with ground-truth geometry would decide whether the improvement is a genuine property of the feedback mechanism or a side effect of teacher forcing. I would therefore keep the reader's CONDITIONAL verdict: the method is plausible and supported internally, but the central claim remains conditional on a test the paper has not run.","tokens_in":14199,"tokens_out":6914,"duration_ms":77730,"concrete_test":"On a fixed set of test rollouts, at frame t=60, branch the generation: continue one branch with the model's own predicted depth and raymap and the other with simulator ground-truth geometry, then compare downstream VBench/FVD. If the ground-truth branch is substantially better, the model is not robust to its own geometry error; if there is no difference, the geometry is not doing causal work. Complement this with per-step absolute relative depth error and camera pose error curves across 32/64/96/128 frames to show whether geometric errors compound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The mechanism in Eq. (2) conditions each autoregressive step on previously predicted geometry, and Algorithm 1 makes this a true feedback loop: each new inference window is generated from states produced by the model itself, including its own depth and raymap estimates, and the spatial retrieval ψ is built from predicted camera poses. Yet the paper reports no intrinsic accuracy for these geometric estimates at any horizon, and no ablation that replaces predicted geometry with ground-truth geometry during rollouts. Training is described over ground-truth synthetic clips (Sec. A.4) with at most condition augmentation on historical frames (Sec. 3.1); the paper describes no training on model-generated rollouts. If predicted depth/raymap carry systematic bias, the next-step network sees inputs outside its training distribution, and errors can compound instead of being corrected. The central quantitative evidence in Table 1 (e.g., subject consistency 0.869 vs. 0.836 at 60 frames) is consistent with this mechanism helping under teacher-forced conditions, but it does not establish that the self-conditioning loop is stable. The 'w/o depth' baseline also feeds back self-generated raymap, so it is not a test of the loop either. Because the abstract's 'significantly reduces drift' depends on the feedback loop being robust, this is the load-bearing assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"DeepVerse proposes an autoregressive 4D world model that augments visual observation tokens with depth and raymap geometric tokens, and maintains a geometry-aware memory for long-horizon inference. The model is trained on synthetic gameplay-like data with ground-truth camera and depth annotations, initialized from a pretrained video diffusion backbone, and evaluated on VBench metrics and FVD for generated 4D future sequences. The paper claims that explicit geometric conditioning reduces drift, improves temporal consistency, and preserves spatial memory, and supports this with two internal ablations: an architecture-level comparison (token-wise vs. channel-wise history concatenation) and a modality-level comparison (with vs. without depth), plus qualitative demonstrations on game images and AI-generated images.","tokens_in":14451,"tokens_out":4266,"duration_ms":46302,"significance":"If the central claims were fully validated, DeepVerse would make a useful contribution by showing that explicit geometric state can be folded into an autoregressive video-generation pipeline and used for long-range consistency, and its geometry-aware memory retrieval is an interesting design direction. The paper has strengths: it reports a large-scale synthetic data pipeline with precise geometry labels, gives detailed architecture/training descriptions, and includes dedicated ablations for the depth modality and for the history-integration architecture. However, the evidence currently falls short of the paper's abstract-level claims because the evaluation is entirely self-referential, the feedback loop over predicted geometry is not stress-tested, and the reported quantitative differences lack uncertainty quantification and are partially confounded by design choices.","major_comments":[{"comment":"The paper's central claim that explicit geometry 'significantly reduces drift' rests on the stability of a closed-loop process in which every autoregressive step conditions on the model's own previously predicted depth and raymap (Algorithm 1; Eq. (2)). However, the paper never measures the intrinsic accuracy of these predicted geometry estimates, never ablates predicted geometry against ground-truth geometry during rollouts, and describes training only on ground-truth clips (Sec. A.4). If predicted depth/raymap carry systematic bias, the next-step network receives inputs outside its training distribution and errors could compound rather than be corrected; the depth ablation in Table 1 does not resolve this because both arms also feed back self-generated raymap, so the stability of the 4D feedback loop is unverified.","section":"Sec. 2.4, Eq. (2), Algorithm 1"},{"comment":"The architecture comparison that motivates the final design is confounded: Model 1 uses SD3-medium initialization, 7 historical frames, an image VAE, and batch size 512, while Model 2 uses Pyramid-Flow initialization, a 57-frame protocol with 8x temporal compression, and batch size 256 (Sec. 3.1 and Appendix A.1). The reported superiority of token-wise concatenation (Model 2) could be due to any of these differences, not necessarily the token concatenation strategy itself; therefore the claim that 'temporal feature aggregation within single tokens exacerbates error accumulation phenomena' (Sec. 3.1) is not established by these data.","section":"Sec. 3.1, Fig. 4"},{"comment":"The central quantitative evidence for the depth modality is a set of single point estimates with no standard deviations, no number of seeds, and no significance tests. For example, subject consistency at 60 frames is reported as 0.86939 with depth versus 0.83602 without depth; a difference of roughly 0.03 may be within run-to-run noise at the reported training scale (23,000 A100 GPU hours, Sec. A.6). Similarly, the FVD curves in Fig. 5a provide no confidence intervals. Without error bars or repeated-run statistics, the strength of the claim 'significantly reduces drift' is disproportionate to the evidence presented.","section":"Sec. 3.2, Table 1, Fig. 5a"},{"comment":"The geometry-aware memory mechanism is a stated core contribution, but its evaluation is only qualitative. The paper presents a visualization (Fig. 7) but provides no quantitative comparison of long-horizon generation with versus without the spatial condition, no metric for the accuracy of the retrieved states, and no ablation isolating the memory retrieval from the depth/raymap-conditioning itself. Consequently, the claim of an 'effective solution' for preserving long-term spatial consistency (Sec. 3.2) is not yet supported by the reported experiments.","section":"Sec. 3.2, 'Spatial Memory', Fig. 7"},{"comment":"There are no comparisons to existing interactive world models such as GameNGen, Oasis, UniSim, or Cosmos, nor to the geometric unified world model Aether, on any shared benchmark or metric. The abstract states 'substantial improvements in prediction accuracy, visual realism, and scene rationality,' but the experiments compare only the method against its own ablated variants on in-distribution synthetic data. Since the paper positions DeepVerse against these systems in Related Works, at least a quantitative comparison on a common task or an explicit statement of why such a comparison is infeasible is needed to support the claimed improvements.","section":"Sec. 3.3 and Related Works"}],"minor_comments":[{"comment":"There is a typo in the paragraph after Eq. (1): 'discribe the methodology' should be 'describe the methodology'.","section":"Sec. 2.1"},{"comment":"The rendered table merges the frame-count column with the first metric column (e.g., '600.86939' should read '60 | 0.86939'), making the numeric values difficult to parse; please fix the table formatting.","section":"Table 1"},{"comment":"The definition of depth encoding would be clearer if written as e_t = sqrt(1/d_t) rather than 'et = p 1/dt' (which could be misread as p times 1/d_t), and the distinction between 'depth d_t' and the encoded 'square root of disparity e_t' should be stated explicitly.","section":"Sec. 2.2, 4D Representation"},{"comment":"The claim of being 'the first to incorporate 4D representations into auto-regressive world models' should be positioned more carefully relative to Aether [44], which already uses depth and raymap-based geometric representations in a unified world model; please specify the precise architectural or methodological difference that makes DeepVerse the first in the claimed category.","section":"Sec. 1 and Sec. 4"},{"comment":"The qualitative evaluation in Fig. 6 uses images from games, real-world photos, and AI-generated images, but all training is on synthetic data; the paper's limitation statement acknowledges restricted real-world generalization, yet the abstract still says 'diverse scenarios.' Please soften the abstract wording to match the actual evaluation scope.","section":"Sec. 3.3"},{"comment":"The caption of Fig. 3 appears garbled in the text ('right front right rear right backward left front left rear left clockwise counter clockwise'); please correct the figure caption to clearly label the movement and rotation classes.","section":"Sec. 2.3"},{"comment":"The raymap compression strategy (keyframe raymap with linear interpolation) is described only in the appendix but is used in the main model; please reference it in Sec. 2.2 so the main text's '4D Representation' description is self-contained.","section":"Sec. A.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well-written in parts and the dataset/architecture details are thorough, but the evaluation is entirely internal and the closed-loop geometry feedback is the main technical risk. The paper cites several own-group works (Aether, GigaGS) and should clarify the novelty boundary; the claim of being the first autoregressive 4D world model should be checked against the updated version of Aether and other recent work. I recommend major revision with emphasis on adding rollout-stability analysis, error bars, and external baselines; these are feasible within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DeepVerse is a legitimate integration of explicit geometry into autoregressive video world models. The combination—feeding previously predicted depth and raymaps into a diffusion transformer, plus a geometry-aware memory retrieval—is new relative to Aether and WorldMem, and the depth ablation in Table 1 gives some internal evidence that the added modality helps. The paper also does the right thing in stating its synthetic-data limitation. That said, the central claim that explicit geometry 'significantly reduces drift' is not backed by the evidence as it stands. There are no error bars, no comparisons to GameNGen, Oasis, UniSim, or Cosmos, and the architecture comparison in Section 3.1 is confounded: Model 1 and Model 2 differ in initialization (SD3 vs Pyramid-Flow), temporal compression, and token counts. More importantly, the stress-test concern is on target: Algorithm 1 feeds the model's own predicted depth and raymaps back at every step, but the paper never measures the accuracy of those predictions or tests what happens when they drift. Training is on ground-truth clips with condition augmentation only, so the self-conditioning loop may behave differently at inference. The 'w/o depth' baseline still feeds back raymap, so it does not test loop stability either. The abstract's 'first' and 'substantial' are overclaims. This is a promising direction, and the authors clearly know the pieces. It deserves a serious referee, but a referee should send it back for external baselines, error bars, and at minimum an ablation that replaces predicted geometry with ground truth during rollouts, plus a measurement of geometry error over time. As it stands, I would not cite it as evidence that geometry-aware feedback works, but I would cite it as an example of the approach.","headline":"DeepVerse is a sensible new combination of explicit geometry and autoregressive video generation, but its headline claim rests on an internal ablation that never measures the accuracy or stability of the very geometry it feeds back.","tokens_in":15004,"tokens_out":2017,"would_cite":false,"duration_ms":23254,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DeepVerse claims that feeding predicted depth and camera maps back into each step of an autoregressive video model reduces drift and holds long-term spatial consistency.","keywords":["world models","autoregressive video generation","4D representation","depth prediction","raymap camera pose","geometry-aware memory","temporal consistency","drift reduction"],"falsifier":"A decisive test is to feed ground-truth depth and raymaps at every autoregressive step instead of the model's own predictions: if the consistency metrics do not improve, the causal story that geometry carries the drift reduction is falsified; a lighter check is to plot predicted-geometry error against horizon and see whether it grows while visual consistency also breaks.","tokens_in":14002,"feed_emoji":"🎬","tokens_out":9110,"duration_ms":72935,"temperature":0.7,"pith_summary":"DeepVerse is an autoregressive video-generation world model that explicitly feeds predicted geometry—depth and camera-ray maps—from past timesteps into each future prediction, alongside the usual RGB frames. The paper's central claim is that this composite 4D state (visual observation plus geometry) gives the model a stable coordinate anchor, so errors compound more slowly and long-generated sequences keep their spatial layout, subject identity, and background consistency. The authors support the claim with ablations showing that removing the depth channel lowers consistency metrics, and with a geometry-aware memory mechanism that retrieves spatially overlapping past states to fight forgetting. If the claim holds, interactive world models can generate much longer, reliable futures from a single image than purely visual autoregressive models.","feed_headline":"Depth maps in video world models cut drift over long horizons","feed_subtitle":"The paper shows depth plus camera pose keeps scenes and subjects steady across hundreds of generated frames.","key_machinery":"The central object is the composite 4D state representation $\\hat{s}_t = (v_t, g_t)$, where $v_t$ is the RGB frame and $g_t$ is the geometric channel formed by depth $d_t$ (parameterized as square root of disparity, $e_t = \\sqrt{1/d_t}$) and a raymap $c_t$ that encodes camera position and ray directions. The raymap lets the model recover camera intrinsics and extrinsics, so all generated frames can be aligned to a single global coordinate frame anchored at the first image. This state is what carries the argument: because every prediction step is conditioned on a self-consistent geometric scaffold rather than pixels alone, the model can keep scale, viewpoint, and scene layout coherent, and the same geometry enables the memory-retrieval function $\\psi$ that selects which past observation to re-inject as spatial context.","core_discovery":"The discovery DeepVerse is trying to establish is that a world model does not have to treat video as raw pixels alone: by predicting, at each timestep, an observation composed of RGB, depth, and a raymap encoding camera pose, and by conditioning the next autoregressive step on its own previous geometry estimates, the model reduces cumulative drift and scale ambiguity and keeps scenes consistent over hundreds of frames. The state is written as $\\hat{s}_t = (v_t, g_t)$ with $g_t$ collecting depth $d_t$ and camera viewpoint $c_t$, and the transition is $f_\\theta = P(\\hat{s}_{t+1:t+k} \\mid a_t, \\hat{s}_t, \\hat{s}_{t-m:t-1}, \\psi(\\hat{s}_{0:t-m-1}))$, where $\\psi$ is a spatial-memory retrieval selecting the historically observed state most aligned with the current camera pose. On VBench metrics the depth-modality variant scores higher in subject and background consistency at 60 and 120 frames than the no-depth baseline, which the paper reads as evidence that explicit geometry, not just extra visual context, is the active ingredient.","pith_inferences":["A sharper test of the causal role of geometry would be to compare models with identical visual input but different geometric conditioning strength, or to inject deliberately corrupted geometry; the paper's with-depth versus without-depth ablation is a start but does not isolate geometry accuracy.","If depth predictions are accurate early but degrade later, a hybrid that trusts geometry only when it is confident could extend the reliable horizon further than the current method.","The approach may transfer to embodied settings: an agent that models state as visual-plus-geometry and retrieves past overlapping views could use the same memory mechanism for spatial localization and path re-entry.","The reliance on synthetic data, which the authors state as a limitation, means the claimed benefits are measured in simulation; the generalization gap to real-world video is an open testable question."],"forward_implications":["Long-horizon generation: with geometry conditioning the model can roll out hundreds of frames from one image while keeping subject and background consistency higher than a visual-only baseline.","Scale ambiguity is addressed: since camera pose and depth are predicted jointly, novel views are extrapolated in a consistent 3D frame rather than being reinvented per frame.","Geometry-aware memory retrieval lets the model revisit spatially overlapping past states, preserving layout across far-apart timesteps and reducing forgetting.","Controller input can be expressed as text (for example, perspective changes computed from camera pose), so the pretrained text-conditioned generation is reused rather than adding new sensor modalities.","The token-wise architectural variant outperforms the channel-wise one on nearly all VBench metrics, indicating that separating temporal tokens mitigates autoregressive error accumulation."],"supporting_citations":[{"why":"Supplies the geometric representation: depth parameterized as square-root disparity and the raymap for camera pose.","marker":"[44]"},{"why":"Provides the base architecture and the token-wise concatenation scheme used for the chosen model.","marker":"[25]"},{"why":"Provides the MM-DiT backbone and the text-conditioning design for both tested architectures.","marker":"[13]"},{"why":"Provides the channel-wise concatenation baseline and the interactive diffusion setting that Model 1 follows.","marker":"[46]"},{"why":"VBench, the benchmark whose six metrics back the ablations and comparisons.","marker":"[22]"},{"why":"FVD, the distributional video metric used in the depth ablation.","marker":"[45]"},{"why":"Supplies the InstructPix2Pix-style classifier-free guidance used to combine text and spatial conditions at inference.","marker":"[4]"},{"why":"Provides the per-pixel 3D-coordinate representation underlying the depth parameterization.","marker":"[49]"},{"why":"WorldMem, the long-term memory model whose 3D-pose retrieval idea the spatial memory mechanism builds on.","marker":"[51]"}],"fun_headline_variants":["Depth-aware world model cuts drift in long video generation","Video world model with depth predicts steady scenes over hundreds of frames","Explicit geometry in world models keeps video consistent over time","Geometry memory improves autoregressive video world model stability","Adding depth to video world models reduces drift and enhances realism"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method conditions each new prediction on the model's own previously estimated depth and camera geometry, and if those estimates drift or are wrong, the errors could compound instead of shrink; the paper does not measure how accurate the predicted geometry is.","fun_headline_variants_meta":{"raw":{"variants":["Depth-aware world model cuts drift in long video generation","Video world model with depth predicts steady scenes over hundreds of frames","Explicit geometry in world models keeps video consistent over time","Geometry memory improves autoregressive video world model stability","Adding depth to video world models reduces drift and enhances realism"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1306,"prompt_tokens":957,"completion_tokens":349,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":270}},"tokens_in":573,"tokens_out":349,"duration_ms":4222,"temperature":1.0,"reasoning_tokens":270,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:50:08.322002+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test is to feed ground-truth depth and raymaps at every autoregressive step instead of the model's own predictions: if the consistency metrics do not improve, the causal story that geometry carries the drift reduction is falsified; a lighter check is to plot predicted-geometry error against horizon and see whether it grows while visual consistency also breaks.","supporting_citations":[],"review_version":1}