{"id":"fb352bf1-61de-4fd5-b4df-610551ec3689","arxiv_id":"2412.12091","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A feed-forward pipeline reconstructs 3D Gaussian scenes from single images by regressing 3DGS directly from camera-conditioned video diffusion latents.","lead":"This paper introduces a system that turns a single photo into a wide-scope 3D scene you can fly through in about five minutes, by having an AI video model imagine the unseen surroundings and then building a 3D Gaussian scene directly from the video model's internal representation. It is a step toward fast, practical 3D content creation from ordinary images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The pipeline's core premise (camera-guided video latents are static and 3D-consistent) is asserted but never measured; Appendix E concedes 'occasionally observed motions' that 'hamper reconstruction,' leaving Tables 1-2 as averages over unquantified failures.","rationale":"The paper has real strengths: the dual-branch camera conditioning is a novel and well-ablated design (Table 1 and Appendix A), the RGB-vs-latent comparison in Table 3 is a fair internal control, the latency advantage is clearly demonstrated, and the qualitative results are compelling. I found no internal inconsistency in the LaLRM architecture or its training objective. The soft spot is upstream: the quality of the 3DGS depends entirely on the multi-view consistency of the video diffusion output, and that property is asserted rather than measured. Appendix E is an honest admission but also a direct concession that the core assumption fails at least sometimes; without a failure rate, the SOTA averages in Table 2 are uninterpretable. The reader's weakest_assumption identified the same load-bearing premise, and my formulation broadens it from 'static' to '3D-consistent with a single rigid scene' and adds a concrete test. This does not change the verdict: CONDITIONAL remains appropriate, conditional on the authors providing the missing quantitative validation of staticity and 3D consistency.","tokens_in":26388,"tokens_out":7834,"duration_ms":75514,"concrete_test":"Sample 100 random images and camera trajectories from the RE10K and Tanks-and-Temples test sets; run the camera-guided video diffusion model on each. Then (1) estimate per-frame optical flow and compare it with the flow predicted from the known camera poses plus an off-the-shelf monocular depth estimate; flag videos where compensated residual flow exceeds a threshold as 'non-static'. (2) Run COLMAP on each generated video and report the fraction of videos with low triangulation inlier ratio or high re-projection error as a direct measure of 3D consistency. (3) For the same samples, run the full LaLRM pipeline and compute novel-view PSNR/SSIM separately for the 'static-consistent' and 'non-static/inconsistent' subsets. If the failure fraction is <=2% and the PSNR drop on failures is <=0.5 dB relative to the clean subset, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the camera-guided diffusion model produces latents encoding multi-view information with 3D consistency, and that LaLRM can lift these latents to a valid 3DGS. For this to hold, every generated video must be realizable as a static 3D scene under the requested camera path. Section 3.2 asserts that the model is adapted to 'generate static scenes,' but this is never verified quantitatively. Appendix E admits: 'We occasionally observe motions in the generated videos, which hampers the reconstruction effect.' 'Occasionally' is unquantified: no motion score, no failure rate, no per-sample breakdown of Table 2. Moreover, 3D consistency is strictly stronger than staticness: a video can be static yet still inconsistent with any single 3D scene (e.g., objects morphing, lighting changes, non-rigid warps). The reported metrics (FID, FVD, Rerr, Terr, PSNR, SSIM, LPIPS) do not measure multi-view geometry; Rerr/Terr only check whether COLMAP can fit a camera to the generated frames, not whether the views satisfy epipolar or re-projection constraints. Because LaLRM is trained to force any latent into a static 3DGS, a mildly inconsistent video will produce blur or floaters that are averaged into the headline numbers without being reported separately. The load-bearing premise is therefore unsupported quantitatively, and the admitted failure mode could materially change the conclusions if it occurs in a non-negligible fraction of test cases.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Wonderland, a single-image 3D scene generation pipeline. A camera-guided video diffusion model generates a 49-frame video conditioned on an input image and a desired camera trajectory; the video is encoded to latents by a frozen 3D VAE, and a Latent Large Reconstruction Model (LaLRM) regresses 3D Gaussian Splatting parameters from these latents in a feed-forward manner. The video diffusion model uses a dual-branch conditioning mechanism (ControlNet and LoRA) with Plücker embeddings for precise pose control, and LaLRM is trained progressively on benchmark videos plus 20K self-generated videos. Experiments report state-of-the-art quantitative results on RealEstate10K, DL3DV, and Tanks-and-Temples for both camera-guided video generation and 3D scene generation, along with qualitative in-the-wild comparisons and a latency comparison.","tokens_in":26729,"tokens_out":4674,"duration_ms":41302,"significance":"If the results hold, the paper makes a meaningful contribution by showing that a 3D reconstruction model can be built directly on the latent space of a video diffusion model, enabling feed-forward, memory-efficient reconstruction of wide-scope scenes from a single image. The quantitative gains over ZeroNVS and ViewCrafter are consistent across three datasets, and the ablations in Table 3 support the latent-space design over RGB-based alternatives. The dual-branch camera control with Plücker embeddings is a clean engineering contribution. The paper also ships detailed architectural descriptions and ablations, which strengthen reproducibility. However, the headline 'wide-scope' claim is not actually measured, and the central assumption that generated videos are static and 3D-consistent is admitted but not quantified, which limits the current evidence for the central claim.","major_comments":[{"comment":"The 'wide-scope' claim is not supported by the evaluation protocol. In §4.2, the 3D scene metrics are computed on 'a total of 14 sampled frames and poses subsequent to the conditional image,' and in §4.1 the visual similarity metrics are similarly reported for the first 14 frames of the generated videos (which are 49 frames long for the proposed method). These frames are the ones closest to the input view, where the task is easiest and where prior methods also perform best. Since 'wide-scope' is a central advertised advantage, the paper should report metrics over larger view offsets or over the full generated trajectories, or at least show a plot of error as a function of frame index. Without this, the numbers in Tables 1 and 2 do not quantify wide-scope behavior.","section":"§4.2, §4.1, and Appendix D.2"},{"comment":"The load-bearing premise that the camera-guided video diffusion model generates static, 3D-consistent videos is asserted but never measured. Appendix E states 'We occasionally observe motions in the generated videos, which hampers the reconstruction effect,' but 'occasionally' is never quantified: there is no motion score, no failure rate, no per-sample breakdown in Tables 1 and 2, and no correlation analysis between consistency and reconstruction quality. The reported metrics (FID, FVD, Rerr, Terr, PSNR, SSIM, LPIPS) do not measure multi-view geometric consistency; Rerr/Terr only measure whether COLMAP can fit a pose to the generated frames, not whether the views satisfy epipolar or re-projection constraints. Because LaLRM is trained to force any latent into a static 3DGS, mildly inconsistent videos will produce blur or floaters that are averaged into the headline numbers without being reported separately. The paper needs a quantitative consistency measure (e.g., optical flow residual, epipolar error, or a static-scene score) and a per-sample report of how often generated videos are sufficiently consistent for reconstruction.","section":"§3.2 and Appendix E"},{"comment":"The fine-tuning of LaLRM on 20K videos generated by the same camera-guided video diffusion model whose latents are later reconstructed at test time introduces a domain-alignment confound. While the paper's Table A3 shows only a small gain from the in-the-wild data on RE10K, the test-time latents always come from the same video model, so LaLRM can learn to decode that model's specific artifacts as if they were valid scene geometry. To isolate this effect, the authors should evaluate LaLRM on latents from an independently trained camera-controlled video model (e.g., a baseline such as ViewCrafter) or on real videos from the RE10K test set encoded with the same 3D VAE, and report the reconstruction metrics separately from those obtained on self-generated latents. Without this control, part of the reported performance may be attributable to training/test distribution alignment rather than to generalizable 3D reconstruction.","section":"§3.3 and Appendix B.1"},{"comment":"The quantitative comparison in Table 2 includes only ZeroNVS and ViewCrafter, yet the abstract and introduction claim 'state-of-the-art' performance in single-view 3D scene generation. Several recent methods that are cited in the paper (e.g., ReconX, CAT3D, Flash3D, Long-LRM, GenEx) are not included in the quantitative benchmark. CAT3D appears only in a qualitative Mip-NeRF comparison (Figure 6), and other concurrent feed-forward or diffusion-based methods are absent entirely. The claim of state-of-the-art performance is therefore not established against the current literature; the authors should either add quantitative comparisons to these methods (using their released models or reported numbers on the same benchmarks) or tone down the claim to 'state-of-the-art among compared methods.'","section":"Table 2 and §4.2"}],"minor_comments":[{"comment":"The claimed '256× spatiotemporal reduction' uses rt=4 and rs=8, so the total reduction is 4⋅8⋅8=256; this is not purely 'spatiotemporal' but also spatial. The phrasing could be made precise.","section":"§3.3"},{"comment":"The figure does not use the notation introduced in the text (octrl, olora) and the two branches are not labeled clearly; annotating the ControlNet and LoRA branches would improve readability.","section":"Figure 2"},{"comment":"Each cell in Table 3 contains two numbers (e.g., '0.137/0.126') without a legend explaining which number corresponds to RGB-49 and which to RGB-14; this should be clarified in the caption or in the text.","section":"Table 3"},{"comment":"The main text does not state which camera poses are used for the Table 2 results; Appendix D.2 says both 'conditional camera poses directly' and 'COLMAP-derived poses from the generated videos' were tried, with the latter yielding superior results. The paper should specify which pose source produced the reported numbers.","section":"Appendix D.2"},{"comment":"There are several typographical errors: 'renown' should be 'renowned' (§3.2), 'descent' should be 'decent' (Appendix D.2), and 'Fréchet' appears without the accent in some places.","section":"§3.2 and Appendix D.2"},{"comment":"The claim that this is 'the first time that a 3D reconstruction model can effectively be built upon the latent space of a diffusion model' is strong given that the paper itself cites concurrent works such as VFusion3D [30] and LiftImage3D [16] that also use video diffusion priors; the novelty claim should be qualified with a discussion of these works.","section":"§1"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically solid in its engineering and the ablations are useful, but the evaluation protocol does not substantiate the 'wide-scope' claim, and the core assumption of static, geometrically consistent videos is unquantified. The self-generated training data and the exclusive use of the same video model at test time should be controlled for. I would be willing to reconsider after the authors add the missing consistency measures and broaden the evaluation, especially toward the later frames of the generated trajectories."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hanwen Liang et al. have a real idea here: instead of decoding video diffusion outputs to RGB and then reconstructing, regress 3DGS directly from the compressed video latents. That is new as far as I know, and it is well motivated — the 256x spatiotemporal reduction makes wide-scope feed-forward reconstruction tractable. The dual-branch camera conditioning (ControlNet-style copy plus LoRA) is also a sensible engineering contribution, and the ablations support it: latent-based beats RGB-based, dual-branch beats either branch alone. The results in Table 2 are consistent across RE10K, DL3DV, and Tanks, and the paper does not oversell the baselines it actually compares against.\n\nThe soft spots are real, though. The load-bearing premise is that the camera-guided video model produces latents that are static and 3D-consistent. The paper asserts this but never measures it. Appendix E admits occasional motions “hamper the reconstruction effect” — “occasionally” is unquantified, with no motion score, failure rate, or per-sample breakdown. Since Rerr/Terr only check whether COLMAP can fit cameras, and the image similarity metrics do not test multi-view geometry, a mildly inconsistent video can still pass evaluation while producing blurry regions that get averaged into the headline numbers. The 14-frame evaluation in Section 4.2 also undercuts the “wide-scope” claim: that is a narrow band near the input, not really wide-scope. The self-generated 20K training videos, produced by the same diffusion model whose latents LaLRM reconstructs, add a mild circularity; the external benchmarks temper it, but the out-of-domain story is the weakest evidence. Missing strong recent baselines (ReconX at minimum) and no error bars make the margin claims hard to calibrate.\n\nNone of this kills the paper. The architecture is coherent, the gains over the chosen baselines are consistent, and the limitations section is honest. What is missing is a quantitative characterization of the core assumption; a careful reader cannot tell whether the reported averages hide a large failure fraction.\n\nFor peer review: yes, a serious editor should send this out. It is novel, reproducible in principle, and the central direction will matter for the field. I would push the authors to quantify motion and consistency, evaluate beyond 14 frames with geometry-aware metrics, and add the missing baselines. I would cite it and bring it to a reading group.","headline":"Genuinely new latent-space recipe with consistent gains, but the static-scene assumption is unquantified and the wide-scope evaluation is narrow; well worth serious review with revisions.","tokens_in":27293,"tokens_out":2381,"would_cite":true,"duration_ms":22111,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single image can generate a wide, navigable 3D scene by reconstructing from video-diffusion latents.","keywords":["single-image 3D generation","video diffusion model","3D Gaussian Splatting","novel view synthesis","camera trajectory control","latent space reconstruction","feed-forward reconstruction","wide-scope scenes"],"falsifier":"Generate a video along a trajectory that returns to the starting camera and reconstruct the 3DGS; if the render from the final pose differs substantially from the input image (e.g., LPIPS above the thresholds the paper reports for its own results), the video latent was not 3D-consistent.","tokens_in":26193,"feed_emoji":"🎥","tokens_out":7087,"duration_ms":59533,"temperature":0.7,"pith_summary":"This paper claims that a single image is enough to generate a wide, navigable 3D scene by chaining two learned modules: a camera-guided video diffusion model that expands the image into a short video following a specified camera trajectory, and a Latent Large Reconstruction Model (LaLRM) that reads the video's compressed latent, not its decoded frames, and regresses 3D Gaussian Splatting parameters in a feed-forward pass. The claim is that this avoids both requirements of prior work—dense multi-view capture and per-scene optimization—while producing high-quality novel views, and that it outperforms existing single-view scene generation methods on RealEstate10K, DL3DV, and Tanks-and-Temples. A stronger conceptual claim is also made: that a diffusion model's latent space can itself serve as the input representation for 3D reconstruction, so generation and reconstruction share the same compressed, 3D-aware representation.","feed_headline":"One image becomes a navigable 3D scene in minutes","feed_subtitle":"A video diffusion model's latent space feeds a reconstruction network, skipping multi-view capture and per-scene optimization.","key_machinery":"The central mechanism is the pairing of a dual-branch camera conditioning module with a latent-space reconstruction transformer. Camera poses are represented as per-pixel Plücker embeddings (6D ray-origin and ray-direction coordinates) and injected into a pretrained video diffusion transformer through two branches: a ControlNet-style branch with a trainable copy of the first transformer blocks, and a LoRA fine-tuning branch, which together make the video follow the requested trajectory while remaining static. The LaLRM then patchifies the video latent and the same camera embeddings into equal-length token sequences, concatenates them, and passes them through transformer blocks to regress a 12-channel Gaussian feature map, which is decoded by a 3D deconvolution into per-pixel 3D Gaussian attributes (RGB, scale, rotation quaternion, opacity, and ray distance). This 3DGS output is rendered and supervised by a combination of MSE and perceptual losses on seen and unseen views.","core_discovery":"Wonderland demonstrates that 3D reconstruction can be built directly on the latent space of a video diffusion model. Given one image and a camera trajectory, a fine-tuned video diffusion transformer generates a video whose latent encodes multi-view, 3D-consistent scene structure; the LaLRM then regresses these latents into 3D Gaussian Splatting (3DGS) in a single feed-forward pass. The paper reports that this yields high-quality, wide-scope novel views and outperforms prior single-view methods on benchmark datasets, including out-of-domain images. The central discovery is the alignment of the generation and reconstruction tasks through the video latent space, which provides a 256x spatiotemporal compression that makes dense transformer-based reconstruction of wide scenes computationally tractable.","pith_inferences":["Testable extension not in the paper: if video latents are the right substrate, then swapping the camera-guidance video backbone for a different pretrained video model may work with only LaLRM retraining, turning LaLRM into a general latent-to-3D module.","The dual-branch conditioning design may transfer to other spatial conditions beyond cameras, such as depth or optical flow, since both branches inject token-level conditions without altering the frozen video transformer's weights.","The stated latency bottleneck is the video generation stage, not reconstruction; a faster sampler or parallel denoising would make the whole pipeline nearly real-time, whereas the reconstruction stage itself is already feed-forward.","A practical safeguard the paper does not discuss: applying a lightweight static-scene consistency check to the generated video before reconstruction would catch the occasional motion failures that degrade results."],"forward_implications":["Single-image 3D scene generation becomes a feed-forward process that completes in about five minutes on one A100 GPU, roughly 3.2x faster than the Cat3D baseline and 36x faster than ZeroNVS.","Wide-scope and out-of-domain scenes can be reconstructed without multi-view capture, because the generated video supplies the multi-view coverage from a single input image and desired trajectory.","Because the reconstructor reads latents rather than decoded pixels, the token count is reduced enough to handle 49-frame, 480x720 clips within a dense transformer's memory budget.","The approach is explicitly limited to static scenes; the authors report that motions occasionally appear in generated videos and hamper reconstruction, so dynamic content remains an open problem."],"supporting_citations":[{"why":"Defines 3D Gaussian Splatting, the scene representation that the latent reconstruction model regresses and renders.","marker":"[47]"},{"why":"Provides the base video diffusion transformer and 3D-VAE whose latent space the whole pipeline builds on.","marker":"[107]"},{"why":"Supplies training and evaluation videos with camera poses; its trajectories are also reused to generate out-of-domain training data.","marker":"[118]"},{"why":"Supplies the DL3DV benchmark used for training and evaluation of both video generation and 3D reconstruction.","marker":"[58]"},{"why":"Supplies the out-of-domain Tanks-and-Temples benchmark used to test generalization.","marker":"[49]"},{"why":"Provides the ControlNet conditioning principle behind one branch of the dual-branch camera control.","marker":"[115]"},{"why":"Provides the LoRA low-rank adaptation used to fine-tune the video model on static-scene datasets.","marker":"[40]"},{"why":"Serves as the main video-diffusion baseline for camera control and 3D scene generation comparisons.","marker":"[112]"},{"why":"Serves as the single-image 3D scene generation baseline (ZeroNVS) for benchmark comparisons.","marker":"[76]"},{"why":"Supplies the transformer backbone design adopted for the Latent Large Reconstruction Model.","marker":"[114]"}],"fun_headline_variants":["Single image to 3D scene via video latents","One image becomes navigable 3D in one pass","Video latent space powers single-image 3D scenes","From one photo to 3D scene in a feed-forward pass","Wonderland: single-image 3D without per-scene optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the camera-guided video diffusion model, after fine-tuning, will generate static, 3D-consistent videos that faithfully follow the specified camera trajectory; if the generated videos drift, contain motion, or ignore the poses, the LaLRM cannot reconstruct a valid 3D scene, and the authors concede in Appendix E that such motions occasionally occur.","fun_headline_variants_meta":{"raw":{"variants":["Single image to 3D scene via video latents","One image becomes navigable 3D in one pass","Video latent space powers single-image 3D scenes","From one photo to 3D scene in a feed-forward pass","Wonderland: single-image 3D without per-scene optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1543,"prompt_tokens":915,"completion_tokens":628,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":558}},"tokens_in":531,"tokens_out":628,"duration_ms":5344,"temperature":1.0,"reasoning_tokens":558,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:17:03.462173+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a video along a trajectory that returns to the starting camera and reconstruct the 3DGS; if the render from the final pose differs substantially from the input image (e.g., LPIPS above the thresholds the paper reports for its own results), the video latent was not 3D-consistent.","supporting_citations":[{"cited_title":"Adding conditional control to text-to-image diffusion models,","cited_arxiv_id":null,"evidence_quote":"Provides the ControlNet conditioning principle behind one branch of the dual-branch camera control."}],"review_version":1}