{"id":"c9f59463-a5d8-4cbe-a14d-c12b5779a95f","arxiv_id":"2412.06699","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"See3D proposes a pose-free visual condition for multi-view diffusion trained on web videos, claiming SOTA single- and sparse-view 3D generation, but the evaluation protocol leaks ground-truth information and mixes benchmark data into training.","lead":"This paper trains a multi-view diffusion model for 3D generation on 16 million internet video clips without camera pose labels, then reports large gains over pose-based methods. The evaluation, however, uses ground-truth brightness adjustment and training data that overlaps the benchmarks, so the headline numbers should be treated with caution.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported PSNR margins may be artifacts of a test-time brightness-scaling step that aligns warped-image conditions to ground truth, a protocol not applied to baselines and not disclosed in the main text.","rationale":"The reader's weakest_assumption focused on whether the time-dependent visual-condition can carry camera-motion information. That assumption is partially supported by the paper's MVImageNet ablation (Table 3), where MV-UnPoseT nearly matches MV-Posed, and by the scaling-up data ablation, so the mechanism itself has credible evidence. The most load-bearing vulnerability is instead the evaluation protocol: the test-time brightness alignment with ground truth is an undisclosed, unfair advantage that directly explains why PSNR (the primary metric) improves. The reader did mention this as an undermining factor, but the reader's stated weakest_assumption is the visual-condition premise, not the evaluation protocol. I therefore partially agree. Since the central quantitative claim is not trustworthy as stated, the REJECT verdict stands; the concern does not move the verdict, but rather reinforces it. A clean re-evaluation without ground-truth alignment is a concrete, feasible check that would determine whether the paper's headline results survive.","tokens_in":30326,"tokens_out":4274,"duration_ms":45401,"concrete_test":"Re-run the single-view and sparse-view evaluations with the test-time brightness-scaling step disabled: feed the raw warped images (or brightness-normalized only by the reference image, never by target ground truth) into Eq. 2/3, for both See3D and baselines, and report PSNR/SSIM/LPIPS on Tanks-and-Temples, RealEstate10K, CO3D, LLFF, DTU, and Mip-NeRF 360. If See3D's PSNR advantage over ViewCrafter drops below ~1 dB or the LLFF sparse-view PSNR drops by more than 2-3 dB, the paper's headline quantitative claims are not supported as stated. Also run a train/test overlap audit for RealEstate10K and CO3D by checking whether any evaluation split video appears in the WebVi3D or supplemental 3D training data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix C.2 (Training Details, Brightness Control) states: 'In our evaluation experiments, brightness scaling is applied to the unmasked regions of warped images to align with ground truth, reducing the impact of brightness, and thus yielding a higher correlation between the generated content and pixel-level metrics.' This procedure injects ground-truth target brightness into the visual-condition (Eq. 2-3) at inference time. Because the reported superiority rests primarily on PSNR (the metric the paper says 'reflects the accuracy of viewpoint control'), this test-time ground-truth alignment can substantially inflate PSNR and SSIM relative to baselines (LucidDreamer, ViewCrafter, CAT3D, etc.) that receive no such adjustment. The 4.63 dB gain over ViewCrafter on Tanks-and-Temples and the 23.23 dB sparse-view PSNR on LLFF are therefore not trustworthy as evidence of 'markedly outperforming' prior work. Additionally, the training configuration (Appendix C.2) mixes clips from RealEstate10K and CO3D into the training set, two of the three single-view evaluation datasets, undermining the 'zero-shot' interpretation for those benchmarks. The paper reports no ablation without brightness scaling, so the magnitude of this effect is unknown. This is the most load-bearing concern because it directly invalidates the central quantitative claim, independent of whether the visual-condition mechanism itself is sound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"See3D proposes a pose-free multi-view diffusion model for 3D generation trained on Internet videos. The paper contributes (i) WebVi3D, a dataset of ~15.99M clips/320M frames curated from ~25.5M stock and drone videos via a four-step pipeline (dynamic-object detection, optical-flow motion filtering, and keypoint-trajectory viewpoint filtering); (ii) a visual condition V_t (Eqs. 2-3) that mixes masked, time-noised target views with the noisy latent, replacing camera-pose conditions; and (iii) a warping-based 3D generation pipeline with per-keypoint depth scale/shift alignment, locally weighted regression, and 3DGS reconstruction. The paper reports strong single-view and sparse-view numbers (e.g., 23.76 PSNR on Tanks-and-Temples vs 19.13 for the re-implemented ViewCrafter in Table 2) and claims zero-shot, open-world capability. A controlled ablation on MVImageNet (Table 3) shows near-parity between the proposed pose-free condition and a pose-conditioned warped-image model. The appendix discloses a test-time brightness-scaling protocol (Appendix C.2) that aligns the warped condition with ground truth, plus a training recipe that mixes RealEstate10K and CO3D clips and 0.5M pose-annotated samples into WebVi3D and initializes from MVDream.","tokens_in":30789,"tokens_out":20855,"duration_ms":203768,"significance":"If validated, the core idea is significant: replacing camera-pose conditioning with a pixel-space time-dependent signal would let 3D priors be learned from web-scale video, and WebVi3D's curation pipeline (with a human-validated 88.6% precision on the filtered set) is a useful community asset. The Table 3 ablation gives credible, controlled evidence that the time-dependent visual condition can approximate pose-conditioned warp inputs, and the paper is honest enough to disclose the disputed evaluation protocol in the appendix. At present, however, the headline quantitative claims are not supported: the test-time brightness scaling leaks ground-truth target statistics into the condition and is applied without a matched treatment for baselines; the training set overlaps two of the three single-view test benchmarks; and the pose-free recipe includes a pose-conditioned initialization and 0.5M annotated samples. These issues bear directly on the Abstract's central claim of markedly outperforming models trained on 3D datasets, so the evidence requires substantial rework before the paper's claims can be assessed.","major_comments":[{"comment":"Appendix C.2 states: 'In our evaluation experiments, brightness scaling is applied to the unmasked regions of warped images to align with ground truth, reducing the impact of brightness, and thus yielding a higher correlation between the generated content and pixel-level metrics.' The scaling factor is computed from the ground-truth target image, so at test time the visual condition (Eqs. 2-3) injected into the model carries the target's global brightness statistics, and PSNR/SSIM are then measured against that same target. This is an oracle leak: PSNR is highly sensitive to global brightness offsets (a uniform offset of 15-20 gray levels can cost several dB), so the reported margins in Table 2—e.g., 4.63 dB over the re-implemented ViewCrafter on Tanks-and-Temples—cannot be attributed to viewpoint-control accuracy. Section 4.1 does not disclose this protocol, the paper does not state that the warping-based baselines (ViewCrafter, LucidDreamer) received an equivalent adjustment, and no ablation quantifies performance without the scaling. Because the paper itself identifies PSNR as the metric that 'reflects the accuracy of viewpoint control,' this issue is load-bearing for the central quantitative claim and must be addressed with a GT-free evaluation or an identical normalization applied to all methods.","section":"Appendix C.2 (Brightness Control); Sec. 4.1"},{"comment":"The training configuration mixes clips 'from datasets such as Objaverse, CO3D, RealEstate10k, MVImgNet, and DL3DV' into WebVi3D, and the model is initialized from MVDream, a pose-conditioned multi-view diffusion model. Two of the three single-view benchmarks in Table 2 (RealEstate10K and CO3D) are therefore inside the training distribution, which contradicts the 'zero-shot' characterization in the Abstract and Section 4.1 and makes the comparison to baselines that were not trained on those datasets uneven; only Tanks-and-Temples is a clean out-of-distribution benchmark among the three. In addition, the disclosed 0.5M pose-annotated supplement and the MVDream initialization mean the claim of learning 3D control 'solely' from pose-free video is stronger than the actual recipe. I request results for a WebVi3D-only variant (at least at the reduced scales used in the Section 4.3 data-proportion ablations), and a rewording of the zero-shot and pose-free claims to match what is trained.","section":"Appendix C.2 (Training Configuration); Abstract; Sec. 4.1"},{"comment":"The definition of the visual condition is incomplete. In Eq. 3, X_t is explicitly a noisy latent, while C_t in Eq. 2 is described as a corrupted image; their weighted sum is well-defined only if both are in the same space, but the paper never states whether C_t is VAE-encoded before the mixture, nor does it specify how many channels V_t and the mask M add to the model's first convolution. Because the entire method depends on V_t, this omitted specification blocks reproduction and should be clarified in a revision.","section":"Eqs. 2-3; Appendix C.1"},{"comment":"The pose-free mechanism rests on three coupled components—random masking, the time-dependent noise schedule t' = f(t) = βt with β = 0.2, and the piecewise mixture W_t—but the only visual-condition ablation (Table 3) contrasts the full model against a masked-only condition (MV-UnPoseM vs MV-UnPoseT). No experiment isolates the contribution of the time-dependent noise (Eq. 2) or of the mixture (Eq. 3), and no sensitivity analysis is reported for β, t_decay_end, v_decay_end, or b, so the claimed robustness of the schedule (Section 3.2.2) is not established.","section":"Sec. 4.3; Appendix C.3"}],"minor_comments":[{"comment":"The reported '77% improvement' in Appendix B is arithmetically inconsistent: the pre-filter and post-filter 3D-aware ratios are 11.6% and 88.6%, a 77 percentage-point increase (or a ~7.6x relative increase), not a 77% improvement. Please correct the wording.","section":"Sec. 3.1; Appendix B (User Study)"},{"comment":"The claims that the method 'outperforms all baseline methods across all metrics' (Sec. 4.1) and 'surpassed state-of-the-art reconstruction models' (Sec. 4.2) are too strong: on DTU 3-view, MuRF reports a higher SSIM (0.885 vs 0.884 in Table 2), and on LLFF 9-view, FSGS, BGGS, and CAT3D all report higher SSIM than See3D (Table 5). Please qualify the claims per dataset and per metric.","section":"Sec. 4.1-4.2; Tables 2 and 5"},{"comment":"The sentence 'we removed the time embedding' is ambiguous: Eq. 1 requires the diffusion timestep embedding, so the removal presumably refers to the temporal/positional embedding of the video backbone (frame ordering is shuffled anyway). Please rephrase to avoid implying the timestep embedding is dropped.","section":"Sec. 3.2.3"},{"comment":"The abstract and Section 1 state that WebVi3D totals 4.41 years in duration, but Table 1's filtered hours (36.27K hours) correspond to roughly 4.14 years; please reconcile the numbers.","section":"Sec. 1; Table 1"},{"comment":"The limitations statement lists inference speed, lack of 4D modeling, and model scalability, but does not mention the evaluation-protocol issues (test-time ground-truth brightness alignment, benchmark overlap) that most affect the interpretation of the reported numbers; these should be acknowledged or refuted in the revision.","section":"Appendix A (Limitations)"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the reader's reject verdict is defensible, and I agree that the evaluation-protocol problem is the decisive issue; I am recommending major_revision rather than reject because the core mechanism has partial independent support (Table 3) and the decisive problems are fixable by re-running the evaluation without oracle alignment and by reporting a WebVi3D-only variant. If the authors cannot provide a ground-truth-free evaluation (or equal treatment of baselines) and a benchmark-overlap analysis, the paper should be rejected. The mismatch between the abstract's 'pose-free' framing and the disclosed recipe (MVDream initialization, 0.5M pose-annotated samples) is also a claim-integrity concern worth raising with the authors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Core idea is genuinely interesting, but the headline numbers should not be trusted. The main comparisons are tainted by a test-time brightness-scaling step that aligns the model's input warped images to ground truth, and the baselines don't get that treatment.\n\nWhat's actually new: the visual-condition formulation (time-dependent noise and mixture, Eq. 2-3) is a clever way to train a multi-view diffusion model without pose annotations, and the WebVi3D dataset at 16M clips is a real scale step. The internal ablation on MVImageNet (MV-UnPoseT vs MV-Posed) shows their pose-free condition is competitive with a pose-conditioned warped-image baseline, which is the most convincing part of the paper.\n\nThe soft spots are in the evaluation. Appendix C.2 states that at evaluation, brightness scaling is applied to the unmasked regions of warped images to align with ground truth. That is a test-time adaptation that uses target statistics; it can inflate PSNR and SSIM. Since the paper's main evidence of 'markedly outperforming' baselines is PSNR, the 4.63 dB gain over ViewCrafter on Tanks-and-Temples and comparable margins elsewhere are not credible as stated. There is no ablation without the scaling, so we can't even estimate the effect. Additionally, the training set mixes in 0.5M clips from RealEstate10K and CO3D, two of the three single-view benchmarks, which undercuts the 'zero-shot' label for those datasets. Tanks-and-Temples is clean on that axis, but the brightness issue applies to all.\n\nThe core mechanism may well be sound. The flaw is in the protocol, not necessarily in the model. But the paper as written overclaims, and the central quantitative contribution is not trustworthy. This is fixable: rerun without brightness scaling, report the delta, and be explicit about train/test separation. As is, I would not cite the numbers.\n\nStill, this deserves a serious referee. The dataset, the pose-free conditioning idea, and the internal ablation are valuable, and the evaluation problems are correctable rather than fundamental. Send it to review, but expect major revision on the experimental protocol. It would also make a good reading-group case study on evaluation pitfalls.","headline":"Interesting pose-free conditioning idea and a large new dataset, but the headline PSNR gains are unreliable because the evaluation applies a test-time brightness alignment to ground truth that baselines don't get.","tokens_in":31189,"tokens_out":4085,"would_cite":true,"duration_ms":41193,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"See3D teaches a multi-view diffusion model to do pose-free 3D generation from web videos, reporting state-of-the-art single- and sparse-view reconstruction without camera annotations.","keywords":["multi-view diffusion","pose-free video","3D generation","visual-condition","novel view synthesis","sparse-view reconstruction","dataset scaling","Gaussian splatting"],"falsifier":"Run See3D on a pose-annotated test set where camera trajectories are known and measure the angular error of generated views relative to ground-truth poses: if the visual-condition genuinely encodes viewpoint, this error should be small and grow smoothly with viewpoint distance, whereas a model that only interpolates appearance would show near-chance angular errors for large viewpoint shifts. A second check: train the same architecture on the same data with the visual-condition replaced by plain masked frames; the reported PSNR gap between MV-UnPoseT and MV-UnPoseM should persist on held-out videos beyond the MVImageNet subset used in the ablation.","tokens_in":30157,"feed_emoji":"🎬","tokens_out":5659,"duration_ms":51825,"temperature":0.7,"pith_summary":"See3D claims that a 3D generative model can learn camera control and multi-view consistency without any pose annotations, by training a multi-view diffusion model on large-scale Internet video. To do this, the authors build WebVi3D, a curated dataset of 16M static-scene clips (320M frames) filtered for large viewpoint variation, and design a purely 2D 'visual-condition' – a time-dependent mixture of masked target frames, added noise, and noisy latents – that substitutes for camera poses. On single-view and 3-view reconstruction benchmarks, See3D reports consistent gains over pose- or warp-conditioned baselines, including a 4.63 dB PSNR gain over ViewCrafter on Tanks-and-Temples and the best sparse-view results on LLFF, DTU, and Mip-NeRF 360. The paper argues this shows costly 3D annotations can be replaced by freely available video data, enabling open-world 3D creation to scale with the web.","feed_headline":"Pose-free video training beats pose-annotated 3D models","feed_subtitle":"See3D learns camera control from noisy masked frames, making 3D creation scalable without pose labels.","key_machinery":"The load-bearing object is the visual-condition Vt = [Wt*Ct + (1−Wt)*Xt; M] (Eq. 3), a concatenation of a corrupted target view Ct, the diffusion latent Xt, and a mask M, mixed by a timestep-dependent weight Wt. Ct itself is the masked target image with time-dependent noise at a reduced schedule t' = βt (β = 0.2), which the paper uses to keep enough signal for camera control while preventing conditional-image leakage at small timesteps. This purely 2D signal replaces camera poses as the control channel. The paper couples it with a warping-based generation pipeline using monocular depth (MoGe), sparse keypoint matching (SuperPoint/LightGlue), independent per-keypoint scale-shift alignment, and locally weighted linear regression to recover dense metric depth, so that warped hints stay close enough to the training-time visual-condition distribution.","core_discovery":"The central claim is that a multi-view diffusion model conditioned solely on a visual signal derived from pixel-space video data – random masks, time-dependent noise with t' = βt, and a schedule-dependent mixture Wt*Ct + (1−Wt)*Xt – learns to control camera movement and synthesize geometrically consistent novel views. Trained on WebVi3D, the model See3D reports zero-shot single-view PSNRs of 23.76 on Tanks-and-Temples, 25.36 on RealEstate10K, and 24.28 on CO3D, each several dB above the warped-image baseline ViewCrafter*, and sparse-view results (23.23 PSNR on LLFF, 28.04 on DTU, 17.35 on Mip-NeRF 360) that exceed diffusion-based few-shot reconstructions like CAT3D and ReconFusion. The authors further show the same model, without fine-tuning, supports open-world 3D editing by masking and reconditioning regions of a reference view. The evidence is intended to show that scaling pose-free video data raises the upper bound of 3D generation beyond what 3D-annotated datasets allow.","pith_inferences":["If the pose-free scaling result holds, 3D generation performance may come to track internet video volume rather than curated 3D asset production, a shift that would change the cost structure of training such systems.","The time-dependent mixture is essentially a learned weighting between a content hint and a noise prior; similar schedules could be ported to other conditional diffusion tasks where pose or dense correspondences are unavailable, such as video editing or dense tracking.","The test-time brightness alignment used in evaluation likely inflates PSNR-type metrics relative to baselines; a fairer comparison would re-run baselines with the same alignment or report perceptual metrics on unaligned images.","Because the model shuffles frames and drops time embeddings, it discards temporal order; this suggests an implicit assumption that static-scene multi-view sets are interchangeable, which may limit extension to dynamic 4D content."],"forward_implications":["Scaling training data from pose-free video is enough to displace pose-conditional multi-view diffusion: See3D reports state-of-the-art results on single- and sparse-view benchmarks without any camera input.","The visual-condition generalizes to warping-based pipelines and mask-based editing without fine-tuning, covering single-view-to-3D, sparse-view 3D, and open-world 3D editing.","The data curation pipeline (static-scene filtering, dynamic-region rejection, viewpoint-radius filtering) can be reused to grow WebVi3D as more Internet video becomes available.","The iterative depth alignment (per-keypoint scale-shift plus locally weighted linear regression) corrects monocular depth errors and enables long-sequence generation along complex camera trajectories."],"supporting_citations":[{"why":"Serves as the primary warping-based baseline and the source of the evaluation protocol for single-view novel view synthesis.","marker":"[121]"},{"why":"Provides the multi-view diffusion architecture and initialization used to build See3D.","marker":"[80]"},{"why":"Motivates the time-dependent noise schedule by identifying the signal-leakage problem in conditional diffusion models.","marker":"[127]"},{"why":"One of the three real-world benchmarks used to evaluate single-view 3D generation.","marker":"[129]"},{"why":"One of the three real-world benchmarks used to evaluate single-view 3D generation.","marker":"[75]"},{"why":"A key diffusion-based baseline for sparse-view reconstruction that See3D is compared against.","marker":"[23]"},{"why":"Supplies the monocular depth estimator used in the warping-based generation pipeline.","marker":"[101]"},{"why":"The base video diffusion model whose architecture is adapted into the multi-view diffusion framework.","marker":"[6]"},{"why":"Provides the DDIM sampling and noise schedule used both in training and inference.","marker":"[85]"},{"why":"Serves as the multi-view dataset for the ablation comparing visual-condition against pose-conditioned and masked-conditioned training.","marker":"[122]"}],"fun_headline_variants":["See3D: 3D creation from pose-free videos at scale","No poses needed: Video data powers 3D generation","See3D learns 3D from 16M video clips, no poses","Scale up 3D with pose-free video training","Videos alone teach 3D creation, says See3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that the time-dependent mixture of masked, noised frames and noisy latents carries enough camera-motion information for a diffusion model to synthesize correct novel views without any pose annotation, and that warped test-time images fall within the distribution of this visual-condition; the paper's main numerical comparisons also rely on ground-truth brightness alignment that is not applied to baselines.","fun_headline_variants_meta":{"raw":{"variants":["See3D: 3D creation from pose-free videos at scale","No poses needed: Video data powers 3D generation","See3D learns 3D from 16M video clips, no poses","Scale up 3D with pose-free video training","Videos alone teach 3D creation, says See3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000545,"raw_usage":{"total_tokens":2703,"prompt_tokens":1136,"completion_tokens":1567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":752,"completion_tokens_details":{"reasoning_tokens":1491}},"tokens_in":752,"tokens_out":1567,"duration_ms":10243,"temperature":1.0,"reasoning_tokens":1491,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:19:29.238068+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run See3D on a pose-annotated test set where camera trajectories are known and measure the angular error of generated views relative to ground-truth poses: if the visual-condition genuinely encodes viewpoint, this error should be small and grow smoothly with viewpoint distance, whereas a model that only interpolates appearance would show near-chance angular errors for large viewpoint shifts. A second check: train the same architecture on the same data with the visual-condition replaced by plain masked frames; the reported PSNR gap between MV-UnPoseT and MV-UnPoseM should persist on held-out videos beyond the MVImageNet subset used in the ablation.","supporting_citations":[{"cited_title":"Stereo magnification: Learning view synthesis using multiplane images","cited_arxiv_id":null,"evidence_quote":"One of the three real-world benchmarks used to evaluate single-view 3D generation."},{"cited_title":"Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the monocular depth estimator used in the warping-based generation pipeline."},{"cited_title":"Mvimgnet: A large-scale dataset of multi-view images","cited_arxiv_id":null,"evidence_quote":"Serves as the multi-view dataset for the ablation comparing visual-condition against pose-conditioned and masked-conditioned training."}],"review_version":1}