{"id":"d8a38f5a-4cf7-4571-bdb5-7f6ccdca3420","arxiv_id":"2411.13291","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Dynamic-aware point tracking with consistent video depth improves camera pose estimation and dense reconstruction from dynamic monocular videos.","lead":"DATAP-SfM combines long-term point tracking with depth-based motion detection to estimate camera paths and build 3D point clouds from ordinary videos containing moving objects. It reports more accurate camera trajectories than prior dynamic-scene structure-from-motion methods on Sintel, TUM, and DAVIS benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's 'Ours' row mixes MiDaS per-frame depth (full-set ATE 0.104) with NVDS consistent depth (subset ATE 0.015); the headline SOTA numbers do not correspond to any single described configuration.","rationale":"I chose the configuration mismatch over the depth-scale ambiguity because it directly undermines the quantitative support for the central claim, whereas the depth-scale concern is a generalization risk that still leaves the reported numbers intact. The reader's weakest assumption about normalized depth is real, but the paper's own ablations already show that depth helps, and the scale issue is an interpretation gap, not a contradiction of the measured results. The configuration mismatch, however, makes the headline numbers ambiguous: the best numbers in Table 2 are aggregates of two different experimental conditions. This violates a basic expectation of reproducibility and prevents a reader from knowing what 'Ours' actually is. The fix is straightforward—report a single configuration or clearly state the per-subset choice—so the appropriate verdict remains conditional. I thus agree with the reader's verdict but not with the choice of weakest assumption.","tokens_in":13808,"tokens_out":7880,"duration_ms":82892,"concrete_test":"Re-run the full evaluation on MPI Sintel with a single fixed depth source: (a) MiDaS per-frame depth for both the COLMAP-success subset and the full set, and (b) NVDS consistent depth for both. Report ATE/RPE for each configuration. If no single configuration reproduces both Table 2 'Ours' numbers (0.015 subset, 0.104 full), then the reported 'Ours' row is not a single method, and the claimed SOTA improvements must be disaggregated or the paper must specify which depth source is used when.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 4 reports four variants of the proposed pipeline. The variant 'w/ depth' (per-frame MiDaS) achieves full-set ATE 0.104 and subset ATE 0.023; the variant 'w/ consistent depth' (NVDS) achieves full-set ATE 0.117 and subset ATE 0.015. Table 2 reports the 'Ours' row with full-set ATE 0.104 and subset ATE 0.015—the best of the two variants on each metric. The paper does not state that different depth sources are used for different subsets, so the 'Ours' row cannot be reproduced as a single pipeline. This is the most load-bearing concern because the central quantitative claim of state-of-the-art pose estimation ('19.37% improvement in ATE on the full MPI Sintel set') depends on the 0.104 full-set number, which stems from the monocular-depth variant, even though the abstract and method section emphasize consistent video depth as the solution to scale ambiguity. If a single configuration is used, the full-set improvement shrinks (to 9.3% for NVDS) and the subset improvement changes; the SOTA claim is then not supported as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DATAP-SfM, a structure-from-motion pipeline for monocular videos containing dynamic objects. The core contribution is a dynamic-aware point-tracking network (DATAP) that jointly estimates long-term 2D trajectories, visibility, and per-point dynamic labels in a sliding-window transformer, aided by a depth-aware motion-segmentation decoder. The resulting static, visible tracks are fed into global bundle adjustment to estimate camera poses and a dense point cloud. Experiments on MPI Sintel, TUM RGBD dynamic sequences, and DAVIS compare against ParticleSfM, DROID-SLAM, COLMAP, and others, reporting state-of-the-art ATE improvements. The paper also provides ablations on the depth source and motion-segmentation components.","tokens_in":14066,"tokens_out":4233,"duration_ms":42023,"significance":"If the reported results are reproducible, the paper makes a useful contribution: it integrates modern long-term point tracking with learned dynamic segmentation to enable global, rather than incremental, SfM in dynamic scenes, and it evaluates on standard benchmarks with several baselines and ablations. The strengths include the use of held-out benchmarks (MPI Sintel, TUM RGBD), a transparent training protocol (FlyingThings3D, 4 GPUs, 2 days), and a clear decomposition of the pipeline into tracking, motion segmentation, and bundle adjustment. However, the central quantitative claim of state-of-the-art pose estimation is currently undermined by an inconsistency between Table 2 and the ablation results in Table 4, as detailed below. The paper also does not discuss code or model release, which limits reproducibility. Overall, the approach is plausible and the experimental design is appropriate, but the headline numbers need to be reconciled with a single, clearly specified configuration before the claims can be accepted.","major_comments":[{"comment":"The 'Ours' row in Table 2 does not correspond to any single ablation variant in Table 4. On the MPI Sintel full set, Table 2 lists ATE 0.104, RPE Trans 0.037, RPE Rot 0.306, which exactly match the 'w/ depth' variant in Table 4; on the COLMAP subset, Table 2 lists ATE 0.015, RPE Trans 0.007, RPE Rot 0.128, which exactly match the 'w/ consistent depth' variant. The paper never states that different depth sources are used for the two subsets, so the 'Ours' row is not reproducible as a single pipeline. This directly affects the headline claims of a 19.37% improvement on the full set and a 28.57% improvement on the COLMAP subset. Please either report one designated configuration for Table 2 or explicitly present the per-subset depth-source choices and justify why such mixing is a fair comparison.","section":"Table 2 vs. Table 4"},{"comment":"The abstract and method repeatedly emphasize consistent video depth (NVDS) as the solution to scale ambiguity and as a performance booster, but Table 4 shows that the monocular per-frame depth variant ('w/ depth') achieves a better full-set ATE (0.104) than the consistent-depth variant (0.117). The only advantage of consistent depth appears on the COLMAP subset (ATE 0.015 vs. 0.023). The paper does not discuss this discrepancy or explain why the advocated depth source is worse on the full set. This is a load-bearing point because the central pose-estimation claim is tied to the full-set number. Please provide an analysis of this trade-off or qualify the claims about consistent depth accordingly.","section":"Sections 1 and 3.1; Table 4"},{"comment":"The depth-aware motion segmentation normalizes per-frame monocular or video depth to (0,1) and back-projects 2D tracks to 3D to construct scene-flow features. Since single-view and video depth estimates are defined only up to an unknown per-frame scale, the absolute scale of the 3D trajectory features is not meaningful, and the normalization may distort the relative motion cue. The paper asserts that consistent video depth alleviates this ambiguity, but the ablations in Table 4 do not support a consistent advantage, and no scale-invariant alternative is tested. Please provide a justification for why this normalization preserves the static-versus-dynamic signal, or add an ablation with scale-invariant features (e.g., normalized scene flow with per-trajectory scaling).","section":"Section 3.1, depth normalization"},{"comment":"The TUM RGBD evaluation is not clearly specified. The table header uses contradictory row labels ('Full set' followed by 'Full subset'), the same 'Ours' method appears in both the ParticleSfM subset and full-set rows with different numbers, and the text does not list which 9 sequences were selected or define the 'ParticleSfM subset.' Without this information, the claimed 26.62% improvement on the subset cannot be independently verified. Please provide the sequence list, the exact evaluation protocol, and a clarified table.","section":"Table 3 and Section 4.5"}],"minor_comments":[{"comment":"The title uses 'DATAP-SfM' while the manuscript header and abstract use 'DA TAP-SfM'; please use one consistent spelling throughout.","section":"Title and abstract"},{"comment":"The header '(Left: Full set, Right: Successful subset of COLMAP)' is misleading because each cell contains two numbers separated by a slash; split the table into two sub-tables or add explicit column headers such as 'Full set / COLMAP subset.'","section":"Table 4"},{"comment":"The trajectory loss is written as a sum over J of ||P(j) - P_hat(j)||, but the relationship between the index j, the number of trajectories N, and the sliding-window length L is not defined; please clarify the notation.","section":"Section 3.1, Eq. (1)"},{"comment":"The paper does not list the 9 selected TUM RGBD sequences; including the sequence names would improve reproducibility, as would stating the exact subset definition used for the 'successful' rows.","section":"Section 4.5"},{"comment":"The paper does not mention whether code or trained models will be released; given the reliance on external components (CoTracker, MiDaS, NVDS) and the difficulty of reproducing the full pipeline, a code-release statement would be valuable.","section":"Throughout"},{"comment":"The bibliography contains duplicate entries for Schonberger & Frahm 2016 and Butler et al. 2012a/b with inconsistent formatting; please unify citations.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the Table 2/Table 4 mismatch: the paper's headline 'Ours' row appears to select the best result per metric from two different ablation variants. This is a fixable reporting flaw, not a fundamental methodological error, and I would be willing to reconsider after a revision that specifies a single configuration or transparently documents the per-subset choices. The discrepancy between the claimed benefit of consistent video depth and the full-set ablation results also needs explicit discussion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is solid: a point tracker that jointly outputs trajectories, visibility, and per-point dynamic labels, then feeds the static visible tracks into global bundle adjustment. That is a sensible extension of ParticleSfM, and the ablations show the depth cue is doing real work. I also like that the method produces dense point clouds of dynamic scenes, not just poses.\n\nThe main problem is the reporting. Table 2's \"Ours\" row takes the full-set ATE from the monocular-depth variant (0.104) and the subset ATE from the consistent-depth variant (0.015). The paper never says it is switching depth sources between subsets, so the reproduced pipeline is not actually defined. This matters because the 19.37% full-set improvement over ParticleSfM relies on the monocular-depth number, while the abstract and method section emphasize consistent video depth. If you use the consistent-depth variant for the full set, the improvement shrinks to about 9%, which is still positive but not the headline claim. The stress-test note is right on this.\n\nThere are also smaller issues. The abstract overstates the role of consistent video depth, since the best full-set result uses per-frame MiDaS. On Sintel, RPE trans is actually worse than ParticleSfM on the full set (0.037 vs 0.031), and the paper does not comment on that. Table 3 has a labeling mess: the bottom block lists \"subset Ours 0.185\" after a full-set row, which is confusing. No error bars are reported, and there is no code release, so I cannot verify whether the differences between ablations are meaningful beyond a single run.\n\nThat said, the empirical evaluation is on held-out benchmarks with external depth and tracking components, so there is no circularity problem. The architecture is plausible, the ablations are informative, and the qualitative results on DAVIS look genuinely better. This is not a case where the central idea fails. It is a case where the paper needs a serious pass to make the claims reproducible.\n\nFor a referee: yes, this deserves peer review. The problem is important, the contribution is real, and the fix is straightforward. Ask the authors to report a single configuration for each benchmark, clarify which depth source is used where, add error bars if possible, and release code. If they do that, the paper could be a solid contribution.","headline":"A genuinely useful dynamic point tracker for SfM, but the headline numbers mix two different depth configurations and need to be fixed before the SOTA claims can be taken at face value.","tokens_in":14651,"tokens_out":2282,"would_cite":true,"duration_ms":24815,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a dynamic-aware point tracking network, combined with consistent video depth and global bundle adjustment over static visible tracks, achieves state-of-the-art camera pose estimation in dynamic scenes, with ATE…","keywords":["dynamic-aware point tracking","structure from motion","motion segmentation","consistent video depth","camera pose estimation","global bundle adjustment","video point tracking"],"falsifier":"Run the pipeline on dynamic sequences with ground-truth depth but per-frame scale perturbations that keep normalized depth in (0,1); if motion segmentation mIoU and ATE degrade significantly, the assumption that normalized depth suffices is wrong. Alternatively, substituting a constant depth map while keeping everything else fixed should reproduce the 'w/o depth' ablation numbers; if it does not, the depth module is doing something else.","tokens_in":13588,"feed_emoji":"🎥","tokens_out":8190,"duration_ms":67171,"temperature":0.7,"pith_summary":"The paper is trying to establish that dynamic-aware dense point tracking, built on long-term video correspondence and consistent video depth, can replace the chain of adjacent-frame optical flow that previous trajectory-based SfM methods rely on. If correct, this removes the cumulative error of chaining pairwise matches and the scale ambiguity of single-view depths, and lets all camera poses be optimized at once over point tracks that are classified as static and visible. The practical payoff is reliable camera trajectory estimation and dense reconstruction from casual monocular videos containing people, vehicles, and other moving objects, where classic SfM and SLAM typically fail.","feed_headline":"Video point tracking beats chained optical flow in dynamic SfM","feed_subtitle":"Replacing pairwise flow matching with a global tracker plus depth makes camera poses stable around moving people and vehicles.","key_machinery":"The load-bearing object is the depth-aware trajectory motion segmentation module inside DATAP. It takes per-frame monocular or video depth, normalizes each depth map to (0,1), back-projects the 2D point tracks into 3D camera coordinates, and concatenates 2D trajectory positions, 3D positions, 2D motion, and scene-flow motion into a $10L$-dimensional feature vector per trajectory. These features are encoded by a transformer and decoded by an OANet-style network with diff-pool clustering to predict dynamic labels. This module is what disambiguates static from moving points; the ablations show removing it degrades mIoU from 54.9 to 49.8 and full-set ATE from 0.104 to 0.138.","core_discovery":"The central claim is that a single sliding-window transformer, which jointly estimates point trajectories, visibility, and per-point dynamic labels, plus global bundle adjustment over static visible tracks, yields state-of-the-art camera pose estimation on dynamic sequences. The paper reports a 19.37% improvement in ATE over ParticleSfM on the full MPI Sintel set and a 26.62% improvement on the TUM RGBD dynamic subset, and shows that adding depth priors lifts motion segmentation mIoU from 49.8 to 54.9. The paper also states that ParticleSfM fails outright on some TUM sequences while the proposed method returns a pose.","pith_inferences":["If the normalized-depth assumption holds, the same architecture should transfer to any video where relative scene flow is informative, even without metric scale; a testable variant would replace depth with a fixed scale and measure the mIoU drop.","The dependence on a learned video-depth model (NVDS) may be the main bottleneck for wild videos; a fully self-supervised depth cue trained jointly with tracking could make the pipeline more general.","The global-bundle-adjustment design suggests that robustness gains should grow with sequence length, because no incremental registration error accumulates; this could be checked by scaling experiments on longer videos.","If the method is made efficient enough for real-time use, it could form the front end of a dynamic SLAM system; the paper flags this direction as future work."],"forward_implications":["Global structure from motion can run directly on long-term point tracks, skipping incremental camera registration and its drift.","Consistent video depth improves motion segmentation enough to matter downstream: mIoU goes from 49.8 to 54.9 in the paper's ablations.","The pipeline produces complete point clouds of dynamic scenes by fusing depth-based dynamic points into the static SfM point cloud.","The method can serve as an initialization for dynamic-scene NeRF reconstruction and editing, as the paper notes in its broader-impact section."],"supporting_citations":[{"why":"Prior trajectory-based SfM baseline and the source of the global bundle adjustment and motion segmentation formulation that DATAP extends.","marker":"Zhao et al., 2022"},{"why":"CoTracker supplies the sliding-window point tracking backbone and transformer architecture that DATAP builds on.","marker":"Karaev et al., 2023"},{"why":"NVDS provides the consistent video depth prior used to initialize depth for dynamic-aware motion segmentation.","marker":"Wang et al., 2023b"},{"why":"OANet decoder with diff-pool clustering is used as the decoder in the depth-aware trajectory motion segmentation.","marker":"Zhang et al., 2019"},{"why":"Translation averaging method used to initialize camera poses in the global SfM pipeline.","marker":"Ozyesil & Singer, 2015"},{"why":"Rotation averaging method used to initialize camera poses in the global SfM pipeline.","marker":"Chatterjee & Govindu, 2013"},{"why":"Midas v3.1 monocular depth model serves as the alternative depth prior in the ablations.","marker":"Birkl et al., 2023"}],"fun_headline_variants":["Global point tracking beats chained flow for dynamic SfM","Depth-aware tracker improves dynamic SfM pose accuracy","Global trajectory estimation beats optical flow chaining for SfM","DATAP-SfM: robust camera poses via tracking and depth","Joint tracking and depth remove dynamic-scene pose drift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that normalizing per-frame depth to (0,1) and back-projecting 2D tracks into 3D preserves enough scene-flow signal to tell static from moving points, even though per-frame depth estimates carry unknown scale.","fun_headline_variants_meta":{"raw":{"variants":["Global point tracking beats chained flow for dynamic SfM","Depth-aware tracker improves dynamic SfM pose accuracy","Global trajectory estimation beats optical flow chaining for SfM","DATAP-SfM: robust camera poses via tracking and depth","Joint tracking and depth remove dynamic-scene pose drift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001137,"raw_usage":{"total_tokens":4720,"prompt_tokens":939,"completion_tokens":3781,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":3698}},"tokens_in":555,"tokens_out":3781,"duration_ms":28055,"temperature":1.0,"reasoning_tokens":3698,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:36:33.993822+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on dynamic sequences with ground-truth depth but per-frame scale perturbations that keep normalized depth in (0,1); if motion segmentation mIoU and ATE degrade significantly, the assumption that normalized depth suffices is wrong. Alternatively, substituting a constant depth map while keeping everything else fixed should reproduce the 'w/o depth' ablation numbers; if it does not, the depth module is doing something else.","supporting_citations":[{"cited_title":"Particlesfm: Exploiting dense point trajectories for localizing moving cameras in the wild","cited_arxiv_id":null,"evidence_quote":"Prior trajectory-based SfM baseline and the source of the global bundle adjustment and motion segmentation formulation that DATAP extends."},{"cited_title":"CoTracker : It is better to track together","cited_arxiv_id":null,"evidence_quote":"CoTracker supplies the sliding-window point tracking backbone and transformer architecture that DATAP builds on."},{"cited_title":"Learning two-view correspondences and geometry using order-aware network","cited_arxiv_id":null,"evidence_quote":"OANet decoder with diff-pool clustering is used as the decoder in the depth-aware trajectory motion segmentation."},{"cited_title":"and Singer, A","cited_arxiv_id":null,"evidence_quote":"Translation averaging method used to initialize camera poses in the global SfM pipeline."},{"cited_title":"and Govindu, V","cited_arxiv_id":null,"evidence_quote":"Rotation averaging method used to initialize camera poses in the global SfM pipeline."}],"review_version":1}