{"id":"0d3c5fa7-33b2-4b46-bb58-9c40f316bada","arxiv_id":"2607.19517","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Crowd4D introduces HSIP scene-anchored proxies and structural coherence regularization to reconstruct scene-consistent 4D crowds from monocular video, outperforming DyCrowd on VirtualCrowd.","lead":"Crowd4D reconstructs 4D crowds from a single monocular video by anchoring each person to a reconstructed ground surface instead of a flat plane. It reports better scene-consistent results than prior methods on the VirtualCrowd benchmark and shows qualitative gains on real gigapixel footage.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"HSIP anchors rely on unvalidated monocular SIS accuracy; synthetic-only evaluation may not transfer to real large-scale scenes.","rationale":"The reader's weakest assumption identifies precisely the condition I find most load-bearing: the fidelity of the SIS and the lowest-vertex ground proxy. My analysis agrees and sharpens it: SIS errors enter through Eq. 18 as a hard quadratic penalty, and the same SIS anchors define the CSCR graph, so errors can be spatially amplified. The paper demonstrates state-of-the-art results on a synthetic benchmark and includes useful ablations, but it does not test the method's core dependency on monocular scene geometry under realistic noise. This is not an internal inconsistency; the method may work well when π3/GeeCalib outputs are accurate. However, the central claim extends to real large-scale scenes, and the absence of any sensitivity analysis or real-world quantitative validation leaves that extension unsupported. Thus the conditional verdict is appropriate: the paper should add an ablation isolating SIS quality before the claims are treated as established. I do not see grounds to reject or to move to unverdictable, given the synthetic evidence and the plausible mechanism.","tokens_in":14946,"tokens_out":6472,"duration_ms":67061,"concrete_test":"On the VirtualCrowd benchmark, re-run the full pipeline with the SIS constructed from ground-truth scene geometry instead of π3 predictions, and also repeat with gravity perturbed by ±2°. Compare PPDS and MPJPE between the predicted-SIS and GT-SIS runs. If the gap exceeds ~10% of Crowd4D's reported improvement over DyCrowd, the method's quantitative gains are contingent on scene-reconstruction accuracy; if the gap is negligible, HSIP is robust to SIS errors.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—scene-consistent metric-scale crowd reconstruction—depends on HSIP anchors s* (Eqs. 10–15) derived from the SIS, which is built from monocular π3 point clouds and GeoCalib gravity (Sec. 3.2.1). The vertical constraint E_HSIP^(y) (Eq. 18) is a quadratic loss that forces the lowest SMPL vertex exactly onto the SIS height. Hence any SIS error—occluded ground, dynamic objects, gravity misalignment, or interpolation across unsupported regions—propagates directly into root translations and, through the CSCR neighborhood graph (Sec. 3.5), into the whole crowd structure. The paper's only quantitative evidence is on VirtualCrowd, a synthetic benchmark with clean geometry; no sensitivity analysis of SIS quality is provided. Real large-scale monocular footage (surveillance, drones) has textureless regions, occlusions, and dynamic scene content that can substantially degrade π3 point maps and GeoCalib gravity estimates. Without quantitative validation of SIS fidelity—or an ablation showing robustness to SIS noise—the observed PPDS/MPJPE gains may not transfer to real scenes, which is the stated target. This is the least-secure condition for the abstract's promise of 'scene-consistent' reconstruction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Crowd4D, an optimization-based pipeline for monocular 4D crowd reconstruction in large scenes. It initializes per-person SMPL tracks and motions from monocular predictors, and scene geometry and camera trajectories from π3 and GeoCalib. It then constructs a terrain-aware Human-Scene Interaction Proxy (HSIP) from a Scene Interaction Point Cloud and Scene Interaction Surface, and performs a three-stage optimization that refines root positions, poses, and motion latents while optimizing a residual global scene scale. A Crowd Structural Coherence Regularization (CSCR) term enforces temporal group-level consistency. Quantitative comparisons on the VirtualCrowd synthetic benchmark report consistent improvements over Crowd3D, GroupRec, DyCrowd, and VideoMimic on PPDS/MPJPE, and qualitative results are shown on the PANDA real-world dataset. Ablations attribute gains to the multi-stage design, HSIP-based alignment, and CSCR.","tokens_in":15349,"tokens_out":6284,"duration_ms":66929,"significance":"If the reported numbers are representative, this is a useful step beyond single-plane crowd reconstruction: the PPDS gains of approximately +5.8 and +6.8 over DyCrowd (Table 2) and the reductions in world-space MPJPE are notable, and the method is training-free at optimization time. The paper is also transparent in providing ablation studies that show each stage and each proposed loss contributes, and the use of external monocular scene reconstruction avoids a fully circular evaluation. However, the significance is conditional: the method's key novelty—HSIP/SIS anchoring—is not stress-tested against scene-reconstruction noise, and the only quantitative benchmark is synthetic, so the claim of robust real-world performance is not yet supported.","major_comments":[{"comment":"The abstract and Sec. 1 claim Crowd4D 'jointly optimizes the crowd and scene.' What is implemented is joint optimization of human parameters and one residual global scale α (Table 1); the scene geometry, SIS, and camera trajectory are fixed after initialization and are not re-estimated. The claim should be narrowed to 'jointly optimizes crowd parameters and a residual scene scale,' or the method should demonstrate actual scene-geometry updates. This matters because the paper's stated contribution is coupling crowd and scene reconstruction.","section":"Abstract; Sec. 3.3–3.4"},{"comment":"The HSIP anchor is constructed from the initialized root translation's viewing ray: a_n,t in Eq. (7) is the xz of \\bar{τ}_{n,t}, and r*_{n,t} is the root-ray intersection in Eq. (11). Stage-1 then optimizes τ toward this same anchor via E_HSIP^{(xz)} and E_HSIP^{(y)} and a direct update τ ← r*. Because the anchor inherits the initial monocular depth error, the constraint is not independent evidence from the scene: a wrong initial depth changes the ray–SIS intersection and can pull the root to a wrong but scene-consistent-looking location. The paper should quantify this by comparing against a baseline that smooths/clamps the initial root depth without ray–SIS anchoring, and by perturbing initial depth in a sensitivity experiment.","section":"Sec. 3.2.2, Eqs. (10)–(15); Sec. 3.4, Eqs. (17)–(18)"},{"comment":"The quantitative evaluation is entirely on VirtualCrowd, a synthetic benchmark with clean geometry and ground-truth annotations; real-world PANDA results are qualitative (Fig. 4). The abstract and conclusion claim robust reconstruction in real large-scale scenes, but no quantitative real-scene evidence is provided, and the conclusion explicitly acknowledges this. Since HSIP/SIS is the core novelty and its success depends on monocular scene reconstruction quality, the paper needs either (i) a pseudo-ground-truth quantitative evaluation on a real dataset (e.g., manually annotated contact points or a few annotated sequences), or (ii) a sensitivity analysis where SIS noise, gravity errors, and occlusion are synthetically injected to show graceful degradation. Without this, the claimed real-world transferability is not established.","section":"Sec. 4.1; Sec. 4.3; Conclusion"},{"comment":"The optimization weights λ_* are never assigned numerical values. Table 1 lists term names, not weights, and Appendix A defines the losses but says weights are in Table 1. For a purely optimization-based method, missing weight values make the experiments irreproducible and prevent assessing sensitivity of the reported PPDS/MPJPE gains to tuning. Please provide the full weight schedule and, if possible, a small sensitivity analysis over the dominant weights (λ_xz, λ_y, λ_proj, λ_crowd).","section":"Sec. 4.2; Table 1; Appendix A"}],"minor_comments":[{"comment":"d_smpl, d_scn, and ω are not formally defined. The text says ω gives higher weight to closer individuals, but the exact formula should be given.","section":"Sec. 3.1, Eq. (2)"},{"comment":"There are typos in the figure and caption: 'Sta;c Camera' should be 'Static Camera' in two places, and the scenario labels are visually inconsistent with the third image. Please clean up.","section":"Fig. 1"},{"comment":"The direction term 1 - (u·v)/(||u||||v||+ε) can be negative for noisy estimates. Since it is inside a robust penalty ρ(·), the behavior on negative values should be clarified or the term should be explicitly nonnegative.","section":"Eq. (21)"},{"comment":"For close-range scenes, the HSIP support range r_HSIP is set to a 'fixed constant,' but no value or criterion for 'close-range' is given. Please specify the rule and the constant.","section":"Sec. 3.2.2"},{"comment":"ACCEL of Ours under unified tracking (16.10) is slightly worse than DyCrowd (15.72), and SLAHMR-Large* is best under GT tracking (12.25). This should be discussed rather than only highlighting MPJPE, especially since the paper emphasizes temporal smoothness.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The numerical core is credible, but the paper currently overclaims joint scene optimization and real-world robustness. I recommend major revision, with emphasis on validating HSIP robustness to scene-reconstruction errors and providing full optimization weights. The synthetic-only evaluation is the main obstacle to acceptance at this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this paper is not a paradigm shift, but it is a real step forward for monocular crowd reconstruction in large scenes. It replaces the flat-ground HVIP anchoring from Crowd3D/DyCrowd with HSIP, built from a scene interaction point cloud and surface, and adds CSCR to stabilize relative crowd structure over time. Those two pieces are new relative to the prior lineage. The VirtualCrowd results support the core claim: PPDS 89.04/91.43 vs DyCrowd 83.21/84.66, MPJPE down by roughly 8–9.5mm, and the occlusion subset improves too. The ablations show each stage and loss pulling its weight. I believe the empirical case is credible, with caveats.\n\nWhat is well done: the SIPC/SIS construction is a sensible way to compress noisy monocular point clouds into a queriable support surface; the HSIP feasible-region formulation with a support range is a reasonable generalization of HVIP; CSCR is cheap and does what it says. The paper is clearly written and the comparison to VideoMimic in Table 4 is fair.\n\nSoft spots, in order of seriousness. First, the 'jointly optimizes the crowd and scene' framing is too strong. Scene geometry is fixed after π3 + GeoCalib and metric scale initialization; the only scene variable optimized is the residual global scale α. That is not joint scene optimization. This is visible in Table 1 and should be fixed in revision. Second, the stress-test concern is real: HSIP anchors come from a monocular SIS, and Eq. 18 forces the lowest SMPL vertex onto that surface. Any error in SIS—occluded ground, dynamic objects, gravity misalignment, interpolation across unsupported regions—propagates directly into root positions and through CSCR into the neighborhood. No sensitivity analysis or SIS-noise ablation is given. VirtualCrowd has clean geometry; real surveillance and drone scenes do not. Third, the quantitative evaluation is single-run and synthetic-only, with PANDA qualitative; no code or full hyperparameters. That limits confidence but does not undermine the main result. Fourth, a mild circularity: HSIP reference positions are built from initialized root rays, and Stage-1 pulls roots toward those anchors. It is not hard circularity because projection consistency and the SIS anchor constrain things, but the anchors are not independent evidence.\n\nWho is this for? People working on dynamic crowd reconstruction or scene-grounded human motion recovery. It deserves a serious referee; I would send it out. Request code, SIS-perturbation analysis, and a tempered abstract.","headline":"Crowd4D is a genuine extension of the DyCrowd line—scene-surface anchoring plus a structural coherence term—with solid synthetic results, but the 'jointly optimizes crowd and scene' claim overstates what is actually optimized and the monocular scene-quality dependence is untested.","tokens_in":15832,"tokens_out":2682,"would_cite":true,"duration_ms":29889,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Crowd4D claims that monocular video alone can yield metric-scale, terrain-accurate 4D crowd motion by optimizing people and scene geometry together instead of assuming a flat ground.","keywords":["4D crowd reconstruction","monocular video","human-scene interaction","scene-aware optimization","SMPL","occlusion robustness","large-scale scenes","terrain surface"],"falsifier":"Take a monocular video of a person walking up a staircase where the reconstructed Scene Interaction Surface flattens the steps into a smooth ramp. If the vertical HSIP term forces the lowest body vertex onto that smooth surface while the person's feet visibly leave it on a step edge, world-space root height and mean joint error against a LiDAR or motion-capture ground truth will deviate by the step height even though 2D reprojection is near-perfect — showing the proxy inherits scene-geometry error rather than correcting it.","tokens_in":14879,"feed_emoji":"🎥","tokens_out":5891,"duration_ms":62987,"temperature":0.7,"pith_summary":"Most monocular crowd reconstruction assumes the ground is a single plane, so on stairs, slopes, or uneven plazas the recovered people drift in scale and position. Crowd4D instead tries to establish that scene-consistent, metric-scale 4D crowd motion can be recovered from one RGB video by tying the crowd to the reconstructed scene itself. The key move is an intermediate proxy — the Human-Scene Interaction Proxy (HSIP) — built from a distilled terrain point cloud and surface, which defines a feasible region for each person's root along the viewing ray and is refined through a three-stage optimization that also adjusts a residual global scale. On the synthetic benchmark used for evaluation, the reported global-consistency score improves from 83.21 to 89.04 under detection-based tracking and from 84.66 to 91.43 with ground-truth tracks, with mean per-joint position error dropping by roughly 8 to 9.5 millimeters. If this stands, large-scale crowd analysis moves from a flat-world approximation to terrain-aware world-space reconstruction from ordinary surveillance or drone video.","feed_headline":"One camera places crowds on real terrain in 4D","feed_subtitle":"A scene-aware optimizer anchors each person to the reconstructed ground, cutting position error and surviving dense occlusions.","key_machinery":"The load-bearing object is the Human-Scene Interaction Proxy (HSIP): for each tracked person at each frame, the lowest SMPL body model vertex is used as a ground-contact proxy, intersected in ray space with a reconstructed Scene Interaction Surface (SIS) — a triangulated height field distilled from a confidence-filtered Scene Interaction Point Cloud (SIPC). HSIP yields a reference root position and a support range that define a feasible region on the terrain. Three losses — horizontal hinge into the feasible region, vertical matching of the lowest vertex to the surface height, and 2D projection consistency — carry the scene constraint into the optimization, while a residual global scale is o","core_discovery":"On its own terms, Crowd4D claims that the long-standing scale ambiguity of monocular 4D crowd reconstruction can be resolved by optimizing humans and scene geometry jointly, rather than treating either as a fixed backdrop. It introduces the Human-Scene Interaction Proxy (HSIP), an intermediate geometric anchor obtained by projecting each person's lowest body vertex along the camera ray onto a Scene Interaction Surface distilled from reconstructed point clouds. Stage-1 optimizes root positions, orientations, and a residual global scale against three HSIP constraints — horizontal feasible-region, vertical terrain height, and image-projection consistency. Stage-2 refines body pose under an anat","pith_inferences":["The HSIP proxy is general enough to be reused for placing virtual characters onto real scenes, checking navigable surfaces, or validating whether a tracked person's support is plausible.","Because the scene surface is reconstructed once and held static, moving objects such as vehicles or temporary barriers are outside the model's world; a per-frame local-surface update would be a natural extension the video input makes possible.","The ratio-based metric-scale initialization could be turned into a training signal for a feed-forward network, which would make the hour-scale optimization unnecessary at inference.","As the paper itself hints, real-scene quantitative evaluation is currently blocked by missing 3D pose annotations; pseudo-labeling the optimized output is the obvious test bed."],"forward_implications":["Single-plane assumptions in monocular crowd reconstruction can be retired: stairs, ramps, and uneven plazas are handled by the same optimization as flat ground, with no change of model.","The method is optimization-based and training-free, so it applies to arbitrary monocular video; the cost is roughly four hours for 100 people over 200 frames on a consumer GPU, reducible to about one hour.","Occlusion robustness comes specifically from group-level coherence: removing the crowd-coherence term raises mean joint error from 61.76 to 66.24 millimeters in the paper's controlled comparison.","World-space accuracy is gated by the quality of scene reconstruction and tracking; when those fail, even perfect image-space replay cannot fix the world-space placement."],"fun_headline_variants":["Joint crowd-scene optimization nails monocular 4D scale","Scene-aware anchor removes monocular crowd depth drift","4D crowds anchored to real terrain from a single camera","Monocular video yields scene-consistent 4D crowd motion","One video, 4D crowds grounded on actual terrain"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The pipeline assumes the reconstructed scene surface and gravity are correct, and that the lowest body vertex of every person is the actual ground contact; if either fails, the anchor drags the person to the wrong height and position.","fun_headline_variants_meta":{"raw":{"variants":["Joint crowd-scene optimization nails monocular 4D scale","Scene-aware anchor removes monocular crowd depth drift","4D crowds anchored to real terrain from a single camera","Monocular video yields scene-consistent 4D crowd motion","One video, 4D crowds grounded on actual terrain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000637,"raw_usage":{"total_tokens":2794,"prompt_tokens":790,"completion_tokens":2004,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":1937}},"tokens_in":534,"tokens_out":2004,"duration_ms":13618,"temperature":1.0,"reasoning_tokens":1937,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T12:29:55.179842+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a monocular video of a person walking up a staircase where the reconstructed Scene Interaction Surface flattens the steps into a smooth ramp. If the vertical HSIP term forces the lowest body vertex onto that smooth surface while the person's feet visibly leave it on a step edge, world-space root height and mean joint error against a LiDAR or motion-capture ground truth will deviate by the step height even though 2D reprojection is near-perfect — showing the proxy inherits scene-geometry error rather than correcting it.","supporting_citations":[],"review_version":1}