Pith. sign in

REVIEW 4 major objections 5 minor 83 references

UAV4D: Dynamic Neural Rendering of Human-Centric UAV Imagery using Gaussian Splatting

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that monocular UAV video of crowded scenes, with up to roughly fifty small moving pedestrians, can be turned into a photorealistic, editable 4D reconstruction by aligning SMPL body meshes with a dense background mesh and…

desk verdict Solid systems paper making dynamic Gaussian splatting work on UAV scenes with many tiny people, but the appendix is a mess and the scale alignment needs validation. read the letter →

arxiv 2506.05011 v1 pith:43WAKMYY submitted 2025-06-05 cs.CV

Jaehoon Choi , Dongki Jung , Christopher Maxey , Yonghan Lee , Sungmin Eum , Dinesh Manocha , Heesung Kwon This is my paper
classification cs.CV
keywords dynamicneuralrendering3DGaussiansplattingUAVimagerynovelviewsynthesishumanmeshreconstructionscaleambiguitySMPLmonocularvideo
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UAV4D aims to make dynamic neural rendering work in the setting where drones actually operate: a single moving camera, a top-down view, and ten to fifty pedestrians who each occupy only a few dozen pixels. The paper's claim is that this becomes tractable if the scene is split into a static background and dynamic humans, each represented by separate 3D Gaussian splats, with the human splats initialized from SMPL body meshes so tiny people are not ignored during optimization. To place the metric human meshes inside the non-metric background mesh, the method fits one global scale parameter by matching bone lengths measured from the body meshes against 3D joints lifted from the foundation model's point maps, then unprojects each pedestrian's ground contact point onto the reconstructed background. If correct, this turns ordinary monocular drone video into photorealistic novel views and editable scenes containing many moving people, a capability existing dynamic rendering methods do not offer for UAV imagery.

What carries the argument

The load-bearing object is a joint human-scene reconstruction carried by one global scale parameter $\sigma$ plus per-person ground contact points. The scale is fit by minimizing an L2 loss between SMPL bone lengths and bone lengths obtained by lifting 2D keypoints through the scaled point maps of the 3D foundation model; the contact point is the unprojection of the bottom-center of each person's bounding box through the depth of the background mesh, giving $\psi_t$ in world coordinates. On top of this geometry, separate Gaussian splats for humans, skinned by SMPL linear blend skinning, and for the background, initialized from the point cloud, are jointly optimized and composited.

What would settle it

Run UAV4D on a drone video of a surveyed area with known ground truth, such as a scene with pre-measured pedestrian locations or a LiDAR scan, and compare each predicted 3D ground contact point and fitted scale to the measured geometry; if the median contact-point error is comparable to body size while the background appears aligned, the scale and placement mechanism is wrong.

Watch

Extended reading notes

Core claim

The central claim is that UAV4D is the first method to enable dynamic Gaussian splatting for UAV-captured scenes with many small moving humans (the paper reports up to 50), and that it achieves state-of-the-art rendering quality on VisDrone, Manipal-UAV, and Okutama-Action, outperforming prior methods by up to 1.5 dB in PSNR while producing visibly sharper human regions. The mechanism behind the claim is that a single optimized scale parameter can unify the metric SMPL human meshes with the background mesh from a 3D foundation model, and that initializing separate human and background Gaussian splats from those unified meshes lets the optimization preserve small, moving humans instead of blending them into the static background. This produces temporally consistent novel-view synthesis and supports scene editing, such as removing or translating individual pedestrians.

Load-bearing premise

The pipeline assumes that the pretrained human-mesh model produces metrically correct bone lengths and that lifting 2D keypoints through the foundation-model point maps gives reliable 3D bone lengths in the same world frame, because those two signals set the single global scale that places every pedestrian in the scene.

Editorial extensions

If this is right

  • Novel views of drone scenes with 30–50 pedestrians can be rendered with sharper humans than prior dynamic NeRF and Gaussian baselines, at a reported gain of up to 1.5 dB PSNR.
  • Because humans and background are separate Gaussian sets, individual pedestrians can be removed or translated in the reconstructed scene, enabling synthetic UAV footage for downstream perception tasks.
  • Explicit SMPL-mesh initialization prevents small moving humans from being absorbed into the static background, which the paper identifies as the reason deformation-field baselines become unstable on such scenes.
  • On all three evaluation datasets—VisDrone, Manipal-UAV, and Okutama-Action—the human-region metrics also improve, indicating the benefit is not just background rendering.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reasonable extension, not pursued in the paper, is to use the same bone-length scale fit outside drones, e.g., ground-level monocular footage where a metric body model must be aligned to monocular depth; the only requirement is visible 2D keypoints and a point map in a consistent frame.
  • The separate decomposition suggests richer scene editing than removal and translation, such as reposing or re-timing an individual pedestrian's SMPL parameters, or inserting synthetic pedestrians; those operations follow directly from the representation but are not demonstrated.
  • The paper states that non-human dynamic objects like swaying trees and bicycles are not modeled; a direct extension would give those regions their own dynamic Gaussians, and its success would determine how far the decomposition generalizes beyond humans.
  • One testable prediction is graceful degradation as pedestrian pixel size shrinks toward the 7×17-pixel Manipal-UAV range and beyond; the method's value for high-altitude surveillance depends on that curve.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. UAV4D addresses dynamic neural rendering of monocular UAV videos containing many small pedestrians. The pipeline first uses VGGT to estimate camera poses and dense point maps, SAM2/HMR2.0 to recover human masks and SMPL meshes, and ViTPose for 2D keypoints. It reconstructs a background mesh by confidence-filtered point maps and Poisson surface reconstruction, then optimizes a single global scale sigma so that VGGT point maps align with metric SMPL bone lengths, unprojects 2D bounding-box bottom centers through the background depth to place each human, and finally optimizes separate human and background 3D Gaussian splats with SMPL-driven skinning. The method is evaluated on three UAV datasets (VisDrone, Manipal-UAV, Okutama-Action) with per-scene breakdowns and ablations, reporting improvement in PSNR/SSIM/LPIPS over TK-Planes, 3DGS, 4DGS, and Deformable-GS.

Significance. The problem is relevant: existing dynamic Gaussian splatting methods are not designed for top-down UAV scenes with 10-50 small pedestrians, and a working monocular pipeline could support simulation and perception applications. The decomposition into human and background Gaussians, and the use of SMPL-based initialization to avoid neglecting small humans, are sensible and well-motivated. The evaluation on three real UAV datasets and the ablation study are useful. However, the central scale-alignment step is not metrically validated, and the reported quantitative advantages are small and not statistically supported; the per-scene tables also contain baseline mislabeling. Hence the contribution is plausible and potentially valuable, but the evidence as presented does not yet establish the claimed metric alignment or consistent state-of-the-art quality.

major comments (4)
  1. [Sec. 3.3, Eqs. (1)-(3)] The scale optimization is the linchpin of the claimed metrically aligned world reconstruction, but its accuracy is never validated against any external metric reference. The single scalar sigma is fit by comparing HMR2.0 SMPL bone lengths with ViTPose/VGGT-lifted 3D joints for humans that are only 7-78 px tall; the paper itself notes that HMR2.0 'often produces noisy initializations' (Sec. 3.2). Because every human is placed by unprojecting a bounding-box bottom center through the sigma-scaled background (Eq. 3), a biased sigma would silently misplace all humans. The 'Ours wo Scale' ablation shows sensitivity to sigma, but not accuracy. Please report the fitted sigma values per scene, cross-validate the fitted scale on held-out persons or frames, and compare human placements against manual contact-point annotations or known metric dimensions (Manipal-UAV provides altitude).
  2. [Tables 1, 2 and Appendix Tables 4-9] The quantitative claims rest on single-run averages with no error bars, confidence intervals, or significance tests. Several margins are very small: human-region PSNR on Okutama is 19.49 vs. 19.14 for TK-Planes, and on Manipal-UAV it is 19.38 vs. 19.23 for 4DGS. The per-scene breakdowns show baseline wins in multiple scenes (e.g., Table 7, scene 1_2_2: 3DGS 19.97 vs. UAV4D 19.01; Table 9, scene 40_VGT_P1: 4DGS 15.83 vs. UAV4D 15.61). Without repeated runs or statistical tests, the 'up to 1.5 dB' improvement and the state-of-the-art claim are not established across datasets and metrics.
  3. [Appendix Tables 4-9] The appendix tables swap the baseline labels: rows named '4DGS' are cited as [67] (Deformable-GS) and rows named 'DeformableGS' are cited as [60] (4DGS), inconsistent with the main-text Tables 1 and 2. This makes the per-scene evidence difficult to interpret and must be corrected in a revision.
  4. [Abstract and Sec. 1 (Main Results)] The abstract's claim that the method 'achieves state-of-the-art rendering quality, achieving a 1.5 dB PSNR improvement' is overbroad without qualification. In Table 1, DeformableGS attains higher PSNR than UAV4D on Manipal-UAV (31.22 vs. 30.94), and the paper's own Sec. 4.2 acknowledges this exception. Please revise the abstract and contributions to state the improvement per dataset and per metric, and specify which comparison yields the 1.5 dB margin.
minor comments (5)
  1. [Sec. 3.2] The refinement criterion 'overlap between dilated human masks' is underspecified; please state whether overlap is measured as IoU or intersection, and give the threshold used.
  2. [Sec. 3.4] The term 'skimming weights' should be 'skinning weights'.
  3. [Table 3 caption] The caption says 'Visdeon dataset'; this should be 'VisDrone dataset'.
  4. [Fig. 6 caption] The word 'thie' is a typo, and the decision to omit the 3DGS baseline from this figure should be justified in the caption or text.
  5. [Appendix A.2, Eq. (6)] The relationship between Lsmpl in Eq. (5) and the regularization terms Lknn and Lsmooth is not stated explicitly; please define Lsmpl precisely and state the role of the opacity regularization Lo.

Circularity Check

0 steps flagged · score 1.0 of 10

No load-bearing circularity: scale σ is an intermediate calibration fit and the headline PSNR/novelty claims rest on held-out benchmark comparisons, not on σ.

full rationale

The paper's central outputs are (i) a claim of first dynamic Gaussian splatting for UAV scenes with many small humans and (ii) a state-of-the-art novel-view synthesis comparison. Neither output is defined in terms of its inputs. The only fitted scalar, σ in Sec. 3.3, is an intermediate calibration parameter solving Eq. (2), which matches VGGT point-map distances to HMR2.0 SMPL bone lengths. σ is never presented as a predictive result and is not reused as evidence for the rendering quality; a biased σ would degrade alignment, which is exactly a correctness/validation concern, not a self-fulfilling derivation. The evaluation is external: Section 4 reports PSNR/SSIM/LPIPS on held-out frames ('every 8th frame as the held-out test set') against TK-Planes, 3DGS, 4DGS, and DeformableGS on three public datasets. The self-citations [38,39] appear only in the related-work and baseline positioning; they do not justify the scale optimization, the Gaussian initialization, or the benchmark numbers. The limitations section candidly attributes failure modes to HMR2.0 inaccuracies and unmodeled dynamics, confirming that the framework's validity is inherited from external pretrained models rather than from the paper's own conclusions. I can exhibit no equation or table in which a claimed result reduces by construction to a fitted parameter or to a self-citation chain. The score of 1, rather than 0, merely acknowledges the presence of self-citations in the related-work survey; they are not load-bearing.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a small inventory of assumed inputs and hand-tuned parameters. The most important are the single global scale sigma, which is fitted to bone lengths, and the domain assumptions that VGGT point maps are consistent up to scale and that the bounding-box bottom is the ground contact. No new physical entities are introduced.

free parameters (5)
  • Global scale sigma = Initialized to 40; optimized via L-BFGS (30 steps, lr 0.1)
    Aligns VGGT background geometry to metric SMPL human meshes using bone-length discrepancies (Eq. 2). A single scalar fitted per dataset controls all human placement.
  • eta_box = 1.2
    Threshold for filtering HMR2.0 meshes whose projected bounding box is too large relative to the detected box.
  • eta_conf = 5 (Manipal), 20 (Okutama), 40 (VisDrone)
    Confidence percentile used to filter VGGT point maps before Poisson reconstruction; chosen per dataset based on altitude.
  • Vertical offset for pelvis = 1 unit
    Offset applied to the detected ground contact point to obtain the root translation, without measurement.
  • Scale optimization sampling = 5-10 human instances, 5 frames at intervals of 5
    Subset of data used to fit sigma; the choice affects the fitted scale.
assumptions (5)
  • domain assumption SMPL human meshes are metric and have correct bone lengths.
    The scale optimization uses HMR2.0's SMPL bone lengths as ground truth to fit sigma (Sec. 3.3).
  • domain assumption VGGT point maps are up to a single global scale and are geometrically consistent across frames.
    The whole alignment applies one scalar sigma to all point maps and poses (Sec. 3.3).
  • ad hoc to paper The bottom-center of the 2D bounding box is the foot-ground contact point.
    Human placement unprojects this point onto the mesh (Sec. 3.3); no validation with actual foot contact.
  • domain assumption The background is static; all dynamics are human.
    Background Gaussians are static; limitations acknowledge swaying trees and bicycles fail (Sec. 5).
  • standard math L-BFGS and Poisson surface reconstruction are standard and reliable for this data.
    Used as off-the-shelf tools (Sec. 3.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of UAV4D: Dynamic Neural Rendering of Human-Centric UAV Imagery using Gaussian Splatting." pith.science (2026). https://pith.science/paper/43WAKMYY

@misc{pith2026250605011,
  author       = {Pith},
  title        = {Pith review of: UAV4D: Dynamic Neural Rendering of Human-Centric UAV Imagery using Gaussian Splatting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/43WAKMYY}},
  note         = {Machine review of arXiv:2506.05011}
}
read the original abstract

Despite significant advancements in dynamic neural rendering, existing methods fail to address the unique challenges posed by UAV-captured scenarios, particularly those involving monocular camera setups, top-down perspective, and multiple small, moving humans, which are not adequately represented in existing datasets. In this work, we introduce UAV4D, a framework for enabling photorealistic rendering for dynamic real-world scenes captured by UAVs. Specifically, we address the challenge of reconstructing dynamic scenes with multiple moving pedestrians from monocular video data without the need for additional sensors. We use a combination of a 3D foundation model and a human mesh reconstruction model to reconstruct both the scene background and humans. We propose a novel approach to resolve the scene scale ambiguity and place both humans and the scene in world coordinates by identifying human-scene contact points. Additionally, we exploit the SMPL model and background mesh to initialize Gaussian splats, enabling holistic scene rendering. We evaluated our method on three complex UAV-captured datasets: VisDrone, Manipal-UAV, and Okutama-Action, each with distinct characteristics and 10~50 humans. Our results demonstrate the benefits of our approach over existing methods in novel view synthesis, achieving a 1.5 dB PSNR improvement and superior visual sharpness.

Figures

Figures reproduced from arXiv: 2506.05011 by the authors.

Figure 1
Figure 1. Comparison between the Existing Benchmarks for Dynamic Neural Rendering and UAV captured Datasets. UAVs typically cover wide areas from a top-down perspective, capturing dynamic scenes [3, 2, 7] with a distinct setup: they contain numerous human instances (ranging from 10 to 50 per frame), each occupying only a tiny fraction of the image area. challenges [17] for both accurate geometry reconstruction and photorealis… view at source ↗
Figure 2
Figure 2. Overview of Our Approach: In Section 3.2, we begin the initialization process for reconstructing the background scene and the human mesh. Then, in Section 3.3, we reconstruct the background mesh, determine the optimal scale for alignment, and position the human mesh by identifying the ground contact points. In Section 3.4, we jointly optimize the human and background Gaussians for rendering. RiSiSi T Ri T . Furtherm… view at source ↗
Figure 3
Figure 3. Human Placement. We identify the 2D contact point from the bounding box and use the depth from the mesh to unproject the 3D ground contact point. With this information, we can accu￾rately position all individual human meshes within the background mesh, with all meshes defined in the world coordinate system, as seen in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Quantitative Comparison of VisDrone Dataset [7]. We visualize the zoomed-in blue and red regions, which emphasize the dynamic humans. Our method demonstrates superior capability in reconstructing small, moving humans compared to other existing approaches. 1_2_2 1_2_8 S…
Figure 5
Figure 5. Figure 5: Quantitative Comparison of Okutama-Action Dataset [3]. We visualize the zoomed-in blue and red regions, which emphasize the dynamic humans. Our method demonstrates superior capability in reconstructing small, moving humans compared to other existing approaches. indicat…
Figure 6
Figure 6. Figure 6: Quantitative Comparison of Manipal-UAV Dataset [2]. We visualize the zoomed-in blue and red regions, which emphasize the dynamic humans. Our method demonstrates superior capability in reconstructing small, moving humans compared to other existing approaches. The render…
Figure 7
Figure 7. Figure 7: Scene Editing Samples: Due to the nature of scene decomposition, we can move or remove the humans within the black-dotted boxes. The fourth image illustrates the decomposition of the background and humans. For additional visual results, please refer to the supplementar…
Figure 8
Figure 8. Figure 8: Visual Examples of Initial Estimates. Each initial estimate is used as input for the subsequent human-scene reconstruction process. A.1 Dataset Details We select four scenes from each dataset, with most scenes comprising approximately 200 images in total, including bot…
Figure 9
Figure 9. Figure 9: shows example visualizations of human regions, comparing our method with the ground truth images. In the VisDrone dataset, human figures are approximately 30×79 pixels in size, while in the Okutama-Action dataset, they are around 17×44 pixels. VisDrone Ours GT Okutama-…
Figure 10
Figure 10. Figure 10: presents the remaining scenes not included in the main paper due to space limitations. Additionally, we provide a project website at https://uav4d.github.io/, which contains rendering videos for all scenes. Scene TK-Planes [39] 3DGS [26] 4DGS [60] UAV4D (Ours) Visual …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

83 extracted references · 45 canonical work pages

  1. [67]

    Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction

    Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20331–20341, 2024

    2024

  2. [60]

    4d gaussian splatting for real-time dynamic scene rendering

    Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20310–20320, 2024

    2024

  3. [1]

    Trajectory space: A dual representation for nonrigid structure from motion

    Ijaz Akhter, Yaser Sheikh, Sohaib Khan, and Takeo Kanade. Trajectory space: A dual representation for nonrigid structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(7): 1442–1456, 2010

    2010

  4. [2]

    Manipal-uav person detection dataset: A step towards benchmarking dataset and algorithms for small object detection

    KR Akshatha, AK Karunakar, B Satish Shenoy, K Phani Pavan, V Dhareshwar Chinmay, et al. Manipal-uav person detection dataset: A step towards benchmarking dataset and algorithms for small object detection. ISPRS Journal of Photogrammetry and Remote Sensing, 195:77–89, 2023

    2023

  5. [3]

    Okutama-action: An aerial view video dataset for concurrent human action detection

    Mohammadamin Barekatain, Miquel Martí, Hsueh-Fu Shih, Samuel Murray, Kotaro Nakayama, Yutaka Matsuo, and Helmut Prendinger. Okutama-action: An aerial view video dataset for concurrent human action detection. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 28–35, 2017

    2017

  6. [4]

    Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855–5864, 2021

    work page 2021

  7. [5]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5470–5479, 2022

    work page 2022

  8. [6]

    Zip-nerf: Anti- aliased grid-based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti- aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19697–19705, 2023

    2023

Show all 83 references
  1. [7]

    Visdrone-det2021: The vision meets drone object detection challenge results

    Yaru Cao, Zhijian He, Lujia Wang, Wenguan Wang, Yixuan Yuan, Dingwen Zhang, Jinglin Zhang, Pengfei Zhu, Luc Van Gool, Junwei Han, et al. Visdrone-det2021: The vision meets drone object detection challenge results. In Proceedings of the IEEE/CVF International conference on comp...

    2021

  2. [8]

    Does 3d gaussian splatting need accurate volumetric rendering? In Computer Graphics Forum, page e70032

    Adam Celarek, George Kopanas, George Drettakis, Michael Wimmer, and Bernhard Kerbl. Does 3d gaussian splatting need accurate volumetric rendering? In Computer Graphics Forum, page e70032. Wiley Online Library, 2025

    2025

  3. [9]

    Omnire: Omni urban scene reconstruction

    Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, Li Song, and Yue Wang. Omnire: Omni urban scene reconstruction. In The Thirteenth International Conference on Learning Representa...

    2025

  4. [10]

    Tapir: Tracking any point with per-frame initialization and temporal refinement

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. Tapir: Tracking any point with per-frame initialization and temporal refinement. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1...

    2023

  5. [11]

    Fusion4d: Real-time performance capture of challenging scenes

    Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al. Fusion4d: Real-time performance capture of challenging scenes. ACM Transactions on Graphics (ToG), 35(4):1...

    2016

  6. [12]

    4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes

    Yuanxing Duan, Fangyin Wei, Qiyu Dai, Yuhang He, Wenzheng Chen, and Baoquan Chen. 4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  7. [13]

    Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion

    Bardienus Duisterhof, Lojze Zust, Philippe Weinzaepfel, Vincent Leroy, Yohann Cabon, and Jerome Revaud. Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion. arXiv preprint arXiv:2409.19152, 2024

    2024 arXiv

  8. [14]

    St4rtrack: Simultaneous 4d reconstruction and tracking in the world

    Haiwen Feng, Junyi Zhang, Qianqian Wang, Yufei Ye, Pengcheng Yu, Michael J Black, Trevor Darrell, and Angjoo Kanazawa. St4rtrack: Simultaneous 4d reconstruction and tracking in the world. arXiv preprint arXiv:2504.13152, 2025

    2025 arXiv

  9. [15]

    K-planes: Explicit radiance fields in space, time, and appearance

    Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12479–12488, 2023

    2023

  10. [16]

    Multi-view stereo: A tutorial

    Yasutaka Furukawa and Carlos Hernández. Multi-view stereo: A tutorial. Foundations and Trends® in Computer Graphics and Vision, 9(1-2):1–148, 2015. 10

    2015

  11. [17]

    Monocular dynamic view synthesis: A reality check

    Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: A reality check. Advances in Neural Information Processing Systems, 35:33768–33780, 2022

    2022

  12. [18]

    Dense variational reconstruction of non-rigid surfaces from monocular video

    Ravi Garg, Anastasios Roussos, and Lourdes Agapito. Dense variational reconstruction of non-rigid surfaces from monocular video. In Proceedings of the IEEE Conference on computer vision and pattern recognition, pages 1272–1279, 2013

    2013

  13. [19]

    Hu- mans in 4d: Reconstructing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Rajasegaran, Angjoo Kanazawa, and Jitendra Malik. Hu- mans in 4d: Reconstructing and tracking humans with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14783–14794, 2023

    2023

  14. [20]

    Cambridge university press, 2003

    Richard Hartley and Andrew Zisserman.Multiple view geometry in computer vision. Cambridge university press, 2003

    2003

  15. [21]

    V olumedeform: Real-time volumetric non-rigid reconstruction

    Matthias Innmann, Michael Zollhöfer, Matthias Nießner, Christian Theobalt, and Marc Stamminger. V olumedeform: Real-time volumetric non-rigid reconstruction. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Pa...

    2016

  16. [22]

    Neuman: Neural human radiance field from a single video

    Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, and Anurag Ranjan. Neuman: Neural human radiance field from a single video. In European Conference on Computer Vision, pages 402–418. Springer, 2022

    2022

  17. [23]

    Dnd: Dense depth estimation in crowded dynamic indoor scenes

    Dongki Jung, Jaehoon Choi, Yonghan Lee, Deokhwa Kim, Changick Kim, Dinesh Manocha, and Dongh- wan Lee. Dnd: Dense depth estimation in crowded dynamic indoor scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12797–12807, 2021

    2021

  18. [24]

    Cotracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track together. In European Conference on Computer Vision, pages 18–35. Springer, 2024

    2024

  19. [25]

    Poisson surface reconstruction

    Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, volume 7, 2006

    2006

  20. [26]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023

    2023

  21. [27]

    Just a few points are all you need for multi-view stereo: A novel semi-supervised learning method for multi-view stereo

    Taekyung Kim, Jaehoon Choi, Seokeon Choi, Dongki Jung, and Changick Kim. Just a few points are all you need for multi-view stereo: A novel semi-supervised learning method for multi-view stereo. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6...

    2021

  22. [28]

    Hugs: Human gaussian splats

    Muhammed Kocabas, Jen-Hao Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. Hugs: Human gaussian splats. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 505–515, 2024

    2024

  23. [29]

    Robust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1611– 1621, 2021

    2021

  24. [30]

    Gart: Gaussian articulated template models

    Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, and Kostas Daniilidis. Gart: Gaussian articulated template models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19876–19887, 2024

    2024

  25. [31]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision, pages 71–91. Springer, 2024

    2024

  26. [32]

    Vastgaussian: Vast 3d gaussians for large scene reconstruction

    Jiaqi Lin, Zhihao Li, Xiao Tang, Jianzhuang Liu, Shiyong Liu, Jiayue Liu, Yangdi Lu, Xiaofei Wu, Songcen Xu, Youliang Yan, et al. Vastgaussian: Vast 3d gaussians for large scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2024

  27. [33]

    Gaussian-flow: 4d reconstruction with dynamic 3d gaussian particle

    Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao. Gaussian-flow: 4d reconstruction with dynamic 3d gaussian particle. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21136–21145, 2024

    2024

  28. [34]

    Citygaussian: Real-time high-quality large-scale scene rendering with gaussians

    Yang Liu, Chuanchen Luo, Lue Fan, Naiyan Wang, Junran Peng, and Zhaoxiang Zhang. Citygaussian: Real-time high-quality large-scale scene rendering with gaussians. In European Conference on Computer Vision, pages 265–282. Springer, 2024. 11

    2024

  29. [35]

    Smpl: A skinned multi-person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, V olume2, pages 851–866. 2023

    2023

  30. [36]

    Align3r: Aligned monocular depth estimation for dynamic videos

    Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic videos. arXiv preprint arXiv:2412.03079, 2024

    2024 arXiv

  31. [37]

    Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 2024 International Conference on 3D Vision (3DV), pages 800–809. IEEE, 2024

    2024

  32. [38]

    Uav-sim: Nerf-based synthetic data generation for uav-based perception

    Christopher Maxey, Jaehoon Choi, Hyungtae Lee, Dinesh Manocha, and Heesung Kwon. Uav-sim: Nerf-based synthetic data generation for uav-based perception. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5323–5329. IEEE, 2024

    2024

  33. [39]

    Tk-planes: Tiered k-planes with high dimensional feature vectors for dynamic uav-based scenes

    Christopher Maxey, Jaehoon Choi, Yonghan Lee, Hyungtae Lee, Dinesh Manocha, and Heesung Kwon. Tk-planes: Tiered k-planes with high dimensional feature vectors for dynamic uav-based scenes. arXiv preprint arXiv:2405.02762, 2024

    2024 arXiv

  34. [40]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65 (1):99–106, 2021

    2021

  35. [41]

    Reconstruct- ing people, places, and cameras

    Lea Müller, Hongsuk Choi, Anthony Zhang, Brent Yi, Jitendra Malik, and Angjoo Kanazawa. Reconstruct- ing people, places, and cameras. arXiv preprint arXiv:2412.17806, 2024

    2024 arXiv

  36. [42]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. arXiv preprint arXiv:2201.05989, 2022

    2022 arXiv

  37. [43]

    Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time

    Richard A Newcombe, Dieter Fox, and Steven M Seitz. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 343–352, 2015

    2015

  38. [44]

    Updating quasi-newton matrices with limited storage

    Jorge Nocedal. Updating quasi-newton matrices with limited storage. Mathematics of computation, 35 (151):773–782, 1980

    1980

  39. [45]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  40. [46]

    Structure-from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4104–4113, 2016

    2016

  41. [47]

    Pixelwise view selection for unstructured multi-view stereo

    Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. Pixelwise view selection for unstructured multi-view stereo. InEuropean Conference on Computer Vision, pages 501–518. Springer, 2016

    2016

  42. [48]

    Dynamic gaussian marbles for novel view synthesis of casual monocular videos

    Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024

    2024

  43. [49]

    Deepv2d: Video to depth with differentiable structure from motion

    Zachary Teed and Jia Deng. Deepv2d: Video to depth with differentiable structure from motion. arXiv preprint arXiv:1812.04605, 2018

    2018 arXiv

  44. [50]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras

    Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 34:16558–16569, 2021

    2021

  45. [51]

    Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs

    Haithem Turki, Deva Ramanan, and Mahadev Satyanarayanan. Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12922–12931, 2022

    2022

  46. [52]

    Vggsfm: Visual geometry grounded deep structure from motion

    Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21686–21697, 2024

    2024

  47. [53]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. arXiv preprint arXiv:2503.11651, 2025. 12

    2025 arXiv

  48. [54]

    Shape of motion: 4d reconstruction from a single video

    Qianqian Wang, Vickie Ye, Hang Gao, Weijia Zeng, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruction from a single video. 2024

    2024

  49. [55]

    Continuous 3d perception model with persistent state

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501.12387, 2025

    2025 arXiv

  50. [56]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697–20709, 2024

    2024

  51. [57]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004

    2004

  52. [58]

    Deepsfm: Structure from motion via deep bundle adjustment

    Xingkui Wei, Yinda Zhang, Zhuwen Li, Yanwei Fu, and Xiangyang Xue. Deepsfm: Structure from motion via deep bundle adjustment. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 230–247. Springer, 2020

    2020

  53. [59]

    Towards linear-time incremental structure from motion

    Changchang Wu. Towards linear-time incremental structure from motion. In2013 International Conference on 3D Vision-3DV 2013, pages 127–134. IEEE, 2013

    2013

  54. [61]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130, 2022

    2022

  55. [62]

    Vitpose: Simple vision transformer baselines for human pose estimation

    Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose: Simple vision transformer baselines for human pose estimation. Advances in neural information processing systems, 35:38571–38584, 2022

    2022

  56. [63]

    4k4d: Real-time 4d view synthesis at 4k resolution

    Zhen Xu, Sida Peng, Haotong Lin, Guangzhao He, Jiaming Sun, Yujun Shen, Hujun Bao, and Xiaowei Zhou. 4k4d: Real-time 4d view synthesis at 4k resolution. In CVPR, 2024

    2024

  57. [64]

    Track anything: Segment anything meets videos

    Jinyu Yang, Mingqi Gao, Zhe Li, Shang Gao, Fangjing Wang, and Feng Zheng. Track anything: Segment anything meets videos. arXiv preprint arXiv:2304.11968, 2023

    2023 arXiv

  58. [65]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024

    2024

  59. [66]

    Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting

    Zeyu Yang, Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:2310.10642, 2023

    2023 arXiv

  60. [68]

    Mvsnet: Depth inference for unstructured multi- view stereo

    Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: Depth inference for unstructured multi- view stereo. In Proceedings of the European conference on computer vision (ECCV), pages 767–783, 2018

    2018

  61. [69]

    Absgs: Recovering fine details in 3d gaussian splatting

    Zongxin Ye, Wenyu Li, Sidun Liu, Peng Qiao, and Yong Dou. Absgs: Recovering fine details in 3d gaussian splatting. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 1053–1061, 2024

    2024

  62. [70]

    Fast-mvsnet: Sparse-to-dense multi-view stereo with learned propagation and gauss-newton refinement

    Zehao Yu and Shenghua Gao. Fast-mvsnet: Sparse-to-dense multi-view stereo with learned propagation and gauss-newton refinement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1949–1958, 2020

    1949

  63. [71]

    Vis-mvsnet: Visibility-aware multi-view stereo network

    Jingyang Zhang, Shiwei Li, Zixin Luo, Tian Fang, and Yao Yao. Vis-mvsnet: Visibility-aware multi-view stereo network. International Journal of Computer Vision, 131(1):199–214, 2023

    2023

  64. [72]

    Monst3r: A simple approach for estimating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825, 2024

    arXiv

  65. [73]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. 13

    2018

  66. [74]

    Structure and motion from casual videos

    Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Rubinstein, Noah Snavely, and William T Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pages 20–37. Springer, 2022

    2022

  67. [75]

    Gstar: Gaussian surface tracking and reconstruc- tion

    Chengwei Zheng, Lixin Xue, Juan Zarate, and Jie Song. Gstar: Gaussian surface tracking and reconstruc- tion. arXiv preprint arXiv:2501.10283, 2025

    2025 arXiv

  68. [76]

    Dynamic scene reconstruction: Recent advance in real-time rendering and streaming

    Jiaxuan Zhu and Hao Tang. Dynamic scene reconstruction: Recent advance in real-time rendering and streaming. arXiv preprint arXiv:2503.08166, 2025

    2025 arXiv

  69. [77]

    Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting

    Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, and Yongdong Zhang. Motiongs: Exploring explicit motion guidance for deformable 3d gaussian splatting. Advances in Neural Information Processing Systems, 37:101790–101817, 2024

    2024

  70. [78]

    Complex non-rigid motion 3d reconstruction by union of subspaces

    Yingying Zhu, Dong Huang, Fernando De La Torre, and Simon Lucey. Complex non-rigid motion 3d reconstruction by union of subspaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1542–1549, 2014

    2014

  71. [79]

    Real-time non-rigid reconstruction using an rgb-d camera

    Michael Zollhöfer, Matthias Nießner, Shahram Izadi, Christoph Rehmann, Christopher Zach, Matthew Fisher, Chenglei Wu, Andrew Fitzgibbon, Charles Loop, Christian Theobalt, et al. Real-time non-rigid reconstruction using an rgb-d camera. ACM Transactions on Graphics (ToG), 33(4)...

    2014

  72. [80]

    Ewa splatting.IEEE Transactions on Visualization and Computer Graphics, 8(3):223–238, 2002

    Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. Ewa splatting.IEEE Transactions on Visualization and Computer Graphics, 8(3):223–238, 2002. A Dataset & Training Details Scene SAM2 [45] ViTPose [62] HMR2.0 [19] Figure 8: Visual Examples of Initial Estima...

    2002

  73. [81]

    VisDrone: uav0013_00000, uav0079_00480, uav0084_00000, uav0099_02109

  74. [82]

    Manipal-UA V: 40_GND_P1, 40_VGT_P1, 50_GND_P1, 50_RD_P1

  75. [83]

    In the Okutama dataset, humans typically appear at an average size of 17×44 pixels

    Okutama-Action: 1_2_2, 1_2_6, 1_2_8, 1_2_9 In the VisDrone dataset, humans have an average size of approximately 29×78 pixels, while in the Manipal-UA V dataset, the average size is around 7×17 pixels. In the Okutama dataset, humans typically appear at an average size of 17×44...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.