Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Vysics: Object Reconstruction Under Occlusion by Fusing Vision and Contact-Rich Physics

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Vysics claims that a robot can reconstruct a heavily occluded object's full shape by fusing visible RGBD geometry with contact geometry inferred from the object's observed motion.

desk verdict A credible vision-physics fusion with genuinely new mutual-supervision losses; geometry results are solid, but the dynamics evaluation is in-sample and friction sensitivity is unexamined. read the letter →

arxiv 2504.18719 v1 pith:AN4CSCRA submitted 2025-04-25 cs.RO

Bibit Bianchini , Minghan Zhu , Mengti Sun , Bowen Jiang , Camillo J. Taylor , Michael Posa This is my paper · ORCID
classification cs.RO
keywords objectreconstructionocclusioncontact-richmanipulationsigneddistancefunctiondeepsupportdynamicsmodellearningRGBDvideorigidbody
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vysics claims that a robot holding a short RGBD video of a heavily occluded object can recover not only the visible geometry but also the hidden shape, by treating the object's observed motion as evidence about where it was touched. It fuses a vision-based tracker and shape estimator with a contact-implicit dynamics learner: the dynamics learner proposes a convex "physible" geometry that explains the trajectory through hypothesized contacts, and both streams supervise a signed distance function. The paper reports that on a new dataset of a robot arm pushing, pivoting, and toppling seven everyday objects behind occluders, Vysics cuts average chamfer distance roughly in half relative to vision-only reconstruction and produces simulated rollouts that stay accurate longer. If right, this gives robots a way to build usable object models on the spot without pretraining, CAD models, or force and tactile sensors.

What carries the argument

The central object is the deep support function, an input-convex neural network that represents the convex hull of a shape by returning, for any unit direction, the distance to the furthest surface point in that direction; its gradient is the support point itself. PLL uses this DSF to learn geometry from observed trajectories through an implicit contact-dynamics loss, while BundleSDF maintains a signed distance function from the RGBD video. Vysics bridges the two by adding a visible-geometry loss to the DSF training, and by converting DSF support points into three SDF supervision terms: a ray-based support point loss, a hyperplane-constrained lower bound on signed distance, and a convexity bias that connects sparse contact points to the visible surface.

What would settle it

Take a known object with a deep concavity, such as a mug's handle side or a bowl's inside, record an RGBD video where the robot's sphere deliberately contacts only the concave region so the object moves, run Vysics, and compare the reconstructed mesh to the ground truth; if the output replaces the concavity with a convex hull or the inferred contact point lies outside the true surface, the central claim is falsified. A cheaper check is to rerun the same video with intentionally wrong friction coefficients and observe whether the recovered geometry shifts systematically.

Watch

Extended reading notes

Core claim

The central discovery claim is that occluded geometry is observable through dynamics: a trajectory of a rigid body interacting with known surfaces encodes its shape wherever contact occurred, and that contact-inferred geometry can be integrated with visual depth into a single signed distance function. Vysics claims that the visible and "physible" geometries are complementary—vision constrains the sides seen, contact dynamics constrains the sides touched—and that jointly optimizing a hybrid SDF under both sources yields higher geometric accuracy and better dynamics predictions than vision alone. This is demonstrated by feeding BundleSDF-estimated poses and meshes into PLL, filtering the hypothesized contact points by force magnitude, and then running BundleSDF a second time with additional support-point, hyperplane, and convexity losses derived from the contact geometry.

Load-bearing premise

The whole inference assumes that every motion the robot sees is caused by rigid-body contact with the flat table or the spherical gripper, and that those contacts only ever touch the object's convex outer surface; if a robot pokes a hollow or recessed part, the model will misattribute the cause.

Editorial extensions

If this is right

  • Occluded faces that make contact get reconstructed even when never visible, so interaction plans can trust the model where vision alone would fail.
  • The same pipeline yields inertial parameters and a URDF, so simulated rollouts can be used to test manipulation strategies before execution.
  • Because no pretraining or fiducials are needed, the method applies to novel objects immediately from a few seconds of RGBD data plus robot joint states.
  • Dynamics prediction quality, measured by pose-error duration and contact-activation overlap, improves over vision-only geometry even in chaotic open-loop rollouts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The loss machinery would support an active perception loop: a robot unsure of hidden geometry could poke or tap the object at selected locations and use the resulting motion to reduce uncertainty, a direction the paper lists as future work.
  • The convex-hull limitation implies the method would systematically misreport concave features such as handle holes or recessed grips if contacts occur there; extending the DSF to a nonconvex implicit contact representation is a natural next step.
  • Because friction coefficients are hand-set, the inferred geometry could absorb errors in friction; holding friction fixed at wrong values on a known object and measuring how the reconstructed hull shifts would quantify that coupling.
  • The two-round BundleSDF-PLL design suggests a fixed-point scheme in which pose and shape mutually refine; if that iteration converges, it could remove the dependence on the initial BundleSDF pose quality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. Vysics proposes to reconstruct a rigid object's geometry from a short RGBD video by fusing BundleSDF's vision-based tracking/reconstruction with PLL's contact-implicit dynamics learning. The pipeline first runs BundleSDF to obtain poses and a visible mesh, runs PLL (with a convex-hull DSF) to infer contact forces, support points, and inertia under both dynamics and vision losses, then runs BundleSDF a second time with support-point, hyperplane, and convexity losses derived from the PLL output. The method is evaluated on a new dataset of Franka Panda interactions under occlusion, reporting lower chamfer distance and higher IoU than vision-only baselines, and better open-loop dynamics rollouts than a BundleSDF-geometry baseline. The authors also release a URDF output and dataset.

Significance. The central idea is attractive and timely: contact observations during manipulation are an underused cue for completing occluded geometry, and the paper's formulation via support points and hyperplane constraints is clear and well motivated. If the reported geometry improvements hold, the work is a useful step beyond vision-only completion, with the practical advantage of requiring no pretraining or tactile sensors. The geometric evaluation against ground-truth meshes (chamfer distance and IoU) is an appropriate external check, and the new RGBD dataset is a valuable resource. The loss derivations in Eqs. (7)-(13) are mathematically sound under the stated convexity assumption. However, the dynamics-prediction evaluation currently appears to be an in-sample fit, and the hand-set friction coefficients are not subjected to sensitivity analysis; until these are addressed, the breadth of the claims should be considered provisional.

major comments (4)
  1. [§V-B, §VI-B] The dynamics-prediction evaluation compares open-loop simulations against the same BundleSDF trajectory that was used to train PLL and the second BundleSDF pass. Because the model is optimized to reproduce that exact trajectory, the reported reductions in pose error and time-before-divergence on that trajectory can reflect overfitting to the training signal rather than genuine predictive skill. Please evaluate on held-out interaction sessions for each object, or explicitly frame the metric as training-set fit and support the generalization claim with cross-validation.
  2. [§V-A, §IV-B] The 'physible' geometry that supervises the final SDF via Eqs. (8) and (11) is derived from PLL's hypothesized contact forces under Coulomb friction with hand-set coefficients mu_table=0.26 and mu_robot=0.15 for all objects. The paper notes in §V-A that friction is difficult to identify uniquely in this dataset, yet no sensitivity study or ablation over these values is reported. Under misspecified friction, the optimizer can trade friction-cone violations against geometry, shifting support points and hyperplane constraints and thus biasing the final shape. Please add a sensitivity analysis over a plausible range of friction values, or otherwise show that the chamfer/IoU results are insensitive to these choices.
  3. [§III-B, §VII] The DSF can only represent the convex hull, and the paper assumes contacts occur only on the object's convex hull. This assumption is central because any contact on a concave region maps to a support point on the hull rather than the true contact point, directly corrupting the supervision in Eqs. (8) and (11). The manuscript provides no quantitative check of whether the recorded robot/table contacts actually satisfy this condition for the seven objects. Please add an analysis of contact locations (e.g., from manual annotation or by comparing PLL's support points to the ground-truth mesh), or explicitly document the convex-hull assumption as an untested limitation with its likely effect on the results.
  4. [§IV, pipeline description] The second BundleSDF run is described as 'BundleSDF runs again' with the new physics losses, but the paper does not state whether object poses are re-estimated in this run. If poses are re-estimated, the trajectory used to train PLL no longer matches the trajectory associated with the final SDF, and the dynamics evaluation becomes ambiguous; if poses are frozen, that should be stated explicitly and the implementation should be made reproducible. Please clarify this point.
minor comments (6)
  1. [Table I and Figures 8-12] Table I and Figures 8-12 would benefit from reporting the number of sessions per object and per-metric standard deviations or confidence intervals; the current per-object 'all' averages do not indicate variance across sessions.
  2. [§IV-A] The 'visibility check' that defines the visible set V is not described; please specify how visible vertices are determined from the RGBD video.
  3. [Abstract, §I] The abstract and introduction state Vysics requires 'no fundamental priors,' yet the method uses a convexity assumption, hand-set friction coefficients, and a table-plane model; consider softening this phrasing.
  4. [§VI-A, Figure 6] The qualitative comparison in Figure 6 and the statement that generative models 'do not generate a complete shape' would be more compelling if at least one representative quantitative result was reported for these methods.
  5. [Eq. (6)] In Eq. (6), the loss is written as a norm without explicitly stating whether it is L1 or L2; please clarify the norm used in the implementation.
  6. [§V-A] The dataset description in §V-A does not list the number of recorded sessions per object; please add this information for reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

The dynamics-prediction evaluation replays the same BundleSDF trajectory used to fit PLL, so the claimed 'prediction' is an in-sample training fit; the geometry claim against ground-truth meshes is independent.

  1. fitted input called prediction [Section IV (PLL input) and Section V-B / VI-B (dynamics prediction evaluation)]
    "Using the estimated object geometry and its tracked pose at the first frame as the initial condition, we generate simulated trajectories of the object. The predicted trajectories are compared with the real-world trajectory tracked by BundleSDF to show how well the estimated geometry explains the dynamics."

    The geometry and inertia used in this rollout were learned by PLL from this same BundleSDF-estimated trajectory: Section IV says 'we obtain the object trajectory ... via BundleSDF' and 'PLL detects physible portions of the geometry by inferring contact events in the observed dynamics.' The predicted trajectory is therefore a re-simulation of the exact data used to fit the model, so the comparison measures training-data reconstruction, not out-of-sample prediction. The claim of 'better dynamics prediction' is not independently supported by this metric; the geometric evaluation against external ground-truth meshes remains valid.

full rationale

The headline geometry claim is not circular: chamfer distance and volumetric IoU are computed against ground-truth meshes that are external to the optimization, and the vision/physics losses (Eqs. 6, 8, 11, 13) are stated modeling assumptions, not restatements of the evaluation target. The circularity is localized to the dynamics-prediction evaluation. PLL optimizes geometry and inertia from the very BundleSDF trajectory of each session, and Section V-B then 'predicts' that same session by replaying the recorded end-effector commands and comparing the rollout to the same BundleSDF trajectory. Thus the simulated trajectories are fitted values by construction, so the quantitative dynamics comparison shows training fit rather than generalization. This affects the 'better dynamics prediction' claim but does not undermine the ground-truth-validated shape reconstruction results. The fixed friction coefficients (0.26, 0.15) are a modeling sensitivity concern, not a circular step, and no load-bearing uniqueness theorem or self-citation chain is invoked to force the result.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central geometry claim rests on rigid-body and convex-hull contact assumptions, on the accuracy of BundleSDF poses, and on manually chosen friction and loss weights. No new physical entities are introduced.

free parameters (7)
  • object-table friction coefficient = 0.26
    Hand-set in §V-A because sliding mostly happens under robot contact, so friction is not identifiable from the data. Used in PLL's contact force decomposition.
  • object-robot friction coefficient = 0.15
    Hand-set in §V-A for all experiments. Affects inferred contact forces and therefore the physible geometry.
  • support point loss weight (Eq. 8) = 2
    Chosen hyperparameter for modified BundleSDF, Appendix A Table II.
  • hyperplane-constrained loss weight (Eq. 11) = 1
    Chosen hyperparameter, Appendix A Table II.
  • convexity loss weight (Eq. 13) = 1
    Chosen hyperparameter, Appendix A Table II.
  • vision supervision weight on PLL (Eq. 6) = 0.04
    Chosen hyperparameter on top of PLL losses, Appendix B Table III.
  • physible point force filter quantile = top 30%
    Only queried points with the top 30% of hypothesized contact forces are retained for SDF supervision, Appendix C.
assumptions (6)
  • domain assumption The object is a single rigid body with no elasticity or deformation during the recorded interaction.
    PLL and BundleSDF assume a rigid body; §VII acknowledges elasticity is ignored and can cause deviations.
  • domain assumption All contacts between object, table, and robot occur on the object's convex hull.
    Stated in §III-B: the DSF can represent only the convex hull, and the assumption is satisfied 'as long as collisions occur only on the object's convex hull.'
  • domain assumption The robot interacts with a single object at a time, so the convexity bias in §IV-B4 is valid.
    Introduced in the convexity loss section: 'With the assumption that the robot is interacting with a single object at a time...'
  • domain assumption The table plane and end-effector sphere are known geometry, and the camera-to-robot transform is calibrated.
    §V-A: table height detected from depth, end-effector sphere from joint states and known transform; PLL uses these as environment contacts.
  • domain assumption BundleSDF pose estimates are accurate enough to serve as the trajectory input for PLL and as the reference for dynamics evaluation.
    The pipeline feeds BundleSDF poses into PLL (§IV-A) and compares simulated rollouts to BundleSDF tracked poses (§V-B).
  • standard math The signed distance function of a convex set satisfies the support-ray equality (8), the hyperplane lower bound (10), and the convex interpolation inequality (12).
    The new SDF supervision losses rely on these convex analysis properties without additional proof in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vysics: Object Reconstruction Under Occlusion by Fusing Vision and Contact-Rich Physics." pith.science (2026). https://pith.science/paper/AN4CSCRA

@misc{pith2026250418719,
  author       = {Pith},
  title        = {Pith review of: Vysics: Object Reconstruction Under Occlusion by Fusing Vision and Contact-Rich Physics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AN4CSCRA}},
  note         = {Machine review of arXiv:2504.18719}
}
read the original abstract

We introduce Vysics, a vision-and-physics framework for a robot to build an expressive geometry and dynamics model of a single rigid body, using a seconds-long RGBD video and the robot's proprioception. While the computer vision community has built powerful visual 3D perception algorithms, cluttered environments with heavy occlusions can limit the visibility of objects of interest. However, observed motion of partially occluded objects can imply physical interactions took place, such as contact with a robot or the environment. These inferred contacts can supplement the visible geometry with "physible geometry," which best explains the observed object motion through physics. Vysics uses a vision-based tracking and reconstruction method, BundleSDF, to estimate the trajectory and the visible geometry from an RGBD video, and an odometry-based model learning method, Physics Learning Library (PLL), to infer the "physible" geometry from the trajectory through implicit contact dynamics optimization. The visible and "physible" geometries jointly factor into optimizing a signed distance function (SDF) to represent the object shape. Vysics does not require pretraining, nor tactile or force sensors. Compared with vision-only methods, Vysics yields object models with higher geometric accuracy and better dynamics prediction in experiments where the object interacts with the robot and the environment under heavy occlusion. Project page: https://vysics-vision-and-physics.github.io/

Figures

Figures reproduced from arXiv: 2504.18719 by the authors.

Figure 1
Figure 1. Vision-based shape reconstruction (projection shown in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A 2D depiction of the physical meaning of a DSF (3) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Detailed Vysics diagram. Blue arrows denote the vision-based information flow through BundleSDF [66], and green arrows for PLL [9, 53]. Purple arrows indicate the unifying connections Vysics makes to factor both vision and contact-rich physics into the geometry learning problem. and gradient, DSF(ˆn) = max si∈S si · ˆn, (2) ∇ˆnDSF (ˆn) = arg max si∈S si · ˆn =: s. (3) The queried normal direction ˆn and its associat… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Visualization of the loss functions as the incorporation [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The 7 objects and their names in our dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: A qualitative example of generative single-view recon [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: The quantitative comparison of the geometric recon [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 12
Figure 12. Figure 12: For quantifying dynamics prediction performance, [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]
Figure 10
Figure 10. Figure 10: The dynamics prediction accuracy evaluated by the [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: The dynamics prediction accuracy evaluated by the [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling

    cs.CV 2026-02 unverdicted novelty 6.0 of 10

    Picasso produces multi-object scene reconstructions that are both geometrically accurate and physically plausible by using physics-constrained rejection sampling over an inferred contact graph, outperforming prior met...

Reference graph

Works this paper leans on

78 extracted references · 65 canonical work pages · cited by 1 Pith paper

  1. [1]

    Physically embodied gaussian splat- ting: A realtime correctable world model for robotics

    Jad Abou-Chakra, Krishan Rana, Feras Dayoub, and Niko Suenderhauf. Physically embodied gaussian splat- ting: A realtime correctable world model for robotics. In 8th Annual Conference on Robot Learning , 2024. URL https://openreview.net/forum?id=AEq0onGrN2

    work page 2024

  2. [2]

    Vali- dating robotics simulators on real-world impacts

    Brian Acosta, William Yang, and Michael Posa. Vali- dating robotics simulators on real-world impacts. IEEE Robotics and Automation Letters, 7(3):6471–6478, 2022

    work page 2022

  3. [3]

    Amodal 3d reconstruction for robotic manipulation via stability and connectivity

    William Agnew, Christopher Xie, Aaron Walsman, Oc- tavian Murad, Yubo Wang, Pedro Domingos, and Sid- dhartha Srinivasa. Amodal 3d reconstruction for robotic manipulation via stability and connectivity. In Confer- ence on Robot Learning, pages 1498–1508. PMLR, 2021

    work page 2021

  4. [4]

    Input convex neural networks

    Brandon Amos, Lei Xu, and J Zico Kolter. Input convex neural networks. In International Conference on Machine Learning, pages 146–155. PMLR, 2017

    2017

  5. [5]

    Rethinking optimization with differentiable simulation from a global perspec- tive

    Rika Antonova, Jingyun Yang, Krishna Murthy Jataval- labhula, and Jeannette Bohg. Rethinking optimization with differentiable simulation from a global perspec- tive. In Conference on Robot Learning , pages 276–286. PMLR, 2023

    work page 2023

  6. [6]

    Neural rgb-d surface reconstruction

    Dejan Azinovi ´c, Ricardo Martin-Brualla, Dan B Gold- man, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6290–6301, 2022

    work page 2022

  7. [7]

    Nonlinear programming: theory and algorithms

    Mokhtar S Bazaraa, Hanif D Sherali, and Chitharanjan M Shetty. Nonlinear programming: theory and algorithms . John wiley & sons, 2013

    work page 2013

  8. [8]

    Generalization bounded implicit learning of nearly discontinuous functions

    Bibit Bianchini, Mathew Halm, Nikolai Matni, and Michael Posa. Generalization bounded implicit learning of nearly discontinuous functions. In Learning for Dynamics and Control Conference , pages 1112–1124. PMLR, 2022

    work page 2022

Show all 78 references
  1. [9]

    Si- multaneous learning of contact and continuous dynamics

    Bibit Bianchini, Mathew Halm, and Michael Posa. Si- multaneous learning of contact and continuous dynamics. In Conference on Robot Learning , pages 3966–3978. PMLR, 2023

    2023

  2. [10]

    Codeslam—learning a compact, optimisable representation for dense visual slam

    Michael Bloesch, Jan Czarnowski, Ronald Clark, Stefan Leutenegger, and Andrew J Davison. Codeslam—learning a compact, optimisable representation for dense visual slam. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2560–2568, 2018

    2018

  3. [11]

    Convex optimization

    Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004

    2004

  4. [12]

    Posematcher: One-shot 6d object pose estimation by deep feature matching

    Pedro Castro and Tae-Kyun Kim. Posematcher: One-shot 6d object pose estimation by deep feature matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 2148–2157, 2023

    2023

  5. [13]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015

    2015 arXiv

  6. [14]

    Visual dexterity: In- hand reorientation of novel and complex object shapes

    Tao Chen, Megha Tippur, Siyang Wu, Vikash Kumar, Ed- ward Adelson, and Pulkit Agrawal. Visual dexterity: In- hand reorientation of novel and complex object shapes. Science Robotics, 8(84):eadc9244, 2023

    2023

  7. [15]

    Category level object pose estimation via neural analysis-by-synthesis

    Xu Chen, Zijian Dong, Jie Song, Andreas Geiger, and Otmar Hilliges. Category level object pose estimation via neural analysis-by-synthesis. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16 , pages 139–156. Spri...

    2020

  8. [16]

    Anchorformer: Point cloud completion from discriminative nodes

    Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, and Tao Mei. Anchorformer: Point cloud completion from discriminative nodes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 13581–13590, 2023

    2023

  9. [17]

    Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model. In European Conference on Computer Vision, pages 640–658. Springer, 2022

    2022

  10. [18]

    Sdfusion: Multimodal 3d shape completion, reconstruction, and generation

    Yen-Chi Cheng, Hsin-Ying Lee, Sergey Tulyakov, Alexander G Schwing, and Liang-Yan Gui. Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4456– 4465, 2023

    2023

  11. [19]

    Diffcom- plete: Diffusion-based generative 3d shape completion

    Ruihang Chu, Enze Xie, Shentong Mo, Zhenguo Li, Matthias Nießner, Chi-Wing Fu, and Jiaya Jia. Diffcom- plete: Diffusion-based generative 3d shape completion. Advances in neural information processing systems , 36: 75951–75966, 2023

    2023

  12. [20]

    End-to-end differentiable physics for learning and control

    Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and J Zico Kolter. End-to-end differentiable physics for learning and control. Advances in neural information processing systems, 31:7178–7189, 2018

    2018

  13. [21]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems , 36:35799– 35813, 2023

    2023

  14. [22]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern r...

    2023

  15. [23]

    Gpv-pose: Category-level object pose estimation via geometry-guided point-wise voting

    Yan Di, Ruida Zhang, Zhiqiang Lou, Fabian Manhardt, Xiangyang Ji, Nassir Navab, and Federico Tombari. Gpv-pose: Category-level object pose estimation via geometry-guided point-wise voting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, ...

    2022

  16. [24]

    Learning data-efficient rigid-body contact models: Case study of planar impact

    Nima Fazeli, Samuel Zapolsky, Evan Drumwright, and Alberto Rodriguez. Learning data-efficient rigid-body contact models: Case study of planar impact. In Confer- ence on Robot Learning , pages 388–397. PMLR, 2017

    2017

  17. [25]

    Addressing Stiffness-Induced Challenges in Modeling and Identification for Rigid-Body Systems With Friction and Impacts

    Mathew Halm. Addressing Stiffness-Induced Challenges in Modeling and Identification for Rigid-Body Systems With Friction and Impacts . PhD thesis, University of Pennsylvania, 2023

    2023

  18. [26]

    Openlrm: Open-source large reconstruction models

    Zexin He and Tengfei Wang. Openlrm: Open-source large reconstruction models. https://github.com/3DTopia/ OpenLRM, 2023

    2023

  19. [27]

    Lrm: Large reconstruction model for single image to 3d

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400 , 2023

    2023 arXiv

  20. [28]

    Mobilepose: Real-time pose estimation for unseen objects with weak shape supervision

    Tingbo Hou, Adel Ahmadyan, Liangkai Zhang, Jianing Wei, and Matthias Grundmann. Mobilepose: Real-time pose estimation for unseen objects with weak shape supervision. arXiv preprint arXiv:2003.03522 , 2020

    2003 arXiv

  21. [29]

    Dojo: A differentiable simulator for robotics

    Taylor A Howell, Simon Le Cleac’h, J Zico Kolter, Mac Schwager, and Zachary Manchester. Dojo: A differentiable simulator for robotics. arXiv preprint arXiv:2203.00806, 2022

    2022 arXiv

  22. [30]

    Zero-shot multi-object scene completion

    Shun Iwase, Katherine Liu, Vitor Guizilini, Adrien Gaidon, Kris Kitani, Rares ¸ Ambrus ¸, and Sergey Za- kharov. Zero-shot multi-object scene completion. In European Conference on Computer Vision , pages 96–

  23. [31]

    Few-view object reconstruction with unknown categories and camera poses

    Hanwen Jiang, Zhenyu Jiang, Kristen Grauman, and Yuke Zhu. Few-view object reconstruction with unknown categories and camera poses. International Conference on 3D Vision (3DV) , 2024

    2024

  24. [32]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139– 1, 2023

    2023

  25. [33]

    Cosypose: Consistent multi-view multi-object 6d pose estimation

    Yann Labb ´e, Justin Carpentier, Mathieu Aubry, and Josef Sivic. Cosypose: Consistent multi-view multi-object 6d pose estimation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part XVII 16 , pages 574–591. Springer, 2020

    2020

  26. [34]

    Megapose: 6d pose estimation of novel objects via render & compare

    Yann Labb ´e, Lucas Manuelli, Arsalan Mousavian, Stephen Tyree, Stan Birchfield, Jonathan Tremblay, Justin Carpentier, Mathieu Aubry, Dieter Fox, and Josef Sivic. Megapose: 6d pose estimation of novel objects via render & compare. arXiv preprint arXiv:2212.06870, 2022

    2022 arXiv

  27. [35]

    Single-level differentiable contact simulation

    Simon Le Cleac’h, Mac Schwager, Zachary Manchester, Vikas Sindhwani, Pete Florence, and Sumeet Singh. Single-level differentiable contact simulation. IEEE Robotics and Automation Letters , 2023

    2023

  28. [36]

    Keypoint-based category-level object pose tracking from an rgb sequence with uncertainty estimation

    Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patri- cio A Vela, and Stan Birchfield. Keypoint-based category-level object pose tracking from an rgb sequence with uncertainty estimation. In 2022 International Conference on Robotics and Automation (ICRA) , pages 1258–1264. IEEE, 2022

    2022

  29. [37]

    Nu-mcc: Multiview compressive coding with neighbor- hood decoder and repulsive udf

    Stefan Lionar, Xiangyu Xu, Min Lin, and Gim Hee Lee. Nu-mcc: Multiview compressive coding with neighbor- hood decoder and repulsive udf. Advances in Neural Information Processing Systems, 36:63011–63022, 2023

    2023

  30. [38]

    One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. In Proceedings of the IEEE/CVF Conference on Comp...

    2024

  31. [39]

    Zero-1-to- 3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to- 3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9298–9309, 2023

    2023

  32. [40]

    Gen6d: Gener- alizable model-free 6-dof object pose estimation from rgb images

    Yuan Liu, Yilin Wen, Sida Peng, Cheng Lin, Xiaoxiao Long, Taku Komura, and Wenping Wang. Gen6d: Gener- alizable model-free 6-dof object pose estimation from rgb images. In European Conference on Computer Vision , pages 298–315. Springer, 2022

    2022

  33. [41]

    Fusion++: V ol- umetric object-level slam

    John McCormac, Ronald Clark, Michael Bloesch, An- drew Davison, and Stefan Leutenegger. Fusion++: V ol- umetric object-level slam. In 2018 international confer- ence on 3D vision (3DV) , pages 32–41. IEEE, 2018

    2018

  34. [42]

    Realfusion: 360deg reconstruction of any object from a single image

    Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, and Andrea Vedaldi. Realfusion: 360deg reconstruction of any object from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8446–8455, 2023

    2023

  35. [43]

    Occupancy networks: Learning 3d reconstruction in function space

    Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4460–4470, 2019

    2019

  36. [44]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM , 65(1):99– 106, 2021

    2021

  37. [45]

    Autosdf: Shape priors for 3d comple- tion, reconstruction and generation

    Paritosh Mittal, Yen-Chi Cheng, Maneesh Singh, and Shubham Tulsiani. Autosdf: Shape priors for 3d comple- tion, reconstruction and generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 306–315, 2022

    2022

  38. [46]

    3dsgrasp: 3d shape-completion for robotic grasp

    Seyed S Mohammadi, Nuno F Duarte, Dimitrios Dimou, Yiming Wang, Matteo Taiana, Pietro Morerio, Atabak Dehban, Plinio Moreno, Alexandre Bernardino, Alessio Del Bue, et al. 3dsgrasp: 3d shape-completion for robotic grasp. In 2023 IEEE International Conference on Robotics and Aut...

    2023

  39. [47]

    Atlas: End-to-end 3d scene reconstruction from posed images

    Zak Murez, Tarrence Van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End-to-end 3d scene reconstruction from posed images. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part VII ...

    2020

  40. [48]

    NOPE: Novel Object Pose Es- timation from a Single Image

    Van Nguyen Nguyen, Thibault Groueix, Georgy Poni- matkin, Yinlin Hu, Renaud Marlet, Mathieu Salzmann, and Vincent Lepetit. NOPE: Novel Object Pose Es- timation from a Single Image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  41. [49]

    Phyrecon: Physically plausible neural scene reconstruction

    Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang, Bin Wang, Bo Dai, Puhao Li, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang. Phyrecon: Physically plausible neural scene reconstruction. 2024

    2024

  42. [50]

    Deepsdf: Learning continuous signed distance functions for shape represen- tation

    Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape represen- tation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019

    2019

  43. [51]

    Pix2pose: Pixel-wise coordinate regression of objects for 6d pose estimation

    Kiru Park, Timothy Patten, and Markus Vincze. Pix2pose: Pixel-wise coordinate regression of objects for 6d pose estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7668–7677, 2019

    2019

  44. [52]

    Fun- damental challenges in deep learning for stiff contact dynamics

    Mihir Parmar, Mathew Halm, and Michael Posa. Fun- damental challenges in deep learning for stiff contact dynamics. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 5181–

    2021

  45. [53]

    ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations

    Samuel Pfrommer, Mathew Halm, and Michael Posa. ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. In The Con- ference on Robot Learning (CoRL) , 2020. URL https: //proceedings.mlr.press/v155/pfrommer21a.html

    2020

  46. [54]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  47. [55]

    Pixels, voxels, and views: A study of shape represen- tations for single view 3d object shape prediction

    Daeyun Shin, Charless C Fowlkes, and Derek Hoiem. Pixels, voxels, and views: A study of shape represen- tations for single view 3d object shape prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3061–3069, 2018

    2018

  48. [56]

    Inferring 3d shapes of unknown rigid objects in clutter through in- verse physics reasoning

    Changkyu Song and Abdeslam Boularias. Inferring 3d shapes of unknown rigid objects in clutter through in- verse physics reasoning. IEEE Robotics and Automation Letters, 4(2):201–208, 2018

    2018

  49. [57]

    Iterative corresponding geometry: Fusing region and depth for highly efficient 3d tracking of textureless objects

    Manuel Stoiber, Martin Sundermeyer, and Rudolph Triebel. Iterative corresponding geometry: Fusing region and depth for highly efficient 3d tracking of textureless objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6855– 6865, 2022

    2022

  50. [58]

    Nodeslam: Neural object descriptors for multi-view shape reconstruction

    Edgar Sucar, Kentaro Wada, and Andrew Davison. Nodeslam: Neural object descriptors for multi-view shape reconstruction. In 2020 International Conference on 3D Vision (3DV) , pages 949–958. IEEE, 2020

    2020

  51. [59]

    Onepose: One-shot object pose estimation without cad models

    Jiaming Sun, Zihao Wang, Siyu Zhang, Xingyi He, Hongcheng Zhao, Guofeng Zhang, and Xiaowei Zhou. Onepose: One-shot object pose estimation without cad models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6825– 6834, 2022

    2022

  52. [60]

    Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation

    Sudharshan Suresh, Haozhi Qi, Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mri- nal Kalakrishnan, Roberto Calandra, Michael Kaess, et al. Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation. arXiv preprint arXiv:2312.13469, 2023

    2023 arXiv

  53. [61]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras

    Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems , 34:16558– 16569, 2021

    2021

  54. [62]

    3d reconstruction of novel object shapes from single images

    Anh Thai, Stefan Stojanov, Vijay Upadhya, and James M Rehg. 3d reconstruction of novel object shapes from single images. In 2021 International Conference on 3D Vision (3DV), pages 85–95. IEEE, 2021

    2021

  55. [63]

    Learning continuous 3d reconstructions for geometrically aware grasping

    Mark Van der Merwe, Qingkai Lu, Balakumar Sundar- alingam, Martin Matak, and Tucker Hermans. Learning continuous 3d reconstructions for geometrically aware grasping. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 11516–11522. IEEE, 2020

    2020

  56. [64]

    6- pack: Category-level 6d pose tracker with anchor-based keypoints

    Chen Wang, Roberto Mart ´ın-Mart´ın, Danfei Xu, Jun Lv, Cewu Lu, Li Fei-Fei, Silvio Savarese, and Yuke Zhu. 6- pack: Category-level 6d pose tracker with anchor-based keypoints. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 10059–10066. IEEE, 2020

    2020

  57. [65]

    Bundletrack: 6d pose tracking for novel objects without instance or category- level 3d models, 2021

    Bowen Wen and Kostas Bekris. Bundletrack: 6d pose tracking for novel objects without instance or category- level 3d models, 2021

    2021

  58. [66]

    Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects

    Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas M ¨uller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. Bundlesdf: Neural 6-dof tracking and 3d reconstruction of unknown objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2023

  59. [67]

    Foundationpose: Unified 6d pose estimation and tracking of novel objects

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. arXiv preprint arXiv:2312.08344, 2023

    2023 arXiv

  60. [68]

    V-prism: Probabilistic mapping of unknown tabletop scenes

    Herbert Wright, Weiming Zhi, Matthew Johnson- Roberson, and Tucker Hermans. V-prism: Probabilistic mapping of unknown tabletop scenes. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), pages 1078–1085. IEEE, 2024

    2024

  61. [69]

    Multiview compressive coding for 3d reconstruction

    Chao-Yuan Wu, Justin Johnson, Jitendra Malik, Christoph Feichtenhofer, and Georgia Gkioxari. Multiview compressive coding for 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9065–9075, 2023

    2023

  62. [70]

    Ipod: Implicit field learning with point diffusion for generalizable 3d object reconstruc- tion from single rgb-d images

    Yushuang Wu, Luyue Shi, Junhao Cai, Weihao Yuan, Lingteng Qiu, Zilong Dong, Liefeng Bo, Shuguang Cui, and Xiaoguang Han. Ipod: Implicit field learning with point diffusion for generalizable 3d object reconstruc- tion from single rgb-d images. In Proceedings of the IEEE/CVF Con...

    2024

  63. [71]

    Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation

    Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wet- zstein. Grm: Large gaussian reconstruction model for efficient 3d reconstruction and generation. In European Conference on Computer Vision , pages 1–20. Springer, 2024

    2024

  64. [72]

    Shape- former: Transformer-based shape completion via sparse representation

    Xingguang Yan, Liqiang Lin, Niloy J Mitra, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. Shape- former: Transformer-based shape completion via sparse representation. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 6239–6249, 2022

    2022

  65. [73]

    Pointr: Diverse point cloud completion with geometry-aware transformers

    Xumin Yu, Yongming Rao, Ziyi Wang, Zuyan Liu, Jiwen Lu, and Jie Zhou. Pointr: Diverse point cloud completion with geometry-aware transformers. In Proceedings of the IEEE/CVF international conference on computer vision , pages 12498–12507, 2021

    2021

  66. [74]

    Relpose: Predicting probabilistic relative rotation for single objects in the wild

    Jason Y Zhang, Deva Ramanan, and Shubham Tulsiani. Relpose: Predicting probabilistic relative rotation for single objects in the wild. In European Conference on Computer Vision, pages 592–611. Springer, 2022

    2022

  67. [75]

    Simultaneous geometry and pose estimation of held objects via 3d foundation models

    Weiming Zhi, Haozhan Tang, Tianyi Zhang, and Matthew Johnson-Roberson. Simultaneous geometry and pose estimation of held objects via 3d foundation models. IEEE Robotics and Automation Letters , 2024

    2024

  68. [76]

    Triplane meets gaussian splatting: Fast and generalizable single- view 3d reconstruction with transformers

    Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single- view 3d reconstruction with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  69. [77]

    Hyperparameters for converting from PLL outputs to modified BundleSDF inputs: §IV-B describes querying the PLL geometry for supervising BundleSDF via (8) and (11). Since odometry-based contact learning may lack signal in many directions, data for supervising the next round of ...

  70. [78]

    We query 5,768 approximately evenly-spaced ˆ nv directions and obtain their corresponding support points sv on the BundleSDF geometry

    Hyperparameters for converting from BundleSDF out- puts to modified PLL inputs: §IV-A describes querying the BundleSDF geometry to obtain DSF input/output pairs {(ˆ nv, sv)i} for supervising PLL via (6). We query 5,768 approximately evenly-spaced ˆ nv directions and obtain the...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.