Pith. sign in

REVIEW 3 major objections 6 minor 53 references

FusionForce: End-to-end Differentiable Neural-Symbolic Layer for Trajectory Prediction

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A learnable rigid-body physics layer predicts off-road robot trajectories more accurately than LSTM baselines.

desk verdict Solid incremental contribution with real accuracy gains over LSTM baselines, but the central out-of-distribution generalization claim is asserted rather than quantitatively evaluated. read the letter →

arxiv 2502.10156 v4 pith:JVTTYY7J submitted 2025-02-14 cs.RO cs.CV

Ruslan Agishev , Karel Zimmermann This is my paper · ORCID
classification cs.ROcs.CV
keywords trajectorypredictiondifferentiablephysicsengineneural-symbolicmodeloff-roadnavigationrobot-terraininteractionsensorfusionout-of-distributiongeneralizationself-supervisedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FusionForce is an end-to-end differentiable model that predicts a tracked robot's trajectory on rough off-road terrain from a camera image, a lidar point cloud, or both. The model learns terrain properties—supporting ground height under vegetation, friction, stiffness, and damping—and feeds them into a differentiable rigid-body physics engine that computes the trajectory by integrating contact forces. The paper's central claim is that this neural-symbolic layer, called ∇Physics, makes trajectory prediction substantially more accurate than a purely data-driven LSTM baseline while generalizing better to out-of-distribution obstacles such as steep cliffs. On the reported test data the RGB version reaches 0.043 m translational error, versus 0.125–0.129 m for the LSTM baselines with the same terrain encoders. The authors argue that this grey-box approach reduces the sim-to-real gap and makes the model usable for control, trajectory shooting, and SLAM.

What carries the argument

The key machinery is the ∇Physics layer: a from-scratch differentiable rigid-body simulator that accepts predicted terrain property maps and control commands, computes robot-terrain contact forces at 223 body points sampled from the robot's URDF, and integrates the six-degree-of-freedom equations of motion with an Euler or Neural-ODE solver. The contact model is a smoothed spring-damper: normal force is stiffness times penetration plus damping, active only where robot points penetrate the predicted supporting heightmap, with a sigmoid smoothness for gradients; friction is a simplified Pacejka-style law proportional to normal force and track slip. Because the whole forward simulation retains its computational graph, gradients can flow from trajectory error back through the physics to the convolutional terrain encoder, which is what turns the simulator into a learnable image- and lidar-conditioned trajectory predictor.

What would settle it

Measure ground-truth terrain reaction forces on the robot while traversing tall grass, mud, and undergrowth—using an instrumented track or force sensing on the flippers—and compare them with the model's predicted normal and friction forces. If the measured forces are not approximately the spring-damper penetration law against the predicted supporting heightmap, the physics layer is imposing the wrong inductive bias in exactly the out-of-distribution regimes the paper targets.

Watch

Extended reading notes

Core claim

At the core of the paper is the claim that replacing a recurrent trajectory predictor with a differentiable physics simulation—one that queries learned normal reaction and friction forces at up to 223 points of a rigid-body robot model—improves both accuracy and out-of-distribution generalization. The terrain encoder predicts the supporting heightmap (the hidden rigid surface under vegetation), a soft-terrain deformation map, friction, stiffness, and damping; the physics engine then solves the six-degree-of-freedom rigid-body ODE with an elastic-foundation contact law, where normal force is a spring-damper proportional to penetration, plus a simplified tangential friction law. The entire graph is differentiable, so the trajectory loss alone can supervise the terrain properties, with two auxiliary losses on predicted heightmaps regularizing the learning. The paper demonstrates the improvement across camera, lidar, and fused inputs, and reports that the physics layer outperforms TrajLSTM in both translation and rotation, with the gap largest on tall-grass and cliff-like out-of-distribution cases.

Load-bearing premise

The load-bearing premise is that all significant robot-terrain interaction on grass, mud, and undergrowth reduces to spring-damper contact forces acting only where the robot penetrates a hidden rigid supporting surface, plus the simplified friction law, an assumption the paper never validates against measured forces or friction.

Editorial extensions

If this is right

  • With the physics layer, trajectory prediction error drops from about 0.125–0.129 m for the LSTM baselines to 0.042–0.045 m across RGB, lidar, and fused inputs.
  • The model produces interpretable terrain properties—supporting terrain height, friction, stiffness, and damping—as by-products, which can serve as self-supervision targets or as inputs to planners.
  • The differentiable simulation runs about 2048 six-second trajectories in roughly one second on a V100 GPU, making real-time trajectory shooting and sampling-based navigation feasible.
  • Because gradients reach the camera and robot parameters, the same architecture can be used for online recalibration, model re-identification, and SLAM-style losses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the contact model is the right inductive bias, the same ∇Physics layer should transfer to other tracked or wheeled robots by re-identifying mass, inertia, and body-point geometry, without retraining the terrain encoder.
  • Editorial inference: The friction and stiffness maps could be validated against instrumented force measurements or a high-fidelity terrain simulator; such a comparison would separate errors of perception from errors of the contact law.
  • Editorial inference: The differentiable simulator could serve as a world model for reinforcement learning and model-predictive control, where the cost of a trajectory is computed from predicted reaction forces as in the paper's navigation experiments.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FusionForce, an end-to-end differentiable model for off-road robot trajectory prediction from monocular camera and/or lidar. A terrain encoder predicts a supporting heightmap, friction, stiffness, and damping maps; a differentiable physics engine integrates rigid-body dynamics with elastic-foundation contact forces (Eqs. 6-8) to roll out trajectories from these predicted properties. Training combines a trajectory loss with geometric and rigid-terrain regularization losses. Experiments on a self-collected off-road dataset compare the physics-based predictor against LSTM baselines across three terrain encoders and report substantially lower translational and rotational errors for FusionForce (Table I), plus qualitative demonstrations of terrain estimation and autonomous navigation.

Significance. The approach is potentially significant: it provides an interpretable, physics-constrained trajectory predictor that is end-to-end trainable and runs fast enough for onboard navigation. The paper's strengths include the consistent large accuracy improvements over LSTM baselines across input modalities and encoders, the publicly released code and data, and the clear ablation of sensor-fusion variants. However, the central out-of-distribution generalization claim is not quantitatively evaluated, and the accuracy claims lack statistical support. These issues are addressable but currently limit the strength of the conclusions.

major comments (3)
  1. [Section IV-C, Table I, Fig. 12] The claim that ∇Physics 'significantly outperforms' TrajLSTM is not supported by any statistical evidence: no error bars, confidence intervals, number of evaluation runs, or significance tests are reported anywhere in Section IV. The point estimates are large, but without variance information or a paired test we cannot judge whether the differences are reliable. Please report the number of test trajectories, per-trajectory error distributions, and appropriate significance tests (or justify why they are unnecessary).
  2. [Section IV-B and Fig. 3 caption] The paper's central claim of improved out-of-distribution generalization is asserted but never directly measured. No split of the evaluation data into in-distribution and out-of-distribution terrain types is described, and no quantitative OOD trajectory-prediction results are presented; the statement in §IV-B that terrain-accuracy improvement 'suggests a better generalization on dangerous (out-of-distribution) terrains' is an inference, not an evaluation. Please define an explicit OOD protocol (e.g., held-out terrain classes such as steep cliffs or tall vegetation) and report trajectory errors on it, or soften the OOD claims throughout the abstract, introduction, and Fig. 3 caption.
  3. [Section III-B, Eqs. (6)-(8)] The contact model assumes robot-terrain interaction is fully captured by a spring-damper normal force against a predicted supporting heightmap plus a simplified friction law. For the non-rigid terrains targeted (mud, tall grass, undergrowth), this is a strong prior that is not validated against any ground-truth force or friction measurements. This is a risk to the OOD-generalization argument: if the physics layer is credited for OOD robustness, the appropriateness of this contact model for such terrains should be tested or at least discussed with concrete failure modes.
minor comments (6)
  1. [Fig. 2 caption and Section IV-D] Fig. 2 caption states the model 'delivers a million simulated trajectories per second,' while Section IV-D reports about 2048 trajectories per second on a V100 GPU; please reconcile these throughput numbers.
  2. [Section I, Table I, Conclusion] The claimed improvements are inconsistent: Section I says 'up to 56% in translation and 48% in orientation,' while Table I implies roughly 66% and 50% reductions (RGB), and the Conclusion says '2-times smaller error'; please use consistent numbers.
  3. [Table I] Table I shows that FusionForce with RGB+PCL achieves 0.045 m translational error, slightly worse than RGB-only (0.043 m) and PCL-only (0.042 m); this non-monotonicity is not discussed and may reflect noise given the absence of error bars.
  4. [Section III-B] Section III-B states that the Neural ODE solver estimates gradients 'through the implicit function theorem'; for an explicit Euler integrator this is unclear, and the gradient computation should be described precisely (e.g., adjoint vs. full backpropagation).
  5. [Section IV-A] Section IV-A provides no details on the number of evaluation trajectories, the train/test split, or the composition of terrain types in the ROUGH dataset; please add these to enable reproducibility.
  6. [Various] There are several typos: 'stat' at the end of Section III-D, 'frameworkZ' in the AnyNav sentence of Section II, 'reslution' in the Fig. 13 caption, and inconsistent spelling of 'VoxelNet' and 'RELIS-3D'/'RELLIS-3D'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the physics layer is a fixed differentiable simulator, and the learned terrain properties are optimized against trajectory, geometric, and terrain losses, so no claimed prediction reduces to its own training inputs.

full rationale

The paper's central derivation chain is self-contained and does not reduce to its inputs by construction. The differentiable physics layer is a fixed rigid-body simulator given by Eq. (5), with robot-terrain interaction forces specified by the spring-damper contact law of Eq. (7) and the simplified friction law of Eq. (8); these equations are hand-specified priors, not outputs of the learned model. The learnable quantities (supporting heightmap Ht, stiffness K, damping D, friction M) are optimized through the explicit losses (10)-(12), and the trajectory prediction is the result of integrating the resulting forces through the ODE solver. The reported trajectory errors are measured against held-out SLAM-reconstructed trajectories, so the evaluation is a standard supervised-learning comparison rather than a fitted parameter being renamed as a prediction. The self-citation to the authors' prior work MonoForce [19] appears only as a previous baseline and as a source of the approximate robot model; it does not carry the claimed generalization improvement, which is instead supported by the comparison against the independently designed TrajLSTM baseline with shared terrain encoders. The OOD-generalization claim is indeed under-evidenced because no explicit out-of-distribution test split is defined, and the statement that improved terrain accuracy 'suggests a better generalization on dangerous (out-of-distribution) terrains' is an inference rather than a direct measurement. However, missing evidence is a correctness or evaluation gap, not circularity: nothing in the derivation equates a prediction to a training target by definition, and no load-bearing argument rests on an author-imposed uniqueness theorem or ansatz smuggled in via citation. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on one learned simulator and several hand-chosen contact parameters. The physics engine is a fixed prior, not a learned entity, but its parameters (stiffness, damping, sigmoid steepness, heightmap bounds) are either learned or chosen by hand and are not reported numerically. The main structural axioms are the elastic-contact and Coulomb-friction assumptions, which are not independently measured. The model does not introduce new physical objects; all entities are standard terrain and robot state variables.

free parameters (3)
  • Sigmoid steepness k in contact force smoothing = not reported
    Eq. (7) uses a sigmoid with steepness k to smooth the Heaviside contact activation. This is a hand-chosen hyperparameter that affects gradient flow and force magnitude near the contact boundary, and its value is not given.
  • Terrain stiffness e (K) and damping d (D) = learned, but constant in implementation
    The paper says stiffness and damping maps are 'considered constant in the current implementation'. These constants determine the normal reaction force in Eq. (6) and are either fixed by hand or learned as scalars; no values are reported, and the trajectory error depends on them.
  • Heightmap clamp upper/lower bounds (±1 m) = ±1 m
    The predicted heightmap is clamped to [-1, 1] m based on robot size and hanging branches. This is a hand-set constraint that removes information about tall obstacles and affects where contact forces are applied.
assumptions (6)
  • standard math The robot is a rigid body with known mass and inertia, and the 6DOF equations of motion in Eq. (5) hold exactly.
    Section III-B uses standard rigid-body dynamics and assumes mass and inertia are known static parameters identifiable in the lab.
  • domain assumption Normal terrain reaction is proportional to penetration depth (Hooke spring) plus linear damping, activated by a smooth sigmoid.
    Eqs. (6-7) define the contact law. This is a simplified model of flexible terrain that is not validated against real force measurements.
  • domain assumption Tangential friction follows the simplified Pacejka model in Eq. (8), with friction coefficient μ predicted per cell.
    Eq. (8) states friction force is proportional to normal force and track slip. This is a strong assumption for mud, grass, and vegetation.
  • domain assumption The supporting terrain heightmap Ht, the rigid surface hidden under vegetation, is learnable from camera and lidar appearance and is the only surface that generates contact forces.
    Section III-A defines Ht as a predicted output, and the physics engine only queries contact at Ht. Vegetation itself is assumed not to exert forces directly.
  • domain assumption Lidar-reconstructed heightmaps and SLAM trajectories are accurate enough to serve as ground truth for the geometric and trajectory losses.
    Eqs. (10-11) use SLAM poses and lidar heightmaps as supervision; the paper cites ICP SLAM but does not quantify its noise.
  • domain assumption SEEM semantic segmentation correctly identifies rigid terrain cells, and the terrain loss in Eq. (12) uses these masks as reliable labels.
    Section III-D uses the foundation model SEEM to mask rigid classes; errors in SEEM propagate to Ht supervision.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FusionForce: End-to-end Differentiable Neural-Symbolic Layer for Trajectory Prediction." pith.science (2026). https://pith.science/paper/JVTTYY7J

@misc{pith2026250210156,
  author       = {Pith},
  title        = {Pith review of: FusionForce: End-to-end Differentiable Neural-Symbolic Layer for Trajectory Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JVTTYY7J}},
  note         = {Machine review of arXiv:2502.10156}
}
abstract

We propose end-to-end differentiable model that predicts robot trajectories on rough offroad terrain from camera images and/or lidar point clouds. The model integrates a learnable component that predicts robot-terrain interaction forces with a neural-symbolic layer that enforces the laws of classical mechanics and consequently improves generalization on out-of-distribution data. The neural-symbolic layer includes a differentiable physics engine that computes the robot's trajectory by querying these forces at the points of contact with the terrain. As the proposed architecture comprises substantial geometrical and physics priors, the resulting model can also be seen as a learnable physics engine conditioned on real sensor data that delivers $10^4$ trajectories per second. We argue and empirically demonstrate that this architecture reduces the sim-to-real gap and mitigates out-of-distribution sensitivity. The differentiability, in conjunction with the rapid simulation speed, makes the model well-suited for various applications including model predictive control, trajectory shooting, supervised and reinforcement learning, or SLAM.

Figures

Figures reproduced from arXiv: 2502.10156 by the authors.

Figure 1
Figure 1. Camera, predicted terrain properties and trajectories. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Model overview: The proposed model can be seen as an image-conditioned differentiable simulation that delivers a million simulated trajectories per second on the terrain depicted in the onboard camera image. The explainable structure also delivers many intermediate interpretable outputs that can serve for efficient self-supervision [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Qualitative and quantitative results: Comparison of the trajectory error with the state-of-the-art architecture based on the LSTM network. Proposed model achieves, due to the ∇Physics layer, significantly better generalization on out-of-distribution data such as steep terrain cliffs. simulation faces challenges or discrepancies when applied in the real world. Although several techniques, such as rapid motor adaptati… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Influence of lidar data Input images (first row). The predicted supporting terrain from lidar + camera (second and third row) - see 3D pointcloud to compare accuracy. The predicted supporting terrain from camera only (last row). enabling a quadruped to run on soft sand…
Figure 5
Figure 5. Figure 5: Detailed architecture overview: Our model consists of a data-driven FusionForce Predictor and physics-driven differentiable Neuro-Symbolic Physics Engine. The FusionForce Predictor estimates rich features in image and voxel domains. Image features are then lifted to vo…
Figure 6
Figure 6. Figure 6: Terrain force model: Simplified 2D sketch demonstrating normal reaction forces acting on a robot body consisting of two points pi and pj . prevents the penetration of the terrain by the robot points, (ii) tangential friction force that generates forward acceleration wh…
Figure 7
Figure 7. Figure 7: Terrain computed by backpropagating through ∇Physics: Shape of the terrain (border of the area where terrain forces start to act) outlined by heightmap surface, its color represents the friction of the terrain. The optimized trajectory is in green, and the ground truth…
Figure 8
Figure 8. Figure 8: TrajLSTM architecture. The model takes as input: initial state x0, terrain H, control sequence ut, t ∈ {0 . . . T}. It predicts the trajectory as a sequence of states xt, t ∈ {0 . . . T}. C. Data-driven Trajectory Prediction Inspired by the work [36], we design a data-…
Figure 9
Figure 9. Figure 9: Two robot platforms used to collect the ROUGH dataset. Notice that one platform has only four massive flippers, and the other has also two main tracks. choice (of not processing the heightmaps at different time moments) is also motivated by computational efficiency rea…
Figure 10
Figure 10. Figure 10: Qualitative examples. Left: The robot is moving through a narrow passage between a wall and tree logs. Right: The robot is moving on a gravel road with rocks on the sides. It starts its motion from the position marked with a coordinate frame and the trajectory is pred…
Figure 12
Figure 12. Figure 12: Generalization and results: The proposed neuro-symbolic layer ∇Physics generalizes to out-of-distribution examples well. The overall results are consistently better independent of the architecture (LSS [27], Voxel￾Net [51], BEV-Fusion [52]) used for terrain properties…
Figure 11
Figure 11. Figure 11: Qualitative results: Given input onboard image from the robot’s camera (first row), the method distinguishes stones and trees as obstacles, while vegetation is suppressed (see 3D view in third row) the stone is preserved despite having almost the same green color as v…
Figure 13
Figure 13. Figure 13: The results are provided with the following configurations: grid [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: The top-down view of the navigation experiment in the forest environment. During the experiment, the robot autonomously traverses the 260-meter [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 15
Figure 15. Figure 15: Autonomous navigation in the forest environment. The robot follows a set of waypoints and at the same time avoids obstacles (trees, bushes, rocks, [PITH_FULL_IMAGE:figures/full_fig_p012_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 48 canonical work pages

  1. [1]

    Pose prediction for mobile ground robots in uneven terrain based on difference of heightmaps,

    S. Fabian, S. Kohlbrecher, and O. V on Stryk, “Pose prediction for mobile ground robots in uneven terrain based on difference of heightmaps,” in 2020 IEEE Int. Symp. on Safety, Security, and Rescue Robotics (SSRR) , 2020, pp. 49–56. 1

    work page 2020

  2. [2]

    An improved kinematic model for skid- steered wheeled platforms,

    S. Dogru and L. Marques, “An improved kinematic model for skid- steered wheeled platforms,” Autonomous Robots, vol. 45, no. 2, pp. 229– 243, 2021. 1

    work page 2021

  3. [3]

    Bi-level trajectory optimization on uneven terrains with differentiable wheel-terrain interaction model,

    A. Manoharan, A. Sharma, H. Belsare, K. Pal, K. M. Krishna, and A. K. Singh, “Bi-level trajectory optimization on uneven terrains with differentiable wheel-terrain interaction model,” in 2024 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS) under review, 2024, pp. 1–8. 1

    work page 2024

  4. [4]

    Learning visual locomotion with cross-modal supervision,

    A. Loquercio, A. Kumar, and J. Malik, “Learning visual locomotion with cross-modal supervision,” in IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 7295–7302. 1

    work page 2023

  5. [5]

    An embarrassingly simple approach for visual navigation of forest environments,

    C. Niu, C. Newlands, K.-P. Zauner, and D. Tarapore, “An embarrassingly simple approach for visual navigation of forest environments,” Frontiers in Robotics and AI , vol. 10, 2023. 1

    work page 2023

  6. [6]

    Where should I walk? Predicting terrain properties from images via self-supervised learning,

    L. Wellhausen, A. Dosovitskiy, R. Ranftl, K. Walas, C. Cadena, and M. Hutter, “Where should I walk? Predicting terrain properties from images via self-supervised learning,” IEEE Robotics and Automation Letters, vol. 4, no. 2, 2019. 1

    work page 2019

  7. [7]

    How does it feel? self-supervised costmap learning for off-road vehicle traversability,

    M. Guaman Castro, S. Triest, W. Wang, J. M. Gregory, F. Sanchez, J. G. Rogers III, and S. Scherer, “How does it feel? self-supervised costmap learning for off-road vehicle traversability,” in ICRA, 2023. 1

    work page 2023

  8. [8]

    BADGR: an autonomous self-supervised learning-based navigation system,

    G. Kahn, P. Abbeel, and S. Levine, “BADGR: an autonomous self-supervised learning-based navigation system,” CoRR, vol. abs/2002.05700, 2020. [Online]. Available: https://arxiv.org/abs/2002. 05700 1

    arXiv 2002

Show all 53 references
  1. [9]

    Rma: Rapid motor adaptation for legged robots,

    A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” in Robotics: Science and Systems , 2021. 2

    2021

  2. [10]

    Differentiable mpc for end-to-end planning and control,

    B. Amos, I. Jimenez, J. Sacks, B. Boots, and J. Z. Kolter, “Differentiable mpc for end-to-end planning and control,” in Advances in Neural Information Processing Systems , vol. 31, 2018. 2

    2018

  3. [11]

    End-to-end interpretable neural motion planner,

    W. Zeng, W. Luo, S. Suo, A. Sadat, B. Yang, S. Casas, and R. Urtasun, “End-to-end interpretable neural motion planner,” in The IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) , 2019, pp. 8660–8669. 2

    2019

  4. [12]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347,

    arXiv

  5. [13]

    Dellaert and M

    F. Dellaert and M. Kaess, Factor Graphs for Robot Perception. Hanover, MA, USA: Now Publishers Inc., 2017. 2

    2017

  6. [14]

    Robust maximum-likelihood on-line lidar- to-camera calibration monitoring and refinement,

    J. Moravec and R. Sara, “Robust maximum-likelihood on-line lidar- to-camera calibration monitoring and refinement,” in Computer Vision Winter Workshop, 02 2018. 2

    2018

  7. [15]

    Understanding physics- informed neural networks: Techniques, applications, trends, and chal- lenges,

    A. Farea, O. Yli-Harja, and F. Emmert-Streib, “Understanding physics- informed neural networks: Techniques, applications, trends, and chal- lenges,” AI, vol. 5, no. 3, pp. 1534–1557, 2024. 2

    2024

  8. [16]

    Warp: A high-performance python framework for gpu simulation and graphics,

    M. Macklin, “Warp: A high-performance python framework for gpu simulation and graphics,” https://github.com/nvidia/warp, March 2022, nVIDIA GPU Technology Conference (GTC). 2

    2022

  9. [17]

    Brax - a differentiable physics engine for large scale rigid body simulation,

    C. D. Freeman, E. Frey, A. Raichuk, S. Girgin, I. Mordatch, and O. Bachem, “Brax - a differentiable physics engine for large scale rigid body simulation,” 2021. [Online]. Available: http://github.com/google/brax 2

    2021

  10. [18]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll´ar, and R. Girshick, “Segment anything,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , 2023, pp. 3992–4003. 2

    2023

  11. [19]

    Monoforce: Self-supervised learning of physics-informed model for predicting robot-terrain interaction,

    R. Agishev, K. Zimmermann, V . Kubelka, M. Pecka, and T. Svoboda, “Monoforce: Self-supervised learning of physics-informed model for predicting robot-terrain interaction,” in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2024, pp. 12 8...

    2024

  12. [20]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” Science Robotics , vol. 5, no. 47, p. eaaz7432, 2020. 3

    2020

  13. [21]

    Learning quadrupedal locomotion on deformable terrain,

    Y . Choi, J. Kim, and J. Hwangbo, “Learning quadrupedal locomotion on deformable terrain,” IEEE Transactions on Robotics , 2023. 3, 4

    2023

  14. [22]

    Anymal parkour: Learning agile navigation skills with vision and motion planning,

    D. Hoeller, N. Chentanez, V . Koltun, and M. Hutter, “Anymal parkour: Learning agile navigation skills with vision and motion planning,” Science Robotics, vol. 8, no. 73, p. eade2256, 2023. 3

    2023

  15. [23]

    Evora: A terrain- aware model-predictive planning and control framework for off-road autonomous driving,

    Y . Cai, L. Tai, Q. Ha, M. Liu, and L. Zhang, “Evora: A terrain- aware model-predictive planning and control framework for off-road autonomous driving,” IEEE Transactions on Robotics , vol. 38, no. 4, pp. 2392–2412, 2022. 3

    2022

  16. [24]

    Self-supervised traversability prediction via video domain adaptation for mobile robot navigation,

    D. Vecchio, D. Caporale, S. Caccamo, G. Antonelli, and E. Simetti, “Self-supervised traversability prediction via video domain adaptation for mobile robot navigation,” The International Journal of Robotics Research, vol. 42, no. 9, pp. 893–914, 2023. 3

    2023

  17. [25]

    Anynav: Learning to navigate with friction-aware path planning,

    K. Fu, J. Zhang, S. Karaman, and D. Rus, “Anynav: Learning to navigate with friction-aware path planning,” in Proceedings of the Robotics: Science and Systems (RSS) , 2025, to appear. 4

    2025

  18. [26]

    Monolayout: Amodal scene layout from a single image,

    K. Mani, S. Daga, S. Garg, S. S. Narasimhan, M. Krishna, and K. M. Jatavallabhula, “Monolayout: Amodal scene layout from a single image,” in The IEEE Winter Conference on Applications of Computer Vision , 2020, pp. 1689–1697. 4

    2020

  19. [27]

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,

    J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,” in Proceedings of the European Conference on Computer Vision , 2020. 4, 8, 9, 10

    2020

  20. [28]

    Footprints and free space from a single color image,

    J. Watson, M. Firman, A. Monszpart, and G. J. Brostow, “Footprints and free space from a single color image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11–

    2020

  21. [29]

    Fishing net: Future inference of semantic heatmaps in grids,

    N. Hendy, C. Sloan, F. Tian, P. Duan, N. Charchut, Y . Xie, C. Wang, and J. Philbin, “Fishing net: Future inference of semantic heatmaps in grids,” arXiv preprint arXiv:2006.09917 , 2020. 4

    2006 arXiv

  22. [30]

    Roadrunner–learning traversability estimation for autonomous off-road driving,

    J. Frey, S. Khattak, M. Patel, D. Atha, J. Nubert, C. Padgett, M. Hut- ter, and P. Spieler, “Roadrunner–learning traversability estimation for autonomous off-road driving,” arXiv preprint arXiv:2402.19341 , 2024. 4

    2024 arXiv

  23. [31]

    Learning repre- sentations by back-propagating errors,

    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning repre- sentations by back-propagating errors,” nature, vol. 323, no. 6088, pp. 533–536, 1986. 4

    1986

  24. [32]

    Long short-term memory,

    S. Hochreiter, “Long short-term memory,” Neural Computation MIT- Press, 1997. 4, 7

    1997

  25. [33]

    Learning phrase representations using rnn encoder-decoder for statistical machine translation,

    K. Cho, B. Van Merri ¨enboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio, “Learning phrase representations using rnn encoder-decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078, 2014. 4

    2014 arXiv

  26. [34]

    Motion trajectory prediction based on a cnn-lstm sequential model,

    G. Xie, A. Shangguan, R. Fei, W. Ji, W. Ma, and X. Hei, “Motion trajectory prediction based on a cnn-lstm sequential model,” Science China Information Sciences , vol. 63, pp. 1–21, 2020. 4

    2020

  27. [35]

    Yoon and K

    Y . Yoon and K. Yi, “Trajectory prediction using graph-based deep learning for longitudinal control of autonomous vehicles: A proactive ap- proach for autonomous driving in urban dynamic traffic environments,” IEEE Vehicular Technology Magazine, vol. 17, no. 4, pp. 18–27, 2022. 4

    2022

  28. [36]

    Aircraft trajectory prediction using lstm neural network with embedded convolutional layer,

    Y . Pang, N. Xu, and Y . Liu, “Aircraft trajectory prediction using lstm neural network with embedded convolutional layer,” in Proceedings of the Annual Conference of the PHM Society , vol. 11. PHM Society Scottsdale, AZ, USA, 2019, pp. 1–8. 4, 7, 8, 9, 11

    2019

  29. [37]

    Segment everything everywhere all at once,

    X. Zou, J. Yang, H. Zhang, F. Li, L. Li, J. Wang, L. Wang, J. Gao, and Y . J. Lee, “Segment everything everywhere all at once,” in Thirty- seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=UHBrWeFWlL 5, 8

    2023

  30. [38]

    R. N. Yong, E. A. Fattah, and N. Skiadas, Vehicle traction mechanics. Elsevier, 2012. 4, 7

    2012

  31. [39]

    P. J. Blau, Friction science and technology: from concepts to applica- tions. CRC press, 2008. 4

    2008

  32. [40]

    V ortex studio,

    C. L. Simulations, “V ortex studio,” https://www.cm-labs.com, 2025, version 2025.1 [Computer software]. 4

    2025

  33. [41]

    An integrated framework for high-performance, high-fidelity simulation of ground vehicle-tyre-terrain interaction,

    R. Serban, D. Negrut, A. Recuero, and P. Jayakumar, “An integrated framework for high-performance, high-fidelity simulation of ground vehicle-tyre-terrain interaction,” International journal of vehicle perfor- mance, vol. 5, no. 3, pp. 233–259, 2019. 4

    2019

  34. [42]

    agxterrain,

    T. Berglund and M. Servin, “agxterrain,” Algoryx Simulation AB, Ume˚a, Sweden, Tech. Rep. v. 1.01, March 2019. [Online]. Available: https://www.algoryx.se/download/agxTerrain tech report.pdf 4, 8

    2019

  35. [43]

    Piaug–physics informed augmentation for learning vehicle dynamics for off-road navigation,

    P. Maheshwari, W. Wang, S. Triest, M. Sivaprakasam, S. Aich, J. G. Rogers III, J. M. Gregory, and S. Scherer, “Piaug–physics informed augmentation for learning vehicle dynamics for off-road navigation,” arXiv preprint arXiv:2311.00815 , 2023. 4

    2023 arXiv

  36. [44]

    Physord: A neuro-symbolic approach for physics-infused motion prediction in off-road driving,

    Z. Zhao, B. Li, Y . Du, T. Fu, and C. Wang, “Physord: A neuro-symbolic approach for physics-infused motion prediction in off-road driving,” arXiv preprint arXiv:2404.01596 , 2024. 4

    2024 arXiv

  37. [45]

    The contact dynamics method,

    K. Bagi, “The contact dynamics method,” in Computational Modeling of Masonry Structures Using the Discrete Element Method. IGI Global, 2016, pp. 103–122. 6

    2016

  38. [46]

    Pacejka, Tire and Vehicle Dynamics

    H. Pacejka, Tire and Vehicle Dynamics. Elsevier, 2012. 7

    2012

  39. [47]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, and F. e. a. Massa, “Pytorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems 32 . Curran Associates, Inc., 2019, pp. 8024–8035. 7

    2019

  40. [48]

    Learning neural event functions for ordinary differential equations,

    M. N. Ricky T. Q. Chen, Brandon Amos, “Learning neural event functions for ordinary differential equations,” in ICLR, 2021. 7, 11

    2021

  41. [49]

    Semantic-sam: Segment and recognize anything at any granularity,

    F. Li, H. Zhang, P. Sun, X. Zou, S. Liu, J. Yang, C. Li, L. Zhang, and J. Gao, “Semantic-sam: Segment and recognize anything at any granularity,” arXiv preprint arXiv:2307.04767 , 2023. 8

    2023 arXiv

  42. [50]

    Rellis-3d dataset: Data, benchmarks and analysis,

    P. Jiang, P. Osteen, M. Wigness, and S. Saripalli, “Rellis-3d dataset: Data, benchmarks and analysis,” 2020. 8

    2020

  43. [51]

    V oxelnet: End-to-end learning for point cloud based 3d object detection,

    Y . Zhou and O. Tuzel, “V oxelnet: End-to-end learning for point cloud based 3d object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 4490–4499. 8, 9, 10

    2018

  44. [52]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,

    Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. L. Rus, and S. Han, “Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,” in 2023 IEEE international conference on robotics and automation (ICRA). IEEE, 2023, pp. 2774–2781. 8, 9, 10

    2023

  45. [53]

    Comparing ICP variants on real-world data sets,

    F. Pomerleau, F. Colas, R. Siegwart, and S. Magnenat, “Comparing ICP variants on real-world data sets,” Autonomous Robots , vol. 34, no. 3, pp. 133–148, 2013. [Online]. Available: https://doi.org/10.1007/ s10514-013-9327-2 10, 11, 12

    2013

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.