REVIEW 4 major objections 6 minor 31 references
From Simulation to Field: Learning Terrain Traversability for Real-World Deployment
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims that a neural network trained exclusively on simulated LIDAR and IMU data can predict continuous, heading-aware terrain traversability costs on a real robot, without any real-world training data.
desk verdict Promising sim-to-real traversability pipeline with a genuinely novel architecture, but the real-world accuracy claim is not backed by quantitative evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is TraverseNet, a PointNet-derived architecture that consumes point cloud features (coordinates, normals, curvature) alongside a 13-dimensional IMU feature vector made of the flattened 3x3 acceleration covariance matrix and a quaternion orientation. LIDAR scans are accumulated over a sliding window and densified into a robot-centric local map using an Extended Kalman Filter state estimate and GPU-accelerated Iterative Closest Point registration. Traversability labels are computed autonomously inside the simulator as $d_n - d_a$, the nominal minus actual distance traveled over a three-second window, which acts as a proxy for how much the terrain resists or assists the robot's motion. The paper's best architecture processes the IMU features through separate fully connected layers before fusing them with point cloud features, which it calls mid-fusion.
What would settle it
On a real outdoor course, compute traversability labels from the actual robot's nominal minus actual travel distance at fixed speed and compare them point-by-point against the network's predictions from simulated training alone: a systematic mismatch on slopes, gravel, or grass would show the sim-to-real transfer does not hold.
Extended reading notes
Core claim
The paper demonstrates that a LIDAR-inertial neural network can predict continuous traversability values from spatiotemporal point cloud maps, where the training labels come from the difference between the nominal distance a simulated robot should travel at a fixed speed and the actual distance it travels over the terrain, clipped to the [0,1] range. On the simulated test set, the best configuration (mid-fusion IMU plus XYZ coordinates and curvature) reaches a mean absolute error of 0.024. The authors further show that the same network, with no real-world fine-tuning, produces locally consistent traversability maps on a real Robotnik AGV, differentiates gravel from asphalt, assigns higher cost to uphill approaches than downhill approaches, and can be used by a point-cloud A* planner to generate collision-free paths.
Load-bearing premise
The transfer rests on the assumption that the simulated robot's wheels, sensors, and terrain interaction behave enough like the real Robotnik AGV that labels derived from simulated locomotion measure the same terrain cost in the field; the paper admits the steering kinematics are not identical.
Editorial extensions
If this is right
- Traversability labels can be generated automatically from simulated robot locomotion, eliminating the need for manual labeling and enabling large-scale data collection.
- The directionality-aware output means planners can distinguish uphill from downhill traversability costs, which is important for energy-efficient and safe navigation in sloped terrain.
- A network trained only in simulation can be deployed directly on a real robot with no real-world data collection, at least for the platform and environments tested.
- The method's continuous cost output, rather than binary traversable/non-traversable classification, supplies richer information for path planning and exploration.
- On an external real-forest dataset, the method produces more conservative traversability estimates than a non-learning baseline, particularly around tree trunks and pedestrians.
Reading between the lines
- If the sim-to-real transfer holds broadly, the same auto-labeling pipeline could be reused for other robot morphologies only if the simulator's wheel-terrain interaction is re-validated; the paper itself notes steering kinematics are not identical.
- A natural testable extension would be to quantify the transfer gap by collecting real-world locomotion labels on the test course and computing MAE against the network's predictions, rather than relying on qualitative deployment results.
- The observed failure mode of dynamic obstacles persisting in the traversability map suggests that combining the learned cost with a temporal filtering or dynamic-object removal step would likely improve real-world navigation robustness.
- The footprint-box discretization and the fixed time window for IMU covariance are parameters that, if changed, would require data re-collection; point-wise or resolution-adaptive cost prediction would be a plausible follow-up.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TraverseNet, a deep neural network for continuous, directionality-aware terrain traversability estimation from LIDAR point clouds and IMU data. Training labels are generated automatically in a Unity-based high-fidelity simulation by comparing the robot's nominal travel distance (from wheel speeds and radius) with the actual distance from an EKF state estimate, clipped to [0,1]. The authors report an ablation study over input features and IMU fusion strategies (Table 1), qualitative results in simulation and on a real Robotnik AGV, a qualitative comparison on an external dataset, and applications to path planning. The central claim is that the method, trained only on simulated data, can accurately predict traversability in real-world applications without any real data collection.
Significance. If the central claim were supported, the work would be a useful contribution to field robotics: an automated simulation-to-real data pipeline for traversability learning, a continuous and heading-aware cost representation, and open-sourced code would lower the barrier for deploying learning-based traversability estimation on outdoor robots. The idea of deriving labels from the robot's own locomotion (nominal minus actual distance) is simple and potentially scalable. However, the paper's evidence does not currently establish the headline claim of accurate real-world prediction: the only quantitative accuracy metric is a simulated test-set MAE, while the real-world evaluation is qualitative. The comparison with prior work is not based on a common benchmark. These gaps are load-bearing because the value of the method for practical deployment rests on sim-to-real transfer.
major comments (4)
- [Abstract and Sec. 4.7] The central claim that the method 'can accurately predict traversability in real-world applications without any real data collection' is not supported by quantitative evidence. The real-world experiments in Sec. 4.7 are presented qualitatively (Figs. 14-16): no real-world traversability labels or independent ground-truth measurements are computed, and no quantitative accuracy metric is reported for the field tests. The external dataset comparison in Sec. 4.6 is explicitly stated to be qualitative because no ground-truth labels exist. I recommend adding a quantitative real-world evaluation, for example by measuring the correlation between predicted costs and robot outcomes (e.g., actual slip, speed reduction, or manual patch labels) on the real platform.
- [Sec. 3.3 and Sec. 4.1] The training label (dn - da) in Eq. (1) depends on wheel radius, fixed speed, and EKF state estimates, and Sec. 4.1 admits that the simulated replica has steering kinematics 'not identical' to the real active-front-and-rear-steering Robotnik AGV. If the simulated wheel-terrain interaction, IMU covariance, or odometry differs from the real platform, then a terrain patch with a given label in simulation may correspond to a different real traversability cost. The paper provides no direct test of this sim-to-real equivalence, yet the abstract's deployment claim rests on it. A concrete calibration experiment (e.g., comparing simulated and real slip or traversability costs on comparable terrain) would address this concern.
- [Table 1 and Sec. 4.3] The reported 'Final Loss' values (e.g., 4.1, 4.0, 4.3) are inconsistent with the MAE values (e.g., 0.05, 0.07) if L1 loss and MAE are the same metric, and the best Test MAE (0.024) is lower than all Train MAE values, which is unusual. The table also reports no variance, number of runs, or dataset sizes, so the ablation differences (0.024 vs. 0.030-0.035) may not be statistically meaningful. Please clarify what 'Final Loss' denotes, report standard deviations or confidence intervals, and state the size of the training, validation, and test splits.
- [Table 2 and Sec. 4.6] The comparison with prior methods is not apples-to-apples: each method is evaluated on its own dataset, with different problem formulations (regression vs. classification), and the performance values are taken directly from the respective papers. The statement in Sec. 4.6 that 'our findings ... indicate an MAE of 0.024 ... for the same resolution' overstates comparability, since Wallin et al.'s MAE of 0.2 was obtained on a different dataset and different grid resolution. I recommend either adapting the methods to a common benchmark (including the Agishev dataset with quantitative metrics, if labels allow) or tempering the comparative claim in the abstract and Table 2.
minor comments (6)
- [Throughout] The paper contains recurring typos, including 'traversibility' in the abstract and headings, 'Comparision' in Sec. 4.6, and 'As City' instead of 'Ås City' in Sec. 3.4.
- [Sec. 3.3] In Eq. (1), the notation r.w1 and r.w2 is ambiguous: it should be clarified whether w1 and w2 are angular or linear wheel speeds, and whether the wheel radius r is assumed identical for both wheels.
- [Sec. 4.5] Fig. 8(b) and (c) are discussed before Fig. 9 introduces the overlapping-box inference method; adding a forward reference or reordering the figures would improve readability.
- [Sec. 4.7] The inference time of 'approximately 0.2 seconds' for around 200 boxes is stated without hardware details; please specify the GPU/CPU used for the real-world deployment.
- [Table 1] The column 'Final Loss' is not defined in Sec. 4.3; a definition should be added so the reader can interpret the ablation table.
- [References] Reference [8] is a course project report (CS229), not a peer-reviewed publication; consider citing a more citable source for the claim that deep networks outperform traditional classifiers in terrain classification.
Circularity Check
No significant circularity: the label generation, network training, and evaluation are not definitionally tied to the claimed real-world prediction.
full rationale
The derivational chain is not circular. The traversability label is the locomotion-based proxy dn - da (Eq. 1, Sec. 3.3), computed from wheel speeds, an EKF state estimate, and IMU readings; the network is trained to predict that label from point-cloud and IMU features. This is ordinary supervised learning rather than a self-definitional reduction: the label is a measured outcome of the robot's motion over a terrain patch, and the held-out test MAE reported in Table 1 is computed on unseen simulated samples, not on the training set. The real-world experiments in Sec. 4.7 do not compute ground-truth real-world labels, so the abstract's claim of 'accurately predict traversability in real-world applications' is under-supported as an external-validity matter, but under-support is not circularity. Similarly, the comparison with the Agishev et al. dataset in Sec. 4.6 is explicitly qualitative because that dataset's labels are method-generated rather than verified ground truth, so no quantitative real-world claim is derived from that comparison. The self-citations [15] and [30] support prior mapping work and the point-cloud planner, but the traversability network's training and evaluation do not reduce to those citations. No equation is equivalent to its input by construction, and no fitted parameter is renamed as a prediction.
Assumptions & free parameters
free parameters (4)
- IMU covariance time window t =
3 seconds
- Robot footprint box =
(1.0, 0.67, 1.0) m
- Fixed data-collection speed =
1 m/s
- Augmentation settings =
voxel 0.05-0.25, Gaussian std 0.05
assumptions (5)
- domain assumption Unity simulation fidelity is sufficient for sim-to-real transfer
- domain assumption Nominal minus actual distance (clipped) is a valid continuous traversability cost
- domain assumption PointNet-style architecture can learn traversability from point cloud patches with IMU features
- domain assumption ICP alignment and EKF state estimates are accurate enough for label crops and map construction
- domain assumption Gaussian noise and random voxel downsampling approximate real LIDAR noise
Cite this review
Pith. "Pith review of From Simulation to Field: Learning Terrain Traversability for Real-World Deployment." pith.science (2026). https://pith.science/paper/4YGE442V
@misc{pith2026250106904,
author = {Pith},
title = {Pith review of: From Simulation to Field: Learning Terrain Traversability for Real-World Deployment},
year = {2026},
howpublished = {\url{https://pith.science/paper/4YGE442V}},
note = {Machine review of arXiv:2501.06904}
}
read the original abstract
The challenge of traversability estimation is a crucial aspect of autonomous navigation in unstructured outdoor environments such as forests. It involves determining whether certain areas are passable or risky for robots, taking into account factors like terrain irregularities, slopes, and potential obstacles. The majority of current methods for traversability estimation operate on the assumption of an offline computation, overlooking the significant influence of the robot's heading direction on accurate traversability estimates. In this work, we introduce a deep neural network that uses detailed geometric environmental data together with the robot's recent movement characteristics. This fusion enables the generation of robot direction awareness and continuous traversability estimates, essential for enhancing robot autonomy in challenging terrains like dense forests. The efficacy and significance of our approach are underscored by experiments conducted on both simulated and real robotic platforms in various environments, yielding quantitatively superior performance results compared to existing methods. Moreover, we demonstrate that our method, trained exclusively in a high-fidelity simulated setting, can accurately predict traversability in real-world applications without any real data collection. Our experiments showcase the advantages of our method for optimizing path-planning and exploration tasks within difficult outdoor environments, underscoring its practicality for effective, real-world robotic navigation. In the spirit of collaborative advancement, we have made the code implementation available to the public.
Figures
Figures from the paper (15 more)
Reference graph
Works this paper leans on
-
[1]
E. Wallin, V. Wiberg, F. Vesterlund, J. Holmgren, H. J. Persson, M. Servin, Learning multiobjective rough terrain traversability, Journal of Terramechanics 102 (2022) 17–26. doi:https://doi.org/10.1016/j.jterra.2022.04.002. URL https://www.sciencedirect.com/science/article/pii/ S0022489822000313
- [2]
-
[3]
I. Vizzo, T. Guadagnino, B. Mersch, L. Wiesmann, J. Behley, C. Stach- niss, KISS-ICP: In Defense of Point-to-Point ICP – Simple, Accurate, and Robust Registration If Done the Right Way, IEEE Robotics and Au- tomation Letters (RA-L) 8 (2) (2023) 1029–1036. doi:10.1109/LRA. 2023.3236571
arXiv 2023
-
[4]
R. Manduchi, A. Castano, A. Talukder, L. Matthies, Obstacle de- tection and terrain classification for autonomous off-road naviga- tion, Autonomous Robots 18 (2005) 81–102. doi:10.1023/B:AURO. 0000047286.62481.1d
-
[5]
M. Bajracharya, J. Ma, M. Malchano, A. Perkins, A. A. Rizzi, L. Matthies, High fidelity day/night stereo mapping with vegetation and negative obstacle detection for vision-in-the-loop walking, in: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 3663–3670. doi:10.1109/IROS.2013.6696879
-
[6]
A. Angelova, L. Matthies, D. Helmick, P. Perona, Fast terrain classifica- tion using variable-length representation for autonomous navigation, in: 34 2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1–8. doi:10.1109/CVPR.2007.383024
-
[7]
M. Bajracharya, B. Tang, A. Howard, M. Turmon, L. Matthies, Learning long-range terrain classification for autonomous navigation, in: 2008 IEEE International Conference on Robotics and Automation, 2008, pp. 4018–4024. doi:10.1109/ROBOT.2008.4543828
-
[8]
K. Shen, M. F. Kelly, Terrain classification for off-road driving cs-229 final report, 2017. URL https://api.semanticscholar.org/CorpusID:30413317
work page 2017
Show all 31 references
-
[9]
T. H. Y. Leung, D. Ignatyev, A. Zolotas, Hybrid terrain traversability analysis in off-road environments, in: 2022 8th International Conference on Automation, Robotics and Applications (ICARA), 2022, pp. 50–56. doi:10.1109/ICARA55094.2022.9738557
2022
-
[10]
Hosseinpoor, J
S. Hosseinpoor, J. Torresen, M. Mantelli, D. Pitto, M. Kolberg, R. Maf- fei, E. Prestes, Traversability analysis by semantic terrain segmenta- tion for mobile robots, in: 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), 2021, pp. 1407–1413....
2021
-
[11]
M. A. Bekhti, Y. Kobayashi, Regressed terrain traversability cost for autonomous navigation based on image textures, Applied Sciences 10 (4) (2020). doi:10.3390/app10041195. URL https://www.mdpi.com/2076-3417/10/4/1195
2020 doi
-
[12]
R. O. Chavez-Garcia, J. Guzzi, L. M. Gambardella, A. Giusti, Learning ground traversability from simulations, CoRR abs/1709.05368 (2017). arXiv:1709.05368. URL http://arxiv.org/abs/1709.05368
2017 arXiv
-
[13]
Agishev, T
R. Agishev, T. Petˇ r ´ ıˇ cek, K. Zimmermann, Trajectory optimization us- ing learned robot-terrain interaction model in exploration of large sub- terranean environments, IEEE Robotics and Automation Letters 7 (2) (2022) 3365–3371. doi:10.1109/LRA.2022.3147332
2022
-
[14]
G. G. Waibel, T. L¨ ow, M. Nass, D. Howard, T. Bandyopadhyay, P. V. K. Borges, How rough is the path? terrain traversability es- timation for local and global path planning, IEEE Transactions on 35 Intelligent Transportation Systems 23 (9) (2022) 16462–16473. doi: 10.1109/TITS...
2022
-
[15]
F. Atas, G. Cielniak, L. Grimstad, Elevation state-space: Surfel- based navigation in uneven environments for mobile robots, in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 5715–5721. doi:10.1109/IROS47612.2022.9981647
2022
-
[16]
J. L. Mart ´ ınez, M. Mor´ an, J. Morales, A. Robles, M. S´ anchez, Super- vised learning of natural-terrain traversability with synthetic 3d laser scans, Applied Sciences 10 (3) (2020). doi:10.3390/app10031140. URL https://www.mdpi.com/2076-3417/10/3/1140
2020 doi
-
[17]
Thakker, N
R. Thakker, N. Alatur, D. D. Fan, J. Tordesillas, M. Paton, K. Otsu, O. Toupet, A.-a. Agha-mohammadi, Autonomous off-road navigation over extreme terrains with perceptually-challenging conditions, in: B. Si- ciliano, C. Laschi, O. Khatib (Eds.), Experimental Robotics, Springer...
2021
-
[18]
H. Xue, H. Fu, L. Xiao, Y. Fan, D. Zhao, B. Dai, Traversability anal- ysis for autonomous driving in complex environment: A LiDAR-based terrain modeling approach, Journal of Field Robotics 40 (7) (2023) 1779–
2023
-
[19]
Fusaro, E
D. Fusaro, E. Olivastri, D. Evangelista, M. Imperoli, E. Menegatti, A. Pretto, Pushing the limits of learning-based traversability analysis for autonomous driving on cpu, in: I. Petrovic, E. Menegatti, I. Markovi´ c (Eds.), Intelligent Autonomous Systems 17, Springer Nature Sw...
2023
-
[20]
Zhang, Y
K. Zhang, Y. Yang, M. Fu, M. Wang, Traversability assessment and tra- jectory planning of unmanned ground vehicles with suspension systems on rough terrain, Sensors 19 (20) (2019). doi:10.3390/s19204372. URL https://www.mdpi.com/1424-8220/19/20/4372
2019 doi
-
[21]
Haddeler, M
G. Haddeler, M. Y. M. Chuah, Y. You, J. Chan, A. H. Adiwahono, W. Y. Yau, C.-M. Chew, Traversability analysis with vision and terrain probing for safe legged robot navigation, Frontiers in Robotics and AI 9 (2022). doi:10.3389/frobt.2022.887910. 36 URL https://www.frontiersin....
2022
-
[22]
T. Guan, Z. He, D. Manocha, L. Zhang, Ttm: Terrain traversability mapping for autonomous excavator navigation in unstructured environ- ments, ArXiv abs/2109.06250 (2021). URL https://api.semanticscholar.org/CorpusID:237503221
2021 arXiv
-
[23]
Borges, T
P. Borges, T. Peynot, S. Liang, B. Arain, M. Wildie, M. Minareci, S. Lichman, G. Samvedi, I. Sa, N. Hudson, et al., A survey on ter- rain traversability analysis for autonomous ground vehicles: Methods, sensors, and challenges, Field Robot 2 (1) (2022) 1567–1627
2022
-
[24]
C. R. Qi, H. Su, K. Mo, L. J. Guibas, Pointnet: Deep learning on point sets for 3d classification and segmentation, arXiv preprint arXiv:1612.00593 (2016)
2016 arXiv
-
[25]
S. Shi, X. Wang, H. Li, Pointrcnn: 3d object proposal generation and detection from point cloud, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019
2019
-
[26]
C. R. Qi, L. Yi, H. Su, L. J. Guibas, Pointnet++: Deep hierarchi- cal feature learning on point sets in a metric space, arXiv preprint arXiv:1706.02413 (2017)
2017 arXiv
-
[27]
Quigley, K
M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, A. Ng, Ros: an open-source robot operating system, Vol. 3, 2009
2009
-
[28]
T. Shan, B. Englot, D. Meyers, W. Wang, C. Ratti, R. Daniela, Lio-sam: Tightly-coupled lidar inertial odometry via smoothing and mapping, in: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2020, pp. 5135–5142
2020
-
[29]
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, CoRR abs/1412.6980 (2014). URL https://api.semanticscholar.org/CorpusID:6628106
2014 arXiv
-
[30]
F. Atas, G. Cielniak, L. Grimstad, Navigating in 3d uneven environ- ments through supervoxels and nonlinear mpc, in: 2023 European Con- ference on Mobile Robots (ECMR), 2023, pp. 1–8. doi:10.1109/ ECMR59166.2023.10256342. 37
2023
-
[1803]
URL https://doi.org/10.1002%2Frob.22209
doi:10.1002/rob.22209. URL https://doi.org/10.1002%2Frob.22209
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.