MCRL4OR: Multimodal Contrastive Representation Learning for Off-Road Environmental Perception

Liang Wang; Yi Yang; Zhang Zhang

REVIEW 3 major objections 7 minor 44 references

MCRL4OR: Multimodal Contrastive Representation Learning for Off-Road Environmental Perception

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A contrastive pre-training objective that aligns fused vision-and-action features with locomotion states yields representations that consistently improve three off-road perception tasks: cross-modal retrieval, dynamics prediction, and…

desk verdict A plausible contrastive pretraining idea whose headline numbers are undermined by mismatched evaluation settings; the segmentation and dynamics claims need matched baselines before they can be trusted. read the letter →

arxiv 2501.13988 v1 pith:VXDAIVY5 submitted 2025-01-23 cs.RO cs.AIcs.CV

Yi Yang , Zhang Zhang , Liang Wang This is my paper · ORCID

classification cs.ROcs.AIcs.CV

keywords multimodalcontrastivelearningoff-roadperceptionself-supervisedpre-trainingdynamicspredictiontraversabilitysegmentationcross-modalretrievalTartanDrivevisualaffordance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that off-road perception can be improved without dense manual annotation by pre-training on raw driving trajectories. It proposes MCRL4OR, a contrastive learning method that aligns the fused features of visual observations and control actions with the resulting locomotion states. Pre-trained on the TartanDrive dataset, the learned encoders are then transferred to downstream tasks. The authors report that this pre-training consistently improves cross-modal retrieval, off-road dynamics prediction, and semantic segmentation of traversable areas. A sympathetic reader would take the central claim to be that the causal structure of driving, where terrain plus control action produces motion, is a useful self-supervision signal for off-road perception.

What carries the argument

The central mechanism is the contrastive alignment objective (observation + action) ↔ locomotion, implemented with three encoders: a Swin-T vision encoder for images, a 1D-CNN sequence encoder for control actions, and a similar sequence encoder for 27-channel locomotion states. Observation and action features are early-fused through a two-layer MLP before computing cosine similarity with locomotion features in a batch, following the CLIP symmetric contrastive loss with a learnable temperature. The causal premise is that a locomotion state is the result of applying a control action on the terrain visible in the observation, so aligning these modalities should force the vision encoder to ground terrain affordances.

What would settle it

Re-run the traversability segmentation evaluation with the OFF-Net baseline and the MCRL4OR-initialized model both trained and tested on the same lower-half crops of the ORFD images, and re-run the dynamics prediction with both the baseline and the proposed model using the same 2-second history window; if the improvements shrink or disappear under matched conditions, the central claim of consistent gains from the learned representations is not supported.

Watch

Extended reading notes

Core claim

MCRL4OR jointly learns three encoders, for visual observations, control actions, and locomotion states, by optimizing a CLIP-style contrastive loss that pulls together the locomotion state embedding and the fused embedding of observation and action from the same time window, while pushing apart mismatched pairs. The paper's central discovery is that this alignment, trained on the large-scale TartanDrive dataset, transfers to multiple off-road perception tasks: the pre-trained locomotion encoder improves dynamics prediction (GRU RMSE dropping from 0.1674 to 0.0593, and with an Informer backbone to 0.0355), and the pre-trained observation encoder improves traversability segmentation accuracy on the ORFD dataset from 88.8% to 95.1% with a comparable encoder, and to 96.8% when combined with multimodal input. The authors interpret this as evidence that the learned representations capture terrain-dependent vehicle dynamics and visual affordances.

Load-bearing premise

The reported improvements are caused by the learned MCRL4OR representations and not by experimental differences such as the lower-half image cropping applied only to the model being tested, or the longer history length used in the main dynamics result.

Editorial extensions

If this is right

If the representations transfer as claimed, off-road perception systems could be initialized from self-supervised driving trajectories rather than requiring expensive dense annotation.
The same (observation + action) ↔ locomotion alignment could be applied to other proprioceptive-exteroceptive sensor pairs, such as lidar and wheel odometry, for different vehicle platforms.
Pre-trained locomotion encoders could serve as reusable feature extractors for multiple dynamics-prediction architectures, reducing fine-tuning epochs from thousands to hundreds.
The segmentation improvements suggest that pre-training on a driving dataset from a different terrain distribution can still benefit traversability segmentation on a new dataset, easing cross-dataset generalization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's reported segmentation gain may be partly attributable to the lower-half image cropping applied during MCRL4OR pre-training and evaluation, since the baseline OFF-Net is evaluated on full images; a controlled comparison with matched cropping would isolate the representation learning effect.
The dynamics prediction comparison is complicated by a longer history window (4 seconds) used in the main result versus the baseline's 2 seconds; after the authors matched settings, the Informer RMSE rises from 0.0355 to 0.0394, still better than baseline but a smaller margin.
The approach could be extended to a world-model pretraining objective for reinforcement learning in off-road navigation, as the authors suggest, but the current paper only evaluates perception tasks, not closed-loop driving.
A direct test of the visual affordance claim would be to verify whether the retrieved images for a given locomotion query indeed correspond to terrains with similar roughness or soil properties, which the paper only illustrates qualitatively.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, this paper is worth a look but the central claim is not currently supported by the numbers. The idea: align (image + control action) with locomotion state using CLIP-style contrastive learning, pre-train on TartanDrive, transfer to retrieval, dynamics prediction, and segmentation. The causal story is plausible and the retrieval results do show the alignment captures terrain-dependent dynamics; code is released. Credit where due: this is more than a trivial CLIP copy, the early fusion is a reasonable choice, and they include ablations (SimCLR, masked action, different backbones) which show effort.

The problem is that the two flagship quantitative results are built on mismatched comparisons. For segmentation, the paper itself says they crop images to the lower half for preprocessing and segmentation, while the OFF-Net baseline numbers are the original full-image results from the ORFD paper. They never report a no-MCRL4OR OFF-Net run on the same crop. The 6.3% accuracy gain can easily come from evaluating on an easier, mostly-ground mask. That's load-bearing. The table with OFF-Net* and OFF-Net+Ours* does not include the cropped no-pretraining control; the ablation in Table 6 compares MCRL4OR-cropped OFF-Net against the original 88.8% full-image number. So the claimed consistent improvement on segmentation is not established.

Similarly, the headline dynamics number is Informer at 0.0355 RMSE with 4-second history, while the baseline uses 2 seconds. The appendix corrects this to 0.0394 under matched settings. Still better than the TartanDrive GRU baseline of 0.1674, but the main table is misleading. The GRU improvement from 0.1674 to 0.0593 is more credible, but again no error bars or statistical tests, and they don't compare against a recent multimodal pretraining method like COMPASS.

So: the paper has a sound intuition and promising signs, but the evidence as presented does not support "consistently improves all three tasks." The fix is straightforward: run matched baselines with the identical crop and identical history length, report variance, and compare with the relevant contrastive multimodal pretraining baselines. That's a major revision, not a desk reject. Send it to review, but expect the authors to redo the experiments.

Referee Report

3 major / 7 minor

Summary. The paper proposes MCRL4OR, a CLIP-style multimodal contrastive pretraining method for off-road driving. It learns three encoders for visual observations, control actions, and locomotion states by aligning locomotion states with early-fused observation-plus-action features. The authors pretrain on TartanDrive and evaluate the learned representations on three downstream tasks: cross-modal retrieval, off-road dynamics prediction, and traversability semantic segmentation on ORFD. The central claim is that the MCRL4OR-pretrained encoders consistently improve all three downstream tasks, with the largest reported gains in dynamics prediction (RMSE 0.1674 to 0.0593) and segmentation accuracy (88.8% to 95.1%). The paper includes code release and several ablation studies.

Significance. If the reported improvements are genuine, the work would be a useful contribution to self-supervised representation learning for off-road perception: it leverages the largest real-world off-road driving dataset, proposes a plausible causal alignment strategy, releases code, and studies transfer to a different dataset and task. However, the current experimental protocol leaves the central claim unproven. The segmentation comparison is confounded by an evaluation-mask mismatch, and the dynamics-prediction table reports an unmatched history length. These are load-bearing issues because the paper's headline conclusion is that MCRL4OR consistently improves all three tasks. With matched baselines and corrected tables, the core idea could be salvageable, but the evidence as presented is not yet convincing.

major comments (3)

[Off-Road Semantic Segmentation / Implementation Details] The segmentation experiments are confounded by the lower-half image crop. The Implementation Details section states that images are cropped to the lower half during pretraining, and the Results section states: 'we crop the images and only segment lower half.' However, Table 3 and Table 6 compare MCRL4OR results against the original OFF-Net baseline of 88.8% Acc., which is the full-image result from (Min et al. 2022). No no-MCRL4OR OFF-Net baseline evaluated with the identical lower-half crop and pixel mask is reported. Because the paper itself notes that cropping removes most of the 'unreachable' class, the baseline is being scored on a harder evaluation set. The claimed 6.3% improvement may therefore be an artifact of the evaluation mask rather than of the learned representations. The authors must rerun the OFF-Net baseline with the same lower-half crop and report both cropped and full-image numbers.
[Off-Road Dynamics Prediction / Appendix: Dynamics Prediction] The main dynamics-prediction table is not internally consistent. Table 2 reports an Informer RMSE of 0.0355 for the MCRL4OR-initialized model, but the appendix states that this number was obtained with a 4-second history window while the TartanDrive baseline uses 2 seconds, and that after matching the settings the RMSE becomes 0.0394. Table 2 also reports the GRU row as 0.0593 without stating whether that number uses the matched 2-second setting. The main text must report all rows under the same history length as the baseline, and the 0.0355 value should either be removed or clearly labeled as an unmatched configuration. As written, the table overstates the improvement and obscures the comparison.
[Results: Off-Road Semantic Segmentation / Table 6] The claim that MCRL4OR 'consistently improves' the Swin-T encoder is contradicted by the paper's own Table 6. For random-initialization Swin-T, Acc. drops from 88.0% without MCRL4OR to 87.2% with MCRL4OR, and precision drops from 84.4% to 72.4%, even though F1 and mIoU improve. The text says MCRL4OR 'consistently improves' except for 'a slight drop' in Acc., which is logically inconsistent. The authors should either restrict the consistency claim to the configurations where all metrics improve or provide a seed-averaged comparison with variance to show that the drop is not significant.

minor comments (7)

[Introduction] There is a typo in 'scable dataset' (should be 'scalable dataset').
[Off-Road Dynamics Prediction] 'rooted mean square error' should be 'root mean square error' (RMSE).
[Throughout] The abbreviation is inconsistent: 'MCRL4OR' in most places but 'MCLR4OR' in Figure 5, Table 5, and some appendix text. Please standardize.
[Table 6] The table header has 'Recll' instead of 'Recall'.
[Appendix: Dynamics Prediction] The sentence 'And the prediction model predicts the following n seconds' position and quaternion results take the followingn seconds' action and output of the encoder as input and .' is grammatically incomplete and should be rewritten.
[Appendix: Semantic Segmentation] The parameter counts are reported as '27.24M and 54.76' with the second number missing the unit 'M'.
[Reproducibility Checklist] The checklist marks 'If an algorithm depends on randomness, then the method used for setting seeds is described' as 'NA', but all deep learning training runs are stochastic; the authors should state the number of seeds and report variance or statistical tests.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MCRL4OR's contrastive pre-training objective is self-supervised on unlabeled TartanDrive trajectories, and the downstream dynamics and segmentation tasks are distinct from that objective.

full rationale

The paper's derivation chain is self-contained with respect to circularity. The pre-training objective (Eqs. 1-4) is a CLIP-style contrastive alignment between fused observation+action features and locomotion-state features, optimized on unlabeled TartanDrive triplets. The downstream tasks are independent of this loss: dynamics prediction uses the frozen locomotion encoder to initialize a GRU or Informer predictor trained on RMSE, and semantic segmentation fine-tunes the visual encoder on ORFD labels. No parameter is fitted to a downstream metric and then renamed as a prediction. Baselines such as TartanDrive GRU, OFF-Net, FuseNet, and SNE-RoadSeg come from external prior work, and the paper includes ablations with matched backbones (OFF-Net encoder, Swin-T from ImageNet) to isolate the effect of MCRL4OR pre-training. The acknowledged lower-half-image crop and the main-text comparison to a full-image OFF-Net baseline raise an experimental-fairness concern about whether the segmentation gain is fully attributable to the learned representation, but that is an evaluation confound, not a circular derivation: the claimed improvement is not true by construction nor by re-importation of the paper's own assumptions. The paper does not lean on self-citations as load-bearing evidence; references to CLIP, TartanDrive, and ORFD are external data and methods. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no hand-tuned constants or new entities. Its central claim rests on standard machine learning components and on the domain assumption that the causal relationship between observations, actions, and locomotion is as stated.

assumptions (4)

domain assumption The causal relationship that a locomotion state is the result of taking a control action under the current terrain condition perceived by visual sensors.
This premise motivates the (observation+action) to locomotion alignment strategy in Section 'MCRL4OR Framework'. If the relationship is not causal or not learnable, the contrastive objective may not produce useful representations.
domain assumption The TartanDrive dataset is representative of off-road driving conditions and its training/test split is a valid basis for measuring generalization.
The authors pre-train on TartanDrive's training split and evaluate retrieval and dynamics on its test split, following Triest et al. (2022), trusting that the dataset covers sufficient terrain diversity.
domain assumption The ORFD dataset's semantic labels and evaluation protocol are reliable for assessing transferability.
The segmentation task fine-tunes on ORFD with the standard split and metrics; the authors assume the labels and protocol yield meaningful comparisons.
standard math Established architectures (Swin-T, 1D-CNN sequence encoder) and the CLIP contrastive loss are suitable building blocks for this multimodal representation learning task.
The method relies on these prior models and loss function without modification; the authors treat their effectiveness as background knowledge.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MCRL4OR: Multimodal Contrastive Representation Learning for Off-Road Environmental Perception." pith.science (2026). https://pith.science/paper/VXDAIVY5

@misc{pith2026250113988,
  author       = {Pith},
  title        = {Pith review of: MCRL4OR: Multimodal Contrastive Representation Learning for Off-Road Environmental Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VXDAIVY5}},
  note         = {Machine review of arXiv:2501.13988}
}

read the original abstract

Most studies on environmental perception for autonomous vehicles (AVs) focus on urban traffic environments, where the objects/stuff to be perceived are mainly from man-made scenes and scalable datasets with dense annotations can be used to train supervised learning models. By contrast, it is hard to densely annotate a large-scale off-road driving dataset manually due to the inherently unstructured nature of off-road environments. In this paper, we propose a Multimodal Contrastive Representation Learning approach for Off-Road environmental perception, namely MCRL4OR. This approach aims to jointly learn three encoders for processing visual images, locomotion states, and control actions by aligning the locomotion states with the fused features of visual images and control actions within a contrastive learning framework. The causation behind this alignment strategy is that the inertial locomotion state is the result of taking a certain control action under the current landform/terrain condition perceived by visual sensors. In experiments, we pre-train the MCRL4OR with a large-scale off-road driving dataset and adopt the learned multimodal representations for various downstream perception tasks in off-road driving scenarios. The superior performance in downstream tasks demonstrates the advantages of the pre-trained multimodal representations. The codes can be found in \url{https://github.com/1uciusy/MCRL4OR}.

Figures

Figures reproduced from arXiv: 2501.13988 by the authors.

**Figure 2.** Construction diagram of triplet samples, where a sample consists of a frame of visual image and its following 6s (240 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Some off-road image samples in our work. (a) the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: The left column is linear accelerations along three axes in locomotion state. The blue, orange and green lines are [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Left is linear acceleration along three axes in locomotion state. The blue, orange and green lines are corresponded to [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 7.** Figure 7: (a) is a simplified diagram of original OFF-Net [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

**Figure 8.** Figure 8: Diagram of segmentation with only camera as in [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 26 canonical work pages

[1]

H.; Vora, S.; Liong, V

Caesar, H.; Bankiti, V.; Lang, A. H.; Vora, S.; Liong, V. E.; Xu, Q.; Krishnan, A.; Pan, Y.; Baldan, G.; and Beijbom, O. 2020. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11621--11631

2020
[2]

Chang, M.-F.; Lambert, J.; Sangkloy, P.; Singh, J.; Bak, S.; Hartnett, A.; Wang, D.; Carr, P.; Lucey, S.; Ramanan, D.; et al. 2019. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8748--8757

work page 2019
[3]

Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020 a . A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597--1607. PMLR

2020
[4]

Chen, T.; Kornblith, S.; Swersky, K.; Norouzi, M.; and Hinton, G. E. 2020 b . Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33: 22243--22255

work page 2020
[5]

Chen, X.; Fan, H.; Girshick, R.; and He, K. 2020 c . Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297

arXiv 2020
[6]

Cho, K.; Merrienboer, B.; Gulcehre, C.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP

work page 2014
[7]

Dosovitskiy, A.; Ros, G.; Codevilla, F.; Lopez, A.; and Koltun, V. 2017. CARLA: An open urban driving simulator. In Conference on robot learning, 1--16. PMLR

work page 2017
[8]

Fan, R.; Wang, H.; Cai, P.; and Liu, M. 2020. Sne-roadseg: Incorporating surface normal information into semantic segmentation for accurate freespace detection. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXX 16, 340--356. Springer

work page 2020

Show all 44 references

[9]

Geiger, A.; Lenz, P.; and Urtasun, R. 2012. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, 3354--3361. IEEE

2012
[10]

Gibson, J. J. 1979. Boston:Houghton Mifflin

1979
[11]

Guzhov, A.; Raue, F.; Hees, J.; and Dengel, A. 2022. Audioclip: Extending clip to image, text and audio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 976--980. IEEE

2022
[12]

Hazirbas, C.; Ma, L.; Domokos, C.; and Cremers, D. 2017. Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture. In Computer Vision--ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected P...

2017
[13]

He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9729--9738

2020
[14]

Jiang, P.; Osteen, P.; Wigness, M.; and Saripalli, S. 2021. Rellis-3d dataset: Data, benchmarks and analysis. In 2021 IEEE international conference on robotics and automation (ICRA), 1110--1116. IEEE

2021
[15]

Kahn, G.; Abbeel, P.; and Levine, S. 2021. Badgr: An autonomous self-supervised learning-based navigation system. IEEE Robotics and Automation Letters, 6(2): 1312--1319

2021
[16]

P.; and Ba, J

Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

2014 arXiv
[17]

M.; Fidler, S.; Feng, C.; and Anandkumar, A

Li, Y.; Yu, Z.; Choy, C.; Xiao, C.; Alvarez, J. M.; Fidler, S.; Feng, C.; and Anandkumar, A. 2023. Voxformer: Sparse voxel transformer for camera-based 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9087--9098

2023
[18]

Li, Z.; Wang, W.; Li, H.; Xie, E.; Sima, C.; Lu, T.; Qiao, Y.; and Dai, J. 2022. Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In European conference on computer vision, 1--18. Springer

2022
[19]

Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; and Guo, B. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, 10012--10022

2021
[20]

K.; Song, Y.; McDufft, D.; and Kapoor, A

Ma, S.; Vemprala, S.; Wang, W.; Gupta, J. K.; Song, Y.; McDufft, D.; and Kapoor, A. 2022. COMPASS: Contrastive Multimodal Pretraining for Autonomous Systems. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 1000--1007

2022
[21]

Maturana, D.; Chou, P.-W.; Uenoyama, M.; and Scherer, S. 2018. Real-time semantic mapping for autonomous off-road navigation. In Field and Service Robotics: Results of the 11th International Conference, 335--350. Springer

2018
[22]

Min, C.; Jiang, W.; Zhao, D.; Xu, J.; Xiao, L.; Nie, Y.; and Dai, B. 2022. ORFD: A Dataset and Benchmark for Off-Road Freespace Detection. In 2022 International Conference on Robotics and Automation (ICRA), 2532--2538. IEEE

2022
[23]

Moon, S.; Madotto, A.; Lin, Z.; Dirafzoon, A.; Saraf, A.; Bearman, A.; and Damavandi, B. 2022. IMU2CLIP: Multimodal Contrastive Learning for IMU Motion Sensors from Egocentric Videos and Text. arXiv preprint arXiv:2210.14395

2022 arXiv
[24]

H.; Radia, K.; Chen, J.; Wang, D.; Gog, I.; and Gonzalez, J

Ng, M. H.; Radia, K.; Chen, J.; Wang, D.; Gog, I.; and Gonzalez, J. E. 2020. Bev-seg: Bird's eye view semantic segmentation using geometry and semantic point cloud. arXiv preprint arXiv:2006.11436

2020 arXiv
[25]

K.; Ramanan, D.; Wellington, C.; Wisely Babu, B

Pezzementi, Z.; Tabor, T.; Hu, P.; Chang, J. K.; Ramanan, D.; Wellington, C.; Wisely Babu, B. P.; and Herman, H. 2018. Comparing apples and oranges: Off-road pedestrian detection on the National Robotics Engineering Center agricultural person-detection dataset. Journal of Fiel...

2018
[26]

W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

2021
[27]

Seo, J.; Sim, S.; and Shim, I. 2023. Learning Off-Road Terrain Traversability With Self-Supervisions Only. IEEE Robotics and Automation Letters, 8(8): 4617--4624

2023
[28]

Sun, P.; Kretzschmar, H.; Dotiwalla, X.; Chouard, A.; Patnaik, V.; Tsui, P.; Guo, J.; Zhou, Y.; Chai, Y.; Caine, B.; et al. 2020. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF conference on computer vision and pattern reco...

2020
[29]

Tremblay, J.-F.; Manderson, T.; Noca, A.; Dudek, G.; and Meger, D. 2021. Multimodal dynamics modeling for off-road autonomous vehicles. In 2021 IEEE International Conference on Robotics and Automation (ICRA), 1796--1802

2021
[30]

J.; Wang, W.; Johnson, A

Triest, S.; Sivaprakasam, M.; Wang, S. J.; Wang, W.; Johnson, A. M.; and Scherer, S. 2022. TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models. In 2022 International Conference on Robotics and Automation (ICRA), 2546--2552. IEEE

2022
[31]

Ulyanov, D.; Vedaldi, A.; and Lempitsky, V. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022

2016 arXiv
[32]

L.; Brox, T.; and Burgard, W

Valada, A.; Oliveira, G. L.; Brox, T.; and Burgard, W. 2017. Deep multispectral semantic scene understanding of forested environments using multimodal fusion. In 2016 International Symposium on Experimental Robotics, 465--477. Springer

2017
[33]

N.; Kaiser, .; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

2017
[34]

G.; Han, D.; and Kwon, H

Wigness, M.; Eum, S.; Rogers, J. G.; Han, D.; and Kwon, H. 2019. A rugd dataset for autonomous navigation and visual perception in unstructured outdoor environments. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 5000--5007. IEEE

2019
[35]

K.; et al

Wilson, B.; Qi, W.; Agarwal, T.; Lambert, J.; Singh, J.; Khandelwal, S.; Pan, B.; Kumar, R.; Hartnett, A.; Pontes, J. K.; et al. ???? Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting. In Thirty-fifth Conference on Neural Information Processing ...
[36]

Wu, Y.; and He, K. 2018. Group normalization. In Proceedings of the European conference on computer vision (ECCV), 3--19

2018
[37]

Xu, H.; Ghosh, G.; Huang, P.-Y.; Okhonko, D.; Aghajanyan, A.; Metze, F.; Zettlemoyer, L.; and Feichtenhofer, C. 2021. Videoclip: Contrastive pre-training for zero-shot video-text understanding. arXiv preprint arXiv:2109.14084

2021 arXiv
[38]

Xue, J.; Fang, J.; Li, T.; Zhang, B.; Zhang, P.; Ye, Z.; and Dou, J. 2019. BLVD: Building a large-scale 5D semantics benchmark for autonomous driving. In 2019 International Conference on Robotics and Automation (ICRA), 6685--6691. IEEE

2019
[39]

Ye, H.; Mei, J.; and Hu, Y. 2023. M2F2-Net: Multi-Modal Feature Fusion for Unstructured Off-Road Freespace Detection. In 2023 IEEE Intelligent Vehicles Symposium (IV), 1--7

2023
[40]

Zhang, Q.; Peng, Z.; and Zhou, B. 2022. Learning to Drive by Watching YouTube videos: Action-Conditioned Contrastive Policy Pretraining. European Conference on Computer Vision (ECCV)

2022
[41]

Zhang, R.; Guo, Z.; Zhang, W.; Li, K.; Miao, X.; Cui, B.; Qiao, Y.; Gao, P.; and Li, H. 2022. Pointclip: Point cloud understanding by clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8552--8562

2022
[42]

Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; and Zhang, W. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 11106--11115

2021
[43]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[44]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

H.; Vora, S.; Liong, V

Caesar, H.; Bankiti, V.; Lang, A. H.; Vora, S.; Liong, V. E.; Xu, Q.; Krishnan, A.; Pan, Y.; Baldan, G.; and Beijbom, O. 2020. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11621--11631

2020

[2] [2]

Chang, M.-F.; Lambert, J.; Sangkloy, P.; Singh, J.; Bak, S.; Hartnett, A.; Wang, D.; Carr, P.; Lucey, S.; Ramanan, D.; et al. 2019. Argoverse: 3d tracking and forecasting with rich maps. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8748--8757

work page 2019

[3] [3]

Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020 a . A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597--1607. PMLR

2020

[4] [4]

Chen, T.; Kornblith, S.; Swersky, K.; Norouzi, M.; and Hinton, G. E. 2020 b . Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33: 22243--22255

work page 2020

[5] [5]

Chen, X.; Fan, H.; Girshick, R.; and He, K. 2020 c . Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297

arXiv 2020

[6] [6]

Cho, K.; Merrienboer, B.; Gulcehre, C.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP

work page 2014

[7] [7]

Dosovitskiy, A.; Ros, G.; Codevilla, F.; Lopez, A.; and Koltun, V. 2017. CARLA: An open urban driving simulator. In Conference on robot learning, 1--16. PMLR

work page 2017

[8] [8]

Fan, R.; Wang, H.; Cai, P.; and Liu, M. 2020. Sne-roadseg: Incorporating surface normal information into semantic segmentation for accurate freespace detection. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXX 16, 340--356. Springer

work page 2020