Pith. sign in

REVIEW 5 major objections 5 minor 17 references

PIG-Nav: Key Insights for Pretrained Image Goal Navigation Models

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that early-fusion of observation and goal views inside a self-supervised Vision Transformer, plus auxiliary global-route predictions, raises zero-shot navigation success by an average of 22.6 percentage points and…

desk verdict Useful navigation-pretraining insights and a genuinely new game-video labeling pipeline, but the zero-shot claim rests on an unstated exclusion of the test game from the pretraining corpus. read the letter →

arxiv 2507.17220 v1 pith:KYLYOXNU submitted 2025-07-23 cs.CV cs.RO

Jiansong Wan , Chengming Zhou , Jinkua Liu , Xiangge Huang , Xiaoyu Chen , Xiaohan Yi , Qisen Yang , Baiting Zhu
show 9 more authors
Xin-Qiang Cai Lixing Liu Rushuai Yang Chuheng Zhang Sherif Abdelfattah Hayong Shin Pushi Zhang Li Zhao Jiang Bian
This is my paper · ORCID
classification cs.CVcs.RO
keywords image-goalnavigationpretrainedmodelsearly-fusionarchitectureVisionTransformergamevideopretraininginversedynamicsmodelzero-shotgeneralizationdata-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that pretrained image-goal navigation models generalize far better when the current observation and the goal image are fused early inside a Vision Transformer initialized with self-supervised weights, rather than compared as a whole late in the network. It further claims that adding auxiliary training targets -- the relative pose to the goal, the navigation distance, and a global waypoint path -- sharpens the learned navigation representation, and that a large corpus of gameplay videos can be converted into effective navigation pretraining data by filtering clips with a vision-language model and labeling actions with an inverse dynamics model. If these claims hold, a navigation foundation model could be built with much less robot-specific labeled data, retaining good performance on about one eighth of the usual fine-tuning set and transferring to a new building or a real robot. The reported improvements are absolute success-rate gains: an average of 22.6 percentage points in zero-shot settings and 37.5 percentage points after fine-tuning, relative to three existing visual navigation foundation models.

What carries the argument

The carrying object is an early-fusion Vision Transformer: patches from the current observation and the goal image are embedded separately, tagged with learnable tokens that say which image they came from, and then fed jointly into the same ViT, so cross-attention between the two views happens at the patch level rather than after each image has been independently summarized. The encoder starts from MAE-pretrained ViT-Base weights (DINOv2 initialization is tested and does better than scratch but worse than MAE), and a [CLS] token feeds three auxiliary heads -- relative pose to goal, navigation distance, and global path prediction -- alongside a head that predicts the next ten waypoint actions in a 4-dimensional delta-pose space. On the data side, the pipeline labels raw gameplay clips by having Qwen2-VL classify a video as navigation-suitable in two filtering steps (52% precision and 75% recall in step one, 85% precision and 68% recall after prompt tuning), then an inverse dynamics model trained on open navigation datasets predicts per-frame actions [∆x, ∆y, cos ∆ψ, sin ∆ψ]. These two mechanisms together -- cross-attention at the patch level plus global-route supervision on richly labeled data -- are what PIG-Nav claims carry the performance gain.

What would settle it

Inspect the list of roughly 2,000 games used to build the pretraining corpus for ShooterGame clips; if any appear, or if removing all ShooterGame clips and retraining substantially lowers the reported Highrise and Sanctuary zero-shot success rates, the out-of-distribution generalization claim is not supported.

Watch

Extended reading notes

Core claim

PIG-Nav's central claim is that a pretrained image-goal navigation policy should (1) let a Vision Transformer attend jointly to the observation and goal image patch tokens from the earliest layer, with the encoder initialized from a self-supervised MAE pretraining, and (2) be trained with auxiliary heads that predict the relative pose to the goal, the total navigation distance, and ten equally spaced global waypoints in addition to the standard waypoint action chunk. The authors argue that these design choices capture low-level visual correspondences and global route structure that late-fusion CNN baselines miss. The paper also claims that adding a large corpus of gameplay videos, filtered by a vision-language model and labeled by a trained inverse dynamics model, improves pretraining: validation loss drops from 0.189 to 0.164 when game data is included. Across the two simulated ShooterGame environments (Highrise and Sanctuary) and one real-wheeled-robot setting, PIG-Nav reports an average absolute gain of 22.6 percentage points in zero-shot success rate and 37.5 percentage points in fine-tuned success rate over GNM, ViNT, and NoMaD, with the largest margins on the Medium and Hard tasks, and it retains competitive performance while using roughly one eighth of the fine-tuning data.

Load-bearing premise

The load-bearing premise is that the two simulated test environments, Highrise and Sanctuary, are genuinely unseen at pretraining time; the paper does not state that ShooterGame videos were excluded from its roughly 220,000-gameplay-video corpus.

Editorial extensions

If this is right

  • If the central claim is right, zero-shot deployment of navigation policies becomes practical: PIG-Nav reports 0.84 success rate on the Sanctuary environment without any fine-tuning, where the strongest baseline reaches 0.42.
  • Pretraining data efficiency improves roughly eightfold: fine-tuning on 1/8 of the downstream dataset keeps PIG-Nav competitive, and its zero-shot performance beats baselines fine-tuned on 1/16 or 1/8 of that data.
  • Early fusion is the load-bearing architectural choice: replacing it with late [CLS] fusion cuts zero-shot SR from 0.693 to 0.487 in Highrise and from 0.540 to 0.280 in Sanctuary.
  • Each auxiliary task contributes: removing the global-path, distance, or relative-goal head lowers zero-shot SR, and the fall is steepest when all three are removed.
  • Game-video pretraining data helps beyond public robotics datasets, lowering average pretraining validation loss from 0.189 to 0.164.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never states whether ShooterGame, the source of both simulated test environments, appears in the roughly 2,000-game pretraining video collection; if it does, the 'zero-shot' results are in-distribution rather than out-of-distribution. A clean test would be to retrain on the corpus with all ShooterGame clips removed.
  • The same VLM-filter-plus-IDM-label pipeline could be reused to generate action labels for other video corpora, including egocentric or drone footage, provided the IDM transfers across visual domains; the paper's reported IDM quality on game video suggests but does not prove such transfer.
  • The early-fusion principle is not specific to navigation: any goal-conditioned policy that must align a current observation with a target image could benefit from patch-level cross-attention, so the design insight is testable in manipulation or visual servoing settings.
  • The auxiliaries make the representation predict route-level quantities, which may be why the model generalizes with less data; this suggests that other robotics foundation models should consider global supervisory signals such as distance-to-go or predicted path rather than only local action chunks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces PIG-Nav, a pretrained image-goal navigation model that combines an early-fusion ViT encoder with auxiliary prediction tasks (relative pose, navigation distance, global path) and a game-video labeling pipeline. The authors report large gains over GNM, ViNT, and NoMaD: an average 22.6% improvement in zero-shot success rate and 37.5% in fine-tuned success rate across two ShooterGame environments and a real-robot environment, plus an approximately eightfold reduction in fine-tuning data. Additional ablations study early fusion, ViT initialization (MAE vs. DINOv2 vs. raw), and auxiliary losses. The paper also describes a VLM-based filtering pipeline and an IDM-based action annotation procedure for converting gameplay videos into navigation training data.

Significance. If the zero-shot generalization claim holds, PIG-Nav would be a credible step toward more general visual navigation foundation models: the architecture is simple, the auxiliary losses are well motivated, and the game-video labeling pipeline is a practical contribution. The paper compares against three established baselines, includes both simulation and real-robot deployments, and provides systematic ablations of all major design choices. However, the headline claims rest on missing support: the test environments are not shown to be absent from the pretraining corpus, the game-data benefit is validated only by the training loss itself, the eightfold data-efficiency claim is not quantified in the text, and no variance or significance information accompanies any result. These gaps make the central claims plausible but not yet established.

major comments (5)
  1. [Section 4.2 vs. Section 5.1] The zero-shot generalization claim requires that the Highrise and Sanctuary test maps (both from ShooterGame) be absent from the pretraining corpus of roughly 2,000 games and 220,000 gameplay videos described in Section 4.2. The paper never states that ShooterGame was excluded, and no game list or frame-overlap check is provided. If any ShooterGame videos were included, the 'zero-shot' results are in-distribution, and the 22.6% average absolute gain over ViNT does not demonstrate generalization to an unseen environment. Please state explicitly whether ShooterGame content appears in the pretraining data, describe how exclusion was enforced, and ideally add a nearest-frame or texture-overlap analysis between the pretraining videos and the test environments.
  2. [Section 5.2.3 / Table 2] The benefit of the labeled game video data is measured solely by average validation loss, which is the same objective being minimized during pretraining. A drop from 0.189 to 0.164 on the training/validation distribution is circular evidence that the game data improves navigation performance; it does not show that downstream SR or SPL improves. Please either report downstream zero-shot and fine-tuned SR/SPL with and without game data, or explicitly soften the claim to 'the game data reduces pretraining loss' rather than 'improves model performance.'
  3. [Section 5.2.2 / Figure 3] The 'approximately eightfold reduction in fine-tuning data' is a headline finding but is not substantiated by any numbers in the text. The section only refers to Figure 3 and states that PIG-Nav maintains high accuracy with less data. Please report the actual data-efficiency numbers: for example, the SR/SPL achieved with 1/8 of the fine-tuning data compared to the full-data baseline, and define the equivalence criterion used to conclude an eightfold reduction.
  4. [Abstract / Section 5.2.1 / Table 1] The reported '22.6% improvement' and '37.5% improvement' are absolute differences in success rate (percentage points), not relative improvements. For example, on the real-robot task the zero-shot SR is 0.50 for PIG-Nav versus 0.13 for ViNT, a 37-point absolute gap but a roughly 285% relative change. The current wording is misleading; please label these as absolute percentage-point gains or report relative improvements alongside.
  5. [Section 5.1 / Table 1 and all result tables] No error bars, confidence intervals, number of random seeds, or statistical tests are reported for any result. The real-robot evaluation consists of only 8 tasks, and several baseline entries (e.g., GNM and NoMaD zero-shot robot SR) are 0.00, making comparisons highly sensitive to single-run noise. Please provide variance estimates (e.g., across seeds or bootstrap over episodes) and a significance test for the headline differences, or clearly state the number of independent runs performed.
minor comments (5)
  1. [Abstract / Introduction] The phrases '22.6%' and '37.5%' should be qualified as absolute percentage-point gains in success rate averaged over tasks, to match the numbers in Table 1.
  2. [Section 4.2 / Section 8.2] The two-step VLM filtering is described with precision/recall numbers, but it is unclear what the '500 positive samples' in the second step are and how the combined 80% precision / 20% recall criterion was chosen; please clarify the labeling protocol.
  3. [Section 8.1] The sentence 'certain data sources remain unavailable compared to those used in previous works' is vague; please specify which public datasets from GNM/ViNT/NoMaD were not included and whether this affects the fairness of the comparison.
  4. [Figure 3] The figure is referenced repeatedly but its axes and quantitative contents are not described in the text beyond the caption; please ensure the figure is actually included and add a brief description of how the data-efficiency curves were generated.
  5. [General] Several reference entries contain formatting errors (e.g., 'InProceedings', 'In2021'), and table captions are inconsistent in punctuation and capitalization; a copyedit pass is needed.

Circularity Check

3 steps flagged · score 4.0 of 10

Central SR/SPL results are externally measured, but two supporting evaluations are self-referential and the zero-shot claim rests on an unstated pretraining/test disjointness.

  1. fitted input called prediction [Section 4.2 (Data Cleaning by VLM); Appendix 8.2]
    "In the second filtering process, we experiment with 12 different prompt instructions and select the one yielding the best weighted balance between precision and recall to improve dataset quality. The final prompts we picked result in 85% precision and 68% recall, demonstrating that our two-step filtering strategy effectively constructs a high-quality video dataset for navigation tasks. ... we test total 12 prompts in Qwen2 with our labeling 500 positive samples."

    The reported 85% precision and 68% recall are computed on the same 500 manually labeled samples that were used to select among the 12 prompts, with the selection criterion being a weighted balance of precision and recall on exactly this set. Reporting the resulting precision/recall on the selection set turns the prompt-tuning procedure into the evidence for filtering quality: the numbers are optimized by construction rather than estimated on held-out data. This is a fitted evaluation presented as a validation of the pipeline.

  2. other [Section 5.2.3, Table 2]
    "we pretrain PIG-Nav using both this dataset and standard public navigation datasets and compare the average validation loss. As illustrated in Table 2, we find that incorporating additional labeled game video data results in lower validation losses, demonstrating the efficiency of our labeled dataset and its processing pipeline."

    The asserted benefit of the game video dataset is supported only by a lower average validation loss, which is the same loss family (waypoint action plus auxiliary objectives) minimized during pretraining. Since the model is trained to minimize exactly this objective, an in-distribution loss reduction is a self-referential consistency check rather than independent evidence of navigation benefit; the paper does not tie this loss reduction to downstream SR/SPL in this experiment. This is not central because the main comparisons use SR/SPL, but it is presented as a demonstration of data value.

1 more flagged steps
  1. self definitional [Abstract; Section 4.2; Section 5.1]
    "Recent studies have explored pretrained (foundation) models for vision-based robotic navigation, aiming to achieve generalizable navigation and positive transfer across diverse environments while enhancing zero-shot performance in unseen settings. ... We collect over 220,000 long gameplay videos, sourced from approximately 2,000 different games, as our original dataset. ... The two simulation environments, Highrise and Sanctuary, are both part of ShooterGame, a representative PC multiplayer First-Person Shooter (FPS) game built on Unreal Engine 4."

    The paper's central zero-shot generalization claim requires Highrise and Sanctuary to be unseen environments. The pretraining corpus is described as about 2,000 games, with no statement that ShooterGame was excluded and no game list or frame-overlap check. If any ShooterGame videos are in the corpus, the zero-shot evaluation is in-distribution by construction, and the reported 22.6% improvement is a recall of pretraining statistics rather than evidence of generalization. This is a missing exclusion that is load-bearing for the headline claim; it is a correctness risk and a definitional precondition that the paper asserts without verification.

full rationale

The headline results in Table 1 are measured on external downstream metrics (SR and SPL) on Highrise, Sanctuary, and a real robot, so they are not fitted tautologies and the central comparison against GNM, ViNT, and NoMaD has independent content. No load-bearing self-citation chain is present; architectural choices are tested by ablations on SR/SPL. However, two supporting evaluations are self-referential: the game-video filtering quality is reported as precision/recall on the same 500 labeled samples used to select among prompts, and the benefit of game video data is evidenced only by a lower validation loss of the very objective minimized during pretraining. Additionally, the zero-shot generalization claim depends on Highrise/Sanctuary being absent from the ~2,000-game pretraining corpus, which the paper never states; if that exclusion fails, the zero-shot result reduces to in-distribution recall. These issues are partial and mostly concern supporting evidence rather than the main externally measured SR/SPL comparisons, so a moderate circularity score is appropriate.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the quality, scale, and non-overlap of the pretraining data, the transfer of the inverse dynamics model, and the VLM filter choices; none of these are independently verified outside the paper.

free parameters (5)
  • Data-cleaning displacement threshold = 5x average displacement
    Section 4.1: filters public trajectories before pretraining; chosen ad hoc, not from a principled criterion.
  • VLM prompt selection weighting = 80% precision / 20% recall
    Section 8.2: the final prompt was chosen under this hand-set weighting, which changes which clips enter the dataset.
  • IDM labeling configuration = 48 frame pairs per 10s clip; 26 hours training
    Section 4.2: chosen from small-batch experiments; these labels supervise all game data.
  • Auxiliary loss weights = not reported
    Section 3.2: the combination of Lwaypoint, Lrelative, Lnav_distance, and Lglobal is not specified; different weights would change the learned representation.
  • Data normalization scheme = not specified which of three
    Section 8.1: the best of three normalization techniques was selected by performance, so the choice is data-dependent.
assumptions (5)
  • domain assumption Public navigation datasets (GoStanford, RECON, CoryHall, BDD100K, SCAND, TartanDrive, SACSoN) provide valid trajectory supervision for pretraining.
    Section 4.1: the model's waypoint and auxiliary losses are computed entirely from these trajectories; any systematic pose or scaling error propagates to all downstream results.
  • domain assumption The IDM trained on public data transfers to gameplay videos from about 2,000 games and yields accurate 4D action labels.
    Section 4.2: the game-video pretraining set is labeled by this IDM; no accuracy metric on held-out game frames is reported, only visual examples.
  • domain assumption The Qwen2-VL two-step filter with the selected prompts keeps a useful fraction of game clips.
    Section 4.2: the filter's best prompt reaches 85% precision and 68% recall on a 500-clip sample; if these rates do not generalize, the dataset composition changes.
  • domain assumption ShooterGame (Highrise and Sanctuary) is not included in the pretraining game-video corpus.
    Implied by the 'zero-shot generalization' claim in Sections 1 and 5.1, but never stated; if false, zero-shot results become in-distribution recall.
  • domain assumption Standard deep learning assumptions: Adam, learning rate 5e-5, batch size 128, 200 epochs, and equal weighting of loss terms converge to a useful policy.
    Section 5.1 hyperparameters; no sensitivity analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PIG-Nav: Key Insights for Pretrained Image Goal Navigation Models." pith.science (2026). https://pith.science/paper/KYLYOXNU

@misc{pith2026250717220,
  author       = {Pith},
  title        = {Pith review of: PIG-Nav: Key Insights for Pretrained Image Goal Navigation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KYLYOXNU}},
  note         = {Machine review of arXiv:2507.17220}
}
read the original abstract

Recent studies have explored pretrained (foundation) models for vision-based robotic navigation, aiming to achieve generalizable navigation and positive transfer across diverse environments while enhancing zero-shot performance in unseen settings. In this work, we introduce PIG-Nav (Pretrained Image-Goal Navigation), a new approach that further investigates pretraining strategies for vision-based navigation models and contributes in two key areas. Model-wise, we identify two critical design choices that consistently improve the performance of pretrained navigation models: (1) integrating an early-fusion network structure to combine visual observations and goal images via appropriately pretrained Vision Transformer (ViT) image encoder, and (2) introducing suitable auxiliary tasks to enhance global navigation representation learning, thus further improving navigation performance. Dataset-wise, we propose a novel data preprocessing pipeline for efficiently labeling large-scale game video datasets for navigation model training. We demonstrate that augmenting existing open navigation datasets with diverse gameplay videos improves model performance. Our model achieves an average improvement of 22.6% in zero-shot settings and a 37.5% improvement in fine-tuning settings over existing visual navigation foundation models in two complex simulated environments and one real-world environment. These results advance the state-of-the-art in pretrained image-goal navigation models. Notably, our model maintains competitive performance while requiring significantly less fine-tuning data, highlighting its potential for real-world deployment with minimal labeled supervision.

Figures

Figures reproduced from arXiv: 2507.17220 by the authors.

Figure 1
Figure 1. Overview of PIG-Nav. contribute to this issue: 1) The design of existing models focuses on the relative information between observations and goals as a whole, failing to capture low-level correspondences. Additionally, they often employ low-capacity encoders that are not initialized with pretrained weights, restricting their ability to learn transferable navigation representations. 2) The datasets used for pretraini… view at source ↗
Figure 2
Figure 2. Game Video Data Processing Pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average SR and Validation Loss performance comparison across different fine-tuning dataset size. Left for [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Three evaluation examples of PIG-Nav on three environments: Highrise, Sanctuary, Real Robot. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Examples of Qwen2 two steps filtering. State I State II State III State IV E1 E2 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Visualization of IDM labeled data. Four stages from Stage I to Stage II for two environment E1 and E2 are [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 5
Figure 5. Figure 5: In details, Figure 5 show the four types of Qwen2 model classification outputs, including True Positive, False [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 7
Figure 7. Figure 7: Rollout example in Highrise environment. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Rollout example in Sanctuary environment. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages

  1. [5]

    Hm3d-ovon: A dataset and benchmark for open-vocabulary object goal navigation

    Naoki Yokoyama, Ram Ramrakhya, Abhishek Das, Dhruv Batra, and Sehoon Ha. Hm3d-ovon: A dataset and benchmark for open-vocabulary object goal navigation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5543–5550. IEEE,

    work page 2024

  2. [6]

    Ovrl-v2: A simple state-of-art baseline for imagenav and objectnav

    Karmesh Yadav, Arjun Majumdar, Ram Ramrakhya, Naoki Yokoyama, Alexei Baevski, Zsolt Kira, Oleksandr Maksymets, and Dhruv Batra. Ovrl-v2: A simple state-of-art baseline for imagenav and objectnav. arXiv preprint arXiv:2303.07798,

    arXiv

  3. [8]

    Target-driven visual navigation in indoor scenes using deep reinforcement learning

    Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA), pages 3357–3364. IEEE,

    work page 2017

  4. [9]

    Affordances-oriented planning using foundation models for continuous vision-language navigation

    Jiaqi Chen, Bingqian Lin, Xinmin Liu, Lin Ma, Xiaodan Liang, and Kwan-Yee K Wong. Affordances-oriented planning using foundation models for continuous vision-language navigation. arXiv preprint arXiv:2407.05890,

    arXiv

  5. [11]

    The pitfalls of next-token prediction

    Gregor Bachmann and Vaishnavh Nagarajan. The pitfalls of next-token prediction. arXiv preprint arXiv:2403.06963,

    arXiv

  6. [12]

    Learning to achieve goals with belief state transformers

    Edward S Hu, Kwangjun Ahn, Qinghua Liu, Haoran Xu, Manan Tomar, Ada Langford, Dinesh Jayaraman, Alex Lamb, and John Langford. Learning to achieve goals with belief state transformers. arXiv preprint arXiv:2410.23506,

    arXiv

  7. [15]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191,

    arXiv

  8. [16]

    We define the provided navigation trajectory as: τ = (o0, o1, · · ·, oT ; p0, p1, · · ·, pT ) where T represents the total number of steps in the trajectory

    12 PIG-Nav: Key Insights for Pretrained Image Goal Navigation Models PREPRINT 7 Training Tasks for PIG-Nav Notations. We define the provided navigation trajectory as: τ = (o0, o1, · · ·, oT ; p0, p1, · · ·, pT ) where T represents the total number of steps in the trajectory. The first observation ocur = o0 corresponds to the current image, while the final...

    work page 2024

Show all 17 references
  1. [17]

    Dpos_yaw(p1, p2) = (x2 − x1)2 + (y2 − y1)2 + (cos(ψ2) − cos(ψ1))2 + (sin(ψ2) − sin(ψ1))2 Relative Pose to Goal

    The following position-yaw L2 metric quantifies the discrepancy between predicted and ground-truth positions and yaw angles, assessing the accuracy of the model’s waypoint predictions. Dpos_yaw(p1, p2) = (x2 − x1)2 + (y2 − y1)2 + (cos(ψ2) − cos(ψ1))2 + (sin(ψ2) − sin(ψ1))2 Rel...

    2024

  2. [2008]

    Ving: Learning open-world navigation with visual goals

    Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, and Sergey Levine. Ving: Learning open-world navigation with visual goals. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 13215–13222. IEEE, 2021a. Andrew Szot, Alexander Clegg, ...

    2021

  3. [2019]

    Rapid exploration for open-world navigation with latent goal models

    Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, and Sergey Levine. Rapid exploration for open-world navigation with latent goal models. arXiv preprint arXiv:2104.05859, 2021b. Gregory Kahn, Adam Villaflor, Bosen Ding, Pieter Abbeel, and Sergey Levine. Self-su...

    2018 arXiv

  4. [2020]

    Tartandrive: A large-scale dataset for learning off-road dynamics models

    Samuel Triest, Matthew Sivaprakasam, Sean J Wang, Wenshan Wang, Aaron M Johnson, and Sebastian Scherer. Tartandrive: A large-scale dataset for learning off-road dynamics models. In 2022 International Conference on Robotics and Automation (ICRA) , pages 2546–2552. IEEE,

    2022

  5. [2021]

    Gnm: A general navigation model to drive any robot

    Dhruv Shah, Ajay Sridhar, Arjun Bhorkar, Noriaki Hirose, and Sergey Levine. Gnm: A general navigation model to drive any robot. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 7226–7233. IEEE, 2023a. Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyl...

    2023 arXiv

  6. [2022]

    Vtnet: Visual transformer network for object goal navigation

    Heming Du, Xin Yu, and Liang Zheng. Vtnet: Visual transformer network for object goal navigation. arXiv preprint arXiv:2105.09447,

    arXiv

  7. [2023]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193,

    arXiv

  8. [2024]

    Navid: Video-based vlm plans the next step for vision-and-language navigation

    Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and-language navigation. arXiv preprint arXiv:2402.15852, 2024a. Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Mi...

    arXiv

  9. [2025]

    The one ring: a robotic indoor navigation generalist

    11 PIG-Nav: Key Insights for Pretrained Image Goal Navigation Models PREPRINT Ainaz Eftekhar, Luca Weihs, Rose Hendrix, Ege Caglar, Jordi Salvador, Alvaro Herrasti, Winson Han, Eli VanderBil, Aniruddha Kembhavi, Ali Farhadi, et al. The one ring: a robotic indoor navigation gen...

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.