Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation

T0 review · 5 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MFRA, a multi-level fusion and reasoning architecture for vision-and-language navigation, claims state-of-the-art success rates on R2R, REVERIE, and SOON by fusing low-, mid-, and high-level visual semantics with language and navigation…

desk verdict The headline SOTA claim is contradicted by the paper's own table; I would not send this to peer review until code and corrected tables are provided. read the letter →

arxiv 2504.16516 v2 pith:NRNTBIZD submitted 2025-04-23 cs.CV cs.AI

Junrong Yue , Yifan Zhang , Chuan Qin , Bo Li , Xiaomin Lie , Xinlei Yu , Wenxin Zhang , Zhendong Zhao This is my paper · ORCID
classification cs.CVcs.AI
keywords vision-and-languagenavigationhierarchicalmulti-modalfusioninstruction-guidedattentiondynamiccontextintegrationobjectgroundingembodiedagentsCLIPembeddingstransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that an embodied navigation agent follows natural-language instructions more effectively when it fuses visual information at several levels of abstraction — low-level object shapes, mid-level spatial arrangements, and high-level scene semantics — rather than relying on global scene embeddings or object features alone. It introduces MFRA, which uses CLIP to align panoramic views and instruction tokens, a four-stage DIRformer-style encoder-decoder to fuse them hierarchically, and a dynamic reasoning module that attends to instruction-relevant spatial regions while integrating a recurrent history state. The paper claims this architecture achieves state-of-the-art navigation accuracy, path efficiency, and object-grounding precision on R2R, REVERIE, and SOON, with R2R validation-unseen success rate of 50.44 percent versus 48.53 percent for the NaviLLM baseline. If correct, the result would show that explicitly hierarchical cross-modal fusion, rather than simply larger pretrained models or topological planning, is the driver of progress in vision-language navigation.

What carries the argument

The central mechanism is the DIRformer-based hierarchical fusion module, a U-shaped multi-stage transformer with Dynamic Multi-head Transposed Attention (DMTA) and Dynamic Gated Feed-Forward Networks (DGFFN) at each stage. DMTA computes attention between spatial visual features and instruction tokens in a transposed form, letting language filter visual features at each abstraction level, and DGFFN applies a gated nonlinearity to aggregate object interactions into larger spatial arrangements. The stages are posited to correspond to low-level object shapes, mid-level spatial arrangements, and high-level semantic contexts, with decoder skip connections preserving fine-grained cues. Object-level features from a detector and a GRU history embedding are injected through auxiliary DMTA layers, and the fused output is summarized by instruction-guided spatial attention before candidate views are scored for the next action.

What would settle it

Train a control model identical to MFRA but with the four-stage DIRformer fusion replaced by a same-capacity plain transformer receiving the same CLIP features, object tokens, and history, and compare R2R validation-unseen success rate; if the control matches 50.44 percent, the hierarchical tiers are not the cause of the improvement. Alternatively, train linear probes on the three fusion stages and check whether they separate object shapes, spatial arrangements, and room-purpose categories as the paper claims.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that one architecture can beat global-scene, object-centric, LLM-assisted, and topological-planning baselines by routing all modalities through a shared multi-scale fusion backbone. The authors report consistent gains across all three benchmarks, and their ablation study attributes the largest single drop to removing the DIRformer fusion module (4.42 points of success rate) and the largest overall drop to replacing CLIP-based representations with CNN+LSTM features. The mechanism is a four-stage encoder-decoder in which Dynamic Multi-head Transposed Attention aligns spatial features with instruction tokens at every scale while Dynamic Gated Feed-Forward Networks selectively activate spatial-semantic patterns, and object features plus history tokens are injected at each stage. The paper interprets this design as mimicking human top-down and bottom-up reasoning, with low-level detail constraining mid-level arrangements and high-level goals pruning those arrangements.

Load-bearing premise

The paper assumes that the four-stage DIRformer fusion actually produces the intended low-, mid-, and high-level semantic hierarchy, and that the reported gains come from that hierarchy rather than from the added capacity of a stronger pretrained fusion backbone.

Editorial extensions

If this is right

  • On R2R validation-unseen, MFRA reports 50.44 success rate and 35.38 SPL, above NaviLLM (48.53 SR, 34.76 SPL), and on R2R test-unseen it reports 52.43 SR and 39.21 SPL.
  • Ablations attribute the largest single performance drop to removing the DIRformer fusion module and the largest overall drop to replacing CLIP features with CNN+LSTM features, so the paper's case rests on both the pretrained aligned embeddings and the hierarchical fusion structure.
  • Removing the language instruction causes the sharpest drop among the three modalities (14.36 SR), which the paper reads as evidence that instruction-guided attention is what makes the fused representation actionable.
  • Because the same architecture performs well on step-by-step R2R, high-level REVERIE, and long-horizon SOON, the paper claims the approach generalizes across instruction formats, path lengths, and grounding requirements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cleanest way to test the paper's hierarchy claim is a layer-wise linear-probe study: it would reveal whether the three fusion stages genuinely separate object shapes, spatial arrangements, and room-purpose semantics, which the paper currently assumes rather than demonstrates.
  • An instruction-adaptive fusion policy would be a direct next step: lean on object features for fine-grained references such as 'the red mug' and on scene features for references such as 'the sunlit lounge', which the paper motivates but does not implement.
  • The same hierarchical fusion could transfer to continuous-environment or real-robot VLN by replacing the fixed 36-view panorama with an online-stitched visual representation, since the history-injection mechanism already handles variable-length trajectories.
  • Comparing the instruction-guided attention weights with human gaze or referring-expression annotations on REVERIE would test the paper's 'act like a human' framing; the paper does not report such an alignment check.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes MFRA, a Multi-level Fusion and Reasoning Architecture for Vision-and-Language Navigation, combining CLIP-based features, a DIRformer-style U-shaped encoder-decoder for 'hierarchical' multi-modal fusion, an instruction-guided attention module, and auxiliary losses (MLM, MVC, object grounding). The authors claim state-of-the-art performance on REVERIE, R2R, and SOON, reporting numbers in Table 1, ablations in Tables 2–3, and a bar-chart comparison in Figure 4. However, the empirical evidence as presented is internally inconsistent: Table 1's caption says R2R while Section 4.2 says REVERIE; RGS/RGSPL are reported for R2R even though Section 4.1 states R2R has no object grounding; the test-unseen SR is lower than DUET's despite the claimed consistent improvement; and the REVERIE/SOON comparisons lack any numerical table. The paper also does not provide code or a probing analysis to support the claimed semantic hierarchy.

Significance. If the reported results were correct and verifiable, MFRA would be a strong empirical contribution to VLN, combining a U-shaped restoration backbone with multi-modal fusion and showing gains on grounding-adjusted metrics. The paper has clear strengths: it targets a real limitation of current VLN models (uniform fusion and weak temporal modeling), uses standard benchmarks, and includes ablation studies. However, the central claim of 'superior performance compared to state-of-the-art methods' is not currently supported by the manuscript's own evidence. The dataset-label inconsistencies, undefined metrics, and missing numerical results for two of the three claimed benchmarks make the empirical contribution unverifiable as written. The hierarchical-interpretation claim is also not backed by analysis, so the paper's core novelty is not demonstrated.

major comments (5)
  1. [Table 1 and Section 4.2] Table 1 is captioned 'Performance comparison with SOTA methods on the R2R dataset', but Section 4.2 refers to the same table as 'results ... on the REVERIE dataset'. This is not a trivial typo: Section 4.1 explicitly states that R2R has no object grounding and that RGS/RGSPL are only defined for REVERIE and SOON. Therefore the RGS/RGSPL columns in Table 1 cannot be computed for R2R. If the table is actually REVERIE, then no R2R results are provided at all, contradicting the abstract and the table caption. This undermines the primary quantitative evidence for the paper's central claim.
  2. [Table 1, test-unseen row] Even taking Table 1 at face value as a comparison on R2R, the numbers contradict the claim in Section 4.2 of 'consistent and significant improvements in both navigation accuracy and object grounding precision'. Specifically, MFRA's test-unseen SR is 52.43%, lower than DUET's 52.51% from the same table. Additionally, Section 4.2 claims 'the performance drop from seen to unseen environments is much smaller than that of all baseline methods', but the val-seen to val-unseen SR drop for MFRA is 76.88−50.44=26.44 points, which is larger than DUET's drop (71.75−46.98=24.77) and NaviLLM's drop (73.12−48.53=24.59).
  3. [Section 4.5 and Figure 4] The cross-dataset evaluation on REVERIE and SOON consists only of a bar chart (Figure 4) with no numerical values, error bars, or sample sizes, and the text refers to 'the supplementary material', which is not present in the manuscript. The claims that MFRA 'consistently achieves the highest performance' on these datasets are therefore unsupported by any verifiable quantitative evidence. The paper needs to provide full tables with concrete numbers for both datasets, including all standard metrics.
  4. [Tables 2 and 3] The ablation study (Table 2) and the feature-contribution analysis (Table 3) also report RGS and RGSPL on the 'R2R val unseen' split, which is impossible given that R2R has no object-grounding annotations, as stated in Section 4.1. Moreover, Section 4.3 says the ablation is on the 'REVERIE validation unseen split' while Table 2's caption says 'R2R val unseen', and Section 4.4 says Table 3 reports results on 'REVERIE validation unseen' while its caption says 'R2R'. These inconsistencies make it impossible to know which dataset any of the ablation numbers come from.
  5. [Section 3.3] The paper's central conceptual claim is that the DIRformer-based fusion realizes a 'human-like cognitive hierarchy' with low-level object shapes, mid-level spatial arrangements, and high-level semantic contexts. No evidence supports this interpretation: there is no probing analysis, no layer-wise visualization, no attention-token analysis, and no ablation isolating the three tiers. The performance gains could plausibly come from the increased transformer capacity or from the CLIP backbone alone, rather than from a semantically meaningful hierarchy. The authors should either provide such an analysis or substantially weaken the 'think hierarchically' claim.
minor comments (4)
  1. [Section 4.2, first paragraph] The opening sentence says the comparison is 'on the R2R dataset', but the next sentence says the results are 'on the REVERIE dataset'. This needs to be reconciled with Table 1's caption.
  2. [References [40]/[41] and [47]/[48]] The same paper appears twice: Radford et al. 2021 CLIP is listed as both [40] and [41], and LXMERT (Tan and Bansal 2019) is listed as both [47] and [48]. Please deduplicate.
  3. [Section 4.2] The phrase 'fact-level grounding mechanism' appears without prior definition or formal description in the methodology; it is unclear how this differs from the object-grounding loss described in Section 3.5.
  4. [General] No code, training hyperparameters (except loss weights λ1=1.0, λ2=0.5, λ3=1.0), or hardware details are provided, which complicates reproduction of the reported results even after the tables are corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MFRA is an empirical architecture evaluated on standard benchmarks; the table inconsistencies are correctness risks, not evidence of circular derivation.

full rationale

MFRA is an empirical systems paper: it combines CLIP encoders, a DIRformer-style U-shaped transformer, a GRU history encoder, and attention heads, trains with behavior cloning plus auxiliary losses, and evaluates on standard REVERIE/R2R/SOON held-out splits. There is no first-principles derivation whose output is fed back as input. The DIRformer citation is to an external image-restoration paper and is used architecturally, not to justify a prediction; LXMERT initialization and CLIP features are external pretrained models, which provide independent grounding. The only self-citation is ref [57] (ICH-PRNet) in a list of 'recent advancements,' and it is not load-bearing. The paper's central claims rest on empirical numbers, so any concerns about them—e.g., Table 1 is captioned as R2R while Section 4.2 calls the same table REVERIE, and RGS/RGSPL are reported for R2R although Section 4.1 says R2R has no object grounding—are correctness/reproducibility risks, not circularity: the numbers are not equivalent to the model's inputs by construction. No fitted parameter is renamed as a prediction, and no uniqueness theorem from the authors' prior work is invoked. Therefore no circular step is present.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim is empirical; it relies on external pretrained models (CLIP, LXMERT, DIRformer) and benchmark assumptions. The only explicitly hand-set numbers are the loss weights and the fixed depth.

free parameters (2)
  • loss weights lambda1, lambda2, lambda3 = 1.0, 0.5, 1.0
    Set empirically in Section 3.5 with no sensitivity analysis; they control the contribution of three auxiliary losses to the total objective.
  • DIRformer depth (encoder and decoder stages) = 4 stages
    The four-stage U-shaped configuration in Section 4.1 is an architecture choice not justified by experiments.
assumptions (5)
  • domain assumption The VLN benchmarks' panoramic observations, candidate action sets, and expert demonstrations accurately represent the navigation task.
    The policy in Section 3 is trained and evaluated on discrete graph-based VLN environments; real-world transfer is not demonstrated.
  • domain assumption CLIP embeddings provide semantically aligned visual and textual representations sufficient for instruction grounding.
    Section 3.2 relies on CLIP for visual and instruction features; the paper does not validate this choice against other encoders beyond one ablation.
  • ad hoc to paper DIRformer's U-shaped encoder-decoder, designed for image restoration, can be repurposed as a multi-level cross-modal fusion backbone for navigation.
    Section 3.3 imports DIRformer with no analysis showing that its spatial hierarchy corresponds to the claimed low/mid/high semantic tiers.
  • domain assumption Initializing cross-modal interaction layers from LXMERT transfers useful knowledge to VLN.
    Stated in Section 4.1 without evaluation of the initialization's contribution.
  • domain assumption Auxiliary MLM, MVC, and object-grounding losses improve navigation policy learning.
    All three are optimized jointly in Section 3.5; the ablations remove whole modules, not individual losses.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation." pith.science (2026). https://pith.science/paper/NRNTBIZD

@misc{pith2026250416516,
  author       = {Pith},
  title        = {Pith review of: Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NRNTBIZD}},
  note         = {Machine review of arXiv:2504.16516}
}
read the original abstract

Vision-and-Language Navigation (VLN) aims to enable embodied agents to follow natural language instructions and reach target locations in real-world environments. While prior methods often rely on either global scene representations or object-level features, these approaches are insufficient for capturing the complex interactions across modalities required for accurate navigation. In this paper, we propose a Multi-level Fusion and Reasoning Architecture (MFRA) to enhance the agent's ability to reason over visual observations, language instructions and navigation history. Specifically, MFRA introduces a hierarchical fusion mechanism that aggregates multi-level features-ranging from low-level visual cues to high-level semantic concepts-across multiple modalities. We further design a reasoning module that leverages fused representations to infer navigation actions through instruction-guided attention and dynamic context integration. By selectively capturing and combining relevant visual, linguistic, and temporal signals, MFRA improves decision-making accuracy in complex navigation scenarios. Extensive experiments on benchmark VLN datasets including REVERIE, R2R, and SOON demonstrate that MFRA achieves superior performance compared to state-of-the-art methods, validating the effectiveness of multi-level modal fusion for embodied navigation.

Figures

Figures reproduced from arXiv: 2504.16516 by the authors.

Figure 1
Figure 1. Illustration of MFRA selected navigable candi [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall pipeline. (a) The baseline method uses a dual-scale graph transformer to encode the panoramic view, the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of navigation examples. The sentence [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance comparison on REVERIE and SOON [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation

    cs.RO 2026-07 accept novelty 5.5 of 10

    VLN methods show a large sim-to-real gap; a hierarchical system reaches 51% real-world success versus 22% for a monolithic RGB-only system across ten physical scenes.

Reference graph

Works this paper leans on

72 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Dong An, Yuankai Qi, Yan Huang, Qi Wu, Liang Wang, and Tieniu Tan

    work page

  2. [2]

    Dong An, Yuankai Qi, Yangguang Li, Yan Huang, Liang Wang, Tieniu Tan, and Jing Shao. 2023. BEVBert: Multimodal Map Pre-training for Language-guided Navigation. arXiv:2212.04385 [cs.CV] https://arxiv.org/abs/2212.04385

    arXiv 2023

  3. [3]

    Dong An, Hanqing Wang, Wenguan Wang, Zun Wang, Yan Huang, Keji He, and Liang Wang. 2024. ETPNav: Evolving Topological Planning for Vision- Language Navigation in Continuous Environments. arXiv:2304.03047 [cs.CV] https://arxiv.org/abs/2304.03047

    arXiv 2024

  4. [4]

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sün- derhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. 2018. Vision-and- Language Navigation: Interpreting visually-grounded navigation instructions in real environments. arXiv:1711.07280 [cs.CV] https://arxiv.org/abs/1711.07280

    arXiv 2018

  5. [5]

    Enrico Cancelli et al. 2025. Static and dynamic approaches for Embodied Social Navigation from the perspective of an autonomous agent. (2025)

    work page 2025

  6. [6]

    Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta. 2020. Neural Topological SLAM for Visual Navigation. arXiv:2005.12256 [cs.CV] https://arxiv.org/abs/2005.12256

    arXiv 2020

  7. [7]

    Li, Gaowen Liu, Mingkui Tan, and Chuang Gan

    Peihao Chen, Xinyu Sun, Hongyan Zhi, Runhao Zeng, Thomas H. Li, Gaowen Liu, Mingkui Tan, and Chuang Gan. 2023.𝐴2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models. arXiv:2308.07997 [cs.CV] https://arxiv.org/abs/2308.07997

    arXiv 2023

  8. [8]

    Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. 2023. History Aware Multimodal Transformer for Vision-and-Language Navigation. arXiv:2110.13309 [cs.CV] https://arxiv.org/abs/2110.13309

    arXiv 2023

Show all 72 references
  1. [9]

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. 2022. Learning from Unlabeled 3D Environments for Vision-and- Language Navigation. arXiv:2208.11781 [cs.CV] https://arxiv.org/abs/2208.11781

    2022 arXiv

  2. [10]

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. 2022. Think Global, Act Local: Dual-scale Graph Transformer for Vision- and-Language Navigation. arXiv:2202.11742 [cs.CV] https://arxiv.org/abs/2202. 11742

    2022 arXiv

  3. [11]

    Lawrence Zitnick

    Xinlei Chen and C. Lawrence Zitnick. 2014. Learning a Recurrent Visual Rep- resentation for Image Caption Generation. arXiv:1411.5654 [cs.CV] https: //arxiv.org/abs/1411.5654

    2014 arXiv

  4. [12]

    J. C. K. Chow. 2017. DRIFT-FREE INDOOR NAVIGATION USING SIMULTA- NEOUS LOCALIZATION AND MAPPING OF THE AMBIENT HETEROGE- NEOUS MAGNETIC FIELD. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W7 (Sept. 2017), 339–344. doi...

    2017 doi

  5. [13]

    J. C. K. Chow, I. Detchev, K. D. Ang, K. Morin, K. Mahadevan, and N. Louie. 2018. ROBOT VISION: CALIBRATION OF WIDE-ANGLE LENS CAMERAS USING COLLINEARITY CONDITION AND K-NEAREST NEIGHBOUR REGRESSION. The International Archives of the Photogrammetry, Remote Sensing and Spatial ...

    2018 doi

  6. [14]

    Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv:1412.3555 [cs.NE] https://arxiv.org/abs/1412.3555

    2014 arXiv

  7. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

    2019 arXiv

  8. [16]

    Leo Feng, Frederick Tung, Hossein Hajimirsadeghi, Mohamed Osama Ahmed, Yoshua Bengio, and Greg Mori. 2024. Attention as an RNN. arXiv preprint arXiv:2405.13956 (2024)

    2024 arXiv

  9. [17]

    Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis- Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018. Speaker-Follower Models for Vision-and-Language Navigation. arXiv:1806.02724 [cs.CV] https://arxiv.org/abs...

    2018 arXiv

  10. [18]

    Muraleekrishna Gopinathan. 2025. Toward embodied navigation through vision and language. (2025)

    2025

  11. [19]

    Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. 2021. Airbert: In-domain Pretraining for Vision-and-Language Naviga- tion. arXiv:2108.09105 [cs.CV] https://arxiv.org/abs/2108.09105

    2021 arXiv

  12. [20]

    Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. 2020. Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training. arXiv:2002.10638 [cs.CV] https://arxiv.org/abs/2002.10638

    2020 arXiv

  13. [21]

    Yicong Hong, Cristian Rodriguez-Opazo, Yuankai Qi, Qi Wu, and Stephen Gould

  14. [22]

    Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazo, and Stephen Gould. 2021. A Recurrent Vision-and-Language BERT for Navigation. arXiv:2011.13922 [cs.CV] https://arxiv.org/abs/2011.13922

    2021 arXiv

  15. [23]

    Cong Hu, Xiao-Zhong Wei, and Xiao-Jun Wu. 2024. DIRformer: A Novel Image Restoration Approach Based on U-shaped Transformer and Diffusion Models. ACM Trans. Multimedia Comput. Commun. Appl. 21, 2, Article 57 (Dec. 2024), 23 pages. doi:10.1145/3703632

    2024 doi

  16. [24]

    Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, and Katerina Fragkiadaki

  17. [25]

    Himangshu Kalita, Steven Morad, and Jekan Thangavelautham. 2018. Path Planning and Navigation Inside Off-World Lava Tubes and Caves. arXiv:1803.02818 [cs.RO] https://arxiv.org/abs/1803.02818

    2018 arXiv

  18. [26]

    Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. 2020. Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding. arXiv:2010.07954 [cs.CV] https://arxiv.org/abs/2010. 07954

    2020 arXiv

  19. [27]

    Jiacheng Li, Yujie Lu, Jinpeng Wang, Yuanfang Guo, Weizhi Ma, Zhumin Chen, Jun Ma, and Peng Jiang. 2021. RecBERT: A Pre-trained Language Model for Sequential Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Informatio...

    2021

  20. [28]

    Xiangyang Li, Zihan Wang, Jiahao Yang, Yaowei Wang, and Shuqiang Jiang. 2023. KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation. arXiv:2303.15796 [cs.CV] https://arxiv.org/abs/2303.15796

    2023 arXiv

  21. [29]

    Bingqian Lin, Yunshuang Nie, Ziming Wei, Jiaqi Chen, Shikui Ma, Jianhua Han, Hang Xu, Xiaojun Chang, and Xiaodan Liang. 2025. NavCoT: Boosting LLM- Based Vision-and-Language Navigation via Learning Disentangled Reasoning. arXiv:2403.07376 [cs.CV] https://arxiv.org/abs/2403.07376

    2025 arXiv

  22. [30]

    Hao Liu, Yang Yang, Fumin Shen, Lixin Duan, and Heng Tao Shen. 2016. Recurrent Image Captioner: Describing Images with Spatial-Invariant Transformation and Attention Filtering. arXiv:1612.04949 [cs.CV] https://arxiv.org/abs/1612.04949

    2016 arXiv

  23. [31]

    Mingyan Liu. 2025. A Unified Virtual Mixture-of-Experts Frame- work:Enhanced Inference and Hallucination Mitigation in Single-Model System. arXiv:2504.03739 [cs.CL] https://arxiv.org/abs/2504.03739

    2025 arXiv

  24. [32]

    Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. 2023. Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions. arXiv:2309.11382 [cs.RO] https://arxiv.org/abs/2309.11382

    2023 arXiv

  25. [33]

    Abhinav Moudgil, Arjun Majumdar, Harsh Agrawal, Stefan Lee, and Dhruv Batra

  26. [34]

    Yuankai Qi, Zizheng Pan, Shengping Zhang, Anton van den Hengel, and Qi Wu. 2020. Object-and-Action Aware Model for Visual Language Navigation. arXiv:2007.14626 [cs.CL] https://arxiv.org/abs/2007.14626

    2020 arXiv

  27. [35]

    Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. 2020. REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments. arXiv:1904.10151 [cs.CV] https://arxiv.org/abs/1904.10151

    2020 arXiv

  28. [36]

    Jianing Qian, Anastasios Panagopoulos, and Dinesh Jayaraman. 2024. Recasting Generic Pretrained Vision Transformers As Object-Centric Scene Encoders For Manipulation Policies. arXiv:2405.15916 [cs.CV] https://arxiv.org/abs/2405.15916

    2024 arXiv

  29. [37]

    arXiv:2110.14143 [cs.CV] https://arxiv.org/abs/2110.14143

    SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation. arXiv:2110.14143 [cs.CV] https://arxiv.org/abs/2110.14143

    arXiv

  30. [38]

    Yanyuan Qiao, Yuankai Qi, Zheng Yu, Jing Liu, and Qi Wu. 2023. March in Chat: Interactive Prompting for Remote Embodied Referring Expression. arXiv:2308.10141 [cs.CV] https://arxiv.org/abs/2308.10141

    2023 arXiv

  31. [39]

    Yanyuan Qiao, Zheng Yu, and Qi Wu. 2023. VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigation. arXiv:2308.10172 [cs.CV] https://arxiv.org/abs/2308.10172

    2023 arXiv

  32. [40]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  33. [41]

    Yanyuan Qiao, Yuankai Qi, Yicong Hong, Zheng Yu, Peng Wang, and Qi Wu

  34. [42]

    arXiv:2203.11591 [cs.CV] https://arxiv.org/abs/2203.11591

    HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigation. arXiv:2203.11591 [cs.CV] https://arxiv.org/abs/2203.11591

    arXiv

  35. [43]

    Erick Schmidt, Zachary Ruble, David Akopian, and Daniel J. Pack. 2019. Software- Defined Radio GNSS Instrumentation for Spoofing Mitigation: A Review and a Case Study. IEEE Transactions on Instrumentation and Measurement 68, 8 (Aug. 2019), 2768–2784. doi:10.1109/tim.2018.2869261

    2019

  36. [44]

    Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, and Kurt Keutzer. 2021. How Much Can CLIP Benefit Vision- and-Language Tasks? arXiv:2107.06383 [cs.CV] https://arxiv.org/abs/2107.06383

    2021 arXiv

  37. [45]

    Chenglu Sun, Shuo Shen, Wenzhi Tao, Deyi Xue, and Zixia Zhou. 2025. Noise- Resilient Symbolic Regression with Dynamic Gating Reinforcement Learning. arXiv preprint arXiv:2501.01085 (2025). Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for...

    2025 arXiv

  38. [46]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.000...

    2021 arXiv

  39. [47]

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster R- CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv:1506.01497 [cs.CV] https://arxiv.org/abs/1506.01497

    2016 arXiv

  40. [48]

    Hao Tan and Mohit Bansal. 2019. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 (2019)

    2019 arXiv

  41. [49]

    Hao Tan, Licheng Yu, and Mohit Bansal. 2019. Learning to Navi- gate Unseen Environments: Back Translation with Environmental Dropout. arXiv:1904.04195 [cs.CL] https://arxiv.org/abs/1904.04195

    2019 arXiv

  42. [50]

    Hanqing Wang, Wenguan Wang, Wei Liang, Caiming Xiong, and Jianbing Shen. 2021. Structured Scene Memory for Vision-Language Navigation. arXiv:2103.03454 [cs.CV] https://arxiv.org/abs/2103.03454

    2021 arXiv

  43. [51]

    Chiranjib Sur. 2019. CRUR: Coupled-Recurrent Unit for Unification, Conceptual- ization and Context Capture for Language Representation – A Generalization of Bi Directional LSTM. arXiv:1911.10132 [cs.CL] https://arxiv.org/abs/1911.10132

    2019 arXiv

  44. [53]

    Zun Wang, Jialu Li, Yicong Hong, Yi Wang, Qi Wu, Mohit Bansal, Stephen Gould, Hao Tan, and Yu Qiao. 2023. Scaling Data Generation in Vision-and-Language Navigation. arXiv:2307.15644 [cs.CV] https://arxiv.org/abs/2307.15644

    2023 arXiv

  45. [54]

    Qi Wu, Chunhua Shen, Anton van den Hengel, Peng Wang, and Anthony Dick

  46. [55]

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. 2022. SimMIM: A Simple Framework for Masked Image Modeling. arXiv:2111.09886 [cs.CV] https://arxiv.org/abs/2111.09886

    2022 arXiv

  47. [56]

    Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan- Fang Wang, William Yang Wang, and Lei Zhang. 2019. Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Naviga- tion. arXiv:1811.10092 [cs.CV] https://arxiv.org/a...

    2019 arXiv

  48. [57]

    Xiaohan Wang, Wenguan Wang, Jiayi Shao, and Yi Yang. 2023. Lana: A Language-Capable Navigator for Instruction Following and Generation. arXiv:2303.08409 [cs.CV] https://arxiv.org/abs/2303.08409

    2023 arXiv

  49. [58]

    Arthur Zhang, Harshit Sikchi, Amy Zhang, and Joydeep Biswas. 2025. CREStE: Scalable Mapless Navigation with Internet Scale Priors and Counterfactual Guid- ance. arXiv:2503.03921 [cs.RO] https://arxiv.org/abs/2503.03921

    2025 arXiv

  50. [59]

    Chongyang Zhao, Yuankai Qi, and Qi Wu. 2023. Mind the Gap: Improving Success Rate of Vision-and-Language Navigation by Revisiting Oracle Success Routes. arXiv:2308.03244 [cs.CV] https://arxiv.org/abs/2308.03244

    2023 arXiv

  51. [60]

    Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. 2024. Towards Learning a Generalist Model for Embodied Navigation. arXiv:2312.02010 [cs.CV] https://arxiv.org/abs/2312.02010

    2024 arXiv

  52. [61]

    Gengze Zhou, Yicong Hong, and Qi Wu. 2023. NavGPT: Explicit Rea- soning in Vision-and-Language Navigation with Large Language Models. arXiv:2305.16986 [cs.CV] https://arxiv.org/abs/2305.16986

    2023 arXiv

  53. [62]

    Shiyang Yan, Jun Xu, Yuai Liu, and Lin Xu. 2019. HorNet: A Hierarchical Off- shoot Recurrent Network for Improving Person Re-ID via Image Captioning. arXiv:1908.04915 [cs.CV] https://arxiv.org/abs/1908.04915

    2019 arXiv

  54. [63]

    Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Jichao Zhu, Lingyan Zhang, Gangyong Jia, Qing Wu, Xiang Wan, Lihua Li, and Changmiao Wang. 2025. ICH-PRNet: a cross-modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism. Neural Networks 18...

    2025

  55. [64]

    Fengda Zhu, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. 2020. Vision- Language Navigation with Self-Supervised Auxiliary Reasoning Tasks. arXiv:1911.07883 [cs.CV] https://arxiv.org/abs/1911.07883

    2020 arXiv

  56. [65]

    Junyou Zhu, Yanyuan Qiao, Siqi Zhang, Xingjian He, Qi Wu, and Jing Liu. 2024. MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation. arXiv:2409.18800 [cs.CV] https://arxiv.org/abs/2409.18800

    2024 arXiv

  57. [66]

    Wanrong Zhu, Yuankai Qi, Pradyumna Narayana, Kazoo Sone, Sugato Basu, Xin Eric Wang, Qi Wu, Miguel Eckstein, and William Yang Wang

  58. [68]

    Xinzhe Zhou, Wei Liu, and Yadong Mu. 2021. Rethinking the Spatial Route Prior in Vision-and-Language Navigation. arXiv:2110.05728 [cs.CV] https://arxiv.org/ abs/2110.05728

    2021 arXiv

  59. [69]

    Fengda Zhu, Xiwen Liang, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. 2021. SOON: Scenario Oriented Object Navigation with Graph-based Exploration. arXiv:2103.17138 [cs.CV] https://arxiv.org/abs/2103.17138

    2021 arXiv

  60. [73]

    arXiv:2103.16561 [cs.CV] https://arxiv.org/abs/2103.16561

    Diagnosing Vision-and-Language Navigation: What Really Matters. arXiv:2103.16561 [cs.CV] https://arxiv.org/abs/2103.16561

    arXiv

  61. [2016]

    arXiv:1603.02814 [cs.CV] https://arxiv.org/abs/1603.02814

    Image Captioning and Visual Question Answering Based on Attributes and External Knowledge. arXiv:1603.02814 [cs.CV] https://arxiv.org/abs/1603.02814

    arXiv

  62. [2020]

    arXiv:2010.09304 [cs.CV] https://arxiv.org/abs/2010.09304

    Language and Visual Entity Relationship Graph for Agent Navigation. arXiv:2010.09304 [cs.CV] https://arxiv.org/abs/2010.09304

    2010 arXiv

  63. [2021]

    arXiv:2107.07201 [cs.CV] https://arxiv.org/abs/2107.07201

    Neighbor-view Enhanced Model for Vision and Language Navigation. arXiv:2107.07201 [cs.CV] https://arxiv.org/abs/2107.07201

    arXiv

  64. [2022]

    In European Conference on Computer Vision

    Bottom up top down detection transformers for language grounding in images and point clouds. In European Conference on Computer Vision . Springer, 417–433

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.