Pith. sign in

REVIEW 4 major objections 4 minor 54 references

DOPE: Dual Object Perception-Enhancement Network for Vision-and-Language Navigation

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DOPE claims that explicit object and action phrase extraction from instructions, combined with cross-modal image-text object modeling, improves navigational success on R2R and REVERIE beyond listed baselines.

desk verdict Competent incremental VLN paper with plausible gains; the R2R object-feature source is unspecified, which is the main thing to fix before trusting the numbers. read the letter →

arxiv 2505.00743 v1 pith:45S5JGN3 submitted 2025-04-30 cs.CV cs.RO

Yinfeng Yu , Dongsheng Yang This is my paper · ORCID
classification cs.CVcs.RO
keywords Vision-and-LanguageNavigationobjectperceptioncross-modalattentioninstructionunderstandingDUETbaselineR2RdatasetREVERIEtargetgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-and-language navigation asks an agent to follow an instruction through an unseen indoor environment. This paper argues that two kinds of object-level information are underused: the nouns and verbs inside the instruction, and the relations between objects named in text and objects visible in a panorama. On top of the DUET navigation model, DOPE adds a parser that pulls out action and object phrases, a text module that re-weights instruction features by multi-head attention with a gate, and an image module that runs object features through a cross-modal encoder and another attention gate. The paper reports that the full model reaches a 74 percent success rate on R2R's unseen test split and 58.38 percent on REVERIE's unseen test split, beating every method it compares against. The reason to care is that object-level grounding in both language and vision is a concrete, testable route to better instruction following and target localization.

What carries the argument

The load-bearing mechanism is a pair of attention-and-gate feature refineries: one for language, one for vision. For text, a parser produces object and action phrase embeddings that act as memory for multi-head attention over the full instruction, and a sigmoid gate mixes the attended features with the original instruction. For vision, object-region features are added to the panoramic view, jointly self-attended, passed through a cross-modal encoder together with the object phrase embeddings, and combined by another gate. These refineries insert explicit object-level cues into DUET's coarse-scale cross-modal encoder, which produces the global and local action scores that are weighted together to choose the next step. The common ingredient that does the work is the attention update followed by a learned gate, applied symmetrically to text and image streams.

What would settle it

Re-run DOPE on R2R with the image-object branch given random crops instead of real object regions; if the test-unseen success rate stays near 74, the reported gains are not caused by object perception. Equally decisive: require the authors to name the object detector used for R2R and confirm that its outputs reproduce the published numbers.

Watch

Extended reading notes

Core claim

The central claim, stated as the authors would state it, is that a navigation agent can navigate more accurately when it explicitly perceives objects on both sides of the language-vision pair. DOPE's TSE module extracts action verbs and object nouns from the instruction; TOPA uses multi-head attention to relate those extracted phrases to the whole instruction and a sigmoid gate to keep a balance between original and enhanced text features; IOPA encodes object regions from the panorama, models their relations to language object phrases through a cross-modal encoder, and gates the result into the image stream. The resulting object-enhanced text and object-enhanced image features are fused into the DUET action scorer. The paper's evidence is the comparison tables: on R2R test-unseen, DOPE reaches SR 74 and SPL 63, and on REVERIE test-unseen it reaches SR 58.38, OSR 63.10, and RGSPL 25.43, all above the listed baselines. The ablations show that removing either module hurts and that removing the shared attention-gate component (OPE) drops the combined model's SR from 51.72 to 47.37 on REVERIE val-unseen.

Load-bearing premise

The method assumes the model always has access to a list of objects in the current panoramic view, but the R2R benchmark does not provide such a list and the paper does not say where it comes from.

Editorial extensions

If this is right

  • On R2R unseen test split, DOPE reaches 74 SR and 63 SPL, a 5-point SR gain over the DUET baseline's 69 and a 1-point SR gain over the strongest listed prior method (BEVBert at 73).
  • On REVERIE unseen test split, DOPE reaches 58.38 SR and 25.43 RGSPL, the highest numbers among the listed methods, with a 4.41 SR gain over ACK.
  • Ablation results on REVERIE val-unseen show that each module contributes: TOPA alone raises SR from 46.98 to 49.47, IOPA alone to 50.01, and both together to 51.72.
  • The shared attention-gate (OPE) component is load-bearing: removing it from the full model drops SR from 51.72 to 47.37 and RGSPL from 24.97 to 22.16.
  • Tuning dropout to 0.7 improves all metrics on REVERIE val-unseen, so part of the reported gain is regularization of a small dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper leaves implicit is that the same two-stream object refinement could be attached to other VLN baselines, not just DUET, so the mechanism's generality is testable by re-running an existing model with these modules.
  • Because R2R provides no predefined object boxes, the reported R2R gains depend on an unnamed object extraction step; a natural test is to substitute an off-the-shelf detector and check whether the numbers reproduce.
  • The gating pattern that mixes original features with object-enhanced features could also be applied in multimodal tasks outside navigation, such as visual question answering, where sparse salient tokens are easily diluted by long contexts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DOPE, an extension of the DUET baseline for vision-and-language navigation. DOPE adds three modules: Text Semantic Extraction (TSE), which uses spaCy POS tagging and a hand-defined action vocabulary to extract object and action phrases; Text Object Perception-Augmentation (TOPA), which feeds these phrases through BERT with a multi-head attention enhancement and gated fusion; and Image Object Perception-Augmentation (IOPA), which uses CLIP object features, self-attention, and an LXMERT cross-modal encoder to model object relationships. The authors evaluate on R2R and REVERIE, reporting state-of-the-art or competitive results: R2R test-unseen SR 74 vs. 73 for BEVBert and +5 OSR, +5 SR, +4 SPL over DUET; REVERIE test-unseen SR 58.38 vs. 52.81 for BEVBert and +4.41 SR over ACK. Ablations on the REVERIE unseen validation set attribute gains to the TOPA and IOPA modules and to the OPE submodule.

Significance. If the results hold, DOPE is a modest but useful incremental contribution to VLN. The idea of separately modeling object and action phrases in instructions and object-level visual features is reasonable and the modular design is clearly described at a high level. The paper's strengths include experiments on two standard benchmarks, ablations that separate the contributions of TSE+TOPA, IOPA, and OPE, a dropout-rate analysis, and qualitative navigation examples. However, the paper lacks code release and statistical confidence: all numbers are single-run point estimates, and the R2R results are not reproducible as written because the source of object features for R2R is never specified. The core architecture is plausible, but the missing implementation details and lack of variance reporting prevent the claimed SOTA improvements from being verified.

major comments (4)
  1. [§3.4 and §4.1/4.3] The IOPA module (Eqs. 6–8) requires a set of object features O_t for every panoramic view, but the paper never states how these are obtained for the R2R dataset. Section 4.1 says only that REVERIE provides predefined object bounding boxes, and Section 4.3 says 'Object bounding boxes are provided in the REVERIE dataset, and we also use CLIP for feature extraction.' The R2R evaluation in Table 1 therefore either uses an undisclosed detector or region proposal mechanism, or it does not use IOPA at all. If the former, the detector architecture, pretraining data, and thresholds are essential implementation details that can materially affect navigation accuracy and must be reported; if the latter, the reported R2R gains cannot be attributed to the proposed object-perception modules. This is a load-bearing reproducibility gap in the paper's central claim of superiority on R2R.
  2. [§3.2.1] The Text Semantic Extraction module depends on an 'action vocabulary relevant to the navigation task' and on normalization of object nouns, but the vocabulary list and exact extraction rules are not provided. Only word clouds in Figure 3 illustrate the categories. Since TSE and TOPA are two of the three proposed contributions, the parser must be specified completely—the action vocabulary, the POS-filtering rule, the regular expression cleaning, and the lemmatization procedure—for the method to be independently instantiated and compared.
  3. [§3.4.2] The notation and data flow in IOPA are ambiguous enough to prevent a reader from reconstructing the architecture. The symbol O_t is reused for both the raw CLIP object features and the LXMERT-integrated object features; the relation among [R'_t, O'_t], O_t, and f_t = {R_hat_t, O_hat_t} is unclear; and the text does not say which features are the query, key, and value in the final MHA that produces the object-enhanced image features. The two types of positional embeddings (Section 3.4.1) are described verbally but not defined in equations. A precise specification of tensor shapes and module connections is needed here.
  4. [§4.6 and Tables 1–2] The dropout rate is tuned on the REVERIE unseen validation split (Figure 6), and all reported results are single-run point estimates without variance, confidence intervals, or significance tests. Given that the R2R test-unseen improvement over BEVBert is one SR point (74 vs. 73), and several REVERIE improvements are a few points, the claim of 'superior navigation performance' is not statistically supported. The authors should either report multiple seeds with standard deviations, or explicitly acknowledge the uncertainty in these comparisons.
minor comments (4)
  1. [§4.6] The title 'Quantitative Results' is misleading; this subsection is a dropout hyperparameter study. A title such as 'Hyperparameter Analysis' would reflect the content.
  2. [Table 4] The ablation shows that the full model without OPE (row 'ALL w/o OPE') has SR 47.37, which is lower than either TSE+TOPA w/o OPE (49.28) or IOPA w/o OPE (49.73). This non-monotonicity is surprising and should be discussed; otherwise the reader cannot tell whether the OPE module is consistently beneficial or whether interactions between modules are unstable.
  3. [References] Several references are incompletely formatted, e.g., Refs. [23] and [32] list only surnames ('I Loshchilov', 'V Sanh') rather than full author names. The reference list should be brought into a consistent style.
  4. [§3.4] The phrase 'for the input panoramic images and image objects, we use CLIP to extract their features' appears before the formal definition of O_t in Eq. (6); clarifying that O_t denotes object-image features would help the reader.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DOPE's claims are empirical benchmark comparisons against external baselines; the missing R2R object-feature source is a reproducibility gap, not a circular reduction.

full rationale

The paper's central claim is that DOPE outperforms existing methods on R2R and REVERIE. This claim is supported by comparisons in Tables 1 and 2 against external baselines (DUET, BEVBert, ACK, etc.) and by ablations in Tables 3 and 4 that compare the full model against the DUET baseline with modules removed. No equation in the paper defines the output metric in terms of a fitted parameter, and no prediction is obtained by re-inserting a fitted value as if it were a forecast. The TSE module uses an action vocabulary and spaCy POS tagging; TOPA and IOPA use BERT, CLIP, and LXMERT, all external off-the-shelf components with stated provenance. The gate functions in Eqs. (4)-(5) and the attention in Eqs. (6)-(8) are architectural operations whose parameters are learned, not quantities that are equivalent to the reported success metrics by construction. The authors' own prior work appears only in audio-visual navigation references ([43]-[46]) and is not load-bearing for the dual-object formulation or for the reported results. The dropout rate selection in Section 4.6 is ordinary hyperparameter tuning on a validation split, not a fitted quantity renamed as a prediction. The most serious flaw is that Section 3.4 requires image object features O_t for every panoramic view, while Section 4.1 only mentions predefined object bounding boxes for REVERIE; no object detector or box source is specified for R2R. This is a reproducibility/completeness defect that could affect whether the R2R numbers are attributable to the proposed modules, but it is not circularity: nothing in the paper defines the R2R result as following tautologically from an unstated box source. Under the required standard, where circularity must be demonstrated by quoting equations or self-citation chains that reduce a claimed result to its own inputs, no such reduction exists here. The appropriate finding is no significant circularity with a score of 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard VLN domain assumptions, on an unstated but load-bearing availability of object features for R2R, and on a hand-built parser vocabulary that is never quantified. The only explicit numeric hyperparameter chosen for the headline results is the dropout rate. No speculative scientific entities are introduced.

free parameters (1)
  • Dropout rate = 0.7
    Selected by sweeping five values on the REVERIE unseen validation subset (Figure 6). The final reported model uses this value, so the headline numbers are conditional on a validation-tuned hyperparameter without a separate confirmatory run.
assumptions (4)
  • domain assumption The task is accurately modeled as discrete graph navigation with GPS, an RGB camera, and panoramic views.
    This is inherited from the R2R/REVERIE setup and is used without justification in §3. It rules out continuous control and low-level obstacle avoidance.
  • domain assumption Object-level image features can be obtained for every panoramic view in both R2R and REVERIE.
    The IOPA module consumes object features, and REVERIE provides predefined bounding boxes, but R2R does not. The paper does not name a detector or proposal mechanism for R2R, so this assumption is load-bearing and unstated.
  • ad hoc to paper A hand-defined action vocabulary plus spaCy POS tagging is sufficient to extract all task-relevant action verbs and object nouns.
    The Object and Action Parser in §3.2.1 defines an action vocabulary "relevant to the navigation task" without listing it or measuring its coverage. The parser is a central input to TOPA and IOPA.
  • domain assumption Pretrained CLIP and LXMERT features capture cross-modal object correspondences well enough to improve downstream navigation decisions.
    The method relies on CLIP for visual features and LXMERT as the cross-modal encoder (§3.4, §4.3). No experiment isolates whether any gain comes from these pretrained feature extractors rather than from the new modules.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DOPE: Dual Object Perception-Enhancement Network for Vision-and-Language Navigation." pith.science (2026). https://pith.science/paper/45S5JGN3

@misc{pith2026250500743,
  author       = {Pith},
  title        = {Pith review of: DOPE: Dual Object Perception-Enhancement Network for Vision-and-Language Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/45S5JGN3}},
  note         = {Machine review of arXiv:2505.00743}
}
read the original abstract

Vision-and-Language Navigation (VLN) is a challenging task where an agent must understand language instructions and navigate unfamiliar environments using visual cues. The agent must accurately locate the target based on visual information from the environment and complete tasks through interaction with the surroundings. Despite significant advancements in this field, two major limitations persist: (1) Many existing methods input complete language instructions directly into multi-layer Transformer networks without fully exploiting the detailed information within the instructions, thereby limiting the agent's language understanding capabilities during task execution; (2) Current approaches often overlook the modeling of object relationships across different modalities, failing to effectively utilize latent clues between objects, which affects the accuracy and robustness of navigation decisions. We propose a Dual Object Perception-Enhancement Network (DOPE) to address these issues to improve navigation performance. First, we design a Text Semantic Extraction (TSE) to extract relatively essential phrases from the text and input them into the Text Object Perception-Augmentation (TOPA) to fully leverage details such as objects and actions within the instructions. Second, we introduce an Image Object Perception-Augmentation (IOPA), which performs additional modeling of object information across different modalities, enabling the model to more effectively utilize latent clues between objects in images and text, enhancing decision-making accuracy. Extensive experiments on the R2R and REVERIE datasets validate the efficacy of the proposed approach.

Figures

Figures reproduced from arXiv: 2505.00743 by the authors.

Figure 1
Figure 1. Illustration of the differences between DOPE and other methods. (a) Conventional methods, (b) Our method. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed Dual Object Perception-Enhancement Network (DOPE), which primarily comprises three [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. (a) Action Word Cloud and (b) Object Word Cloud. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Illustration of the Object Perception-Enhancement [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Illustration of cross-modal object modeling. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Explanation of the impact of different Dropout rates [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Visualization of navigation examples. The red font and red boxes represent object information. (a) Ground truth [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 35 canonical work pages

  1. [1]

    Dong An, Yuankai Qi, Yangguang Li, Yan Huang, Liang Wang, Tieniu Tan, and Jing Shao. 2023. Bevbert: Multimodal map pre-training for language-guided DOPE: Dual Object Perception-Enhancement Network for Vision-and-Language Navigation ICMR ’25, June 30-July 3, 2025, Chicago, IL, USA navigation. In Proceedings of the IEEE/CVF International Conference on Compu...

    work page 2023

  2. [2]

    Dong An, Hanqing Wang, Wenguan Wang, Zun Wang, Yan Huang, Keji He, and Liang Wang. 2024. Etpnav: Evolving topological planning for vision-language navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    work page 2024

  3. [3]

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünder- hauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. 2018. Vision-and- language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3674–3683

    2018

  4. [4]

    Jinyu Chen, Chen Gao, Erli Meng, Qiong Zhang, and Si Liu. 2022. Reinforced structured state-evolution for vision-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15450–15459

    2022

  5. [5]

    Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. 2021. History aware multimodal transformer for vision-and-language navigation. Advances in neural information processing systems 34 (2021), 5834–5847

    work page 2021

  6. [6]

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. 2022. Think global, act local: Dual-scale graph transformer for vision- and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16537–16547

    work page 2022

  7. [7]

    Samyak Datta, Sameer Dharur, Vincent Cartillier, Ruta Desai, Mukul Khanna, Dhruv Batra, and Devi Parikh. 2022. Episodic memory question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 19119–19128

    work page 2022

  8. [8]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

Show all 54 references
  1. [9]

    Heming Du, Xin Yu, and Liang Zheng. 2020. Learning object relation graph and tentative policy for visual navigation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16 . Springer, 19–34

    2020

  2. [10]

    Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis- Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018. Speaker-follower models for vision-and-language navigation. Advances in neural information processing syst...

    2018

  3. [11]

    Chen Gao, Jinyu Chen, Si Liu, Luting Wang, Qiong Zhang, and Qi Wu. 2021. Room-and-object aware knowledge reasoning for remote embodied referring expression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3064–3073

    2021

  4. [12]

    Chen Gao, Xingyu Peng, Mi Yan, He Wang, Lirong Yang, Haibing Ren, Hongsheng Li, and Si Liu. 2023. Adaptive zone-aware hierarchical planner for vision-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14911–14920

    2023

  5. [13]

    Fang Gao, Jingfeng Tang, Jiabao Wang, Shaodong Li, and Jun Yu. 2024. Enhanc- ing Scene Understanding for Vision-andLanguage Navigation by Knowledge Awareness. IEEE Robotics and Automation Letters (2024)

    2024

  6. [14]

    Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. 2021. Airbert: In-domain pretraining for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 1634– 1643

    2021

  7. [15]

    Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. 2017. Cognitive mapping and planning for visual navigation. InProceedings of the IEEE conference on computer vision and pattern recognition . 2616–2625

    2017

  8. [16]

    Keji He, Chenyang Si, Zhihe Lu, Yan Huang, Liang Wang, and Xinchao Wang. 2024. Frequency-enhanced data augmentation for vision-and-language navigation. Advances in Neural Information Processing Systems 36 (2024)

    2024

  9. [17]

    Jialu Li, Hao Tan, and Mohit Bansal. 2022. Envedit: Environment editing for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15407–15417

    2022

  10. [18]

    Xiangyang Li, Zihan Wang, Jiahao Yang, Yaowei Wang, and Shuqiang Jiang. 2023. Kerm: Knowledge enhanced reasoning for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2583–2592

    2023

  11. [19]

    Bingqian Lin, Yunshuang Nie, Ziming Wei, Yi Zhu, Hang Xu, Shikui Ma, Jianzhuang Liu, and Xiaodan Liang. 2024. Correctable Landmark Discovery Via Large Models for Vision-Language Navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  12. [20]

    Kunyang Lin, Peihao Chen, Diwei Huang, Thomas H Li, Mingkui Tan, and Chuang Gan. 2023. Learning vision-and-language navigation from youtube videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8317–8326

    2023

  13. [21]

    Xiangru Lin, Guanbin Li, and Yizhou Yu. 2021. Scene-intuitive agent for re- mote embodied visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 7036–7045

    2021

  14. [22]

    Rui Liu, Xiaohan Wang, Wenguan Wang, and Yi Yang. 2023. Bird’s-Eye-View Scene Graph for Vision-Language Navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 10968–10980

    2023

  15. [23]

    I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  16. [24]

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Vilbert: Pretrain- ing task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems 32 (2019)

    2019

  17. [25]

    Bahram Mohammadi, Yicong Hong, Yuankai Qi, Qi Wu, Shirui Pan, and Javen Qin- feng Shi. 2024. Augmented Commonsense Knowledge for Remote Object Ground- ing. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 4269– 4277

    2024

  18. [26]

    Yuankai Qi, Zizheng Pan, Yicong Hong, Ming-Hsuan Yang, Anton Van Den Hen- gel, and Qi Wu. 2021. The road to know-where: An object-and-room informed sequential bert for indoor vision-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision...

    2021

  19. [27]

    Yuankai Qi, Zizheng Pan, Shengping Zhang, Anton van den Hengel, and Qi Wu

  20. [28]

    Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. 2020. Reverie: Remote embodied visual referring expression in real indoor environments. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ....

    2020

  21. [29]

    Yanyuan Qiao, Yuankai Qi, Yicong Hong, Zheng Yu, Peng Wang, and Qi Wu

  22. [30]

    Yanyuan Qiao, Yuankai Qi, Yicong Hong, Zheng Yu, Peng Wang, and Qi Wu. 2023. Hop+: History-enhanced and order-aware pre-training for vision-and-language navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 7 (2023), 8524–8537

    2023

  23. [31]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  24. [32]

    V Sanh. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    2019 arXiv

  25. [33]

    Raphael Schumann, Wanrong Zhu, Weixi Feng, Tsu-Jui Fu, Stefan Riezler, and William Yang Wang. 2024. Velma: Verbalization embodiment of llm agents for vision and language navigation in street view. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18924–18933

    2024

  26. [34]

    Hao Tan and Mohit Bansal. 2019. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 (2019)

    2019 arXiv

  27. [35]

    Hao Tan, Licheng Yu, and Mohit Bansal. 2019. Learning to navigate unseen environments: Back translation with environmental dropout. arXiv preprint arXiv:1904.04195 (2019)

    2019 arXiv

  28. [36]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  29. [37]

    Liuyi Wang, Zongtao He, Ronghao Dang, Huiyi Chen, Chengju Liu, and Qijun Chen. 2023. Res-sts: Referring expression speaker via self-training with scorer for goal-oriented vision-language navigation. IEEE Transactions on Circuits and Systems for Video Technology 33, 7 (2023), 3441–3454

    2023

  30. [38]

    Xiaohan Wang, Wenguan Wang, Jiayi Shao, and Yi Yang. 2023. Lana: A language- capable navigator for instruction following and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 19048–19058

    2023

  31. [39]

    Zihan Wang, Xiangyang Li, Jiahao Yang, Yeqi Liu, and Shuqiang Jiang. 2023. Gridmm: Grid memory map for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 15625–15636

    2023

  32. [40]

    Shuhuan Wen, Simeng Gong, Ziyuan Zhang, F Richard Yu, and Zhiwen Wang

  33. [41]

    Jie Wu, Chunlei Wu, Xiuxuan Shen, and Leiquan Wang. 2025. Adaptive Cross- Modal Experts Network with Uncertainty-Driven Fusion for Vision–Language Navigation. Knowledge-Based Systems 307 (2025), 112735

    2025

  34. [42]

    Siying Wu, Xueyang Fu, Feng Wu, and Zheng-Jun Zha. 2024. Vision-and- Language Navigation via Latent Semantic Alignment Learning.IEEE Transactions on Multimedia (2024)

    2024

  35. [43]

    Yinfeng Yu, Lele Cao, Fuchun Sun, Xiaohong Liu, and Liejun Wang. 2022. Pay Self- Attention to Audio-Visual Navigation. In 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21-24, 2022 . BMVA Press, 46

    2022

  36. [44]

    Yinfeng Yu, Lele Cao, Fuchun Sun, Chao Yang, Huicheng Lai, and Wenbing Huang

  37. [45]

    Yinfeng Yu, Changan Chen, Lele Cao, Fangkai Yang, Wenbing Huang, and Fuchun Sun. 2023. Measuring Acoustics with Collaborative Multiple Agents. In The 32nd International Joint Conference on Artificial Intelligence, IJCAI 2023, Macao, 19th- 25th August 2023

    2023

  38. [46]

    Yinfeng Yu, Wenbing Huang, Fuchun Sun, Changan Chen, Yikai Wang, and Xiaohong Liu. 2022. Sound Adversarial Audio-Visual Navigation. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, ICMR ’25, June 30-July 3, 2025, Chicago, IL, USA Yin...

    2022

  39. [47]

    Gengze Zhou, Yicong Hong, Zun Wang, Xin Eric Wang, and Qi Wu. 2024. Navgpt- 2: Unleashing navigational reasoning capability for large vision-language models. In European Conference on Computer Vision . Springer, 260–278

    2024

  40. [48]

    Fengda Zhu, Xiwen Liang, Yi Zhu, Qizhi Yu, Xiaojun Chang, and Xiaodan Liang

  41. [49]

    Fengda Zhu, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. 2020. Vision-language navigation with self-supervised auxiliary reasoning tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 10012–10022

    2020

  42. [2020]

    InEuropean Conference on Computer Vision

    Object-and-action aware model for visual language navigation. InEuropean Conference on Computer Vision . Springer, 303–317

  43. [2021]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Soon: Scenario oriented object navigation with graph-based exploration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12689–12699

  44. [2022]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Hop: History-and-order aware pre-training for vision-and-language navi- gation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15418–15427

  45. [2023]

    Neural Computation 35, 5 (2023), 958–976

    Echo-Enhanced Embodied Visual Navigation. Neural Computation 35, 5 (2023), 958–976

    2023

  46. [2024]

    Knowledge-Based Systems 305 (2024), 112610

    Vision-and-language navigation based on history-aware cross-modal feature fusion in indoor environment. Knowledge-Based Systems 305 (2024), 112610

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.