Pith. sign in

REVIEW 1 major objections 1 minor 2 cited by

GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance

T0 review · 1 major / 1 minor · reviewed 2026-05-22 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗

Pith's one-line read A new dataset of real-world scenes shows current multimodal models struggle with depth perception for blind navigation guidance.

desk verdict GuideDog delivers a large, multi-country egocentric dataset grounded in BLV standards, but the model performance claims rest on thin evidence. read the letter →

arxiv 2503.12844 v2 submitted 2025-03-17 cs.CV

Junhyeok Kim , Jaewoo Park , Junhee Park , Sangeyl Lee , Jiwan Chung , JiSung Kim , Ji Hoon Joung , Youngjae Yu This is my paper
classification cs.CV
keywords egocentricdatasetblindlowvisionaccessibilityguidancemultimodalmodelsdepthperceptionhuman-AIannotationpedestrianscenesnavigationassistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents GuideDog, a collection of 22,000 egocentric image-description pairs from pedestrian scenes across 46 countries, built with a human-AI pipeline that verifies outputs against established blind and low-vision guidance standards instead of starting from scratch. This tackles the prior bottleneck of labor-intensive expert annotation that limited progress on assistive navigation tools. The accompanying GuideDogQA benchmark with 818 samples tests object recognition and depth perception specifically. Experiments on the benchmark indicate that existing multimodal large language models still fall short on depth cues and on following the accessibility standards. A reader would care because reliable AI guidance could support safer independent movement for the billions affected by vision loss.

What carries the argument

The human-AI annotation pipeline that moves the task from full generation to verification against established blind and low-vision guidance standards.

What would settle it

Independent expert raters score a random sample of the dataset descriptions for adherence to the guidance standards and find large systematic deviations from expert-level quality.

Watch

Extended reading notes

Core claim

The central claim is that GuideDog supplies a scalable, standards-grounded dataset of 22K image-description pairs plus an 818-sample QA benchmark, and that testing current multimodal large language models on this benchmark reveals persistent difficulties with depth perception and with producing descriptions that adhere to blind and low-vision guidance standards.

Load-bearing premise

The human-AI pipeline yields descriptions that match the quality and fidelity of full expert annotation when judged against blind and low-vision guidance standards.

Editorial extensions

If this is right

  • Models can now be benchmarked on realistic pedestrian scenes for object recognition and depth perception in accessibility contexts.
  • The verification-based pipeline reduces the cost of creating further accessibility-aware datasets while preserving standard compliance.
  • Persistent model failures on depth perception point to concrete targets for improvement in multimodal systems used for navigation.
  • The 2K human-verified subset provides a high-quality seed for training or fine-tuning assistive applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same verification approach could extend to other domains where expert standards must be followed at scale, such as medical imaging or legal document review.
  • If models improve on this benchmark, real-time mobile apps could translate the descriptions into audio cues that help users avoid obstacles more effectively.
  • Dataset creators in related fields might adopt similar hybrid pipelines to incorporate domain standards without full expert labor.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper introduces GuideDog, a dataset of 22K egocentric image-description pairs (2K human-verified) from real-world pedestrian scenes across 46 countries, created via a human-AI annotation pipeline grounded in established BLV guidance standards. It also presents the GuideDogQA benchmark consisting of 818 samples to evaluate MLLMs on object recognition and depth perception, with the central claim that experiments show depth perception and adherence to these standards remain challenging for current MLLMs.

Significance. If the human-AI pipeline produces annotations of quality comparable to full expert annotation and the 818-sample benchmark is constructed rigorously, the dataset would provide a valuable, scalable resource for developing accessibility-aware navigation systems for BLV users, addressing the scarcity of real-world egocentric data in this domain.

major comments (1)
  1. [Abstract] Abstract: The statement that 'experiments reveal that depth perception and adherence to these standards remain challenging for current MLLMs' is not accompanied by any quantitative results, performance metrics, error analysis, or details on the construction, sampling method, or composition of the 818-sample GuideDogQA benchmark, leaving the central empirical claim only moderately supported.
minor comments (1)
  1. [Abstract] Abstract: Clarify whether the 2K human-verified pairs are a subset of the 22K or represent an additional verification step applied to the full set.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive comment on the abstract. We address the concern point-by-point below and will incorporate changes in the revised manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The statement that 'experiments reveal that depth perception and adherence to these standards remain challenging for current MLLMs' is not accompanied by any quantitative results, performance metrics, error analysis, or details on the construction, sampling method, or composition of the 818-sample GuideDogQA benchmark, leaving the central empirical claim only moderately supported.

    Authors: We agree that the abstract would be strengthened by including brief quantitative support for the central claim. The full details on GuideDogQA construction (including sampling method and composition), performance metrics, and error analysis appear in Sections 3.3 and 4 of the manuscript. In revision we will add concise quantitative highlights to the abstract (e.g., key accuracy figures on depth-perception and standard-adherence tasks across evaluated MLLMs) while preserving length constraints. This directly addresses the concern without altering the underlying experimental results. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper is an empirical dataset release introducing GuideDog (22K image-description pairs via human-AI pipeline grounded in external BLV expert standards) and GuideDogQA benchmark (818 samples). No mathematical derivations, equations, parameter fitting, or predictions exist. Claims about MLLM performance on depth perception and standards adherence are direct empirical observations on the benchmark, not reductions to fitted inputs or self-citations. The annotation process is presented as shifting to verification with 2K human-verified pairs, without self-definitional loops or ansatzes smuggled via citation. The work is self-contained against external benchmarks and standards.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claims rest on the assumption that established BLV guidance standards exist and can be reliably applied by the described pipeline; no free parameters or invented entities are introduced.

assumptions (1)
  • domain assumption Established BLV guidance standards from experts and research provide a valid and complete basis for annotation.
    The human-AI pipeline is explicitly grounded in these standards.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance." pith.science (2026). https://pith.science/paper/2503.12844

@misc{pith2026250312844,
  author       = {Pith},
  title        = {Pith review of: GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2503.12844}},
  note         = {Machine review of arXiv:2503.12844}
}
read the original abstract

For people affected by blindness and low vision (BLV), safe and independent navigation remains a major challenge, impacting over 2.2 billion individuals worldwide. Although multimodal large language models (MLLMs) offer new opportunities for assistive navigation, progress has been limited by the scarcity of accessibility-aware datasets, because creating them requires labor-intensive expert annotation. To this end, we introduce GuideDog, a novel dataset containing 22K image-description pairs (2K human-verified) capturing real-world pedestrian scenes across 46 countries. Our human-AI pipeline shifts annotation from generation to verification, grounded in established BLV guidance standards from experts and research, improving scalability while maintaining quality. We also present GuideDogQA, an 818-sample benchmark evaluating object recognition and depth perception. Experiments reveal that depth perception and adherence to these standards remain challenging for current MLLMs.

Figures

Figures reproduced from arXiv: 2503.12844 by the authors.

Figure 1
Figure 1. An overview of the accessibility-aware guidance gen [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Side-by-side comparison of examples from (a) G [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. An overview of the GUIDEDOG generation pipeline, ensuring all stages adhere to GUIDEDOG standards (S1, S2, S3). For the collected scene image displayed at the center top, (1) in accordance with S1, the MLLM first extracts a comprehensive scene description; (2) following S2, both off-the-shelf models and the MLLM identify obstacles; (3) next, the extracted information is incorporated into a BLV-specific instruction d… view at source ↗
Figures from the paper (16 more)
Figure 5
Figure 5. Figure 5: Country distribution of samples in the GUIDEDOG dataset. # Source Videos 269 # Total Frames in Source Videos 59.8M Total Source Videos Duration 291 hours # GUIDEDOG Samples 22,084 # GUIDEDOG Gold Label 2,106 # GUIDEDOGQA Samples 818 # Cities in GUIDEDOG 183 # Countries…
Figure 6
Figure 6. Figure 6: Examples of object recognition and relative depth com [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: The global information extraction prompt filters out inappropriate images and extracts scene information and location details in [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The local information extraction prompt, which evaluates detected objects and classifies each as either dangerous or not, in order [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: The silver label generation prompt leverages classified dangerous objects and extracted scene information to generate silver [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: The GPT-Eval prompt, which evaluates how well it adheres to G [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: The zero-shot prompt designed for MLLM to generate accessibility-aware guidance generation in G [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: The 3-shot prompt designed for MLLM to generate accessibility-aware guidance generation in G [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: The zero-shot prompt designed for SM to generate accessibility-aware guidance generation in G [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: The 3-shot prompt designed for SM to generate accessibility-aware guidance generation in G [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: An example of a gold label in GUIDEDOG. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: An example of an object recognition task on G [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: An example of a depth recognition task on G [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: An interface for evaluating and refining silver labels into gold labels. [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]
Figure 19
Figure 19. Figure 19: An interface for validating object detection and depth estimation. [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]
Figure 20
Figure 20. Figure 20: An interface for human evaluation of model-generated descriptions. [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance

    cs.CV 2026-07 conditional novelty 7.0 of 10

    VIABench provides 761 long-form egocentric videos from blind individuals with 14,526 annotations across three assistance tasks, and shows current multimodal LLMs achieve best overall scores below 30.

  2. EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning

    cs.CV 2025-11 conditional novelty 7.0 of 10

    EgoVITA, a GRPO-based plan-then-verify framework with dense visual-grounding rewards, improves egocentric video reasoning by up to +7.7 points and keeps exocentric video performance intact.

Reference graph

Works this paper leans on

71 extracted references · 71 canonical work pages · cited by 2 Pith papers

  1. [1]

    GPT-4 Technical Report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    work page Pith review arXiv

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 6

    work page Pith review arXiv 2025

  3. [3]

    METEOR: An auto- matic metric for MT evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. METEOR: An auto- matic metric for MT evaluation with improved correlation with human judgments. InProceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan, 2005. Association for Computational Linguistics. 6

    work page 2005

  4. [4]

    Depth Pro: Sharp Monocular Metric Depth in Less Than a Second

    Aleksei Bochkovskii, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. arXiv preprint arXiv:2410.02073, 2024. 4

    work page Pith review arXiv 2024

  5. [5]

    Mag- nitude, temporal trends, and projections of the global preva- lence of blindness and distance and near vision impairment: a systematic review and meta-analysis

    Rupert RA Bourne, Seth R Flaxman, Tasanee Braithwaite, Maria V Cicinelli, Aditi Das, Jost B Jonas, Jill Keeffe, John H Kempen, Janet Leasher, Hans Limburg, et al. Mag- nitude, temporal trends, and projections of the global preva- lence of blindness and distance and near vision impairment: a systematic review and meta-analysis. The Lancet Global Health, 5(...

    work page 2017

  6. [6]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 4

    work page 2021

  7. [7]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xing- gang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16901–16911, 2024. 4

    work page 2024

  8. [8]

    Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tri- pathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2024. 6

    work page Pith review arXiv 2024

Show all 71 references
  1. [9]

    V-eye: A vision-based navigation system for the visually impaired

    Ping-Jung Duh, Yu-Cheng Sung, Liang-Yu Fan Chiang, Yung-Ju Chang, and Kuan-Wen Chen. V-eye: A vision-based navigation system for the visually impaired. IEEE Transac- tions on Multimedia, 23:1567–1580, 2020. 3

    2020

  2. [10]

    How to guide someone who is blind or partially sighted

    Emma Turner, Sense. How to guide someone who is blind or partially sighted. https://www.sense.org.uk/ blog/how-to-guide-someone-who-is-blind- or-partially-sighted, 2023. 3

    2023

  3. [11]

    A review of assistive spatial orientation and navigation technologies for the visually impaired

    Hugo Fernandes, Paulo Costa, Vitor Filipe, Hugo Paredes, and Jo ˜ao Barroso. A review of assistive spatial orientation and navigation technologies for the visually impaired. Uni- versal Access in the Information Society, 18:155–168, 2019. 3

    2019

  4. [12]

    Introducing gemini 2.0: our new ai model for the agentic era

    Google DeepMind. Introducing gemini 2.0: our new ai model for the agentic era. https://blog.google/ technology / google - deepmind / google - gemini - ai - update - december - 2024/ , 2024. 6, 8

    2024

  5. [13]

    Vizwiz grand challenge: Answering visual questions from blind people

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3608–3617,

  6. [14]

    Vizwiz-priv: A dataset for recognizing the presence and purpose of private visual information in images taken by blind people

    Danna Gurari, Qing Li, Chi Lin, Yinan Zhao, Anhong Guo, Abigale Stangl, and Jeffrey P Bigham. Vizwiz-priv: A dataset for recognizing the presence and purpose of private visual information in images taken by blind people. In Pro- ceedings of the IEEE/CVF Conference on Computer ...

    2019

  7. [15]

    Space-aware instruction tuning: Dataset and bench- mark for guide dog robots assisting the visually impaired

    ByungOk Han, Woo-han Yun, Beom-Su Seo, and Jaehong Kim. Space-aware instruction tuning: Dataset and bench- mark for guide dog robots assisting the visually impaired. arXiv preprint arXiv:2502.07183, 2025. 2

    2025

  8. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 6

    2022

  9. [17]

    Long-form answers to visual questions from blind and low vision people

    Mina Huh, Fangyuan Xu, Yi-Hao Peng, Chongyan Chen, Hansika Murugu, Danna Gurari, Eunsol Choi, and Amy Pavel. Long-form answers to visual questions from blind and low vision people. arXiv preprint arXiv:2408.06303, 2024. 8

    2024

  10. [18]

    System configuration and navigation of a guide dog robot: Toward animal guide dog-level guiding work

    Hochul Hwang, Tim Xia, Ibrahima Keita, Ken Suzuki, Joy- deep Biswas, Sunghoon I Lee, and Donghyun Kim. System configuration and navigation of a guide dog robot: Toward animal guide dog-level guiding work. In 2023 IEEE Inter- national Conference on Robotics and Automation (ICRA...

    2023

  11. [19]

    Identifying crucial ob- jects in blind and low-vision individuals’ navigation

    Md Touhidul Islam, Imran Kabir, Elena Ariel Pearce, Md Al- imoor Reza, and Syed Masum Billah. Identifying crucial ob- jects in blind and low-vision individuals’ navigation. InPro- ceedings of the 26th International ACM SIGACCESS Con- ference on Computers and Accessibility, pag...

    2024

  12. [20]

    Enhancing multimodal large language models with vision detection models: An empirical study

    Qirui Jiao, Daoyuan Chen, Yilun Huang, Yaliang Li, and Ying Shen. Enhancing multimodal large language models with vision detection models: An empirical study. arXiv preprint arXiv:2401.17981, 2024. 4

    2024

  13. [21]

    Techniques for constructing indoor navigation systems for the visually impaired: A review

    Roya Norouzi Kandalan and Kamesh Namuduri. Techniques for constructing indoor navigation systems for the visually impaired: A review. IEEE Transactions on Human-Machine Systems, 50(6):492–506, 2020. 1

    2020

  14. [22]

    Meganno+: A human-llm collabora- tive annotation system

    Hannah Kim, Kushan Mitra, Rafael Li Chen, Sajjadur Rah- man, and Dan Zhang. Meganno+: A human-llm collabora- tive annotation system. arXiv preprint arXiv:2402.18050 ,

  15. [23]

    Understanding expec- tations for a robotic guide dog for visually impaired people

    J Taery Kim, Morgan Byrd, Jack L Crandell, Bruce N Walker, Greg Turk, and Sehoon Ha. Understanding expec- tations for a robotic guide dog for visually impaired people. arXiv preprint arXiv:2501.04594, 2025. 2 9

    2025

  16. [24]

    Deepnavi: A deep learning based smartphone navigation as- sistant for people with visual impairments

    Bineeth Kuriakose, Raju Shrestha, and Frode Eika Sandnes. Deepnavi: A deep learning based smartphone navigation as- sistant for people with visual impairments. Expert Systems with Applications, 212:118720, 2023. 3

    2023

  17. [25]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 6

    2024 arXiv

  18. [26]

    A survey of multimodel large language models

    Zijing Liang, Yanjie Xu, Yifan Hong, Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. A survey of multimodel large language models. In Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering, pages 405–409, 2024. 4

    2024

  19. [27]

    A technique for the measurement of attitudes

    Rensis Likert. A technique for the measurement of attitudes. Archives of psychology, 1932. 7

    1932

  20. [28]

    ROUGE: A package for automatic evalua- tion of summaries

    Chin-Yew Lin. ROUGE: A package for automatic evalua- tion of summaries. In Text Summarization Branches Out , pages 74–81, Barcelona, Spain, 2004. Association for Com- putational Linguistics. 6

    2004

  21. [29]

    Deep learning based wearable assistive system for visually im- paired people

    Yimin Lin, Kai Wang, Wanxin Yi, and Shiguo Lian. Deep learning based wearable assistive system for visually im- paired people. In Proceedings of the IEEE/CVF interna- tional conference on computer vision workshops, pages 0–0,

  22. [30]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. 1, 8

    2023

  23. [31]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 6

    2024

  24. [32]

    Open scene understanding: Grounded situation recognition meets segment anything for helping people with visual im- pairments

    Ruiping Liu, Jiaming Zhang, Kunyu Peng, Junwei Zheng, Ke Cao, Yufan Chen, Kailun Yang, and Rainer Stiefelhagen. Open scene understanding: Grounded situation recognition meets segment anything for helping people with visual im- pairments. In Proceedings of the IEEE/CVF Internat...

    2023

  25. [33]

    Objectfinder: Open-vocabulary assistive sys- tem for interactive object search by blind people

    Ruiping Liu, Jiaming Zhang, Angela Sch ¨on, Karin M ¨uller, Junwei Zheng, Kailun Yang, Kathrin Gerling, and Rainer Stiefelhagen. Objectfinder: Open-vocabulary assistive sys- tem for interactive object search by blind people. arXiv preprint arXiv:2412.03118, 2024. 8

    2024

  26. [34]

    G-eval: NLG evaluation using gpt- 4 with better human alignment

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt- 4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro- cessing, pages 2511–2522, Singapore, 2023. Associa...

    2023

  27. [35]

    A qualitative and quantitative analysis of research in mobility technologies for visually impaired peo- ple

    Jyoti Madake, Shripad Bhatlawande, Anjali Solanke, and Swati Shilaskar. A qualitative and quantitative analysis of research in mobility technologies for visually impaired peo- ple. IEEE Access, 11:82496–82520, 2023. 3

    2023

  28. [36]

    Mobility-related ac- cidents experienced by people with visual impairment

    Roberto Manduchi and Sri Kurniawan. Mobility-related ac- cidents experienced by people with visual impairment. AER Journal: Research and Practice in Visual Impairment and Blindness, 4(2):44–54, 2011. 1, 3

    2011

  29. [37]

    Generating contextually- relevant navigation instructions for blind and low vision peo- ple

    Zain Merchant, Abrar Anwar, Emily Wang, Souti Chat- topadhyay, and Jesse Thomason. Generating contextually- relevant navigation instructions for blind and low vision peo- ple. arXiv preprint arXiv:2407.08219, 2024. 2, 3, 8

    2024

  30. [38]

    Ai and accessibility

    Meredith Ringel Morris. Ai and accessibility. Communica- tions of the ACM, 63(6):35–37, 2020. 2

    2020

  31. [39]

    We don’t need no bounding-boxes: Train- ing object class detectors using only human verification

    Dim P Papadopoulos, Jasper RR Uijlings, Frank Keller, and Vittorio Ferrari. We don’t need no bounding-boxes: Train- ing object class detectors using only human verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 854–863, 2016. 2

    2016

  32. [40]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , pages 311– 318, Philadelphia, Pennsylvania, USA, 2002. Asso...

    2002

  33. [41]

    Egoblur: Responsible innovation in aria, 2023

    Nikhil Raina, Guruprasad Somasundaram, Kang Zheng, Sagar Miglani, Steve Saarinen, Jeff Meissner, Mark Schwesinger, Luis Pesqueira, Ishita Prasad, Edward Miller, Prince Gupta, Mingfei Yan, Richard Newcombe, Carl Ren, and Omkar M Parkhi. Egoblur: Responsible innovation in aria, ...

    2023

  34. [42]

    Optimal walking in terms of variability in step length

    Noboru Sekiya, Hiroshi Nagasaki, Hajime Ito, and Taketo Furuna. Optimal walking in terms of variability in step length. Journal of Orthopaedic & Sports Physical Therapy, 26(5):266–272, 1997. 4

    1997

  35. [43]

    Touch your way: hap- tic sight for visually impaired people to walk with indepen- dence

    Ji-Won Song and Sung-Ho Yang. Touch your way: hap- tic sight for visually impaired people to walk with indepen- dence. In CHI’10 Extended Abstracts on Human Factors in Computing Systems, pages 3343–3348. 2010. 3

    2010

  36. [44]

    Assisting the blind and visually impaired: guidelines for eye health workers and other helpers

    Sue Stevens. Assisting the blind and visually impaired: guidelines for eye health workers and other helpers. Com- munity Eye Health, 16(45):7, 2003. 3

    2003

  37. [45]

    Nuanced perspectives toward disabil- ity simulations from digital designers, blind, low vision, and color blind people

    Garreth W Tigwell. Nuanced perspectives toward disabil- ity simulations from digital designers, blind, low vision, and color blind people. In Proceedings of the 2021 CHI confer- ence on human factors in computing systems , pages 1–15,

    2021

  38. [46]

    Label Studio: Data labeling soft- ware, 2020-2025

    Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. Label Studio: Data labeling soft- ware, 2020-2025. Open source software available from https://github.com/HumanSignal/label-studio. 12

    2020

  39. [47]

    Cambrian-1: A fully open, vision-centric explo- ration of multimodal llms

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric explo- ration of multimodal llms. Advances in Neural Information Proces...

    2025

  40. [48]

    Helpful resources for business con- sulting

    Vision Australia. Helpful resources for business con- sulting. https://www.visionaustralia.org/ business - consulting / helpful - resources ,

  41. [49]

    How to be a sighted guide

    Vision Loss Resources. How to be a sighted guide. https: //visionlossresources.org/resources/how- to-be-a-guide/

  42. [50]

    Etiquette for interacting with people who are blind or have low vision

    Washington State University. Etiquette for interacting with people who are blind or have low vision. https:// 10 studentaffairs.vancouver.wsu.edu/access- center / etiquette - interacting - people - who-are-blind-who-have-low-vision . 3

  43. [51]

    Dos and don’ts for people with vision loss

    Wisconsin Department of Health Services. Dos and don’ts for people with vision loss. https : / / www . dhs . wisconsin.gov/obvi/adjustment/dos-donts. htm, 2023. 3

    2023

  44. [52]

    Sighted guide techniques

    Wisconsin Department of Health Services. Sighted guide techniques. https://www.dhs.wisconsin.gov/ obvi / adjustment / sightedguidetech . htm ,

  45. [53]

    Emerging practices for large multimodal model (lmm) assistance for people with vi- sual impairments: Implications for design

    Jingyi Xie, Rui Yu, He Zhang, Sooyeon Lee, Syed Ma- sum Billah, and John M Carroll. Emerging practices for large multimodal model (lmm) assistance for people with vi- sual impairments: Implications for design. arXiv preprint arXiv:2407.08882, 2024. 1, 2, 8

    2024

  46. [54]

    Mm-vet: Evaluating large multimodal models for integrated capabilities

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490, 2023. 13

    2023 arXiv

  47. [55]

    Socratic models: Composing zero-shot multimodal rea- soning with language

    Andy Zeng, Maria Attarian, Krzysztof Marcin Choroman- ski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael S Ryoo, Vikas Sindhwani, Johnny Lee, et al. Socratic models: Composing zero-shot multimodal rea- soning with language. In The Eleventh International ...

  48. [56]

    A design of interface for visual-impaired people to access visual information from images featuring large language models and visual language models

    Zhe-Xin Zhang and Yoichi Ochiai. A design of interface for visual-impaired people to access visual information from images featuring large language models and visual language models. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pages 1–4, 2024. 8

    2024

  49. [57]

    Vialm: A survey and benchmark of visually impaired assis- tance with large models

    Yi Zhao, Yilin Zhang, Rong Xiang, Jing Li, and Hillming Li. Vialm: A survey and benchmark of visually impaired assis- tance with large models. arXiv preprint arXiv:2402.01735,

  50. [58]

    drive”, “car

    2, 8 11 A. Automatic Pipeline Details A.1. Gathering Videos To construct a high-quality dataset of outdoor walking scenes, we implemented a systematic approach to video selection. We manually identified YouTube channels spe- cializing in walking tours across urban and natural ...

  51. [59]

    Filter out the following non-street cases from the label {is street}: • Certain objects block most of the screen, making it difficult to recognize the street

    Review the image and decide if it represents a street scene (”Yes”) or not (”No”). Filter out the following non-street cases from the label {is street}: • Certain objects block most of the screen, making it difficult to recognize the street. • The viewer is looking at somethin...

  52. [60]

    • scene location: Describe the location and surroundings using only 10, 11, 12, 1, and 2 o’clock positions

    If "Yes": • scene description: Provide an overview of the street, including pedestrians, buildings, vehicles, or any key elements that make it a street. • scene location: Describe the location and surroundings using only 10, 11, 12, 1, and 2 o’clock positions. The leftmost par...

  53. [61]

    No": • scene description: Briefly explain why this is not a street scene. • scene location: Must be

    If "No": • scene description: Briefly explain why this is not a street scene. • scene location: Must be "None"

  54. [62]

    is_street

    Use only the JSON format below. 5. Do not include any text outside of this JSON format. Output JSON example: { "is_street": "Yes", "scene_description": "A detailed description of the street, including key elements such as pedestrians, shops, and vehicles.", "scene_location": "...

  55. [63]

    • Mark an object as dangerous (”Yes”) only if it poses a collision risk (e.g., curbs, potholes, poles, stairs, con- struction barriers, parked vehicles, etc.)

    Complete Danger Zone (within 5 meters): • Evaluate whether each object is directly in the user’s walking path and could lead to a collision. • Mark an object as dangerous (”Yes”) only if it poses a collision risk (e.g., curbs, potholes, poles, stairs, con- struction barriers, ...

  56. [64]

    object": the object’s identifier or name. •

    Ordinary Zone (beyond 5 meters) • Focus on moving objects in this zone (e.g., approaching motorcycles, cars, bicycles, pedestrians). • Mark an object as dangerous (”Yes”) if it is moving toward the user and could pose a threat. • Otherwise, mark it as not dangerous (”No”) and ...

  57. [66]

    Navigation: After describing all hazards, provide a single, concise sentence on how to safely navigate or avoid them overall. Potential Hazards: {object info} Scene Information: {scene info} Remember to provide a single, flowing explanation without labeled sections, as if talk...

  58. [68]

    Remember to provide a single, flowing explanation without labeled sections, as if talking directly to the visually impaired individual

    Navigation: After describing all hazards, provide a single, concise sentence on how to safely navigate or avoid them overall. Remember to provide a single, flowing explanation without labeled sections, as if talking directly to the visually impaired individual. Figure 11. The ...

  59. [70]

    Examples: • You’re on a bustling city street with buildings on your left, and the sidewalk and storefronts on your right

    Navigation: After describing all hazards, provide a single, concise sentence on how to safely navigate or avoid them overall. Examples: • You’re on a bustling city street with buildings on your left, and the sidewalk and storefronts on your right. At 10 o’clock, about five ste...

  60. [72]

    Remember to provide a single, flowing explanation without labeled sections, as if talking directly to the visually impaired individual

    Navigation: After describing all hazards, provide a single, concise sentence on how to safely navigate or avoid them overall. Remember to provide a single, flowing explanation without labeled sections, as if talking directly to the visually impaired individual. Scene Descripti...

  61. [73]

    Surroundings and Position: Summarize where the person is, the general environment, their current position, and any nearby landmarks in 1-2 sentences

  62. [74]

    • Follow the order of 10, 11, 12, 1, and 2 o’clock

    Hazards: • For each direction (10, 11, 12, 1, and 2 o’clock), combine all hazards in that direction into exactly one sentence, mentioning approximate distance(s) and reason(s) they are dangerous. • Follow the order of 10, 11, 12, 1, and 2 o’clock

  63. [75]

    Remember to provide a single, flowing explanation without labeled sections, as if talking directly to the visually impaired individual

    Navigation: After describing all hazards, provide a single, concise sentence on how to safely navigate or avoid them overall. Remember to provide a single, flowing explanation without labeled sections, as if talking directly to the visually impaired individual. Scene Descripti...

Pith tools

Reviewed May 22, 2026 · model on record in the stance chip above.