pith. sign in

arxiv: 2606.25202 · v1 · pith:TQVWMHENnew · submitted 2026-06-23 · 💻 cs.HC · cs.RO

ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task

Christian Fronk , Hanting Ye , Zhehan Qu , Maria Gorlatova This is my paper

Pith reviewed 2026-06-25 21:49 UTC · model grok-4.3

classification 💻 cs.HC cs.RO
keywords augmented realityhuman-robot collaborationobstruction mitigationwarehouse taskssituated analyticsAR-HRCuser study
0
0 comments X

The pith

AR situated analytics improve warehouse HRC efficiency by 46% only when paired with obstruction mitigation to preserve real-world visibility.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents ARTOO-DARTU as an AR system for human-robot collaboration in warehouses that delivers real-time robot feedback while using an obstruction detection and mitigation pipeline to avoid blocking views of important real objects. In a user study with 34 participants using a gamified task called Pocket MonstARs, the full system produced a 46% efficiency gain overall and 61% faster performance on visibility-critical subtasks. Without the mitigation, the analytics showed no benefit, indicating that the ability to keep real-world elements visible is what enables the gains in dynamic settings with moving robots. This matters because HRC tasks require both information from the robot and clear sight for safe and effective work.

Core claim

The central claim is that the ARTOO-DARTU system, equipped with its obstruction detection and mitigation pipeline, allows AR situated analytics to significantly enhance efficiency and user experience in AR-HRC warehouse scenarios by preventing obstructions to real-world visibility, as shown by the 46% increase in overall task efficiency and 61% faster subtasks when the pipeline is active compared to when it is not.

What carries the argument

The obstruction detection and mitigation (ODM) pipeline, which identifies and reduces AR content that would otherwise obstruct real-world elements during mobile robot movements.

If this is right

  • When the ODM is active, AR situated analytics increase overall HRC task efficiency by 46%.
  • Participants are 61% faster on subtasks that require visibility of the real world when using the ODM.
  • The system enables real-time robot situated analytics and control without posing safety risks from visual obstructions.
  • AR content can be dynamically positioned relative to robot movements while maintaining usability.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar mitigation techniques could extend to other mobile robot environments where AR overlays risk hiding physical hazards.
  • Real warehouse deployments might show different gains if the gamified elements are replaced with actual inventory items.
  • The results suggest that future AR-HRC designs should prioritize visibility preservation as a core requirement rather than an add-on.

Load-bearing premise

The gamified Pocket MonstARs task with virtual monsters and labeled boxes accurately reflects the real-world identification and visibility requirements of warehouse inventory picking with mobile robots.

What would settle it

A study measuring task completion times and error rates in an actual warehouse setting with physical boxes and moving robots, comparing conditions with and without the obstruction mitigation.

Figures

Figures reproduced from arXiv: 2606.25202 by Christian Fronk, Hanting Ye, Maria Gorlatova, Zhehan Qu.

Figure 1
Figure 1. Figure 1: (a) The area allocated for our developed Pocket MonstARs task, where users collaborate with a robot [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The simulated warehouse environment. (a) Zoomed-in views of inventory labels. (b) Worker’s perspec [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: ARTOO-DARTU system diagram. 2. Convenient AR control of a robotic collaborator requiring minimal interface additions. 3. Mitigation of AR obstructions of the real-world in a dynamic environment with mobile elements, without mitigation breaking spatial alignment of situated analytics. We do not position ARTOO-DARTU as a system for responding to warehouse accidents after they occur. Rather, our intended appl… view at source ↗
Figure 4
Figure 4. Figure 4: The setup and AR view of ARTOO-DARTU. (a) Situated analytics indicators for the robot. (b) Use [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The flow of the ODM for a case in which an AR item obstructs a real-world labeled inventory box. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The two monster types in Pocket MonstARs: visible monsters and box-associated monsters identified using real-world cues. complete the game, users must catch 15 monsters by moving the TurtleBot to each monster location and pressing a virtual stun button to make the TurtleBot stun the monster. A monster is considered stunned, and subsequently caught, if the TurtleBot’s Euclidean separation distance to the mo… view at source ↗
Figure 7
Figure 7. Figure 7: Process of catching a visible monster during the Pocket MonstARs SA+ARR trial with ODM inactive. (a) A visible monster appears, and the user follows the instruction text by placing the map marker at the monster’s location. (b) The TurtleBot navigates to the map marker while trajectory indicators appear and the command status screen updates. (c) Once the TurtleBot reaches the destination, the user presses t… view at source ↗
Figure 8
Figure 8. Figure 8: Task performance data for all users of each group across all trials. (a): Average overall catch times. (b): [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
read the original abstract

Human-robot collaboration (HRC) often requires robot intentions and internal states to be conveyed to users for task efficiency and safety. Recently, augmented reality (AR) situated analytics provide such real-time robot feedback in HRC contexts. However, AR situated analytics can obstruct important real-world elements, posing safety and usability risks, especially when content is dynamically positioned relative to movements of mobile robots in a warehouse HRC scenario. In this paper, we introduce the Augmented Reality Technique Of Obstruction Deterrence while Aiding Robotic Teaming for Users (ARTOO-DARTU), an AR system tailored specifically for warehouse HRC that enables real-time robot situated analytics and control while preserving visibility of the real world through an obstruction detection and mitigation pipeline (ODM) that is uniquely suited for AR-HRC. To evaluate ARTOO-DARTU, we developed Pocket MonstARs, a controlled gamified abstraction of HRC warehouse inventory picking in which virtual monsters serve as proxies for pick targets, while labeled and object-marked boxes preserve the real-world identification demands of the picking task. In a 34-participant user study, we found that our designed AR situated analytics yielded a 46% increase in efficiency on the overall HRC task, but only when the ODM was active. Participants with the ODM active were also 61% faster on subtasks requiring visibility of the real world. Our findings demonstrate that, when paired with our developed ODM to prevent real-world obstructions, the situated analytics in ARTOO-DARTU can significantly enhance efficiency and user experience in AR-HRC warehouse scenarios.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces ARTOO-DARTU, an AR system for warehouse human-robot collaboration (HRC) featuring a tailored obstruction detection and mitigation (ODM) pipeline that enables real-time situated robot analytics while preserving visibility of the real world. Evaluation uses the Pocket MonstARs gamified task (virtual monsters as pick targets, labeled boxes for identification) in a 34-participant study, reporting a 46% efficiency increase on the overall HRC task and 61% faster performance on visibility-requiring subtasks, but only when ODM is active.

Significance. If the results hold, the work demonstrates the practical importance of obstruction mitigation for safe and efficient AR-HRC in dynamic settings with mobile robots. The specialized ODM pipeline is a clear engineering contribution, and the controlled gamified evaluation enables direct condition comparisons. The quantitative claims, if supported by appropriate statistics and design details, provide evidence that pairing situated analytics with ODM can improve user performance.

major comments (1)
  1. [§4] §4 (User Study and Task Design): The 46% overall efficiency gain and 61% subtask speedup are reported only with ODM active, but these rest on the assumption that the Pocket MonstARs task (virtual monsters as proxies for pick targets plus labeled boxes) reproduces the visibility and identification demands of real warehouse inventory picking. Real scenarios involve variable physical occlusions, lighting changes, and unpredictable robot trajectories that the controlled abstraction may not replicate; if the ODM benefit is an artifact of this simplification, the conditional claim does not transfer. This is load-bearing for the central applicability argument.
minor comments (2)
  1. [Abstract] Abstract: The quantitative percentages are stated without reference to the accompanying statistical tests, sample sizes per condition, or error measures; while the body likely contains these, the abstract should at minimum indicate that results are statistically supported.
  2. [Figures] Figure captions (e.g., system overview figures): Ensure all pipeline stages in the ODM description are explicitly labeled so readers can trace the obstruction detection logic without cross-referencing the text.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their thoughtful review and constructive feedback on the applicability of our controlled evaluation. We address the single major comment below.

read point-by-point responses

  1. Referee: [§4] §4 (User Study and Task Design): The 46% overall efficiency gain and 61% subtask speedup are reported only with ODM active, but these rest on the assumption that the Pocket MonstARs task (virtual monsters as proxies for pick targets plus labeled boxes) reproduces the visibility and identification demands of real warehouse inventory picking. Real scenarios involve variable physical occlusions, lighting changes, and unpredictable robot trajectories that the controlled abstraction may not replicate; if the ODM benefit is an artifact of this simplification, the conditional claim does not transfer. This is load-bearing for the central applicability argument.

    Authors: We appreciate the referee's concern about ecological validity. The Pocket MonstARs task was deliberately constructed as a controlled gamified abstraction to isolate the precise visibility and identification demands that ODM is designed to protect. Virtual monsters serve as proxies for pick targets while the labeled and object-marked physical boxes directly preserve the real-world identification requirements of inventory picking; this design choice allows us to measure ODM's effect on subtasks that require unobstructed real-world visibility. Such abstractions are standard in HCI for rigorously comparing interface conditions before field deployment. We do not claim that the 46% and 61% gains will transfer unchanged to every real warehouse; our claims are scoped to the controlled setting in which ODM demonstrably mitigates the obstruction problem introduced by situated AR analytics. We will add a dedicated paragraph in the Discussion section that (a) explicitly states the rationale for the task elements and (b) acknowledges the additional real-world variables (lighting, unpredictable trajectories) not present in the study, thereby clarifying the boundary of the applicability argument without altering any results or claims. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical user study with no derivations or fitted parameters

full rationale

The paper describes an AR system (ARTOO-DARTU) and reports results from a 34-participant controlled user study on a gamified task (Pocket MonstARs). No equations, derivations, parameters, or predictive models are present in the provided text or abstract. The efficiency claims (46% overall, 61% on visibility subtasks) are direct empirical measurements, not outputs of any fitted or self-referential construction. The ODM pipeline is a described implementation, not a mathematical reduction. Self-citations, if any, are not load-bearing for any derivation. This matches the default case of a self-contained empirical study with no opportunity for the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The work is an empirical system design and user study. It introduces no mathematical free parameters, axioms, or postulated physical entities. The ODM pipeline is a software component rather than an invented entity with independent evidence requirements.

pith-pipeline@v0.9.1-grok · 5838 in / 1215 out tokens · 22235 ms · 2026-06-25T21:49:54.086308+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 1 canonical work pages

  1. [1]

    Kaveh Azadeh, M. B. M. René de Koster, and Debjit Roy. 2017. Robotized and Automated Warehouse Systems: Review and Recent Developments.SSRN Electronic Journal(2017)

    2017

  2. [2]

    Blaine Bell, Steven Feiner, and Tobias Höllerer. 2001. View Management for Virtual and Augmented Reality. InProc. ACM UIST

    2001

  3. [3]

    John Brooke. 1995. SUS: A Quick and Dirty Usability Scale.Usability Eval. Ind.(11 1995)

    1995

  4. [4]

    Chan, Geoffrey Hanks, Maram Sakr, Haomiao Zhang, Tiger Zuo, H

    Wesley P. Chan, Geoffrey Hanks, Maram Sakr, Haomiao Zhang, Tiger Zuo, H. F. Machiel van der Loos, and Elizabeth Croft. 2022. Design and Evaluation of an Augmented Reality Head-mounted Display Interface for Human Robot Teams Collaborating in Physically Shared Manufacturing Tasks.J. Hum.-Robot Interact.(2022)

    2022

  5. [5]

    Kishan Chandan, Vidisha Kudalkar, Xiang Li, and Shiqi Zhang. 2021. ARROCH: Augmented Reality for Robots Collaborating with a Human. InProc. IEEE ICRA

    2021

  6. [6]

    Linfeng Chen, Kazuki Takashima, Kazuyuki Fujita, and Yoshifumi Kitamura. 2021. PinpointFly: An Egocentric Position-control Drone Interface using Mobile AR. InProc. ACM CHI

    2021

  7. [7]

    Tian, Tadayoshi Kohno, and Franziska Roesner

    Kaiming Cheng, Jeffery F. Tian, Tadayoshi Kohno, and Franziska Roesner. 2023. Exploring User Reactions and Mental Models Towards Perceptual Manipulation Attacks in Mixed Reality. InProc. USENIX Security

    2023

  8. [8]

    Shakiba Davari, Feiyu Lu, and Doug A. Bowman. 2020. Occlusion Management Techniques for Everyday Glanceable AR Interfaces. InProc. IEEE VRW

    2020

  9. [9]

    Shakiba Davari, Feiyu Lu, and Doug A. Bowman. 2022. Validating the Benefits of Glanceable and Context-Aware Augmented Reality for Everyday Information Access Tasks. InProc. IEEE VR

    2022

  10. [10]

    Philipp Fleck, Aimée Sousa Calepso, Sebastian Hubenschmid, Michael Sedlmair, and Dieter Schmalstieg. 2023. RagRug: A Toolkit for Situated Analytics.IEEE TVCG(2023)

    2023

  11. [11]

    Gay-Bellile, P

    V. Gay-Bellile, P. Lothe, S. Bourgeois, E. Royer, and S. Naudet Collette. 2010. Augmented Reality in Large Environments: Application to Aided Navigation in Urban Context. InProc. IEEE ISMAR

    2010

  12. [12]

    Raphaël Grasset, Tobias Langlotz, Denis Kalkofen, Markus Tatzgern, and Dieter Schmalstieg. 2012. Image-driven View Management for Augmented Reality Browsers. InProc. IEEE ISMAR

    2012

  13. [13]

    Erico Guizzo. 2008. Three Engineers, Hundreds of Robots, One Warehouse.IEEE Spectrum(2008)

    2008

  14. [14]

    Hooman Hedayati, Michael Walker, and Daniel Szafir. 2018. Improving Collocated Robot Teleoperation with Augmented Reality. InProc. ACM HRI

    2018

  15. [15]

    Sebastian Hubenschmid, Jonathan Wieland, Daniel Immanuel Fink, Andrea Batch, Johannes Zagermann, Niklas Elmqvist, and Harald Reiterer. 2022. ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies. InProc. ACM CHI. https://doi.org/10.1145/3491102.3517550

    work page doi:10.1145/3491102.3517550 2022

  16. [16]

    Bryce Ikeda and Daniel Szafir. 2024. PRogramAR: Augmented Reality End-User Robot Programming.J. Hum.-Robot Interact.(2024)

    2024

  17. [17]

    Rafia Inam, Klaus Raizer, Alberto Hata, Ricardo Souza, Elena Forsman, Enyu Cao, and Shaolei Wang. 2018. Risk Assessment for Human-Robot Collaboration in an Automated Warehouse Scenario. InProc. IEEE ETFA

    2018

  18. [18]

    Sarah Isherwood. 2009. Graphics and Semantics: The Relationship between What Is Seen and What Is Meant in Icon Design. InEngineering Psychology and Cognitive Ergonomics. Springer

    2009

  19. [19]

    Jaided AI et al. [n. d.]. EasyOCR. https://github.com/JaidedAI/EasyOCR

  20. [20]

    Glenn Jocher et al. 2022. ultralytics/yolov5: v7.0 – YOLOv5 SOTA Realtime Instance Segmentation. Proc. ACM Hum.-Comput. Interact., Vol. 10, No. 5, Article MHCI7668. Publication date: August 2026. ARTOO-DARTU MHCI7668:21

    2022

  21. [21]

    Jim Lawrence, Javier Bernal, and Christoph Witzgall. 2019. A Purely Algebraic Justification of the Kabsch-Umeyama Algorithm.Journal of Research of the National Institute of Standards and Technology(2019)

    2019

  22. [22]

    Kiron Lebeck, Kimberly Ruth, Tadayoshi Kohno, and Franziska Roesner. 2017. Securing Augmented Reality Output. In Proc. IEEE SP

    2017

  23. [23]

    Weizhou Luo, Zhongyuan Yu, Rufat Rzayev, Marc Satkowski, Stefan Gumhold, Matthew McGinity, and Raimund Dachselt. 2023. Pearl: Physical Environment based Augmented Reality Lenses for In-Situ Human Movement Analysis. InProc. ACM CHI

    2023

  24. [24]

    Stefan Maass and Jürgen Döllner. 2006. Efficient View Management for Dynamic Annotation Placement in Virtual Landscapes. InSmart Graphics

    2006

  25. [25]

    Simone Macciò, Alessandro Carfì, and Fulvio Mastrogiovanni. 2022. Mixed Reality as Communication Medium for Human-Robot Collaboration. InProc. IEEE ICRA

    2022

  26. [26]

    Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. 2022. Robot Operating System 2: Design, architecture, and uses in the wild.Science Robotics(2022)

    2022

  27. [27]

    Microsoft. 2024. Mixed Reality Toolkit 3. https://github.com/MixedRealityToolkit/MixedRealityToolkit- Unity?tab=readme-ov-file

    2024

  28. [28]

    1990.Operators’ Trust in and Use of Automatic Controllers in a Supervisory Process Control Task

    Bonnie Marlene Muir. 1990.Operators’ Trust in and Use of Automatic Controllers in a Supervisory Process Control Task. University of Toronto

    1990

  29. [29]

    Occupational Safety and Health Administration (OSHA). 2023. Establishment-Specific Injury and Illness Data

    2023

  30. [30]

    Koichi Owaki, Nattaon Techasarntikul, and Hideyuki Shimonishi. 2023. Human Behavior Analysis in Human-Robot Cooperation with AR Glasses. InProc. IEEE ISMAR

    2023

  31. [31]

    Frieder Pankratz, Andreas Dippon, Tayfur Coskun, and Gudrun Klinker. 2013. User Awareness of Tracking Uncertainties in AR Navigation Scenarios. InProc. IEEE ISMAR

    2013

  32. [32]

    Hannah Pelikan and Emily Hofstetter. 2023. Managing Delays in Human-Robot Interaction.Proc. ACM CHI(2023)

    2023

  33. [33]

    Poels, Y.A.W

    K. Poels, Y.A.W. de Kort, and W.A. IJsselsteijn. 2007.D3.3 : Game Experience Questionnaire: Development of a Self-Report Measure to Assess the Psychological Impact of Digital Games. Technische Universiteit Eindhoven

    2007

  34. [34]

    Shuwen Qiu, Hangxin Liu, Zeyu Zhang, Yixin Zhu, and Song-Chun Zhu. 2020. Human-Robot Interaction in a Shared Augmented Reality Workspace. InProc. IEEE/RSJ IROS

    2020

  35. [35]

    Sebastian Ramirez. [n. d.]. FastAPI. https://fastapi.tiangolo.com

  36. [36]

    Maha Sajid, Syed Ibrahim Mustafa Shah Bukhari, Bo Ji, and Brendan David-John. 2025. Just Stop Doing Everything for Now!: Understanding Security Attacks in Remote Collaborative Mixed Reality

    2025

  37. [37]

    Marc Satkowski, Rufat Rzayev, Eva Goebel, and Raimund Dachselt. 2022. ABOVE & BELOW: Investigating Ceiling and Floor for Augmented Reality Content Placement. InProc. IEEE ISMAR

    2022

  38. [38]

    Tim Scargill, Ying Chen, Tianyi Hu, and Maria Gorlatova. 2023. SiTAR: Situated Trajectory Analysis for In-the-Wild Pose Error Estimation. InProc. IEEE ISMAR

    2023

  39. [39]

    Manisah Mohd Shah, Haslina Arshad, and Riza Sulaiman. 2012. Occlusion in Augmented Reality. InProc. IEEE ICIDT

    2012

  40. [40]

    Shubham Sonawani, Yifan Zhou, and Heni Ben Amor. 2023. Projecting Robot Intentions Through Visual Cues: Static vs. Dynamic Signaling. InProc. IEEE/RSJ IROS

    2023

  41. [41]

    Tianyu Song, Felix Pabst, Ulrich Eck, and Nassir Navab. 2025. Enhancing Patient Acceptance of Robotic Ultrasound through Conversational Virtual Agent and Immersive Visualizations.IEEE TVCG(2025)

    2025

  42. [42]

    Unity Technologies. 2025. NavMesh. https://docs.unity3d.com/6000.0/Documentation/ScriptReference/AI.NavMesh.h tml

    2025

  43. [43]

    Unity Technologies Japan. 2024. Unity Warehouse. https://assetstore.unity.com/packages/3d/environments/industrial /unity-warehouse-276394

    2024

  44. [44]

    Walker, Hooman Hedayati, and Daniel Szafir

    Michael E. Walker, Hooman Hedayati, and Daniel Szafir. 2019. Robot Teleoperation with Augmented Reality Virtual Surrogates. InProc. ACM HRI

    2019

  45. [45]

    Sean White and Steven Feiner. 2009. SiteLens: Situated Visualization Techniques for Urban Site Visits. InProc. SIGCHI Human Factors in Computing Systems

    2009

  46. [46]

    Yanming Xiu, Tim Scargill, and Maria Gorlatova. 2025. ViDDAR: Vision Language Model-Based Task-Detrimental Content Detection for Augmented Reality.IEEE TVCG(2025)

    2025

  47. [47]

    Vicky Zhang, Alexander Albers, Christine Saeedi-Givi, Per Ola Kristensson, Thomas Bohné, and Sławomir Tadeja

  48. [48]

    Should I Evaluate My Augmented Reality System in an Industrial Environment? Investigating the Effects of Classroom and Shop Floor Settings on Guided Assembly.IEEE TVCG(2024)

    2024

  49. [49]

    Yu Zhao, Jeanine Stefanucci, Sarah Creem-Regehr, and Bobby Bodenheimer. 2023. Evaluating Augmented Reality Landmark Cues and Frame of Reference Displays with Virtual Reality.IEEE TDSC(2023). Proc. ACM Hum.-Comput. Interact., Vol. 10, No. 5, Article MHCI7668. Publication date: August 2026

    2023