Pith. sign in

REVIEW 42 references

No Fuss, Just Function -- A Proposal for Non-Intrusive Full Body Tracking in XR for Meaningful Spatial Interactions

T0 review · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A concept proposal, with no evaluation, that human pose estimation can provide non-intrusive full body tracking in XR for accessibility and natural interaction.

arxiv 2504.11987 v1 pith:CINEH7UE submitted 2025-04-16 cs.HC

Elisabeth Mayer , Thomas Odaker , Dieter Kranzlmüller This is my paper · ORCID
classification cs.HC
keywords non-intrusivebodyinteractionsusersaccessibilityexperiencefullgoal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Extended reality (XR) covers virtual and augmented reality. Many XR systems use handheld controllers for input, but controllers can be hard to use for people with motor limitations. Full body tracking (FBT) lets a person's own body be the input device. Classic FBT needs markers or trackers strapped to the body, which is expensive and can be uncomfortable. Human pose estimation (HPE) is a computer vision technique that finds a person's joints from camera images. Recent HPE systems work with a single RGB or depth camera, so no extra hardware is needed. This paper calls that 'non-intrusive' tracking and proposes that HPE can make FBT available to all XR users. The authors define two conditions: the user does not wear additional hardware, and the user is not aware of the tracking hardware. They review existing work on markerless pose estimation and on adapted systems for wheelchair users, and they argue that HPE can be tuned for tremors or limited mobility. The paper does not build or test a system. It is a position paper meant to start a discussion and to set an agenda for future evaluation. The authors themselves state that the next step is to run expert interviews, build a framework, and evaluate the approach in terms of user experience, usability, and workload.
Extended reading notes

Core claim

The paper's central assertion is that human pose estimation (HPE) can provide a low-cost non-intrusive full body tracking system for XR, satisfying two conditions: C1 (the user wears no additional hardware) and C2 (the user is unaware of the tracking hardware), thereby improving accessibility and natural interaction for all users. This is stated in Section 3 and reiterated in the Conclusion: 'We are confident that HPE can help provide non-intrusive FBT and bring natural interactions in XR to a broader audience.'

Load-bearing premise

The proposal assumes HPE models can be made accurate for users with diverse bodies and motor limitations, including wheelchair users and people with tremors, in unconstrained camera setups. The authors flag this in Section 5: 'the majority of HPE models are trained with able-bodied people.' Additionally, C2 is admitted to be unevaluated. If HPE cannot generalize to these populations, the accessibility benefit collapses even if the non-intrusive design is achievable.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities. The proposal rests on literature-derived assumptions about HPE capability and accessibility benefits. The two conditions C1/C2 are the authors' framing and are untested.

assumptions (4)
  • domain assumption HPE can provide accurate markerless 3D pose estimation sufficient for FBT in XR.
    Invoked in Section 3 as the basis for meeting C1; supported by cited works [4, 28, 33, 34].
  • domain assumption Non-intrusive tracking improves user experience and accessibility.
    The paper's framing in Sections 3 and 4 relies on this; cited works [5, 11, 12, 27].
  • ad hoc to paper Conditions C1 and C2 adequately define non-intrusive FBT.
    The authors propose these conditions in Section 3 based on prior work; no empirical validation is provided.
  • domain assumption HPE models can be adapted to users with motor limitations without major accuracy loss.
    Section 4 discusses tremor counter-balancing and dynamic adaptation; flagged as unverified in Section 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of No Fuss, Just Function -- A Proposal for Non-Intrusive Full Body Tracking in XR for Meaningful Spatial Interactions." pith.science (2026). https://pith.science/paper/CINEH7UE

@misc{pith2026250411987,
  author       = {Pith},
  title        = {Pith review of: No Fuss, Just Function -- A Proposal for Non-Intrusive Full Body Tracking in XR for Meaningful Spatial Interactions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CINEH7UE}},
  note         = {Machine review of arXiv:2504.11987}
}
read the original abstract

Extended Reality (XR) is a rapidly growing field with a wide range of hardware from head mounted displays to installations. Users have the possibility to access the entire Mixed Reality (MR) continuum. Goal of the human-computer-interaction (HCI) community is to allow natural and intuitive interactions but in general interactions for XR often rely on handheld controllers. One natural interaction method is full body tracking (FBT), where a user can use their body to interact with the experience. Classically, FBT systems require markers or trackers on the users to capture motion. Recently, there have been approaches based on Human Pose Estimation (HPE), which highlight the potential of low-cost non-intrusive FBT for XR. Due to the lack of handheld devices, HPE may also improve accessibility with people struggling with traditional input methods. This paper proposes the concept of non-intrusive FBT for XR for all. The goal is to spark a discussion on advantages for users by using a non-intrusive FBT system for accessibility and user experience.

Figures

Figures reproduced from arXiv: 2504.11987 by the authors.

Figure 1
Figure 1. Intrusive Full Body Tracking vs Non-Intrusive Full Body Tracking for XR: One user wearing marker-based FBT (left) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 23 canonical work pages

  1. [1]

    Vladislav Angelov, Emiliyan Petkov, Georgi Shipkovenski, and Teodor Kalushkov

    work page

  2. [2]

    Christoph Anthes, Rubén Jesús García-Hernández, Markus Wiedemann, and Dieter Kranzlmüller. 2016. State of the art of virtual reality technology. (2016), 1–19. https://doi.org/10.1109/AERO.2016.7500674

    arXiv 2016

  3. [3]

    Bowman, Ernst Kruijff, Joseph J

    Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola, and Ivan Poupyrev. 2001. An Introduction to 3-D User Interface Design. Presence 10, 1 (2001), 96–108. https://doi.org/10.1162/105474601750182342

    work page doi:10.1162/105474601750182342 2001

  4. [4]

    Nicola Capece, Ugo Erra, and Giuseppe Romaniello. 2018. A Low-Cost Full Body Tracking System in Virtual Reality Based on Microsoft Kinect. (2018), 623–635. https://doi.org/10.1007/978-3-319-95282-6_44

    work page doi:10.1007/978-3-319-95282-6_44 2018

  5. [5]

    Polona Caserman, Augusto Garcia-Agundez, and Stefan Göbel. 2020. A Survey of Full-Body Motion Reconstruction in Immersive Virtual Reality Applications.IEEE Transactions on Visualization and Computer Graphics 26, 10 (2020), 3089–3108. https://doi.org/10.1109/TVCG.2019.2912607

    arXiv 2020

  6. [6]

    Polona Caserman, Augusto Garcia-Agundez, Robert Konrad, Stefan Göbel, and Ralf Steinmetz. 2019. Real-time body tracking in virtual reality using a Vive tracker. Virtual Reality 23 (2019), 155–168. https://doi.org/10.1007/s10055-018- 0374-z

    doi:10.1007/s10055-018- 2019

  7. [7]

    Colombo, A

    C. Colombo, A. Del Bimbo, and A. Valli. 2001. Real-time tracking and reproduction of 3D human body motion. In Proceedings 11th International Conference on Image Analysis and Processing. 108–112. https://doi.org/10.1109/ICIAP.2001.956993

    work page arXiv 2001

  8. [8]

    Comport, E

    A.I. Comport, E. Marchand, and F. Chaumette. 2003. A real-time tracker for markerless augmented reality. In The Second IEEE and ACM International Sym- posium on Mixed and Augmented Reality, 2003. Proceedings. 36–45. https: //doi.org/10.1109/ISMAR.2003.1240686

    arXiv 2003

Show all 42 references
  1. [9]

    Carolina Cruz-Neira, Daniel J Sandin, Thomas A DeFanti, Robert V Kenyon, and John C Hart. 1992. The CAVE: Audio visual experience automatic virtual environment. Commun. ACM 35, 6 (1992), 64–73

    1992

  2. [10]

    Yann Desmarais, Denis Mottet, Pierre Slangen, and Philippe Montesinos. 2021. A review of 3D human pose estimation algorithms for markerless motion capture. Computer Vision and Image Understanding 212 (2021), 103275. https://doi.org/10. 1016/j.cviu.2021.103275

    2021

  3. [11]

    Xavier Desurmont, Isabel Martinez-Ponte, Jerome Meessen, and Jean-francois Delaigle. 2006. Nonintrusive viewpoint tracking for 3D for perception in smart video conference. 6056 (2006), 106–117. https://doi.org/10.1117/12.642924

    2006 doi

  4. [12]

    John Dudley, Lulu Yin, Vanja Garaj, and Per Ola Kristensson. 2023. Inclu- sive Immersion: a review of efforts to improve accessibility in virtual reality, augmented reality and the metaverse. Virtual Reality 27, 4 (2023), 2989–3020. https://doi.org/10.1007/s10055-023-00850-8

    2023 doi

  5. [13]

    Moshe Gabel, Ran Gilad-Bachrach, Erin Renshaw, and Assaf Schuster. 2012. Full body gait analysis with Kinect. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society . 1964–1967. https://doi.org/10. 1109/EMBC.2012.6346340

    2012

  6. [14]

    Thomas Helten, Andreas Baak, Meinard Müller, and Christian Theobalt. 2013. Full-Body Human Motion Capture from Monocular Depth Images. (2013), 188–

    2013

  7. [15]

    William Huang, Sam Ghahremani, Siyou Pei, and Yang Zhang. 2024. Wheel- Pose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Users. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Ass...

    2024

  8. [16]

    Inman, Ken Loge, Aaron Cram, and Missy Peterson

    Dean P. Inman, Ken Loge, Aaron Cram, and Missy Peterson. 2011. Learning to Drive a Wheelchair in Virtual Reality. Journal of Special Education Technology 26, 3 (2011), 21–34. https://doi.org/10.1177/016264341102600303

    2011 doi

  9. [17]

    B Jo, S Kim, and S Kim. 2023. Enhancing Virtual and Augmented Reality In- teractions with a MediaPipe-Based Hand Gesture Recognition User Interface. Ingénierie des Systèmes d’Information 28 (2023), 633–638. https://doi.org/10.18280/ isi.280311

    2023

  10. [18]

    Jia Jun, Qi Yue, and Zuo Qing. 2010. An Extended Marker-Based Tracking System for Augmented Reality. In 2010 Second International Conference on Modeling, Simulation and Visualization Methods . 94–97. https://doi.org/10.1109/WMSVM. 2010.52

    2010 doi

  11. [19]

    S.M. Kim, M. Sked, and Q. Ji. 2004. Non-intrusive eye gaze tracking under natural head movements. In The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society , Vol. 1. 2271–2274. https: //doi.org/10.1109/IEMBS.2004.1403660

    2004 arXiv

  12. [20]

    Alexander Kulik. 2009. Building on Realism and Magic for Designing 3D Interac- tion Techniques. IEEE Computer Graphics and Applications 29, 6 (2009), 22–33. https://doi.org/10.1109/MCG.2009.115

    2009 doi

  13. [21]

    Gongjin Lan, Yu Wu, Fei Hu, and Qi Hao. 2023. Vision-Based Human Pose Estimation via Deep Learning: A Survey. IEEE Transactions on Human-Machine Systems 53, 1 (2023), 253–268. https://doi.org/10.1109/THMS.2022.3219242

    2023

  14. [22]

    Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, and Cewu Lu. 2021. HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Hu- man Pose and Shape Estimation. (2021), 3382–3392. https://doi.org/10.1109/ CVPR46437.2021.00339

    2021

  15. [23]

    Yunzhi Li, Vimal Mollyn, Kuang Yuan, and Patrick Carrington. 2024. WheelPoser: Sparse-IMU Based Body Pose Estimation for Wheelchair Users. In Proceedings of CHI ’25, Yokohama, JPN, Mayer et al. the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St....

    2024

  16. [24]

    Phil Lopes, Jan-Niklas Voigt-Antons, Jaime Garcia, and David Melhart. 2023. Editorial: User states in extended reality media experiences for entertainment games. https://doi.org/10.3389/frvir.2023.1235004

    2023

  17. [25]

    Jorge Martín-Gutiérrez, Carlos Efrén Mora, Beatriz Añorbe-Díaz, and Antonio González-Marrero. 2017. Virtual Technologies Trends in Education. Eurasia journal of mathematics, science and technology education 13, 2 (2017), 469–486. https://doi.org/10.12973/eurasia.2017.00626a

    2017

  18. [26]

    Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1995. Aug- mented reality: A class of displays on the reality-virtuality continuum. In Tele- manipulator and telepresence technologies , Vol. 2351. Spie, 282–292

    1995

  19. [27]

    Martez Mott, Edward Cutrell, Mar Gonzalez Franco, Christian Holz, Eyal Ofek, Richard Stoakley, and Meredith Ringel Morris. 2019. Accessible by Design: An Opportunity for Virtual Reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct...

    2019

  20. [28]

    Bruce Xiaohan Nie, Ping Wei, and Song-Chun Zhu. 2017. Monocular 3D Human Pose Estimation by Predicting Depth on Joints. (2017), 3467–3475. https://doi. org/10.1109/ICCV.2017.373

    2017 doi

  21. [29]

    Michael Nielsen, Moritz Störring, Thomas B Moeslund, and Erik Granum. 2004. A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Gesture-Based Communication in Human-Computer Interaction: 5th International Gesture Workshop, GW 2003, Genova, Italy, ...

    2004

  22. [30]

    Joseph O’Rourke and Norman I. Badler. 1980. Model-based image analysis of human motion using constraint propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-2, 6 (1980), 522–536. https://doi.org/10. 1109/TPAMI.1980.6447699

    1980

  23. [31]

    Georgios Papadopoulos, Alexandros Doumanoglou, and Dimitrios Zarpalas. 2023. VRGestures: Controller and Hand Gesture Datasets for Virtual Reality. (2023), 336–350. https://doi.org/10.1007/978-3-031-50075-6_26

    2023 doi

  24. [32]

    Jiyoung Park, S Rhee, and Myoung-Hee Kim. 2006. Non-Intrusive Tracking of Multiple Users in a Spatially Immersive Display. InProceedings of the First Interna- tional Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, . INSTICC, SciTePress, 464–467. https...

    2006 doi

  25. [33]

    Amaan Rahman, Mili Shah, Ya-Shian Li-Baboud, and Ann Virts. 2023. Towards a Markerless 3D Pose Estimation Tool. (2023-04-28 04:04:00 2023). https://doi.org/ 10.1145/3544549.3583950

    2023

  26. [34]

    Dennis Reimer, Iana Podkosova, Daniel Scherzer, and Hannes Kaufmann. 2023. Evaluation and improvement of HMD-based and RGB-based hand tracking solu- tions in VR. Frontiers in Virtual Reality 4 (2023). https://doi.org/10.3389/frvir. 2023.1169313

    2023

  27. [35]

    Sminchisescu and B

    C. Sminchisescu and B. Triggs. 2001. Covariance scaled sampling for monocular 3D body tracking. 1 (2001), I–I. https://doi.org/10.1109/CVPR.2001.990509

    2001

  28. [36]

    Anthony Steed, Tuukka M Takala, Daniel Archer, Wallace Lages, and Robert W Lindeman. 2021. Directions for 3D User Interface Research from Consumer VR Games. IEEE Transactions on Visualization and Computer Graphics 27, 11 (2021), 4171–4182. https://doi.org/10.1109/TVCG.2021.3106431

    2021

  29. [37]

    Anastasios Theodoropoulos, Dimitra Stavropoulou, Panagiotis Papadopoulos, Nikos Platis, and George Lepouras. 2023. Developing an Interactive VR CAVE for Immersive Shared Gaming Experiences. 2, 2 (2023), 162–181. https://doi.org/ 10.3390/virtualworlds2020010

    2023 doi

  30. [38]

    Victor Vieira, Eder Oliveira, Aline Menin, Esteban Clua, Marco Winckler, and Daniela Trevisan. 2024. Understanding affordances in XR interactions through a design space. In Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems (IHC ’24) . Associati...

    2024

  31. [39]

    Lam, and James A

    Jackie (Junrui) Yang, Tuochao Chen, Fang Qin, Monica S. Lam, and James A. Landay. 2022. HybridTrak: Adding Full-Body Tracking to VR Using an Off-the- Shelf Webcam. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA)(CHI ’22). ...

    2022

  32. [40]

    Yaying Zhang, Rongkai Shi, and Hai-Ning Liang. 2024. Designing Stick-Based Extended Reality Controllers: A Participatory Approach. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’24). Association for Computing Mach...

    2024

  33. [206]

    https://doi.org/10.1007/978-3-642-44964-2_9

    doi

  34. [2020]

    (2020), 1–5

    Modern Virtual Reality Headsets. (2020), 1–5. https://doi.org/10.1109/ HORA49412.2020.9152604

    2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.