Pith. sign in

REVIEW 4 major objections 6 minor 50 references

sMoRe: Enhancing Object Manipulation and Organization in Mixed Reality Spaces with LLMs and Generative AI

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes sMoRe, a mixed-reality system that turns voice or text commands into newly generated 3D objects placed at user-specified physical locations.

desk verdict A real and well-documented MR prototype whose central placement-accuracy claim is not actually measured; worth reviewing but needs a major reframing and objective evaluation. read the letter →

arxiv 2411.11752 v1 pith:6KFO3NY4 submitted 2024-11-18 cs.HC

Yunhao Xing , Que Liu , Jingwu Wang , Diego Gomez-Zara This is my paper · ORCID
classification cs.HC
keywords mixedrealitylargelanguagemodelsgenerativeAItext-to-3Dgenerationnaturalinterfacesobjectplacementspatialmappinguserstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes sMoRe, a mixed-reality application that turns voice or typed commands into newly generated 3D objects placed in the user's physical space. Its central claim is that an LLM-driven pipeline paired with text-to-3D generative AI can replace asset libraries, controllers, and 3D-modeling skill as the main way users create and organize virtual objects. In a six-participant user study, the authors report that users found the system easy to learn, low in physical and mental demand, and effective for achieving what they had in mind. The paper argues this points toward a more intuitive, accessible way to interact with mixed-reality environments.

What carries the argument

The carrying mechanism is the five-module pipeline (Planner, Analyzer, Object Preparer, Location Optimizer, Mesh Generator), with the Location Optimizer doing the placement work by generating random points on LLM-chosen surface types and accepting only candidates that satisfy the twelve spatial-relation criteria in Table 1. The Object Preparer's LLM-produced scale determines collider boundaries, so the entire placement chain depends on that estimate. The 2.5D layout map synchronizes positions with the virtual scene through a linear scaling equation and gives users an overview and an indirect manipulation handle.

What would settle it

Run a placement-accuracy study in which a fixed set of commands with known intended locations is repeated, measuring the distance between the placeholder position and the intended position and checking whether objects stay within the identified surface boundaries; systematic error would show that the reported usability gains come from the interaction style rather than reliable placement.

Watch

Extended reading notes

Core claim

sMoRe establishes a framework in which a natural-language request is decomposed by a Planner LLM into an object name, attributes, and target locations; an Analyzer LLM links those locations to objects in a scanned scene; an Object Preparer LLM assigns real-world dimensions, shape, and gravity; a Location Optimizer samples candidate surface points and validates them against pre-defined spatial-relation rules; and a Mesh Generator calls a text-to-3D service and attaches physics and interaction components. The resulting virtual object appears at the chosen location, persists as the user moves, and is mirrored on a 2.5D layout map. The reported user study supports that this improves comprehension, interaction, and organization of the MR space.

Load-bearing premise

The system assumes that the dimensions the LLM assigns to an object and the random surface points the Location Optimizer samples will place virtual objects where users mean them to go, and the user study never measures placement accuracy.

Editorial extensions

If this is right

  • Users can generate bespoke 3D objects from voice or text descriptions and have them appear on specified physical surfaces without asset libraries or 3D modeling experience.
  • The 2.5D layout map provides a persistent bird's-eye overview and lets users reposition virtual objects indirectly by moving their low-poly representations.
  • Virtual objects respond to gravity and collide with physical and virtual objects during hand or controller manipulation.
  • The same framework can be applied to visual reminders, interior design previews, and creative prototyping in physical spaces.
  • If the usability results generalize, speech-driven generation can lower the cognitive and physical load of manipulating mixed-reality content.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not measure whether the predicted placement matches the user's intended location; a controlled accuracy study would test whether the positive usability ratings come from reliable spatial reasoning or from the novelty of voice-generated objects.
  • Because the Location Optimizer falls back to the floor in front of the user when no valid point is found, failures could silently place objects in unintended spots; making that fallback visible in the interface would let users correct placement immediately.
  • The one-object-per-prompt constraint suggests the Planner would need extension before the system can handle compound organization tasks such as moving several objects into a new arrangement at once.
  • The 2.5D map's real-time synchronization implies the map could function as an external spatial memory aid, but whether it actually improves recall over headset-only views is untested.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents sMoRe, a mixed-reality system that combines an LLM-based pipeline (Planner, Analyzer, Object Preparer, Location Optimizer) with a text-to-3D generator (Genie) to let users create and place virtual objects using voice or typed text. The system also provides a 2.5D layout map and hand/controller interaction. The authors evaluate sMoRe in a lab study with six participants using NASA TLX, SUS, Likert items, and open-ended questions, reporting generally positive descriptive scores and qualitative feedback. The paper claims that sMoRe enhances user comprehension, interaction, and organization in MR environments.

Significance. If the framework works as described, sMoRe is a useful integration of LLM-driven spatial reasoning with generative 3D asset creation in MR, with a clearly described architecture, algorithm pseudocode, and an appendix of module prompts that would allow replication. The 2.5D layout map and the voice-to-3D workflow are interesting interaction ideas. The main limitation is that the evaluation does not provide objective or comparative evidence for placement accuracy or for the claimed usability improvements; therefore the significance is conditional on a strengthened evaluation. The paper is honest about several limitations in Section 5.2, but the headline claims go beyond what the data support.

major comments (4)
  1. [§4.3 and §6] The conclusion that sMoRe "significantly enhances user interaction" and the abstract's claim that the user study "demonstrates the effectiveness" of the system are not supported by the reported evidence. The study has six participants, no control condition, no inferential statistics, and no objective task-performance measures; only descriptive means and standard deviations are reported. Because Contribution 3 claims a "comprehensive user study to validate the usability and effectiveness," this is a load-bearing gap rather than a cosmetic one.
  2. [§5.2 and Algorithm 2 / Table 1] Placement accuracy, which is central to DG2 and to the first contribution (creating and locating virtual objects), is never measured. Algorithm 2 validates candidate locations only against the coarse static criteria in Table 1; for example, "On" checks only whether the candidate is inside the target object's x/z boundaries, with no explicit check that the object rests on the surface or that the generated mesh does not interpenetrate the environment. The Object Preparer's dimensions come from LLM estimates (Appendix A.3), and these dimensions define the placeholder collider and the validation geometry. The paper itself concedes in Section 5.2 that "Future studies should evaluate how accurate the respective matching between the predicted and the actual locations is." Without any objective measurement, the reported SUS/TLX and Likert responses cannot be causally attributed to the locate-and-place pipeline.
  3. [§3.3.4 and Algorithm 2] The fallback behavior of the Location Optimizer is not instrumented. When no candidate point passes validation after up to 10,000 attempts, the system silently places the object on the floor in front of the user. The paper does not report how often this fallback occurs, in which usage scenarios it occurs, or whether any participant's objects were placed by fallback. Since fallback placement overrides the user-specified location, its frequency and consequences are directly relevant to the usability claims and should be measured.
  4. [§4.2 and §4.3] The user-study procedure does not include predefined placement targets or a baseline condition. Participants freely explored generating and placing objects for ten minutes, so the paper cannot report a task success rate or compare sMoRe against an alternative interaction method (e.g., controller-based placement). A within-subjects comparison with a baseline, or at least a task-based accuracy metric with predefined target locations, would substantially strengthen the efficacy claim that the current descriptive Likert scores are meant to support.
minor comments (6)
  1. [§4.2] The NASA TLX and SUS instruments are cited with "[?]" placeholders; full references are needed.
  2. [Appendix A.3 and Figure 5] The Object Preparer example contains "chariity" where the object name appears to be misspelled (likely "charity" or the intended word); this should be corrected.
  3. [§3.4.2] The coordinate-mapping formulas are typeset without clear multiplication symbols and with ambiguous subscripts; defining P_virtual, P_layout, and S_virtual as single variables and presenting the equations with standard math notation would improve readability.
  4. [Title page and ACM Reference Format] The page footer and ACM reference format display the year 2018 and a placeholder DOI, which are inconsistent with the manuscript's actual submission year (2024); these metadata should be fixed.
  5. [§3.3] The text refers to "Pseudo-code 1" and "Pseudo-code 2" while the displayed items are labeled "Algorithm 1" and "Algorithm 2"; the terminology should be made consistent. Additionally, the sentence introducing the Planner's prompt processing is incomplete and should read as a full sentence.
  6. [§6] The conclusion contains the duplicated phrase "significantly enhances user interaction and interaction within MR environments"; one occurrence of "interaction" should be removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: sMoRe's claims rest on an empirical user study and engineering integration, not on a fitted derivation or self-citation chain.

full rationale

The paper's central claims are (1) a framework combining LLMs and GenAI to create and locate virtual objects, and (2) a user study showing improved comprehension, interaction, and organization. Neither claim is derived from an equation fitted to its own output. The workflow in Algorithm 1 chains Planner, Analyzer, Object Preparer, Location Optimizer, and Mesh Generator; each step consumes inputs and produces outputs without redefining a target quantity in terms of itself. The 2.5D layout synchronization formula is a linear coordinate transform, not a prediction. The Object Preparer asks an LLM for real-world dimensions, and the Location Optimizer samples surfaces and applies static rules from Table 1; this is system design and behavior modeling, not a fitted parameter relabeled as a prediction. There are no self-citations or imported uniqueness theorems; the reference list consists of external prior work. The acknowledged limitation in Section 5.2 that placement accuracy was not objectively measured is a validity threat to the causal claim, but it is not circular: no output of the study is an input to the system's placement logic. The n=6 self-report study is weak evidence for the conclusion that sMoRe 'significantly enhances' interaction, but weakness of evidence is a correctness concern, not circularity. Accordingly no circular step meets the quoted-reduction bar.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on unvalidated domain assumptions about the scanned scene, the sufficiency of spatial rules, the reliability of LLM-generated object properties, and the validity of self-report measures. No physical entities are invented, and no parameters are fitted to ground-truth data.

free parameters (2)
  • Object dimensions (w,h,d) from the Object Preparer LLM = varies per LLM output; e.g., car key 0.12, 0.04, 0.07
    Chosen by LLM prompt rather than measured. Placement, collision, and visual integration depend on these values, and their accuracy is never evaluated.
  • Spatial relationship thresholds in Table 1 = null
    The criteria for 'on', 'next to', 'far from', and similar relations are ad hoc binary checks. The Location Optimizer's validity decisions rely on these uncalibrated rules.
assumptions (4)
  • domain assumption A room scanned with PolyCam and imported into Unity as FBX accurately represents physical geometry for placement purposes
    Invoked in Sections 3.3.2 and 3.5; the Analyzer and Location Optimizer operate entirely on this scanned representation with no validation against ground truth.
  • domain assumption The static rules in Table 1 plus LLM synonym matching are sufficient to capture the spatial relations users express
    Section 3.3.4 defines placement validation; the paper notes that 'inside' is unsupported and Section 5.2 acknowledges misinterpretation of complex commands.
  • domain assumption Genie text-to-3D output can be scaled, rotated, and collider-fitted to the placeholder without errors that break the user experience
    Section 3.3.5 replaces placeholders with generated meshes; Section 5.2 concedes that generated objects do not always align with user expectations.
  • domain assumption Self-report scales measure the claimed constructs of comprehension, interaction, and organization
    Section 4 uses NASA TLX, SUS, and custom Likert items, but no task performance or behavioral measure links those responses to actual organizational or comprehension improvements.

how reviews work

0 comments
Cite this review

Pith. "Pith review of sMoRe: Enhancing Object Manipulation and Organization in Mixed Reality Spaces with LLMs and Generative AI." pith.science (2026). https://pith.science/paper/6KFO3NY4

@misc{pith2026241111752,
  author       = {Pith},
  title        = {Pith review of: sMoRe: Enhancing Object Manipulation and Organization in Mixed Reality Spaces with LLMs and Generative AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6KFO3NY4}},
  note         = {Machine review of arXiv:2411.11752}
}
read the original abstract

In mixed reality (MR) environments, understanding space and creating virtual objects is crucial to providing an intuitive and rich user experience. This paper introduces sMoRe (Spatial Mapping and Object Rendering Environment), an MR application that combines Generative AI (GenAI) with large language models (LLMs) to assist users in creating, placing, and managing virtual objects within physical spaces. sMoRe allows users to use voice or typed text commands to create and place virtual objects using GenAI while specifying spatial constraints. The system leverages LLMs to interpret users' commands, analyze the current scene, and identify optimal locations. Additionally, sMoRe integrates text-to-3D generative AI to dynamically create 3D objects based on users' descriptions. Our user study demonstrates the effectiveness of sMoRe in enhancing user comprehension, interaction, and organization of the MR environment.

Figures

Figures reproduced from arXiv: 2411.11752 by the authors.

Figure 1
Figure 1. Illustration of the sMoRe system in action. Users provide voice or typed text commands, such as "Create an Eiffel Tower [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The workflow of sMoRe Manuscript submitted to ACM [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Example of the Planner Manuscript submitted to ACM [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Example of the Analyzer 3.3.2 Analyzer. To generate interactive 3D scenes in MR, sMoRe needs to map the relationships between the physical environment and the virtual objects. The goal of the Analyzer is to identify locations relevant to users’ commands Manuscript subm…
Figure 5
Figure 5. Figure 5: Example of the Object Preparer The module’s LLM component outputs these properties for the object, represented as: 𝑃𝜏 = {𝑤, ℎ, 𝑑, physics,shape}. The Object Preparer then instantiates an object with the detected geometrical shape in Unity with the specified Manuscript …
Figure 6
Figure 6. Figure 6: Example of the Location Optimizer Once a candidate point 𝑝𝑐 = (𝑥, 𝑦, 𝑧) is generated, it is transformed into a candidate location object 𝐿𝑐 that includes properties derived from 𝑃𝜏 and 𝑝𝑐 . This transformation allows the system to consider both the physical attributes …
Figure 7
Figure 7. Figure 7: Example of the Mesh Generator 3.3.5 Mesh Generator. Given the target object (𝜏), the location object (𝐿𝜏 ), and the physical properties 𝑃𝜏 , the Mesh Generator renders the final virtual object (𝑂) for the scene Ω. This module serves as the creator module and is respons…
Figure 8
Figure 8. Figure 8: The procedure of generating the target object ( [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: An example layout map of a conference room with some randomly generated objects. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Selected Results from the post-study questionnaire [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: NASA TLX Results from the post-study questionnaire [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 26 canonical work pages

  1. [1]

    Setareh Aghel Manesh, Tianyi Zhang, Yuki Onishi, Kotaro Hara, Scott Bateman, Jiannan Li, and Anthony Tang. 2024. How People Prompt Generative AI to Create Interactive VR Scenes. In Proceedings of the 2024 ACM Designing Interactive Systems Conference (Copenhagen, Denmark) (DIS ’24). Association for Computing Machinery, New York, NY, USA, 2319–2340. https:/...

    arXiv 2024

  2. [2]

    Claudio Aguayo and Chris Eames. 2023. Using mixed reality (XR) immersive learning to enhance environmental education. The Journal of Environmental Education 54, 1 (2023), 58–71

    work page 2023

  3. [3]

    Bashar I Ahmad, Patrick M Langdon, Simon J Godsill, Richard Donkor, Rebecca Wilde, and Lee Skrypchuk. 2016. You do not have to touch to select: A study on predictive in-car touchscreen with mid-air selection. In proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications. 113–120

    work page 2016

  4. [4]

    Siddhanth Jayaraj Ajri, Dat Nguyen, Swati Agarwal, Arun Kumar Reddy Padala, and Caglar Yildirim. 2023. Virtual AIVantage: Leveraging Large Language Models for Enhanced VR Interview Preparation among Underrepresented Professionals in Computing. In Proceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia (Vienna, Austria) (MUM ’2...

    work page arXiv 2023

  5. [5]

    Takeshi Asano, Ehud Sharlin, Yoshifumi Kitamura, Kazuki Takashima, and Fumio Kishino. 2005. Predictive interaction using the delphian desktop. In Proceedings of the 18th annual ACM symposium on User interface software and technology . 133–141

    work page 2005

  6. [6]

    Huidong Bai, Prasanth Sasikumar, Jing Yang, and Mark Billinghurst. 2020. A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1...

    arXiv 2020

  7. [7]

    Doug A Bowman and Larry F Hodges. 1997. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In Proceedings of the 1997 symposium on Interactive 3D graphics . 35–ff

    work page 1997

  8. [8]

    Vinay Chamola, Gaurang Bansal, Tridib Kumar Das, Vikas Hassija, Siva Sai, Jiacheng Wang, Sherali Zeadally, Amir Hussain, Fei Richard Yu, Mohsen Guizani, and Dusit Niyato. 2024. Beyond Reality: The Pivotal Role of Generative AI in the Metaverse. IEEE Internet of Things Magazine 7, 4 (2024), 126–135. https://doi.org/10.1109/IOTM.001.2300174

    work page doi:10.1109/iotm.001.2300174 2024

Show all 50 references
  1. [9]

    Aldrich Clarence, Jarrod Knibbe, Maxime Cordeil, and Michael Wybrow. 2021. Unscripted retargeting: Reach prediction for haptic retargeting in virtual reality. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR) . IEEE, 150–159

    2021

  2. [10]

    Fernanda De La Torre, Cathy Mengying Fang, Han Huang, Andrzej Banburski-Fahey, Judith Amores Fernandez, and Jaron Lanier. 2024. LLMR: Real-time Prompting of Interactive Worlds using Large Language Models. In Proceedings of the CHI Conference on Human Factors in Computing Syste...

    2024

  3. [11]

    Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. 2022. Get3d: A generative model of high quality 3d textured shapes learned from images. Advances In Neural Information Processing Systems 35 (2022), 31841–31854

    2022

  4. [12]

    Franca Garzotto, Emanuele Torelli, Francesco Vona, and Beatrice Aruanno. 2018. HoloLearn: Learning through Mixed Reality for People with Cognitive Disability. 189–190. https://doi.org/10.1109/AIVR.2018.00042

    2018

  5. [13]

    Michele Gattullo, Enricoandrea Laviola, Antonio Boccaccio, Alessandro Evangelista, Michele Fiorentino, Vito Modesto Manghisi, and Antonio Em- manuele Uva. 2022. Design of a mixed reality application for STEM distance education laboratories. Computers 11, 4 (2022), 50

    2022

  6. [14]

    Daniele Giunchi, Nels Numan, Elia Gatti, and Anthony Steed. 2024. DreamCodeVR: Towards Democratizing Behavior Design in Virtual Reality with Speech-Driven Programming. In2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR). 579–589. https://doi.org/10.1109/VR58804....

    2024

  7. [15]

    Jens Grubert. 2021. Mixed reality interaction techniques. arXiv preprint arXiv:2103.05984 (2021)

    2021 arXiv

  8. [17]

    Shafina Abd Karim Ishigaki, Ajune Wanis Ismail, Nur Ameerah Abdul Halim, and Norhaida Mohd Suaib. 2023. Voice Commands with Virtual Assistant in Mixed Reality Telepresence. In International Conference on Advanced Engineering, Technology and Applications . Springer, 148–158

    2023

  9. [18]

    Jacek Jankowski and Martin Hachet. 2015. Advances in interaction with 3D environments. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 152–190

    2015

  10. [19]

    Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy J. Mitra. 2023. HOLODIFFUSION: Training a 3D Diffusion Model Using 2D Images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 18423–18433. Manuscript submitted to ACM 22 Xing et al

    2023

  11. [20]

    Jae Yeol Lee, Gue Won Rhee, and Dong Woo Seo. 2010. Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. The International Journal of Advanced Manufacturing Technology 51 (2010), 1069–1082

    2010

  12. [21]

    SooHyung Lee, HyeRin Lee, and KiSuk Lee. 2023. Knowledge Generation Pipeline using LLM for Building 3D Object Knowledge Base. In 2023 14th International Conference on Information and Communication Technology Convergence (ICTC) . 1303–1305. https://doi.org/10.1109/ICTC58733.202...

    2023

  13. [22]

    Yemon Lee, Andy M Connor, and Stefan Marks. 2024. Mixed interaction: evaluating user interactions for object manipulations in virtual space. Journal on Multimodal User Interfaces (2024), 1–15

    2024

  14. [23]

    Zhipeng Li, Christoph Gebhardt, Yves Inglin, Nicolas Steck, Paul Streli, and Christian Holz. 2024. SituationAdapt: Contextual UI Optimization in Mixed Reality with Situation Awareness via LLM Reasoning. arXiv:2409.12836 [cs.HC] https://arxiv.org/abs/2409.12836

    2024 arXiv

  15. [24]

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin

  16. [25]

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin Johnson. 2024. Scalable 3d captioning with pretrained models. Advances in Neural Information Processing Systems 36 (2024)

    2024

  17. [26]

    Daniel Mendes, Fabio Marco Caputo, Andrea Giachetti, Alfredo Ferreira, and Joaquim Jorge. 2019. A survey on 3d virtual object manipulation: From the desktop to immersive virtual environments. In Computer graphics forum, Vol. 38. Wiley Online Library, 21–45

    2019

  18. [27]

    Mark R Mine. 1995. Virtual environment interaction techniques. UNC Chapel Hill CS Dept (1995)

    1995

  19. [28]

    Mo, John J Dudley, and Per Ola Kristensson

    George B. Mo, John J Dudley, and Per Ola Kristensson. 2021. Gesture Knitter: A Hand Gesture Design Tool for Head-Mounted Mixed Reality Applications. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Com...

    2021

  20. [29]

    Hee-Seung Moon, Yi-Chi Liao, Chenyu Li, Byungjoo Lee, and Antti Oulasvirta. 2024. Real-time 3D Target Inference via Biomechanical Simulation. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Mac...

    2024

  21. [30]

    Hyacinth S Nwana. 1996. Software agents: An overview. The knowledge engineering review 11, 3 (1996), 205–244

    1996

  22. [31]

    Barron, and Ben Mildenhall

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. 2022. DreamFusion: Text-to-3D using 2D Diffusion. arXiv:2209.14988 [cs.CV] https://arxiv.org/abs/2209.14988

    2022 arXiv

  23. [32]

    Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, and Tadao Ichikawa. 1996. The go-go interaction technique: non-linear mapping for direct manipulation in VR. In Proceedings of the 9th annual ACM symposium on User interface software and technology . 79–80

    1996

  24. [33]

    Ivan Poupyrev, Tadao Ichikawa, Suzanne Weghorst, and Mark Billinghurst. 1998. Egocentric object manipulation in virtual environments: empirical evaluation of interaction techniques. In Computer graphics forum, Vol. 17. Wiley Online Library, 41–52

    1998

  25. [34]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.000...

    2021 arXiv

  26. [35]

    Rafael Radkowski and Christian Stritzke. 2012. Interactive hand gesture-based assembly for augmented reality applications. In Proceedings of the 2012 International Conference on Advances in Computer-Human Interactions . 303–308

    2012

  27. [36]

    Hafizur Rahaman, Erik Champion, and Mafkereseb Bekele. 2019. From photo to 3D to mixed reality: A complete workflow for cultural heritage visualisation and experience. Digital Applications in Archaeology and Cultural Heritage 13 (2019), e00102

    2019

  28. [37]

    Jasmine Roberts, Andrzej Banburski-Fahey, and Jaron Lanier. 2022. Steps towards prompt-based creation of virtual worlds. arXiv:2211.05875 [cs.HC] https://arxiv.org/abs/2211.05875

    2022 arXiv

  29. [38]

    Jasmine Roberts, Andrzej Banburski-Fahey, and Jaron Lanier. 2022. Surreal vr pong: Llm approach to game design. In 36th Conference on Neural Information Processing Systems (NeurIPS 2022) , Vol. 1

    2022

  30. [39]

    Yulin Shen, Yifei Shen, Jiawen Cheng, Chutian Jiang, Mingming Fan, and Zeyu Wang. 2024. Neural Canvas: Supporting Scenic Design Prototyping by Integrating 3D Sketching and Generative AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, ...

    2024

  31. [40]

    Kevin Skyba and Thies Pfeiffer. 2024. Towards natural language understanding for intuitive interactions in XR using large language models. GI VR / AR Workshop. https://doi.org/10.18420/vrar2024_0021

    2024 doi

  32. [41]

    Yuk-Ming Tang, Kin Man Au, Henry CW Lau, George TS Ho, and Chun-Ho Wu. 2020. Evaluating the effectiveness of learning design with mixed reality (MR) in higher education. Virtual Reality 24, 4 (2020), 797–807

    2020

  33. [42]

    Manuel Veit, Antonio Capobianco, and Dominique Bechmann. 2009. Influence of degrees of freedom’s manipulation on performances during orientation tasks in virtual reality environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology (Kyoto, J...

    2009

  34. [43]

    Ryan Volum, Sudha Rao, Michael Xu, Gabriel DesGarennes, Chris Brockett, Benjamin Van Durme, Olivia Deng, Akanksha Malhotra, and Bill Dolan. 2022. Craft an Iron Sword: Dynamically Generating Interactive Game Characters by Prompting Large Language Models Tuned on Code. In Procee...

    2022

  35. [45]

    Hongyu Wan, Jinda Zhang, Abdulaziz Arif Suria, Bingsheng Yao, Dakuo Wang, Yvonne Coady, and Mirjana Prpa. 2024. Building LLM-based AI Agents in Social Virtual Reality. In Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems (CHI EA ’24) . Associa...

    2024

  36. [46]

    Cheng, Alwena Lin, Yu Chen, and Mike Y

    Shun-Yu Wang, Wei-Chung Su, Serena Chen, Ching-Yi Tsai, Marta Misztal, Katherine M. Cheng, Alwena Lin, Yu Chen, and Mike Y. Chen. 2024. RoomDreaming: Generative-AI Approach to Facilitating Iterative, Preliminary Interior Design Exploration. In Proceedings of the CHI Conference...

    2024

  37. [47]

    Yudan Wu, Shanhe You, Zixuan Guo, Xiangyang Li, Guyue Zhou, and Jiangtao Gong. 2023. MR. Brick: designing a remote mixed-reality educational game system for promoting children’s social & collaborative skills. InProceedings of the 2023 CHI Conference on Human Factors in Computi...

    2023

  38. [48]

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. 2023. PointLLM: Empowering Large Language Models to Understand Point Clouds. arXiv:2308.16911 [cs.CV] https://arxiv.org/abs/2308.16911

    2023 arXiv

  39. [49]

    Luke Yoffe, Aditya Sharma, and Tobias Höllerer. 2023. OCTOPUS: Open-vocabulary Content Tracking and Object Placement Using Semantic Understanding in Mixed Reality. In 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) . 587–588. https://do...

    2023

  40. [50]

    Difeng Yu, Xueshi Lu, Rongkai Shi, Hai-Ning Liang, Tilman Dingler, Eduardo Velloso, and Jorge Goncalves. 2021. Gaze-Supported 3D Object Manipulation in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21)....

    2021

  41. [51]

    name": "PC Machine

    Mingyuan Zhong, Gang Li, Peggy Chi, and Yang Li. 2021. HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for ...

    2021

  42. [2023]

    In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Magic3D: High-Resolution Text-to-3D Content Creation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 300–309. https://doi.org/10.1109/CVPR52729.2023.00037

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.