Pith. sign in

REVIEW 2 major objections 50 references

Spatula turns generative motion graphics into an elastic, on-canvas control space that users can discover, zoom, group, and expand on demand.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 11:59 UTC pith:MWDUMKH3

load-bearing objection Solid HCI systems paper that operationalizes four practical dimensions for on-demand in-situ attribute control; the user-study comparison is confounded by fixed order and lacks objective metrics, but the formative work, prototype, and honest limitations still make it worth a referee. the 2 major comments →

arxiv 2607.10405 v1 pith:MWDUMKH3 submitted 2026-07-11 cs.HC cs.AIcs.GR

Spatula: Exploring On-Demand In-Situ Interfaces and Interaction for Attribute Control

Boyu Li , Linjie Qiu , Lin-Ping Yuan , Duotun Wang , Yue Jiang , Zeyu Wang , Hongbo Fu This is my paper
classification cs.HC cs.AIcs.GR
keywords On-Demand UICreativity SupportAttribute ControlMotion GraphicsIn-Situ InteractionLLM InterfacesDirect Manipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

After an AI generates a motion graphic, creators still need precise control over attributes such as color, speed, and scale. Pure text prompting is imprecise and slow; traditional panels are dense and cognitively heavy. Spatula treats the set of controllable attributes as an elastic landscape that can be explored along four dimensions: discoverability (in-situ hints that show what can be adjusted and how), resolution (widgets that expand from coarse presets to fine curves or color spaces), scope (semantic grouping so related elements move together), and expandability (proactive suggestions plus lightweight text to add missing parameters). An LLM analyzes the animation code, maps attributes to interaction primitives, and injects temporary on-canvas controls. A 12-person study and demos in web design and 3D modeling show that this approach supports both novices exploring and experts sculpting, while keeping the interface light.

Core claim

Attribute control for generative motion graphics can be made actionable by reframing it as an Elastic Attribute Control Space whose structure, granularity, and boundary adapt on demand, rather than as either a fixed hierarchical panel or a black-box prompt. Spatula operationalizes that space through four coordinated mechanisms—context-aware in-situ discovery, multi-resolution widgets, semantic scope grouping, and proactive expansion—and shows via user study and cross-domain demos that the resulting scaffolds support fine-grained, low-latency refinement while remaining lightweight.

What carries the argument

The Elastic Attribute Control Space: an adaptive interaction scaffold that dynamically reveals (Discoverability), refines (Resolution), groups (Scope), and extends (Expandability) the parameters of a generated motion graphic via LLM-driven analysis and in-situ UI injection.

Load-bearing premise

That an LLM can reliably read arbitrary animation code, extract a useful primary set of attributes, and map them to correct interaction widgets without frequent failure on complex or hard-coded scenes.

What would settle it

On a held-out suite of complex p5.js (or equivalent) animations, measure how often the system either misses essential primary attributes or injects incorrect/broken interaction bindings; if failure rates remain high after expansion, the elastic-space claim does not hold in practice.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The paper presents Spatula, a proof-of-concept system that generates on-demand, in-situ attribute-control interfaces for motion graphics. Starting from formative interviews and a technical probe that uses an LLM to analyze p5.js code and inject UI/interaction bindings, the authors reframe attribute control as an Elastic Attribute Control Space organized along four dimensions: Discoverability (context-aware hints), Resolution (multi-LOD widgets), Scope (semantic grouping), and Expandability (proactive attribute addition). A comparative user study (N=12, novices and experts) with four sequential conditions (LLM prompting, separate panel, tech probe, Spatula) reports higher subjective ratings for controllability and enjoyment; qualitative findings describe divergent novice/expert strategies. Plug-and-play demos to web design and 3D modeling, plus an appendix technical evaluation of attribute extraction, are offered as supporting evidence.

Significance. If the claims hold, the work supplies a useful design framing and concrete interaction mechanisms for the post-generation refinement gap that currently separates generative AI from traditional authoring tools. The four-dimension elastic-space construct, the knowledge-driven UI synthesis pipeline, and the explicit generalization demos are concrete contributions that other HCI systems can reuse. The formative-to-probe-to-system trajectory is carefully documented and the appendix technical evaluation of LLM attribute extraction is a welcome addition. These strengths make the paper a solid systems contribution even if the comparative evidence remains largely subjective.

major comments (2)
  1. Section 5.1.2 and Figure 12: the four conditions were presented in fixed sequential order (LLM → panel → probe → Spatula) with no counterbalancing or washout. Learning, fatigue, and progressive familiarity with the same targets therefore systematically favor the final condition. The reported advantage of Spatula over the tech probe (which already supplies in-situ widgets) rests almost entirely on subjective Likert ratings and quotes; no objective performance measures (time-to-target, parameter error, number of adjustments, success rate against Stage-1 targets) are provided. This confounds the central claim that the four elastic dimensions themselves produce the observed benefit.
  2. Section 4.4, Limitations, and Appendix 10: the system’s viability rests on the assumption that an LLM can reliably extract a useful primary-attribute set and map it to correct interaction primitives. The technical evaluation reports precision/recall/F1 only for primary attributes on 50 scripts and does not quantify failure modes on hard-coded or complex scenes (the very cases flagged in the Limitations). Without a clearer characterization of extraction reliability and recovery strategies, the generalizability claim remains under-supported.

Circularity Check

0 steps flagged

No significant circularity; standard iterative HCI design paper whose empirical claims rest on independent formative observations and a comparative user study.

full rationale

The paper's derivation chain is observational and constructive rather than predictive or definitional. Formative interviews and a tech probe surface four challenges (C1–C4); these directly motivate four design guidelines (D1–D4) that are then operationalized as the Elastic Attribute Control Space dimensions. The mapping is explicit design response, not a tautology: the dimensions are not defined in terms of the later user-study outcomes, nor are any parameters fitted to data and then re-presented as predictions. The N=12 comparative study and the Appendix technical evaluation of LLM attribute extraction (precision/recall against expert-annotated ground truth) constitute independent empirical measurements. Self-citations to the authors' prior systems appear only in Related Work and Applications as contextual examples; none supply a uniqueness theorem, ansatz, or load-bearing premise that forces the present results. No equations equate an output quantity to an input by construction. Consequently the central claims remain falsifiable by the reported study and do not reduce to their own inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 1 invented entities

HCI systems paper; load-bearing content is empirical and design-derived rather than axiomatic. Free parameters are mainly study and model choices. Invented entity is the elastic control space itself. Domain assumptions about LLM reliability and user intent are the main external premises.

free parameters (3)
  • LLM choice and temperature/settings (Gemini-3-Pro primary)
    Attribute extraction quality and interaction mapping depend on the specific model; technical evaluation shows variance across Gemini/GPT/Kimi/Qwen, so results are model-dependent.
  • Primary vs secondary attribute hierarchy thresholds
    Human annotators and the system decide which parameters count as 'primary'; the cut is judgmental and affects what is initially exposed.
  • LOD widget ordering and gesture thresholds (drag distance, long-press time)
    Hand-tuned interaction rules that determine when multi-resolution expansion or gesture disambiguation fires.
axioms (3)
  • domain assumption Users need fine-grained, in-situ parameter control after generative creation and that text prompts alone are insufficient for precise refinement.
    Stated from formative interviews (Sec. 3.1) and used to motivate the entire design; treated as given rather than re-proven.
  • domain assumption An LLM can parse executable animation code (p5.js) and produce a usable structured attribute + interaction schema.
    Core technical premise of the Analyzing/Applying agents (Sec. 4.4); technical evaluation quantifies but does not eliminate failure modes.
  • ad hoc to paper The four dimensions (Discoverability, Resolution, Scope, Expandability) adequately span the attribute-control needs observed in the probe.
    Derived from the four challenges C1–C4; presented as the organizing framework without independent completeness proof.
invented entities (1)
  • Elastic Attribute Control Space no independent evidence
    purpose: Conceptual scaffold that unifies the four adaptive dimensions and justifies dynamic UI generation instead of fixed panels.
    New framing introduced by the authors; independent evidence is limited to the user-study outcomes and demos within this paper.

pith-pipeline@v1.1.0-grok45 · 26373 in / 2726 out tokens · 35404 ms · 2026-07-14T11:59:07.760272+00:00 · methodology

0 comments
read the original abstract

Controlling attributes is a critical step toward achieving the final creative outcome, yet current approaches fall short in supporting users in the iterative refinement of generative content. We propose Spatula, a proof-of-concept system that generates on-demand, in-situ attribute control interfaces and interactions for creating motion graphics. Building on a technical probe that automatically analyzes animation context and generates corresponding attributes and UI, we frame attribute control as an explorable landscape and explore the attribute control space along four key dimensions: Discoverability, Resolution, Scope, and Expandability. Findings from a user study (N=12) show that our system provides intuitive and convenient interactions while supporting diverse needs for fine-grained parameter control. Furthermore, our applications demonstrate that the plug-and-play design generalizes to other domains, such as web design and 3D modeling.

Figures

Figures reproduced from arXiv: 2607.10405 by Boyu Li, Duotun Wang, Hongbo Fu, Linjie Qiu, Lin-Ping Yuan, Yue Jiang, Zeyu Wang.

Figure 1
Figure 1. Figure 1: We introduce Spatula, a system for generating on-demand, in-situ attribute control interfaces for motion graphics. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Limitations of current attribute control paradigms. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Pipeline of the tech probe for on-demand interac [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The framework of the Elastic Attribute Control Space. Spatula maps common UI examples and interaction modalities [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Example Hints of what can be adjusted. Animatable [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example hints of how to adjust. On-hover tooltips [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Detailed attribute values are revealed during inter [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Examples of Multi-Resolution Control. (Left) color [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 12
Figure 12. Figure 12: User ratings results from the user study. [PITH_FULL_IMAGE:figures/full_fig_p009_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Application examples of Spatula. (Left) In web [PITH_FULL_IMAGE:figures/full_fig_p010_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: UI Widget Examples from commercial tools. [PITH_FULL_IMAGE:figures/full_fig_p013_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Interface for annotating motion graphics at [PITH_FULL_IMAGE:figures/full_fig_p015_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Precision, recall, and F1 scores for predicting pri [PITH_FULL_IMAGE:figures/full_fig_p015_16.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 6 canonical work pages

  1. [1]

    2026.RGB curves

    Adobe. 2026.RGB curves. https://helpx.adobe.com/premiere/desktop/correct- color/add-color-effects/correct-color-using-rgb-curves.html

    2026

  2. [2]

    Cai, Michael Terry, Quoc Le, and Charles Sutton

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program Synthesis with Large Language Models. arXiv:2108.07732 [cs.PL] https://arxiv.org/abs/2108.07732

    Pith/arXiv arXiv 2021

  3. [3]

    Jazbo Beason, Ruijia Cheng, Eldon Schoop, and Jeffrey Nichols. 2025. Athena: Intermediate Representations for Iterative Scaffolded App Generation with an LLM.arXiv preprint arXiv:2508.20263(2025)

    Pith/arXiv arXiv 2025

  4. [4]

    Jiaqi Chen, Yanzhe Zhang, Yutong Zhang, Yijia Shao, and Diyi Yang. 2025. Generative Interfaces for Language Models. arXiv:2508.19227 [cs.CL] https: //arxiv.org/abs/2508.19227

    Pith/arXiv arXiv 2025

  5. [5]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    Pith/arXiv arXiv 2021

  6. [6]

    Adobe After Effect. 2025. Adobe After Effects - Motion graphics software. https: //www.adobe.com/products/aftereffects.html

    2025

  7. [7]

    2026.Color Wheel

    Figma. 2026.Color Wheel. https://www.figma.com/color-wheel/

    2026

  8. [8]

    Gajos, Daniel S

    Krzysztof Z. Gajos, Daniel S. Weld, and Jacob O. Wobbrock. 2010. Automatically generating personalized user interfaces with Supple.Artificial Intelligence174, 12 (2010), 910–950. doi:10.1016/j.artint.2010.05.005

    doi:10.1016/j.artint.2010.05.005 2010

  9. [9]

    François Guimbretiére and Terry Winograd. 2000. FlowMenu: combining com- mand, text, and data entry. InProceedings of the 13th Annual ACM Sympo- sium on User Interface Software and Technology(San Diego, California, USA) (UIST ’00). Association for Computing Machinery, New York, NY, USA, 213–216. doi:10.1145/354401.354778

    work page doi:10.1145/354401.354778 2000

  10. [10]

    Aditya Gunturu, Yi Wen, Nandi Zhang, Jarin Thundathil, Rubaiat Habib Kazi, and Ryo Suzuki. 2024. Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook Diagrams. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology(Pittsburgh, PA, USA)(UIST ’24). Association for Computing Machinery...

    doi:10.1145/3654777.3676392 2024

  11. [11]

    Robert Held, Ankit Gupta, Brian Curless, and Maneesh Agrawala. 2012. 3D Puppetry: A Kinect-based Interface for 3D Animation. InProceedings of the 25th Annual ACM Symposium on User Interface Software and Technology(Cambridge, Massachusetts, USA)(UIST ’12). Association for Computing Machinery, New York, NY, USA, 423–434. doi:10.1145/2380116.2380170

    doi:10.1145/2380116.2380170 2012

  12. [12]

    Ken Hinckley and Mike Sinclair. 1999. Touch-sensing input devices. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(Pittsburgh, Pennsylvania, USA)(CHI ’99). Association for Computing Machinery, New York, NY, USA, 223–230. doi:10.1145/302979.303045

    work page doi:10.1145/302979.303045 1999

  13. [13]

    Yuki Koyama and Masataka Goto. 2022. BO as Assistant: Using Bayesian Op- timization for Asynchronously Generating Design Suggestions. InProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA)(UIST ’22). Association for Computing Machinery, New York, NY, USA, Article 77, 14 pages. doi:10.1145/3526113.3545664

    doi:10.1145/3526113.3545664 2022

  14. [14]

    Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2016. SelPh: Progressive Learning and Support of Manual Photo Color Enhancement. InProceedings of the 2016 CHI Conference on Human Factors in Computing Systems(San Jose, California, USA)(CHI ’16). Association for Computing Machinery, New York, NY, USA, 2520–2532. doi:10.1145/2858036.2858111

    doi:10.1145/2858036.2858111 2016

  15. [15]

    Boyu Li, Linjie Qiu, Duotun Wang, Qianxi Liu, Ryo Suzuki, Mingming Fan, and Zeyu Wang. 2025. DesignMemo: Integrating Discussion Context into Online Collaboration with Enhanced Design Rationale Tracking.Proc. ACM Hum.- Comput. Interact.9, 7, Article CSCW398 (Oct. 2025), 32 pages. doi:10.1145/3757579

    work page doi:10.1145/3757579 2025

  16. [16]

    Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, and Wojciech Matusik. 2025. VLMaterial: Procedural Material Generation with Large Vision-Language Models. InProceedings of the International Conference on Learning Representations (ICLR)(Singapore). https://openreview.net/forum? id=wHebuIb6IH

    2025

  17. [17]

    Boyu Li, Linping Yuan, Zhe Yan, Qianxi Liu, Yulin Shen, and Zeyu Wang. 2024. AniCraft: Crafting Everyday Objects as Physical Proxies for Prototyping 3D Character Animation in Mixed Reality. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology(Pittsburgh, PA, USA) (UIST ’24). Association for Computing Machinery, New York...

    doi:10.1145/3654777.3676325 2024

  18. [18]

    Boyu Li, Lin-Ping Yuan, and Zeyu Wang. 2025. VideoCraft: A Mixed Reality- empowered Video Generation Workflow with Spatial Layer Editing for Concept Video Creation. InProceedings of the 38th Annual ACM Symposium on User Inter- face Software and Technology (UIST ’25). Association for Computing Machinery, New York, NY, USA, Article 19, 16 pages. doi:10.1145...

    doi:10.1145/3746059.3747606 2025

  19. [19]

    Yi-Chi Liao, Paul Streli, Zhipeng Li, Christoph Gebhardt, and Christian Holz

  20. [20]

    InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25)

    Continual Human-in-the-Loop Optimization. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 795, 26 pages. doi:10. 1145/3706598.3713603

    arXiv 2025

  21. [21]

    Shaoteng Liu, Tianyu Wang, Jui-Hsien Wang, Qing Liu, Zhifei Zhang, Joon-Young Lee, Yijun Li, Bei Yu, Zhe Lin, Soo Ye Kim, and Jiaya Jia. 2024. Generative Video Propagation. arXiv:2412.19761 [cs.CV] https://arxiv.org/abs/2412.19761

    Pith/arXiv arXiv 2024

  22. [22]

    Vivian Liu, Rubaiat Habib Kazi, Li-Yi Wei, Matthew Fisher, Timothy Langlois, Seth Walker, and Lydia Chilton. 2025. LogoMotion: Visually-Grounded Code Synthesis for Creating and Editing Animation. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Conference’17, July 2017, Washington, DC, USA Boyu Li, ...

    arXiv 2025

  23. [23]

    Xiang Liu, Peijie Dong, Xuming Hu, and Xiaowen Chu. 2024. LongGenBench: Long-context Generation Benchmark. InFindings of the Association for Computa- tional Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 865–883. doi:10.18653/v1/2024.findings-emnlp.48

    doi:10.18653/v1/2024.findings-emnlp.48 2024

  24. [24]

    Yiren Liu, Si Chen, Haocong Cheng, Mengxia Yu, Xiao Ran, Andrew Mo, Yiliu Tang, and Yun Huang. 2024. How AI Processing Delays Foster Creativity: Explor- ing Research Question Co-Creation with an LLM-based Agent. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, ...

    doi:10.1145/3613904.3642698 2024

  25. [25]

    Jiaju Ma and Maneesh Agrawala. 2025. MoVer: Motion Verification for Motion Graphics Animations.ACM Trans. Graph.44, 4, Article 33 (July 2025), 17 pages. doi:10.1145/3731209

    doi:10.1145/3731209 2025

  26. [26]

    Damien Masson, Sylvain Malacria, Géry Casiez, and Daniel Vogel. 2024. Direct- GPT: A Direct Manipulation Interface to Interact with Large Language Models. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 975, 16 pages. doi:10.1145/36...

    doi:10.1145/3613904.3642462 2024

  27. [27]

    2002.The Design and Evaluation of Multiple Interfaces: A Solution for Complex Software

    Joanna McGrenere. 2002.The Design and Evaluation of Multiple Interfaces: A Solution for Complex Software. Ph. D. Dissertation. University of Toronto

    2002

  28. [28]

    Brad A. Myers. 1998. A brief history of human-computer interaction technology. Interactions5, 2 (March 1998), 44–54. doi:10.1145/274430.274436

    doi:10.1145/274430.274436 1998

  29. [29]

    2026.Audio Routing, Remote Control, and Macro Con- trols

    Native Instruments. 2026.Audio Routing, Remote Control, and Macro Con- trols. https://www.native-instruments.com/ni-tech-manuals/maschine-plus- manual/en/audio-routing%2C-remote-control%2C-and-macro-controls.html

    2026

  30. [30]

    Ryogo Niwa, Shigeo Yoshida, Yuki Koyama, and Yoshitaka Ushiku. 2025. Cooper- ative Design Optimization through Natural Language Interaction. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST ’25). Association for Computing Machinery, New York, NY, USA, Article 121, 25 pages. doi:10.1145/3746059.3747789

    doi:10.1145/3746059.3747789 2025

  31. [31]

    Peter O’Donovan, Aseem Agarwala, and Aaron Hertzmann. 2015. DesignScape: Design with Interactive Layout Suggestions. InProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(Seoul, Republic of Korea)(CHI ’15). Association for Computing Machinery, New York, NY, USA, 1221–1224. doi:10.1145/2702123.2702149

    doi:10.1145/2702123.2702149 2015

  32. [32]

    Sharon Oviatt. 2006. Human-centered design meets cognitive load theory: de- signing interfaces that help people think. InProceedings of the 14th ACM Interna- tional Conference on Multimedia(Santa Barbara, CA, USA)(MM ’06). Association for Computing Machinery, New York, NY, USA, 871–880. doi:10.1145/1180639. 1180831

    work page doi:10.1145/1180639 2006

  33. [33]

    Michael Sedlmair, Miriah Meyer, and Tamara Munzner. 2012. Design Study Methodology: Reflections from the Trenches and the Stacks.IEEE Transactions on Visualization and Computer Graphics18, 12 (2012), 2431–2440. doi:10.1109/ TVCG.2012.213

    2012

  34. [34]

    Bernstein

    Omar Shaikh, Shardul Sapkota, Shan Rizvi, Eric Horvitz, Joon Sung Park, Diyi Yang, and Michael S. Bernstein. 2025. Creating General User Models from Com- puter Use. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST ’25). Association for Computing Machinery, New York, NY, USA, Article 35, 23 pages. doi:10.1145/3...

    doi:10.1145/3746059.3747722 2025

  35. [35]

    Yulin Shen, Yifei Shen, Jiawen Cheng, Chutian Jiang, Mingming Fan, and Zeyu Wang. 2024. Neural Canvas: Supporting Scenic Design Prototyping by Integrating 3D Sketching and Generative AI. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Articl...

    arXiv 2024

  36. [36]

    Xinyu Shi, Yinghou Wang, Yun Wang, and Jian Zhao. 2024. Piet: Facilitating Color Authoring for Motion Graphics Video. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 148, 17 pages. doi:10.1145/3613904.3642711

    doi:10.1145/3613904.3642711 2024

  37. [37]

    Ben Shneiderman. 1981. Direct manipulation: A step beyond programming languages (abstract only).SIGSOC Bull.13, 2–3 (May 1981), 143. doi:10.1145/ 1015579.810991

    arXiv 1981

  38. [38]

    2016.Designing the User Interface: Strategies for Effective Human–Computer Interaction(6 ed.)

    Ben Shneiderman, Catherine Plaisant, Maxine Cohen, Steven Jacobs, Niklas Elmqvist, and Nicholas Diakopoulos. 2016.Designing the User Interface: Strategies for Effective Human–Computer Interaction(6 ed.). Pearson

    2016

  39. [40]

    Ryo Suzuki, Rubaiat Habib Kazi, Li-yi Wei, Stephen DiVerdi, Wilmot Li, and Daniel Leithinger. 2020. RealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic Sketching. InProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology(Virtual Event, USA)(UIST ’20). Association for Computing Machinery, New...

    doi:10.1145/3379337.3415892 2020

  40. [41]

    2010.Designing Interfaces: Patterns for Effective Interaction Design

    Jenifer Tidwell. 2010.Designing Interfaces: Patterns for Effective Interaction Design. O’Reilly Media

    2010

  41. [42]

    Theophanis Tsandilas and m. c. schraefel. 2007. Bubbling menus: a selective mechanism for accessing hierarchical drop-down menus. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(San Jose, California, USA)(CHI ’07). Association for Computing Machinery, New York, NY, USA, 1195–1204. doi:10.1145/1240624.1240806

    work page doi:10.1145/1240624.1240806 2007

  42. [43]

    Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey Bigham, and Jeffrey Nichols. 2024. UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pa...

    doi:10.18653/v1/2024.naacl-long.417 2024

  43. [44]

    Haijun Xia, Tony Wang, Aditya Gunturu, Peiling Jiang, William Duan, and Xiaoshuo Yao. 2023. CrossTalk: Intelligent Substrates for Language-Oriented Interaction in Video-Based Communication and Collaboration. InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (San Francisco, CA, USA)(UIST ’23). Association for Computin...

    doi:10.1145/3586183.3606773 2023

  44. [45]

    Zhijie Xia, Kyzyl Monteiro, Kevin Van, and Ryo Suzuki. 2023. RealityCanvas: Augmented Reality Sketching for Embedded and Responsive Scribble Animation Effects. InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23). Association for Computing Machinery, New York, NY, USA, Article 115, 14 pages. doi:10.1145/35861...

    doi:10.1145/3586183.3606716 2023

  45. [46]

    Liwenhan Xie, Yanna Lin, Can Liu, Huamin Qu, and Xinhuan Shu. 2025. DataWink: Reusing and Adapting SVG-based Visualization Examples with Large Multimodal Models.IEEE Transactions on Visualization and Computer Graphics (2025), 1–11. doi:10.1109/TVCG.2025.3634635

    doi:10.1109/tvcg.2025.3634635 2025

  46. [47]

    Hui Ye, Chufeng Xiao, Jiaye Leng, Pengfei Xu, and Hongbo Fu. 2026. Mo- GraphGPT: Creating Interactive Scenes Using Modular LLM and Graphical Con- trol.IEEE Transactions on Visualization and Computer Graphics(2026), 1–16. doi:10.1109/TVCG.2026.3667904

    doi:10.1109/tvcg.2026.3667904 2026

  47. [48]

    Shengdong Zhao and Ravin Balakrishnan. 2004. Simple vs. compound mark hierarchical marking menus. InProceedings of the 17th Annual ACM Symposium on User Interface Software and Technology(Santa Fe, NM, USA)(UIST ’04). Association for Computing Machinery, New York, NY, USA, 33–42. doi:10.1145/1029632. 1029639

    work page doi:10.1145/1029632 2004

  48. [49]

    Yuheng Zhao, Xueli Shu, Liwen Fan, Lin Gao, Yu Zhang, and Siming Chen

  49. [50]

    ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent.IEEE Transactions on Visualization and Computer Graphics32, 1 (2026), 451–461. doi:10. 1109/TVCG.2025.3642628

    arXiv 2026

  50. [51]

    Chenfei Zhu, Shao-Kang Hsia, Xiyun Hu, Ziyi Liu, Jingyu Shi, and Karthik Ramani. 2025. agentAR: Creating Augmented Reality Applications with Tool- Augmented LLM-based Autonomous Agents. InProceedings of the 38th Annual ACM Symposium on User Interface Software and Technology (UIST ’25). Association for Computing Machinery, New York, NY, USA, Article 54, 23...

    arXiv 2025