The Attention-Aware Pipeline: Design Tensions from Making Attention Visible in XR

Arvind Srinivasan; Niklas Elmqvist

arxiv: 2606.03492 · v1 · pith:4HZXPRTFnew · submitted 2026-06-02 · 💻 cs.HC

The Attention-Aware Pipeline: Design Tensions from Making Attention Visible in XR

Arvind Srinivasan , Niklas Elmqvist This is my paper

Pith reviewed 2026-06-28 08:30 UTC · model grok-4.3

classification 💻 cs.HC

keywords attentionXRgaze visualizationdesign tensionsfeedback loopcollaborative interfacesdiminished realityeye tracking

0 comments

The pith

Making gaze visible in XR creates a feedback loop where visual responses alter what users attend to next and generate stage-specific design tensions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces the Attention-Aware Pipeline of Capture, Record, and Revisualize to frame how XR systems that expose gaze patterns produce closed loops of attention change. These loops turn each design choice at one stage into an input that reshapes attention at the next, creating predictable tensions. The authors trace the pipeline through three prototype roles for attention (mirror, medium, mediator) and show that each role surfaces a distinct tension the loop model anticipates. A formative eye-tracking study with musicians then confirms real problems of attentional tunneling and disconnection that the pipeline predicts. The work ends by proposing subtractive intervention as a targeted response to one of those tensions.

Core claim

The Attention-Aware Pipeline (Capture, Record, Revisualize) generates design tensions whose form depends on each stage's configuration because the system's visual response alters what users attend to next, triggering further responses; tracing the pipeline through mirror, medium, and mediator systems reveals tensions that motivate subtractive diminished-reality intervention.

What carries the argument

The Attention-Aware Pipeline (Capture, Record, Revisualize) whose closed feedback loop turns each stage's output into the next stage's input.

If this is right

Each choice of revisualization technique directly reshapes subsequent gaze patterns and must be evaluated for its effect on the loop.
Casting attention as mirror, medium, or mediator surfaces distinct tensions that follow from the same feedback structure.
Subtractive intervention through diminished reality offers one concrete way to reduce tunneling without adding new visual elements.
Formative eye-tracking studies of shared tasks can surface disconnection problems before full system deployment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same loop logic could be used to anticipate attention problems in non-musical XR collaboration such as remote repair or education.
Designers of any gaze-visible interface may need explicit tests for whether the revisualization itself narrows the user's field of view.
The pipeline framing suggests that attention-aware systems should be evaluated on the stability of their feedback loop rather than on isolated accuracy metrics.

Load-bearing premise

The three-stage pipeline and its predicted tensions accurately capture the causal dynamics of attention feedback in real XR use rather than serving only as a post-hoc lens on observed behaviors.

What would settle it

A controlled XR study implementing the full pipeline in a new collaborative task that produces no measurable attentional tunneling, disconnection, or stage-specific tensions matching the three reported forms.

Figures

Figures reproduced from arXiv: 2606.03492 by Arvind Srinivasan, Niklas Elmqvist.

**Figure 1.** Figure 1: The Attention-Aware Pipeline. Three stages process gaze data into visual interventions. The feedback arrow (bottom) is the source of design tensions: whatever Revisualize produces enters Capture as new perceptual content. Design questions at each stage (adapted from the 4W1H approach similar to prior works [4]) define the configuration space. We have been working on this problem across three projects, each… view at source ↗

**Figure 2.** Figure 2: PupilLabs Analysis Workflow. Multi-view snapshot from the band rehearsal showing dynamically tracked areas of interest (AOIs) across four musicians. Bounding boxes are generated via automated detection and manually refined to support gaze-to-target mapping. The interface illustrates the annotation workflow used to construct dynamic, person-centered AOIs required for analyzing attention directed at moving s… view at source ↗

read the original abstract

Where people look during shared activity carries coordination cues that speech and gesture cannot replace, but these patterns remain invisible to participants. XR headsets make gaze available as real-time input, yet few systems feed it back visually. We frame our work using the Attention-Aware Pipeline (Capture, Record, Revisualize), whose feedback loop means the systems visual response alters what users attend to next, triggering further responses. This generates design tensions whose form depends on each stages configuration. We trace the pipeline through three systems casting attention as a mirror (reflecting gaze history), a medium (sharing it across collaborators), and a mediator (intervening through diminished reality). Each encountered a tension the loop predicted, motivating the next. A formative eye-tracking study of four musicians surfaced attentional tunneling and near-total disconnection, confirming the need for intervention. We present these tensions and a next step: testing whether subtractive intervention reduces tunneling for a single sight-reader.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper gives a clean three-stage pipeline for gaze feedback in XR but treats the feedback loop as predictive when the evidence is only post-hoc tracing of prototypes built inside that frame.

read the letter

The main takeaway is that this work introduces the Attention-Aware Pipeline (Capture, Record, Revisualize) as an organizing frame for how making gaze visible in XR creates a feedback loop and resulting design tensions. They map the stages onto three concrete roles—mirror, medium, and mediator—and show how each of three built systems ran into a tension the pipeline describes.

What the paper does well is actually implement the three systems and run a small formative eye-tracking study with four musicians. The observations of attentional tunneling and near-total disconnection are concrete and point to a real usability issue when attention cues are added to shared activity. That part feels like it comes from hands-on design work rather than pure abstraction.

The soft spot is the predictive status given to the loop. The systems were developed within the same framing, so the claim that the pipeline “predicts” the tensions they encountered reads as retrospective description rather than an independent test. The study has no manipulation of stage parameters, no separate measure of attention change, and n=4 with no quantitative results or controls. Without that, the loop risks being circular by construction, as the stress-test note flags.

This is for HCI researchers working on collaborative XR tools who need a way to think through gaze visualization choices. A reader already in that subfield could get practical value from the three examples and the identified tensions. It is not positioned as a broad theoretical advance.

I would bring it to a reading group as maybe, to talk through whether the pipeline can be turned into something falsifiable. I would not cite it in the next year on current evidence. It deserves peer review because the prototypes and study give reviewers something tangible to evaluate, even if the central loop claim needs more empirical grounding.

Referee Report

3 major / 2 minor

Summary. The paper introduces the Attention-Aware Pipeline (Capture, Record, Revisualize) as a framework whose feedback loop causes visual responses to alter subsequent user attention, thereby generating design tensions whose form depends on each stage's configuration. It illustrates the pipeline by tracing three XR prototypes that cast attention as a mirror (reflecting gaze history), a medium (sharing across collaborators), and a mediator (intervening via diminished reality), each encountering a tension the loop predicted. A formative eye-tracking study with four musicians observed attentional tunneling and near-total disconnection, confirming the need for intervention and motivating a next step of testing subtractive intervention for sight-readers.

Significance. If the pipeline holds as a model of causal dynamics, it supplies a structured lens for anticipating configuration-dependent issues when visualizing attention in collaborative XR, with potential to guide designs that mitigate tunneling or disconnection. The work's strengths lie in its clear three-stage staging, the logical progression across the three prototypes, and the explicit identification of tensions arising from the feedback loop; these elements provide a reusable organizing device even if the predictive claims require further grounding.

major comments (3)

[Abstract and Pipeline introduction] Abstract and the section introducing the Attention-Aware Pipeline: the claim that the feedback loop 'generates design tensions whose form depends on each stage's configuration' is presented as both descriptive and predictive, yet the only evidence consists of post-hoc tracing of three systems developed inside the same framing plus a formative study; no a priori predictions, controlled variation of stage parameters, or independent measures of attention change (separate from the visualization) are reported, so the loop risks functioning as a retrospective lens rather than a generator of falsifiable dynamics.
[Formative eye-tracking study] Formative eye-tracking study section: the study with n=4 musicians is cited as confirming the need for intervention after observing tunneling and disconnection, but it supplies no quantitative results, error analysis, manipulation of Capture/Record/Revisualize parameters, or test of whether the observed behaviors were produced by the loop itself rather than other factors; this limits its support for the central predictive claim.
[Tracing the three systems] Section tracing the three systems (mirror/medium/mediator): each is said to have 'encountered a tension the loop predicted,' but the manuscript does not specify how those tensions were anticipated before prototype development or how differing stage configurations were shown to produce distinct tension forms, leaving the configuration-dependence claim without direct empirical mapping.

minor comments (2)

[Abstract and study description] The abstract and study description should clarify the exact quantitative or qualitative measures used to identify 'near-total disconnection' and 'attentional tunneling' so readers can assess the strength of the observational data.
[Pipeline and prototype sections] Notation for the three stages (Capture, Record, Revisualize) is introduced without an accompanying diagram or table that explicitly maps each prototype's implementation choices onto the stages; adding such a mapping would improve traceability of the claimed configuration-dependent tensions.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. The work is exploratory and uses the Attention-Aware Pipeline as a conceptual organizing device derived from design practice rather than a validated predictive model. We address each major comment below with planned revisions to clarify scope and claims.

read point-by-point responses

Referee: The claim that the feedback loop 'generates design tensions whose form depends on each stage's configuration' is presented as both descriptive and predictive, yet the only evidence consists of post-hoc tracing of three systems developed inside the same framing plus a formative study; no a priori predictions, controlled variation of stage parameters, or independent measures of attention change are reported.

Authors: We agree the current phrasing risks implying predictive power. The pipeline was developed iteratively from our design process to surface and organize observed tensions; the three prototypes were not constructed to test a priori predictions from the framework. We will revise the abstract and pipeline introduction to frame it explicitly as a descriptive and generative lens for anticipating configuration-dependent issues in future XR attention systems, removing any suggestion of falsifiable dynamics without further empirical work. revision: yes
Referee: The study with n=4 musicians is cited as confirming the need for intervention after observing tunneling and disconnection, but it supplies no quantitative results, error analysis, manipulation of Capture/Record/Revisualize parameters, or test of whether the observed behaviors were produced by the loop itself.

Authors: The study is explicitly formative and was conducted to surface real attentional patterns in a music context that motivated our intervention designs. It does not claim to validate the pipeline's causal claims or manipulate its stages. We will expand the section to state its limitations and purpose more explicitly. No quantitative data or parameter manipulations exist to add. revision: partial
Referee: Each system is said to have 'encountered a tension the loop predicted,' but the manuscript does not specify how those tensions were anticipated before prototype development or how differing stage configurations were shown to produce distinct tension forms.

Authors: The tracing section is retrospective, documenting how the pipeline helped articulate tensions that emerged during sequential prototype development. No controlled mapping of configurations to tension forms was performed. We will revise the section to clarify the post-hoc nature of the analysis and remove language suggesting a priori prediction, while retaining the pipeline's value as a reusable organizing device. revision: yes

Circularity Check

0 steps flagged

No significant circularity; pipeline is a retrospective organizing lens on prototypes

full rationale

The manuscript introduces the Attention-Aware Pipeline (Capture-Record-Revisualize) as a framing device whose feedback loop is posited to generate configuration-dependent design tensions. Three prototype systems are then traced through this framing, with the text noting that each 'encountered a tension the loop predicted.' However, no equations, parameter fits, or self-citations are present that would reduce any claimed prediction to its own inputs by construction. The pipeline functions as a post-hoc conceptual organizer for observed prototype behaviors rather than a deductive chain whose outputs are definitionally equivalent to its premises. The small formative study (n=4) is presented separately as motivation for intervention and does not serve as a fitted input renamed as prediction. No load-bearing self-citation or uniqueness theorem is invoked. The derivation chain therefore remains self-contained as design research and does not meet the criteria for circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The work introduces the Attention-Aware Pipeline as a new conceptual entity without external empirical grounding beyond the described prototypes. No free parameters are fitted. Domain assumptions include that gaze carries unique coordination value and that visual feedback will reliably alter subsequent attention.

axioms (2)

domain assumption Gaze patterns during shared activity carry coordination cues that speech and gesture cannot replace.
Stated in the opening sentence of the abstract as the motivating premise.
domain assumption XR headsets make gaze available as real-time input that can be fed back visually.
Assumed as technical capability enabling the pipeline.

invented entities (1)

Attention-Aware Pipeline (Capture, Record, Revisualize) no independent evidence
purpose: Organizing framework that predicts design tensions from feedback loops in attention visualization.
Newly introduced in the abstract as the central framing device; no independent evidence outside the three systems is provided.

pith-pipeline@v0.9.1-grok · 5691 in / 1414 out tokens · 17060 ms · 2026-06-28T08:30:04.129108+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

17 extracted references · 13 canonical work pages

[1]

Srinivasan, J

A. Srinivasan, J. Ellemose, P. W. S. Butcher, P. D. Ritsos, N. Elmqvist, Attention-aware visualization: Tracking and responding to user perception over time, IEEE Transactions on Visualization and Computer Graphics 31 (2025) 1017–1027. doi:10.1109/TVCG.2024.3456300

work page doi:10.1109/tvcg.2024.3456300 2025
[2]

Srinivasan, N

A. Srinivasan, N. Elmqvist, HeedVision: Attention awareness in collaborative immersive analytics environments, arXiv preprint arXiv:2505.07069 (2025)

Pith/arXiv arXiv 2025
[3]

S. Mori, S. Ikeda, H. Saito, A survey of diminished reality: Techniques for visually concealing, elim- inating, and seeing through real objects, IPSJ Transactions on Computer Vision and Applications 9 (2017) 1–14. doi:10.1186/s41074-017-0028-1

work page doi:10.1186/s41074-017-0028-1 2017
[4]

S. Shin, A. Batch, P. W. S. Butcher, P. D. Ritsos, N. Elmqvist, The Reality of the Situation: A Survey of Situated Analytics, IEEE TVCG (2023). doi:10.1109/TVCG.2023.3285546, to appear

work page doi:10.1109/tvcg.2023.3285546 2023
[5]

Lebeck, K

K. Lebeck, K. Ruth, T. Kohno, F. Roesner, Towards security and privacy for multi-user augmented reality: Foundations with end users, in: Proc. IEEE Symposium on Security and Privacy, IEEE, 2018, pp. 392–408. doi:10.1109/SP.2018.00051

work page doi:10.1109/sp.2018.00051 2018
[6]

J. A. Easterbrook, The effect of emotion on cue utilization and the organization of behavior, Psychological Review 66 (1959) 183–201

1959
[7]

W. C. Hill, J. D. Hollan, D. Wroblewski, T. McCandless, Edit wear and read wear, in: Proc. ACM CHI, ACM, New York, NY, USA, 1992, pp. 3–9. doi:10.1145/142750.142751

work page doi:10.1145/142750.142751 1992
[8]

L. E. Matzen, M. J. Haass, K. M. Divis, Z. Wang, A. T. Wilson, Data visualization saliency model: A tool for evaluating abstract data visualizations, IEEE TVCG 24 (2018) 563–573. doi: 10.1109/ TVCG.2017.2743939

arXiv 2018
[9]

Steichen, G

B. Steichen, G. Carenini, C. Conati, User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities, in: Proceedings of the 2013 International Conference on Intelligent User Interfaces, IUI ’13, ACM, New York, NY, USA, 2013, p. 317–328. doi:10.1145/2449396.2449439

work page doi:10.1145/2449396.2449439 2013
[10]

Grassé, La reconstruction du nid et les coordinations interindividuelles chez Bellicositermes natalensis et Cubitermes sp

P.-P. Grassé, La reconstruction du nid et les coordinations interindividuelles chez Bellicositermes natalensis et Cubitermes sp. la théorie de la stigmergie: essai d’interprétation du comportement des termites constructeurs, Insectes Sociaux 6 (1959) 41–80. doi:10.1007/BF02223791

work page doi:10.1007/bf02223791 1959
[11]

Herling, W

J. Herling, W. Broll, Advanced self-contained object removal for realizing real-time diminished reality in unconstrained environments, in: Proc. IEEE ISMAR, IEEE, 2010, pp. 207–212. doi: 10. 1109/ISMAR.2010.5643572

arXiv 2010
[12]

Yolov8: A novel object detection algorithm with enhanced performance and robustness

R. Varghese, S. M., Yolov8: A novel object detection algorithm with enhanced performance and robustness, in: 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), 2024, pp. 1–6. doi:10.1109/ADICS58448.2024.10533619

work page doi:10.1109/adics58448.2024.10533619 2024
[13]

Sato, et al., Gaze transition entropy as a measure of attention allocation in a dynamic workspace involving automation, Scientific Reports 14 (2024) 74244

N. Sato, et al., Gaze transition entropy as a measure of attention allocation in a dynamic workspace involving automation, Scientific Reports 14 (2024) 74244. doi: 10.1038/s41598-024-74244-4

work page doi:10.1038/s41598-024-74244-4 2024
[14]

G. R. Dirkin, Cognitive tunneling: Use of visual information under stress, Perceptual and Motor Skills 56 (1983) 191–198. doi:10.2466/pms.1983.56.1.191

work page doi:10.2466/pms.1983.56.1.191 1983
[15]

Paprocki, A

R. Paprocki, A. Lenskiy, What does eye-blink rate variability dynamics tell us about cognitive performance?, Frontiers in Human Neuroscience 11 (2017) 620. doi: 10.3389/fnhum.2017. 00620

work page doi:10.3389/fnhum.2017 2017
[16]

Kopiez, J

R. Kopiez, J. I. Lee, Towards a general model of skills involved in sight reading music, Music Education Research 10 (2008) 41–62. doi:10.1080/14613800701871363

work page doi:10.1080/14613800701871363 2008
[17]

Madell, S

J. Madell, S. Hébert, Eye movements and music reading: Where do we look next?, Music Perception 26 (2008) 157–170. doi:10.1525/mp.2008.26.2.157

work page doi:10.1525/mp.2008.26.2.157 2008

[1] [1]

Srinivasan, J

A. Srinivasan, J. Ellemose, P. W. S. Butcher, P. D. Ritsos, N. Elmqvist, Attention-aware visualization: Tracking and responding to user perception over time, IEEE Transactions on Visualization and Computer Graphics 31 (2025) 1017–1027. doi:10.1109/TVCG.2024.3456300

work page doi:10.1109/tvcg.2024.3456300 2025

[2] [2]

Srinivasan, N

A. Srinivasan, N. Elmqvist, HeedVision: Attention awareness in collaborative immersive analytics environments, arXiv preprint arXiv:2505.07069 (2025)

Pith/arXiv arXiv 2025

[3] [3]

S. Mori, S. Ikeda, H. Saito, A survey of diminished reality: Techniques for visually concealing, elim- inating, and seeing through real objects, IPSJ Transactions on Computer Vision and Applications 9 (2017) 1–14. doi:10.1186/s41074-017-0028-1

work page doi:10.1186/s41074-017-0028-1 2017

[4] [4]

S. Shin, A. Batch, P. W. S. Butcher, P. D. Ritsos, N. Elmqvist, The Reality of the Situation: A Survey of Situated Analytics, IEEE TVCG (2023). doi:10.1109/TVCG.2023.3285546, to appear

work page doi:10.1109/tvcg.2023.3285546 2023

[5] [5]

Lebeck, K

K. Lebeck, K. Ruth, T. Kohno, F. Roesner, Towards security and privacy for multi-user augmented reality: Foundations with end users, in: Proc. IEEE Symposium on Security and Privacy, IEEE, 2018, pp. 392–408. doi:10.1109/SP.2018.00051

work page doi:10.1109/sp.2018.00051 2018

[6] [6]

J. A. Easterbrook, The effect of emotion on cue utilization and the organization of behavior, Psychological Review 66 (1959) 183–201

1959

[7] [7]

W. C. Hill, J. D. Hollan, D. Wroblewski, T. McCandless, Edit wear and read wear, in: Proc. ACM CHI, ACM, New York, NY, USA, 1992, pp. 3–9. doi:10.1145/142750.142751

work page doi:10.1145/142750.142751 1992

[8] [8]

L. E. Matzen, M. J. Haass, K. M. Divis, Z. Wang, A. T. Wilson, Data visualization saliency model: A tool for evaluating abstract data visualizations, IEEE TVCG 24 (2018) 563–573. doi: 10.1109/ TVCG.2017.2743939

arXiv 2018

[9] [9]

Steichen, G

B. Steichen, G. Carenini, C. Conati, User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities, in: Proceedings of the 2013 International Conference on Intelligent User Interfaces, IUI ’13, ACM, New York, NY, USA, 2013, p. 317–328. doi:10.1145/2449396.2449439

work page doi:10.1145/2449396.2449439 2013

[10] [10]

Grassé, La reconstruction du nid et les coordinations interindividuelles chez Bellicositermes natalensis et Cubitermes sp

P.-P. Grassé, La reconstruction du nid et les coordinations interindividuelles chez Bellicositermes natalensis et Cubitermes sp. la théorie de la stigmergie: essai d’interprétation du comportement des termites constructeurs, Insectes Sociaux 6 (1959) 41–80. doi:10.1007/BF02223791

work page doi:10.1007/bf02223791 1959

[11] [11]

Herling, W

J. Herling, W. Broll, Advanced self-contained object removal for realizing real-time diminished reality in unconstrained environments, in: Proc. IEEE ISMAR, IEEE, 2010, pp. 207–212. doi: 10. 1109/ISMAR.2010.5643572

arXiv 2010

[12] [12]

Yolov8: A novel object detection algorithm with enhanced performance and robustness

R. Varghese, S. M., Yolov8: A novel object detection algorithm with enhanced performance and robustness, in: 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), 2024, pp. 1–6. doi:10.1109/ADICS58448.2024.10533619

work page doi:10.1109/adics58448.2024.10533619 2024

[13] [13]

Sato, et al., Gaze transition entropy as a measure of attention allocation in a dynamic workspace involving automation, Scientific Reports 14 (2024) 74244

N. Sato, et al., Gaze transition entropy as a measure of attention allocation in a dynamic workspace involving automation, Scientific Reports 14 (2024) 74244. doi: 10.1038/s41598-024-74244-4

work page doi:10.1038/s41598-024-74244-4 2024

[14] [14]

G. R. Dirkin, Cognitive tunneling: Use of visual information under stress, Perceptual and Motor Skills 56 (1983) 191–198. doi:10.2466/pms.1983.56.1.191

work page doi:10.2466/pms.1983.56.1.191 1983

[15] [15]

Paprocki, A

R. Paprocki, A. Lenskiy, What does eye-blink rate variability dynamics tell us about cognitive performance?, Frontiers in Human Neuroscience 11 (2017) 620. doi: 10.3389/fnhum.2017. 00620

work page doi:10.3389/fnhum.2017 2017

[16] [16]

Kopiez, J

R. Kopiez, J. I. Lee, Towards a general model of skills involved in sight reading music, Music Education Research 10 (2008) 41–62. doi:10.1080/14613800701871363

work page doi:10.1080/14613800701871363 2008

[17] [17]

Madell, S

J. Madell, S. Hébert, Eye movements and music reading: Where do we look next?, Music Perception 26 (2008) 157–170. doi:10.1525/mp.2008.26.2.157

work page doi:10.1525/mp.2008.26.2.157 2008