Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Comparing Pass-Through Quality of Mixed Reality Devices: A User Experience Study During Real-World Tasks

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In a 31-participant study, the Apple Vision Pro rated highest on every pass-through quality measure compared with the Meta Quest 3 and Varjo XR-3, while also producing the lowest task load and cybersickness.

desk verdict Useful first comparative pass-through data, but the headline claim about AVP beating Quest 3 outruns the statistics. read the letter →

arxiv 2502.06382 v1 pith:2PX7FFA3 submitted 2025-02-10 cs.HC

Francesco Vona , Julia Schorlemmer , Michael Stern , Navid Ashrafi , Maurizio Vergari , Tanja Kojic , Jan-Niklas Voigt-Antons This is my paper · ORCID
classification cs.HC
keywords pass-throughmixedrealityuserstudyAppleVisionProMetaQuest3VarjoXR-3cybersicknesstaskload
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether modern mixed-reality headsets with video pass-through let users perform ordinary real-world tasks without extra effort or discomfort. Thirty-one participants read texts and solved puzzles while wearing the Apple Vision Pro, Meta Quest 3, and Varjo XR-3, and also completed the tasks with no headset as a baseline. The authors report that the Apple Vision Pro produced the lowest task load, the least cybersickness, and the highest subjective scores on all six pass-through quality dimensions: clarity, resolution, color accuracy, depth perception, environmental awareness, and distortion. The paper argues that pass-through quality drives usability and comfort, and that the Vision Pro's combination of display and tracking makes it the most suitable of the three for sustained practical use.

What carries the argument

The argument rests on a within-subjects experimental design: each of the 31 participants performs the same two tasks, reading aloud and solving a puzzle, under four conditions (no headset, Apple Vision Pro, Meta Quest 3, and Varjo XR-3) with randomized order to minimize order effects. The measured outcomes are the NASA-TLX workload score, the CSQ-VR cybersickness questionnaire, and six custom pass-through quality ratings for clarity, resolution, color accuracy, depth perception, environmental awareness, and distortion. Repeated-measures ANOVAs test for condition effects, and the mean score order across devices is the evidence for the reported ranking.

What would settle it

Run the same three-headset comparison in a double-blind protocol where the vendor identity is concealed, and check the subjective ratings against objective pass-through measurements such as latency, resolution, color accuracy, and distortion. If the Vision Pro's subjective edge shrinks or reverses when branding is hidden, or if objective measurements do not match the subjective ranking, the central claim that pass-through quality causes the lower workload and cybersickness would be undercut.

Watch

Extended reading notes

Core claim

The study's central claim is that, among three commercially available video see-through headsets, the Apple Vision Pro offers the best pass-through experience for seated real-world tasks. On the NASA-TLX workload measure, the Vision Pro scored lowest for both reading (M = 18.84) and puzzle solving (M = 12.71), and on the CSQ-VR cybersickness measure it produced the lowest discomfort (M = 9.68), followed by Meta Quest 3 and then Varjo XR-3. On the authors' custom six-item pass-through quality scale, the Vision Pro received the highest ratings for every dimension. From this the paper concludes that high-quality pass-through reduces cognitive load and discomfort, and that this is a key factor for adopting extended reality in healthcare, education, and similar settings.

Load-bearing premise

The load-bearing premise is that the six custom, unvalidated pass-through quality questions measure what they claim to measure, and that participants' ratings reflected the headset display rather than familiarity, ergonomic differences, or the order in which devices were tried.

Editorial extensions

If this is right

  • If the ranking holds, organizations choosing headsets for prolonged real-world work in healthcare or education would get measurably lower operator fatigue and discomfort from the Apple Vision Pro than from the Meta Quest 3 or Varjo XR-3.
  • The alignment of workload and cybersickness scores with pass-through quality scores supports the idea that visual fidelity is a primary driver of comfort in mixed-reality headsets.
  • The Meta Quest 3 emerges as a moderate middle-ground device, needing refinements in image processing or ergonomics to close the gap on task load and motion discomfort.
  • The Varjo XR-3's high-end display specifications do not by themselves translate into better subjective pass-through, suggesting that ergonomics and image-processing pipeline matter as much as raw resolution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the Apple Vision Pro's advantage may partly reflect its newer hardware generation and far higher price; an unstated corollary is that pass-through quality may track device generation more than display resolution alone.
  • Because the custom quality ratings were never validated against objective measurements, a natural extension is to correlate subjective clarity, resolution, and color ratings with measurable device properties such as camera resolution, latency, color gamut, and distortion maps.
  • The seated reading and puzzle tasks may understate differences that would appear in standing, walking, or socially interactive use; testing pass-through during locomotion or object manipulation would probe how general the ranking is.
  • A blinded protocol, in which participants do not know which headset they are wearing, could reveal how much of the Vision Pro's lead comes from the actual display versus brand expectations or physical comfort differences.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This manuscript reports a within-subjects user study (N = 31) comparing the pass-through quality of three mixed-reality headsets — Apple Vision Pro, Meta Quest 3, and Varjo XR-3 — plus a no-headset baseline condition, during two real-world tasks: reading a text aloud and solving a puzzle. Participants rated each condition on the NASA-TLX workload questionnaire, the CSQ-VR cybersickness questionnaire, and six custom pass-through quality metrics (clarity, resolution, color accuracy, depth perception, environmental awareness, distortion). The authors report that the Apple Vision Pro recorded the lowest task load and cybersickness scores and the highest pass-through quality ratings on all six dimensions, and they conclude that the Vision Pro outperformed the Quest 3 and Varjo XR-3. The central empirical claim is therefore that the Vision Pro delivers measurably better pass-through quality and user comfort than the other two headsets, with implications for applications in education, healthcare, and manufacturing.

Significance. The question is timely: comparative, task-based pass-through quality data on three current commercial headsets, including the recently released Apple Vision Pro, are scarce, and a controlled within-subjects study with ecologically plausible tasks is a useful contribution to the XR evaluation literature. The authors deserve credit for a clean study design — randomized condition order, a roughly gender-balanced sample (18 male, 13 female), validated instruments for workload (NASA-TLX) and cybersickness (CSQ-VR), and a controlled laboratory protocol with per-participant headset cleaning. If the comparative claim survives proper pairwise testing, the paper would give practitioners an actionable ranking of the three devices for pass-through-dependent applications. However, the headline claim is currently not backed by the statistics actually reported, so the paper's significance depends on the authors completing the inferential analysis rather than on the data as presented.

major comments (3)
  1. [§3, Pass-Through Quality paragraph] The central claim of the abstract — that the Apple Vision Pro 'outperformed the Meta Quest 3 and Varjo XR-3, receiving the highest ratings for pass-through quality' — is not supported by the statistics reported. The Pass-Through Quality paragraph gives means for the Vision Pro only (clarity M = 5.39, resolution M = 5.55, environmental awareness M = 5.58) and then states that 'Repeated-measures ANOVA for pass-through dimensions revealed significant differences as well,' without F statistics, effect sizes, means or standard deviations for the Quest 3 and Varjo XR-3, or any post-hoc tests. An omnibus ANOVA only rejects the global null; with three headsets, the significance could be driven entirely by Varjo's low scores while the Vision Pro and Quest 3 are statistically indistinguishable. The abstract's specific 'outperformed the Meta Quest 3' assertion requires a significant pairwise contrast, which is never reported. The authors should report full descriptive statistics for all six metrics and all three devices, and add pairwise post-hoc comparisons with correction for multiple comparisons (or nonparametric equivalents, given the ordinal rating scale).
  2. [§3, Task Load and Cybersickness paragraphs] The Discussion draws comparative conclusions about all three devices — for example, that the Quest 3 'demonstrated a balanced performance' and that the Varjo XR-3 imposes 'higher cognitive demands' — but the Task Load and Cybersickness paragraphs report only omnibus ANOVAs (F(3,90) = 28.12 and F(3,90) = 16.11) with no post-hoc tests. For the reading task, the Vision Pro and Quest 3 means are very close (M = 18.84, SD = 8.66 versus M = 19.26, SD = 8.17), so without pairwise tests the claim that the Vision Pro has 'the lowest task load' relative to the Quest 3 is not established. The same applies to the cybersickness comparison, where the Vision Pro–Quest 3 difference (M = 9.68 versus 12.68) may or may not reach significance after correction for multiple comparisons.
  3. [§2, Pass-Through Quality Metrics] The six pass-through quality metrics (clarity, resolution, color accuracy, depth perception, environmental awareness, distortion) are described as 'custom metrics,' but no item wording, response scale, reliability information, or validity evidence is provided, and no reference is given for their construction. Because these ratings are the direct basis of the paper's headline claim, the reader cannot determine what participants actually assessed or whether the dimensions are measured independently. The authors should provide the exact questionnaire items, the response scale, and a brief justification or validation of the dimensions, or alternatively state explicitly that these are single-item subjective ratings introduced for this study and interpret them with that caveat.
minor comments (6)
  1. [§2 Methods and §3 Results] Section 2 states that a fourth baseline condition (no headset) was included and that conditions were randomized, and the ANOVAs use df = (3, 90), consistent with four conditions; however, no baseline results are reported anywhere in Section 3 or Figure 2, so the reader cannot gauge how much each headset degrades task load or increases cybersickness relative to the no-device reference. The authors should either report the baseline descriptives or state why they were excluded.
  2. [§3, Cybersickness paragraph] The sentence 'Repeated-measures ANOVA revealed significant differences in task load across conditions' in the Cybersickness paragraph appears to be a copy-paste error and should read 'cybersickness across conditions.'
  3. [Abstract and §2] Minor typos: 'Thirtyone' in the abstract should be 'Thirty-one,' 'receving' in Section 2 should be 'receiving,' and 'V arjo XR-3' in Section 2 contains a stray formatting space.
  4. [§2 Pass-Through Quality Metrics and §3 Results] The response scale of the pass-through quality items is never specified; means around 5.4–5.6 suggest a 7-point Likert scale, but this should be stated explicitly, along with whether each dimension was a single item or a multi-item subscale.
  5. [Figure 2] Figure 2 shows box-plots for the NASA-TLX and CSQ-VR scores only; a comparable visualization or table for the six pass-through metrics across the three headsets is needed to support the discussion's comparative statements.
  6. [§2 Methods and §3 Results] Instrument scoring details are missing: it is not stated whether the raw or weighted NASA-TLX version was administered (the two yield different totals), and although the CSQ-VR has three subscales (nausea, vestibular, oculomotor) that are listed in the methods, only the total score is reported in the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports empirical participant ratings and contains no derivation chain, fitted parameters, or self-citation that reduces its claims to their inputs.

full rationale

The paper is an empirical user experience study, not a derivation. Its central claim that the Apple Vision Pro received the highest pass-through quality ratings is a direct summary of self-reported questionnaire data collected under the stated protocol. There are no equations relating the pass-through quality metrics to each other, no fitted parameters renamed as predictions, and no modeled quantity whose definition presupposes the outcome. The custom pass-through quality metrics are introduced as rating scales and used as outcome measures; whether they validly capture pass-through quality is a measurement-validity concern, not a circularity concern. The manuscript also does not rely on a load-bearing self-citation: the cited prior work is background literature, and the comparisons among headsets are presented as new empirical results. The absence of pairwise post-hoc tests and unvalidated scales are statistical and methodological limitations, but they do not make the claimed result equivalent to its inputs by construction. Hence no circular step is present and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

This is an empirical study with no free parameters or invented entities. It relies on the validity of self-report questionnaires and standard statistical assumptions.

assumptions (3)
  • standard math Repeated-measures ANOVA assumptions (sphericity, normality) hold for the TLX, CSQ-VR, and custom quality ratings.
    The paper reports F-tests without checking or reporting ANOVA assumptions; if violated, the reported p-values are unreliable.
  • domain assumption The custom pass-through quality metrics are valid operationalizations of pass-through quality.
    The metrics are introduced without validation, so the central quality comparison depends on this assumption.
  • domain assumption The devices were set up equivalently (firmware, pass-through mode, IPD, lighting) across participants.
    The paper does not describe device configuration controls, so differences in setup could confound the comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparing Pass-Through Quality of Mixed Reality Devices: A User Experience Study During Real-World Tasks." pith.science (2026). https://pith.science/paper/2PX7FFA3

@misc{pith2026250206382,
  author       = {Pith},
  title        = {Pith review of: Comparing Pass-Through Quality of Mixed Reality Devices: A User Experience Study During Real-World Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2PX7FFA3}},
  note         = {Machine review of arXiv:2502.06382}
}
read the original abstract

In extended reality, pass-through enables users to view their real-world surroundings via cameras on the headset, displaying live video inside the device. This study compared the pass-through quality of three devices: Apple Vision Pro, Meta Quest 3, and Varjo XR3. Thirtyone participants performed two tasks, reading a text and solving a puzzle, while using each headset with the pass-through feature activated. Participants then rated their experiences, focusing on workload and cybersickness. Results showed that the Apple Vision Pro outperformed the Meta Quest 3 and Varjo XR3, receiving the highest ratings for pass-through quality.

Figures

Figures reproduced from arXiv: 2502.06382 by the authors.

Figure 1
Figure 1. Participant engaged in the puzzle task (left) and the reading [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Box-plots showing the distribution of total NASA TLX scores and CSQ-VR scores across the different conditions. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Recent Advances and Future Directions in Extended Reality (XR): Exploring AI-Powered Spatial Intelligence

    cs.HC 2025-04 unverdicted

    A survey-style preprint describes XR hardware, software, and products and argues that multi-modal AI and IoT digital twins will drive future spatial intelligence.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Banquiero, G

    M. Banquiero, G. Valdeolivas, D. Ram ´on, and M.-C. Juan. A color passthrough mixed reality application for learning piano. Virtual Real., 28(2), mar 2024. doi: 10.1007/s10055-024-00953-w 1

    work page doi:10.1007/s10055-024-00953-w 2024

  2. [2]

    Banquiero, G

    M. Banquiero, G. Valdeolivas, S. Trincado, N. Garc´ıa, and M.-C. Juan. Passthrough mixed reality with oculus quest 2: A case study on learn- ing piano. IEEE MultiMedia , 30(2):60–69, apr 2023. doi: 10.1109/ MMUL.2022.3232892 1

    arXiv 2023

  3. [3]

    Chaurasia, A

    G. Chaurasia, A. Nieuwoudt, A.-E. Ichim, R. Szeliski, and A. Sorkine- Hornung. Passthrough+ real-time stereoscopic view synthesis for mo- bile mixed reality. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 3(1):1–17, 2020. 1

    work page 2020

  4. [4]

    Apple Vision Pro for Healthcare: "The Ultimate Display"? -- Entering the Wonderland of Precision Medicine

    J. Egger, C. Gsaxner, X. Chen, J. Bian, J. Kleesiek, and B. Puladi. Apple vision pro for healthcare:” the ultimate display”?arXiv preprint arXiv:2308.04313, 2023. 1, 2

    work page Pith review arXiv 2023

  5. [5]

    A. L. S. Fernandes and S. K. Feiner. Combating vr sickness through subtle dynamic field-of-view modification. In 2016 IEEE Symposium on 3D User Interfaces (3DUI) , pp. 201–210. IEEE, 2016. 2

    work page 2016

  6. [6]

    S. G. Hart and L. E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. In P. A. Han- cock and N. Meshkati, eds., Human Mental Workload, vol. 52 of Ad- vances in Psychology , pp. 139–183. North-Holland, 1988. doi: 10. 1016/S0166-4115(08)62386-9 1

    work page 1988

  7. [7]

    Ishihara, H

    A. Ishihara, H. Aga, Y . Ishihara, H. Ichikawa, H. Kaji, K. Kawasaki, D. Kobayashi, T. Kobayashi, K. Nishida, T. Hamasaki, H. Mori, and Y . Morikubo. Integrating both parallax and latency compensation into video see-through head-mounted display. IEEE Transactions on Vi- sualization and Computer Graphics , 29(5):2826–2836, 2023. doi: 10 .1109/TVCG.2023.3249578 1

    arXiv 2023

  8. [8]

    Y . Itoh, T. Langlotz, J. Sutton, and A. Plopski. Towards indistin- guishable augmented reality: A survey on optical see-through head- mounted displays. ACM Computing Surveys, 2021. 1

    work page 2021

Show all 19 references
  1. [9]

    G. A. Koulieris, K. Aks ¸it, M. Stengel, R. K. Mantiuk, K. Mania, and C. Richardt. Near-eye display and tracking technologies for virtual and augmented reality. Computer Graphics F orum, 38(2):493–519,

  2. [10]

    Kourtesis, J

    P. Kourtesis, J. Linnell, R. Amir, F. Argelaguet, and S. E. MacPher- son. Cybersickness in virtual reality questionnaire (csq-vr): A valida- tion and comparison against ssq and vrsq. Virtual Worlds, 2(1):16–35,

  3. [11]

    G. Kuo, E. Penner, S. Moczydlowski, A. Ching, D. Lanman, and N. Matsuda. Perspective-correct vr passthrough without reprojection. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1–9, 2023. 1

    2023

  4. [12]

    N. Kyaw, M. Gu, E. Croft, and A. Cosgun. Comparing usability of augmented reality and virtual reality for creating virtual bound- ing boxes of real objects. Applied Sciences , 13(21), 2023. doi: 10. 3390/app132111693 1

    2023

  5. [13]

    McGill, G

    M. McGill, G. Li, A. Ng, L. Bajorunaite, J. Williamson, F. Pollick, and S. Brewster. Augmented, virtual and mixed reality passenger experi- ences. In Studies in Computational Intelligence , Studies in compu- tational intelligence, pp. 445–475. Springer International Publishing...

    2022

  6. [14]

    Milgram, H

    P. Milgram, H. Takemura, A. Utsumi, and F. Kishino. Augmented reality: A class of displays on the reality-virtuality continuum. Tele- manipulator and Telepresence Technologies, 2351, 01 1994. doi: 10. 1117/12.197321 1

    1994

  7. [15]

    Palmisano, R

    S. Palmisano, R. Mursic, and J. Kim. Cybersickness in virtual reality head-mounted displays: Examining the influence of scene complexity and arm cycles on cybersickness and postural stability. Frontiers in Virtual Reality, 1:602954, 2020. doi: 10.3389/frvir.2020.602954 1

    2020

  8. [16]

    Rebenitsch and C

    L. Rebenitsch and C. Owen. Review on cybersickness in applications and visual displays. Virtual Reality, 20(2):101–125, 2016. 2

    2016

  9. [17]

    L. Xiao, S. Nouri, J. Hegland, A. G. Garcia, and D. Lanman. Neural- passthrough: Learned real-time view synthesis for vr. In ACM SIG- GRAPH 2022 Conference Proceedings (SIGGRAPH ’22) , pp. Article 40, 1–9. Association for Computing Machinery, New York, NY , USA,

    2022

  10. [2022]

    doi: 10.1145/3528233.3530701 1

  11. [2023]

    doi: 10.3390/virtualworlds2010002 1

    doi

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.