Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gaze-guided foveated vision cuts robot ViT compute by 94% while keeping performance.

desk verdict Efficiency results are believable and the open benchmark adds value, but success-rate claims need a fixed-center foveation control and seed-based evaluation. read the letter →

arxiv 2507.15833 v3 pith:5MLRLRK6 submitted 2025-07-21 cs.RO cs.AIcs.CV

Ian Chuang , Jinyu Zou , Andrew Lee , Dechen Gao , Iman Soltani This is my paper
classification cs.ROcs.AIcs.CV
keywords gaze-guidedfoveationfoveatedvisiontransformersimitationlearningactiveflowmatchingpolicyeyetrackingdistractorrobustnessbimanualmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that robot manipulation policies do not need to process camera images uniformly: by focusing a vision transformer's tokens where a human operator looks, a policy can match or beat full-resolution processing while using 94% fewer visual tokens and much less computation. The authors extend an active-vision ALOHA robot platform with VR eye tracking to collect gaze alongside demonstrations, and they evaluate two ways to predict gaze at test time—a two-stage gaze model and an end-to-end gaze-and-action policy. Across six simulated bimanual tasks and two real-robot tasks, the foveated policies match uniform baselines on success rate, improve robustness to background distractors, and in some high-precision tasks succeed more often. If this holds, human gaze is a practical inductive bias for efficient robot learning, not just a biological curiosity.

What carries the argument

The central mechanism is the foveated patch tokenization pattern adapted from foveated image segmentation: a 20-patch layout whose resolution decays from a central gaze-aligned point to the periphery, combined with gaze estimation (a two-stage UNet plus spatial softmax model or an end-to-end flow-matching gaze predictor). The token pattern is what carries the argument: it maps high visual detail exactly where human demonstrators look and throws away peripheral detail, and this is what produces the compute savings and distractor robustness.

What would settle it

Take a trained foveated policy and replace its predicted gaze points with gaze points corrupted by increasing spatial offsets, or with random fixation points, at evaluation time; if success rates remain high under large offsets, then the benefit comes from the tokenization pattern itself rather than from accurate gaze, and the central claim that human gaze guides the foveation would be falsified. Conversely, a monotonic drop in success rate as gaze error increases would confirm that gaze localization is the load-bearing signal.

Watch

Extended reading notes

Core claim

The central discovery is that foveated patch tokenization—placing small dense patches at a predicted gaze point and larger sparser patches toward the periphery—preserves, and in some settings improves, the task success of vision-transformer-based imitation policies while cutting the token count from 324 to 20. Foveation shifts the image so the predicted gaze lands at the center of the pattern, pads the borders, and downsamples each patch to a common size, so a standard ViT can consume them. The resulting policies train roughly seven times faster and infer about three times faster, and they generalize better to scenes with distracting objects. The authors interpret this as evidence that a gaze-guided foveation inductive bias is useful for robotic vision, and they release the GIAVA platform, simulation benchmark, and gaze-labeled dataset to support further work.

Load-bearing premise

The whole result rests on the assumption that the eye-tracking coordinates recorded from the VR headset, after synchronization and mapping to the robot camera image, actually point at the task-relevant region of the image; if they are noisy or misaligned, foveation would concentrate resolution on the wrong location and the reported benefits would not be evidence for the gaze hypothesis.

Editorial extensions

If this is right

  • Foveated tokenization cuts the ViT token count from 324 to 20, reducing encoder GFLOPs by roughly 16x and overall training and inference time by 7x and 3x, respectively, at batch size 64.
  • On six simulated manipulation tasks, the foveated policies match or exceed uniform fine-grained tokenization in success rate, with the largest gains on high-precision tasks such as ThreadNeedle and PourTestTube.
  • Foveated policies generalize better to scenes with randomly placed distractor objects, a robustness improvement that the uniform baselines do not show consistently.
  • On two real-robot tasks (Ball and Toothbrush), the foveated policy is comparable to or better than uniform tokenization, particularly when distractors are present.
  • Gaze can be predicted either by a separate two-stage model or jointly with actions via flow matching; the two-stage approach is generally more accurate, while the end-to-end version is cheaper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If gaze is doing the causal work, then tasks with high visual precision or heavy clutter should gain the most from foveation; the paper's results are consistent with that but do not systematically vary task precision, so this remains a testable prediction.
  • A straightforward ablation the authors do not run is to corrupt or randomize the gaze input to a trained foveated policy; if the robustness and speed gains persist with random gaze, then the benefit is essentially token sparsity or augmentation rather than evidence that human gaze carries task-relevant signal.
  • The GIAVA data collection pipeline records gaze and head pose simultaneously, so combining foveation with the active camera arm's perspective control is a natural next step: the policy would decide both where to point the camera and where to fixate within that view.
  • Because foveation downscales the periphery, tasks that require monitoring a peripheral target—like the HookPackage failure mode the authors report—suggest foveated policies may need gaze memory or explicit saccades to be fully general.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes GIAVA, an extension of AV-ALOHA that records human gaze during teleoperated demonstrations and trains imitation-learning policies with foveated ViT tokenization. Two gaze estimators are compared: a two-stage UNet-based estimator and an end-to-end method that appends gaze to the action space. Experiments on six simulation tasks and two real-robot tasks compare a Foveated tokenization scheme with Fine (324 uniform 16x16 patches) and Coarse (20 uniform 64x64 patches) baselines under standard and distractor settings, with and without MAE pretraining. The paper reports a 94% reduction in visual tokens and ViT GFLOPs, roughly 7x faster training and 3x faster inference, and claims that foveation preserves or improves success rates while increasing robustness to distractors.

Significance. If the attribution to human gaze is established, this is a useful contribution to robot learning: the efficiency gains are large and well supported, the open-source platform, dataset, and simulation benchmark are valuable community resources, and the two gaze-integration recipes are clearly described. The evidence as presented, however, supports the efficiency claim much more strongly than the performance and robustness claims, because the experimental design does not isolate human gaze from the foveal tokenization geometry. The work is significant but the central scientific claim about gaze-guided vision is currently incomplete.

major comments (3)
  1. [Section III.E and Table III] The experimental design cannot attribute the reported performance and robustness differences to human gaze. The Fine and Coarse baselines differ from Foveated both in token count and in where high resolution is placed: Fine uses 324 uniform 16x16 patches, Coarse uses 20 uniform 64x64 patches, and Foveated uses 20 patches concentrated around the predicted gaze. Without a control that uses the same 20-patch foveal pattern with gaze fixed at the image center, or with gaze replaced by a non-human heuristic, the observed success-rate gains could be produced solely by the multiresolution 'center-plus-context' inductive bias of the foveal pattern itself. Since the paper's title and Contribution 1 claim 'gaze-guided' foveation, this control is load-bearing and should be added.
  2. [Section V.B and Tables III-IV] All simulation success rates are reported as the best checkpoint among 10 evaluations, and no multiple seeds or confidence intervals are reported. Differences of a few percentage points, such as Ball 64 vs 62 and Toothbrush 24 vs 18 in Table IV, are within the noise of 50-rollout evaluations, so the claims that foveation 'preserves or potentially improves' performance are not reliably quantified. Please report means and standard deviations over at least three seeds, or otherwise provide confidence intervals, and present the full evaluation distribution rather than only the best checkpoint.
  3. [Section III.A] The paper does not validate that the VR eye-tracking coordinates, after synchronization and interpolation for untagged frames, correspond to the task-relevant image region on the robot camera. A systematic headset calibration error or latency would place the foveal high-resolution region in the wrong location during training, making any apparent benefit a regularization artifact rather than evidence for the gaze hypothesis. Please report calibration or synchronization error, or provide an ablation with artificially shifted or perturbed gaze labels, to support the data-collection premise.
minor comments (5)
  1. [Table III caption] The table layout is difficult to parse because each method name appears once while there are two training settings; please clarify by labeling the No Pretraining and With MAE Pretraining subcolumns explicitly.
  2. [Section V.B, HookPackage paragraph] The discussion of Fov-Act's difficulty fixating on the hook is anecdotal and not quantified; please add gaze-error statistics or a supplementary video analysis to support this explanation.
  3. [Section III.B] The flow matching loss and the ODE integration are presented inline; display equations would improve readability and make the notation easier to verify.
  4. [Tables I and II] The hardware and software versions used for latency and memory measurements are not stated; please include them so the efficiency numbers can be reproduced.
  5. [Throughout] The manuscript contains inconsistent spacing in names such as 'A V-ALOHA' and 'GIA V A'; please standardize these terms.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the foveated-vs-uniform comparisons are empirical and self-contained; self-citations to AV-ALOHA provide the testbed, not the claimed result.

full rationale

This is an empirical systems paper, not a deductive derivation. The central efficiency claim (94% token/compute reduction) follows directly from the implemented tokenization patterns: Foveated uses 20 patches while Fine uses 324, as reported in Table II. That is transparent arithmetic from the chosen designs, not a fitted parameter renamed as a prediction. The performance and robustness claims come from head-to-head rollouts of four policies (Fine, Coarse, Fov-Act, Fov-UNet) in Tables III and IV, trained with the same flow-matching objective, the same ViT initialization schemes, and the same MAE pretraining procedure applied separately to each tokenization pattern. The foveated policies are not derived from the uniform baselines; they are alternative architectures evaluated under matched conditions. The main self-citations are to AV-ALOHA [12] for the active-vision hardware and simulation benchmark, and to the authors' earlier bimanual works [3], [4]; these provide the testbed but do not assert or entail the foveation result, so the self-citation is not load-bearing. The foveated tokenization method itself is adopted from external work [18] (Schmidt and Newcombe), not from the authors' prior publications, so no ansatz is smuggled in via self-citation. The paper also explicitly notes a fairness caveat that uniform tokenization might improve with DINOv2 pretrained weights but uses the same MAE procedure across all patterns, which is a limitation, not a circular step. The HookPackage discussion discloses a failure mode of Fov-Act, cutting against any suggestion that the foveation outcome was forced. A skeptic could ask whether the gains come from human gaze specifically or from the multiresolution inductive bias alone, since no fixed-center foveation control is evaluated; that is a legitimate experimental-design concern, but it is a confound, not a case of the conclusion being equivalent to its inputs by construction. No circular step can be exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the transferability of human gaze to robot camera coordinates and on synchronization of eye tracking, plus standard deep learning assumptions. No new theoretical entities are introduced; the main hand-designed element is the foveation pattern geometry.

free parameters (1)
  • Foveated tokenization pattern geometry (20 patches, concentric rings) = 20 patches; central patches 16x16 pixels; peripheral patches larger and downscaled; exact ring radii not specified in…
    Hand-designed choice to match the Coarse baseline token count; it determines where resolution is concentrated and directly affects performance, but it is not fitted to data or derived from first principles.
assumptions (4)
  • domain assumption Human gaze from the VR headset maps to the task-relevant region of the robot camera image.
    The whole foveation premise relies on gaze pointing at task-relevant regions; if the headset-to-camera mapping is miscalibrated, foveation could discard important information. Section III.A.
  • domain assumption Image-ID-based synchronization plus interpolation yields correct gaze-image alignment.
    The authors interpolate gaze for frames without labels; if latency is larger than assumed, training labels are noisy. Section III.A.
  • domain assumption MAE pretraining on a 60k-image ImageNet subset transfers enough to manipulation to provide a fair comparison across tokenization patterns.
    The paper deliberately avoids ImageNet-scale pretraining; transfer of these small-scale MAE features is assumed. Section III.E.
  • standard math Flow matching provides a valid conditional generative model of action chunks.
    Standard result imported from [38-40]; not proven in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers." pith.science (2026). https://pith.science/paper/5MLRLRK6

@misc{pith2026250715833,
  author       = {Pith},
  title        = {Pith review of: Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5MLRLRK6}},
  note         = {Machine review of arXiv:2507.15833}
}
read the original abstract

Human vision is a highly active process driven by gaze, which directs attention to task-relevant regions through foveation, dramatically reducing visual processing. In contrast, robot learning systems typically rely on passive, uniform processing of raw camera images. In this work, we explore how incorporating human-like active gaze into robotic policies can enhance efficiency and robustness. We develop GIAVA (Gaze Integrated Active-Vision ALOHA), a robot vision system that emulates human head and neck movement, and gaze adjustment for foveated processing. Extending the AV-ALOHA robot platform, we introduce a framework for simultaneously collecting eye-tracking, perspective control, and robot manipulation demonstration data from a human operator. We also open-source a simulation benchmark and dataset for training robot policies that incorporate human gaze. Inspired by recent work in foveated image segmentation and given the widespread use of Vision Transformers (ViTs) in robot learning, we integrate gaze information into ViTs using a foveated patch tokenization scheme. Compared to uniform patch tokenization, this significantly reduces the number of tokens, and thus computation. Our results show that our method for foveated robot vision drastically reduces computational overhead, and enhances robustness to background distractors. Notably, on certain high-precision tasks, foveated vision also improves performance, as reflected in higher success rates. Together, these findings suggest that human-inspired foveated visual processing offers untapped potential and should be further considered as a useful inductive bias in robotic vision systems. https://soltanilara.github.io/giava/

Figures

Figures reproduced from arXiv: 2507.15833 by the authors.

Figure 1
Figure 1. GIAVA System Overview: a) Bimanual demonstra￾tions with eye-tracking are recorded using a VR headset to b) control the AV-ALOHA robot system. After data collection, a robot policy is trained to imitate human gaze, head/neck movement and manipulation actions. c) The policy first predicts gaze, foveates the image around the gaze, and generates actions via a flow matching policy. across the whole visual field would req… view at source ↗
Figure 2
Figure 2. GIAVA Data Collection: The stereo camera images along with an image ID are transmited to the VR headset. The VR headset sends back head and hand controller poses to control the manipulation and active vision robotic arms, along with gaze coordinates and the corresponding image ID, synchronizing gaze data with the images. were then fed into a policy [35]. Subsequent works used gaze to switch from low-resolution full … view at source ↗
Figure 3
Figure 3. Gaze Estimation: Fov-UNet uses UNet and spatial softmax to predict gaze; Fov-Act predicts future gaze and action together via policy. Tokenization: Fov-UNet and Fov-Act use foveated tokenization around predicted gaze; Fine and Coarse use uniform tokenization. Policy Architecture: Image observations Oimg are tokenized, processed by ViT, and compressed with Q-Former module into tokens cimg, which conditions Flow Trans… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of the patch tokenization methods. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: MAE reconstructions after pretraining with different [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Illustration of simulation and real tasks. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Act, Sense, Act: Learning Active Perception from Large-Scale Egocentric Human Data

    cs.RO 2026-02 conditional novelty 6.0 of 10

    CoMe-VLA combines cognitive subtask labels and dual-track memory with human egocentric pretraining, reaching 83% mean success on five active-perception manipulation tasks.

Reference graph

Works this paper leans on

45 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023

    arXiv 2023

  2. [2]

    Aloha unleashed: A simple recipe for robot dexterity,

    T. Z. Zhao, J. Tompson, D. Driess, P. Florence, K. Ghasemipour, C. Finn, and A. Wahid, “Aloha unleashed: A simple recipe for robot dexterity,”arXiv preprint arXiv:2410.13126, 2024

    arXiv 2024

  3. [3]

    Interact: Inter- dependency aware action chunking with hierarchical attention trans- formers for bimanual manipulation,

    A. Lee, I. Chuang, L.-Y . Chen, and I. Soltani, “Interact: Inter- dependency aware action chunking with hierarchical attention trans- formers for bimanual manipulation,”arXiv preprint arXiv:2409.07914, 2024

    arXiv 2024

  4. [4]

    Vita: Vision-to-action flow matching policy,

    D. Gao, B. Zhao, A. Lee, I. Chuang, H. Zhou, H. Wang, Z. Zhao, J. Zhang, and I. Soltani, “Vita: Vision-to-action flow matching policy,” arXiv preprint arXiv:2507.13231, 2025

    arXiv 2025

  5. [5]

    Generalizable humanoid manipulation with improved 3d diffusion policies,

    Y . Ze, Z. Chen, W. Wang, T. Chen, X. He, Y . Yuan, X. B. Peng, and J. Wu, “Generalizable humanoid manipulation with improved 3d diffusion policies,”arXiv preprint arXiv:2410.10803, 2024

    arXiv 2024

  6. [6]

    Flow matching imitation learning for multi-support manipulation,

    Q. Rouxel, A. Ferrari, S. Ivaldi, and J.-B. Mouret, “Flow matching imitation learning for multi-support manipulation,” in2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids). IEEE, 2024, pp. 528–535

    work page 2024

  7. [7]

    Hover: Versatile neural whole-body controller for humanoid robots,

    T. He, W. Xiao, T. Lin, Z. Luo, Z. Xu, Z. Jiang, J. Kautz, C. Liu, G. Shi, X. Wang,et al., “Hover: Versatile neural whole-body controller for humanoid robots,”arXiv preprint arXiv:2410.21229, 2024

    arXiv 2024

  8. [8]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,”The International Journal of Robotics Research, p. 02783649241273668, 2023

    2023

Show all 45 references
  1. [9]

    The foveal confluence in human visual cortex,

    M. M. Schira, C. W. Tyler, M. Breakspear, and B. Spehar, “The foveal confluence in human visual cortex,”Journal of Neuroscience, vol. 29, no. 28, pp. 9050–9058, 2009

    2009

  2. [10]

    Embedded foveation image coding,

    Z. Wang and A. C. Bovik, “Embedded foveation image coding,”IEEE Transactions on image processing, vol. 10, no. 10, pp. 1397–1410, 2001

    2001

  3. [11]

    Object detection through search with a foveated visual system,

    E. Akbas and M. P. Eckstein, “Object detection through search with a foveated visual system,”PLOS Computational Biology, vol. 13, no. 10, pp. 1–28, 10 2017. [Online]. Available: https: //doi.org/10.1371/journal.pcbi.1005743

    2017 doi

  4. [12]

    Active vision might be all you need: Exploring active vision in bimanual robotic manipulation,

    I. Chuang, A. Lee, D. Gao, and I. Soltani, “Active vision might be all you need: Exploring active vision in bimanual robotic manipulation,” arXiv preprint arXiv:2409.17435, 2024

    2024 arXiv

  5. [13]

    Open-television: Teleoperation with immersive active visual feedback,

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,”arXiv preprint arXiv:2407.01512, 2024

    2024 arXiv

  6. [14]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly,et al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  7. [15]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khali- dov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby,et al., “Dinov2: Learning robust visual features without supervision,”arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  8. [16]

    Data scaling laws in imitation learning for robotic manipulation,

    F. Lin, Y . Hu, P. Sheng, C. Wen, J. You, and Y . Gao, “Data scaling laws in imitation learning for robotic manipulation,”arXiv preprint arXiv:2410.18647, 2024

    2024 arXiv

  9. [17]

    Cage: Causal attention en- ables data-efficient generalizable robotic manipulation,

    S. Xia, H. Fang, C. Lu, and H.-S. Fang, “Cage: Causal attention en- ables data-efficient generalizable robotic manipulation,”arXiv preprint arXiv:2410.14974, 2024

    2024 arXiv

  10. [18]

    Segment this thing: Foveated tok- enization for efficient point-prompted segmentation,

    T. Schmidt and R. Newcombe, “Segment this thing: Foveated tok- enization for efficient point-prompted segmentation,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 29 428–29 437

    2025

  11. [19]

    Non-extensive distribution of human eye photoreceptors,

    A. Mehri, “Non-extensive distribution of human eye photoreceptors,” Journal of Theoretical Biology, vol. 419, pp. 305–309, 2017

    2017

  12. [20]

    Marr,Vision: A computational investigation into the human repre- sentation and processing of visual information

    D. Marr,Vision: A computational investigation into the human repre- sentation and processing of visual information. MIT press, 2010

    2010

  13. [21]

    Recognition-by-components: a theory of human image understanding

    I. Biederman, “Recognition-by-components: a theory of human image understanding.”Psychological review, vol. 94, no. 2, p. 115, 1987

    1987

  14. [22]

    Neural mechanisms of selective visual attention,

    R. Desimone, J. Duncan,et al., “Neural mechanisms of selective visual attention,”Annual review of neuroscience, vol. 18, no. 1, pp. 193–222, 1995

    1995

  15. [23]

    How does the brain solve visual object recognition?

    J. J. DiCarlo, D. Zoccolan, and N. C. Rust, “How does the brain solve visual object recognition?”Neuron, vol. 73, no. 3, pp. 415–434, 2012

    2012

  16. [24]

    Performance-optimized hierarchical models predict neural responses in higher visual cortex,

    D. L. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. J. DiCarlo, “Performance-optimized hierarchical models predict neural responses in higher visual cortex,”Proceedings of the national academy of sciences, vol. 111, no. 23, pp. 8619–8624, 2014

    2014

  17. [25]

    Context mitigates crowding: Peripheral object recognition in real-world images,

    M. W. Wijntjes and R. Rosenholtz, “Context mitigates crowding: Peripheral object recognition in real-world images,”Cognition, vol. 180, pp. 158–164, 2018

    2018

  18. [26]

    Emergent properties of foveated perceptual systems,

    A. Deza and T. Konkle, “Emergent properties of foveated perceptual systems,”arXiv preprint arXiv:2006.07991, 2020

    2006 arXiv

  19. [27]

    Biologically inspired deep learning model for efficient foveal-peripheral vision,

    H. Lukanov, P. K ¨onig, and G. Pipa, “Biologically inspired deep learning model for efficient foveal-peripheral vision,”Frontiers in Computational Neuroscience, vol. 15, p. 746204, 2021

    2021

  20. [28]

    Foveater: Foveated transformer for image classification,

    A. Jonnalagadda, W. Y . Wang, B. Manjunath, and M. P. Eck- stein, “Foveater: Foveated transformer for image classification,”arXiv preprint arXiv:2105.14173, 2021

    2021 arXiv

  21. [29]

    Peripheral vision transformer,

    J. Min, Y . Zhao, C. Luo, and M. Cho, “Peripheral vision transformer,” Advances in Neural Information Processing Systems, vol. 35, pp. 32 097–32 111, 2022

    2022

  22. [30]

    An eye-gaze tracking system for teleoperation of a mobile robot,

    C. Carreto, D. G ˆego, and L. Figueiredo, “An eye-gaze tracking system for teleoperation of a mobile robot,”Journal of Information Systems Engineering & Management, vol. 3, no. 2, p. 16, 2018

    2018

  23. [31]

    The human gaze helps robots run bravely and efficiently in crowds,

    Q. Zhang, Z. Hu, Y . Song, J. Pei, and J. Liu, “The human gaze helps robots run bravely and efficiently in crowds,” in2023 IEEE international conference on robotics and automation (ICRA). IEEE, 2023, pp. 7540–7546

    2023

  24. [32]

    Gaze-based intention estimation: principles, method- ologies, and applications in hri,

    A. Belardinelli, “Gaze-based intention estimation: principles, method- ologies, and applications in hri,”ACM Transactions on Human-Robot Interaction, vol. 13, no. 3, pp. 1–30, 2024

    2024

  25. [33]

    Visarl: Visual reinforcement learning guided by human saliency,

    A. Liang, J. Thomason, and E. Bıyık, “Visarl: Visual reinforcement learning guided by human saliency,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 2907–2912

    2024

  26. [34]

    Eye, robot: Learning to look to act with a bc-rl perception-action loop,

    J. Kerr, K. Hari, E. Weber, C. M. Kim, B. Yi, T. Bonnen, K. Goldberg, and A. Kanazawa, “Eye, robot: Learning to look to act with a bc-rl perception-action loop,”arXiv preprint arXiv:2506.10968, 2025

    2025

  27. [35]

    Using human gaze to improve robustness against irrelevant objects in robot manipulation tasks,

    H. Kim, Y . Ohmura, and Y . Kuniyoshi, “Using human gaze to improve robustness against irrelevant objects in robot manipulation tasks,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4415–4422, 2020

    2020

  28. [36]

    Gaze-based dual resolution deep imitation learning for high- precision dexterous robot manipulation,

    ——, “Gaze-based dual resolution deep imitation learning for high- precision dexterous robot manipulation,”IEEE Robotics and Automa- tion Letters, vol. 6, no. 2, pp. 1630–1637, 2021

    2021

  29. [37]

    Multi-task real-robot data with gaze attention for dual-arm fine manipulation,

    ——, “Multi-task real-robot data with gaze attention for dual-arm fine manipulation,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 8516–8523

    2024

  30. [38]

    Improving and generalizing flow-based generative models with minibatch optimal transport,

    A. Tong, K. Fatras, N. Malkin, G. Huguet, Y . Zhang, J. Rector-Brooks, G. Wolf, and Y . Bengio, “Improving and generalizing flow-based generative models with minibatch optimal transport,”arXiv preprint arXiv:2302.00482, 2023

    2023 arXiv

  31. [39]

    Affordance-based robot manipulation with flow matching,

    F. Zhang and M. Gienger, “Affordance-based robot manipulation with flow matching,”arXiv preprint arXiv:2409.01083, 2024

    2024

  32. [40]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,”arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  33. [41]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” inInternational conference on machine learning. PMLR, 2023, pp. 19 730–19 742

    2023

  34. [42]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” inProceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4195–4205

    2023

  35. [43]

    Movie gen: A cast of media foundation models,

    A. Polyak, A. Zohar, A. Brown, A. Tjandra, A. Sinha, A. Lee, A. Vyas, B. Shi, C.-Y . Ma, C.-Y . Chuang,et al., “Movie gen: A cast of media foundation models,”arXiv preprint arXiv:2410.13720, 2024

    2024 arXiv

  36. [44]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255

    2009

  37. [45]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. Girshick, “Masked autoencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.