pith. sign in

arxiv: 2606.21011 · v1 · pith:G5NL6UNAnew · submitted 2026-06-19 · 💻 cs.RO

R2HandoverSim: A Simulation Framework and Benchmark for Robot-to-Human Object Handovers

Hanxin Zhang , Abdulqader Dhafer , Hongbiao Dong , Zhou Daniel Hao This is my paper

Pith reviewed 2026-06-26 14:43 UTC · model grok-4.3

classification 💻 cs.RO
keywords robot-to-human handoversimulation benchmarkshared grasp posehandover metricsuser studygrasp stabilityreachabilitysafety evaluation
0
0 comments X

The pith

R2HandoverSim benchmark shows five metrics predict user-perceived robot handover quality better than success rate alone.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents R2HandoverSim as a simulation framework and benchmark to enable reproducible evaluation of robot-to-human object handovers, addressing the lack of standardized protocols. It systematically compares four baseline methods on their predicted shared grasp poses and validates the setup through a user study with 30 participants that demonstrates correlation between simulation outcomes and real-world results. The central argument is that the five metrics of planning feasibility, reachability, grasp stability, grasp affordance, and safety provide a more accurate reflection of handover quality as experienced by users than an overall success rate metric.

Core claim

R2HandoverSim enables objective comparison of robot-to-human handover methods by evaluating predicted shared grasp poses in simulation across the five metrics, with those metrics shown to align more closely with user judgments from the 30-participant study than success rate alone.

What carries the argument

R2HandoverSim simulation benchmark that evaluates shared grasp poses using the five metrics of planning feasibility, reachability, grasp stability, grasp affordance, and safety.

If this is right

  • Standardized simulation evaluation becomes possible for comparing different robot-to-human handover approaches.
  • Methods can be ranked by how well they satisfy the five metrics rather than by binary success.
  • Simulation results can serve as a proxy for expected real-world user satisfaction in tested scenarios.
  • Future handover systems can target optimization of grasp affordance and safety alongside reachability.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Designers of physical robots could use the same five metrics to tune controllers before hardware deployment.
  • The benchmark approach might apply to evaluating human-to-robot handovers if the metrics are adapted.
  • Expanding the set of tested objects could reveal whether the metric correlations hold across more varied shapes and weights.

Load-bearing premise

The simulation environment together with the 30-participant user study accurately represent real-world physical dynamics and human perceptions of handover quality.

What would settle it

An experiment with new objects or a larger participant group in which the five metrics show no stronger correlation with user ratings than success rate alone would falsify the claim.

Figures

Figures reproduced from arXiv: 2606.21011 by Abdulqader Dhafer, Hanxin Zhang, Hongbiao Dong, Zhou Daniel Hao.

Figure 1
Figure 1. Figure 1: Overview of the R2HandoverSim benchmark envi [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Trial protocol of R2HandoverSim: initialization, target pose assignment, object placement, robotic grasping, approach [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The 16 benchmark objects from OakInk, including [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison of predicted handover configurations across four baselines. Each row shows the grasp pose, [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Real hardware setup for the sim-to-real experiment. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Subjective ratings for handover comfort, safety, and naturalness ( [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
read the original abstract

We present R2HandoverSim, a simulation benchmark for robot-to-human (R2H) object handovers. Although R2H handover methods have advanced rapidly, the lack of standardized evaluation protocols impedes objective comparison. Our benchmark enables reproducible evaluation by systematically comparing four baselines on their predicted shared grasp poses. We conduct a user study with 30 participants, analyze baseline performance, and show that simulation results correlate with real-world evaluation outcomes. Crucially, five complementary metrics (planning feasibility, reachability, grasp stability, grasp affordance, and safety) better reflect user-perceived handover quality than overall success rate alone. Website and code: https://robot-future.github.io/r2handoversim/.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript presents R2HandoverSim, a simulation benchmark for robot-to-human object handovers. It systematically evaluates four baseline methods on predicted shared grasp poses using simulation, reports results from a 30-participant user study, demonstrates correlation between simulation and real-world outcomes, and argues that five complementary metrics (planning feasibility, reachability, grasp stability, grasp affordance, and safety) better capture user-perceived handover quality than overall success rate alone. Code and a project website are provided for reproducibility.

Significance. If the reported sim-to-real correlation and metric superiority hold under scrutiny, the work would provide a valuable standardized benchmark for R2H handovers, addressing a clear gap in evaluation protocols. Explicit credit is due for releasing code and the website, which supports reproducibility in an empirical robotics benchmark. The user study linking metrics to human perception could strengthen evaluation practices if the statistical links are robust.

major comments (2)
  1. [User Study] User Study section: The central claim that the five metrics better reflect user-perceived quality than success rate alone rests on the 30-participant study and reported correlations, but the manuscript provides insufficient detail on the statistical tests, data exclusion criteria, multiple-comparison corrections, or effect sizes used to establish this superiority; without these, it is not possible to assess whether the result is robust or affected by post-hoc analysis choices.
  2. [Experiments and Results] Experiments and Results section: The reported correlation between simulation outcomes and real-world evaluation is load-bearing for the benchmark's validity, yet the text does not specify the objects/scenarios tested, the exact protocol for real-world trials, or controls for participant bias and distribution shift; this leaves the generalization claim (beyond the tested set) under-supported.
minor comments (2)
  1. [Abstract] The abstract and introduction could more clearly distinguish between the simulation benchmark contribution and the user-study validation component.
  2. [Figures] Figure captions for baseline comparisons should explicitly state the number of trials per condition and any error bars used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and positive recognition of the benchmark's reproducibility and potential impact. We address each major comment below and will revise the manuscript to incorporate the requested details.

read point-by-point responses

  1. Referee: [User Study] User Study section: The central claim that the five metrics better reflect user-perceived quality than success rate alone rests on the 30-participant study and reported correlations, but the manuscript provides insufficient detail on the statistical tests, data exclusion criteria, multiple-comparison corrections, or effect sizes used to establish this superiority; without these, it is not possible to assess whether the result is robust or affected by post-hoc analysis choices.

    Authors: We acknowledge that the current manuscript does not provide these statistical details, which limits assessment of robustness. In the revised manuscript, we will expand the User Study section with a dedicated paragraph specifying: the exact statistical tests (Pearson and Spearman correlations with exact p-values), data exclusion criteria (no participants were excluded), multiple-comparison corrections (Bonferroni applied where relevant), and effect sizes (correlation coefficients r and associated confidence intervals). This addition will directly support evaluation of the claim that the five metrics better reflect user-perceived quality. revision: yes

  2. Referee: [Experiments and Results] Experiments and Results section: The reported correlation between simulation outcomes and real-world evaluation is load-bearing for the benchmark's validity, yet the text does not specify the objects/scenarios tested, the exact protocol for real-world trials, or controls for participant bias and distribution shift; this leaves the generalization claim (beyond the tested set) under-supported.

    Authors: We agree that additional specifics are required to substantiate the sim-to-real correlation. The revised Experiments and Results section will include: the full list of objects and handover scenarios tested in real-world trials, the precise protocol (number of trials per participant, trial ordering, and setup details), and controls for bias and distribution shift (randomized presentation order, participant blinding to simulation metrics, and post-hoc checks for scenario coverage). These expansions will strengthen the support for generalization claims while preserving the reported correlation results. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical benchmark validated by independent user study

full rationale

The paper introduces a simulation benchmark for robot-to-human handovers, compares four baselines on shared grasp poses, and reports results from a 30-participant user study plus sim-to-real correlation. No load-bearing step reduces a claimed prediction or metric to a fitted parameter, self-citation chain, or definitional equivalence. The five metrics are presented as complementary empirical measures whose superiority is assessed against user-perceived quality via external study data, not by construction from the simulation inputs themselves. The work is self-contained against external benchmarks and user evaluations.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper introduces an empirical benchmark rather than a theoretical derivation; no free parameters, axioms, or invented entities are described in the abstract.

pith-pipeline@v0.9.1-grok · 5658 in / 1054 out tokens · 28311 ms · 2026-06-26T14:43:01.817135+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 4 canonical work pages · 1 internal anchor

  1. [1]

    Object handovers: A review for robotics,

    V . Ortenzi, A. Cosgun, T. Pardi, W. P. Chan, E. Croft, and D. Kuli ´c, “Object handovers: A review for robotics,”IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1855–1873, 2021

    2021

  2. [2]

    Optimizing robot-to-human object handovers using vision-based affordance infor- mation,

    D. Lehotsky, A. Christensen, and D. Chrysostomou, “Optimizing robot-to-human object handovers using vision-based affordance infor- mation,” in2023 IEEE International Conference on Imaging Systems and Techniques (IST). IEEE, 2023, pp. 1–6

    2023

  3. [3]

    Contacthandover: Contact-guided robot-to-human object handover,

    Z. Wang, Z. Liu, N. Ouporov, and S. Song, “Contacthandover: Contact-guided robot-to-human object handover,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 9916–9923

    2024

  4. [4]

    At- tention deep learning based model for predicting the 3D Human Body Pose using the Robot Human Handover Phases,

    J. Laplaza, A. Pumarola, F. Moreno-Noguer, and A. Sanfeliu, “At- tention deep learning based model for predicting the 3D Human Body Pose using the Robot Human Handover Phases,” in2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN). IEEE, 2021, pp. 161–166

    2021

  5. [5]

    HandoverSim: A simulation framework and benchmark for human-to-robot object handovers,

    Y .-W. Chao, C. Paxton, Y . Xiang, W. Yang, B. Sundaralingam, T. Chen, A. Murali, M. Cakmak, and D. Fox, “HandoverSim: A simulation framework and benchmark for human-to-robot object handovers,” in 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022, pp. 6941–6947

    2022

  6. [6]

    GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simu- lation Demonstration and Imitation,

    Z. Wang, J. Chen, Z. Chen, P. Xie, R. Chen, and L. Yi, “GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simu- lation Demonstration and Imitation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 362–16 372

    2024

  7. [7]

    H2O: A benchmark for visual human-human object handover analysis,

    R. Ye, W. Xu, Z. Xue, T. Tang, Y . Wang, and C. Lu, “H2O: A benchmark for visual human-human object handover analysis,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 15 762–15 771

    2021

  8. [8]

    GRAB: A dataset of whole-body human grasping of objects,

    O. Taheri, N. Ghorbani, M. J. Black, and D. Tzionas, “GRAB: A dataset of whole-body human grasping of objects,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 581–600

    2020

  9. [9]

    ARCTIC: A dataset for dexterous bimanual hand- object manipulation,

    Z. Fan, O. Taheri, D. Tzionas, M. Kocabas, M. Kaufmann, M. J. Black, and O. Hilliges, “ARCTIC: A dataset for dexterous bimanual hand- object manipulation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 12 943–12 954

    2023

  10. [10]

    Human–robot object handover: Recent progress and future direction,

    H. Duan, Y . Yang, D. Li, and P. Wang, “Human–robot object handover: Recent progress and future direction,”Biomimetic Intelligence and Robotics, vol. 4, no. 1, p. 100145, 2024

    2024

  11. [11]

    The grasp strategy of a robot passer influences perfor- mance and quality of the robot-human object handover,

    V . Ortenzi, F. Cini, T. Pardi, N. Marturi, R. Stolkin, P. Corke, and M. Controzzi, “The grasp strategy of a robot passer influences perfor- mance and quality of the robot-human object handover,”Frontiers in Robotics and AI, vol. 7, p. 542406, 2020

    2020

  12. [12]

    Leveraging semantic and geometric information for zero-shot robot-to-human handover,

    J. Liu, W. Dong, J. Wang, and M. Q.-H. Meng, “Leveraging semantic and geometric information for zero-shot robot-to-human handover,” Arxiv Preprint Arxiv:2409.17621, 2024. [Online]. Available: https: //arxiv.org/abs/2409.17621

    work page arXiv 2024

  13. [13]

    Assistance to older adults with comfortable robot-to-human handovers,

    J. Nowak, P. Fraisse, A. Cherubini, and J.-P. Daures, “Assistance to older adults with comfortable robot-to-human handovers,” in2022 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO), 2022, pp. 1–6

    2022

  14. [14]

    Naturalistic robot-to-human bimanual handover in complex environments through multi-sensor fusion,

    S. E. Ovur and Y . Demiris, “Naturalistic robot-to-human bimanual handover in complex environments through multi-sensor fusion,”IEEE Transactions on Automation Science and Engineering, vol. 21, no. 3, pp. 3730–3741, 2023

    2023

  15. [15]

    ContactPose: A dataset of grasps with object contact and hand pose,

    S. Brahmbhatt, C. Tang, C. D. Twigg, C. C. Kemp, and J. Hays, “ContactPose: A dataset of grasps with object contact and hand pose,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 361–378

    2020

  16. [16]

    DexYCB: A benchmark for capturing hand grasping of objects,

    Y .-W. Chao, W. Yang, Y . Xiang, P. Molchanov, A. Handa, J. Tremblay, Y . S. Narang, K. Van Wyk, U. Iqbal, S. Birchfield, and Others, “DexYCB: A benchmark for capturing hand grasping of objects,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9044–9053

    2021

  17. [17]

    MobileH2R: Learning generalizable human to mobile robot handover exclusively from scalable and diverse synthetic data,

    Z. Wang, Z. Chen, J. Chen, J. Wang, Y . Yang, Y . Liu, X. Liu, H. Wang, and L. Yi, “MobileH2R: Learning generalizable human to mobile robot handover exclusively from scalable and diverse synthetic data,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 17 315–17 325

    2025

  18. [18]

    Dexh2r: A benchmark for dynamic dexterous grasping in human-to-robot handover,

    Y . Wang, J. Ye, C. Xiao, Y . Zhong, H. Tao, H. Yu, Y . Liu, J. Yu, and Y . Ma, “Dexh2r: A benchmark for dynamic dexterous grasping in human-to-robot handover,”arXiv preprint arXiv:2506.23152, 2025

    work page arXiv 2025

  19. [19]

    NVIDIA Isaac Sim: Robotics simulation and synthetic data,

    NVIDIA, “NVIDIA Isaac Sim: Robotics simulation and synthetic data,” https://developer.nvidia.com/isaac-sim, 2023

    2023

  20. [20]

    Oakink: A large-scale knowledge repository for understanding hand-object interaction,

    L. Yang, K. Li, X. Zhan, F. Wu, A. Xu, L. Liu, and C. Lu, “Oakink: A large-scale knowledge repository for understanding hand-object interaction,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 20 953–20 962

    2022

  21. [21]

    ShapeNet: An Information-Rich 3D Model Repository

    A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Suet al., “ShapeNet: An information-rich 3d model repository,”arXiv preprint arXiv:1512.03012, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  22. [22]

    The YCB Object and Model Set: Towards common bench- marks for manipulation research,

    B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar, “The YCB Object and Model Set: Towards common bench- marks for manipulation research,” in2015 International Conference on Advanced Robotics (ICAR). IEEE, 2015, pp. 510–517

    2015

  23. [23]

    ContactDB: Analyzing and predicting grasp contact via thermal imaging,

    S. Brahmbhatt, C. Ham, C. C. Kemp, and J. Hays, “ContactDB: Analyzing and predicting grasp contact via thermal imaging,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8709–8719

    2019

  24. [24]

    Multi-GraspLLM: A multimodal LLM for multi-hand semantic guided grasp generation,

    H. Li, W. Mao, W. Deng, C. Meng, H. Fan, T. Wang, P. Tan, H. Wang, and X. Deng, “Multi-GraspLLM: A multimodal LLM for multi-hand semantic guided grasp generation,” Dec. 2024. [Online]. Available: https://arxiv.org/abs/2412.08468

    work page arXiv 2024

  25. [25]

    Fast and comfortable interactive robot-to-human object handover,

    C. Meng, T. Zhang, and T. lun Lam, “Fast and comfortable interactive robot-to-human object handover,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 3701–3706

    2022

  26. [26]

    GanHand: Predicting human grasp affordances in multi-object scenes,

    E. Corona, A. Pumarola, G. Alenya, F. Moreno-Noguer, and G. Ro- gez, “GanHand: Predicting human grasp affordances in multi-object scenes,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 5031–5041

    2020

  27. [27]

    Graspit! a versatile simulator for robotic grasping,

    A. T. Miller and P. K. Allen, “Graspit! a versatile simulator for robotic grasping,”IEEE Robotics and Automation Magazine, vol. 11, no. 4, pp. 110–122, 2004

    2004

  28. [28]

    Learning to segment object affordances on synthetic data for task-oriented robotic handovers,

    A. D. Christensen, D. Lehotsk ´y, M. W. Jørgensen, and D. Chrysos- tomou, “Learning to segment object affordances on synthetic data for task-oriented robotic handovers,” inThe 33rd British Machine Vision Conference. British Machine Vision Association, 2022

    2022

  29. [29]

    V oxNet: A 3d convolutional neural net- work for real-time object recognition,

    D. Maturana and S. Scherer, “V oxNet: A 3d convolutional neural net- work for real-time object recognition,” in2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Ieee, 2015, pp. 922–928

    2015

  30. [30]

    Contact- Graspnet: Efficient 6-dof grasp generation in cluttered scenes,

    M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contact- Graspnet: Efficient 6-dof grasp generation in cluttered scenes,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 13 438–13 444

    2021

  31. [31]

    Text2HOI: Text-guided 3d motion generation for hand-object interaction,

    J. Cha, J. Kim, J. S. Yoon, and S. Baek, “Text2HOI: Text-guided 3d motion generation for hand-object interaction,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1577–1585

    2024