Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A behaviour classifier trained on teleoperated joint trajectories can rank imitation-learning policies offline, with scores that line up with on-robot success rate more closely than validation loss or dynamic time warping.

desk verdict A fresh idea for offline HRI policy selection, but the validation is undermined by a teacher-forced evaluation protocol that never meets the robot. read the letter →

arxiv 2507.06404 v1 pith:IDZMEDZA submitted 2025-07-08 cs.RO cs.CVcs.LG

Matteo Tiezzi , Tommaso Apicella , Carlos Cardenas-Perez , Giovanni Fregonese , Stefano Dafarra , Pietro Morerio , Daniele Pucci , Alessio Del Bue This is my paper
classification cs.ROcs.CVcs.LG
keywords NeuralMetaEvaluatorimitationlearningevaluationhuman-robotinteractionbehaviourclassificationofflinepolicyjointtrajectoryhumanoidrobotsmodelselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Evaluating imitation-learning policies for humanoid robots usually means deploying each candidate on hardware and measuring success rate, which is slow, variable across human demonstrators, and blind to the quality of the produced motion. This paper proposes NeME, a neural network trained only on teleoperated joint trajectories to classify which behaviour the robot is performing, and then reused as an offline meta-evaluator: it scores a policy by feeding in the joint trajectories that the policy outputs and checking whether the recognized behaviour matches the intended one. The paper argues that NeME-based scores align with the success rate measured on the physical robot more closely than validation loss or dynamic time warping, while requiring no robot deployment and no human in the loop. If this holds, imitation-learning practitioners can select checkpoints and compare architectures reproducibly before touching hardware.

What carries the argument

The central object is NeME, a behaviour classifier over joint-position trajectories: a behaviour is defined as the sequence of joint outputs in a time window of length L, and NeME minimizes cross-entropy to predict the behaviour label from teleoperated trajectories. The load-bearing mechanism is using classification accuracy as a quality score: a policy trajectory that NeME labels correctly is judged to be close to the characteristic teleoperated patterns, hence better. The best variant reported is an LSTM over 32-frame windows of 24-dimensional joint features.

What would settle it

Take a policy that achieves equal or higher on-robot success rate than a demonstrated-style policy but executes the behaviour with a clearly different joint trajectory, such as a handshake with an alternate elbow motion; if NeME's meta-F1 ranks that policy below the worse-performing one, the claimed alignment with success rate is falsified.

Watch

Extended reading notes

Core claim

NeME is a sequence classifier trained with cross-entropy on teleoperation data to map a window of joint positions to a behaviour label such as handshake or payload reception. Once trained, it is applied to trajectories produced by an imitation-learning policy, and the predicted label is compared with the intended behaviour to compute a meta-accuracy or meta-F1 score. The central claim is that this meta-score is a better offline predictor of which policy or checkpoint will achieve high success rate on the real ergoCub robot than validation loss or DTW distance, because it judges the movement itself rather than pixel-level or joint-position alignment. In the epoch-selection experiment, NeME's mF1 curve peaks near the success-rate peak while validation loss selects a one-epoch model with about 20 percent success; in the two-policy comparison, NeME scores XBG above XBG-RGB, matching the deployed success rates of 70.0 versus 61.7.

Load-bearing premise

The whole evaluation rests on the premise that better task performance always produces joint trajectories resembling the teleoperated demonstrations; a policy that succeeds in a novel but functionally valid style would be penalized by NeME even though it works.

Editorial extensions

If this is right

  • During training, NeME lets a practitioner pick the best checkpoint by meta-F1 instead of validation loss, avoiding the 20-percent-success trap shown for one-epoch selection.
  • Different imitation-learning architectures can be compared offline on identical test trajectories, removing the human-variability and environmental noise of repeated on-robot success-rate trials.
  • Because NeME scores partial trajectories, evaluation can report graded, per-behaviour meta-accuracy rather than a binary success or failure.
  • The evaluation pipeline transfers to a new action set or platform only by retraining the behaviour classifier, which is cheaper than deploying every candidate policy on hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: NeME implicitly ranks policies by fidelity to the demonstrated style, so for tasks with multiple successful movement styles the evaluator would need per-style training data or it may rank a valid novel-style policy below a worse one.
  • Editorial inference: Since NeME consumes only joint trajectories, the same trained evaluator could serve as a reusable scoring layer for any policy on the same robot and behaviour set, effectively becoming a cheap, reproducible benchmark for that task suite.
  • Editorial inference: A natural extension is to use NeME's offline score as a training signal, for example early stopping or guiding data augmentation, which the paper does not test but its setup makes directly possible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes NeME, a sequence-classification network trained on teleoperated joint-position trajectories to recognize seven human-robot interaction behaviors (wave, shake, pick, walk, pick+walk, still, pick+still). Once trained, NeME is used as an offline meta-evaluator: a policy's rollout trajectories are fed to NeME and the resulting meta-accuracy/meta-F1 is used to select among training epochs or among policies, with the claim that this offline score aligns with success rate measured on the physical ergoCub robot more closely than validation loss or DTW. The experiments comprise a NeME architecture/context-length comparison (Table 2), an intra-policy epoch-selection study (Figure 1), and an inter-policy comparison of XBG versus XBG-RGB (Table 3). The central claim is therefore an empirical alignment claim between an offline classifier score and online robot success rate.

Significance. The idea of training a behavior classifier on teleoperation data and then reusing it as a performance surrogate is attractive and could reduce the cost of model selection in human-robot interaction and collaboration. The authors make a strong empirical claim, however, and the evidence presented in the manuscript is currently not sufficient to support it. The strengths of the paper are the realistic humanoid platform, the seven-task dataset, the systematic comparison of sequence models for NeME, and the explicit acknowledgment of the main assumption in Section 6. If the teacher-forcing issue were resolved and the alignment claim were quantified with proper statistics, the contribution would be a useful evaluation tool; at present the load-bearing evidence is largely qualitative and potentially confounded.

major comments (4)
  1. [Section 4.2 and Section 6] The offline evaluation feeds the ground-truth teleoperated vision and proprioception signals into the policy, rather than rolling out the policy closed-loop. Because the proprioception input at each step is the expert's joint state, the policy's predictions are conditioned on demonstration states, so the 'trajectories produced by the policy' are not the trajectories the policy would generate in deployment. This makes NeME's score and the success rate measure different distributions, so the alignment in Figure 1 and Table 3 could be an artifact of teacher forcing. Please provide closed-loop rollouts (or a simulator with a policy-generated state trajectory) for the same epochs and policies and recompute NeME scores, and report open-loop versus closed-loop score differences. Alternatively, explicitly restrict the claim to open-loop prediction quality and rename the contribution accordingly.
  2. [Figure 1 and Table 3] The empirical support for alignment is qualitative: Figure 1 shows no error bars for NeME mF1, DTW, or validation loss, and no quantitative measure of alignment (e.g., rank correlation or Kendall's tau between per-epoch NeME scores and average success rate) is reported. The single pair of policies in Table 3 provides two data points, and the mA/mF1 differences (73.5 versus 68.5 mA, 71.3 versus 69.9 mF1) are not accompanied by confidence intervals or a statistical test; the 1.4-point mF1 gap is comparable to the run-to-run variation of NeME training reported in Table 2. Please add repeated seeds for policy evaluation, error bars, and a formal alignment statistic.
  3. [Table 3 and Section 4.2] Table 3 covers only four of the seven behaviors (handwave, handshake, payload reception, standstill); walk, pick+walk, and pick+still are absent from the NeME-versus-SR comparison, although the abstract and conclusion claim alignment for the general framework. Since the dataset and success-rate protocol are described for seven tasks, the paper should either report NeME and SR for the excluded behaviors or clearly state that the validation is restricted to a subset and temper the general claim accordingly.
  4. [Section 3, 'Policy evaluation'] The validity assumption stated in Section 3, that 'the closer the trajectory produced by pi is to the characteristic patterns seen in the teleoperated data, the better its performance is evaluated', is not independently tested. A policy that succeeds via a functionally correct but stylistically novel trajectory would be penalized by NeME. The limitations section acknowledges the related assumption, but the paper does not provide evidence about the distribution of successful trajectories relative to teleoperated ones. Please test this assumption, for example by measuring NeME scores on successful versus failed closed-loop rollouts of the same policy, or at least by quantitatively comparing successful policies' trajectories to the teleoperation distribution.
minor comments (5)
  1. [Section 3] The notation is inconsistent: a trajectory is first defined as tau_i = {(s_t, a_t, b_t)} with length T_i, and later the proprioceptive trajectory is defined as tau_i^J = {j_t}_{t=0}^L with length L; the definition of a behavior as 'the sequence of output joints positions in a time window of length L' is circular. Please define a behavior formally and specify how temporal windows are chunked and labeled for NeME training and evaluation.
  2. [Figure 1] The left and right panels use different x-axes and scales, and the y-axis labels are incomplete: 'Loss value' has no units and the DTW y-axis is unlabeled. Please make the figure self-contained and include error bars or multiple runs.
  3. [Table 3] The column header 'D S' should be expanded in the table itself (for example, 'DTW dist.' and 'SR (%)'), and it should be stated whether mA/mF1 values are averaged over four behaviors or over all behaviors in the test set.
  4. [Section 2] The sentence 'success rates only fail to capture the nuanced qualities of the whole robot motion itself' is grammatically awkward; consider 'success rates fail to capture ... only part of the picture' or a similar rephrasing.
  5. [General] No code or data availability statement is provided. For a contribution whose stated goal is reproducible and systematic evaluation, releasing NeME training code and the dataset split would substantially strengthen the paper.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: NeME is an independently trained behaviour classifier, and the claimed alignment with success rate is an empirical result rather than a fitted or self-referential quantity.

full rationale

The central claim is that NeME, trained on teleoperated joint trajectories with behaviour labels, can select IL policy checkpoints and policies in a way that aligns with physical-robot success rate better than validation loss or DTW. Nothing in the derivation makes success rate an input to NeME: NeME is trained by cross-entropy over behaviour labels (Section 3), while SR is measured through a separate deployment protocol (Section 4.2). The alignment shown in Figure 1 and Table 3 is therefore an empirical finding, not a reduction by construction. The self-citations (XBG [4], ergoCub [29], the iCub3 avatar system [39], and the architecture survey [28]) are supporting infrastructure, baselines, or setup references, not load-bearing justification of the evaluation claim itself. Section 3 states the assumption that trajectories closer to teleoperation are evaluated better; this is a domain assumption, and Section 4.2 supplies the external check against SR. The acknowledged limitation in Section 6, namely 'we feed the vision and the proprioception signals of the testing samples into the robot policy, even if the robot joints moves in a different way', is a real validity concern about teacher-forced evaluation distributions, but it is not a definitional circularity: NeME's labels and the SR ground truth remain independent of each other, and no equation or fitted parameter reduces the claimed alignment to its own input.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim introduces no new physical or conceptual entity; NeME is a standard sequence classifier. Load-bearing assumptions are the fidelity-proxy premise and the offline-to-open-world transfer premise, both acknowledged by the authors. Free parameters are the standard hyperparameter choices (context window, architecture) for the selected NeME variant.

free parameters (2)
  • NeME context window L = 32
    The best performing input trajectory length, chosen by hyperparameter search over L in {16, 32, 64} based on validation accuracy; the central claim uses the L=32 LSTM variant.
  • NeME architecture = LSTM
    Selected after comparing LSTM, Transformer, xLSTM, Mamba, and Mamba2; the paper relies on this choice for all subsequent meta-evaluator results.
assumptions (3)
  • domain assumption Trajectory fidelity implies task success: better policies produce trajectories that a classifier trained on teleoperated demonstrations will classify correctly.
    Stated in Section 3 'Policy evaluation': "The rationale behind NeME is that the closer the trajectory produced by pi is to the characteristic patterns seen in the teleoperated data, the better its performance is evaluated." This is an unproved premise; a successful behavior could deviate stylistically from demonstrations.
  • domain assumption Offline evaluation trajectories are representative of on-robot behavior when feeding the collected visual and proprioceptive inputs to the policy.
    Acknowledged in Section 6: the assumption that "during testing, the robot behaves in a similar way compared to the collected samples" is a main limitation, and the evaluation cannot cover the open-world interaction setting.
  • domain assumption The seven behavior categories fully cover the interaction space, and the policy evaluation can be restricted to four of them without changing the conclusion.
    Table 3 reports NeME and success rate only for handwave, handshake, payload reception, and standstill; walk and pick+walk behaviors are omitted, yet the abstract states a general alignment claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction." pith.science (2026). https://pith.science/paper/IDZMEDZA

@misc{pith2026250706404,
  author       = {Pith},
  title        = {Pith review of: Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IDZMEDZA}},
  note         = {Machine review of arXiv:2507.06404}
}
read the original abstract

Evaluating and comparing the performance of autonomous Humanoid Robots is challenging, as success rate metrics are difficult to reproduce and fail to capture the complexity of robot movement trajectories, critical in Human-Robot Interaction and Collaboration (HRIC). To address these challenges, we propose a general evaluation framework that measures the quality of Imitation Learning (IL) methods by focusing on trajectory performance. We devise the Neural Meta Evaluator (NeME), a deep learning model trained to classify actions from robot joint trajectories. NeME serves as a meta-evaluator to compare the performance of robot control policies, enabling policy evaluation without requiring human involvement in the loop. We validate our framework on ergoCub, a humanoid robot, using teleoperation data and comparing IL methods tailored to the available platform. The experimental results indicate that our method is more aligned with the success rate obtained on the robot than baselines, offering a reproducible, systematic, and insightful means for comparing the performance of multimodal imitation learning approaches in complex HRI tasks.

Figures

Figures reproduced from arXiv: 2507.06404 by the authors.

Figure 1
Figure 1. Model selection in Imitation Learning is challenging. The success rate is the most used [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. NeME is a neural network that classifies joint trajectories into behaviours and is employed as a meta-evaluator for learned imitation policies πθ. by the policies, supports reproducible testing of multiple policies on the same data, and allows assessment of model performance prior to robot deployment. Background & Problem formulation. Let us consider an Imitation Learning (IL) problem, where an autonomous robotic ag… view at source ↗
Figure 3
Figure 3. Statistical analysis of the dataset. We show the number of frames (1st column), number of [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Critical Interval MSE: Toward Reliable Offline Validation for Robot Manipulation Policies

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    CI-MSE improves Spearman's rank correlation between offline validation error and real rollout performance from -0.61 (raw MSE) to -0.87 across policy checkpoints in simulation and real-world robot manipulation experiments.

Reference graph

Works this paper leans on

49 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Hussein, M

    A. Hussein, M. M. Gaber, E. Elyan, and C. Jayne. Imitation learning: A survey of learning methods. ACM Computing Surveys, 50(2):1–35, 2017

    work page 2017

  2. [2]

    Ross and D

    S. Ross and D. Bagnell. Efficient reductions for imitation learning. InInternational Conference on Artificial Intelligence and Statistics, 2010

    work page 2010

  3. [3]

    D. J. Foster, A. Block, and D. Misra. Is behavior cloning all you need? understanding horizon in imitation learning. In The Thirty-eighth Annual Conference on Neural Information Process- ing Systems

    work page

  4. [4]

    Cardenas-Perez, G

    C. Cardenas-Perez, G. Romualdi, M. Elobaid, S. Dafarra, G. L’Erario, S. Traversaro, P. More- rio, A. Del Bue, and D. Pucci. Xbg: End-to-end imitation learning for autonomous behaviour in human-robot interaction and collaboration. IEEE Robotics and Automation Letters, 2024

    work page 2024

  5. [5]

    Zhang, Z

    T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep imita- tion learning for complex manipulation tasks from virtual reality teleoperation. In 2018 IEEE international conference on robotics and automation (ICRA), pages 5628–5635. Ieee, 2018

    work page 2018

  6. [6]

    Z. Fu, Q. Zhao, Q. Wu, G. Wetzstein, and C. Finn. Humanplus: Humanoid shadowing and imitation from humans. In Conference on Robot Learning, 2024

    work page 2024

  7. [7]

    Rahmatizadeh, P

    R. Rahmatizadeh, P. Abolghasemi, L. B ¨ol¨oni, and S. Levine. Vision-based multi-task manip- ulation for inexpensive robots using end-to-end learning from demonstration. In 2018 IEEE international conference on robotics and automation (ICRA), pages 3758–3765. IEEE, 2018

    2018

  8. [8]

    Mizuchi, Y

    Y . Mizuchi, Y . Tanno, and T. Inamura. Designing evaluation metrics for quality of human- robot interaction in guiding human behavior. In International Conference on Human-Agent Interaction, 2023

    work page 2023

Show all 49 references
  1. [9]

    G. A. Abbo, T. Belpaeme, and M. Spitale. Concerns and values in human-robot interactions: A focus on social robotics. arXiv preprint arXiv:2501.05628, 2025

    2025 arXiv

  2. [10]

    Coronado, T

    E. Coronado, T. Kiyokawa, G. A. G. Ricardez, I. G. Ramirez-Alpizar, G. Venture, and N. Ya- manobe. Evaluating quality in human-robot interaction: A systematic search and classification of performance and human-centered factors, measures and metrics towards an industry 5.0. Jou...

    2022

  3. [11]

    Bensch, A

    S. Bensch, A. Jevtic, and T. Hellstr ¨om. On interaction quality in human-robot interaction. In ICAART 2017 Proceedings of the 9th International Conference on Agents and Artificial Intelligence, vol. 1, pages 182–189. SciTePress, 2017

    2017

  4. [12]

    Kokotinis, G

    G. Kokotinis, G. Michalos, Z. Arkouli, and S. Makris. On the quantification of human-robot collaboration quality. International Journal of Computer Integrated Manufacturing , 36(10): 1431–1448, 2023. 9

    2023

  5. [13]

    C. Wang, L. Fan, J. Sun, R. Zhang, L. Fei-Fei, D. Xu, Y . Zhu, and A. Anandkumar. Mimicplay: Long-horizon imitation learning by watching human play. In Conference on Robot Learning, 2023

    2023

  6. [14]

    Mandlekar, D

    A. Mandlekar, D. Xu, R. Mart ´ın-Mart´ın, S. Savarese, and L. Fei-Fei. Learning to generalize across long-horizon tasks from human demonstrations. InRobotics: Science and System, 2020

    2020

  7. [15]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023

    2023 arXiv

  8. [16]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. In Conference on Robot Learning, 2024

    2024

  9. [17]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. In arXiv preprint arXiv:2212.06817, 2022

    2022 arXiv

  10. [18]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  11. [19]

    G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakr- ishna, R. Baruch, M. Bauza, M. Blokzijl, et al. Gemini robotics: Bringing ai into the physical world. In arXiv preprint arXiv:2503.20020, 2025

    2025 arXiv

  12. [20]

    Bjorck, F

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. In arXiv preprint arXiv:2503.14734, 2025

    2025 arXiv

  13. [21]

    Taghavi, J

    N. Taghavi, J. Berdichevsky, N. Balakrishnan, K. C. Welch, S. K. Das, and D. O. Popa. On- line dynamic time warping algorithm for human-robot imitation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 3843–3849. IEEE, 2021

    2021

  14. [22]

    Dietzel and P

    C. Dietzel and P. J. Martin. Assessing similarity measures for the evaluation of human-robot motion correspondence. arXiv preprint arXiv:2412.04820, 2024

    2024 arXiv

  15. [23]

    Afzal, D

    A. Afzal, D. S. Katz, C. L. Goues, and C. S. Timperley. A study on the challenges of using robotics simulators for testing. arXiv preprint arXiv:2004.07368, 2020

    2004 arXiv

  16. [24]

    Gu, Y .-J

    X. Gu, Y .-J. Wang, and J. Chen. Humanoid-gym: Reinforcement learning for humanoid robot with zero-shot sim2real transfer. arXiv preprint arXiv:2404.05695, 2024

    2024 arXiv

  17. [25]

    J. Dao, H. Duan, and A. Fern. Sim-to-real learning for humanoid box loco-manipulation. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 16930– 16936, 2024. doi:10.1109/ICRA57147.2024.10610977

    2024

  18. [26]

    Gulcehre, Z

    C. Gulcehre, Z. Wang, A. Novikov, T. Paine, S. G ´omez, K. Zolna, R. Agarwal, J. S. Merel, D. J. Mankowitz, C. Paduraru, G. Dulac-Arnold, J. Li, M. Norouzi, M. Hoffman, N. Heess, and N. de Freitas. Rl unplugged: A suite of benchmarks for offline reinforce- ment learning. In H....

    2020

  19. [27]

    T. L. Paine, C. Paduraru, A. Michi, C. Gulcehre, K. Zolna, A. Novikov, Z. Wang, and N. de Freitas. Hyperparameter selection for offline reinforcement learning. arXiv preprint arXiv:2007.09055, 2020. 10

    2007 arXiv

  20. [28]

    Tiezzi, M

    M. Tiezzi, M. Casoni, A. Betti, T. Guidi, M. Gori, and S. Melacci. Back to recurrent processing at the crossroad of transformers and state-space models. Nature Machine Intelligence, pages 1–11, 2025

    2025

  21. [29]

    Sartore, M

    C. Sartore, M. Elobaid, L. Rapetti, G. Romualdi, S. Dafarra, N. A. Piga, I. Sorrentino, P. M. Viceconte, S. Traversaro, U. Pattacini, L. Fiorio, F. Draicchio, G. Tranfo, L. Natale, M. Mag- giali, and D. Pucci. Towards a shared embodied intelligence of humanoid robots: Optimiza...

  22. [30]

    J. Fu, M. Norouzi, O. Nachum, G. Tucker, Z. Wang, A. Novikov, M. Yang, M. R. Zhang, Y . Chen, A. Kumar, C. Paduraru, S. Levine, and T. L. Paine. Benchmarks for deep off-policy evaluation. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Au...

    2021

  23. [31]

    Mandlekar, D

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart´ın-Mart´ın. What matters in learning from offline human demonstrations for robot manipulation. In 5th Annual Conference on Robot Learning

  24. [32]

    D. A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In Advances in neural information processing systems, volume 1, 1988

    1988

  25. [33]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model. In arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  26. [34]

    Damacharla, A

    P. Damacharla, A. Y . Javaid, J. J. Gallimore, and V . K. Devabhaktuni. Common metrics to benchmark human-machine teams (hmt): A review. IEEE Access, 6:38637–38655, 2018

    2018

  27. [35]

    M ¨uller

    M. M ¨uller. Dynamic time warping. Information retrieval for music and motion, pages 69–84, 2007

    2007

  28. [36]

    Bain and C

    M. Bain and C. Sammut. A framework for behavioural cloning. In Machine intelligence 15, pages 103–129, 1995

    1995

  29. [37]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics: Science and Systems (RSS), 2023

    2023

  30. [38]

    Romualdi, S

    G. Romualdi, S. Dafarra, Y . Hu, P. Ramadoss, F. J. A. Chavez, S. Traversaro, and D. Pucci. A Benchmarking of DCM Based Architectures for Position, Velocity and Torque Controlled Humanoid Robots. International Journal of Humanoid Robotics, 17(01):1950034, Feb. 2020. ISSN 0219-...

    2020 doi

  31. [39]

    Dafarra, U

    S. Dafarra, U. Pattacini, G. Romualdi, L. Rapetti, R. Grieco, K. Darvish, G. Milani, E. Valli, I. Sorrentino, P. M. Viceconte, et al. icub3 avatar system: Enabling remote fully immersive embodiment of humanoid robots. Science Robotics, 9(86):eadh3834, 2024

    2024

  32. [40]

    https://ifeeltech.eu/

    ifeel. https://ifeeltech.eu/

  33. [41]

    N. L. Olsen, B. Markussen, and L. L. Raket. Simultaneous inference for misaligned multivari- ate functional data. Journal of the Royal Statistical Society Series C: Applied Statistics, 67(5): 1147–1176, 2018

    2018

  34. [42]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735– 1780, 1997. 11

    1997

  35. [43]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Ad- vances in Neural Information Process...

  36. [44]

    M. Beck, K. P ¨oppel, M. Spanring, A. Auer, O. Prudnikova, M. K. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter. xlstm: Extended long short-term memory. In The Thirty- eighth Annual Conference on Neural Information Processing Systems

  37. [45]

    Gu and T

    A. Gu and T. Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  38. [46]

    Dao and A

    T. Dao and A. Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060, 2024

    2024 arXiv

  39. [47]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 12

    2017 arXiv

  40. [2017]

    URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

    2017

  41. [2025]

    Version 1, available at https://doi.org/10.21203/rs.3.rs-6122130/v1

    doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.