Remote Mobile Manipulation with the Centauro Robot: Full-body Telepresence and Autonomous Operator Assistance

Antonio Di Guardo; Antonio Frisoli; Arturo Laurenzi; Arul Selvam Periyasamy; Christian Lenz; Daniele Leonardis; David Droeschel; Diego Rodriguez; Dmytro Pavlichenko; Domenico Buongiorno

REVIEW 4 major objections 5 minor 56 references

Remote Mobile Manipulation with the Centauro Robot: Full-body Telepresence and Autonomous Operator Assistance

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims an integrated telepresence-and-autonomy system on the Centauro robot can perform a wide range of remote mobile manipulation tasks without task-specific training, demonstrated in tests by a nuclear disaster-response…

desk verdict A solid systems-integration paper whose broad claims slightly outrun its evaluation; worth refereeing with requests to temper the 'no task-specific training' framing. read the letter →

arxiv 1908.01617 v1 pith:ARJUMBET submitted 2019-08-05 cs.RO

Tobias Klamt , Max Schwarz , Christian Lenz , Lorenzo Baccelliere , Domenico Buongiorno , Torben Cichon , Antonio Di Guardo , David Droeschel

show 16 more authors

Massimiliano Gabardi Malgorzata Kamedula Navvab Kashiri Arturo Laurenzi Daniele Leonardis Luca Muratore Dmytro Pavlichenko Arul Selvam Periyasamy Diego Rodriguez Massimiliano Solazzi Antonio Frisoli Michael Gustmann Jürgen Ro{ss}mann Uwe Süss Nikos G. Tsagarakis Sven Behnke

This is my paper · ORCID

classification cs.RO

keywords Centauromobilemanipulationtelepresenceteleoperationhybriddriving-steppinglocomotionexoskeletonautonomousgraspingfieldrobotics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to establish that the CENTAURO system, built around the Centauro robot and a suite of operator interfaces, can solve a broad set of realistic mobile manipulation tasks in environments too dangerous for humans. The key claim is that the integration of a full-body telepresence suit, autonomous locomotion and manipulation functions, and a simulation-based operator visualization goes beyond prior systems and enables untrained task execution. The authors evaluate this claim in a field setting with tasks such as opening doors, overcoming gaps and step fields, operating valves, using power tools, and connecting plugs. A sympathetic reading is that the system demonstrates a practical template for remote maintenance, construction, and disaster response, where flexibility across unknown tasks matters more than optimizing any single task.

What carries the argument

The central object is the integrated CENTAURO architecture: a 52-DoF robot with four 5-DoF legs ending in 360-degree steerable wheels and an anthropomorphic upper body with two 7-DoF arms and two complementary hands, coupled with a full-body telepresence suit that transfers arm, wrist, and finger motion and provides force feedback. The architecture also includes a simulation-based digital twin for operator situation awareness, a hybrid driving-stepping locomotion planner, and an autonomous manipulation pipeline that segments objects, estimates poses, transfers grasping skills from known to novel instances, and optimizes arm trajectories. The work of this machinery is to let a human operator retain high-level task judgment while offloading low-level control and repetitive actions to autonomy, which is what allows the system to address tasks it has never seen before.

What would settle it

Run the full task battery with a new operator team that has no prior exposure to the interfaces, no site inspection, and exactly one attempt per task, counting any physical assistance as a failure; if the success rates drop substantially from the reported ones, the claim that the system works without task-specific training is not supported.

Watch

Extended reading notes

Core claim

The central discovery is that a holistically integrated remote mobile manipulation system, combining a 52-DoF centaur-like robot with torque-controlled compliant actuators, a full-body telepresence suit with force feedback, and autonomous locomotion and manipulation planners, can accomplish a wide variety of realistic tasks without previous task-specific training. The paper argues that while individual components have been shown before, their integration into a single system evaluated across many tasks is the novel step. The results show successful teleoperated manipulation with the exoskeleton, precise adjustments with a 6D mouse, autonomous stair climbing with a hybrid driving-stepping planner, and autonomous grasping of a previously unseen drill via transferred grasp knowledge.

Load-bearing premise

The breadth claim rests on the evaluation showing that successes come from the system's general capability rather than from site familiarity, operator practice, or assistance; because operators could inspect sites in advance and some tasks were re-attempted after failures, that boundary is not strictly controlled.

Editorial extensions

If this is right

Operators can attempt previously unseen maintenance and disaster-response tasks without dedicated training runs, relying on complementary interfaces and autonomous assistance.
The hybrid driving-stepping planner turns a single operator-specified goal pose into executable paths over ramps, gaps, step fields, and stairs, substantially lowering the cognitive load of locomotion.
Force feedback in the exoskeleton lets operators detect mechanical limits such as valve stops and plug insertion forces, while the 6D mouse provides precise axis-constrained adjustments for fine alignment.
Autonomous grasping transfers grasps from known drill models to novel drill instances, indicating that category-level grasp knowledge can reduce the need for per-object engineering.
The staircase test exposed concrete weak points, particularly actuator cooling and localization precision, that define clear improvement targets for future field iterations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The breadth claim would be easier to compare across systems if the evaluation distinguished first-try performance from re-attempts and prohibited pre-inspection of the task site, making success rates a stricter measure of generality.
The human push allowed during the autonomous staircase climb suggests the full-autonomy claim currently assumes benign terrain detail; wheel-foot contact with holes is an identifiable failure mode for future planning and localization work.
The 7-of-14 success rate in autonomous grasping suggests perception, not motion planning, is the main bottleneck, so uncertainty-aware grasp selection could improve reliability without new hardware.
The deliberate pairing of complementary interfaces points toward a design principle for remote robots: keep autonomy for navigation and grasps, but retain a human in the loop for task-level decisions and force-sensitive manipulation.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Best take: this is a genuinely useful system paper. The CENTAURO robot, the full-body telepresence suit, and the autonomous locomotion and manipulation functions are integrated into one operating system and tested across a broad suite of realistic tasks at a nuclear response facility. Most components were published before; the integration is the new result, and that is a legitimate contribution. The paper is transparent: failures are reported, per-task sample sizes are small, and the lessons-learned section actually reflects on interface trade-offs. For those reasons I would send it to reviewers, not desk-reject.

What is done well: the hardware description is detailed enough to be useful (actuator classes, kinematics, exoskeleton mapping), and the evaluation covers driving, stepping, doors, valves, power tools, plugs, and surface scanning. The authors also credit the earlier component papers, so the novelty claim is scoped to integration rather than to each subsystem. The empirical numbers are thin by ML standards—between 1 and 9 attempts per task—but that is normal for a full-robot field exercise, and the paper does not hide the failures.

Soft spots: the abstract says the interfaces enable solving a wide variety of tasks 'without previous task-specific training.' The evaluation does not fully support that. The cutting-tool trigger was enlarged after failures, the snap hook was modified, a webcam was added for the screwdriver task, and the staircase test was moved to a lab after a hardware fix—and one of the three 'autonomous' stair successes required a human push. Repeated attempts were added after failures, and Table 2 aggregates successes over attempts, so the summary rates overstate first-try general capability. The paper openly reports all of this, so the issue is framing rather than concealment. The fix is to soften the 'no training' claim and to report assisted attempts separately.

The other soft spot is smaller: 'goes beyond the state of the art' is asserted without a quantitative baseline against, say, Momaro or a DRC-era system. I do not need a formal comparison for a system paper, but that sentence invites one.

Who should read it: anyone building a teleoperated mobile manipulator, especially with exoskeletons or wheeled-legged platforms. It is a solid integration reference, and the lessons learned are worth a skim.

Recommendation: send it to peer review. It deserves referee time. The main revisions are to recalibrate the claims about task-specific training and to separate post-modification successes from first-try ones. I would cite it for the system description.

Referee Report

4 major / 5 minor

Summary. The paper presents the CENTAURO system, a 52-DoF wheeled-legged centaur-like robot with compliant actuators, a full-body telepresence suit, and several autonomous assistance functions for locomotion and manipulation. The authors argue that the integration of these components into a holistic remote mobile manipulation system is novel and enables a wide range of realistic tasks without previous task-specific training. The system is evaluated in an intensive testing period at KHG facilities, with tasks ranging from ramp driving and door opening to valve operation, power-tool use, autonomous grasping, and autonomous stair climbing. The paper reports success rates, task times, failure cases, and lessons learned.

Significance. If the central claim is accepted, the paper is a valuable system-level contribution to field robotics: it demonstrates a complex, torque-controlled hybrid wheeled-legged platform with a rich operator interface, and it reports honest failure data and lessons learned that are useful to the community. The autonomous grasping component is evaluated on a novel instance of a familiar object category, which is a legitimate generalization setting rather than a circular test. The paper also openly acknowledges hardware failures and interface limitations. However, the load-bearing breadth claim that the system can solve a wide variety of tasks 'without previous task-specific training' rests on an evaluation protocol that includes mid-evaluation task modifications, repeated attempts, and at least one assisted autonomy success. These issues do not undermine the value of the system demonstration, but they do require a substantial reframing or re-analysis before the paper's strongest claims can be supported.

major comments (4)

[Section 9, Table 2; Section 9.2] The claim that the user interfaces enable solving tasks 'without previous task-specific training' is not cleanly supported by the evaluation protocol. Several headline successes depended on task or system modifications made after failures: the cutting-tool trigger was enlarged after a series of failures (Section 9.2), the snap hook was 'modified slightly to make it more easily graspable' (Section 9.2), a webcam was added to the other hand for the screwdriver task (Section 9.2), and the staircase autonomy test was moved to a lab after an actuator fan redesign (Section 9.4). In addition, Section 9 states that 'When failures were encountered, more attempts were added to gain insight into the possible failure modes,' and Table 2 aggregates successes across all attempts. Repeated attempts with system modifications are a form of task-specific tuning at the system level, so the reported success rates (e.g., Cutting tool 3/9, Auto grasping 7/14) cannot cleanly support the no-training claim. I recommend reporting the chronological sequence of attempts and modifications, and either restricting the no-training claim to the first attempt per task or removing it.
[Section 9.4, Table 2] The autonomous staircase experiment reports 3/3 successes, but one of those successes required a human push to regain balance, and the experiment was performed in a lab after hardware redesign rather than at the original evaluation site. Counting the assisted attempt as an unqualified autonomous success overstates the system's autonomous capability. The paper should clearly separate assisted from unassisted attempts, and should note that the failure mode was not resolved by the system itself. Since Table 2 is the central quantitative evidence for the autonomous locomotion claim, this is a load-bearing evaluation-integrity issue.
[Section 2; Section 9, Table 2] The claim that the integrated system 'goes beyond the state of the art' is not supported by any baseline, ablation, or comparison to prior systems. The evaluation reports no comparison to Momaro, DRC-HUBO, CHIMP, RoboSimian, or any other relevant platform, and the success rates are based on 1-9 attempts per task with operator-estimated difficulty scores. As a demonstration of integration this is informative, but as evidence for a comparative claim it is insufficient. I recommend either adding a structured comparison or explicitly reframing the contribution as an integrated system demonstration with lessons learned, without the comparative 'beyond the state of the art' wording.
[Section 9.3] The autonomous grasping experiment reports that 'the success rate improved during testing' across 14 attempts, and that operators could trigger re-computation of the planned trajectory before execution. Re-computation triggered by an operator is a form of human assistance, and improvement over attempts without any reported change to the system suggests either operator learning or implicit task-specific tuning. The paper should report the per-attempt outcome sequence, distinguish fully autonomous attempts from those with operator-triggered re-computation, and clarify whether the reported 7/14 success rate counts only fully autonomous executions.

minor comments (5)

[Section 9, Table 2] The 'Difficulty' scores are estimated subjectively by operators and are not tied to any hypothesis or used in the analysis; consider presenting them only as an informal ordering or removing them from the quantitative table.
[Table 2, caption] For the 'Auto grasping' row, the table lists '7/14' and '220 s', but it is unclear whether the time is the average over successes, the median, or the final attempt; please state the statistic and clarify how failed attempts are treated.
[Section 9.2, Snap hook] The modification of the snap hook to make it 'more easily graspable' is described in a single sentence; since this modification directly affects the success rate, it should be described in enough detail for a reader to judge the task difficulty and the validity of the 3/3 result.
[Section 9.1 and Section 9.4] The original Stairs task (0/1) is reported in Table 2, while the later autonomous staircase experiment appears as a separate row 'Auto locomotion'; the relationship between the two should be stated explicitly so that the reader does not interpret the later 3/3 as a retest of the same task under the original evaluation conditions.
[Section 5.4] The pose estimation section states that the single-block variant 'performed slightly better in the presence of occlusion' but does not report the data supporting that comparison; a reference or a brief quantitative statement would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the CENTAURO system claim is an empirical integration claim; component methods are built on independent prior work and no prediction reduces to a fitted input.

full rationale

This is a systems-integration paper rather than a derivation paper, and none of its load-bearing claims reduces by construction to its own inputs. The strongest claim is that 'the integration into a holistic remote mobile manipulation system which is evaluated in a wide range of realistic tasks is novel and goes beyond the state of the art' (Section 2). That claim is supported by a physical system, a described architecture, and a reported evaluation; it is not obtained by plugging a fitted parameter into an equation and then reading the same quantity back out. The autonomous grasping component is the closest thing to a learned predictor: grasping poses are transferred from known category instances to 'a previously unknown' driller instance (Section 9.3), which is a legitimate generalization setting, not a fitted-input-called-prediction loop. The trajectory optimizer uses a cost function with obstacle, joint-limit, and duration terms and is evaluated on new query trajectories; the costs do not encode the evaluation outcomes. Self-citations appear frequently, but they point to component papers (locomotion planning, XBotCore, exoskeletons, pose estimation) whose assumptions do not include the present paper's integration claim, and the integration itself is demonstrated by the system's behavior rather than by citing those papers. The paper's own limitation statements are about empirical validity, not circularity: in Section 9.4, 'a person at location was allowed to give the robot a slight push to regain balance'; in Section 9.2, the snap hook 'was modified slightly to make it more easily graspable,' the cutting tool trigger was enlarged after failures, and a webcam was added for the screwdriver task; and Section 9 states 'When failures were encountered, more attempts were added to gain insight into the possible failure modes.' These passages weaken the 'without previous task-specific training' generalization claim and the breadth claim built on it, but they do not make the claim self-definitional: the system is not defined in terms of its test outcomes, and the success rates are empirical observations rather than quantities derived from the assumptions. A correctness or evaluation-protocol critique would be appropriate, but it is not a circularity critique. Accordingly, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim is an integration and demonstration claim, so the ledger is dominated by engineering modeling choices rather than fitted parameters. The main hand-set values are velocity setpoints, cost weights, task difficulty ratings, and the latent space size for grasp transfer; none are fitted to the reported success rates. Key assumptions include static environments for trajectory optimization, static-stability-only planning, reliable ground contact estimation, and operator experience transferring to unseen tasks. No new physical entities are postulated.

free parameters (4)

Task difficulty ratings = integer 1-10 per task
Assigned by operators before attempts (Table 2); used to categorize task complexity, not fitted to outcomes, but subjective and unvalidated.
Planner velocity setpoints vdes = 0.1 m/s near obstacles, 0.25 m/s otherwise
Section 7.4: desired linear velocity chosen by hand; affects path execution time and safety but not verified against alternatives.
Trajectory optimization cost weights lambda_j = in [0,1] per cost component
Section 8.2: weights such as lambda_obst set priorities among obstacle, duration, and torque costs; values are chosen per task, no tuning procedure reported.
Grasp transfer latent space dimension = not reported
Section 8.1: PCA-EM on deformation fields retains a lower-dimensional manifold; the retained dimensionality is a modeling choice not reported or sensitivity-analyzed, and it directly governs the expressiveness of the grasp transfer.

assumptions (4)

domain assumption The environment is static and the robot base does not move during manipulation trajectory execution.
Section 8.2 states this assumption explicitly; it excludes dynamic obstacles and base motion during autonomous arm movement, limiting the generality of the autonomous manipulation claim.
domain assumption Static stability is sufficient for stepping locomotion; dynamic effects can be neglected.
Section 7.2: 'Stability computation is limited to static stability since motion execution is sufficiently slow and thus, dynamic effects can be neglected.' This underpins the planner's step feasibility.
domain assumption Ground contact can be reliably detected from leg joint torques via forward dynamics.
Section 5.1; the semi-autonomous stepping controller halts downward foot motion on detected contact, so systematic contact errors would propagate to stepping failures; no quantitative validation of contact detection accuracy is given.
domain assumption Operators experienced with the interfaces can solve unseen tasks without task-specific training, and inspecting the task site beforehand does not constitute training.
Section 9: 'the tasks themselves were new to them' and 'training runs were not allowed', while operators were 'generally experienced in the operation of robots through the provided interfaces'; the transferability of prior interface experience to new tasks is assumed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Remote Mobile Manipulation with the Centauro Robot: Full-body Telepresence and Autonomous Operator Assistance." pith.science (2026). https://pith.science/paper/ARJUMBET

@misc{pith2026190801617,
  author       = {Pith},
  title        = {Pith review of: Remote Mobile Manipulation with the Centauro Robot: Full-body Telepresence and Autonomous Operator Assistance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ARJUMBET}},
  note         = {Machine review of arXiv:1908.01617}
}

read the original abstract

Solving mobile manipulation tasks in inaccessible and dangerous environments is an important application of robots to support humans. Example domains are construction and maintenance of manned and unmanned stations on the moon and other planets. Suitable platforms require flexible and robust hardware, a locomotion approach that allows for navigating a wide variety of terrains, dexterous manipulation capabilities, and respective user interfaces. We present the CENTAURO system which has been designed for these requirements and consists of the Centauro robot and a set of advanced operator interfaces with complementary strength enabling the system to solve a wide range of realistic mobile manipulation tasks. The robot possesses a centaur-like body plan and is driven by torque-controlled compliant actuators. Four articulated legs ending in steerable wheels allow for omnidirectional driving as well as for making steps. An anthropomorphic upper body with two arms ending in five-finger hands enables human-like manipulation. The robot perceives its environment through a suite of multimodal sensors. The resulting platform complexity goes beyond the complexity of most known systems which puts the focus on a suitable operator interface. An operator can control the robot through a telepresence suit, which allows for flexibly solving a large variety of mobile manipulation tasks. Locomotion and manipulation functionalities on different levels of autonomy support the operation. The proposed user interfaces enable solving a wide variety of tasks without previous task-specific training. The integrated system is evaluated in numerous teleoperated experiments that are described along with lessons learned.

Figures

Figures reproduced from arXiv: 1908.01617 by the authors.

**Figure 1.** The Centauro robot. 1 Introduction Capable mobile manipulation robots are desperately needed in environments which are inaccessible or dangerous for humans. Missions include construction and maintenance of manned and unmanned stations, as well as exploration of unknown environments on the moon and other planets. Furthermore, such systems can be employed in search and rescue missions on earth. It applies to all thes… view at source ↗

**Figure 2.** System architecture overview. 3 System Overview The proposed system architecture possesses multiple hardware and software components. An overview over these components and the communication architecture between them is given in this section. Starting point is the Centauro robot. It perceives information about its internal state and the environment through a 3D rotating laser scanner, a RGB-D sensor, multiple RGB cam… view at source ↗

**Figure 3.** Kinematic layout. Left: The right arm. Joint axes are marked with colored lines. Center: [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (36 more)

**Figure 4.** Figure 4: Centauro manipulation end-effectors: 1 DoF SoftHand (l.) and anthropomorphic 9 DoF Schunk [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Manufactured actuation units of the identified classes (f.l.t.r.): Large, Medium, Small. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: XBotCore threads and communication architecture [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: The full-body telepresence suit, components and implementation. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

**Figure 8.** Figure 8: (a),(b) The arm exoskeleton features four actuated DoFs (shoulder and elbow) in a wide operative [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

**Figure 9.** Figure 9: Wrist exoskeleton design: (a) kinematic scheme, (b) CAD model, (c) schematic representation of [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

**Figure 10.** Figure 10: The wrist exoskeleton worn by an operator. Left: Isolated test. Right: Mounted on the arm [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

**Figure 11.** Figure 11: The hand exoskeleton worn by an operator. [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

**Figure 12.** Figure 12: Centauro overcoming a step field: Scenario (left), localized robot and registered point cloud color [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]

**Figure 13.** Figure 13: Turntable capture and scene synthesis. Left: Different drills on the turntable as captured by a [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]

**Figure 14.** Figure 14: Pose estimation network architecture. a) Single-block output variant; b) Multi-block output. [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]

**Figure 15.** Figure 15: Pipeline for 3D simulation. Sensors are colored red and pipeline components yellow. [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]

**Figure 16.** Figure 16: l.: VEROSIM provides a digital twin of the Centauro robot encompassing all links, joints, sensors etc., c.: third person view (standard and stereoscopic) on the current scene with a rigid body height map overlaid by a cost map accompanied by color-coded point cloud da…

**Figure 17.** Figure 17: Overview of visualization possibilities: t.l.: head-up display visualization of internal parameters, [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]

**Figure 18.** Figure 18: Centauro in a simulated Mars mission. 3D VEROSIM visualization Robot state & Keyframe editor Panoramic view & RGB Kinect image Foot cameras Pointcloud, ground contact & COM markers Task specific GUI Monitor 1 Monitor 2 Monitor 3 [PITH_FULL_IMAGE:figures/full_fig_p020…

**Figure 19.** Figure 19: Environment and robot state visualization for the support operators. [PITH_FULL_IMAGE:figures/full_fig_p020_19.png]

**Figure 20.** Figure 20: Locomotion control interfaces: a) 4D joystick and 3D pedal controller, b) keyframe editor, c) [PITH_FULL_IMAGE:figures/full_fig_p021_20.png]

**Figure 21.** Figure 21: An operator opening a door teleoperating Centauro with the upper-body exoskeleton. [PITH_FULL_IMAGE:figures/full_fig_p022_21.png]

**Figure 22.** Figure 22: Teleoperation scheme between the Hand Exoskeleton and the Schunk robotic hand. [PITH_FULL_IMAGE:figures/full_fig_p023_22.png]

**Figure 23.** Figure 23: 6D input device (l.) and corresponding GUI (r.) for dexterous wrist control. [PITH_FULL_IMAGE:figures/full_fig_p023_23.png]

**Figure 24.** Figure 24: Overview of the pipeline for locomotion planning. Sensors are colored red, pipeline components [PITH_FULL_IMAGE:figures/full_fig_p024_24.png]

**Figure 25.** Figure 25: Environment representation: The map shows two walls, a ramp and two poles of different height. [PITH_FULL_IMAGE:figures/full_fig_p025_25.png]

**Figure 26.** Figure 26: Neighbor states can be reached by driving or stepping related motions: a) Omnidirectional [PITH_FULL_IMAGE:figures/full_fig_p025_26.png]

**Figure 27.** Figure 27: The planning representation is split into three levels of abstraction. Coarser representations are [PITH_FULL_IMAGE:figures/full_fig_p026_27.png]

**Figure 28.** Figure 28: Representation level positioning. A fine planning representation is only provided in the vicinity [PITH_FULL_IMAGE:figures/full_fig_p026_28.png]

**Figure 29.** Figure 29: Pipeline for autonomous manipulation. Sensors are colored red, pipeline components yellow, and [PITH_FULL_IMAGE:figures/full_fig_p027_29.png]

**Figure 30.** Figure 30: Training phase. Deformations between the canonical model and each instance are calculated by [PITH_FULL_IMAGE:figures/full_fig_p028_30.png]

**Figure 31.** Figure 31: Transferring grasping knowledge to the presented novel instance. The input point cloud is at the [PITH_FULL_IMAGE:figures/full_fig_p029_31.png]

**Figure 32.** Figure 32: Two qualitatively different trajectories generated by our trajectory optimization: priority on [PITH_FULL_IMAGE:figures/full_fig_p030_32.png]

**Figure 33.** Figure 33: Left: Opening the door and driving through it. Right: Overcoming a gap with the Centauro [PITH_FULL_IMAGE:figures/full_fig_p032_33.png]

**Figure 34.** Figure 34: Complex locomotion tasks. Left: Traversing a step field. Right: Climbing stairs. [PITH_FULL_IMAGE:figures/full_fig_p032_34.png]

**Figure 35.** Figure 35: Valve experiments. Left to right: Grasping and turning of the green gate type valve, overview [PITH_FULL_IMAGE:figures/full_fig_p033_35.png]

**Figure 36.** Figure 36: Left: Clipping a snap hook on a fixed metal bar. Right: The fire hose plug to be removed. [PITH_FULL_IMAGE:figures/full_fig_p033_36.png]

**Figure 37.** Figure 37: Manipulation experiments. Left: Inserting a power plug. Right: Cutting a fixed wire using a [PITH_FULL_IMAGE:figures/full_fig_p033_37.png]

**Figure 38.** Figure 38: Power tool usage. Left: Drilling a hole into wood. Right: Driving a screw into wood. [PITH_FULL_IMAGE:figures/full_fig_p035_38.png]

**Figure 39.** Figure 39: Autonomous grasping: approaching (left), grasping (center), and lifting the drill (right). [PITH_FULL_IMAGE:figures/full_fig_p035_39.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages

[1]

P., and Rizzi, A

Abe, Y., Stephens, B., Murphy, M. P., and Rizzi, A. A. (2013). Dynamic whole-body robotic manipulation. Unmanned Systems Technology XV , 8741

work page 2013
[2]

Atorf, L., Cichon, T., and Ro mann, J. (2015). Flexible data logging, management, and analysis of simulation results of complex systems for eRobotics applications. In European Simulation and Modelling Conference (ESM)

work page 2015
[3]

Atorf, L., Schluse, M., and Rossmann, J. (2014). Simulation-based optimization, reasoning, and control: The eRobotics approach towards intelligent robots. In International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS)

work page 2014
[4]

Baccelliere, L., Kashiri, N., Muratore, L., Laurenzi, A., Kamedula, M., Margan, A., Cordasco, S., Malzahn, J., and Tsagarakis, N. G. (2017). Development of a human size and strength compliant bi-manual platform for realistic heavy manipulation tasks. In IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS)

work page 2017
[5]

Buongiorno, D., Sotgiu, E., Leonardis, D., Marcheschi, S., Solazzi, M., and Frisoli, A. (2018). WRES : a novel 3DoF WR ist E xo S keleton with tendon-driven differential transmission for neuro-rehabilitation and teleoperation. IEEE Robotics and Automation Letters (RAL) , 3(3):2152--2159

work page 2018
[6]

G., Grioli, G., Farnioli, E., Serio, A., Piazza, C., and Bicchi, A

Catalano, M. G., Grioli, G., Farnioli, E., Serio, A., Piazza, C., and Bicchi, A. (2014). Adaptive synergies for the design and control of the P isa/ IIT S oft H and. International Journal of Robotics Research (IJRR) , 33(5):768--782

work page 2014
[7]

Cichon, T., Loconsole, C., Buongiorno, D., Solazzi, M., Schlette, C., Frisoli, A., and Ro mann, J. (2016). Combining an exoskeleton with 3D simulation in-the-loop. In International Workshop on Human Friendly Robotics (HFR) , pages 31--34

work page 2016
[8]

and Ro mann, J

Cichon, T. and Ro mann, J. (2017a). Robotic teleoperation: Mediated and supported by virtual testbeds. In International Symposium on Safety, Security, and Rescue Robotics (SSRR)

work page 2017

Show all 56 references

[9]

and Ro mann, J

Cichon, T. and Ro mann, J. (2017b). Simulation-based user interfaces for digital twins: Pre-, in-, or post-operational analysis and exploration of virtual testbeds. In European Simulation and Modelling Conference (ESM)

2017
[10]

de Boor, C. (1978). A Practical Guide to Splines . Springer

1978
[11]

Droeschel, D., Schwarz, M., and Behnke, S. (2017). Continuous mapping and localization for autonomous navigation in rough terrain using a 3D laser scanner. Robotics and Autonomous Systems , 88:104 -- 115

2017
[12]

Fankhauser, P., Bloesch, M., Rodriguez, D., Kaestner, R., Hutter, M., and Siegwart, R. (2015). Kinect v2 for mobile robot navigation: Evaluation and modeling. In International Conference on Advanced Robotics (ICAR)

2015
[13]

Gabardi, M., Solazzi, M., Leonardis, D., and Frisoli, A. (2018). Design and evaluation of a novel 5 DoF underactuated thumb-exoskeleton. IEEE Robotics and Automation Letters (RAL) , 3(3):2322--2329

2018
[14]

a nen, I., Suomela, J., Yl \

Halme, A., Lepp \"a nen, I., Suomela, J., Yl \"o nen, S., and Kettunen, I. (2003). Work P artner: Interactive human-like service robot for outdoor applications. International Journal of Robotics Research (IJRR) , 22(7-8):627--640

2003
[15]

Hebert, P., Bajracharya, M., Ma, J., Hudson, N., Aydemir, A., Reid, J., Bergh, C., Borders, J., Frost, M., Hagman, M., et al. (2015). Mobile manipulation and mobility as manipulation—design and algorithms of R obo S imian. Journal of Field Robotics , 32(2):255--274

2015
[16]

Hirose, S., Yokota, S., Torii, A., Ogata, M., Suganuma, S., Takita, K., and Kato, K. (2005). Quadruped walking robot centered demining system - development of TITAN-IX and its operation. In IEEE International Conference on Robotics and Automation (ICRA)

2005
[17]

M., Rocchi, A., Laurenzi, A., and Tsagarakis, N

Hoffman, E. M., Rocchi, A., Laurenzi, A., and Tsagarakis, N. G. (2017). Robot control for dummies: Insights and examples using opensot. In IEEE - RAS International Conference on Humanoid Robots (Humanoids)

2017
[18]

Johnson, M., Shrewsbury, B., Bertrand, S., Wu, T., Duran, D., Floyd, M., Abeles, P., Stephen, D., Mertins, N., Lesman, A., et al. (2015). Team IHMC 's lessons learned from the DARPA robotics challenge trials. Journal of Field Robotics , 32(2):192--208

2015
[19]

Kalakrishnan, M., Chitta, S., Theodorou, E., Pastor, P., and Schaal, S. (2011). S T O M P : Stochastic trajectory optimization for motion planning. In IEEE International Conference on Robotics and Automation (ICRA)

2011
[20]

G., and Tsagarakis, N

Kamedula, M., Kashiri, N., Caldwell, D. G., and Tsagarakis, N. G. (2016). A compliant actuation dynamics Gazebo-ROS plugin for effective simulation of soft robotics systems: Application to CENTAURO robot. In International Conference on Informatics in Control , Automation and R...

2016
[21]

G., and Tsagarakis, N

Kashiri, N., Ajoudani, A., Caldwell, D. G., and Tsagarakis, N. G. (2016). Evaluation of hip kinematics influence on the performance of a quadrupedal robot leg. In International Conference on Informatics in Control , Automation and Robotics (ICINCO)

2016
[22]

Kashiri, N., Malzahn, J., and Tsagarakis, N. (2017). On the sensor design of torque controlled actuators: A comparison study of strain gauge and encoder based principles. IEEE Robotics and Automation Letters (RAL) , 2(2):1186--1194

2017
[23]

G., Laffranchi, M., and Caldwell, D

Kashiri, N., Tsagarakis, N. G., Laffranchi, M., and Caldwell, D. G. (2013). On the stiffness design of intrinsic compliant manipulators. In IEEE / ASME International Conference on Advanced Intelligent Mechatronics (AIM)

2013
[24]

and Behnke, S

Klamt, T. and Behnke, S. (2017). Anytime hybrid driving-stepping locomotion planning. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2017
[25]

and Behnke, S

Klamt, T. and Behnke, S. (2018). Planning hybrid driving-stepping locomotion on multiple levels of abstraction. In IEEE International Conference on Robotics and Automation (ICRA)

2018
[26]

Klamt, T., Rodriguez, D., Schwarz, M., Lenz, C., Pavlichenko, D., Droeschel, D., and Behnke, S. (2018). Supervised autonomous locomotion and manipulation for disaster response with a centaur-like robot. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2018
[27]

Krotkov, E., Hackett, D., Jackel, L., Perschbacher, M., Pippine, J., Strauss, J., Pratt, G., and Orlowski, C. (2017). The DARPA robotics challenge finals: Results and perspectives. Journal of Field Robotics , 34(2):229--240

2017
[28]

J., Forss \'e n, P.-E., and Ovr \'e n, H

Lawin, F. J., Forss \'e n, P.-E., and Ovr \'e n, H. (2016). Efficient multi-frequency phase unwrapping using kernel density estimation. In European Conference on Computer Vision (ECCV)

2016
[29]

J., and Thrun, S

Likhachev, M., Gordon, G. J., and Thrun, S. (2003). ARA *: Anytime A* with provable bounds on sub-optimality. In Advances in Neural Information Processing Systems (NIPS)

2003
[30]

Lin, G., Milan, A., Shen, C., and Reid, I. (2017). Refinenet: Multi-path refinement networks with identity mappings for high-resolution semantic segmentation. In International Conference on Computer Vision and Pattern Recognition (CVPR)

2017
[31]

S., Strawser, P., Bridgwater, L., Verdeyen, W

Mehling, J. S., Strawser, P., Bridgwater, L., Verdeyen, W. K., and Rovekamp, R. (2007). Centaur: NASA 's mobile humanoid designed for field work. In IEEE International Conference on Robotics and Automation (ICRA)

2007
[32]

M., Rocchi, A., Caldwell, D

Muratore, L., Laurenzi, A., Hoffman, E. M., Rocchi, A., Caldwell, D. G., and Tsagarakis, N. G. (2017). XBotCore : A real-time cross-robot software platform. In IEEE International Conference on Robotic Computing (IRC)

2017
[33]

and Song, X

Myronenko, A. and Song, X. (2010). Point set registration: Coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 32(12):2262--2275

2010
[34]

and Behnke, S

Pavlichenko, D. and Behnke, S. (2017). Efficient stochastic multicriteria arm trajectory optimization. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2017
[35]

S., and Behnke, S

Pavlichenko, D., Rodriguez, D., Schwarz, M., Lenz, C., Periyasamy, A. S., and Behnke, S. (2018). Autonomous dual-arm manipulation of familiar objects. In IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids)

2018
[36]

Pirondini, E., Coscia, M., Marcheschi, S., Roas, G., Salsedo, F., Frisoli, A., Bergamasco, M., and Micera, S. (2014). Evaluation of a new exoskeleton for upper limb post-stroke neuro-rehabilitation: Preliminary results. In Replace, Repair, Restore, Relieve--Bridging Clinical a...

2014
[37]

Raibert, M., Blankespoor, K., Nelson, G., and Playter, R. (2008). Big D og, the rough-terrain quadruped robot. IFAC Proceedings Volumes , 41(2):10822--10825

2008
[38]

F., Muratore, L., Laurenzi, A., Hoffman, E

Rigano, G. F., Muratore, L., Laurenzi, A., Hoffman, E. M., and Tsagarakis, N. G. (2018). Towards a robot hardware abstraction layer (R-HAL) leveraging the xbot software framework. In IEEE International Conference on Robotic Computing (IRC)

2018
[39]

Rodehutskors, T., Schwarz, M., and Behnke, S. (2015). Intuitive bimanual telemanipulation under communication restrictions by immersive 3D visualization and motion tracking. In IEEE-RAS International Conference on Humanoid Robots (Humanoids)

2015
[40]

and Behnke, S

Rodriguez, D. and Behnke, S. (2018). Transferring category-based functional grasping skills by latent space non-rigid registration. In IEEE Robotics and Automation Letters (RA-L) , pages 2662--2669

2018
[41]

Rodriguez, D., Cogswell, C., Koo, S., and Sven, B. (2018). Transferring grasping skills to novel instances by latent space non-rigid registration. In IEEE International Conference on Robotics and Automation (ICRA)

2018
[42]

Roennau, A., Heppner, G., Nowicki, M., and Dillmann, R. (2014). LAURON V : A versatile six-legged walking robot with advanced maneuverability. In IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM)

2014
[43]

G., and Tsagarakis, N

Roozing, W., Malzahn, J., Kashiri, N., Caldwell, D. G., and Tsagarakis, N. G. (2017). On the stiffness selection for torque-controlled series-elastic actuators. IEEE Robotics and Automation Letters (RAL) , 2(4):2255--2262

2017
[44]

W., Parlitz, C., Heppner, G., Hermann, A., Roennau, A., and Dillmann, R

Ruehl, S. W., Parlitz, C., Heppner, G., Hermann, A., Roennau, A., and Dillmann, R. (2014). Experimental evaluation of the S chunk 5-finger gripping hand for grasping tasks. In IEEE International Conference on Robotics and Biomimetics (ROBIO)

2014
[45]

Sarac, M., Solazzi, M., Sotgiu, E., Bergamasco, M., and Frisoli, A. (2017). Design and kinematic optimization of a novel underactuated robotic hand exoskeleton. Meccanica---International Journal of Theoretical and Applied Mechanics , 52(3):749--761

2017
[46]

S., Lenz, C., Schreiber, M., and Behnke, S

Schwarz, M., Beul, M., Droeschel, D., Sch \"u ller, S., Periyasamy, A. S., Lenz, C., Schreiber, M., and Behnke, S. (2016a). Supervised autonomy for exploration and mobile manipulation in rough terrain with a centaur-like robot. Frontiers in Robotics and AI , 3:57

2016
[47]

M., Koo, S., Periyasamy, A

Schwarz, M., Lenz, C., García, G. M., Koo, S., Periyasamy, A. S., Schreiber, M., and Behnke, S. (2018). Fast object learning and dual-arm coordination for cluttered stowing, picking, and packing. In IEEE International Conference on Robotics and Automation (ICRA)

2018
[48]

Schwarz, M., Rodehutskors, T., Droeschel, D., Beul, M., Schreiber, M., Araslanov, N., Ivanov, I., Lenz, C., Razlaw, J., Sch \"u ller, S., Schwarz, D., Topalidou-Kyniazopoulou, A., and Behnke, S. (2017). Nimb R o R escue: Solving disaster-response tasks with the mobile manipula...

2017
[49]

Schwarz, M., Rodehutskors, T., Schreiber, M., and Behnke, S. (2016b). Hybrid driving-stepping locomotion with the wheeled-legged robot Momaro . In IEEE International Conference on Robotics and Automation (ICRA)

2016
[50]

G., Guglielmino, E., Focchi, M., Cannella, F., and Caldwell, D

Semini, C., Tsagarakis, N. G., Guglielmino, E., Focchi, M., Cannella, F., and Caldwell, D. G. (2011). Design of H y Q - a hydraulically and electrically actuated quadruped robot. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Eng...

2011
[51]

C., Stager, D., Zajac, B., Bagnell, J

Stentz, A., Herman, H., Kelly, A., Meyhofer, E., Haynes, G. C., Stager, D., Zajac, B., Bagnell, J. A., Brindza, J., Dellin, C., et al. (2015). CHIMP , the CMU highly intelligent mobile platform. Journal of Field Robotics , 32(2):209--228

2015
[52]

St \"u ckler, J., Schwarz, M., Schadler, M., Topalidou-Kyniazopoulou, A., and Behnke, S. (2016). Nimb R o E xplorer: Semi-autonomous exploration and mobile manipulation in rough terrain. Journal of Field Robotics , 33(4):411--430

2016
[53]

G., Caldwell, D

Tsagarakis, N. G., Caldwell, D. G., Negrello, F., Choi, W., Baccelliere, L., Loc, V., Noorden, J., Muratore, L., Margan, A., Cardellino, A., et al. (2017). WALK-MAN: A high-performance humanoid platform for realistic environments. Journal of Field Robotics , 34(7):1225--1259

2017
[54]

G., and Semini, C

Ur Rehman, B., Focchi, M., Frigerio, M., Goldsmith, J., Caldwell, D. G., and Semini, C. (2015). Design of a hydraulically actuated arm for a quadruped robot. In International Conference on Assistive Robotics ( CLAWAR )

2015
[55]

Yamauchi, B. M. (2004). Pack B ot: A versatile platform for military robotics. In Proceedings of SPIE, Unmanned Ground Vehicle Technology VI , volume 5422

2004
[56]

X., Rasmussen, C., Huang, E., Stilman, M., and Bobick, A

Zucker, M., Joo, S., Grey, M. X., Rasmussen, C., Huang, E., Stilman, M., and Bobick, A. (2015). A general-purpose system for teleoperation of the DRC-HUBO humanoid robot. Journal of Field Robotics , 32(3):336--351

2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

P., and Rizzi, A

Abe, Y., Stephens, B., Murphy, M. P., and Rizzi, A. A. (2013). Dynamic whole-body robotic manipulation. Unmanned Systems Technology XV , 8741

work page 2013

[2] [2]

Atorf, L., Cichon, T., and Ro mann, J. (2015). Flexible data logging, management, and analysis of simulation results of complex systems for eRobotics applications. In European Simulation and Modelling Conference (ESM)

work page 2015

[3] [3]

Atorf, L., Schluse, M., and Rossmann, J. (2014). Simulation-based optimization, reasoning, and control: The eRobotics approach towards intelligent robots. In International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS)

work page 2014

[4] [4]

Baccelliere, L., Kashiri, N., Muratore, L., Laurenzi, A., Kamedula, M., Margan, A., Cordasco, S., Malzahn, J., and Tsagarakis, N. G. (2017). Development of a human size and strength compliant bi-manual platform for realistic heavy manipulation tasks. In IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS)

work page 2017

[5] [5]

Buongiorno, D., Sotgiu, E., Leonardis, D., Marcheschi, S., Solazzi, M., and Frisoli, A. (2018). WRES : a novel 3DoF WR ist E xo S keleton with tendon-driven differential transmission for neuro-rehabilitation and teleoperation. IEEE Robotics and Automation Letters (RAL) , 3(3):2152--2159

work page 2018

[6] [6]

G., Grioli, G., Farnioli, E., Serio, A., Piazza, C., and Bicchi, A

Catalano, M. G., Grioli, G., Farnioli, E., Serio, A., Piazza, C., and Bicchi, A. (2014). Adaptive synergies for the design and control of the P isa/ IIT S oft H and. International Journal of Robotics Research (IJRR) , 33(5):768--782

work page 2014

[7] [7]

Cichon, T., Loconsole, C., Buongiorno, D., Solazzi, M., Schlette, C., Frisoli, A., and Ro mann, J. (2016). Combining an exoskeleton with 3D simulation in-the-loop. In International Workshop on Human Friendly Robotics (HFR) , pages 31--34

work page 2016

[8] [8]

and Ro mann, J

Cichon, T. and Ro mann, J. (2017a). Robotic teleoperation: Mediated and supported by virtual testbeds. In International Symposium on Safety, Security, and Rescue Robotics (SSRR)

work page 2017