{"id":"d4c11708-ffce-40b5-80d5-b7c947b99a50","arxiv_id":"2506.16475","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Pretraining a modular transformer policy on human demonstrations then finetuning on a small robot dataset improves success on six real quadruped manipulation tasks, including out-of-distribution objects.","lead":"This paper shows that training a quadruped robot's control policy first on human demonstration videos, then on a small set of robot demonstrations, improves the robot's household manipulation success rate. The authors built a teleoperation system that records matched human and robot actions, plus a new neural network architecture to make the human data useful for the four-legged robot.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'OOD' transfer claim is not out-of-distribution for the pretrained model: Section IV-A.1 places the OOD objects in human pretraining, so the headline 82.7% gain conflates object familiarity with cross-embodiment skill transfer.","rationale":"The reader's weakest_assumption concerns unquantified teleoperation tracking error in Eq. 1 and Section III-B. That is a legitimate robustness concern, but it is less load-bearing for the central transfer claim than the OOD contamination, because imperfect tracking degrades the absolute quality of all robot demonstrations and affects every method equally; it does not selectively undermine the causal attribution of transfer to human pretraining. The OOD overlap is directly acknowledged in the paper's own task description, so it is not a speculative alternative explanation. It gives a concrete, testable way in which the measured OOD gain could be real but not evidence of cross-embodiment generalization. The paper has independent support: open-sourced code, hardware, and datasets, and a consistent positive trend across tasks, which is why I do not call for rejection. However, the headline OOD numbers need to be re-derived with the test objects held out of pretraining, or the claims must be reworded as generalization to objects unseen in robot finetuning but present in human pretraining. Conditional acceptance with that requirement remains the appropriate outcome.","tokens_in":22950,"tokens_out":5347,"duration_ms":50681,"concrete_test":"Hold the OOD objects out of human pretraining. For TC-Uni and TC-Bi, pretrain MXT only on human trajectories involving the same 10 objects used for robot finetuning (or a disjoint held-out split), then finetune on robot data and evaluate on the current OOD object set. If the pretrained model retains its roughly 83-100% OOD success rates, the leakage explanation is falsified; if OOD success drops toward the MXT-scratch level, the 82.7% headline gain is an artifact of pretraining exposure rather than cross-embodiment transfer. Additionally, report per-task OOD deltas separately for tasks with and without pretraining overlap so the aggregate can be recomputed without contaminated rows.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section IV-A.1, for the Toy Collection tasks the paper states that \"we use 10 objects for robot finetuning and all objects for human pretraining and real-robot evaluation,\" and Fig. 4 labels the held-out test items as OOD objects. For the pretrained MXT policy these objects are therefore not out-of-distribution: their appearance and affordances were seen during human-data pretraining. The central quantitative claims that \"pretraining with human data contributes a 38.6% success rate improvement overall and 82.7% under OOD settings,\" and the corresponding 79.7% OOD advantage over HIT, are averaged over rows that include these leaked objects. The OOD rows are the strongest evidence for embodiment-agnostic skill transfer, yet they cannot distinguish cross-embodiment transfer from plain test-object familiarity acquired during pretraining. The same leakage also inflates the overall 38.6% number. What remains supported is a positive ID transfer effect on tasks with overlapping or generic object sets, but the headline OOD generalization claim, as currently measured, is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"Human2LocoMan introduces an XR-based teleoperation and data-collection pipeline that unifies human and LocoMan observations and actions, together with a modularized cross-embodiment Transformer (MXT) that is pretrained on human demonstrations and then finetuned on small robot datasets. The paper reports real-robot success rates on six household tasks in unimanual and bimanual modes, and claims that human pretraining contributes a 38.6% success-rate improvement overall and 82.7% under OOD settings, with larger gains over the HIT baseline. The paper also releases code, hardware, and data.","tokens_in":23222,"tokens_out":10043,"duration_ms":83534,"significance":"If the central transfer claim held, this would be a meaningful contribution to cross-embodiment imitation learning for quadrupedal manipulation, combining a practical data-collection interface with an open-source dataset and policy architecture. The within-distribution comparisons in Table II do show consistent positive deltas for MXT-Pretrained over MXT-Scratch on several tasks, and the modular MXT design outperforms the HPT-style aggregated baseline in the ablation. The paper is also commendable for open-sourcing code, hardware, and data and for evaluating on real hardware rather than only in simulation. However, the headline OOD generalization claim is currently confounded by pretraining-data leakage for the toy-collection tasks, and the aggregate percentages in the abstract do not reproduce from the reported table under standard aggregation rules.","major_comments":[{"comment":"For the Toy Collection tasks, Section IV-A.1 states that \"we use 10 objects for robot finetuning and all objects for human pretraining and real-robot evaluation,\" and Figure 4 labels the held-out test items as OOD objects. For MXT-Pretrained, these objects are therefore not out-of-distribution: their appearance and affordances were seen during human-data pretraining. Since the OOD rows are the strongest evidence for embodiment-agnostic transfer, the claimed 82.7% OOD gain conflates object familiarity acquired during pretraining with cross-embodiment skill transfer. Please either exclude the OOD evaluation objects from human pretraining, or rename and re-analyze the condition as \"novel to robot finetuning but seen in human pretraining,\" and adjust the central OOD claims accordingly.","section":"IV-A.1, IV-B, Table II"},{"comment":"The aggregated improvements \"38.6% overall and 82.7% under OOD\" for MXT-Pretrained over MXT-Scratch, and \"41.9% overall and 79.7% under OOD\" over HIT, are not reproducible from Table II under standard aggregation. For example, averaging the per-task OOD success-rate deltas between MXT-Pretrained (larger) and MXT-Scratch (larger) gives 31.95 percentage points, and including both dataset sizes gives about 33.3 percentage points; I could not reproduce 82.7 with natural weightings of the 24 ID and 12 OOD trials. Please state the exact aggregation rule (which rows, which baseline, absolute or relative changes) and verify the headline numbers, since the abstract and introduction lead with these quantities.","section":"IV-B, Table II"},{"comment":"The caption of Figure 5 states that results are based on 36 trials comprising 24 OOD and 12 ID trials, whereas Section IV-A.6 and Table II state that ID results use 24 trials and OOD results use 12 trials. Please correct this inconsistency; if OOD trials were actually 24, the reported OOD percentages and the statistical strength of the OOD claims would need to be recomputed.","section":"Figure 5 caption and Table II"}],"minor_comments":[{"comment":"The notation in Eq. (1) refers to \"xgripper max\" and \"xgripper min\" as gripper angles, but the variables elsewhere are denoted by θ; please unify the notation.","section":"III-B, Eq. (1)"},{"comment":"Success rates with 12 OOD trials distinguish only a handful of successes per cell; several reported differences correspond to 1–4 successful rollouts. Including raw counts or confidence intervals would help the reader calibrate the OOD comparisons.","section":"IV-A.6, Table II"},{"comment":"The whole-body controller's tracking error is not quantified; since the teleoperation pipeline's validity depends on the controller following the mapped target poses, reporting measured end-effector and torso tracking error would strengthen the data-collection claims.","section":"III-B"}],"recommendation":"major_revision","confidential_remarks":"The core idea and the within-distribution transfer evidence are worth keeping, but the paper's most prominent quantitative claim—large OOD gains from human pretraining—is currently undermined by an explicit leakage in the toy-collection tasks and by aggregate numbers that do not match the table. A major revision that redefines the OOD protocol, reproduces the aggregates, and either drops or re-scopes the headline claim would be appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper's core finding—that pretraining on human demonstrations helps a quadruped with leg-mounted manipulators learn manipulation policies, especially with less robot data—is believable and backed by consistent in-distribution (ID) improvements on real hardware. But the attention-grabbing OOD numbers (82.7% improvement, 79.7% vs HIT) should not be taken at face value. The stress-test note is correct: for Toy Collection, Section IV-A.1 says all objects are used for human pretraining and real-robot evaluation, while only 10 are used for robot finetuning. Those 'OOD' objects are therefore in-distribution for the pretrained policy. The OOD rows in Table II include those leaked objects, so the OOD transfer claim conflates cross-embodiment skill transfer with plain object familiarity.\n\nWhat the paper does well: it ships. The teleoperation pipeline, the LocoMan dataset, the code, and the hardware details are open-sourced. MXT is a sensible modularization of the HPT-style trunk/tokenizer/detokenizer design, and the ablation against MXT-Agg gives some evidence that modality-level decomposition helps. The real-robot evaluations cover six tasks, unimanual and bimanual, and the ID deltas from pretraining are positive on every task (e.g., TC-Uni 87.5 to 95.8, TC-Bi 70.8 to 91.7, Scoop 62.5 to 87.5, Pour 79.2 to 91.7 for the larger data versions). That is a real result.\n\nSoft spots, in proportion. The OOD leakage is the main one; it hits the paper's most impressive-sounding claims. Second, the abstract's 41.9% and 38.6% improvements are not obviously reproducible from Table II—the aggregation method isn't stated, and a reader summing cells gets different numbers. Third, the whole-body controller tracking accuracy is never quantified; the teleop mapping in Eq. 1 assumes the controller follows the target poses, and if tracking is poor the demonstration quality is uncertain. That's a minor issue for the transfer claim, since the demonstrations still worked well enough to produce positive results, but it would be good to see end-effector tracking error. Fourth, the HPT baseline appears only on toy collection, and HIT is not pretrained with human data, so the strong baseline comparisons are a bit narrower than the prose suggests.\n\nWho should read it: anyone working on cross-embodiment imitation, legged manipulation, or human-data pretraining. It deserves a serious referee. The fix is straightforward: rerun or clearly relabel the OOD evaluation with objects truly unseen in pretraining, report the aggregation formula, and add confidence intervals. If the authors can't rerun, they should soften the OOD claim and let the ID result carry the paper. I'd accept it into review, but with the expectation of a major revision on the evaluation.","headline":"A solid cross-embodiment IL system with real robot gains from human pretraining, but the headline OOD numbers are inflated because the 'OOD' objects were already in the human pretraining data.","tokens_in":23754,"tokens_out":3161,"would_cite":true,"duration_ms":30751,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pretraining a quadruped-manipulation policy on human demonstrations, then finetuning it on a small robot dataset, raises real-world success rates by 38.6% on average and 82.7% on out-of-distribution objects.","keywords":["quadrupedal manipulation","imitation learning","cross-embodiment transfer","human pretraining","teleoperation","modular transformer policy","loco-manipulation","out-of-distribution generalization"],"falsifier":"Log the commanded target poses from Eq. (1) against the robot's actual torso and end-effector poses throughout teleoperation on every task, and compute the per-axis pose tracking error; then retrain the finetuned policy on only those demonstrations whose tracking error stays below a threshold. If success rates do not drop when poorly tracked demonstrations are removed, or if tracking error is large even in successful rollouts, the human-pretraining gain is not explained by faithful demonstrations.","tokens_in":22779,"feed_emoji":"🤖","tokens_out":6138,"duration_ms":57092,"temperature":0.7,"pith_summary":"This paper tries to establish that a quadrupedal robot with leg-mounted manipulators can learn household manipulation skills by first pretraining a policy on human demonstrations and then finetuning it on a small set of robot demonstrations. To make that possible, the authors build a teleoperation and data-collection pipeline that records human head, wrist, and grip motions and maps them to the robot's torso, end effectors, and grippers in a common coordinate frame, along with a Modularized Cross-embodiment Transformer (MXT) whose shared trunk is flanked by embodiment-specific tokenizers and detokenizers. On six real-world tasks spanning unimanual and bimanual, prehensile and non-prehensile, tool-use and long-horizon manipulation, the system reports large gains: pretraining with human data adds 38.6% success overall and 82.7% under out-of-distribution settings, and a pretrained model matches or beats baselines using only half the robot data. If correct, this is evidence that easy-to-collect human demonstrations can substitute for expensive robot demonstrations even when the human and robot bodies look very different.","feed_headline":"Human pretraining lifts quadruped manipulation success by 82.7%","feed_subtitle":"Pretraining on human demos, then finetuning on half the robot data, wins on six household tasks.","key_machinery":"The Modularized Cross-embodiment Transformer (MXT) — a transformer policy with a shared encoder-decoder trunk and per-embodiment, per-modality tokenizers and detokenizers — is the mechanism that carries the argument. It makes the policy consume a fixed sequence of tokens in which each semantic modality (main image, wrist image, body pose, end-effector pose, gripper action) is encoded and decoded separately, so the shared trunk sees aligned modality structure even when the human and quadruped embodiments differ; missing modalities are handled by masks. The teleoperation mapping in Eq. (1) plays the supporting role of turning human head and wrist motions into target torso and end-effector poses in a unified frame, so the human and robot datasets are spatially commensurable before training begins.","core_discovery":"The central claim is that positive cross-embodiment transfer from humans to a quadruped works when both embodiments' observations and actions are decomposed into aligned modalities (images, body pose, end-effector pose, gripper state) and processed by separate tokenizers and detokenizers around a shared transformer trunk. The human data are collected first, the MXT policy is pretrained on them with behavior cloning, then the tokenizers and detokenizers are reinitialized and the trunk is finetuned on LocoMan trajectories recorded through the same teleoperation interface. Across six tasks the pretrained policy reaches above 79% success on in-distribution settings and substantially higher out-of-distribution success than without pretraining: an average success-rate gain of 38.6% overall (41.9% vs the stronger imitation baseline) and 82.7% under OOD conditions, with gains concentrated in precise, long-horizon substeps such as scooping and pouring. The paper interprets this as the trunk learning modality-level manipulation structure from human data that transfers to a very different body.","pith_inferences":["If the transfer effect is as general as the paper's mechanism suggests, the same XR-captured human data could pretrain a trunk for other legged or wheeled manipulator platforms, provided their observations and actions are expressed in the same modality-aligned format; this is a testable extension the paper does not run.","The result implies that kinematic similarity between demonstrator and robot may not be necessary for imitation pretraining, only structured alignment of modalities; that is an editorial inference because the paper only demonstrates one robot body.","Because human data are cheaper and faster to collect (over 200 trajectories in 30 minutes, compared with over 50 robot trajectories), the measured gains suggest the bottleneck in robot imitation learning could shift from data volume to the quality of the teleoperation-to-robot mapping.","A natural next experiment, which the paper lists as future work, is simultaneous co-training on human and robot data in a multi-task setting; the sequential pretrain-finetune design leaves open whether mixing data during training would preserve or amplify the transfer."],"forward_implications":["Human demonstrations can replace about half of the robot demonstrations without hurting success, cutting the time and hardware cost of data collection for quadrupedal manipulation.","A single pretrained trunk can be finetuned into different manipulation modes (unimanual and bimanual) of the same robot, so pretraining cost is amortized across tasks.","Out-of-distribution generalization — new object shapes, colors, and placements — is where human pretraining helps most, not just average in-distribution performance.","The modular modality split, not merely the shared transformer trunk, is what preserves transfer; aggregating modalities into single tokenizers removes most of the benefit.","Human pretraining appears to matter most for precise substeps inside long-horizon tasks, such as localizing and scooping litter or pouring a ball, suggesting the transferred knowledge is about accuracy of coordinated motion, not just task semantics."],"supporting_citations":[{"why":"Supplies the quadruped hardware and the whole-body controller that the teleoperation pipeline adapts to track target torso and end-effector poses.","marker":"[14]"},{"why":"Provides the XR headset and motion-capture interface used to record human head and wrist motions and stream first-person or first-robot views.","marker":"[21]"},{"why":"Contributes the cross-embodiment transformer design (cross-attention tokenization, shared trunk, action readout) that MXT builds on and is compared against.","marker":"[78]"},{"why":"Serves as the strong imitation-learning baseline whose average success rate MXT's 41.9% overall improvement is measured against.","marker":"[20]"},{"why":"Represents the prior approach to cross-embodiment human-to-robot transfer that treats humans as a similar embodiment; the paper positions its own modality-structured transfer against it.","marker":"[31]"},{"why":"Supplies the action-chunking technique used by the detokenizers to predict fixed-horizon action sequences and reduce compounding errors.","marker":"[18]"},{"why":"Used by the whole-body controller to detect self-collisions during teleoperation, a safety filter that keeps recorded demonstrations valid.","marker":"[83]"}],"fun_headline_variants":["Human pretraining boosts quadruped manipulation 82.7% OOD","Learning from human demos halves robot data, 82.7% OOD gain","Cross-embodiment pretraining: human demos teach quadrupeds to manipulate","Quadruped learns from human hands, OOD success up 82.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result rests on the assumption that the teleoperation mapping and the whole-body controller actually deliver the target torso and end-effector poses with enough fidelity that the recorded robot trajectories are valid demonstrations; tracking error is never measured, so if the robot lags or distorts the human commands, both the robot dataset and the measured transfer gains are called into question.","fun_headline_variants_meta":{"raw":{"variants":["Human pretraining boosts quadruped manipulation 82.7% OOD","Learning from human demos halves robot data, 82.7% OOD gain","Cross-embodiment pretraining: human demos teach quadrupeds to manipulate","Quadruped learns from human hands, OOD success up 82.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001549,"raw_usage":{"total_tokens":6228,"prompt_tokens":1017,"completion_tokens":5211,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":5124}},"tokens_in":633,"tokens_out":5211,"duration_ms":34032,"temperature":1.0,"reasoning_tokens":5124,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:26:22.656132+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log the commanded target poses from Eq. (1) against the robot's actual torso and end-effector poses throughout teleoperation on every task, and compute the per-axis pose tracking error; then retrain the finetuned policy on only those demonstrations whose tracking error stays below a threshold. If success rates do not drop when poorly tracked demonstrations are removed, or if tracking error is large even in successful rollouts, the human-pretraining gain is not explained by faithful demonstrations.","supporting_citations":[{"cited_title":"The pinocchio c++ library: A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives","cited_arxiv_id":null,"evidence_quote":"Used by the whole-body controller to detect self-collisions during teleoperation, a safety filter that keeps recorded demonstrations valid."}],"review_version":1}