{"id":"1512538a-31fe-4e6c-952e-5c6b254fc7f3","arxiv_id":"2412.13196","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A teacher-student whole-body tracking controller with automated motion-data filtering and specialist fine-tuning outperforms prior methods on a Unitree G1 humanoid.","lead":"ExBody2 is a control system that teaches a humanoid robot to imitate many human motions, from walking and crouching to dancing and punching, using a mix of motion capture and simulation. It pairs a generalist policy trained on automatically filtered data with fine-tuned specialist policies, and reports better tracking accuracy than prior methods in simulation and on a real Unitree G1 robot.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The optimal filtering threshold is selected and evaluated on the same D_CMU split, so the reported curation benefit and the claimed transfer of tau* are not yet established; a holdout or cross-dataset test is needed.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the automated curation threshold is selected and evaluated on D_CMU, with transfer asserted but never tested. I agree with that assessment. This is the most direct threat to the paper's contribution (i), the automated curation pipeline, because the main comparison in Table II is not a clean estimate of the filtering benefit. The concern is not fatal to the entire central claim, since Exbody2-w/o-Filter already outperforms the three baselines on D_CMU, so the 'outperforms previous methods' statement may survive even if filtering provides no out-of-sample benefit. That is why the appropriate verdict remains CONDITIONAL rather than REJECT. I also note that the decoupled motion-velocity control is not independently ablated, but that omission is less decisive here because the combined system and its unfiltered variant both beat the baselines. The paper deserves credit for reporting the unfiltered variant, which makes the curation claim separable, and for including DAgger and history-length ablations. The missing piece is a holdout or cross-dataset evaluation of the threshold selection itself.","tokens_in":19820,"tokens_out":7193,"duration_ms":73020,"concrete_test":"Re-run the curation pipeline with D_CMU randomly split into selection and held-out evaluation halves: train pi_0 and choose tau* on the selection half only, then evaluate the resulting pi_tau* on the held-out half against pi_0 and an unfiltered policy trained on the selection half. If the advantage of pi_tau* over the unfiltered policy disappears or shrinks materially on the held-out half, the Table II curation gain is selection overfit and the claimed transfer of tau* is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section III.A.1 the authors define tau* = argmax_tau E_{s in D}[Performance(pi_tau, s)] and select it by evaluating several pi_tau on D_CMU. The main headline comparison in Table II is then also reported on D_CMU. Thus the improvement of Exbody2 over Exbody2-w/o-Filter (e.g., Empjpe 0.1355 -> 0.1079, Elower_mpjpe 0.1579 -> 0.1253) is measured on the same data used to choose the filter, so part of the apparent curation gain may be selection overfit. The paper's claim in Section IV.C that this threshold 'generalizes to other datasets' is asserted without a test; no experiment fixes tau* = 0.15 and evaluates on an independent dataset or a held-out split. The procedure also uses the initial policy pi_0's error e(s) = alpha Ekey(s) + beta Edof(s) as a proxy for infeasibility; because pi_0 is suboptimal, motions that are feasible for the final (better) policy but hard for pi_0 can be filtered out, conflating 'hard for pi_0' with 'infeasible.' This is load-bearing because the automated-curation component is presented as the source of the generalist-policy gains. Notably, Exbody2-w/o-Filter also beats the three baselines on D_CMU, so the overall outperformance claim might survive, but the specific benefit attributed to filtering and the claimed transfer to new datasets are not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ExBody2, a teacher-student reinforcement-learning framework for whole-body tracking on a Unitree G1 humanoid. A teacher policy trained with privileged information is distilled into a student policy via DAgger, and the method adds three claimed innovations: automated dataset curation that filters infeasible motion clips using the tracking error of an initial policy, a generalist-to-specialist fine-tuning paradigm, and a decoupled motion-velocity control strategy that tracks local keypoints while using root velocity for global guidance. Experiments on the CMU dataset and on the real robot report lower keypoint and joint errors than ExBody, ExBody*, and OmniH2O* baselines, and show that specialist fine-tuning improves accuracy over the generalist and from-scratch training. The paper also reports ablations of history length and DAgger distillation, and an appendix explores dataset size and feasibility-diversity trade-offs.","tokens_in":20325,"tokens_out":6498,"duration_ms":58812,"significance":"If the results hold, ExBody2 would be a strong step for expressive whole-body control: a single generalist policy outperforming prior whole-body trackers in simulation and on real hardware, with a principled, automated answer to the question of which mocap clips are worth training on. The paper's strengths are its real-robot deployment, the consistent use of multiple error metrics, and the informative ablations of history length and DAgger distillation, as well as the direct comparison of generalist, specialist, and from-scratch training in Table IV. The central unresolved issue is that the curation threshold is chosen and evaluated on the same benchmark, the decoupled-control innovation is never isolated, and the main tables lack variance reporting; these gaps weaken the attribution of the reported gains to the paper's specific contributions. If the missing holdout or cross-dataset validation and proper statistical reporting are added, the paper would be a solid contribution to the field.","major_comments":[{"comment":"The automated-curation threshold is selected and evaluated on the same benchmark. The objective 'τ* = arg max_τ E_{s∈D}[Performance(π_τ,s)]' in §III.A.1 is instantiated with D = D_CMU: Fig. 4 chooses τ*=0.15 from policies evaluated on D_CMU, and Table II then reports ExBody2 versus ExBody2-w/o-Filter on that same dataset. The curation benefit (e.g., Empjpe 0.1355→0.1079, Elower_mpjpe 0.1579→0.1253) can therefore include selection overfit, and the statement in §IV.C that the threshold 'generalizes to other datasets' is not backed by any experiment that fixes τ*=0.15 and evaluates on a held-out split or an independent dataset. In addition, the filter score e(s)=αE_key(s)+βE_dof(s) is produced by the initial policy π0; because π0 is suboptimal, motions that a later, better policy could track are removed if π0 finds them hard, conflating 'difficult for π0' with 'infeasible.' Please add a holdout/cross-dataset validation of the threshold and an explicit test of the infeasibility proxy before claiming the curation gain.","section":"§III.A.1 and §IV.C"},{"comment":"The decoupled motion-velocity control strategy is not ablated in isolation. The paper's comparison of ExBody2 with OmniH2O* in Tables II and III changes observation space, privileged information, reward design, and teacher-student training simultaneously, so the better tracking cannot be attributed specifically to the decoupled local-keypoint/velocity architecture described in §III.B.3. Please add a variant that keeps all other components fixed and replaces the local keypoint tracking plus velocity term with global keypoint tracking (or removes the decoupled velocity term), and report its errors on D_CMU and on the real-world subset.","section":"§III.B.3"},{"comment":"Headline results are reported without variance or significance. Tables II, III, IV and Fig. 4 give single numbers, whereas Appendix Table X says 'Statistically significant results are highlighted in bold across 5 random seeds'; no seed-level standard deviations or tests are reported for the main claims. Without error bars, small differences—for example ExBody2-w/o-Filter versus ExBody2 on upper-body metrics in Table II—cannot be distinguished from run-to-run noise. Please report mean±std over at least five seeds (or trials, for the real-robot results) for all headline comparisons, and state the number of seeds used.","section":"Tables II–IV"}],"minor_comments":[{"comment":"The text states that the specialist 'significantly outperforms' both the generalist and scratch policies on DACCAD, but in Table IV(d) the Generalist policy has a lower velocity error (Evel=0.3361) than the Specialist (0.4021); please qualify this claim and discuss the velocity-versus-pose trade-off.","section":"§IV.D.2"},{"comment":"In the reward-design paragraph, 'would discussed' should read 'will be discussed.'","section":"§III.B.1"},{"comment":"References [3] and [4] are the same arXiv paper (Expressive Whole-Body Control for Humanoid Robots) and should be merged.","section":"References"},{"comment":"The name of the method is spelled inconsistently as 'Exbody2' in the abstract and text and 'ExBody2' in the title; please standardize.","section":"Title/Abstract"},{"comment":"The notation 'Performance(πτ, s)' in the threshold objective is never formally defined; please specify which metric over a sequence is used (e.g., mean MPJPE) so that the optimization is reproducible.","section":"§III.A.1"}],"recommendation":"major_revision","confidential_remarks":"The reference list includes HOVER (arXiv:2410.21229) as [19] but the related-work section does not discuss it; since HOVER is a closely related neural whole-body controller, the authors should compare with or at least discuss it. I also recommend asking the authors to disclose the seed-level data behind Tables II–IV, because the appendix claims statistical significance across five seeds while the main tables report neither variance nor significance tests."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Ben, quick read of ExBody2 (2412.13196). Bottom line: this is a competent, well-engineered advance in humanoid whole-body tracking, with real G1 demos that back the main claims. The two genuinely new pieces are the decoupled motion-velocity control (local keypoint tracking plus explicit velocity goal) and the automated feasibility filtering based on the initial policy's error. The teacher-student PPO/DAgger pipeline and generalist-specialist finetune are standard, but they're applied cleanly and ablated properly (DAgger ablation, history length).\n\nThe paper's strongest evidence is the sim-to-real transfer: four baselines, chosen motions, real-robot joint errors that mirror simulation. The appendix dataset ablation with D50/D250/DCMU and evaluation on DACCAD is a nice addition, and reporting 5-seed significance there is better than the main tables, which lack error bars entirely.\n\nNow the soft spots, proportionate. The stress-test note is right about the curation threshold: tau* is picked by maximizing performance on D_CMU and then the headline comparison is on D_CMU. Section IV.C claims the threshold generalizes to other datasets without testing it. That's a real selection-on-evaluation issue. However, it's not load-bearing for the whole paper: ExBody2-w/o-Filter also beats Exbody and OmniH2O on D_CMU, so the single-policy outperformance claim does not depend on the filter. What is not established is the additional gain attributed to curation and the transfer of tau*. The appendix's D250 result is suggestive that moderate filtering helps, but it's a different selection procedure. A holdout or cross-dataset test would fix this.\n\nSecond soft spot: the decoupled control strategy is not ablated in isolation. The claim that it contributes to the gains is plausible, but the paper never removes it to show what degrades. Minor, because the real-robot results are strong, but the attribution is not pinned.\n\nThe paper is honest about its limitation (no seamless recombining of specialist policies). Citation pattern is fine; self-citation is to their own ExBody, which is the direct predecessor.\n\nWho is this for? People working on learning-based humanoid control, especially sim-to-real whole-body tracking. It deserves a serious referee – the engineering is solid, the videos presumably show the claims, and the curation question is a fixable experimental issue, not a fundamental flaw. I'd send it to review with a request for a cross-dataset threshold test and an ablation of the decoupled control.\n\nFinal recommendation: send to peer review. My own verdict would be conditional accept after those experiments.","headline":"Solid whole-body control engineering with real-robot results; the curation threshold is selected and evaluated on the same split, so that specific gain is not yet established, but the rest of the pipeline holds up.","tokens_in":20686,"tokens_out":1805,"would_cite":true,"duration_ms":17188,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ExBody2 claims that a humanoid robot can track diverse human whole-body motions with a single control policy when the controller decouples local keypoint tracking from root-velocity tracking and the training data is automatically filtered…","keywords":["humanoid whole-body control","motion imitation","teacher-student policy distillation","reinforcement learning","sim-to-real transfer","dataset filtering","expressive motion tracking","motion-velocity decoupled control"],"falsifier":"Hold out a random half of the motion corpus before training. Select the error cutoff using only the retained half, train the generalist on the filtered retained half, and measure its error on the held-out half against an unfiltered policy. If the filtered policy is not better on the held-out half, the claimed curation gain is overfitting to the threshold-selection set. A second check: rerun the filtering with a much stronger initial policy and see whether the same motion clips are discarded; if the rankings flip substantially, the filter is measuring the weakness of the first policy, not the feasibility of the motions.","tokens_in":19660,"feed_emoji":"🤖","tokens_out":12417,"duration_ms":107061,"temperature":0.7,"pith_summary":"ExBody2 argues that a humanoid robot can imitate expressive, dynamic human whole-body motions with a single control policy if the training data is automatically pruned to keep only motions the robot can physically track, and if the controller separates two goals: matching body landmarks in the robot's own coordinate frame and following the commanded root velocity. The paper builds a teacher-student pipeline in which a privileged reinforcement-learned teacher produces the data used to filter the motion corpus, and a student policy is distilled for deployment on a real humanoid. On the evaluation corpus, the authors report that this one generalist policy tracks keypoints and joint angles more accurately than the compared whole-body controllers, and that brief fine-tuning on a motion group yields specialist policies with still higher precision. The reason to care is that whole-body expressiveness, not just stable walking, is the missing piece between humanoid hardware and natural behavior in human spaces.","feed_headline":"A single policy now beats prior whole-body humanoid trackers","feed_subtitle":"Automated filtering plus decoupled velocity and keypoint tracking drives the gain; fine-tuning adds precision.","key_machinery":"The load-bearing mechanism is the motion-velocity decoupled control strategy combined with automated feasibility filtering. Instead of chasing global keypoint coordinates, the controller maps keypoints into the robot's current body frame and tracks them locally, while a separate channel tracks the commanded root linear velocity, direction, and orientation; this prevents small positional errors from accumulating and allows a long dancing sequence to stay stable. The filtering step uses the initial policy $\\pi_0$ as an oracle: for each motion sequence $s$ it computes $e(s) = \\alpha E_{\\text{key}}(s) + \\beta E_{\\text{dof}}(s)$ with $\\alpha = 0.1$ and $\\beta = 0.9$, and the paper selects $\\tau^*$ so that $D_{\\tau^*} = \\{s : e(s) \\le \\tau^*\\}$ is the training set for the final generalist, embodying a Feasibility-Diversity Principle: keep upper-body diversity while removing lower-body motions the robot cannot physically track. A teacher-student distillation step then transfers the privileged teacher's behavior to a student that observes only proprioception and reference targets, with history length and online imitation-style rollouts shown to be necessary for the final accuracy.","core_discovery":"The paper's central claim is that the right formulation of whole-body tracking is two decoupled objectives - local body-landmark tracking plus global motion-velocity tracking - and that the main obstacle to expressive control is not policy capacity but the feasibility of the training data. Its automated curation scores every motion clip by the initial teacher's lower-body tracking error $e(s) = \\alpha E_{\\text{key}}(s) + \\beta E_{\\text{dof}}(s)$ with $\\alpha = 0.1$, $\\beta = 0.9$, keeps the clips below a threshold $\\tau^*$ chosen to maximize performance on the full evaluation corpus, and trains the generalist policy on this subset. The reported result is that this generalist, trained and deployed as a single policy, achieves lower keypoint and joint tracking errors than the compared baselines in both simulation and real-world experiments, and that resuming from it to fine-tune on a specialized motion group gives specialists that outperform both the generalist and models trained from scratch on matched compute. The authors frame the trade-off explicitly: filtering slightly increases velocity-tracking error while substantially improving stability and precision, and fine-tuning gains accuracy on target motions at the cost of broad coverage.","pith_inferences":["A testable consequence the paper leaves open: apply the threshold-selection procedure to a held-out motion library and verify that the $\\tau^*$ tuned on the main corpus is still near-optimal; the paper asserts this transfer but does not evaluate it.","Because the filter is defined by the initial policy's errors, better initial policies would likely change which clips are labeled infeasible, so the curation and the policy are coupled rather than the filter being a property of the data alone.","The acknowledged limitation that specialist policies cannot be recombined in one session points to a natural next step: a router or gating network that blends specialists in real time based on the incoming motion class.","If decoupled velocity control rather than filtering is the main source of gain, an ablation that keeps the filter and swaps in global-keypoint tracking should degrade performance more than removing any single reward term; the paper does not run this isolation."],"forward_implications":["A single ExBody2 generalist policy can replace separate controllers for walking, crouching, dancing, and upper-body gestures on the same humanoid platform.","New motion capture data can be pruned automatically by the same error-threshold procedure, removing the need for manual or language-based filtering.","Fine-tuning the generalist on a small, motion-specific dataset yields higher tracking accuracy than training from scratch under matched compute, with the advantage growing as motions become more difficult.","The accepted cost of filtering is a slightly larger root-velocity error, traded for substantially lower keypoint and joint errors and better real-world stability."],"supporting_citations":[{"why":"Supplies the motion-capture corpus used both as the training pool and as the evaluation set for the generalist policy.","marker":"[1]"},{"why":"Direct predecessor and baseline: ExBody2 extends its joystick-command whole-body tracking formulation and compares against it.","marker":"[4]"},{"why":"Baseline whose global-keypoint tracking strategy ExBody2 contrasts with its local keypoint and decoupled velocity control.","marker":"[17]"},{"why":"Motivates the decoupled control design by demonstrating that global keypoint tracking drifts and fails in dynamic scenarios.","marker":"[18]"},{"why":"Together with [29], provides the two-stage teacher-student training template that ExBody2 follows.","marker":"[28]"},{"why":"Establishes the privileged-teacher then student-distillation recipe for sim-to-real transfer that the method adopts.","marker":"[29]"},{"why":"Supplies additional human motion capture data used to guide the imitation training.","marker":"[38]"},{"why":"Provides the parallel physics simulation environment in which teacher and student policies are trained and evaluated.","marker":"[39]"},{"why":"The online imitation-learning procedure used to train the student policy from teacher rollouts.","marker":"[50]"},{"why":"The policy-gradient reinforcement learning algorithm used to train the privileged teacher policy.","marker":"[51]"}],"fun_headline_variants":["Single whole-body policy wins via decoupled velocity and keypoint tracking","ExBody2: one policy for all expressive humanoid motion, from walk to dance","Automated clipping of infeasible motions is the real enabler","Fine-tuning trades generality for precision in humanoid control","Decoupled objectives and data filtering beat specialized trackers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole gain of the generalist policy hinges on the assumption that the first, unfiltered policy's lower-body tracking errors are the right measure of which motions the robot can never physically perform, and that the cutoff chosen on the evaluation corpus works on other motion data without retuning.","fun_headline_variants_meta":{"raw":{"variants":["Single whole-body policy wins via decoupled velocity and keypoint tracking","ExBody2: one policy for all expressive humanoid motion, from walk to dance","Automated clipping of infeasible motions is the real enabler","Fine-tuning trades generality for precision in humanoid control","Decoupled objectives and data filtering beat specialized trackers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000297,"raw_usage":{"total_tokens":1721,"prompt_tokens":944,"completion_tokens":777,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":688}},"tokens_in":560,"tokens_out":777,"duration_ms":7941,"temperature":1.0,"reasoning_tokens":688,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:19:18.549654+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a random half of the motion corpus before training. Select the error cutoff using only the retained half, train the generalist on the filtered retained half, and measure its error on the held-out half against an unfiltered policy. If the filtered policy is not better on the held-out half, the claimed curation gain is overfitting to the threshold-selection set. A second check: rerun the filtering with a much stronger initial policy and see whether the same motion clips are discarded; if the rankings flip substantially, the filter is measuring the weakness of the first policy, not the feasibility of the motions.","supporting_citations":[{"cited_title":"Carnegie-Mellon mocap database","cited_arxiv_id":null,"evidence_quote":"Supplies the motion-capture corpus used both as the training pool and as the evaluation set for the generalist policy."},{"cited_title":"Omnih2o: Universal and dexterous human- to-humanoid whole-body teleoperation and learning,","cited_arxiv_id":null,"evidence_quote":"Baseline whose global-keypoint tracking strategy ExBody2 contrasts with its local keypoint and decoupled velocity control."},{"cited_title":"Troje, Gerard Pons-Moll, and Michael J","cited_arxiv_id":null,"evidence_quote":"Supplies additional human motion capture data used to guide the imitation training."},{"cited_title":"A reduction of imitation learning and structured prediction to no-regret online learning","cited_arxiv_id":null,"evidence_quote":"The online imitation-learning procedure used to train the student policy from teacher rollouts."}],"review_version":1}