{"id":"1ea40097-fa83-479d-9331-2c1878b64469","arxiv_id":"2506.14770","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GMT trains a single unified humanoid policy using adaptive sampling and mixture-of-experts, achieving lower tracking errors than a re-implemented ExBody2 across diverse whole-body motions.","lead":"A new training framework lets one neural-network policy drive a humanoid robot through dozens of different whole-body motions, from walking to kung fu, in simulation and on real hardware. The result brings general-purpose humanoid control closer by replacing separate task policies with a single unified controller.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Data curation in Sec 3.3 may remove exactly the hard motions a stronger policy could learn; without reporting removal thresholds and testing on unfiltered data, the 'general' claim is unverified.","rationale":"The reader identifies the two-stage data curation as the weakest assumption; I agree, and I would sharpen it: the concern is not merely that filtering may remove learnable motions, but that the evaluation protocol does not establish independence between the curation filter and the test set. The paper states that all policies are trained on the filtered dataset and evaluated on AMASS-test and LAFAN1, but it never specifies whether the evaluation split was also filtered. If the same filtering criterion is applied at training and test time, the simulation comparison in Table 2 is an in-distribution comparison on a curated subset, not a test of general motion tracking. This directly threatens the abstract's 'general and scalable' claim. I considered other potential concerns, such as the absence of quantitative real-world metrics and the limited baseline set, but those are evidence-strength issues rather than potential internal-selection biases. The curation issue is more load-bearing because it could make the central claim true only on an easy, filtered subset. The proposed concrete test—retraining on unfiltered data and evaluating on unfiltered test motions—would settle whether the filtering is a benign feasibility filter or a source of overclaiming. Since the reader already reached CONDITIONAL, my analysis does not change the verdict; it reinforces the need for the conditional acceptance to require transparency about the curation step and an unfiltered evaluation.","tokens_in":11335,"tokens_out":5182,"duration_ms":52649,"concrete_test":"Instrument the curation pipeline: report the number and category breakdown of clips removed in stage 2, the completion-rate threshold, and the motion statistics (duration, joint speed, contact changes) of removed versus kept clips. Then retrain the full GMT policy (with MoE and Adaptive Sampling) on the unfiltered stage-1 dataset and evaluate it on an unfiltered AMASS test split that includes the previously discarded motions. If the final policy tracks a nontrivial fraction of the discarded clips, the curation step is overly aggressive and the 'general' claim is overstated. If it fails on the same clips as the preliminary policy, the filtering is justified and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that GMT is a general, scalable motion-tracking framework trained on diverse human motion data. The load-bearing assumption sits in Sec 3.3: after rule-based filtering, the authors train a preliminary policy on the remaining data and then remove motions that this preliminary policy fails to complete. This second-stage curation is described only qualitatively: no completion-rate threshold is given, no fraction of removed clips is reported, and no analysis of what categories or motion complexities are discarded is provided. If the preliminary policy is weaker than the final GMT policy, then the curated 8,925-clip dataset may be biased toward motions that are easy for a weaker policy, and the more diverse or dynamic motions—precisely the ones that stress the 'general' claim—may have been removed before evaluation. The ablation results in Table 2(a) show that MoE and Adaptive Sampling substantially reduce tracking error, so the final policy is indeed more capable than a plain baseline; this raises the real possibility that the curation policy is not representative of the final policy's ability. Moreover, the paper does not state whether the AMASS test set and LAFAN1 evaluation data were themselves filtered by the same pipeline. If both training and evaluation exclude the same hard motions, the reported simulation numbers and the SOTA comparison in Table 2 are measured on a subset from which challenging motions have already been eliminated. This would not invalidate the ablations, but it would make the headline claim of generality unverifiable and possibly overstated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces GMT, a two-stage teacher-student framework for training a single unified whole-body motion-tracking policy for a humanoid robot (Unitree G1). The two main technical contributions are an Adaptive Sampling strategy that reweights training motions by tracking difficulty, and a Motion Mixture-of-Experts (MoE) architecture for the policy network. The policy is trained on a filtered combination of AMASS and LAFAN1 (8,925 clips, 33.12 hours) and evaluated in simulation against a re-implemented ExBody2 baseline, with ablations for each component and for the motion-input representation. Real-world deployment is shown qualitatively on a variety of motor skills, and an additional MuJoCo experiment demonstrates tracking of motion-diffusion-model-generated motions. The central empirical claim is that a single policy can track a broad spectrum of whole-body motions more accurately than the compared baseline, with each component contributing to improved simulation metrics.","tokens_in":11637,"tokens_out":3091,"duration_ms":32819,"significance":"If the claims hold, GMT would be a useful step toward general-purpose whole-body controllers for humanoid robots, and the Adaptive Sampling and MoE design choices are plausible and well motivated by the observed category imbalance in AMASS. The ablation study is systematic and the simulation metrics are standard; the real-world deployment videos are compelling evidence of basic feasibility. The paper also deserves credit for explicitly stating limitations (no contact-rich skills, no challenging terrains) and for evaluating on an external motion-generation pipeline (MDM), which is a good generalization test. However, the significance is currently bounded by two issues: the 'state-of-the-art' claim rests on a single re-implemented baseline, and the dataset-curation procedure is not documented quantitatively, so the breadth of the 'general' claim cannot be fully verified from the manuscript as written.","major_comments":[{"comment":"The second stage of dataset curation is described only qualitatively: a preliminary policy is trained on the rule-filtered data and 'based on the completion rates achieved by this policy, we further filter out failed motions.' No completion-rate threshold, no fraction of removed clips, and no analysis of which motion categories or complexity levels were discarded are reported. Since the final policy is more capable than the preliminary policy (as the ablations in Table 2(a) suggest), this filtering may preferentially remove exactly the hard motions that a stronger policy could learn, and the 'general' claim is therefore unverified. Please report the threshold, the removal statistics, and a breakdown of removed motions by category and difficulty, and consider evaluating on an unfiltered or independently filtered test set.","section":"Section 3.3"},{"comment":"The abstract and conclusion claim 'state-of-the-art performance,' but the only quantitative baseline is a re-implemented ExBody2. Other recent whole-body controllers mentioned in the related work (HumanPlus, OmniH2O, VMP, ASAP) are not compared numerically, and the real-world evidence is qualitative. The comparison would support a claim of 'improvement over the re-implemented ExBody2 baseline' but not a general state-of-the-art claim. Please either add quantitative comparisons to at least one or two additional recent methods or revise the claim to match the evidence.","section":"Section 4.2 and Table 2"},{"comment":"The simulation evaluation is reported on 'AMASS test set' and 'LAFAN1,' but the text does not state whether these evaluation sets were filtered by the same two-stage curation pipeline described in Section 3.3. If the evaluation sets are subsets produced by the same preliminary-policy filter, the reported numbers may not reflect performance on the full diversity of human motion, and the comparison with the baseline would be measured on a potentially biased subset. The manuscript should clarify the relationship between the training filter and the evaluation sets, and ideally report results on an unfiltered test set as well.","section":"Section 4.1 and Table 2"}],"minor_comments":[{"comment":"The text says all baseline comparisons and ablations use privileged policies, but Table 2 also lists a 'Student Policy' row; please clarify whether the student row is part of the main comparison and how it relates to the stated evaluation protocol.","section":"Table 2 and Section 4.1"},{"comment":"The variable E_max in the sampling level definition is not clearly defined; please specify whether it is a per-motion maximum key-body error, a threshold, or a normalization constant, and give its value or estimation procedure.","section":"Equation (1)"},{"comment":"The 'top percentile tracking errors' are not defined; please state how the top percentiles are computed (e.g., worst episodes, worst timesteps, or worst motion clips) and whether the errors are aggregated over all motions or per category.","section":"Figure 5"},{"comment":"The real-world deployment section provides no quantitative metrics; adding even basic measured tracking errors or success rates for a few representative motions would considerably strengthen the real-world claim.","section":"Section 4.4"},{"comment":"The motion-input ablation 'GMT-L2' is said to degrade because the immediate next frame is omitted, but the comparison also varies the total input window; please make explicit whether the window length is the only difference between GMT-L2 and GMT-L2-M, or whether the encoding architecture also differs.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems contribution, but the 'general and state-of-the-art' positioning is stronger than the current evidence supports. The two key fixes are (1) documenting the curation filter and its effect on the evaluation sets, and (2) either adding a second baseline or softening the SOTA claim. Both are within the scope of a revision and do not require invalidating the core result."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GMT is a solid, well-engineered systems paper that advances humanoid whole-body motion tracking, but the 'state-of-the-art' claim outruns the evidence. The most valuable piece is the motion-input design: feeding the policy both a two-second window of future frames (compressed by a conv encoder) and the immediate next frame gives a clear, consistent tracking improvement across both test sets (Table 2b). Adaptive Sampling and MoE are familiar ideas, but the paper applies them sensibly and the ablations show each component helps, with the full system beating a re-implemented ExBody2 by a decent margin on all four metrics.\n\nWhat the paper does well: the ablations are clean and component-wise contributions are consistent; the limitations section is honest (no contact-rich skills, no terrain); the real-world videos on a Unitree G1 demonstrate a wide range of skills. The teacher-student pipeline and domain randomization are standard but competently executed.\n\nThe soft spots, in order of importance:\n\n1. The SOTA claim is overreached. The only quantitative baseline is a re-implemented ExBody2. There is no quantitative comparison to HumanPlus, OmniH2O, or VMP on the same metrics, and real-world results are qualitative (videos). One baseline does not support 'state-of-the-art.'\n\n2. The two-stage data curation in Sec 3.3 is underreported. The paper says a preliminary policy's completion rates are used to filter out failed motions, but no threshold, no fraction removed, and no category analysis are given. The stress-test concern is legitimate: if the curation policy is weaker than the final policy, the filter could systematically remove exactly the hard motions that test the 'general' claim. The evaluation sets (AMASS-test, LAFAN1) appear to be unfiltered, which mitigates the concern, but the authors should state this explicitly and report the curation statistics.\n\n3. Minor: 'general and scalable' is a big claim for one robot and one dataset combination.\n\nThe central empirical story holds together: the proposed components improve tracking in simulation, and the qualitative real-world results are convincing. The missing curation details and the sparse baseline comparison are fixable in revision. This paper deserves a serious referee and, with those concerns addressed, likely acceptance.\n\nRecommendation: send to peer review. I'd ask for curation statistics, a statement about test-set filtering, and either an additional baseline or a quantitative real-world metric before accepting the SOTA label.","headline":"A solid, incremental contribution to humanoid control with a genuinely useful motion-input design, but the SOTA claim needs more baselines and curation details.","tokens_in":12149,"tokens_out":3862,"would_cite":true,"duration_ms":38591,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GMT claims a single policy can track diverse whole-body human motions on a real humanoid robot, with lower simulation tracking error than a re-implemented ExBody2 baseline.","keywords":["Humanoid","Locomotion","Learning-based Control","Motion Imitation","Whole-body Control","Mixture-of-Experts","Adaptive Sampling","Teacher-student Training"],"falsifier":"Retrieve the clips discarded by the second-stage filter, add them back into the training set, and re-train GMT; if a substantial fraction of those clips still cannot be tracked by the final policy, the curated 'general' claim is not supported.","tokens_in":11187,"feed_emoji":"🤖","tokens_out":8518,"duration_ms":87596,"temperature":0.7,"pith_summary":"The paper claims that a single policy can be trained to make a real humanoid robot track a broad spectrum of whole-body human motions, from walking and stretching to kicks, dancing, and kung-fu. It argues that two obstacles stand in the way: large motion datasets are skewed toward easy, common movements, and a single neural network does not have enough expressive power to cover a wide motion manifold. GMT addresses the first with Adaptive Sampling, which re-weights training clips by how well the current policy completes them, and the second with a Mixture-of-Experts policy head that lets different experts specialize in different motion regions. In simulation, the resulting policy reports lower tracking errors than a re-implemented ExBody2 baseline, and the same policy is deployed on a Unitree G1 humanoid to reproduce a range of dynamic skills. If correct, this moves toward a general whole-body controller that a high-level planner could sequence into tasks.","feed_headline":"One policy tracks dancing, kicking, and kung-fu on a real robot","feed_subtitle":"Adaptive sampling plus mixture-of-experts lets one controller track dancing, kicking, and kung-fu.","key_machinery":"The load-bearing machinery is a two-stage teacher-student training loop whose teacher combines a Motion Mixture-of-Experts head with an Adaptive Sampling curriculum. The MoE is a set of expert networks plus a gating network that reads the same observations and outputs a probability distribution; the final action is the probability-weighted combination of the experts' sampled actions, giving the model a way to specialize across the motion manifold. Adaptive Sampling sets each clip's sampling probability from its completion count and current tracking error, so easy motions are down-weighted and hard motions dominate training. Around this core, GMT also relies on a convolutional encoder that compresses about two seconds of future motion frames into a latent vector fed to the policy alongside the immediate next frame. The whole system is trained first as a privileged PPO policy and then distilled into a deployable student policy by DAgger.","core_discovery":"On its own terms, GMT's discovery is that joint curation of the data distribution and the model architecture unlocks a single general motion-tracking policy for a physical humanoid. The training set is first filtered by rules and then by a preliminary policy's completion rates, leaving 8,925 clips, or 33.12 hours, from AMASS and LAFAN1. The teacher policy is trained with PPO under domain randomization, using a Mixture-of-Experts network whose gating network weights the outputs of specialized expert policies, while Adaptive Sampling raises the sampling probability of motions the current policy still fails. Motion targets are represented with local key-body positions aligned to the robot's heading, and the policy sees both the immediate next frame and roughly two seconds of future frames compressed by a convolutional encoder. The paper reports that this single policy outperforms its re-implemented ExBody2 baseline on AMASS-test and LAFAN1 tracking metrics and reproduces the motions on a real Unitree G1, including dynamic skills.","pith_inferences":["A testable consequence the paper does not pursue: if the single-policy claim holds, tracking error on a held-out set of skills should keep dropping as the curated dataset grows; a plateau would point to the MoE's capacity or the sampling schedule as the bottleneck.","An extension would apply the same two mechanisms to contact-rich skills such as getting up or rolling, which the paper lists as unsupported; this would probe whether the limitation is the data filter or the architecture.","A practical benchmark would report per-category completion rates on a fixed motion taxonomy, so future general controllers can be compared by coverage rather than aggregate error."],"forward_implications":["A single deployed policy can replace a bank of specialist policies for walking, kicking, dancing, and other whole-body skills on a 23-DoF humanoid.","Because the policy tracks text-generated motions from a motion diffusion model in simulation, it can act as the low-level executor for high-level planners that generate or select motions on the fly.","Most of the performance gain from Adaptive Sampling and the MoE appears on the hardest motion categories, so the framework's advantage grows with dataset diversity.","Using local key-body positions and a multi-frame future-motion window is a transferable design for other motion-imitation pipelines."],"supporting_citations":[{"why":"Supplies the large AMASS mocap dataset, the main training source and the source of category imbalance the paper addresses.","marker":"[6]"},{"why":"ExBody2 is the closest prior whole-body tracking work and the baseline GMT re-implements and reports lower tracking errors against.","marker":"[7]"},{"why":"LAFAN1 is the second mocap dataset used for training and for evaluation.","marker":"[40]"},{"why":"PPO is the reinforcement learning algorithm used to train the privileged teacher policy.","marker":"[38]"},{"why":"DAgger is the imitation algorithm used to distill the teacher policy into the deployable student policy.","marker":"[39]"},{"why":"Provides the two-stage data-curation approach and the physics-based character-control setting that GMT adapts for real robots.","marker":"[5]"},{"why":"The motion diffusion model used to generate test motions for the sim-to-sim application experiment.","marker":"[46]"},{"why":"Supplies the domain-randomization method used for sim-to-real transfer.","marker":"[42]"}],"fun_headline_variants":["GMT: one policy, many motions, real humanoid","Adaptive sampling and MoE make a single policy track diverse motions","One controller tracks dancing, kicking, and kung-fu on a real humanoid","GMT: adaptive sampling plus MoE gives one policy for all motions","Unified humanoid policy tracks diverse motions in the real world"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim of generality rests on the assumption that the second-stage data filter, which drops motions a preliminary policy fails to complete, is not removing motions that a better final policy could have learned and that would change the reported performance.","fun_headline_variants_meta":{"raw":{"variants":["GMT: one policy, many motions, real humanoid","Adaptive sampling and MoE make a single policy track diverse motions","One controller tracks dancing, kicking, and kung-fu on a real humanoid","GMT: adaptive sampling plus MoE gives one policy for all motions","Unified humanoid policy tracks diverse motions in the real world"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000445,"raw_usage":{"total_tokens":2237,"prompt_tokens":920,"completion_tokens":1317,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":1225}},"tokens_in":536,"tokens_out":1317,"duration_ms":15416,"temperature":1.0,"reasoning_tokens":1225,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:09:10.021723+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrieve the clips discarded by the second-stage filter, add them back into the training set, and re-train GMT; if a substantial fraction of those clips still cannot be tracked by the final policy, the curated 'general' claim is not supported.","supporting_citations":[{"cited_title":"Mahmood, N","cited_arxiv_id":null,"evidence_quote":"Supplies the large AMASS mocap dataset, the main training source and the source of category imbalance the paper addresses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LAFAN1 is the second mocap dataset used for training and for evaluation."},{"cited_title":"Tessler, Y","cited_arxiv_id":null,"evidence_quote":"Provides the two-stage data-curation approach and the physics-based character-control setting that GMT adapts for real robots."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the domain-randomization method used for sim-to-real transfer."}],"review_version":1}