{"id":"7fb99963-f54f-4d4d-a55e-1ce98b0fbb1b","arxiv_id":"2608.10162","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Masked autoregressive diffusion over disentangled continuous latent streams generates, completes, infills, and terminates hand-object interactions from text on ARCTIC and GRAB.","lead":"MAD-HOI is a text-to-motion model that generates 3D hand and object interaction clips, and it can extend, fill in, or stop on its own instead of needing a fixed length. It combines continuous motion detail with language-model-style sequence flexibility, which could simplify animation pipelines and seed robot manipulation policies.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The distributional SOTA claim rests on an unpublished author-trained evaluator and retrained baselines; until the evaluator is independently validated and baseline training is matched to original pipelines, the headline comparison is not conclusive.","rationale":"The reader's weakest assumption matches my own: the empirical superiority claim is conditional on an unpublished evaluator and on the authors' retraining choices. I reviewed the method for internal inconsistencies. The masked autoregressive flow-matching construction is coherent, and the capability set (completion, infilling, composite generation, EOM) follows plausibly from masking, EOM tokens, and composite training data; I did not find a fatal technical flaw. The strongest internal inconsistency is the abstract's physical-plausibility claim versus Table 3 on GRAB, where MAD-HOI trails Text2HOI and DiffH2O on Contact Percent, Contact Ratio, and Physical Plausibility. That overstatement matters but is secondary to the evaluator and baseline-retraining issue, which directly underpins the headline 'outperforms open-sourced baselines' claim. A leave-one-object-out evaluator check would test whether the ranking is an artifact of the author-trained evaluator. The paper's novelty and breadth are real, and the empirical claims could be established with code release and external validation, so CONDITIONAL remains the appropriate verdict.","tokens_in":19096,"tokens_out":13287,"duration_ms":131929,"concrete_test":"Train the authors' evaluator architecture under leave-one-object-out cross-validation on ARCTIC and GRAB (hold out each test object class during evaluator training), then recompute Table 2 FID, KID, and Top-3 for MAD-HOI and the baselines; also release DiffH2O, LatentHOI, and OpenHOI retraining configs and confirm they reach their originally reported performance on their native pipeline. If MAD-HOI no longer ranks first in FID and KID under the held-out evaluator, the distributional SOTA claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 'Evaluation Protocol' is the load-bearing point. All FID, KID, Top-3, and Matching Score numbers in Table 2 are computed in the embedding space of a text-motion evaluator that the authors train from scratch on ARCTIC and GRAB, encoding object motion and articulation jointly with hand motion. This departs from the standard HumanML3D protocol used by the cited baselines, and the baselines DiffH2O, LatentHOI, and OpenHOI are retrained by the authors under their own preprocessing (Sec. 4.3), with no released checkpoints, training curves, or hyperparameter settings. If the evaluator's motion autoencoder is biased toward the continuous global-coordinate representation MAD-HOI outputs, or if the retrained baselines are undertuned relative to their original pipelines, then the large FID/KID margins (ARCTIC FID 0.071 vs 0.495 for OpenHOI) are not evidence of general superiority. The claim is also internally overstated: Table 3 on GRAB shows MAD-HOI trailing Text2HOI and DiffH2O on Contact Percent, Contact Ratio, and Physical Plausibility, so the abstract's 'more physically plausible' is not supported by the paper's own data. The strongest fix is to make the evaluator and baseline training reproducible and to check ranking stability under an independent evaluator.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MAD-HOI, a text-conditioned generative model of articulated hand-object interactions. It consists of a cascaded VAE that encodes object, left-hand, and right-hand motion into disentangled continuous latent streams, followed by a masked autoregressive transformer whose outputs condition a flow-matching head. The authors argue that this single training objective yields atomic and composite generation, conditioned completion and infilling, handedness control, and learned EOM termination without VQ discretization. Experiments on ARCTIC and GRAB report retrieval, distributional, geometric, and plausibility metrics against Text2HOI, DiffH2O, LatentHOI, and OpenHOI, together with ablations, a small user study, and a qualitative robot-retargeting application. The main quantitative claim is that MAD-HOI outperforms open-sourced baselines on distributional fidelity while remaining competitive on physical plausibility.","tokens_in":19306,"tokens_out":7194,"duration_ms":64132,"significance":"If substantiated, MAD-HOI would be a practically useful method because it combines autoregressive flexibility with a continuous motion representation, avoiding quantization losses that are particularly damaging for contact-sensitive hand motion. The paper gives credit-worthy ablations: Table 2 shows that removing the modality embeddings collapses generation (FID from 0.071 to 56.7 on ARCTIC), and removing EOM prediction degrades FID substantially, supporting the two key design choices. The VAE reconstruction comparison in Sec. 3 also favors continuous latents over a VQ-VAE in contact preservation. However, the headline comparison currently rests on an author-trained evaluator and retrained baselines, so the quantitative SOTA claim, while plausible, is not yet demonstrated to be protocol-independent.","major_comments":[{"comment":"The distributional metrics (R@3, FID, KID, Diversity, Matching Score) are computed in the embedding space of a text-motion evaluator that the authors train from scratch and that encodes object motion and articulation jointly with hand motion. This departs from the standard HumanML3D protocol under which the cited baselines were originally evaluated, and the manuscript does not validate the new evaluator (e.g., retrieval accuracy on held-out data, agreement with the old evaluator, or ranking stability across evaluators). Since the evaluator is trained on the same datasets and preprocessing as the generative models, the large FID margins reported in Table 2 (ARCTIC FID 0.071 for MAD-HOI versus 0.495 for OpenHOI) could be an artifact of evaluator bias rather than of overall generation quality. The authors should validate the evaluator and re-express the headline ranking under at least one independent evaluation protocol before claiming state-of-the-art results.","section":"§5, Evaluation Protocol; Table 2"},{"comment":"DiffH2O, LatentHOI, and OpenHOI are retrained by the authors under their own preprocessing, while Text2HOI uses its released checkpoints, creating a mixed comparison. The manuscript gives no training curves, hyperparameter budgets, convergence checks, or released checkpoints for the retrained baselines, so a reader cannot determine whether the reported margin reflects architectural superiority or unequal tuning. Because the paper's central claim is empirical superiority over these open-sourced methods, the authors should provide the retraining setup in full, ideally compare against the original checkpoints under both the original and the new protocols, and release all checkpoints for verification.","section":"§4.3 (Baselines) and Table 2"},{"comment":"The abstract states that MAD-HOI generates \"more diverse and physically plausible interactions\", but the paper's own Table 3 shows that on GRAB, MAD-HOI trails Text2HOI and DiffH2O on Contact Percent, Contact Ratio, and Physical Plausibility, and trails DiffH2O on Penetration Percent. The paper is strongest on ARCTIC plausibility metrics and on distributional fidelity, but the abstract's wording overclaims the physical-plausibility result. The authors should qualify the conclusion to say that MAD-HOI is competitive, not superior, on physical plausibility, and specify the datasets and metrics on which it leads.","section":"Abstract; §5.1, Table 3"},{"comment":"The paper lists \"reliable termination\" as a headline capability, yet Table 5 shows that on GRAB MAD-HOI's EOM prediction has a Median AE of 14.4 frames and a Within-8 accuracy of only 38.3%, which is not self-evidently \"reliable\". The interval-based metrics are more favorable (Within Interval 70.5% on GRAB), but the manuscript does not report the widths of the [q10,q90] intervals or the per-condition standard deviations, so these numbers cannot be assessed. The authors should report interval widths and per-condition variability, and they should temper the \"reliable termination\" claim or state the accuracy levels at which termination should be considered reliable for downstream use.","section":"§5.1, Table 5 (EOM prediction)"}],"minor_comments":[{"comment":"Equation (1) lists the VAE loss with weights α, β, γ, δ, and λ, but their values appear only in Sec. B.4 of the supplementary; the main text should give the values or an explicit pointer at first use.","section":"§4.1, Eq. (1)"},{"comment":"The sentence describing jerk profiles contains an unbalanced parenthesis and ends with \"supplementary material.\" without closing the parenthetical; it should be reworded to complete the sentence correctly.","section":"§3"},{"comment":"Table 4 is captioned \"Task Tokens (#): 1 2 3 4 5\" but the text explains that these columns correspond to the number of conditioning latents; the caption should state this explicitly so that the monotonic decrease in ADE and FDE is interpretable.","section":"Table 4"},{"comment":"The sentence \"The gains are most signifantlyseeninFIDforARCTICwhereourmethodOpenHOIisthesecondbestperformingmethod...\" contains a typo and an unclear clause; it should be rewritten, for example as \"FID on ARCTIC shows the largest gain, with MAD-HOI at 0.071 and OpenHOI as the second-best method at 0.495.\"","section":"§5.1"},{"comment":"The robot-retargeting application is supported only by one qualitative figure; the authors should state explicitly that this is a qualitative proof of concept and is not yet a quantitative robotics benchmark.","section":"§6"},{"comment":"The paper would benefit from a reproducibility statement: the authors do not state whether MAD-HOI code, trained checkpoints, or the evaluation pipeline will be released, which is particularly important because the comparison protocol is nonstandard.","section":"General"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is better than the usual text-to-HOI submission. The core idea is sound: keep motion in continuous latents, use masked autoregressive flow matching, and you get atomic generation, composite sequences, completion, infilling, and learned termination from one objective. That capability set is real and, as far as I can tell from Tab. 1, not offered by any cited open-source method. The ablations are meaningful—modality embeddings are clearly load-bearing, and EOM prediction helps. The VAE-vs-VQVAE tokenizer comparison is also good evidence for continuous latents, not just hand-waving. This deserves a serious referee.\n\nThe soft spots are real but not disqualifying. The biggest is the evaluation protocol. All FID/KID/Top-3 numbers come from an evaluator trained from scratch that encodes object motion together with hands, and several baselines are retrained by the authors. That is a legitimate choice—the standard HumanML3D evaluator does ignore object motion—but until the evaluator is released or independently validated, the large margins (ARCTIC FID 0.071 vs 0.495) cannot be treated as established superiority. I would not call this a fatal flaw; it is a reproducibility condition. The paper should also report error bars or at least state variance explicitly, since the supplementary says standard deviations were negligible and omitted—that is a claim a referee will want to see substantiated.\n\nThe abstract overstates physical plausibility. On ARCTIC the geometric metrics are genuinely strong, but on GRAB the paper's own Tab. 3 shows MAD-HOI trailing DiffH2O and Text2HOI on contact percentage, contact ratio, and physical plausibility. The conclusion is more careful, and the user study is helpful, but the abstract should match the data.\n\nOne more thing: composite generation is demonstrated on a training set that the authors augmented with composite sequences. That is not circular in a harmful way—it is an honest way to teach the model chained motions—but the paper should be explicit that the composite capability is enabled by this augmented training data, not emergent from the architecture alone. The EOM comparison against Text2HOI's cVAE is fair; MAD-HOI is worse on raw MAE but better on distribution-aware interval metrics, which is the right way to think about a single prompt having multiple valid durations.\n\nOverall: solid paper, important within its subfield, with a real capability contribution. The central claims hold up in spirit, but the quantitative SOTA claims need the evaluator and baseline training to be public before they are fully convincing. I would send it to reviewers, with the expectation that the authors will be asked to release code and evaluator weights, report error bars, and soften the abstract. This is a revise-and-resubmit trajectory, not a reject.","headline":"A genuinely capable HOI generator with a new capability set, whose headline numbers rest on an author-trained evaluator and retrained baselines—worth publishing after the evaluation protocol is made reproducible and the plausibility claims are toned down.","tokens_in":19947,"tokens_out":808,"would_cite":true,"duration_ms":9993,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One masked-autoregressive flow-matching objective, trained once on continuous hand and object latents, gives text-driven HOI generation atomic and composite generation, completion, infilling, and automatic termination.","keywords":["text-to-motion generation","hand-object interaction","masked autoregressive diffusion","flow matching","variational autoencoder","articulated objects","motion completion and infilling","end-of-motion prediction"],"falsifier":"Retrain every open-source baseline with its own original pipeline and score all methods with both the standard hand-only text-motion protocol and the paper's object-aware evaluator; if MAD-HOI does not stay ahead on FID, R@3, and matching score under both protocols, the claimed advantage is protocol-dependent. A second check: measure downstream grasp success of policies trained on VAE versus VQ-VAE reconstructions to see whether the reconstruction-level contact gains actually transfer.","tokens_in":18792,"feed_emoji":"🖐️","tokens_out":12642,"duration_ms":104687,"temperature":0.7,"pith_summary":"The paper tries to establish that one text-to-motion model can cover the full range of practical hand-object interaction generation without losing contact-sensitive detail. It introduces MAD-HOI, which encodes object, left-hand, and right-hand motion as separate continuous latent streams and trains a masked autoregressive transformer whose context features condition a flow-matching head under a single masked flow-matching objective. The paper argues that this single objective yields atomic and composite generation, conditioned completion and infilling, handedness control, and automatic end-of-motion prediction. On ARCTIC and GRAB it reports better distributional fidelity and competitive physical plausibility than the open-source baselines it compares against. If the claim holds, variable-length bimanual text-driven interactions can be produced without separate length predictors, post-hoc masking, or task-specific heads.","feed_headline":"One objective drives hand-object motion, completion, and stopping","feed_subtitle":"MAD-HOI's single masked flow-matching model also predicts when motion ends, outperforming open baselines on fidelity.","key_machinery":"The central object is the MAD-HOI generator: a cascaded variational autoencoder that encodes object motion, left-hand motion, and right-hand motion into separate 512-dimensional continuous latent streams; a masked autoregressive transformer with bidirectional attention that sees every unmasked latent token, plus text and positional embeddings, and outputs conditioning signals for masked positions; and a flow-matching head that generates the masked latents by regressing a velocity field along linear interpolants. Learnable End-of-Motion tokens are appended to the interleaved latent sequence, and the VAE is trained to reconstruct static poses as 'idle latents', so the transformer learns to associate the tail of an interaction with termination and to keep an unused hand still. The single objective — velocity prediction at masked positions under a cosine masking schedule — drives all of these capabilities.","core_discovery":"The paper's central claim is that a single training objective — masked conditional flow matching over a VAE latent space with object, left-hand, and right-hand streams kept separate — is sufficient for a text-to-HOI model to generate atomic and composite interactions, complete and infill motions, respect handedness, and predict its own end-of-motion. The authors support this with experiments on ARCTIC and GRAB where MAD-HOI reports the best retrieval accuracy, FID, KID, and matching score among compared open-source baselines and remains competitive on physical-plausibility metrics. They also argue that continuous latents preserve contact-critical detail better than discrete VQ codes: in their reconstruction comparison the VAE penetrates in only 2.1% of frames versus 24.7% for a matched VQ-VAE, with contact in 92.2% versus 77.9% of frames. Ablations show that the learnable modality tokens are needed for the model to converge and that EOM prediction improves rather than burdens generation quality.","pith_inferences":["Earlier hand-only evaluations may have missed differences that only appear when object motion and articulation are encoded; this is an inference, not the paper's claim.","The masked-autoregressive-flow recipe could transfer to other multi-stream articulated motions, such as two-person hand-object coordination, where stream separation and learned termination matter.","A natural stress test is to compare closed-loop task success of manipulation policies trained on MAD-HOI-generated trajectories against policies trained on baseline-generated data, since reconstruction-level contact metrics may not predict downstream performance.","The EOM mechanism could be exposed as a controllable prior, letting users bias generation toward shorter or longer interactions instead of accepting the learned length distribution."],"forward_implications":["A single trained model can generate atomic and composite interactions from chained prompts up to its context window, removing the need for a separate composition module.","Completion and infilling require no task-specific heads: masking the front or the middle of the latent sequence turns the same model into a forecaster or an infiller.","Handedness becomes a learned property of the output, with the unused hand predicted as an idle stream rather than masked after generation.","End-of-motion prediction is learned as part of the same objective, so inference needs no user-supplied length or external length regressor.","Because the latent representation stays continuous, reconstructed contact and penetration are closer to ground truth, which the paper argues matters for downstream robotics priors."],"supporting_citations":[{"why":"Supplies the continuous autoregressive diffusion formulation that MAD-HOI adapts from image generation to motion latents.","marker":"(Li et al. 2024)"},{"why":"Provides the bidirectional masked autoregressive diffusion design for continuous motion generation that MAD-HOI's transformer and diffusion head build on.","marker":"(Meng et al. 2025)"},{"why":"HOIGPT is the discrete VQ language-model baseline and the comparison point for tokenizer choice and sequence-level capabilities.","marker":"(Huang et al. 2025)"},{"why":"Text2HOI supplies the data preprocessing protocol, a diffusion baseline, and the cVAE length regressor used for EOM comparison.","marker":"(Cha et al. 2024)"},{"why":"DiffH2O is a two-stage diffusion baseline that the authors retrain under their preprocessing for comparison.","marker":"(Christen et al. 2024)"},{"why":"LatentHOI is a diffusion baseline and the source of the physical-plausibility metric suite used in evaluation.","marker":"(Li et al. 2025a)"},{"why":"OpenHOI is the composite-motion baseline whose post-hoc composition module MAD-HOI claims to replace.","marker":"(Zhang et al. 2025c)"},{"why":"Provides the ARCTIC articulated-object dataset used for training and evaluation.","marker":"(Fan et al. 2023)"},{"why":"Provides the GRAB grasping dataset used for training and evaluation.","marker":"(Taheri et al. 2020)"},{"why":"Defines the feature-based text-motion evaluation protocol that the paper adapts to jointly encode object and hand motion.","marker":"(Guo et al. 2022a)"}],"fun_headline_variants":["MAD-HOI: one objective for text-driven hand-object motion, completion, and EOM","Single masked flow-matching model predicts end-of-motion for hand-object interactions","Continuous latent hand-object motion beats discrete VQ codes in contact fidelity","MAD-HOI: generate, complete, infill, and predict end-of-motion from text","One training objective unifies generation, completion, infilling, and EOM for HOI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative comparisons assume the authors' jointly-trained evaluator and their re-trained baselines are fair, representative measurements; if that evaluator is biased toward MAD-HOI's latent structure or the retrained baselines are under-tuned, the reported margins would not show general superiority over the standard protocol.","fun_headline_variants_meta":{"raw":{"variants":["MAD-HOI: one objective for text-driven hand-object motion, completion, and EOM","Single masked flow-matching model predicts end-of-motion for hand-object interactions","Continuous latent hand-object motion beats discrete VQ codes in contact fidelity","MAD-HOI: generate, complete, infill, and predict end-of-motion from text","One training objective unifies generation, completion, infilling, and EOM for HOI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000742,"raw_usage":{"total_tokens":3341,"prompt_tokens":1006,"completion_tokens":2335,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":2224}},"tokens_in":622,"tokens_out":2335,"duration_ms":13457,"temperature":1.0,"reasoning_tokens":2224,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:11:30.639484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain every open-source baseline with its own original pipeline and score all methods with both the standard hand-only text-motion protocol and the paper's object-aware evaluator; if MAD-HOI does not stay ahead on FID, R@3, and matching score under both protocols, the claimed advantage is protocol-dependent. A second check: measure downstream grasp success of policies trained on VAE versus VQ-VAE reconstructions to see whether the reconstruction-level contact gains actually transfer.","supporting_citations":[],"review_version":1}