{"id":"bec13fd3-63fc-4848-a904-f5f5a566cc8d","arxiv_id":"2412.16670","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"This paper introduces a latent diffusion transformer that represents two-person interactive motions as one unified latent token sequence, improving text-to-motion generation quality and speed on InterHuman.","lead":"An AI system that creates two-person animations, like one person bowing and the other accepting the apology, from a single sentence, by modeling both motions together in one compressed code. It reports higher quality and about four times faster inference than earlier two-branch models on the InterHuman benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline quantitative claim is undercut by an implausible MM Dist: generated 1.862 vs real 3.755 on Table I, suggesting the VAE reconstruction may inflate text-motion similarity, and no reconstruction-only retrieval metrics are reported to isolate the diffusion model.","rationale":"The reader's verdict is CONDITIONAL because the unified latent space is not isolated. I agree that the missing two-branch VAE comparison is a limitation, but I see a sharper, more load-bearing problem in the reported evaluation itself. The MM Dist value of 1.862 for generated motions, against 3.755 for real motions, is internally inconsistent with the meaning of the metric: a generator cannot be, on average, twice as aligned with captions as the real motions that were paired with those captions, unless the metric is being gamed by smoothing or by a distribution shift in the evaluated embeddings. This is not a matter of consensus; it is a quantitative anomaly that the paper never explains. The same pattern is reflected in MModality being far below real and below InterGen, and the paper's only discussion of MModality is a post hoc reinterpretation rather than a validation. A reconstruction-only pass would cleanly separate the VAE's contribution from the diffusion model's contribution and would also reveal whether compression destroys interaction cues. If the artifact is confirmed, the headline 'superior in text-following' and '4x faster while maintaining high quality' would need to be substantially revised; if the reconstruction-only metrics are close to the real row, the concern is resolved and the current Tables stand. I therefore keep the reader's CONDITIONAL verdict but on a different, more specific basis.","tokens_in":8167,"tokens_out":7747,"duration_ms":67138,"concrete_test":"Run a reconstruction-only evaluation on the InterHuman test set: take each ground-truth motion, encode it with the frozen InterVAE encoder, decode it with the frozen decoder (no diffusion, no text conditioning), and compute R-Precision, MM Dist, MModality, and FID on the reconstructed motions. Compare these numbers to Table I and to the Real row. If the reconstruction alone reaches MM Dist near 1.862 (versus 3.755 for real) or R-Precision near 0.427, then the reported generation gains are dominated by VAE smoothing and the central claim is not supported by the current evidence. In the same pass, measure contact distance and relative joint offsets between reconstructed partners to check interaction preservation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim of superior text-conditioned generation rests on Table I. InterLDM's MM Dist is 1.862±0.007, less than half of the real-data value 3.755±0.008, and its MModality is 1.032±0.089 versus 7.948±0.064 for real motions and 2.141±0.063 for InterGen. MM Dist is the mean L2 distance between text and motion embeddings in a pretrained evaluator space; it is not plausible that a generative model trained with text conditioning produces motions that are twice as close to their captions than the real motions that were paired with those captions. The most natural explanation is that the InterVAE decoder acts as a low-pass filter, moving reconstructed motions toward a text-conditional centroid; this would simultaneously lower MM Dist, inflate top-1 R-Precision, and reduce MModality. The paper does not report retrieval metrics on the VAE reconstruction alone, so the diffusion model's contribution to these gains is not isolated. Section II-A motivates the unified latent as preserving 'full information' but provides no two-branch VAE comparison and no interaction-specific metrics (contact, relative offset, role assignment), so the compression is not directly validated. The MModality paragraph in Section III-B is an explicit post hoc reinterpretation ('if the generated motions are not relevant to the text condition, the higher MModality means the worse instruction-following ability'), which does not resolve the per-text diversity deficit. Without a reconstruction-only evaluation, the Table I gains could be an artifact of the VAE rather than evidence that unified latent diffusion outperforms two-branch methods.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified latent-space framework for two-person interactive motion generation from text. An interaction variational autoencoder (InterVAE) compresses two-person motion sequences into a single latent token sequence, and a diffusion transformer (InterLDM) generates latents conditioned on text, followed by decoding to full motions. Experiments on the InterHuman benchmark report improvements over prior two-branch methods in R-Precision, FID, MM Dist, and Diversity, as well as roughly 4x faster inference. The central claim is that treating two-person motions as one integrated data point in a unified latent space preserves interaction information and improves instruction-following, especially for asymmetric motions.","tokens_in":8471,"tokens_out":2478,"duration_ms":24015,"significance":"If the central claim is sustained, the paper makes a useful contribution: it is, to my knowledge, the first to apply a single latent VAE to multi-person interaction data, and the reported inference speedup over InterGen is practically relevant. The paper also provides quantitative comparisons on the public InterHuman benchmark with confidence intervals, and the qualitative examples in Figure 2 illustrate a plausible advantage for asymmetric interactions. However, the evidence for the central causal claim that the unified latent space is responsible for the gains is currently incomplete, and one of the headline metrics is implausible as reported. The contribution would be strengthened by isolating the VAE's contribution and by validating interaction-specific information preservation.","major_comments":[{"comment":"The reported MM Dist for InterLDM is 1.862±0.007, which is less than half of the real-data value 3.755±0.008. This is not plausible for a text-conditioned generative model: one would not expect generated motions to be twice as close to their captions in a pretrained evaluator embedding space than the real motions paired with those captions. The most likely explanation is that the InterVAE decoder acts as a low-pass filter or moves reconstructions toward a text-conditional centroid, which would simultaneously lower MM Dist, inflate top-1 R-Precision, and reduce MModality. Because the paper reports no retrieval or MM Dist metrics on VAE reconstructions alone, the diffusion model's contribution to these gains is not isolated. I request reconstruction-only R-Precision and MM Dist, and if the VAE alone already achieves near-interpretable text-motion similarity, the claims about instruction-following need substantial revision.","section":"Table I / Section III-B"},{"comment":"The ablation only varies the latent token length f (18, 24, 30, 36), so it tests compression rate versus reconstruction/generation quality, but it does not test the central design choice of a unified latent space against the alternative of two separate per-person branches. Without a matched comparison to a two-branch VAE or a two-branch diffusion model under similar parameter count and compute, the paper's central claim that 'two-in-one' unified modeling drives the improvement is not directly supported. In addition, Section II-A motivates the unified latent as preserving 'full information' of individual motions and inter-human interaction, yet no interaction-specific metrics are reported (e.g., relative joint offset error, contact accuracy, or role assignment accuracy), so the claim that the compression preserves interaction details is unverified.","section":"Section III-C / Table II"},{"comment":"The discussion of MModality is an explicit post hoc reinterpretation: the paper writes that 'if the generated motions are not relevant to the text condition, the higher MModality means the worse instruction-following ability.' This does not resolve the per-text diversity deficit: InterLDM's MModality is 1.032±0.089 versus 7.948±0.064 for real motions and 2.141±0.063 for InterGen. The paper should either report per-text diversity restricted to text-relevant samples (e.g., by retrieval filtering) or acknowledge that the model collapses the diversity of motions for a fixed caption. As written, the text contradicts the claim in the abstract of generating 'two motions with huge differences from one text condition,' and the resolution cannot be a reinterpretation of the metric alone.","section":"Section III-B"}],"minor_comments":[{"comment":"The phrase 'performing text condition' appears to be a typo; it should likely read 'performing text conditioning' or 'performing text following.'","section":"Abstract"},{"comment":"There is a duplicated word in 'Based on on the unified interaction latent space'; please correct to 'Based on the unified interaction latent space.'","section":"Section II-B"},{"comment":"The confidence interval for InterLDM's Top2 R-Precision is reported as 0.559±0.05, while all other intervals use three decimal places; this is likely 0.559±0.005 and should be corrected for consistency.","section":"Table I"},{"comment":"Reference [28], cited as the T2M baseline in Table I, appears to be 'Text2motion: from natural language instructions to feasible plans' in Autonomous Robots, which is a different line of work from the text-to-motion generation method T2M by Guo et al. Please verify and correct this citation, as incorrect attribution affects reproducibility.","section":"References"},{"comment":"The evaluation protocol does not specify how many generated samples are used for each metric, how the 95% confidence intervals are computed, or whether the evaluators (e.g., the motion/text encoders) are the same ones used by InterGen. Please provide these details in the experimental setup.","section":"Section III-A"},{"comment":"The trade-off figure uses dot size for parameter count, but the quantitative parameter counts are not given in text or caption; please provide the exact numbers for all compared methods.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's central design idea is clean and potentially useful, but the evaluation needs to address the implausible MM Dist and the missing reconstruction-only baseline before the quantitative claims are credible. If the authors can show that the VAE alone does not produce the text-similarity gains and that interaction-specific signals survive compression, the paper could become a solid contribution. I do not see grounds for rejection, because the concerns are addressable with additional experiments rather than being inherent to the approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi,\n\nI read the Two-in-One paper. The headline numbers are better than the method's description supports. MM Dist of 1.862 against 3.755 for real motions is not credible on its face, and the paper gives no reconstruction-only metrics to explain it. That alone makes me want to see a referee's copy.\n\nWhat's actually new: treating two-person motion as a single sequence, compressing it with a VAE into one latent token stream, and running a DiT-based latent diffusion on that stream. That's a clean, practical design. The VAE uses sensible geometric losses (foot contact, bone length, velocity) and the latent length is about 1/10 of the raw data, so the claimed 4x inference speedup is plausible. The writing is clear and the method section is easy to follow.\n\nThe soft spots are in the evaluation. First, the low MM Dist and the very low MModality (1.03 vs 7.95 for real) suggest the model is producing motions that are text-close but not diverse. The paper's paragraph reinterpreting MModality as a sign of good instruction-following is post hoc and doesn't address the diversity deficit. Second, the ablation in Table II only varies the latent token length. It never compares a unified latent branch against a two-branch architecture under the same VAE, so the causal claim that the unified space drives the improvement is untested. Third, no code or training details are released, so I can't audit the speedup or the numbers. Finally, the 'first VAE for multi-person motion' claim is narrow given FreeMotion's unified framework, though that's a minor point.\n\nFor a reader in human motion generation, this is a reasonable incremental contribution with a promising direction. But the evidence for the central claim is not tight enough. If I were the editor, I'd send it to peer review because the question is important and the method is plausible, but the authors should be required to release code, report reconstruction-only retrieval metrics, and add a proper unified-vs-two-branch ablation.\n\nBest,\n[You]","headline":"The unified latent idea is sensible, but the implausibly low MM Dist and weak ablations keep me from trusting the numbers.","tokens_in":9071,"tokens_out":3632,"would_cite":false,"duration_ms":31732,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating two people's motion as one data point in a shared latent space improves text-guided generation and runs about four times faster than separate-branch models.","keywords":["multi-person motion generation","interactive motion","latent diffusion model","variational autoencoder","text-to-motion","diffusion transformer","asymmetric interactions","InterHuman"],"falsifier":"Measure reconstruction errors of InterVAE specifically on inter-person quantities, such as relative joint distances, contact flags, and role-swap consistency, for the $f=24$ latent code, and compare those errors directly against a two-branch VAE on the same quantities; if the unified latent does not preserve these interaction details better, the paper's central explanation falls apart.","tokens_in":7919,"feed_emoji":"🕺","tokens_out":6470,"duration_ms":51566,"temperature":0.7,"pith_summary":"This paper aims to show that two-person interactive motion is generated better when the two motions are treated as one indivisible data point rather than as two separate outputs coordinated afterward. The authors build an interaction Variational AutoEncoder (InterVAE) that compresses a two-person motion pair into a single short latent code, and a text-conditioned latent diffusion transformer (InterLDM) that generates that code from natural language. They report that this unified design beats separate-branch baselines such as InterGen on text-following metrics (R-Precision, MM-Dist) and on distribution quality (FID, Diversity), while being roughly four times faster at inference. The practical payoff: text-to-animation systems could produce coordinated two-person scenes with asymmetric roles, such as one person bowing while the other accepts the apology, directly from one sentence.","feed_headline":"One latent space makes two-person motion generation 4x faster","feed_subtitle":"Unified VAE plus latent diffusion follows asymmetric text prompts better than two-branch models, the paper reports.","key_machinery":"The load-bearing object is the unified latent space produced by InterVAE, a transformer VAE that takes the normalized two-person motion sequence together with a learnable latent token and outputs a short latent code $z\\in\\mathbb{R}^{f\\times256}$ (with $f=24$ in the best configuration), so that every latent token carries information about both persons at once. The counterpart is InterLDM, a Diffusion Transformer denoiser that operates on that entire latent code rather than on per-person sequences, using text embeddings from frozen CLIP and T5 encoders. Because the diffusion process denoises one joint latent instead of two separate ones, the model cannot drift into generating each person independently; interaction cues such as facing direction and relative offset live in the same tokens. The reconstruction objective also includes foot-contact, bone-length, and joint-velocity losses, which push the latent to retain the geometric structure of the human skeleton.","core_discovery":"The central claim is that interaction information, including relative spatial relations, contact, and role assignment, is better preserved when both persons' motions are encoded together into one latent space than when each person is generated by a separate branch with cross-attention. InterVAE maps the concatenated two-person motion sequence to a latent variable $z\\in\\mathbb{R}^{24\\times256}$, about a tenth of the raw sequence length, using a transformer encoder-decoder trained with reconstruction, KL, joint-velocity, bone-length, and foot-contact losses. A single Diffusion Transformer denoiser then learns to reverse a Gaussian noising process on this latent, conditioned on text via AdaLN-Zero and classifier-free guidance. On the InterHuman test set, the paper reports that this design raises R-Precision from 0.371 to 0.427, lowers FID from 5.918 to 5.619, lowers MM-Dist from 5.108 to 1.862, and runs about four times faster than InterGen, with the clearest qualitative gains on descriptions that require the two people to perform very different actions.","pith_inferences":["The paper evaluates InterVAE only through downstream generation FID; a direct reconstruction test on interaction-specific quantities such as relative joint distances, contact frames, and role-swap consistency would clarify whether the unified latent truly preserves the interaction information it is claimed to preserve.","If the unified latent is the mechanism behind the text-following gains, the gap between unified and two-branch models should widen as the asymmetry of the described motions increases; this could be tested by binning InterHuman descriptions by role asymmetry.","The same one-data-point strategy may transfer to other multi-agent generative tasks, such as hand-object interaction or group dance, where interaction coherence matters more than per-agent fidelity.","Because the method uses fixed CLIP and T5 text encoders and a single guidance scale, the reported quality is a lower bound on what the architecture could achieve with stronger text conditioning or tuned guidance."],"forward_implications":["Text-to-motion systems can generate coordinated two-person scenes from one sentence, including descriptions with strongly asymmetric roles, such as one person bowing while the other accepts the apology.","The reported gains on R-Precision and MM-Dist imply that generated motions match the text better than separate-branch baselines, not just that they look plausible in isolation.","A single latent diffusion pass with a 24-token code runs about four times faster than two-branch generation at similar model size, making interactive motion generation more practical for real-time animation workflows.","The token-length ablation shows a trade-off: shorter codes speed up inference but hurt reconstruction, while longer codes hurt generation; $f=24$ is the reported sweet spot."],"supporting_citations":[{"why":"Supplies the InterHuman dataset and InterGen baseline, the two-branch cross-attention approach the paper argues against.","marker":"[1]"},{"why":"Provides the denoising diffusion probabilistic model formulation that the latent diffusion process is built on.","marker":"[29]"},{"why":"Provides the Diffusion Transformer (DiT) architecture used as the denoiser, including AdaLN-Zero.","marker":"[31]"},{"why":"Provides classifier-free guidance, used to balance conditional and unconditional generation during sampling.","marker":"[30]"},{"why":"Provides the DPMSolver++ scheduler that enables 25-step sampling.","marker":"[33]"},{"why":"Provides the variational autoencoder framework that InterVAE extends to interactive motion.","marker":"[34]"},{"why":"Frozen CLIP text encoder that produces word-level text condition embeddings.","marker":"[40]"},{"why":"Frozen T5 text encoder that produces sentence-level text condition embeddings.","marker":"[42]"}],"fun_headline_variants":["Single latent space generates two-person motion 4x faster","Unified VAE+Diffuser beats two-branch motion models","One latent space, one model, 4x faster two-person animation","Asymmetric motion prompts? One latent space handles them all","Two-person motion in one latent: 4x speedup, better quality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method hinges on the assumption that a single 24-token latent code can faithfully preserve the interaction details between two people, such as relative positions, contact, and role assignment; if compression loses those details, no amount of diffusion training can recover them.","fun_headline_variants_meta":{"raw":{"variants":["Single latent space generates two-person motion 4x faster","Unified VAE+Diffuser beats two-branch motion models","One latent space, one model, 4x faster two-person animation","Asymmetric motion prompts? One latent space handles them all","Two-person motion in one latent: 4x speedup, better quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1293,"prompt_tokens":919,"completion_tokens":374,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":284}},"tokens_in":535,"tokens_out":374,"duration_ms":3745,"temperature":1.0,"reasoning_tokens":284,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:22:00.751256+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure reconstruction errors of InterVAE specifically on inter-person quantities, such as relative joint distances, contact flags, and role-swap consistency, for the $f=24$ latent code, and compare those errors directly against a two-branch VAE on the same quantities; if the unified latent does not preserve these interaction details better, the paper's central explanation falls apart.","supporting_citations":[{"cited_title":"Intergen: Diffusion- based multi-human motion generation under complex interactions,","cited_arxiv_id":null,"evidence_quote":"Supplies the InterHuman dataset and InterGen baseline, the two-branch cross-attention approach the paper argues against."},{"cited_title":"Scalable diffusion models with transformers,","cited_arxiv_id":null,"evidence_quote":"Provides the Diffusion Transformer (DiT) architecture used as the denoiser, including AdaLN-Zero."}],"review_version":1}