{"id":"113a31ad-4b4c-4611-9e66-39106a2f19dc","arxiv_id":"2412.15664","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A diffusion model generates human motion that simultaneously follows text instructions and adapts to complex 3D terrain, using goal-centric canonicalization and an ego-centric distance field.","lead":"SCENIC is a diffusion model that generates realistic human animation in 3D scenes from text instructions, such as walking upstairs like a zombie or stepping over obstacles. It combines goal-based navigation with local scene geometry so the motion adapts to stairs, slopes, and clutter while following natural-language control.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scene-constraint metrics are computed on synthetic terrains fitted to the motions via the same error function as the metrics; real-scene generalization is only qualitative, so the quantitative terrain-adaptation claim is not yet established.","rationale":"The paper's strongest claim is that SCENIC simultaneously enables text-controlled style editing and adaptation to complex terrains, with physically plausible motion in diverse 3D scenes. The quantitative evidence for terrain adaptation rests on the SCENIC test set, which is constructed by the same synthetic fitting procedure used to create training data. Because the fitting objective (Appendix 2.1, Eqs. 4–7) is effectively the same as the evaluation metrics (Sec. 4.3), and the test patches are refined to fit ground-truth foot contacts, the low penetration and contact-distance numbers partly reflect the data-construction process rather than the model's scene-aware reasoning. Real-scene datasets are only shown qualitatively, so the domain gap between fitted heightmaps and real-world scans is not quantified. This is a correctness risk for the central claim. The concern is addressable by releasing code and computing the same metrics on real scenes, or by providing a statistical comparison of terrain complexity between synthetic and real scenes. The reader identified the same weakest assumption, and the conditional verdict is appropriate: the paper's contribution is plausible and the ablations support the design choices, but the current evidence does not fully establish generalization to real scenes. Secondary issues, such as the text-controllability metric being under-specified and the inference guidance not being ablated, reinforce the need for further evaluation but do not replace the terrain-fitting circularity as the most load-bearing concern.","tokens_in":15865,"tokens_out":5237,"duration_ms":46034,"concrete_test":"Re-run the evaluation on real-scene datasets: take HPS and Matterport3D scenes, compute a watertight mesh or SDF for each real scene, generate SCENIC motions using the same sub-goals and text prompts, and compute the identical Penetration and Contact distance metrics with the released code. If the average penetration and contact distance on real scenes degrade substantially relative to the synthetic-test values, the quantitative scene-adaptation claim does not transfer. Additionally, compare the distribution of heightmap gradient and curvature statistics between the fitted synthetic patches and the real scenes; if they differ strongly, the synthetic test set is not representative of real-world terrain.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing weakness is the circularity between the terrain-fitting data pipeline (Appendix 2.1, Eqs. 4–7) and the evaluation of scene constraints in Table 1. Training and test motion segments are paired with heightmap patches by minimizing Efit = Econtact + Epenetration + Ejump (Eq. 4), where Econtact and Epenetration are computed from foot-joint heights, contact labels, and terrain height—exactly the ingredients used in the physics guidance (Eq. 3) and in the evaluation metrics 'Penetration' and 'Contact distance' (Sec. 4.3). Moreover, the patches are first retrieved by minimizing contact/penetration error and then further refined with RBF editing, so the test terrains are deformed to fit ground-truth foot contacts. Hence the low penetration (1.57 cm) and contact distance (4.51 cm) are partly inherited from the data-construction objective, not necessarily produced by the model's scene reasoning. All quantitative scene-constraint numbers in Table 1 are on these fitted synthetic patches; for real-scene datasets (HPS, Matterport3D, Replica, LaserHuman) only qualitative examples are provided. Therefore the paper's central claim—physically plausible motion adaptation to complex real-world terrains—is not quantitatively supported. The model may generalize, but the evidence as presented does not rule out that it has memorized the statistics of fitted heightmaps rather than learned terrain-adaptive control.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SCENIC, a diffusion-based method for synthesizing human motion that adapts to uneven terrain while following text instructions. The model uses a hierarchical representation: a goal-centric canonical coordinate frame for long-horizon navigation and a local ego-centric distance field for terrain geometry, combined with frame-wise text tokens and an autoregressive segment generation process. A physics-based guidance term is applied at inference to encourage foot-ground contact and prevent penetration. The authors build a synthetic dataset by fitting motion-capture segments onto 4m x 4m heightmap patches from Source Engine terrains, and evaluate on scene constraints (penetration, contact distance), goal accuracy, motion realism (FID, diversity, foot-skate), and a 24-participant user study, reporting improvements over text- and scene-conditioned baselines. Qualitative generalization to four real-scene datasets is shown.","tokens_in":16229,"tokens_out":10156,"duration_ms":77784,"significance":"If substantiated, SCENIC would be a valuable step toward unified text- and scene-controlled motion synthesis, combining two lines of research that are usually separate. The dual canonical/local representation and the frame-wise text conditioning are sensible design choices, and the synthetic data pipeline addresses the scarcity of paired motion-scene-text data. The paper is generally well written and includes ablations and a user study. However, the quantitative evidence for terrain adaptation is weakened by the circularity of the terrain-fitting evaluation, and the text-control claim lacks a direct quantitative metric. With additional experiments on real scenes or non-fitted terrains and proper statistical reporting, the work could be a solid contribution.","major_comments":[{"comment":"The scene-constraint metrics (Penetration and Contact distance) in Table 1 are computed on test terrain patches that were selected and deformed using the same objective as the evaluation metrics: the fitting process in Appendix 2.1 minimizes Efit = Econtact + Epenetration + Ejump (Eqs. 4-7), where Econtact and Epenetration are the exact errors later reported as Contact distance and Penetration in Sec. 4.3. Because the test terrains were fitted to the ground-truth foot contacts, the low numbers (1.57 cm, 4.51 cm) are partly inherited from data construction and do not by themselves demonstrate that the model generalizes to arbitrary terrain geometry. Real-scene results on HPS, Matterport3D, Replica, and LaserHuman are presented only qualitatively. This is the main load-bearing gap for the central claim of terrain adaptation. I recommend evaluating on the actual 3D meshes of real-scene datasets (e.g., by measuring penetration and contact of the generated SMPL bodies against the scans) or on a held-out terrain set generated without fitting to the test motions.","section":"Appendix 2.1, §4.3, Table 1"},{"comment":"All metrics in Table 1 are reported as single values without error bars, multiple sampling seeds, or significance tests. Several of the claimed improvements are small in absolute terms (e.g., penetration 1.57 vs. 1.98 cm; contact distance 4.51 vs. 5.55 cm), and diffusion sampling is stochastic. The authors should report mean plus/minus std over at least three seeds (or multiple sampling runs per model) and include a paired significance test (e.g., bootstrap or Wilcoxon) for the headline scene-constraint and goal-reaching metrics.","section":"§4.4, Table 1"},{"comment":"The contribution of the proposed physics guidance Jphys is not ablated, and the text 'For a fair comparison with baselines, the inference guidance is not activated for all comparisons' is ambiguous. If guidance was disabled in Table 1, then the reported scene-constraint values do not reflect the paper's proposed guidance mechanism; if it was enabled, the comparison is not fair because the baselines lack an equivalent component. Please clarify which setting produced Table 1 and add an ablation 'Ours without Jphys' to isolate its effect.","section":"§3.6, §4.4, Table 1"},{"comment":"No quantitative metric directly measures text-motion alignment. The 'Multimodality' column is described as 'the alignment between the generated motion and the text instruction,' but the reported values (ground truth 6.023, Ours 6.354) are a diversity measure, not an alignment score. The text-control claim rests mainly on qualitative examples and the user study, which is a holistic preference judgment. I recommend adding a standard retrieval-based metric (e.g., motion-to-text retrieval accuracy) or computing CLIP-space similarity between the prompt and the generated motion to substantiate fine-grained text control.","section":"§4.3, Table 1"}],"minor_comments":[{"comment":"The citation 'Botsch and Kobbelt et al. [?]' is missing the full reference; please add the complete bibliographic entry.","section":"Appendix 2.1"},{"comment":"The caption does not define the arrows (down-arrow, up-arrow, right-arrow) nor specify whether lower or higher is better for each column; please add a legend.","section":"Table 1"},{"comment":"The description of 'Multimodality' as a text-alignment measure is inconsistent with its actual definition as a diversity metric; correct either the description or the metric.","section":"§4.4"},{"comment":"The dataset statistics are inconsistent: Table 2 sums to about 168 minutes, which at 30 fps and 60-frame clips yields roughly 5,000 unique segments, while the text claims 15,000 sequences. Please clarify how mirroring and the three fitted terrains inflate the count, and confirm whether the 1,000 test sequences are unique motion clips or augmented copies.","section":"§4.1"},{"comment":"The phrase 'For a fair comparison with baselines, the inference guidance is not activated for all comparisons' is confusing; please rewrite to state explicitly which configuration (with or without guidance) generated the numbers in Table 1.","section":"§3.6"},{"comment":"The scene embedding is described as 'projecting from the point grid perpendicularly toward the scene'; please specify the projection direction (e.g., downward) and how the distance is computed when the ray does not intersect the scene.","section":"§3.2"},{"comment":"Qualitative results are shown only for HPS and Matterport3D in Figures 3 and 5; Replica and LaserHuman are mentioned but not displayed. Either include representative examples for these datasets or refer the reader to the supplementary video.","section":"§4.5"},{"comment":"The user study reports 75.6% preference, but no confidence interval or statistical test is given for this proportion; please report a binomial confidence interval given the 24 participants and 7 questions.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper's 'first' claim is plausible but difficult to verify given several concurrent works; the authors do discuss differences with [13] and [90]. The main concern is the circular evaluation. I would be willing to reconsider after the authors add quantitative real-scene evaluation or non-fitted test terrains, plus error bars and the missing guidance ablation. The paper fits the venue's scope if these issues are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The paper does something genuinely new: it puts text-conditioned style control and complex terrain adaptation into one autoregressive diffusion model, and the goal-centric canonicalization plus ego-centric distance field is a sensible way to combine high-level navigation with local geometry. The data pipeline—fitting short mocap segments to game-engine heightmaps—is a practical answer to the paired-data scarcity problem, and the ablations show both main design choices matter. The user study, while small, is at least a real preference test. I'd send this to a serious venue.\n\nThe soft spots are mostly about what the numbers can and cannot claim. The scene-constraint metrics (penetration, contact distance) are computed on test terrains that were selected and then deformed by minimizing essentially the same error functions used as evaluation metrics. That doesn't make the metrics meaningless—the model still has to produce motion on fixed test terrain—but it does mean low penetration/contact numbers are partly inherited from data construction. The real-scene generalization (HPS, Matterport, Replica, LaserHuman) is qualitative only, so the central claim that the model adapts to complex real-world terrains is not quantitatively supported yet. I also note the paper does not report error bars or multiple seeds, and the proposed physics guidance is explicitly disabled in the comparisons, so its contribution is untested. These are fixable: report metrics on a real-scene benchmark or at least on hold-out terrain patches that were not fitted with the same objective, ablate the guidance, and give variance estimates.\n\nI don't think this is a fatal flaw. The model likely does learn something about terrain adaptation, and the qualitative examples are suggestive. But the paper currently overstates what the quantitative evidence shows. If I were the editor, I'd send it to review with the expectation that the authors tighten the evaluation. The code/data release promise, if honored, would also make the data pipeline auditable.","headline":"A solid unification of text-controlled motion synthesis with terrain adaptation, but the quantitative scene-constraint evidence is partly inherited from the terrain-fitting pipeline and real-scene generalization is only qualitative.","tokens_in":16723,"tokens_out":1881,"would_cite":true,"duration_ms":13785,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SCENIC is a diffusion model that generates physically plausible human motion adapting to complex 3D scenes while following natural-language instructions such as \"walking upstairs like a zombie.\"","keywords":["human motion synthesis","diffusion models","text-conditioned generation","scene-aware motion","terrain adaptation","navigation","goal-centric canonicalization","distance field scene embedding"],"falsifier":"Measure penetration depth and foot-contact distance on held-out raw real-scene scans that were never run through the terrain-fitting pipeline, using motion capture of a person walking the same paths, and compare SCENIC against its baselines; if SCENIC's advantage disappears, the fitting process rather than the model was carrying the scene-constraint score.","tokens_in":15684,"feed_emoji":"🚶","tokens_out":5978,"duration_ms":50789,"temperature":0.7,"pith_summary":"SCENIC is an attempt to close a gap that has split the motion-synthesis field: either a model adapts to uneven terrain, stairs, and obstacles, or it obeys text commands, but not both. The paper claims a single diffusion-based architecture can do both, generating arbitrarily long animations in which a virtual character reaches user-specified sub-goals, keeps its feet on the ground, avoids sinking into surfaces, and changes gait or style mid-motion according to prompts like \"walking upstairs like a zombie.\" The proposed recipe is hierarchical: a goal-centric canonical frame handles navigation, an ego-centric distance field handles local geometry, and per-frame text tokens align language with each moment of motion. If the claim holds, it gives game and embodied-AI developers a way to script believable navigation with natural language instead of hand-tuned controllers or flat-floor assumptions.","feed_headline":"SCENIC: one model steers virtual humans by text over rough terrain","feed_subtitle":"SCENIC adds local scene geometry to language control, so \"zombie upstairs walk\" stays grounded on stairs.","key_machinery":"The load-bearing object is the pair of scene representations. Goal-centric canonicalization re-expresses the motion and the scene in the coordinate frame of the current sub-goal, so the model learns to generate motion that converges to the target rather than merely conditioning on it. The ego-centric scene embedding is a heightmap-like distance field sampled on a 144-point grid centered at the root with orientation relative to the root's yaw, which gives the diffusion transformer local geometry without global coordinates. Frame-wise text tokens, obtained by reducing CLIP embeddings with PCA, align language with each frame so style can switch mid-sequence, and autoregressive diffusion with physics guidance, which penalizes foot penetration and enforces contact, stitches the segments into long plausible motions.","core_discovery":"The paper's central claim, stated on its own terms, is that scene-aware and semantically controlled motion synthesis can be solved in one model rather than composed from separate modules. Concretely, SCENIC takes a 3D scene, a trajectory of sub-goals, and a text prompt, and outputs SMPL-parameterized motion segments that converge on each goal, adapt to the terrain underfoot, avoid penetrations, and follow the requested style. The authors report the lowest penetration, contact distance, goal-reaching error, FID, and foot-skate values among the compared methods on their test set, and 75.6% of user-study participants preferred SCENIC over the two baselines. They also show qualitative generalization to four real-scene datasets, where the model was trained only on synthetic fitted terrain patches.","pith_inferences":["Left implicit in the paper: the quantitative scene-constraint metrics are computed on synthetic test patches produced by the same fitting pipeline, so the reported penetration and contact numbers may partly measure the fitting process rather than the model's generalization.","A testable extension would evaluate SCENIC on unmodified real-world scans with quantitative foot-ground contact measurements, separating model skill from data construction.","The dual representation could transfer to dynamic obstacles and object-carrying tasks, since the ego-centric distance field is agnostic to whether the geometry it encodes is static.","The frame-wise text alignment suggests SCENIC could serve as a controllable motion prior for downstream planning or reinforcement-learning pipelines, though the paper does not explore that use."],"forward_implications":["A single model can follow compound instructions such as \"walk upstairs like a zombie\" while keeping feet on the stairs and avoiding body-scene penetration.","Because synthesis is autoregressive over sub-goals, the same model produces arbitrarily long navigation sequences rather than fixed-length clips.","Per-frame text tokens let motion style change mid-sequence, so a character can transition from crouching to hopping without breaking scene constraints.","A companion interaction module extends the framework to object contact, so navigation can end in actions such as sitting on a sofa or lying down.","Training on synthetic fitted terrain patches transfers to unseen real indoor scenes from Replica, Matterport3D, HPS, and LaserHuman."],"supporting_citations":[{"why":"Supplies the terrain heightmaps and the motion dataset that the SCENIC training set is built from.","marker":"[26]"},{"why":"Provides the planar-encoding idea that the ego-centric distance field scene embedding adapts.","marker":"[7]"},{"why":"Supplies the per-frame text alignment approach used for frame-wise motion-text tokens.","marker":"[38]"},{"why":"Provides a text-instruction human-scene interaction method and dataset reference that SCENIC extends.","marker":"[29]"},{"why":"TRUMANS is the main scene-interaction baseline, with its occupancy representation and action encoding modified for comparison.","marker":"[31]"},{"why":"FlowMDM is the motion-composition baseline that SCENIC compares against for seamless segment transitions.","marker":"[2]"},{"why":"TeSMO provides the text-controlled collision-free navigation baseline and the collision guidance formulation.","marker":"[80]"},{"why":"SMPL is the body model used to parameterize and animate the generated human motion.","marker":"[47]"}],"fun_headline_variants":["SCENIC: text-controlled motion that adapts to complex 3D scenes","Tell SCENIC to 'walk like a zombie' — it obeys, even uphill","One diffusion model fuses scene geometry and text to steer motion","SCENIC grounds language commands in 3D scene geometry","Text tells SCENIC where to walk; the scene tells it how"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that motion clips paired with automatically chosen and deformed synthetic heightmap patches form a training distribution that transfers to real scenes, since every quantitative scene-constraint number is measured on patches produced by that same fitting pipeline.","fun_headline_variants_meta":{"raw":{"variants":["SCENIC: text-controlled motion that adapts to complex 3D scenes","Tell SCENIC to 'walk like a zombie' — it obeys, even uphill","One diffusion model fuses scene geometry and text to steer motion","SCENIC grounds language commands in 3D scene geometry","Text tells SCENIC where to walk; the scene tells it how"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001183,"raw_usage":{"total_tokens":4910,"prompt_tokens":992,"completion_tokens":3918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":3820}},"tokens_in":608,"tokens_out":3918,"duration_ms":24720,"temperature":1.0,"reasoning_tokens":3820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:12:00.738976+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure penetration depth and foot-contact distance on held-out raw real-scene scans that were never run through the terrain-fitting pipeline, using motion capture of a person walking the same paths, and compare SCENIC against its baselines; if SCENIC's advantage disappears, the fitting process rather than the model was carrying the scene-constraint score.","supporting_citations":[{"cited_title":"ACM Trans","cited_arxiv_id":null,"evidence_quote":"Supplies the terrain heightmaps and the motion dataset that the SCENIC training set is built from."},{"cited_title":"In: SIGGRAPH Asia Confer- ence Papers (2024) 2, 3, 4, 5","cited_arxiv_id":null,"evidence_quote":"Provides a text-instruction human-scene interaction method and dataset reference that SCENIC extends."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"TRUMANS is the main scene-interaction baseline, with its occupancy representation and action encoding modified for comparison."},{"cited_title":"ACM Trans","cited_arxiv_id":null,"evidence_quote":"SMPL is the body model used to parameterize and animate the generated human motion."}],"review_version":1}