{"id":"ef78759c-f80f-46e2-8228-7555c789e97c","arxiv_id":"2608.02236","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Using text-to-motion generation to inject kinematic structure into text prototypes improves zero-shot skeleton action recognition, though most of the reported gain comes from test-time prototype self-refinement.","lead":"A computer-vision team shows that text-to-motion generative models can supply the kinematic structure that text-only descriptions lack, improving zero-shot skeleton action recognition on standard benchmarks. The method fuses generated motion statistics into text prototypes via a learned gate, then refines unseen-class prototypes with high-confidence test samples.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unspecified Shift-GCN pre-training may leak unseen-class labels, undermining zero-shot validity; generated-motion joint topology conversion also unverified.","rationale":"The reader's weakest assumption—leakage-free, feature-compatible Shift-GCN—is exactly the load-bearing concern. The paper's silence on Φ's pre-training is a genuine validity threat, not just a missing detail. If Φ was supervised on the full NTU dataset, the zero-shot setting is broken, and the central claim that T2M priors help would not survive. The joint-topology mismatch is an additional concrete risk that the paper does not address. Both concerns are testable through code inspection and a controlled rerun. The generative-prior idea itself is plausible and the ablations (Table 4) show GESF alone gives modest gains, but the headline SOTA numbers are driven by GPR, which amplifies any leakage in Φ. Thus the reader's CONDITIONAL verdict remains appropriate: the concern does not currently invalidate the work, but it must be resolved before the central claim is accepted. I agree with the reader's identification and would not adjust the verdict based on this pass.","tokens_in":17131,"tokens_out":3368,"duration_ms":32206,"concrete_test":"Inspect the released code and checkpoints to identify the exact Shift-GCN weights used as Φ. If the checkpoint was trained with labels on NTU-60/120 (or includes the 'unseen' classes in any way), re-run the Table 1 NTU-120 96/24 experiment with Φ replaced by a strictly self-supervised or cross-dataset skeleton encoder (e.g., a model pre-trained only on a different dataset like PKU-MMD or a contrastive encoder trained without labels), keeping all other components fixed. If the accuracy drops materially (e.g., >5 points), the reported results are inflated by leakage. Separately, log the joint counts and coordinate conversion details; verify the generated motions are re-targeted to NTU's 25-joint format before feeding into Shift-GCN, or confirm the encoder accepts 22-joint input. A successful reproduction with a clean encoder would resolve the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that T2M-generated motions improve zero-shot skeleton action recognition. This requires a shared, leakage-free feature space in which both real NTU skeletons and synthetic HumanML3D-style motions are represented equivalently. §5.2 states only 'We use Shift-GCN [8] to extract both real and generated skeleton features' without specifying how Φ was pre-trained. If Φ is a supervised model trained on NTU-60/120, including the classes later designated as 'unseen,' then the zero-shot evaluation is invalid: unseen-class features would already be encoded with label-relevant information, and the test-time prototype refinement (Eqs. 8–10) could exploit this leakage, attributing to T2M priors what is actually memorized discrimination.\n\nA second, related gap: §4.1 says generated motions are 'converted from relative spatial configurations to absolute joint coordinates,' but T2M models like StableMoFusion are trained on HumanML3D's 22-joint SMPL skeleton, while Shift-GCN (as used for NTU) expects a 25-joint topology. The paper does not describe re-targeting or dimension alignment. If the conversion does not produce features comparable to real NTU skeletons, then the kinematic prototypes (Eq. 3) and dispersion (Eq. 4) are computed in an incompatible space, and the gated fusion (Eqs. 5–6) injects misleading signals.\n\nBoth issues are load-bearing because the entire framework routes real and generated data through the same encoder Φ. If Φ is not a clean, validated shared space, the claimed gains—especially the +16.2% on NTU-120 96/24—cannot be attributed to generative priors. The paper offers no control experiment to rule out these confounds.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"GenPrior proposes a zero-shot skeleton-based action recognition (ZSAR) framework that incorporates priors from a frozen text-to-motion (T2M) model. For each action class, the method generates multiple motions from LLM-expanded descriptions, extracts skeleton features with Shift-GCN, and forms generative kinematic prototypes and per-dimension dispersion vectors. A dispersion-gated network blends these generative cues with CLIP text embeddings, and the fused representations are aligned to real skeleton features via a cross-modal VAE. At inference, a test-time prototype refinement step (GPR) pseudo-labels test samples against the semantic prototypes, selects high-confidence samples per class, and replaces each class prototype with the mean of those selected features. The paper reports state-of-the-art results on NTU-60, NTU-120, and PKU-MMD under both ZSL and GZSL, with ablations over components, T2M backbones, and the selection ratio.","tokens_in":17393,"tokens_out":5411,"duration_ms":50866,"significance":"If the technical assumptions are validated, the core idea — using externally pre-trained T2M models as a source of kinematic structure for text-only prototypes — is a novel and plausible direction for ZSAR. The paper is also commendable for releasing code, evaluating on multiple benchmarks and splits, abalating the main components, and showing robustness across three different T2M backbones. The significance, however, hinges on two currently unverified assumptions: (i) the frozen skeleton encoder is trained in a way that does not leak unseen-class supervision, and (ii) generated motions from a HumanML3D-style 22-joint skeleton are actually feature-compatible with real NTU skeletons in a 25-joint space after the described conversion. In addition, the transductive nature of GPR is not reflected in the abstract or the main SOTA claim. These issues are central rather than cosmetic.","major_comments":[{"comment":"The paper states 'We use Shift-GCN [8] to extract both real and generated skeleton features' but does not specify how Φ was pre-trained. If Φ was supervised on NTU-60 or NTU-120 using all classes, including those later designated as unseen, then the feature space already encodes unseen-class label information. This would invalidate the zero-shot setting and could inflate every reported number, since the kinematic prototypes (Eq. 3), the gated fusion (Eqs. 5-6), and the refinement step (Eq. 10) all operate on this shared feature space. The authors must disclose the exact pre-training data and protocol of Φ, or re-run the experiments with an encoder trained only on the seen classes (or a self-supervised encoder) and show that results are maintained. A leakage test — e.g., linear probe accuracy on unseen classes using frozen features — would also be informative.","section":"§5.2, §4.1, Eq. (2)-(4)"},{"comment":"StableMoFusion and other HumanML3D-trained T2M models produce motions on a 22-joint SMPL skeleton, whereas NTU datasets and Shift-GCN expect 25 joints. The paper says generated motions are 'converted from relative spatial configurations to absolute joint coordinates,' but this does not address the joint-topology / dimension mismatch. Without an explicit retargeting or dimension-alignment procedure, it is unclear in what coordinate space the features in Eq. (2) are computed, and consequently the kinematic prototype (Eq. 3) and dispersion (Eq. 4) may be statistics of a space incompatible with real NTU features. Please provide the exact joint mapping/padding, and validate feature compatibility quantitatively (e.g., t-SNE overlap of real and generated features for seen classes, or a nearest-neighbor accuracy between generated and real features).","section":"§4.1, §5.2, Eq. (2)-(4)"},{"comment":"GenPrior uses unlabeled test samples at inference through GPR (Eqs. 8-10), making it a transductive / test-time adaptation method. Table 1 nevertheless compares it with inductive baselines, and the abstract claims 'state-of-the-art performance' without this caveat. The transductive advantage is only disclosed in the table footnote. This is an apples-to-oranges comparison: a transductive method can trivially outperform inductive methods on the same benchmark. Please report an inductive variant (GESF only, without GPR) alongside the transductive variant, and compare with inductive baselines on equal footing. The transductive version should be compared separately with SC under the same protocol, and the main SOTA claim should be qualified.","section":"Table 1, Abstract, §5.3"},{"comment":"The GPR refinement is self-referential: test samples are assigned to classes using the current prototypes (Eq. 8), the top-η most confident samples for each class are those closest to the prototypes, and the refined prototype is the mean of those same samples' features (Eq. 10). The final evaluation then classifies the same test samples using these refined prototypes. This is a form of transductive self-training, and the observed gains may partly reflect confirmation bias rather than genuine prototype correction. To support the claimed causal benefit, please add control experiments: (i) random selection of the same fraction η instead of confidence-based selection, (ii) oracle selection using true labels, and (iii) an analysis of per-class accuracy changes alongside the number of selected samples. Report how much of the gain comes from the confidence selection mechanism versus the averagi","section":"§4.2, Eqs. (8)-(10), Table 4"}],"minor_comments":[{"comment":"The phrase 'first framework' should be qualified to 'first to our knowledge' or supported by a systematic related-work statement; in any case, the transductive nature of GPR should be mentioned in the abstract if SOTA is claimed.","section":"Abstract"},{"comment":"Please state explicitly that the square root and squaring are element-wise operations. Currently the equation is ambiguous without the surrounding text.","section":"Eq. (4)"},{"comment":"The logical notation with ⊕ and the masking constant λ is hard to parse. Please define clearly how the seen/unseen decision is made and how γ is selected. Also report the chosen values of γ, λ, and λ_Dist in the implementation details.","section":"Eq. (16)"},{"comment":"The GenPrior row has a dash in the 'Venue' column. If the paper is under review, use 'Ours' or '—'; this is fine, but please keep the footnote about test-time adaptation in the main text as well.","section":"Table 1"},{"comment":"Typo in the figure: 'V AE' should be 'VAE'.","section":"Fig. 2"},{"comment":"For random splits, the paper reports averages but no standard deviation or per-split results. Adding standard deviations would strengthen the robustness claim.","section":"§5.2 / Table 2"},{"comment":"The generation budget J=40, K=10 is fixed. A sensitivity study over J and K (or total N_g) would help confirm that the method is not dependent on a particular generation budget.","section":"§5.4, Table 6"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The two load-bearing concerns — the unknown pre-training of Shift-GCN and the 22-joint vs 25-joint topology mismatch — are potentially fatal if they cannot be resolved. I do not recommend rejection immediately because both are addressable with additional experiments and careful reporting, but without them the central zero-shot claim is not established. The comparison protocol also needs fixing: the paper should present the inductive variant as the main zero-shot result and treat GPR as a transductive add-on."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GenPrior's genuinely new contribution is using a frozen text-to-motion generator as an external motion-prior source for zero-shot skeleton action recognition. Prior work enriched text with LLMs or trained internal aligners; nobody had queried a T2M model for class-level kinematic structure. That idea is worth something, and the dispersion-gated fusion is a sensible way to keep synthetic artifacts out. The ablations show the gated fusion alone gives consistent, if modest, gains over a text-only baseline (+1–4 points). The paper is clearly written and the math is straightforward.\n\nThe soft spots are real but proportionate. The largest is that the paper's SOTA claim mixes inductive and transductive settings. Generative Prototype Refinement pseudo-labels test samples and re-estimates prototypes from those same samples. That is test-time adaptation, not inductive zero-shot recognition, and Table 4 attributes up to +12 points to GPR. The footnote in Table 1 discloses this, but the abstract and Section 5.3 present the results as a uniform SOTA comparison. Flora and other inductive baselines are being compared against a method with access to test features at inference. That is an apples-to-oranges comparison for the headline numbers.\n\nTwo load-bearing details are missing. The paper never says how the Shift-GCN encoder was pre-trained. If it was supervised on the same datasets including the 'unseen' classes, the evaluation is invalid. And T2M models like StableMoFusion are trained on 22-joint SMPL skeletons, while NTU uses 25 joints; the relative-to-absolute conversion mentioned in §4.1 doesn't address joint re-targeting. If the feature spaces of real and generated skeletons aren't actually compatible, the kinematic prototypes are artifacts. These are not necessarily fatal — they may be easily answered — but they can't be checked from the paper.\n\nAlso missing: exact values of γ, λ, λ_Dist, and any error bars. Minor relative to the above.\n\nMy judgment: the generative-prior idea is novel and should be explored, but the current paper overstates the SOTA. It deserves a serious referee, not a desk reject, and the review should focus on repositioning the method as transductive/test-time adaptive ZSAR, adding the encoder pre-training and topology details, and reining in the comparisons. I'd bring this to a reading group as a case study in how a clean novel prior gets entangled with transductive leakage, and I'd probably cite the T2M-prior contribution once the protocol questions are answered.","headline":"Genuinely new T2M-prior idea, but the SOTA numbers are driven by transductive test-time refinement, and encoder pre-training / topology conversion details are missing.","tokens_in":18060,"tokens_out":3758,"would_cite":true,"duration_ms":31206,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GenPrior shows that text-to-motion generative priors can be repurposed to close the semantic–kinematic gap in zero-shot skeleton action recognition, setting a new state of the art on three benchmarks.","keywords":["zero-shot skeleton action recognition","text-to-motion generation","generative priors","semantic-kinematic gap","prototype refinement","cross-modal alignment","dispersion gating"],"falsifier":"Inspect the pretraining of the skeleton encoder: if it was trained on any NTU class that appears among the 'unseen' test categories, the zero-shot evaluation is invalid. Separately, encode the same generated motion before and after the relative-to-absolute conversion and measure the feature distance; if the conversion shifts features substantially, the kinematic prototypes are an artifact of the conversion rather than of motion content.","tokens_in":16883,"feed_emoji":"🦴","tokens_out":5807,"duration_ms":45537,"temperature":0.7,"pith_summary":"Zero-shot skeleton action recognition aligns skeleton sequences with text descriptions, but text-derived prototypes lack geometric structure and physical constraints. GenPrior tests a new source of information: a pre-trained text-to-motion model that generates synthetic motions for each action class. From these generated motions, the method distills a kinematic prototype and an intra-class dispersion vector, fuses them into CLIP text embeddings through a learned gate, and aligns the fused semantics with real skeleton features in a shared latent space. At test time, it refines class prototypes by mining high-confidence test samples. The paper reports state-of-the-art ZSL and GZSL results on NTU-60, NTU-120, and PKU-MMD, with the largest gains when unseen classes are numerous.","feed_headline":"Synthetic motions raise zero-shot skeleton accuracy to 90.4%","feed_subtitle":"Distilling kinematic structure from generated motions into text prototypes beats prior zero-shot baselines on three benchmarks.","key_machinery":"The load-bearing object is the generative kinematic prototype plus intra-class dispersion vector distilled from text-to-motion outputs (mean and per-dimension standard deviation of encoded generated motions), injected into text embeddings by the Dispersion-Gated Feature Fusion module, and later used as anchors in the Generative Prototype Refinement module. The gate conditions on text, projected prototype, and dispersion to suppress synthetic artifacts; the refinement mines high-confidence test samples by entropy ranking and replaces semantic prototypes with the mean of those samples.","core_discovery":"The central claim is that synthetically generated motion sequences carry structural kinematic information that pure text embeddings lack, and that this information can be injected into textual prototypes without contaminating them with synthetic artifacts. GenPrior does this by using a frozen text-to-motion model to generate diverse motions per class, encoding them with a pre-trained skeleton encoder, and computing per-class means and per-dimension standard deviations. A learned dispersion-gated fusion network decides how much of the kinematic prototype to add to each text embedding, using the dispersion vector as a reliability measure. The fused features are aligned with real skeleton featu","pith_inferences":["The dispersion vector could serve double duty as a per-class uncertainty estimate, enabling applications like active labeling or rejection of low-confidence generated prototype classes.","Because the method only needs text descriptions and a frozen T2M model, it is applicable to novel action vocabularies without retraining; one testable extension is whether the same priors transfer to cross-dataset recognition with different skeleton topologies.","The ratio-based seen/unseen calibration in GZSL could be replaced or learned rather than fixed at gamma, potentially improving the balance further on datasets with skewed prior class frequencies."],"forward_implications":["The reported gains grow as the number of unseen classes increases (e.g., +14.9% on the 96/24 NTU-120 split), suggesting generative priors help most where text-only prototypes are most ambiguous.","Performance is stable across three different text-to-motion backbones (within 1.3%), implying the benefit is a property of the prior source class rather than one specific generator.","The ablation shows ungated injection of kinematic prototypes hurts accuracy, while the dispersion-gated fusion recovers and exceeds the baseline — evidence that reliability-aware fusion is necessary for exploiting synthetic data.","The Generative Prototype Refinement improves 19 of 24 unseen classes on the hardest split, with large gains on fine-grained object-interaction actions like taking off a hat or glasses.","Since the method is training-free at test time apart from prototype re-estimation, it can be layered on existing embedding-based ZSAR models."],"fun_headline_variants":["GenPrior reaches 90.4% zero-shot skeleton accuracy via synthetic motion priors","Generated motions sharpen zero-shot skeleton recognition to 90.4%","Text-to-motion priors close semantic-kinematic gap for zero-shot action","Synthetic sequences calibrate prototypes for 90.4% zero-shot skeleton recognition","Kinematic priors from motion generators boost zero-shot action recognition"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the frozen skeleton encoder maps both real NTU skeletons and text-to-motion generated motions into the same trustworthy feature space without leaking unseen-class supervision; the paper does not state how that encoder was pretrained and does not verify that the relative-to-absolute coordinate conversion preserves feature compatibility.","fun_headline_variants_meta":{"raw":{"variants":["GenPrior reaches 90.4% zero-shot skeleton accuracy via synthetic motion priors","Generated motions sharpen zero-shot skeleton recognition to 90.4%","Text-to-motion priors close semantic-kinematic gap for zero-shot action","Synthetic sequences calibrate prototypes for 90.4% zero-shot skeleton recognition","Kinematic priors from motion generators boost zero-shot action recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000337,"raw_usage":{"total_tokens":1689,"prompt_tokens":720,"completion_tokens":969,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":879}},"tokens_in":464,"tokens_out":969,"duration_ms":8761,"temperature":1.0,"reasoning_tokens":879,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T10:55:30.875728+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the pretraining of the skeleton encoder: if it was trained on any NTU class that appears among the 'unseen' test categories, the zero-shot evaluation is invalid. Separately, encode the same generated motion before and after the relative-to-absolute conversion and measure the feature distance; if the conversion shifts features substantially, the kinematic prototypes are an artifact of the conversion rather than of motion content.","supporting_citations":[],"review_version":1}