{"id":"59470aab-bf3d-452a-b24e-e61c8270d3e3","arxiv_id":"2506.05397","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Gen4D creates diverse and photorealistic synthetic sports videos from text and estimated motion, producing the SportPAL dataset for training human pose estimation models.","lead":"This paper presents Gen4D, an automated pipeline that turns text prompts and internet sports videos into synthetic 4D human animations with posed bodies and realistic backgrounds. It uses this pipeline to build SportPAL, a synthetic sports dataset of more than half a million annotated frames, and shows pose estimators can be trained on the generated frames.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SportPAL's utility claim is only tested synthetic-to-synthetic: Table 2 trains and evaluates on the same Gen4D renderer's output, so high AP reflects self-consistency, not transfer to real sports imagery.","rationale":"I read Gen4D as a dataset-generation paper: the central claim is that a fully automated pipeline can create large-scale, diverse synthetic sports data whose annotations are useful for training in-the-wild HPE models. The load-bearing condition is therefore that synthetic frames carry transferable signal for real sports imagery. The paper's experiments test internal consistency, not that condition. Table 2 reports models trained and evaluated on SportPAL; Table 4 transfers between synthetic sport subsets. Because the renderer projects and annotates the same estimated pose, high AP is expected and says little about the real domain. This is not a mathematical error but an evidentiary gap at the center of the contribution. The reader's weakest_assumption about SMPLer-X pseudo-groundtruth is a plausible mechanism for failure, but the more direct check is external transfer: if SportPAL-trained models transfer to real benchmarks, the pseudo-groundtruth concern is empirically moot, and if they fail, the self-consistent synthetic numbers do not save the claim. A secondary internal inconsistency (Table 3 lists 50 total subjects while per-sport subject counts sum to 85, implying subject reuse across sports) reinforces the need for careful split documentation, but it is not the main concern. I therefore recommend retaining the CONDITIONAL verdict (UNCHANGED), conditioned on external real-data evaluation and release of data/code.","tokens_in":10744,"tokens_out":8190,"duration_ms":92044,"concrete_test":"Train a standard HPE model (e.g., TokenPose) on a SportPAL training split and evaluate zero-shot on a real sports pose benchmark with ground-truth keypoints, such as ASPset-510 or SportsPose for outdoor sports or AthletePose3D for ice hockey, under the same protocol as (a) a model trained on the real benchmark's training split and (b) a model trained on an existing synthetic dataset such as BEDLAM. If the SportPAL-trained model transfers at or near real-trained performance, the concern is resolved; if performance collapses, the synthetic-to-synthetic results in Table 2 do not establish utility.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central purpose of Gen4D/SportPAL is to supply training data for in-the-wild human-centric vision tasks (Abstract, §1). The quantitative support is Table 2/Figure 5, where TokenPose and a DETR baseline are trained and evaluated on SportPAL frames, plus Table 4, which transfers between synthetic sport subsets. Both are internal to the Gen4D renderer. §3.1 deliberately renders the SMPLer-X pose estimate back into the synthetic image, so image-pose pairs are exactly consistent by construction; near-saturated AP15 scores (icehockey 99.91, soccer 98.51) are therefore a learnability check, not evidence of quality or real-world utility. If SMPLer-X systematically distorts sports-specific articulation or occluded joints (bulky gear, skating), that bias is invisible in synthetic testing but would transfer to real data. The Future Work paragraph concedes that pseudo-groundtruth jitter and physical plausibility are not optimized. No experiment compares against a real sports pose benchmark (AthletePose3D, SportsPose, ASPset-510, WorldPose), against a real-data-trained model, or against an existing synthetic dataset (BEDLAM, AGORA, SURREAL), and no code or data are released. Thus the central claim that SportPAL is a scalable foundation for in-the-wild tasks is not yet supported; the experiments show the simulator is self-consistent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gen4D, a fully automated pipeline that extracts SMPLX pose and shape parameters from internet videos via a pretrained mesh-recovery method, optimizes canonical human Gaussians using dual-branch Score Distillation Sampling guided by text prompts, deforms and rasterizes these avatars under varied camera viewpoints, and composites human-aware backgrounds using an IC-Light diffusion module. Based on this pipeline, the authors introduce SportPAL, a synthetic dataset of over 583,000 frames spanning baseball, ice hockey, and soccer, with 2D/3D pose, SMPLX parameters, bounding boxes, segmentation masks, and action labels. The experimental section evaluates two 2D human pose estimators trained and tested on SportPAL and reports a cross-sport fine-tuning study.","tokens_in":11012,"tokens_out":3959,"duration_ms":46847,"significance":"If the central claims are supported, the pipeline would be a useful contribution: it removes the need for manual 3D modeling and scene design, sources motion from public internet videos, and produces a large-scale annotated synthetic dataset for an under-served domain. The paper gives a reasonably detailed description of the pipeline and provides a dataset statistics table. Its main strength is the proposed automation and scale. However, the evidence currently demonstrates only internal self-consistency of the renderer and annotation pipeline. The claimed photorealism, diversity, annotation accuracy, and utility for in-the-wild tasks are not measured against real data, and no code or data are released, so independent verification is impossible.","major_comments":[{"comment":"The evaluation is entirely synthetic-to-synthetic and therefore does not support the central claim that SportPAL is a scalable foundation for in-the-wild human-centric vision tasks. Both the training and test sets are generated by the same Gen4D pipeline, so near-saturated AP15 scores (99.91 for ice hockey, 98.51 for soccer) reflect self-consistency of the renderer and annotation generator rather than transferable data quality. Please add a real-data transfer experiment on at least one public sports pose benchmark (e.g., AthletePose3D, SportsPose, ASPset-510, or WorldPose), and ideally a comparison with an existing synthetic dataset such as BEDLAM or AGORA under matched training conditions.","section":"§5.2, Table 2"},{"comment":"The annotations are not independent measurements. SMPLX parameters estimated by SMPLer-X from internet videos are normalized and used both as the pseudo-motion that drives rendering and as the ground truth in SportPAL. The paper's statement in §3.1 that perfect image–pose pairs will be generated is true only in the sense of internal consistency, not in the sense of correctness relative to real human motion. If SMPLer-X systematically errs on sports-specific articulation, occlusions, or bulky gear, that error is baked into every rendered frame and remains invisible in the synthetic test set. Please provide a validation study against independent motion capture or manual annotation on a subset of the three sports, and report per-joint error or failure modes.","section":"§3.1, §4"},{"comment":"The Future Work paragraph concedes that pseudo-ground-truth jitter and physical plausibility are not optimized, yet the abstract and introduction describe the generated sequences as 'lifelike' and 'photorealistic.' No experiment quantifies motion jitter, foot skating, or physical plausibility, and no perceptual study or image-quality metric supports the photorealism claim. Please add quantitative motion-quality metrics or a user study, or soften the claims accordingly.","section":"§6, Future Work"}],"minor_comments":[{"comment":"Table 2 cites 'TokenPose [17]', but reference [17] is Dwivedi et al. (TokenHMR), while the in-text citation for TokenPose is [29]; please correct the citation mismatch.","section":"Table 2"},{"comment":"In the Soccer row, '57110' should read '57,110' for readability, and the Total row should be checked for consistency with the per-sport frame counts.","section":"Table 3"},{"comment":"The symbols L1, L2, M, and Id are not defined precisely in the surrounding text; please define each variable next to the equation or in a notation list.","section":"Equation (2)"},{"comment":"The claim that SportPAL is 'the largest dataset featuring the most diverse range of actions from various sports' is not supported by a quantitative comparison with the datasets listed in Table 1; please add diversity or coverage metrics or qualify the claim.","section":"§4"},{"comment":"The SDS loss weights λ1 and λ2 and the background-diffusion weights λv and λic are fixed without an ablation or sensitivity analysis; please provide a brief study or at least a rationale for the chosen values.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central claims outpace its experiments: the paper markets a scalable foundation for in-the-wild tasks but provides no real-data transfer evaluation, no independent validation of the pseudo ground truth, and no release of code or data. These gaps are fixable within the paper's scope, so I recommend major revision rather than rejection. I would also ask the editor to check that the several self-citations in the reference list are used only where appropriate and not to inflate the novelty claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is SportPAL: a half-million-frame synthetic dataset with rich annotations, built by a plausible pipeline that combines motion extraction from internet videos, text-guided Gaussian avatar generation, and diffusion-based background synthesis. That is genuinely useful, especially for sports pose estimation where real data with 3D annotations is hard to get. The prompt modeling scheme for avatar diversity is a reasonable new integration, and the paper is honest about assembling existing parts rather than claiming new first-principles theory.\n\nThe soft spot is not the pipeline—it is the validation. The central claim is that SportPAL can serve as a scalable foundation for in-the-wild human-centric tasks. But every quantitative experiment in Section 5 is synthetic-to-synthetic. Table 2 trains and evaluates on SportPAL itself, and the near-saturated scores (icehockey AP15 99.91) reflect that the annotations are exactly consistent with the rendered images by construction, not that the dataset transfers to real sports footage. Table 4 shows cross-sport fine-tuning between synthetic subsets, which again only tests internal consistency. There is no comparison against a real sports pose benchmark, no test on real images, no comparison with BEDLAM, AGORA, or SURREAL, and no code or data released. The pseudo-ground-truth from SMPLer-X is load-bearing: if it systematically misestimates sports-specific articulation, bulky gear, or occluded joints, that bias is invisible in synthetic testing but would transfer to real data. The paper's own Future Work concedes motion jitter and physical plausibility are not optimized, which further tempers the 'photorealistic and diverse' claim.\n\nHaving said that, these are addressable weaknesses. The pipeline description is clear, the dataset is large and thoughtfully annotated, and the paper does not oversell—it explicitly notes the accuracy of extracted motion does not need to match the original video, which is a fair statement of the self-consistency approach. No critical mathematical error appears in the equations; the free parameters are standard weighting terms.\n\nWho gets value: anyone working on synthetic data for human-centric vision, especially sports. This paper deserves a serious referee, but it needs substantial revision before acceptance: release the dataset and code, add a real-data transfer experiment (e.g., AthletePose3D or SportsPose), compare against existing synthetic datasets, and ideally run a small motion-capture check on the pseudo-ground-truth. Without those, the dataset is a promising resource whose utility is asserted rather than demonstrated.","headline":"A sensible synthetic-data pipeline and a large new sports dataset, but the experiments only test self-consistency, not real-world transfer.","tokens_in":11575,"tokens_out":1608,"would_cite":false,"duration_ms":20586,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a fully automated pipeline can generate diverse, photorealistic 4D human animation datasets from text prompts and internet videos, demonstrated by SportPAL, a 583,403-frame synthetic sports dataset.","keywords":["4D human animation","Gaussian splatting","synthetic data","human pose estimation","text-to-avatar","diffusion models","sports dataset","SMPLX"],"falsifier":"Record a set of expert sports movements with both the video-based pose estimator used by Gen4D and a synchronized marker-based motion-capture rig; if the recovered poses differ systematically on fast articulations or bulky gear, and a pose estimator trained on SportPAL inherits those errors when tested on real footage, the paper's claim of accurate image-pose pairs would be contradicted.","tokens_in":10537,"feed_emoji":"🏃","tokens_out":7795,"duration_ms":79406,"temperature":0.7,"pith_summary":"Gen4D aims to remove the manual 3D modeling bottleneck from synthetic human data. It claims that text prompts, a pretrained pose estimator, and internet videos are enough to automatically produce diverse, photorealistic 4D human animations with aligned pose annotations. The paper builds SportPAL, a 583,403-frame synthetic sports dataset across baseball, ice hockey, and soccer, and shows that pose estimators trained on it transfer across sports. If the claim holds, large-scale training data for uncommon human-centric vision tasks can be manufactured on demand, without motion-capture studios or asset libraries.","feed_headline":"Auto pipeline yields 583K photorealistic sports frames from prompts","feed_subtitle":"Prompt-guided avatars and internet video motion replace manual 3D modeling, yielding a half-million-frame sports pose library.","key_machinery":"The load-bearing construction is the fully automated motion-to-scene pipeline. Motion extraction uses a pretrained human mesh recovery method to get SMPLX body parameters from internet videos; canonical avatars are optimized as 3D Gaussians supervised by a text-conditioned diffusion model through a dual-branch Score Distillation Sampling loss; scene composition deforms the Gaussians along the motion, rasterizes them from randomized viewpoints, and inpaints backgrounds with an illumination-consistent diffusion backbone that enforces physically grounded light transport for shadows and lighting. The dataset is the material expression: SportPAL, with 50 subjects, 2,012 clips, and 583,403 frames.","core_discovery":"The central claim is that existing synthetic human datasets are limited by rigid asset libraries and hand-crafted rendering pipelines, and that Gen4D overcomes this by chaining three automated stages: extracting SMPLX pose and shape sequences from internet videos of expert demonstrations; synthesizing varied canonical human avatars by optimizing Gaussian splatting parameters under a dual-branch Score Distillation Sampling loss conditioned on text prompts; and deforming, rasterizing, and compositing those avatars into diffusion-generated, illumination-consistent backgrounds. The paper asserts this produces accurate image-pose pairs even when the initial pose estimates do not perfectly match the source video, because the estimated poses become the ground truth that drives rendering. SportPAL then provides 2D and 3D keypoints, SMPLX parameters, segmentation masks, bounding boxes, and action labels for over half a million frames, and experiments show a token-based pose estimator trained on the dataset generalizes across baseball, ice hockey, and soccer.","pith_inferences":["A natural extension, not argued in the paper, is that the three pipeline stages are modular enough to be upgraded independently: better motion sources or physics-constrained refinement could improve dataset realism without regenerating any avatars.","The dataset's self-consistency is guaranteed by construction, so its largest risk is not annotation noise but distribution shift from real sports imagery; a direct test would be to train on SportPAL and evaluate on real sports videos with held-out camera angles and athlete builds.","One testable extension is to run the same pipeline on motion-capture-verified sequences and compare downstream pose accuracy, isolating how much of the gain comes from pose fidelity versus image diversity."],"forward_implications":["If the pipeline works as claimed, generating a new synthetic human dataset reduces to writing prompts and collecting videos, so sports or niche activities with scarce real data can get large training sets quickly.","Because every frame is rendered rather than recorded, annotations such as 2D and 3D poses, SMPLX parameters, segmentation masks, bounding boxes, and action labels come for free and are perfectly aligned with the image.","Pose estimators pretrained on one sport can be fine-tuned on another with measured gains, so the data's diversity aids cross-domain transfer.","The same framework can be pointed at other in-the-wild human activities beyond the three demonstrated sports, since the motion and scene stages are not sport-specific."],"supporting_citations":[{"why":"Supplies the per-frame SMPLX pose and shape estimates from internet videos that become the pseudo ground truth animating the avatars.","marker":"[10]"},{"why":"Provides the text-conditioned Gaussian avatar generation via dual-branch Score Distillation Sampling and the deformation setup used in Gen4D.","marker":"[31]"},{"why":"Provides the background diffusion backbone whose light-transport consistency gives the composited scenes realistic illumination and shadows.","marker":"[55]"},{"why":"Defines the SMPL/SMPLX body model that supplies the canonical surface on which Gaussians are initialized and from which joints are regressed.","marker":"[32]"},{"why":"The transformer-based pose estimator trained and evaluated on SportPAL in the cross-sport fine-tuning experiments.","marker":"[29]"},{"why":"The DETR-based transformer used as the baseline pose estimator in the SportPAL experiments.","marker":"[12]"}],"fun_headline_variants":["Gen4D auto-generates 583K photorealistic sports frames","Prompt-guided pipeline yields diverse 4D human animations","Synthetic sports dataset: half a million frames from prompts","Gen4D: automated creation of diverse sports scenes","From video motion and prompts to 583K sports frames"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that per-frame body pose and shape estimates pulled from internet videos are reliable enough, after normalization, to serve as the ground truth that animates synthetic avatars, even for fast, occluded, or gear-heavy sports movements.","fun_headline_variants_meta":{"raw":{"variants":["Gen4D auto-generates 583K photorealistic sports frames","Prompt-guided pipeline yields diverse 4D human animations","Synthetic sports dataset: half a million frames from prompts","Gen4D: automated creation of diverse sports scenes","From video motion and prompts to 583K sports frames"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000789,"raw_usage":{"total_tokens":3471,"prompt_tokens":929,"completion_tokens":2542,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":2470}},"tokens_in":545,"tokens_out":2542,"duration_ms":18907,"temperature":1.0,"reasoning_tokens":2470,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:04:34.776724+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record a set of expert sports movements with both the video-based pose estimator used by Gen4D and a synchronized marker-based motion-capture rig; if the recovered poses differ systematically on fast articulations or bulky gear, and a pose estimator trained on SportPAL inherits those errors when tested on real footage, the paper's claim of accurate image-pose pairs would be contradicted.","supporting_citations":[{"cited_title":"SMPLer-X: Scaling up expressive human pose and shape estimation","cited_arxiv_id":null,"evidence_quote":"Supplies the per-frame SMPLX pose and shape estimates from internet videos that become the pseudo ground truth animating the avatars."},{"cited_title":"Humangaus- sian: Text-driven 3d human generation with gaussian splat- ting","cited_arxiv_id":null,"evidence_quote":"Provides the text-conditioned Gaussian avatar generation via dual-branch Score Distillation Sampling and the deformation setup used in Gen4D."},{"cited_title":"Ic-light github page, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the background diffusion backbone whose light-transport consistency gives the composited scenes realistic illumination and shadows."},{"cited_title":"Black.SMPL: A Skinned Multi- Person Linear Model","cited_arxiv_id":null,"evidence_quote":"Defines the SMPL/SMPLX body model that supplies the canonical surface on which Gaussians are initialized and from which joints are regressed."},{"cited_title":"End-to- end object detection with transformers","cited_arxiv_id":null,"evidence_quote":"The DETR-based transformer used as the baseline pose estimator in the SportPAL experiments."}],"review_version":1}