{"id":"a47c5350-2262-4164-bc14-f249df0fdf71","arxiv_id":"2505.13437","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FinePhys adds a learned PhysNet that re-estimates 3D skeletons through an Euler-Lagrange parameterization and uses the fused skeletons as 2D heatmap guidance for a latent diffusion model, reporting improved generation on FineGym FX-JUMP and FX-TURN.","lead":"This paper presents a video-generation system that uses 2D pose, 3D lifting, and learned Euler-Lagrange-style dynamics to guide diffusion models when generating gymnastic actions. The authors report large gains over existing text-to-video baselines on two fine-grained gymnastic subsets, with the hardest salto subset left out of the main quantitative table.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PhysNet's Euler-Lagrange parameters are unconstrained learned functions of position only, so Eq. 16 is a learned acceleration map rather than an enforced physical law; the reported gains may stem from extra capacity or smoothing, not from physics.","rationale":"The central assertion is a causal one: FinePhys outperforms baselines because PhysNet injects Euler-Lagrange/Newtonian physics into skeletal guidance. That causal attribution is the paper's claimed novelty, and it is exactly where the support is weakest. The implementation estimates all physical terms (M^{-1}, J, C, N) with networks trained only against pose reconstruction losses, and the equation is then integrated as a smoother. With no constraints tying these terms to mass, inertia, gravity, or measured forces, Eq. 16 is formally equivalent to a generic learned map from position features to accelerations; the EL notation is an interpretation, not an enforcement. A matched-capacity control (an MLP producing qddot from the same features with the same integration) would distinguish genuine physical inductive bias from added model capacity. This is load-bearing because if the EL structure is not the cause, the paper's contribution reduces to another conditioned video generator with a pose-denoising auxiliary branch, and the physical-plausibility claims in the abstract, Section 3, and conclusion are unsupported. I therefore agree with the reader's weakest_assumption. The abstract's claim over three subsets when Table 1 reports two, and the absence of a described train/test split, are additional validity concerns, but they are secondary: a split can be supplied post hoc and the overclaim is limited to presentation. If the PhysNet concern lands, the main scientific claim of explicitly incorporating physical laws fails regardless of split details. The existing conditional verdict remains appropriate: the paper should be revised to provide the matched-ablation control, physical consistency checks, and code/data release, or soften the physics claim.","tokens_in":25278,"tokens_out":8002,"duration_ms":82708,"concrete_test":"Retrain FinePhys with PhysNet replaced by a matched-capacity MLP that takes the same global/local temporal features and produces qddot directly, then integrates bidirectionally via Eq. 17 with identical losses and training schedule. Repeat the Tab. 1 and Tab. 2 comparisons. If the MLP matches FinePhys within statistical error, the Euler-Lagrange parameterization is not load-bearing; if PhysNet clearly wins, inspect whether the learned (Mhat^{-1}+Nhat) is positive-definite and whether its qddot matches finite-difference accelerations on held-out ground-truth MoCap data as an independent check of physical consistency.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.3, the physical content of PhysNet is entirely in learned estimators: Jhat_t=E_J(q_t), Chat_t=E_C(q_t), Mhat^{-1}_t=S(E_M(q_t)), and Nhat_t=G(E_N(q_t)) (Eqs. 10-16). These are trained only with pose reconstruction losses (Eqs. 19-20). No physical units, biomechanical ranges, gravity, or positivity constraints are imposed; only symmetry of Mhat^{-1} is assumed, and the advertised dependence on qdot in Eq. 9 is absent from the estimator inputs, which take q_t and not velocities. Thus qddot_t = (Mhat^{-1}_t + Nhat_t)(Jhat_t - Chat_t) is an unconstrained stochastic function of q_t, and Eq. 17 merely adds second-order temporal smoothing. The system is not forced to satisfy Euler-Lagrange or Newtonian dynamics, so the central causal claim, that gains are due to explicit physics, is not established. The ablation in Tab. 2 (\"Sdd+SMLP\") uses an unspecified simple MLP and does not match PhysNet's recurrent bidirectional integration or parameter count, so it cannot isolate the EL inductive bias. Since no code or data are released, there is no external check on physical plausibility.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FinePhys, a framework for fine-grained human action video generation that incorporates physics-based skeletal guidance. The pipeline first detects 2D poses online, lifts them to 3D via an in-context learning module, then refines the 3D poses with a PhysNet module that instantiates Euler-Lagrange equations through learned estimators of generalized forces, joint constraints, inverse inertia, and a stochastic noise term. The data-driven and physics-predicted 3D skeletons are fused, projected to 2D, and converted into multi-scale heatmaps that guide a latent diffusion model (Stable Diffusion with AnimateDiff motion module and LoRA). Experiments are reported on three FineGym subsets (FX-JUMP, FX-TURN, FX-SALTO), with quantitative comparisons to several baselines using user studies, CLIP-SIM*, PickScore, and FVD. The paper also introduces CLIP-SIM*, a modified version of CLIP-SIM intended to better capture fine-grained action dynamics.","tokens_in":25562,"tokens_out":5078,"duration_ms":47689,"significance":"If validated, the idea of combining explicit physical laws with skeletal guidance for diffusion-based video generation would be a meaningful step toward controlling fine-grained human actions. The paper is honest about its limitations, explicitly stating in the Limitations section and supplementary material that salto actions remain intractable and that initial pose detection can fail completely; this transparency is a strength. The qualitative visualizations are thorough and suggest the method produces anatomically more plausible results than several baselines. However, the central claim that gains stem from explicit physics is not yet supported by the evidence: the PhysNet estimators are unconstrained learned functions, the ablation against a simple MLP does not match model capacity, and the quantitative evaluation rests on a self-defined metric (CLIP-SIM*) that may favor the proposed method. The omission of FX-SALTO results from the main table contradicts the abstract's claim of evaluation on three subsets.","major_comments":[{"comment":"The claim that PhysNet enforces Euler-Lagrange/Newtonian dynamics is not established. The estimators in Eqs. (10)-(16) depend only on q_t (position) and are trained solely with pose reconstruction losses (Eqs. 19-20); no physical units, positivity constraints, velocity dependence, or gravity terms are imposed. Consequently, Eq. (16) defines an unconstrained learned acceleration map, and Eq. (17) is a second-order smoothing update. The reported gains could therefore reflect added model capacity or temporal smoothing rather than physical laws. Please provide a matched-capacity learned baseline without the EL structure and include physical plausibility metrics (e.g., joint-angle limits, contact consistency, ground-reaction forces) to support the attribution to physics.","section":"Section 3.3, Eqs. (9)-(17)"},{"comment":"The abstract and Section 4.1 state that FinePhys is evaluated on three FineGym subsets, but Table 1 only reports results for FX-JUMP and FX-TURN. The Limitations section and supplementary Fig. 14 admit that salto actions are intractable and are not calculated in Table 1. The manuscript should either report FX-SALTO results (even if unsatisfactory) or explicitly revise the abstract and claims to reflect that only two subsets are evaluated. Additionally, no train/test split is described; please specify how videos are partitioned and how many videos per class are used for evaluation.","section":"Table 1 and Section 4.1"},{"comment":"The proposed CLIP-SIM* metric samples reference videos and frames from FineGym for domain and temporal consistency. Since FinePhys is fine-tuned on FineGym, these references are in-distribution for the proposed method, which may bias the metric in its favor. The paper should demonstrate that CLIP-SIM* is not biased (e.g., by reporting human correlation on a subset where baselines are known to be strong) and should provide confidence intervals or statistical significance tests for the reported gains. The user study also reports only mean MOS scores without variance or significance testing.","section":"Section 4.2, Eqs. (25)-(26) and Fig. 4"},{"comment":"The ablation 'Sdd+SMLP' replaces PhysNet with a 'simple MLP' of unspecified architecture. It does not match PhysNet's design (global and local heads, bidirectional forward/inverse updating, symmetric inverse inertia, noise sampling) or its parameter count, so this ablation cannot isolate the effect of the Euler-Lagrange inductive bias. Please provide a controlled comparison where the only difference is the EL structure, and report parameter counts. Furthermore, no code or data are released, which prevents external verification of the physical plausibility of the generated poses; please consider releasing code and data (or a data agreement for FineGym).","section":"Table 2"},{"comment":"The 2D evaluation on FineGym uses S2D_detect as the reference, yet the paper itself notes in Fig. 6 and Section 4.3 that online 2D pose detection is noisy and often fails for gymnastics. Evaluating against a noisy reference can mislead, as the proposed pipeline is designed to correct these very errors. Please also report errors against manually annotated keypoints on a small subset, or treat S2D_detect as pseudo-ground-truth with a clear disclaimer about its limitations.","section":"Section 4.3 and Table 2 (2D evaluation)"}],"minor_comments":[{"comment":"The notation Vout = F(Noise,D,S2D) uses D without a definition; please clarify what D denotes (likely the data-driven 3D skeletons or the input video).","section":"Section 3.2"},{"comment":"The table header uses the metric 'MPVPE' while the text defines 'MPJVE' (mean per-joint velocity error); please unify the acronym.","section":"Table 2"},{"comment":"The sentence 'This work was founded by' should read 'funded by'.","section":"Acknowledgments"},{"comment":"The notation V^Ref is used inconsistently between the equation and the surrounding text; please align the definition of the reference set.","section":"Section 4.2, Eq. (26)"},{"comment":"The caption states 'FinePhys demonstrates superior performance' but does not refer to the method description; consider referencing the proposed framework as described in Section 3.3 for clarity.","section":"Figure 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and challenging problem, and the qualitative results are promising. However, the central physics-attribution claim requires a matched-capacity ablation and a more convincing evaluation, and the discrepancy between the abstract's three-subset claim and the reported two-subset table should be resolved. I recommend inviting a revision that addresses these load-bearing issues; the current form is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real engineering contribution to a hard problem—text-to-video generation for fine-grained gymnastics—and the qualitative results are noticeably better than the pose-guided baselines. But the physics claim in the title and abstract does not survive contact with the equations. The paper is worth refereeing, but the authors need to either constrain and validate the physical parameters or reframe the contribution honestly.\n\nWhat's new: the combination of in-context 2D-to-3D lifting, a bidirectional second-order refinement module, and multi-scale skeleton heatmaps injected into an AnimateDiff-style backbone is not something I've seen in one system. On FX-JUMP and FX-TURN, FinePhys consistently beats Follow-Your-Pose, AnimateDiff, and others on the user study and on the proposed CLIP-SIM*. The pose restoration from noisy detections (Fig. 6) is convincing, and the paper's admission that FX-SALTO is intractable is a good sign of honesty—though it makes the abstract's 'three subsets' claim wrong.\n\nSoft spots, in descending order of severity. (1) The physics. Equations 10–16 estimate J, C, M^{-1}, and N with networks that take only q_t, not velocities, and are trained with pose reconstruction losses. Nothing in the architecture enforces Newtonian dynamics or even units. Eq. 16 is a learned acceleration map; Eq. 17 is second-order temporal smoothing. The SMLP ablation doesn't control for parameter count or the bidirectional structure, so it cannot isolate any physical inductive bias. This is the paper's load-bearing flaw. (2) Evaluation. No train/test split is described; the main table omits FX-SALTO; CLIP-SIM* is self-defined and not calibrated against human judgments; the user study draws on 20 participants and reports no variance or significance. (3) No code or data, so none of this is independently checkable.\n\nThe engineering improvements are probably real—the fusion of lifted 3D poses with a learned smoother may genuinely help a diffusion model produce more stable gymnastics videos. But the paper should stop calling it 'explicitly incorporating physical laws' unless the physical terms are constrained, measured, or falsified. I would send this to referees with a request for major revision: narrow the claims, justify the metric, and provide artifacts or a clear split.\n\nWho this is for: people working on controllable human video generation, especially with skeleton guidance. They'll find it a useful baseline and a cautionary tale about physics-inspired naming. I'd take it to a reading group, but I wouldn't cite it as evidence that Lagrangian mechanics improves video generation.","headline":"A capable engineering contribution to fine-grained human video generation, but the 'explicit physics' framing is mostly window dressing: PhysNet is a learned second-order smoother, not an enforced Euler-Lagrange system.","tokens_in":26137,"tokens_out":2226,"would_cite":false,"duration_ms":23858,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FinePhys claims that incorporating Euler-Lagrange physics into skeletal guidance markedly improves fine-grained human action video generation, producing more physically plausible gymnastics videos than pose-only or text-only baselines.","keywords":["fine-grained human action generation","physics-informed deep learning","Euler-Lagrange equations","skeletal guidance","diffusion models","video generation","2D-to-3D pose lifting","FineGym"],"falsifier":"Take the PhysNet module and replace the Euler-Lagrange estimator with a same-capacity unconstrained network that receives the same $q_t$ inputs and produces the same $q_{t+1}$ updates under identical losses, then rerun the Human3.6M and FineGym pose evaluations and the generation metrics; if the unconstrained variant matches or beats the physics-structured one, the Euler-Lagrange form is not the cause of the reported gains.","tokens_in":25044,"feed_emoji":"🤸","tokens_out":5206,"duration_ms":46002,"temperature":0.7,"pith_summary":"The paper sets out to show that fine-grained human action video generation—such as gymnastics leaps, turns, and saltos—fails when skeletal guidance is purely data-driven, because the estimated poses can be noisy and physically implausible. The authors propose FinePhys, which first lifts online-detected 2D poses to 3D via in-context learning, then re-estimates the 3D motion with a PhysNet module that instantiates the Euler-Lagrange equations of rigid-body dynamics. The physics-refined and data-driven skeletons are fused, projected back to 2D, and encoded as multi-scale heatmaps that guide a latent diffusion model. On three FineGym subsets the method reports better user-study ratings and higher CLIP-SIM* scores than competitive baselines. If the claim holds, physical equations can serve as a practical inductive bias for video generation, not just a post-hoc filter.","feed_headline":"Physics-pinned skeletons improve gymnastics video generation","feed_subtitle":"A diffusion model guided by Euler-Lagrange-consistent skeletons generates fine-grained gymnastics with fewer limb distortions.","key_machinery":"The load-bearing object is the PhysNet module plus the Euler-Lagrange equation of motion $M(q)\\ddot{q} = J(q,\\dot{q}) - C(q,\\dot{q})$. PhysNet's encoders predict the force vector, the constraint vector, a symmetric inverse inertia matrix (estimated via a triangular matrix, symmetrized, and perturbed with Gaussian noise), and then compute accelerations $\\ddot{q} = (\\hat{M}^{-1} + \\hat{N})(\\hat{J} - \\hat{C})$. A second-order central difference formula turns those accelerations into updated joint positions, applied both forward and backward in time, and the two directional estimates are averaged. The work this machinery does is to convert data-driven 3D poses into physics-constrained poses that are then fused with the original estimates to form the skeletal heatmap guidance for the diffusion model.","core_discovery":"The central claim is that explicitly encoding Lagrangian mechanics inside the pose-estimation branch is what makes fine-grained action generation work: a PhysNet module estimates the generalized forces $J$, constraint terms $C$, a symmetric inverse inertia matrix $M^{-1}$ (with a learned noise perturbation), and then computes joint accelerations $\\ddot{q} = (\\hat{M}^{-1} + \\hat{N})(\\hat{J} - \\hat{C})$, integrating bidirectionally with a central-difference ODE solver to produce physics-revised 3D skeletons. These are averaged with data-driven 3D skeletons, projected to 2D, and used as multi-scale heatmap conditioning for the denoising UNet. The paper argues that this physical re-estimation not only corrects distorted or missing online poses but also yields more natural movement dynamics than skeleton conditioning alone, and it reports consistent gains over baselines on FX-JUMP, FX-TURN, and FX-SALTO.","pith_inferences":["If the Euler-Lagrange parameterization is doing real work, then the symmetry constraint on the inverse inertia and the learned noise perturbation could transfer to other physics-conditioned generators, such as motion prediction or human mesh recovery, as a lightweight regularizer.","A sharper test of the physical claim would check whether the learned $\\ddot{q}$ values approximate independent finite-difference accelerations from motion-capture ground truth; the paper's pose-error ablations do not directly measure physical fidelity.","The improved CLIP-SIM* metric, which compares generated videos against multiple reference videos from the same action class, could generalize to other domains where temporal dynamics rather than frame smoothness define quality.","The framework's dependence on an online pose detector that sometimes outputs nothing suggests that detection robustness, not physics, is the next bottleneck; adding optical flow as an auxiliary cue could compensate."],"forward_implications":["Physically re-estimated skeletons reduce the impact of noisy online 2D pose detection, restoring missing or distorted joints for downstream generation.","Fusing data-driven and physics-predicted 3D poses, rather than using either alone, gives lower 2D and 3D pose error on both Human3.6M and FineGym evaluations.","Replacing PhysNet with a plain MLP degrades pose estimation, indicating that the Euler-Lagrange structure, not merely extra capacity, carries the reported benefit.","FinePhys outperforms pose-guided, depth-guided, and text-only baselines on user study and improved CLIP-SIM* metrics across FX-JUMP, FX-TURN, and FX-SALTO.","High-speed salto classes that combine large body rotation with rapid deformation remain intractable for the method, as the paper reports."],"supporting_citations":[{"why":"Supplies the multibody dynamics equation of motion $M(q)\\ddot{q} = J - C$ that PhysNet instantiates.","marker":"[42]"},{"why":"Justifies strong inductive biases in generative networks, used to motivate the symmetry and noise assumptions on the inverse inertia matrix.","marker":"[34]"},{"why":"Physics-guided diffusion baseline that corrects global artifacts; FinePhys positions its fine-grained joint-level physics against this approach.","marker":"[81]"},{"why":"Provides the FineGym dataset and its FX-JUMP, FX-TURN, and FX-SALTO subsets used for fine-tuning and evaluation.","marker":"[62]"},{"why":"The pose-guided video generation baseline whose skeletal heatmap encoder training recipe FinePhys adopts.","marker":"[46]"},{"why":"Supplies the AnimateDiff motion module and temporal attention backbone inside the 3D-UNet denoising architecture.","marker":"[23]"},{"why":"Provides the two-stream transformer used in the in-context 2D-to-3D lifting module.","marker":"[89]"},{"why":"AMASS, one of the 3D motion datasets used to pre-train the 2D-to-3D lifting and PhysNet modules.","marker":"[47]"},{"why":"Human3.6M, the other 3D motion dataset used for pre-training and for the 3D pose evaluation of the lifting and physics modules.","marker":"[32]"},{"why":"CLIP embeddings that underlie both the original CLIP-SIM metrics and the paper's improved CLIP-SIM* evaluation.","marker":"[53]"}],"fun_headline_variants":["Lagrangian physics steers realistic gymnastics video generation","Euler-Lagrange skeletons fix fine-grained action video quality","Physics-grounded skeletons boost fine-grained action generation","ODE-solved poses make gymnastics videos more plausible","Biomechanics via equations improves fine-grained action synthesis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Euler-Lagrange module only enforces physics if the learned neural networks actually estimate meaningful forces, constraints, and an inertia matrix; if they are just flexible curve-fitters trained against pose losses, then the 'physics' is a reparameterization and the method's gains may come from added model capacity.","fun_headline_variants_meta":{"raw":{"variants":["Lagrangian physics steers realistic gymnastics video generation","Euler-Lagrange skeletons fix fine-grained action video quality","Physics-grounded skeletons boost fine-grained action generation","ODE-solved poses make gymnastics videos more plausible","Biomechanics via equations improves fine-grained action synthesis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000628,"raw_usage":{"total_tokens":2910,"prompt_tokens":958,"completion_tokens":1952,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":1887}},"tokens_in":574,"tokens_out":1952,"duration_ms":13135,"temperature":1.0,"reasoning_tokens":1887,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:13:41.139982+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the PhysNet module and replace the Euler-Lagrange estimator with a same-capacity unconstrained network that receives the same $q_t$ inputs and produces the same $q_{t+1}$ updates under identical losses, then rerun the Human3.6M and FineGym pose evaluations and the generation metrics; if the unconstrained variant matches or beats the physics-structured one, the Euler-Lagrange form is not the cause of the reported gains.","supporting_citations":[{"cited_title":"A quick tutorial on multibody dynamics","cited_arxiv_id":null,"evidence_quote":"Supplies the multibody dynamics equation of motion $M(q)\\ddot{q} = J - C$ that PhysNet instantiates."},{"cited_title":"Generalization in diffusion models arises from geometry-adaptive harmonic representations","cited_arxiv_id":null,"evidence_quote":"Justifies strong inductive biases in generative networks, used to motivate the symmetry and noise assumptions on the inverse inertia matrix."},{"cited_title":"Finegym: A hierarchical video dataset for fine-grained action under- standing","cited_arxiv_id":null,"evidence_quote":"Provides the FineGym dataset and its FX-JUMP, FX-TURN, and FX-SALTO subsets used for fine-tuning and evaluation."},{"cited_title":"Follow your pose: Pose- guided text-to-video generation using pose-free videos","cited_arxiv_id":null,"evidence_quote":"The pose-guided video generation baseline whose skeletal heatmap encoder training recipe FinePhys adopts."},{"cited_title":"For each gymnastics move described in the labels below, write a detailed description as if explaining to someone who is unfamiliar with gymnastics","cited_arxiv_id":null,"evidence_quote":"Provides the two-stream transformer used in the in-context 2D-to-3D lifting module."},{"cited_title":"Amass: Archive of motion capture as surface shapes","cited_arxiv_id":null,"evidence_quote":"AMASS, one of the 3D motion datasets used to pre-train the 2D-to-3D lifting and PhysNet modules."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Human3.6M, the other 3D motion dataset used for pre-training and for the 3D pose evaluation of the lifting and physics modules."},{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"CLIP embeddings that underlie both the original CLIP-SIM metrics and the paper's improved CLIP-SIM* evaluation."}],"review_version":1}