{"id":"6bdeab1e-d265-45d8-8424-02824d543570","arxiv_id":"2412.14444","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"GenHMR applies masked generative token prediction and 2D-pose-guided latent refinement to monocular human mesh recovery, reporting state-of-the-art MPJPE on Human3.6M, 3DPW, and EMDB.","lead":"This paper introduces GenHMR, a generative model that reconstructs 3D human body meshes from single images by converting body poses into discrete tokens and predicting them with a masked transformer. It reports large error reductions on standard benchmarks, aided by a test-time refinement step that aligns the 3D result to 2D keypoints.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline gains are dominated by a test-time 2D-pose optimization that baselines do not receive; without optimization-augmented baselines, the SOTA claim is not yet established.","rationale":"The reader's weakest_assumption focuses on the accuracy of 2D detections and the simplified camera model inside the refinement step. My concern is broader: even if the 2D detections and camera model are adequate, the comparison is unfair because GenHMR receives a test-time optimization that the baselines do not. This is explicitly acknowledged in the reader's rationale ('the comparison is not fully controlled because the method uses a test-time 2D pose-guided optimization that the deterministic baselines do not receive'), so there is partial agreement. The paper's own ablation (Table 3) shows that the refinement is responsible for a large portion of the final improvement, making the uncontrolled comparison the most load-bearing threat to the central claim. The concrete test—applying the same optimization to baselines—would settle whether the claimed SOTA advantage is genuine or an artifact of the test-time procedure. I keep the verdict at CONDITIONAL rather than REJECT because the generative model alone (UGS only) still outperforms HMR2.0 on 3DPW and EMDB, so the method has plausible merit, but the magnitude of the claimed improvement is not yet substantiated without controlled baselines and code release.","tokens_in":22113,"tokens_out":5289,"duration_ms":40894,"concrete_test":"Re-run HMR2.0 and TokenHMR with the identical 2D pose-guided refinement on 3DPW and EMDB: optimize each baseline's output SMPL pose parameters (θ) against OpenPose 2D keypoints using the same simplified camera model, regularization strength, and 20 refinement iterations, then report MPJPE/MVE. If the refined baselines achieve errors within a few mm of GenHMR's, the claimed advantage is largely an artifact of the test-time optimization rather than the generative framework.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim of '20-30% error reduction over SOTA' is not supported by a controlled comparison. GenHMR's final numbers are obtained after a test-time 2D pose-guided refinement (Table 3) that optimizes pose tokens to match OpenPose 2D keypoints. This optimization is applied only to GenHMR; the deterministic baselines (HMR2.0, TokenHMR) are evaluated without any equivalent test-time fitting. Table 3 shows that on 3DPW, the refinement reduces MPJPE from 68.1 mm (UGS only) to 54.7 mm (20 iterations), a 19.7% drop; on EMDB, from 88.2 to 68.5, a 22.3% drop. These gains are comparable to or larger than the headline differences over the baselines. If the same 2D-guidance optimization were applied to baseline outputs (e.g., directly optimizing their predicted SMPL pose parameters against the same OpenPose keypoints), it is plausible that their errors would decrease substantially, potentially closing the gap. The paper does not report such optimization-augmented baselines, nor does it provide code to test this. Additionally, the simplified perspective camera (identity rotation, fixed focal length; Appendix 'Camera Model') may limit the accuracy of the reprojection loss, and the paper does not specify the regularization weight λθ' or the number of refinement iterations used for the final Table 1 numbers. Without controlling for the test-time optimization and extra 2D information, the claim that the generative framework itself outperforms SOTA is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GenHMR, a two-stage generative framework for monocular 3D human mesh recovery. In the first stage, a VQ-VAE pose tokenizer converts SMPL pose parameters into discrete tokens. In the second stage, an image-conditioned masked transformer with multi-scale deformable cross-attention learns to predict randomly masked pose tokens from image features. At inference, uncertainty-guided iterative sampling progressively decodes high-confidence tokens, followed by a 2D pose-guided refinement that optimizes the latent pose embeddings to align the projected mesh with OpenPose 2D keypoints. The authors report state-of-the-art results on Human3.6M, 3DPW, and EMDB, with 20-30% MPJPE reductions over prior methods, and provide extensive ablations in the appendix.","tokens_in":22483,"tokens_out":7414,"duration_ms":55527,"significance":"If the controlled comparison can be established, GenHMR would be a noteworthy contribution: it demonstrates that masked generative modeling of tokenized poses can match or beat deterministic regression, and the latent-space refinement is a novel mechanism for injecting 2D evidence at test time. The appendix is unusually thorough in covering tokenizer design, codebook size, token counts, temperature schedules, masking functions, feature resolutions, and loss ablations, which is a clear strength. However, the headline claim is currently confounded by the test-time 2D optimization that is not applied to baseline methods, and the absence of code, seeds/error bars, and precise inference settings limits verification.","major_comments":[{"comment":"The headline comparison is not controlled because the final GenHMR numbers in Table 1 are produced after test-time refinement against OpenPose 2D keypoints, whereas the deterministic baselines (HMR2.0, TokenHMR) are evaluated without any equivalent test-time optimization. Table 3 shows that this refinement alone reduces 3DPW MPJPE from 68.1 to 54.7 mm (19.7%) and EMDB MPJPE from 88.2 to 68.5 mm (22.3%), which are of the same magnitude as the reported gains over the baselines. To support the SOTA claim, the authors should either add optimization-augmented baselines (for example, running the same 2D reprojection optimization on HMR2.0 and TokenHMR SMPL outputs) or explicitly separate the contribution of the generative model from the contribution of the test-time 2D optimization.","section":"Comparison to SOTA (Table 1) and 2D Pose-Guided Refinement (Table 3)"},{"comment":"The optimization in Eq. (4) updates continuous pose embeddings Yp, but the pose tokenizer was trained with discrete codebook entries and a straight-through gradient estimator. The paper does not specify how the refined continuous embeddings are mapped to SMPL pose parameters θ' used in the reprojection loss L2D and regularization Lθ', nor whether the final result is quantized back into the codebook. Because this refinement stage is central to the reported accuracy, the authors should describe the exact decoding path from refined embeddings to SMPL parameters and verify that the optimization respects the discrete latent structure.","section":"2D Pose-Guided Refinement (Eqs. 2-4)"},{"comment":"The training protocol for the main results in Table 1 is unclear. Appendix Table 7 reports two GenHMR variants, SD+ITW and SD+ITW+BL, with 3DPW MPJPE values of 58.6 and 52.1 mm and EMDB MPJPE values of 74.6 and 67.5 mm, neither of which matches the 54.7 and 68.5 mm reported in Table 1. The paper should state which training set, backbone, and inference settings were used in Table 1 and explain the discrepancy with Table 7. Additionally, no multiple-seed statistics or error bars are reported, so it is unclear whether the observed differences over baselines are within run-to-run variance.","section":"Experiments (Table 1) and Appendix (Table 7)"},{"comment":"The exact inference configuration for the headline numbers is not specified in the main text. The main text states that 5-10 refinement iterations are sufficient, but the values in Table 1 correspond to the 20-iteration row of Table 3; the regularization weight λθ′ in Eq. (2) is never assigned a value; and the camera model in the Appendix fixes focal length and rotation to identity with no sensitivity analysis. Please report the exact number of refinement iterations, the value of λθ′, and the camera parameters used in Table 1, and include an ablation showing how sensitive the final numbers are to these choices.","section":"Implementation Details and Camera Model (Appendix)"}],"minor_comments":[{"comment":"There is a typo in the sentence '5 to 10 iterations is sufficient to yield satisfaroy enhancement' — 'satisfaroy' should be 'satisfactory'.","section":"2D Pose-Guided Refinement"},{"comment":"The sequence length L and codebook size are introduced in the method section but their values (96 tokens, 2048 x 256 codebook) appear only in the appendix implementation details; please define these symbols in the main text.","section":"Method (Pose Tokenizer)"},{"comment":"The AITI metric is defined as average inference time per image, but it is unclear whether it includes the time of the 2D pose detector (OpenPose) used in the refinement stage; please clarify what is included in the reported AITI values.","section":"Evaluation Metrics"},{"comment":"The main text describes 'stochastic sampling' of tokens from their prediction distributions, while the appendix specifies 'greedy sampling with top-k = 1'; these descriptions should be reconciled because top-k = 1 with argmax is deterministic.","section":"Uncertainty-Guided Sampling"},{"comment":"In the row with only L2D and β, the reported MPJPE is 1160.6 mm and MVE is 1167.7 mm, which are orders of magnitude larger than all other rows; please double-check these values for a possible typo (for example, a decimal point or unit error).","section":"Appendix, Impact of Losses (Table 10)"},{"comment":"The H36M columns in Table 3 report only MPJPE and not MVE, while 3DPW and EMDB report both; for consistency, please include MVE for H36M as well.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer vision venue. The core concern is the controlled comparison: the test-time 2D pose-guided refinement is applied only to GenHMR, so the reported SOTA advantage may not be attributable to the generative framework. This is fixable with additional baseline experiments, but the authors should be pushed to provide them. The absence of code and of any multiple-seed statistics is also a concern for the current review culture. The paper's extensive appendix is a genuine strength and should be acknowledged in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a serious generative HMR paper with unusually thorough ablations, but the 20-30% SOTA claim is not yet controlled, because the final numbers include a test-time 2D refinement that the baselines don't receive.\n\nWhat's actually new: the combination of VQ-VAE pose tokenization, a MaskGIT-style image-conditioned masked transformer, and latent-space 2D-guided refinement into one HMR pipeline, plus uncertainty-guided iterative decoding that re-masks low-confidence tokens. The components are known, but the composition and inference strategy are new. The paper ships extensive ablations—tokenizer codebook size and token count, masking schedules, top-k, Gumbel temperature, feature scales, loss ablations—and reports consistent gains on H36M, 3DPW, and EMDB. It even includes failure cases and a limitations paragraph. That is more than most HMR papers do.\n\nThe soft spot is the one the stress test flags, and I think it lands. Table 3 shows 2D pose-guided refinement is responsible for a 19.7% MPJPE drop on 3DPW (68.1 to 54.7) and a 22.3% drop on EMDB (88.2 to 68.5). The headline comparisons in Table 1 use the refined numbers, while HMR2.0 and TokenHMR are evaluated without any equivalent test-time fitting. Since the refinement uses OpenPose keypoints and a simplified camera, it is reasonable to expect those baselines would also improve from the same optimization. Without optimization-augmented baselines, the claim that the generative framework itself beats SOTA is not established. That is a load-bearing gap, not a nitpick.\n\nOther issues are smaller: no code release, no error bars or multiple seeds, some typos in Table 10, and the regularization weight lambda_theta' and the number of refinement iterations used for Table 1 are not specified. The camera model (identity rotation, fixed focal length) is a deliberate simplification and is acknowledged; fine, but the refinement's robustness to noisy 2D detections is untested. The circularity concern from the skeptic does not bother me: the refinement uses 2D observations, not the 3D target, so it is not a circular argument, just an underspecified comparison.\n\nWho this is for: people working on HMR and generative pose modeling. It deserves a serious referee. My recommendation: send it to review, but make the reviewers' main job verifying the decomposition of gains, and require code or at least optimization-augmented baselines before accepting.","headline":"Good generative HMR paper with unusually thorough ablations, but the headline 20-30% SOTA gain is not yet controlled because the final numbers include a test-time 2D refinement that the deterministic baselines do not receive.","tokens_in":22999,"tokens_out":2142,"would_cite":false,"duration_ms":18111,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GenHMR claims that monocular 3D human mesh recovery can be reformulated as an image-conditioned generative task over discrete pose tokens, reducing MPJPE by 20-30% against state-of-the-art methods on Human3.6M, 3DPW, and EMDB.","keywords":["human mesh recovery","monocular 3D pose estimation","masked generative transformer","discrete pose tokens","VQ-VAE","uncertainty-guided sampling","2D pose-guided refinement","SMPL"],"falsifier":"On a test set with ground-truth 3D meshes, add increasing amounts of noise to the OpenPose 2D keypoints used by the refinement stage; if MPJPE after refinement does not degrade monotonically with keypoint noise, or if a single corrupted keypoint flips the reconstruction, the claim that 2D-guided refinement drives the reported gains is falsified.","tokens_in":21943,"feed_emoji":"🧍","tokens_out":8293,"duration_ms":60361,"temperature":0.7,"pith_summary":"GenHMR is a proposed answer to a standing problem in monocular human mesh recovery: a single 2D image carries depth ambiguity and occlusions, so a single deterministic prediction can be wrong even when the image is visually clear. The paper argues that the 2D-to-3D mapping should be modeled explicitly as a probabilistic image-conditioned generative task, and that the resulting uncertainty can then be reduced during inference by iterative sampling and by aligning the projected 3D mesh to 2D keypoint detections. Concretely, GenHMR quantizes SMPL pose parameters into discrete tokens with a vector-quantized variational autoencoder, trains an image-conditioned masked transformer to predict masked pose tokens, and at test time decodes high-confidence tokens while re-masking low-confidence ones, followed by a latent-space refinement that forces reprojected 3D joints to match 2D keypoints. The paper reports that this pipeline lowers Mean Per Joint Position Error by roughly 20-30% compared with state-of-the-art deterministic and probabilistic methods on Human3.6M, 3DPW, and EMDB. If these numbers hold, generative token modeling becomes a competitive alternative to deterministic regression for recovering human bodies from single images.","feed_headline":"Sampling pose tokens cuts 3D human mesh error by 20-30%","feed_subtitle":"GenHMR treats a single image's 3D pose as a masked generative puzzle, then aligns the mesh to 2D keypoints to sharpen the result.","key_machinery":"The load-bearing mechanism is a two-stage generative pipeline. First, a vector-quantized variational autoencoder (VQ-VAE) based pose tokenizer maps SMPL pose parameters $\\theta \\in \\mathbb{R}^{24\\times 3}$ into a discrete sequence of 96 tokens from a $2048\\times 256$ codebook, giving the model a finite latent vocabulary of body poses. Second, an image-conditioned masked transformer with multi-scale deformable cross-attention learns $p(y_i \\mid Y_M, X)$, the categorical distribution of each pose token given the image prompt $X$ and the currently unmasked token sequence $Y_M$; training uses random cosine masking and Gumbel-Softmax differentiable sampling so that SMPL-space, 3D-joint, and 2D-projection losses can flow through the discrete tokens. At inference, uncertainty-guided sampling re-masks low-confidence tokens across iterations, and then 2D pose-guided refinement directly updates the pose token embeddings by gradient descent on $|\\Pi(K(J'_{3D})) - J_{2D}|^2$ plus a pose-regularization term, keeping the whole network frozen.","core_discovery":"The central claim is that treating monocular human mesh recovery as an image-conditioned generation problem over discrete pose tokens, rather than as a one-shot regression of SMPL parameters, lets a model explicitly capture 2D-to-3D ambiguity and then remove most of it at inference. The pose tokenizer converts SMPL pose parameters into a sequence of 96 discrete tokens drawn from a 2048-entry codebook, and the masked transformer learns the conditional distribution of each token given the image and the currently unmasked tokens. At inference, uncertainty-guided sampling iteratively predicts tokens, keeps the confident ones, and re-masks the rest, after which a 2D pose-guided refinement step optimizes the token embeddings in latent space so that the projected 3D joints match OpenPose 2D detections while the pose stays near its initial estimate. The paper reports state-of-the-art results on Human3.6M, 3DPW, and EMDB; for example, on 3DPW MPJPE drops to 54.7 mm versus 70.0 mm for HMR2.0, and on EMDB MPJPE drops to 68.5 mm versus 97.8 mm for HMR2.0.","pith_inferences":["The largest part of the accuracy gain is bought by optimization against external 2D keypoints, so GenHMR's margin over deterministic baselines may shrink whenever OpenPose detections are noisy, truncated, or systematically biased; this dependency is not stress-tested in the paper.","The simplified camera model used in refinement (identity rotation and a fixed focal length) is a likely ceiling: images with strong perspective or unusual camera intrinsics could be served better by estimating per-image focal length, a variant the paper does not explore.","The per-token confidence scores produced during uncertainty-guided sampling could be exported as an ambiguity or occlusion detector, which would make the method useful beyond reconstruction accuracy; this application is not claimed by the authors."],"forward_implications":["If the reported benchmark numbers hold, GenHMR would set a new state of the art on Human3.6M, 3DPW, and EMDB simultaneously, with the largest gains on the in-the-wild EMDB dataset.","The ablation in Table 3 shows that the 2D pose-guided refinement stage, not the generative prior alone, is responsible for most of the final accuracy: on 3DPW, MPJPE falls from 68.1 mm after uncertainty-guided sampling to 54.7 mm after 20 refinement iterations.","Because the method is evaluated on 3DPW and EMDB without training on them, the paper's claim of generalizing to unseen in-the-wild data is part of the contribution.","The reported 20-30% MPJPE reductions relative to prior methods, if reproducible, would make probabilistic or generative human mesh recovery competitive with deterministic regression, a gap the paper identifies as previously open."],"supporting_citations":[{"why":"Defines the SMPL parametric body model that maps pose and shape parameters to a mesh, which the pose tokenizer quantizes and the refinement stage reprojects.","marker":"Loper et al. 2015"},{"why":"Supplies the VQ-VAE discrete latent representation method used to train the pose tokenizer.","marker":"Van Den Oord, Vinyals et al. 2017"},{"why":"Provides the masked generative transformer paradigm with cosine masking and iterative decoding that GenHMR adapts for pose tokens.","marker":"Chang et al. 2022"},{"why":"HMR2.0 is the main deterministic state-of-the-art baseline that GenHMR compares against and reports beating.","marker":"Goel et al. 2023"},{"why":"TokenHMR supplies the tokenized pose representation baseline and is another state-of-the-art deterministic comparison.","marker":"Dwivedi et al. 2024"},{"why":"OpenPose is the off-the-shelf 2D keypoint detector whose detections drive the 2D pose-guided refinement objective.","marker":"Cao et al. 2017"},{"why":"AMASS is the motion-capture dataset used to pretrain the pose tokenizer, together with MOYO.","marker":"Mahmood et al. 2019"},{"why":"Provides the Gumbel-Softmax technique that makes categorical token sampling differentiable during training, enabling SMPL, 3D, and 2D losses.","marker":"Jang, Gu, and Poole 2016"}],"fun_headline_variants":["Generative token sampling fixes depth ambiguity in human mesh recovery","GenHMR: Masked pose tokens beat deterministic HMR by 20-30%","Image-conditioned masked transformer sharpens 3D pose from single images","Sampling and refining pose tokens reduces mesh error on 3DPW and EMDB","Turn 2D ambiguity into a generative puzzle: GenHMR improves HMR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The refinement stage assumes that the 2D keypoints from the off-the-shelf detector are accurate and that a simplified perspective camera with identity rotation and a fixed focal length is adequate for reprojection, and this is the premise on which the headline error reductions rest.","fun_headline_variants_meta":{"raw":{"variants":["Generative token sampling fixes depth ambiguity in human mesh recovery","GenHMR: Masked pose tokens beat deterministic HMR by 20-30%","Image-conditioned masked transformer sharpens 3D pose from single images","Sampling and refining pose tokens reduces mesh error on 3DPW and EMDB","Turn 2D ambiguity into a generative puzzle: GenHMR improves HMR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000228,"raw_usage":{"total_tokens":1558,"prompt_tokens":1112,"completion_tokens":446,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":728,"completion_tokens_details":{"reasoning_tokens":344}},"tokens_in":728,"tokens_out":446,"duration_ms":3732,"temperature":1.0,"reasoning_tokens":344,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:14:01.831180+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a test set with ground-truth 3D meshes, add increasing amounts of noise to the OpenPose 2D keypoints used by the refinement stage; if MPJPE after refinement does not degrade monotonically with keypoint noise, or if a single corrupted keypoint flips the reconstruction, the claim that 2D-guided refinement drives the reported gains is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the SMPL parametric body model that maps pose and shape parameters to a mesh, which the pose tokenizer quantizes and the refinement stage reprojects."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HMR2.0 is the main deterministic state-of-the-art baseline that GenHMR compares against and reports beating."},{"cited_title":"K.; Sun, Y.; Patel, P.; Feng, Y.; and Black, M","cited_arxiv_id":null,"evidence_quote":"TokenHMR supplies the tokenized pose representation baseline and is another state-of-the-art deterministic comparison."},{"cited_title":"F.; Pons-Moll, G.; and Black, M","cited_arxiv_id":null,"evidence_quote":"AMASS is the motion-capture dataset used to pretrain the pose tokenizer, together with MOYO."}],"review_version":1}