{"id":"b03329b8-f4cf-4e66-8c1a-3bb4c8081d75","arxiv_id":"2412.02690","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A 2D-keypoint-conditioned diffusion model trained on a new 10M-image hand dataset enables controllable hand reposing, appearance transfer, novel view synthesis, and zero-shot hand video generation.","lead":"FoundHand is a diffusion model trained on 10 million hand images to generate and edit single or dual hands from 2D keypoint controls. It claims state-of-the-art hand repose, appearance transfer, novel views, and zero-shot hand video synthesis, and it introduces the FoundHand-10M dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gesture-transfer SOTA claim rests on a same-pose reconstruction protocol that does not measure pose transfer; Table 1 could be passed by copying the reference.","rationale":"The reader's verdict is CONDITIONAL and I agree with that overall assessment; the concern identified here does not change the verdict but sharpens why the condition is necessary. The reader's weakest_assumption focuses on whether 2D keypoints can encode viewpoint; that is a real ambiguity, and the paper's own NVS procedure (Section 4.3) uses a 3D hand estimator and explicit camera projection to create the target keypoints, so the 'no camera parameters' claim should be scoped. However, the single most load-bearing problem for the central 'state-of-the-art on six tasks' assertion is that the only quantitative gesture-transfer table does not measure gesture transfer: when target pose equals reference pose, high PSNR, SSIM, and LPIPS can be obtained by copying. This is not a minor reporting issue; it changes what Table 1 can support. The concern is addressable: replacing or augmenting the metric with a true pose-transfer evaluation on distinct target poses, plus error bars, would either substantiate or refute the claim. Because the method itself may still be sound and has qualitative support plus a downstream domain-transfer experiment, the appropriate verdict remains conditional acceptance pending this evidence.","tokens_in":17589,"tokens_out":6473,"duration_ms":71631,"concrete_test":"Construct a test set of reference-target pairs with target keypoints drawn from a different hand or subject, and where available use ground-truth target images from synchronized cameras. Generate outputs with FoundHand and the baselines, then compute target-pose fidelity using a 2D keypoint detector (e.g., MediaPipe) with PCK or OKS against the specified target keypoints, alongside the existing PSNR, SSIM, LPIPS, and FID metrics. Also include a trivial 'copy reference' control. If FoundHand's pose-fidelity advantage over the copy control is small, or its PCK/OKS is no better than baselines, the Table 1 SOTA claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is in the quantitative evaluation of gesture transfer (Section 5, Table 1). The paper defines the task as adopting a target pose different from the reference, but for 'quantitative evaluation without ground truth targets' it sets the target pose equal to the reference pose and computes PSNR, SSIM, LPIPS, and FID between the generated image and the reference. Under this protocol, a model that simply copies or heavily autoencodes the reference image can achieve near-perfect scores without performing any gesture transfer. It is therefore not evidence for the claimed state-of-the-art pose-transfer quality; no metric on genuinely different target poses (e.g., keypoint PCK or OKS against the requested target keypoints) is reported. Gesture transfer is one of the core capabilities and one of the six tasks supporting the SOTA claim, so the central claim is not quantitatively supported until this is fixed. A secondary issue is that the 'no explicit camera parameters' claim in Section 4.3 is qualified by the NVS pipeline, which explicitly estimates 3D joints and projects them into target cameras using assumed camera parameters.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FoundHand-10M, a dataset of over 10M hand images assembled from 12 existing video and multi-view datasets, with automatic MediaPipe 2D keypoints and SAM segmentation masks. It then presents FoundHand, a latent diffusion transformer that performs image-to-image translation conditioned on 2D keypoint heatmaps, an optional reference image, and a binary flag indicating whether the input pair is a temporal pose change or a synchronized viewpoint change. The claimed core capabilities are gesture transfer, domain transfer, and novel view synthesis, with zero-shot applications for fixing malformed hands, synthesizing motion-controlled video, and generating hand-object interaction video. The paper reports state-of-the-art quantitative results across six hand-related tasks and emphasizes that 2D keypoints encode both articulation and viewpoint, avoiding explicit camera parameters.","tokens_in":17691,"tokens_out":4744,"duration_ms":49332,"significance":"If the central claims are substantiated, the contribution is significant: a large unified hand dataset with a common annotation convention, a controllable two-frame diffusion formulation that avoids full video training, and evidence that 2D-keypoint conditioning can drive pose and appearance changes across diverse in-the-wild inputs. The strengths include the scale of the dataset (10M images from 12 sources), the tractable two-frame 3D self-attention design, and the breadth of qualitative demonstrations from photorealistic images to artistic styles and hand-object interactions. However, the quantitative support for the state-of-the-art claim is thin in several places, and the load-bearing evaluation of gesture transfer and the camera-parameter claim need revision before the results can be considered established.","major_comments":[{"comment":"The quantitative protocol for gesture transfer sets the target pose equal to the reference pose and reports PSNR/SSIM/LPIPS/FID between the generated image and the reference. This measures pose-preserving reconstruction—essentially an identity or autoencoding test—rather than the ability to adopt a different target pose, and a model that copies the reference can score perfectly. Since gesture transfer is a core capability and one of the six tasks supporting the SOTA claim, the comparison needs a metric on genuinely different target poses, such as PCK/OKS of the generated keypoints against the requested target keypoints, or paired data with ground-truth target images.","section":"Section 5, Table 1"},{"comment":"The text reports FID \"between reference and generated images.\" FID is defined for distributions, not image pairs; without a precise description of which image sets form the two distributions and how the same-pose reconstruction setup is used, the FID numbers cannot be interpreted or compared across methods. The authors should specify the reference and generated distributions, sample sizes, and whether the metric is computed across the full test set or on a per-image basis.","section":"Section 5, Table 1"},{"comment":"The claim that 2D keypoints \"eliminate the need for explicit camera parameters during training or inference\" contradicts the described pipeline, which explicitly assumes a camera projection K to lift reference keypoints to 3D joints J via an off-the-shelf estimator and then projects J into target cameras. Thus camera parameters are assumed and used at inference time. The claim should be revised to what is actually true—for example, that camera parameters are not provided as network inputs during training—or the pipeline description must change. This matters because the NVS results are cited as evidence for the versatility of the 2D-keypoint representation.","section":"Section 4.3, Novel View Synthesis"},{"comment":"The paper gives no annotation-quality control for the MediaPipe and SAM labels on the 10M images. Because the model conditions on these 2D keypoints and masks, annotation errors directly bound the achievable pose fidelity and the claimed \"precise control.\" The authors should report a validation statistic on a subset (e.g., agreement with manual labels or a downstream pose-estimation metric) and describe any filtering steps used to remove low-quality annotations.","section":"Section 4.1, FoundHand-10M"},{"comment":"The zero-shot video synthesis comparison is based on 12 in-the-wild videos with no standard errors, no significance tests, and no description of how the reported metrics are aggregated. Given the paper's claim of outperforming task-specific video baselines, this quantitative evidence is too thin; the authors should provide per-video results or a larger evaluation, and clarify whether the metric is computed on generated frames only.","section":"Section 5, Table 3(b)"}],"minor_comments":[{"comment":"The dataset name is inconsistent between \"FoundHand-10M\" (Sections 1 and 4.1) and \"FoundHand10M\" (Section 7); please unify the spelling.","section":"Section 7"},{"comment":"The caption reads \"Ours w/ skeleton Ours,\" which appears to be a typo; the intended comparison is unclear.","section":"Figure 3"},{"comment":"The caption describes the metrics as \"identity consistency metrics,\" but FID and LPIPS are perceptual or distribution-level metrics; consider a more accurate label.","section":"Table 1"},{"comment":"Table 3 lists \"Zero123 [44]\" while reference [44] is titled \"Zero-1-to-3\"; please ensure the citation label matches the method being compared.","section":"Table 3"},{"comment":"The classifier-free guidance equation uses the convention ϵ̂θ = wϵθ(zτ;τ,c)+(1−w)ϵθ(zτ;τ,∅); the text should state the value of w used in the experiments and any dependence on the binary flag y.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The method has a plausible technical core, and the fixes requested in the major comments are achievable within the manuscript's scope. My main concern is the gap between the six-task state-of-the-art claim and the evidence, particularly the self-referential gesture-transfer protocol and the contradiction between the camera-parameter claim and the NVS pipeline. If the authors cannot supply genuine pose-transfer evaluation and revise the NVS claim, I would not be able to recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the paper's real contribution is the dataset: 10M hand images with 2D keypoints and masks assembled from 12 existing datasets, plus a DiT-based diffusion model conditioned on keypoint heatmaps that shows genuinely broad capabilities—reposing, domain transfer, novel view synthesis, hand fixing, video. That is a useful system and the qualitative results are often convincing. Second, the gesture-transfer SOTA claim does not survive a close look at Table 1. The quantitative evaluation sets the target pose equal to the reference pose and measures reconstruction quality. That tests identity preservation, not pose transfer; a model that copies the reference could ace it. Without a metric on genuinely different target poses (e.g., PCK against the requested target keypoints), the central claim of SOTA gesture transfer is unsupported.\n\nWhat the paper does well: the dataset integration is real work. Using 2D keypoints as a universal conditioning representation is a sensible and scalable choice, and the results suggest it works in practice. The zero-shot applications—hand fixing and video synthesis—are shown with enough qualitative evidence to be plausible. The NVS comparison against ZeroNVS/ImageDream on InterHand2.6M is a fair test, though the 'no explicit camera parameters' claim is qualified by the pipeline that estimates 3D joints and projects them with assumed camera parameters. The limitations section is honest but narrow—it only flags the 256px resolution, while the evaluation weaknesses are the real issue.\n\nSoft spots in proportion. The gesture-transfer metric is the load-bearing flaw; it needs to be redesigned. FID usage is ambiguous ('between reference and generated images'). No error bars or statistical tests anywhere, and no code or data released, so independent verification is impossible. The paper also omits comparisons to HandDiffuser and Hand1000, which are cited in related work and are the closest hand-specific generators; that omission is conspicuous. Annotation quality from MediaPipe/SAM on 10M images is not audited; a human-check sample would help.\n\nWho this is for: anyone working on controllable hand image generation, hand-conditioned video, or hand datasets. The dataset and model are likely to be useful even if the SOTA claims are overstated. It deserves a serious referee—the core idea is sound and the dataset alone is a contribution—but the paper needs a real gesture-transfer evaluation, baseline additions, and artifact release before acceptance.\n\nRecommendation: send to peer review, expect major revision. The evaluation needs rework.","headline":"A useful hand dataset and multi-task generation model whose flagship gesture-transfer claim rests on a same-pose reconstruction metric that does not measure pose transfer.","tokens_in":18312,"tokens_out":2883,"would_cite":false,"duration_ms":27446,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion model trained on 10 million hand images makes pose-controllable hand generation state of the art across six tasks.","keywords":["hand image generation","diffusion model","2D keypoint conditioning","novel view synthesis","gesture transfer","domain transfer","hand dataset","controllable generation"],"falsifier":"Take a single hand image with known multi-view ground truth, estimate 3D joints, project them into a target camera far outside the small cone used in the paper (for example, a 90-degree rotation), generate the view with FoundHand, and measure the error between the generated image's detected 2D keypoints and the ground-truth projected keypoints. If the error grows sharply with baseline or the generated hand becomes anatomically implausible, the sufficiency of 2D keypoints for viewpoint control is disproven.","tokens_in":17323,"feed_emoji":"🖐️","tokens_out":5222,"duration_ms":46537,"temperature":0.7,"pith_summary":"The paper aims to show that hand-image generation can be made controllable and state-of-the-art by scaling up domain-specific training data and using 2D keypoints as the sole control signal. To do this, the authors assemble FoundHand-10M, a dataset of more than ten million hand images from twelve existing sources, annotated automatically with MediaPipe keypoints and SAM masks. They train FoundHand, a latent diffusion transformer that takes a reference image plus target 2D keypoint heatmaps and generates a new hand image matching the pose and viewpoint. The central claim is that 2D keypoints naturally encode both articulation and camera viewpoint, so no explicit camera parameters or 3D supervision are needed. If true, a single model can repose hands, transfer appearance across domains, synthesize novel views, fix malformed hands, and generate hand and hand-object videos in a zero-shot manner.","feed_headline":"With 10M hand images, 2D keypoints alone control pose and view","feed_subtitle":"One model reposes hands, transfers style, synthesizes new views, and fixes malformed AI hands without retraining.","key_machinery":"The load-bearing mechanism is pair-based training with spatially aligned multi-modal conditioning. The model is a latent diffusion vision transformer (DiT) pretrained on ImageNet; the reference and target frames are encoded with a Stable Diffusion VAE, and the reference image latents, keypoint heatmaps, and hand masks are aligned and passed through a shared embedder, with 3D self-attention between the two frames. Target heatmaps are Gaussian heatmaps over 42 keypoints, 21 per hand, with zero channels for absent hands, so the model can handle single or dual hands and can be conditioned without masks at test time. A binary flag tells the model whether the frame pair comes from synchronized views (viewpoint change) or a temporal sequence (pose change), and stochastic conditioning on previously generated views is used at inference for novel view synthesis and video generation.","core_discovery":"FoundHand's central discovery is that a diffusion model trained on a large, diverse collection of hand image pairs can treat 2D keypoint heatmaps as a universal representation for hand articulation and camera viewpoint. During training, the model sees pairs of frames, either consecutive frames from hand videos or synchronized views from multi-view captures, and learns to map a reference image plus target heatmaps to the target image. At inference, the same conditioning enables reposing, where reference appearance is preserved while the keypoints change; domain transfer, where the keypoints stay fixed but the reference drives style; and novel view synthesis, where 3D joints estimated from a single view are projected into new cameras and the model generates consistent views without explicit camera parameters. The paper reports state-of-the-art quantitative results on gesture transfer, domain-transfer aided 3D hand estimation, novel view synthesis on InterHand2.6M, and zero-shot video and hand-object interaction synthesis, outperforming task-specific baselines.","pith_inferences":["Because the viewpoint sampling is restricted to a small cone around the reference camera, the paper's novel-view evidence does not establish that 2D keypoints fully resolve large-baseline depth ambiguity; a testable extension is to quantify the view cone angle at which reprojection error degrades.","The automatic MediaPipe and SAM annotations receive no reported quality-control filtering, so the dataset may contain noisy keypoint and mask labels; a useful extension would be to measure how generation quality changes when a subset is cleaned or human-verified.","The same pair-based conditioning could be applied to other articulated structures, such as faces or bodies, where 2D landmarks plus a reference image might support reposing, style transfer, and view synthesis without 3D supervision.","The stochastic conditioning used for video could be turned into a simple consistency metric: freeze the first frame and measure pixel-level drift across long generated sequences to test whether the model maintains long-term identity."],"forward_implications":["Hand pose editing becomes a simple keypoint-painting interface: users specify target 2D keypoints and the model preserves appearance while articulating the hand.","Domain transfer can be used as a data-augmentation tool: fine-tuning a 3D hand estimator on FoundHand-transferred images improves mesh recovery metrics on a target domain.","Novel views can be generated from a single hand image without explicit 3D supervision, outperforming NeRF-based zero-shot view synthesis baselines on InterHand2.6M.","Malformed hands produced by text-to-image models can be repaired zero-shot via Repaint-style inpainting conditioned only on keypoints.","Hand and hand-object videos can be synthesized by conditioning each frame on the first frame plus recent generated frames, with no video-specific training."],"supporting_citations":[{"why":"Supplies the automatic 2D keypoint annotations for all 10M images in FoundHand-10M.","marker":"[48]"},{"why":"Supplies the segmentation mask annotations used as auxiliary conditioning.","marker":"[37]"},{"why":"Provides the latent diffusion transformer backbone that FoundHand builds upon.","marker":"[66]"},{"why":"Provides the pretrained VAE latent space and the general image prior used for encoding images.","marker":"[79]"},{"why":"Estimates 3D hand joints that are projected into target cameras to create conditioning keypoints for novel view synthesis.","marker":"[65]"},{"why":"Supplies the stochastic conditioning mechanism used for autoregressive multi-view generation.","marker":"[105]"},{"why":"Provides the InterHand2.6M benchmark used for novel view synthesis evaluation and as a dataset source.","marker":"[54]"},{"why":"Serves as the main task-specific baseline for gesture transfer and hand-object interaction comparison.","marker":"[94]"}],"fun_headline_variants":["2D keypoints alone now drive hand pose and camera view","FoundHand: 10M hand images enable keypoint-only control","One model reposes hands, swaps styles, and synthesizes views","Keypoint-only conditioning now controls pose, view, and style"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 2D keypoints encode enough information about camera viewpoint and depth for the model to synthesize correct novel views, so a large-baseline viewpoint change or a pose with strong depth ambiguity could break the claim that no explicit camera parameters are needed.","fun_headline_variants_meta":{"raw":{"variants":["2D keypoints alone now drive hand pose and camera view","FoundHand: 10M hand images enable keypoint-only control","One model reposes hands, swaps styles, and synthesizes views","Keypoint-only conditioning now controls pose, view, and style"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000695,"raw_usage":{"total_tokens":3130,"prompt_tokens":917,"completion_tokens":2213,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":2140}},"tokens_in":533,"tokens_out":2213,"duration_ms":15836,"temperature":1.0,"reasoning_tokens":2140,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:09:27.165782+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a single hand image with known multi-view ground truth, estimate 3D joints, project them into a target camera far outside the small cone used in the paper (for example, a 90-degree rotation), generate the view with FoundHand, and measure the error between the generated image's detected 2D keypoints and the ground-truth projected keypoints. If the error grows sharply with baseline or the generated hand becomes anatomically implausible, the sufficiency of 2D keypoints for viewpoint control is disproven.","supporting_citations":[{"cited_title":"Re- constructing hands in 3D with transformers","cited_arxiv_id":null,"evidence_quote":"Estimates 3D hand joints that are projected into target cameras to create conditioning keypoints for novel view synthesis."},{"cited_title":"Interhand2","cited_arxiv_id":null,"evidence_quote":"Provides the InterHand2.6M benchmark used for novel view synthesis evaluation and as a dataset source."},{"cited_title":"Controlling the world by sleight of hand, 2024","cited_arxiv_id":null,"evidence_quote":"Serves as the main task-specific baseline for gesture transfer and hand-object interaction comparison."}],"review_version":1}