{"id":"fcf4393b-c8de-4bbe-946a-8668c58ca329","arxiv_id":"2411.18068","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PersonaCraft adds SMPLx depth and normal conditioning, occlusion boundary enhancement, and occlusion-aware classifier-free guidance to diffusion models, enabling controllable multi-person images that preserve both face and body identity.","lead":"PersonaCraft is a system that lets a user generate group photos with several specific people, keeping their faces and body shapes while following a given 3D pose. It uses 3D body models and special occlusion handling to reduce the distortions and identity mixing that current 2D pose methods produce in crowded or overlapping scenes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Body-shape preservation in Tab. 1 is measured by MultiHMR β cosine similarity (Eq. S7), but the same MultiHMR β is also the conditioning input; an independent body-shape regressor is needed before the full-body personalization claim is accepted.","rationale":"The paper is a serious systems contribution with extensive ablations, large user studies, and supplementary fine-tuned baselines; I do not see grounds for rejection. The most load-bearing point is not the diffusion architecture or the OccNet/OccCFG details but the validity of the headline body-shape metric. The central claim of full-body multi-human personalization is supported by Sbody, and that metric is circular in a specific way: MultiHMR generates both the conditioning β and the measured β. The reader's weakest assumption—that inaccurate MultiHMR fits degrade results—is related but not identical; my concern is that even with accurate fits the metric measures agreement with the conditioning estimator, not true body-shape fidelity. This can inflate PersonaCraft's advantage because the baselines lack the same conditioning. The OccNet/OccCFG ablations show small MPJPE/AP changes and no error bars, which makes the occlusion-handling sub-claims harder to assess, but the qualitative and user-study evidence supports a softer version of the claim. Tab. S8 mitigates the fine-tuning objection. Overall, the right resolution is unchanged from the reader: conditional acceptance requiring an independent body-shape evaluation and release of code and data. The concrete test above would settle the central concern.","tokens_in":21928,"tokens_out":4772,"duration_ms":43689,"concrete_test":"Recompute the body-shape preservation numbers of Tab. 1 (and Tab. S8) on the exact same generated images and references, but estimate β_ref and β_gen with a body regressor that was not used in conditioning and has a different architecture and training set (e.g., OSX, PyMAF-X, or SMPLer-X). Keep the cosine-similarity formula and the image set unchanged. If the Ours-vs-baseline margin collapses (say, below 0.05) or Ours no longer ranks first, the Tab. 1 body-shape claim is an artifact of MultiHMR self-consistency and the central full-body personalization claim must be downgraded. If the margin survives with a strong independent regressor and per-subject confidence intervals, the concern is resolved. A secondary check is to verify on a subset that a synthetic image that perfectly rerenders the SMPLx mesh scores near 1.0 with MultiHMR but substantially lower with the independent regressor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"PersonaCraft's strongest quantitative evidence for full-body personalization is Tab. 1's body-shape preservation column (0.560 vs. 0.401). This metric, Eq. S7, computes the cosine similarity between β_ref and β_gen, where both vectors come from MultiHMR. Critically, β_ref is not an external ground truth: Sec. 3.3 uses MultiHMR to estimate β from each reference and renders SMPLx depth/normal maps from that β as the SCNet conditioning signal. The generator is therefore optimized, both at training time and through conditioning, to produce images that are consistent with MultiHMR's β-space. Re-fitting the generated image with the same MultiHMR then measures self-consistency with the conditioning estimator, not independently verified anatomical fidelity. Baselines conditioned only on 2D pose do not receive β_ref, so their Sbody scores are not measuring the same quantity; the large margin in Tab. 1 may largely reflect this asymmetry. The paper's Sec. 5 limitation about fitting accuracy acknowledges that bad fits hurt output, but it does not address the circularity: a MultiHMR-consistent but anatomically wrong image can score high Sbody. The separate user study on body shape is suggestive, but it is not reported with the identity-masked, randomized protocol needed to rule out the metric artifact.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PersonaCraft proposes a diffusion-based framework for multi-human personalized image synthesis. It conditions an SDXL generator on SMPLx depth and normal maps through a fine-tuned ControlNet (SCNet), adds an Occlusion Boundary Enhancer Network (OccNet) trained on masked depth edges, and applies Occlusion-Aware Classifier-Free Guidance (OccCFG) to reinforce conditioning in occluded regions. Full-body personalization is achieved by injecting SMPLx shape parameters through SCNet and combining them with Face Identity ControlNet, plus an optional textual body-shape refinement pathway. The paper reports extensive experiments on COCO-WholeBody, with quantitative and user-study results that claim large improvements over baselines in face identity, body shape, pose accuracy, and perceptual quality.","tokens_in":22147,"tokens_out":6080,"duration_ms":52091,"significance":"The architectural components are well motivated, and the ablations in Tab. 5 show that each proposed module contributes to pose consistency. If the body-shape preservation metric is replaced with an independent evaluation, the method would represent a substantial advance in controllable multi-human generation with explicit occlusion handling and full-body personalization. The paper also provides a useful efficiency analysis and a broad qualitative comparison, and it ships a supplementary description of training and evaluation details.","major_comments":[{"comment":"The body-shape preservation score S_body in Supp. Eq. (S7) is self-referential: β_ref is estimated by MultiHMR from the reference image and is exactly the vector rendered into SMPLx depth/normal maps used as conditioning in Sec. 3.3, while β_gen is also obtained by running MultiHMR on the generated image. The score therefore measures consistency with the conditioning estimator, not independently verified anatomical fidelity, and the comparison in Tab. 1 is asymmetric because the OMG/IPAdapter baselines do not receive β_ref as conditioning. An independent body-shape regressor (a different SMPL/SMPLx fitter or a learned shape descriptor) should be used for both reference and generated images, or the claim should be restricted to conditioning fidelity. The user study in Tab. 2 is suggestive, but the protocol in Sec. B.4 does not report identity-masked, randomized presentation, so it does not currently rule out the metric artifact.","section":"Supp. Eq. (S7), Sec. 3.3, Tab. 1"},{"comment":"The inference-time application of OccNet appears inconsistent with its training objective. Eq. (3) trains OccNet on e_SMPLx ⊙ M_occ (depth edges inside the occlusion mask), and the text of Sec. 3.2 states that the model is provided only these edges. However, Eq. (5) defines R_occ = E^Occ_ω(d_SMPLx), i.e., the full depth map rather than the edge signal. If this is a typo, it should be corrected to e_SMPLx ⊙ M_occ; if not, the discrepancy between training and inference inputs must be explained, since the ablation in Tab. 5 attributes a pose-consistency gain to OccNet.","section":"Eq. (5), Eq. (3), Sec. 3.2, Sec. 3.3"}],"minor_comments":[{"comment":"The row 'w/ body shape' reports Total = 0.615, but Tab. 1 reports Total = 0.560 for the same Single (0.630) and Multi (0.548) values; this numerical inconsistency should be corrected or explained.","section":"Supp. Tab. S6"},{"comment":"The MPJPE and AP-0.5 metrics are computed by running MultiHMR on generated images while the target poses also come from MultiHMR; this is acceptable as a relative comparison, but the paper should state that these scores measure compatibility with MultiHMR's pose space rather than absolute 3D accuracy.","section":"Sec. 4, Metrics"},{"comment":"The user study description should state whether participants were blinded to method identity and whether the order of presented images was randomized; without this information, the preference rates in Tabs. 2 and 4 are difficult to interpret.","section":"Sec. B.4, User Study"},{"comment":"The baseline 'ControlNet-Flux' is cited as [48, 91]; it would be cleaner to cite the Flux model alone and note that ControlNet is applied on top, since [48] is the base model and [91] is the ControlNet method.","section":"Sec. 4.2, Baselines"},{"comment":"The textual body-shape pathway is only evaluated qualitatively (Fig. 11); a quantitative ablation of this pathway, even on a small set, would strengthen the claim of dual-pathway benefit.","section":"Sec. 3.3, Dual-Pathway Body Shape Personalization"}],"recommendation":"major_revision","confidential_remarks":"The main obstacle to acceptance is the circular body-shape metric in Supp. Eq. (S7). If the authors supply an evaluation with an independent body-shape regressor, the paper is likely to become acceptable after minor revisions; the remaining issues, such as the OccNet input inconsistency in Eq. (5) and the Table S6 data inconsistency, are readily fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — here’s the short version: PersonaCraft is a real systems contribution, and you should have the referees look at it, but its headline body-shape number (Sbody 0.560 vs 0.401) does not support the full-body personalization claim as cleanly as the paper presents it. The metric in Eq. S7 is self-referential: β_ref is estimated by MultiHMR, used to render the SMPLx conditioning, and then the generated image is re-fit with the same MultiHMR and compared against that same β_ref. Baselines conditioned only on 2D pose never receive β_ref, so the comparison is not apples-to-apples. That is an evaluator artifact, not necessarily a failure of the method — but the claim needs an independent body-shape regressor or an explicit relabeling as a conditioning-fidelity check.\n\nWhat is genuinely new: this is the first multi-human SMPLx-ControlNet that combines depth and normal maps with occlusion masks, OccNet, and OccCFG. The ablations show each piece helps, the efficiency numbers are credible, and the user studies are large (18.5k and 15.4k responses). The supp also fine-tunes the main baselines on their training set, which is the right way to answer the “unfair baseline” objection. The paper is honest in Sec. 5 that output depends on MultiHMR fitting accuracy.\n\nSoft spots, in order of importance. First, the body-shape metric as above. Second, the pose comparisons are asymmetric: MPJPE compares a 3D-conditioned method against 2D-skeleton baselines, so the method is being handed richer conditioning by construction. The 2D AP results are less vulnerable to that objection, and the qualitative figures do show better occlusion handling. Third, there are no error bars or confidence intervals anywhere; with a 1,000-image test set, a referee should ask for variance. Fourth, no code or data is released, which matters more here because the body-shape evaluation depends on MultiHMR fits. Minor: in Tab. 5, adding OccCFG slightly lowers AP-0.5 (0.499 -> 0.495) while improving MPJPE; not a problem, but worth a sentence. The body-shape user study is suggestive, but the protocol is not described with identity masking, so face identity could leak into body-shape judgments.\n\nThe central occlusion-handling claim holds up qualitatively and in the user study. I would not desk-reject this. Send it to a serious referee; the revision should require code/data release and an independent body-shape metric, and should reframe the body-shape score as conditioning fidelity unless re-validated. I would not cite it in the next year until those numbers are cleaned up, but I’d be curious to see the resubmission.","headline":"The system is a real multi-person 3D-conditioned generation contribution, but the headline body-shape metric is self-referential and the pose comparisons are asymmetric; deserves a serious referee, conditional on an independent body-shape evaluation and code/data release.","tokens_in":22783,"tokens_out":3114,"would_cite":false,"duration_ms":28612,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PersonaCraft uses SMPLx-derived 3D geometry—depth and normal maps—to condition a diffusion model for multi-human image generation, reporting large gains in pose accuracy, face identity, and body-shape preservation over 2D-skeleton…","keywords":["diffusion models","controllable image generation","SMPLx","3D human pose conditioning","occlusion handling","full-body personalization","multi-human scene generation","classifier-free guidance"],"falsifier":"Take a set of heavily occluded multi-person images for which SMPLx fits are known to be poor, or compare against motion-capture ground-truth meshes in the same poses. If PersonaCraft with those poor fits does not outperform a 2D-skeleton baseline, or if its MPJPE advantage disappears when the fitter is held fixed across methods, then the occlusion robustness is coming from the fitter rather than from the diffusion-side machinery.","tokens_in":21649,"feed_emoji":"🧍","tokens_out":5766,"duration_ms":49344,"temperature":0.7,"pith_summary":"PersonaCraft's central claim is that conditioning a diffusion image generator on 3D human-body geometry, rather than on 2D skeletons, removes the ambiguity behind occlusion failures and anatomical distortions in multi-person scenes. The paper builds SMPLx mesh renderings (depth and normal maps) into a ControlNet-style branch, adds an Occlusion Boundary Enhancer Network that trains on depth edges inside occlusion masks, and applies occlusion-aware classifier-free guidance so conditioning is strengthened exactly where bodies overlap. It combines these with a face-identity branch and a dual-pathway body-shape representation to personalize both face and body. In their evaluations, the method reports substantially lower 3D pose error (MPJPE 60.65 vs. 112.6 for the strongest personalized baseline) and higher body-shape preservation, and user studies rank it first for naturalness, identity, and pose consistency.","feed_headline":"3D body conditioning fixes occluded multi-person image generation","feed_subtitle":"SMPLx depth and normal maps halve pose error and preserve face and body identity in generated scenes.","key_machinery":"The machinery has three cooperating parts. SMPLx-ControlNet (SCNet) feeds SMPLx depth and normal renderings through a ControlNet, giving the U-Net depth cues and surface orientation; depth alone is best among single renderings, and depth-plus-normal is the chosen default. The Occlusion Boundary Enhancer Network (OccNet) receives only SMPLx depth edges inside occlusion masks, forcing the model to learn boundary structure where bodies overlap. Occlusion-Aware Classifier-Free Guidance (OccCFG) then scales the conditional noise prediction by a larger factor inside the occlusion mask and a smaller factor outside, so occluded regions get strengthened conditioning without over-saturating open areas. Full-body personalization is carried by SMPLx shape coefficients in the rendered geometry plus an optional CLIP-based textual body-shape pathway, with face identity injected through a face ControlNet.","core_discovery":"The discovery the paper tries to establish is that a single diffusion pipeline can jointly solve full-body identity preservation and occlusion-heavy multi-person generation when pose is represented by SMPLx depth and normal maps. The authors show that 3D conditioning alone still misses fine-grained occlusions, so they add occlusion masks (pixels whose ray intersects more than two surfaces), an Occlusion Boundary Enhancer Network that learns from depth-edge signals inside those masks, and an occlusion-aware classifier-free guidance rule that applies a higher CFG scale only in occluded regions. Combined with a face identity ControlNet and a body-shape pathway that uses SMPLx shape coefficients plus optional textual refinement, the system claims personalized and controllable multi-human synthesis. Reported numbers include MPJPE 62.647 vs. 102.64 for pose-controlled baselines and top-1 user preference above 40% across all evaluated criteria.","pith_inferences":["If the reported MPJPE gains largely reflect the single-shot SMPLx fitter's own estimates acting as both conditioning and evaluation ground truth, then a fair comparison against methods using motion-capture or manually registered 3D poses would separate the value of the 3D representation from the value of the fitter; the paper does not run that experiment.","The occlusion-mask idea is not human-specific: counting intersected surfaces and masking depth edges could be applied to any articulated object in a ControlNet-style diffusion model, though the paper does not test this.","Because OccCFG only needs an occlusion mask at inference, it could be plugged into other 3D-conditioned human generators; a direct test would be running it on a single-person generator with self-occlusion and checking whether the mask-based guidance alone reproduces the reported gains.","A likely failure mode the paper acknowledges is that poor SMPLx fits in partially visible references will propagate into both generation and evaluation, so a user-facing system would need a rejection rule for low-confidence fits; the paper does not propose one."],"forward_implications":["Replacing 2D skeleton conditioning with SMPLx depth and normal conditioning is enough to reduce 3D pose error in multi-person scenes by roughly half in their tests.","Occlusion masks derived from 3D mesh intersections let the model locate fine occlusions, and depth edges inside those masks are a sufficient training signal to sharpen occluded anatomy.","Raising classifier-free guidance inside occluded regions only, rather than globally, improves anatomical consistency without the usual over-saturation.","Full-body personalization can be built by separating body shape (SMPLx coefficients plus text) from face identity, which existing face-only personalizers do not do.","Because the conditioning is modular, the same SCNet and OccNet residual additions can be combined with different face modules and style adapters without retraining the whole pipeline."],"supporting_citations":[{"why":"supplies the single-shot SMPLx pose and shape fits used to render depth, normal, and occlusion conditions for training and evaluation.","marker":"[12]"},{"why":"defines the SMPLx parametric body model whose depth and normal renderings carry the 3D conditioning.","marker":"[61]"},{"why":"provides the ControlNet framework that SCNet and OccNet fine-tune for conditioning injection.","marker":"[91]"},{"why":"supplies the face Identity ControlNet used for facial identity preservation in the personalized pipeline.","marker":"[81]"},{"why":"is the multi-concept personalization baseline that PersonaCraft is compared against and combined with in ablations.","marker":"[43]"},{"why":"provides the SDXL text-to-image backbone and the pose-conditioning baselines used in the pose-controlled experiments.","marker":"[64]"}],"fun_headline_variants":["Occlusion-aware 3D diffusion for full-body multi-person image synthesis","SMPLx depth maps enable occlusion-robust full-body personalization","Full-body multi-person generation with occlusion-aware 3D conditioning","Occlusion-aware 3D conditioning for personalized multi-human scenes","3D conditioning with occlusion masks preserves identity in multi-person scenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the SMPLx pose and shape parameters recovered from ordinary RGB photos are accurate enough to serve as both the generation condition and the evaluation ground truth; if those fits are wrong in crowded or occluded scenes, the whole pipeline and its metrics inherit the error.","fun_headline_variants_meta":{"raw":{"variants":["Occlusion-aware 3D diffusion for full-body multi-person image synthesis","SMPLx depth maps enable occlusion-robust full-body personalization","Full-body multi-person generation with occlusion-aware 3D conditioning","Occlusion-aware 3D conditioning for personalized multi-human scenes","3D conditioning with occlusion masks preserves identity in multi-person scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000875,"raw_usage":{"total_tokens":3805,"prompt_tokens":982,"completion_tokens":2823,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":2730}},"tokens_in":598,"tokens_out":2823,"duration_ms":17921,"temperature":1.0,"reasoning_tokens":2730,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:32:15.207528+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of heavily occluded multi-person images for which SMPLx fits are known to be poor, or compare against motion-capture ground-truth meshes in the same poses. If PersonaCraft with those poor fits does not outperform a 2D-skeleton baseline, or if its MPJPE advantage disappears when the fitter is held fixed across methods, then the occlusion robustness is coming from the fitter rather than from the diffusion-side machinery.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the SMPLx parametric body model whose depth and normal renderings carry the 3D conditioning."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"provides the ControlNet framework that SCNet and OccNet fine-tune for conditioning injection."},{"cited_title":"Omg: Occlusion-friendly personalized multi-concept generation in diffusion models","cited_arxiv_id":null,"evidence_quote":"is the multi-concept personalization baseline that PersonaCraft is compared against and combined with in ablations."}],"review_version":1}