{"id":"4cb1aa65-6d8d-4c84-b459-e34a509aa290","arxiv_id":"2412.15200","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DI-PCG learns to invert procedural 3D generators by diffusing over their parameters, conditioned on DINOv2 image features, producing editable assets in seconds from a photo.","lead":"This paper trains a small diffusion model to guess the knobs and switches of procedural 3D generators directly from a single photo, then feeds those guesses back into the generator to build editable 3D chairs, tables and vases. A smart generalist might read it because it turns hard parameter tuning into a learned, seconds-fast step, and shows a cheaper alternative to giant image-to-3D models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim of accurate parameter recovery is never directly evaluated; Table 1 reports only geometric metrics (CD/EMD/F-Score), not comparisons of predicted vs. ground-truth parameters.","rationale":"The reader identified generator expressiveness as the weakest assumption, which is indeed a fundamental ceiling and is acknowledged in Section 4.5 and the supplementary failure cases. My stress-test focus is different: even within the expressible set, the paper never verifies that the diffusion model recovers the actual parameters, only that the resulting geometry is close. This is more directly tied to the central claim of 'recovering parameters accurately' in the abstract and to the method's identity as inverse PCG. Geometric fidelity is a necessary but not sufficient condition for parameter accuracy, especially if the generator has sloppy or redundant parameters. The editing results (Section 4.4) are suggestive but only qualitative. The concern is concrete and testable: add a parameter-error metric on the held-out test split. If parameters are accurate, the central claim stands; if not, the paper's contribution is better described as constrained image-to-3D generation rather than inverse PCG. This does not change the overall verdict (CONDITIONAL), because the method is still promising and the gap is fixable with additional evaluation. It does, however, strengthen the case that the stated conditions must include direct parameter-accuracy verification before the headline claim is accepted. I also note the MCMC comparison is illustrative only, which further supports CONDITIONAL rather than ACCEPT.","tokens_in":16145,"tokens_out":4868,"duration_ms":49364,"concrete_test":"On the chair test split (2000 held-out image-parameter pairs), render the condition image from ground-truth parameters, run DI-PCG sampling, and compare the predicted (canonicalized) parameters to the ground truth. Report per-parameter mean absolute error (continuous parameters) and classification accuracy (discrete parameters), plus an aggregate error. If the aggregate MAE is small (e.g., <0.1 on the normalized [-1,1] scale), the 'accurate parameter recovery' claim is supported; if MAE is large despite good geometric metrics, the method is fitting geometry without actually recovering the intended parameters, undermining the inverse-PCG and editing claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim (abstract; Section 1) is that DI-PCG 'recovers parameters accurately,' and the method is framed as inverse PCG (Section 3.2). However, every quantitative result in Table 1 evaluates geometry: Chamfer Distance, Earth Mover's Distance, and F-Score between the generated mesh and the ground-truth mesh. Predicted parameters are never compared to the ground-truth parameters that generated the condition image. This matters because the parameter-to-geometry map of the Infinigen generators is not necessarily injective: different parameter settings can produce similar or near-identical geometry, and some parameters may have only a weak effect on the rendered shape. Consequently, low CD/EMD and high F-Score can be achieved even when the recovered parameters are systematically wrong. The editing application (Section 4.4) relies on the recovered parameters being semantically correct — adjusting 'leg height' or 'has_arm' only works if those parameters were accurately inferred. Since the paper's central contribution is inverse parameter estimation, the absence of a direct parameter-accuracy metric leaves the core claim unsupported. The expressiveness limitation (Section 4.5) is acknowledged by the authors, but this parameter-measurement gap is not; it is a fixable omission rather than an inherent ceiling. A second related gap is that the MCMC comparison (Section 'Comparison with MCMC') is purely qualitative, with no measured accuracy or runtime table, so the claimed efficiency advantage over sampling-based inverse PCG is also not quantified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DI-PCG, a diffusion-transformer method for inverse procedural content generation from a single image. The PCG parameters themselves are treated as the denoising target, while DINOv2 image features are injected via cross-attention as conditioning. Training pairs are generated by randomly sampling parameters from six Infinigen generators and rendering the resulting meshes. The authors report that the method needs only 7.6M parameters and about 30 GPU hours per generator, that it outperforms image-to-3D baselines on chair geometry metrics (CD, EMD, F-Score) on both an internal test split and a filtered ShapeNet chair subset, and that it supports sketch conditioning and parameter-level editing. The main claims are efficiency, accuracy of parameter recovery, and generalization to in-the-wild images.","tokens_in":16421,"tokens_out":3982,"duration_ms":29963,"significance":"If the central claim holds, this is a valuable contribution to inverse PCG and image-to-3D generation: a lightweight, generator-agnostic diffusion prior over PCG parameters would enable fast inference, editable outputs, and high-fidelity meshes by construction. The pipeline is self-contained, avoids external 3D data collection, and the architecture is simple and reproducible. The paper explicitly acknowledges the expressiveness limitation of the underlying procedural generators, which is a strength. However, the quantitative evaluation as presented does not yet establish the headline claim of accurate parameter recovery, and the generalization evidence is narrower than the abstract suggests. The significance is therefore conditional on closing this evaluation gap.","major_comments":[{"comment":"The central claim that DI-PCG 'recovers parameters accurately' (Abstract, Section 1) is not directly evaluated. Table 1 reports only geometric metrics (CD, EMD, F-Score) between the generated and ground-truth meshes, with no comparison between predicted and ground-truth parameters. Because the parameter-to-geometry map of the Infinigen generators is not necessarily injective and some parameters may have only a weak effect on the rendered shape, low CD/EMD and high F-Score can coexist with systematically wrong parameter estimates. This matters because the editing application in Section 4.4 depends on the recovered parameters being semantically correct. Please add a direct parameter-accuracy evaluation on the internal test split, where ground-truth parameters are known (for example, normalized L1 error for continuous parameters and per-choice accuracy for discrete parameters), and discuss the identifiability of the parameter space.","section":"Section 3.2 and Table 1"},{"comment":"The quantitative evaluation is limited to the chair category, and the ShapeNet generalization test is post-hoc filtered to 218 models after excluding sofa-like or artistic chairs. The claim that DI-PCG is 'generalizing well to in-the-wild images' is therefore supported quantitatively for only one category and a filtered subset; the other five categories are supported only qualitatively. Please either provide quantitative results for the other procedural generators or revise the generalization claim to match the evidence actually presented.","section":"Section 4.2"},{"comment":"The MCMC comparison is purely qualitative: Figure 6 shows a single example with iteration counts and wall-clock times, but no measured accuracy or runtime table. Since the paper's efficiency claim is framed relative to MCMC, this comparison is load-bearing. Please provide a quantitative comparison, for example parameter error or geometric metrics as a function of number of iterations and wall-clock time, using the same condition image and scoring metric as the proposed method.","section":"Section 4.1, 'Comparison with MCMC'"},{"comment":"The internal test split is rendered by the same procedural generators that are used to create the training pairs. This largely measures the model's ability to interpolate within the training distribution, not its ability to generalize to out-of-distribution conditions. The paper should state this limitation explicitly near Table 1, and if possible quantify robustness to unseen render viewpoints, lighting, or generator variants, since the training render views are limited to azimuths 0/30/60, elevations 30/60, and distances 1.8/2.0.","section":"Section 4.1, 'Data preparation' and Table 1"}],"minor_comments":[{"comment":"The canonicalization projection ϕ is described only verbally; please specify exactly how discrete parameters are mapped to continuous intervals and how the inverse projection resolves values near bin boundaries, and discuss whether uniform binning distorts the posterior distribution.","section":"Section 3.2, 'Representation'"},{"comment":"The tables report single numbers with no variance estimates. Please report standard deviations or confidence intervals across multiple runs or random seeds so that the margin over the baselines can be assessed.","section":"Tables 1 and 2"},{"comment":"The sentence 'It clearly shows that DI-PCG can reliably fit the procedural generator and inversely estimate the parameters with high accuracy' overstates what Table 1 demonstrates, since Table 1 contains no parameter-level metrics.","section":"Section 4.2"},{"comment":"The MCMC timing labels should specify the hardware, the proposal distribution, the scoring function, and the number of chains; without these details the reported wall-clock times are difficult to interpret.","section":"Figure 6"},{"comment":"The phrase 'generalizing well to in-the-wild images' should be qualified in the abstract, given that the quantitative generalization evidence is restricted to filtered ShapeNet chairs and that in-the-wild results for other categories are qualitative only.","section":"Abstract and Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper presents an interesting direction and the pipeline is simple and reproducible in principle. My main concern is that the evaluation does not yet establish the paper's central claim of accurate parameter recovery; the missing parameter-level metrics, the qualitative-only MCMC comparison, and the limited scope of the quantitative tests make the current conclusions too strong. I would be willing to reconsider after a revision that adds direct parameter-accuracy evaluation, a quantitative MCMC comparison, and a clearer statement of the scope of the generalization claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Genuinely new idea, honestly scoped, but the paper's flagship claim—accurate parameter recovery—is never directly measured. All quantitative results are geometric (CD/EMD/F-Score). That missing table is the thing to ask for.\n\nWhat's new: DI-PCG treats the canonicalized parameter vector of an Infinigen generator as the denoising target of a small DiT, conditioned on DINOv2 tokens via cross-attention. That combination is not in the cited prior work, and it is a sensible design. Qualitative results across six categories look consistent, and the efficiency numbers (7.6M params, ~30 GPU hours, seconds at inference) are plausible for parameter-space diffusion. The authors also explicitly name the generator-expressiveness ceiling in Section 4.5. That honesty earns credit.\n\nSoft spots, in order of size. First, the abstract and intro say 'recovering parameters accurately,' but Table 1 reports only CD, EMD, F-Score. The parameter-to-geometry map is not injective, so low CD does not imply correct parameters. A simple predicted-vs-ground-truth normalized parameter error on the test split would close this. Second, the ShapeNet evaluation is post-hoc filtered to 218 chairs, dropping exactly the cases the generator cannot express—acceptable as a generalization probe only within the generator's reach, and the paper should say that in the main body. Third, quantitative metrics cover one of six categories, there are no error bars or variance numbers, and no code/weights are released. The MCMC comparison is a single qualitative figure; the efficiency advantage over MCMC is plausible but unmeasured. None of this breaks the central idea; it is an evaluation gap, not a load-bearing flaw.\n\nThis is a paper I'd send to a serious referee—the novelty is clear and the limitation is acknowledged. I'd want the parameter-accuracy table and a more careful framing of the ShapeNet filter before accepting. For my own work, I'd cite it as related work, with a caveat on the quantitative claims.","headline":"New diffusion-over-parameters idea for inverse PCG, but the claimed parameter accuracy is never directly measured and the ShapeNet eval is filtered—send to review with requests for a parameter-error table.","tokens_in":17032,"tokens_out":3125,"would_cite":true,"duration_ms":18335,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 7.6M-parameter diffusion model can invert procedural generation from a single image, producing editable 3D assets in seconds.","keywords":["inverse procedural content generation","diffusion model","Diffusion Transformer","image-to-3D generation","parametric 3D representation","DINOv2 conditioning","procedural generator","single-image 3D reconstruction"],"falsifier":"Use the unfiltered ShapeNet chair test split, which includes sofa-like and artistic chairs outside the Infinigen chair generator's range, and measure F-Score and Chamfer distance for DI-PCG; if the scores collapse toward baseline levels or a human judge sees the output as a different chair type, the method is not generally inverting the observed geometry but only selecting among expressible variants.","tokens_in":15892,"feed_emoji":"🪑","tokens_out":4899,"duration_ms":35355,"temperature":0.7,"pith_summary":"DI-PCG claims that inverse procedural content generation does not need thousands of sampling iterations: a small diffusion model can learn the posterior over a generator's parameters directly and sample it in seconds from a single image. The paper's core claim is that by treating normalized generator parameters as the denoising variable and injecting a DINOv2 encoding of the observed image as the condition, a 7.6M-parameter Diffusion Transformer trained for about 30 GPU hours recovers parameters accurately and generalizes to internet images. On the chair test split it reports Chamfer distance 0.033, EMD 0.028, and F-Score 0.896, well ahead of large reconstruction baselines. A sympathetic reader would care because this reframes image-to-3D as choosing parameters of an explicit, editable procedural program rather than hallucinating geometry from scratch.","feed_headline":"Tiny diffusion model turns images into 3D assets in seconds","feed_subtitle":"A 7.6M-parameter network recovers procedural-generator parameters, beating large baselines on chairs.","key_machinery":"The load-bearing object is the pair consisting of a procedural generator and its canonicalized parameter vector. The reversible projection $\\phi$ maps generator parameters into $[-1,1]^N$, making mixed continuous and discrete parameters a single diffusion data space, while the Diffusion Transformer denoiser treats that vector as a token sequence and injects DINOv2 image patch features through cross-attention. This machinery carries the argument because the diffusion model amortizes the inverse problem: instead of running the generator thousands of times and scoring each result, one learned model predicts the full posterior and samples it in a few seconds.","core_discovery":"The central discovery is that a procedural generator's parameter vector is itself a compact, semantically meaningful 3D representation that a diffusion model can learn to sample. Discrete parameters are converted to continuous pieces of the interval $[-1,1]$ and continuous parameters are normalized into $[-1,1]$ through a reversible projection $\\phi$, so the whole parameter set becomes a single data space $x \\in [-1,1]^N$. DI-PCG trains a Diffusion Transformer to predict noise on $x$ conditioned on DINOv2 patch tokens via cross-attention, then maps sampled parameters back through $\\phi^{-1}$ into the generator. The resulting method, with 7.6M parameters and six category-specific models trained on 18,000 synthetic image-parameter pairs each, matches or beats much larger image-to-3D systems on the chair benchmark while keeping outputs clean, editable, and consistently meshed.","pith_inferences":["The paper's evaluation filters ShapeNet chairs to exclude sofa-like or artistic chairs; an unfiltered benchmark would likely show a sharp performance drop and would better expose the generator-expressiveness ceiling.","The same conditional diffusion-over-parameters recipe could transfer to other procedural domains with explicit parameters, such as CAD programs, materials, or city layouts, where the usual bottleneck is a scoring function or gradient.","Combining DI-PCG-style inverse models with automatically generated procedural programs would attack the stated limitation directly, letting the parametric representation itself expand.","Parameter-space diffusion also gives a natural diversity knob: sampling multiple times from the posterior yields several plausible assets for one image, something single-shot regressors do not offer."],"forward_implications":["Image-to-3D becomes controllable: sampled parameters can be edited one at a time, such as leg height or arm presence, and re-run through the generator.","Inference cost drops to seconds with only 7.6M parameters, making inverse PCG practical on a single GPU or in resource-constrained settings.","Because training data come from the generator itself, no external 3D collection is needed, and any procedural generator can be paired with DI-PCG with nearly no code change.","Sketch inputs work as well as RGB images, giving designers a simple way to specify shapes.","Quantitative results suggest that on categories a generator can express, parameter-space diffusion can outperform much larger feed-forward reconstruction models."],"supporting_citations":[{"why":"Supplies the procedural generators whose parameters and renders form the training data for DI-PCG.","marker":"[59]"},{"why":"Supplies additional procedural generators such as basket, flower, and dandelion used in the experiments.","marker":"[60]"},{"why":"Provides the pre-trained visual features that serve as the condition injected into the diffusion model.","marker":"[50]"},{"why":"Defines the Diffusion Transformer backbone used as the denoising network.","marker":"[53]"},{"why":"Serves as the large reconstruction baseline that DI-PCG outperforms on the quantitative chair metrics.","marker":"[87]"},{"why":"Serves as a multi-view diffusion based image-to-3D baseline for qualitative and quantitative comparison.","marker":"[35]"},{"why":"Serves as a 3D-native diffusion baseline that DI-PCG compares against on chairs and in-the-wild categories.","marker":"[31]"},{"why":"Serves as a latent 3D diffusion baseline used in the quantitative chair evaluation.","marker":"[93]"}],"fun_headline_variants":["Tiny diffusion maps images to editable procedural 3D parameters","7.6M-parameter diffusion recovers PCG params from images","Inverse PCG at 30 GPU hours: DI-PCG matches huge 3D models","Diffusion transformer decodes procedural knobs from image patches"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the procedural generator's parameter family can actually express the observed object; if no parameter combination yields the target shape, DI-PCG returns the nearest expressible cousin rather than the true asset.","fun_headline_variants_meta":{"raw":{"variants":["Tiny diffusion maps images to editable procedural 3D parameters","7.6M-parameter diffusion recovers PCG params from images","Inverse PCG at 30 GPU hours: DI-PCG matches huge 3D models","Diffusion transformer decodes procedural knobs from image patches"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000783,"raw_usage":{"total_tokens":3458,"prompt_tokens":947,"completion_tokens":2511,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":2432}},"tokens_in":563,"tokens_out":2511,"duration_ms":17370,"temperature":1.0,"reasoning_tokens":2432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:33:19.193752+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use the unfiltered ShapeNet chair test split, which includes sofa-like and artistic chairs outside the Infinigen chair generator's range, and measure F-Score and Chamfer distance for DI-PCG; if the scores collapse toward baseline levels or a human judge sees the output as a different chair type, the method is not generally inverting the observed geometry but only selecting among expressible variants.","supporting_citations":[{"cited_title":"Infinite photorealistic worlds using procedural generation","cited_arxiv_id":null,"evidence_quote":"Supplies the procedural generators whose parameters and renders form the training data for DI-PCG."},{"cited_title":"In- finigen indoors: Photorealistic indoor scenes using procedu- ral generation","cited_arxiv_id":null,"evidence_quote":"Supplies additional procedural generators such as basket, flower, and dandelion used in the experiments."},{"cited_title":"Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation","cited_arxiv_id":null,"evidence_quote":"Serves as a latent 3D diffusion baseline used in the quantitative chair evaluation."}],"review_version":1}