{"id":"feb974b7-4785-4810-ad49-f9d0aa84f188","arxiv_id":"2412.06089","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A generate-plan-edit wrapper, using GPT-4o to plan atomic edits and a diffusion editor to execute them, improves compositional text-to-image fidelity across many T2I models.","lead":"GraPE wraps text-to-image generation in a three-stage loop: generate an image, ask a multimodal language model to list the visual mistakes as small edit steps, then apply those steps one by one with an image editor. The paper reports that this modular, training-free add-on improves compositional prompt following on three benchmarks across 13 T2I models, with the largest gains on weaker generators.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline gains rest on GPT-4o judging images edited under GPT-4o's own plans; no human final-image check, so reported DSG improvements may overstate real fidelity.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: GPT-4o functions as both planner and judge, and the human study does not directly score final GraPE images. I agree with that assessment. This is not an accusation of bad faith; it is a structural confound in the evaluation instrument. All headline comparisons, including the 'up to 3 points' SOTA improvement and the reduction of the weaker-to-stronger gap, are expressed in GPT-4o-judged DSG scores. If GPT-4o preferentially rewards edits that match its own planned corrections, the gains could be inflated even though the pipeline is genuinely modular, training-free, and reproducible. The paper deserves credit for the breadth of models tested, the PixEdit ablation, the separate editing-benchmark evaluation, and the detailed error analysis, all of which suggest the method has real merit. But the central quantitative claim cannot be accepted at face value until an independent human or at least a different VLM judge confirms the final-image fidelity gains. The proposed human DSG test is a direct, feasible check: it uses the same decomposed questions and the same samples, so it isolates the judge effect. If the human results match GPT-4o, the concern is resolved and the conditional verdict can be upgraded. If they do not, the paper's headline numbers would need downward revision. Because the concern is real but addressable, the appropriate verdict remains conditional rather than acceptance or rejection.","tokens_in":23330,"tokens_out":7021,"duration_ms":73872,"concrete_test":"Take a random subset of 100 ConceptMix K=7 prompts and 100 T2I-Compbench prompts for SDXL and SD3.5-Large. Generate base and GraPE-PixEdit images, then have three independent human annotators answer the same decomposed yes/no DSG questions for both image sets, blinded to method and without seeing the edit plan. Compute the human-judged DSG gain of GraPE over base. If the human-judged gain is not significant or is less than half the GPT-4o-judged gain, the self-as-judge concern is confirmed and the headline claim must be tempered. If human gains match GPT-4o gains, the circularity concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that GraPE improves compositional prompt fidelity for arbitrary T2I generators is measured almost entirely by GPT-4o QA: DSG scores on T2I-Compbench/Flickr-Bench (Fig 3, Tables 10-11) and GPT-4o VQA on ConceptMix questions (Table 2). The human study in Section 4.4 does not validate final images: Q1 asks about the pre-edit generated image, Q2 asks raters to judge the hypothetical effect of the plan, and Q3 asks about atomicity. None of these score the actual GraPE output. GPT-4o is therefore both the component that writes the corrective plan and the judge that decides whether the corrected image satisfies the prompt. Because the judge sees only the prompt and image, not the plan, the bias would be subtle: GPT-4o may be more likely to recognize its own intended corrections as present, even when the edit was not visually realized. Supplementary Section 10's own error breakdown is consistent with this worry: of 100 representative plans, only about 12.5 final images from initially erroneous cases are judged 'completely correct,' yet DSG scores for strong models sit above 0.85. A secondary issue is that the RPG baseline in Table 2 is reported only on the subset of samples where RPG produced valid plans (over 50% failure per Supp. Section 8), so the comparison is not on the same evaluation set.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GraPE, a training-free generate-plan-edit pipeline for compositional text-to-image generation. A base T2I model generates an initial image; a GPT-4o planner compares textual and visual object-attribute elements and emits atomic edit instructions; an instruction-guided editing model (AURORA or the authors' newly trained PixEdit) applies these edits sequentially. The authors evaluate on T2I-Compbench, ConceptMix, and Flickr-Bench across 13 T2I models, reporting DSG and GPT-4o QA gains, ablations of the planner prompt, a human study of plan quality, error analyses, and cost/runtime estimates.","tokens_in":23559,"tokens_out":7657,"duration_ms":72656,"significance":"GraPE is a simple and practical recipe: it requires no training, is modular across generator/editor pairs, and its object-centric planner decomposition is interpretable. The breadth of experiments (13 T2I models, two editors, three benchmarks) and the explicit cost and runtime analysis are strengths; the paper also introduces a new editor, PixEdit, and evaluates it on standard editing benchmarks. If the headline gains survive independent final-image evaluation, the approach would be a useful inference-time tool for improving compositional fidelity and for narrowing the gap between weaker and stronger generators.","major_comments":[{"comment":"The final-image evaluation is carried out almost entirely by GPT-4o QA (DSG in Fig. 3/Tables 10–11 and ConceptMix in Table 2), while GPT-4o is also the model that generates the edit plans. The human study in §4.4 (Table 4) does not score the actual GraPE output: Q1 asks about the pre-edit image, Q2 asks raters to judge the hypothetical effect of the plan, and Q3 asks about atomicity. This creates a self-consistency risk: the judge may be more likely to recognize its own intended corrections as present, even when the edit was not visually realized. Supp. §10's own breakdown (only about 12.5 of 100 final images from initially erroneous cases are judged completely correct) suggests that final-image correctness is not yet established. Please add a human evaluation of the final GraPE images or an evaluation with a different, independent VQA model, and report those scores alongside the GPT-4o scores.","section":"§4.4, Supp. §10, Fig. 3, Tables 10–11"},{"comment":"The RPG comparison is not on a comparable evaluation set. Supp. §8 states that RPG fails to generate valid plans for over 50% of samples and that the reported numbers are on the successfully generated samples only. Because the other rows in Table 2 and Fig. 3 are computed on the full sample, the RPG row is not an apples-to-apples baseline. Please either report RPG on the full sample (e.g., scoring failed samples as zero) or re-evaluate all methods on the common subset of prompts for which RPG produces valid plans.","section":"Supp. §8, Table 2, Fig. 3"},{"comment":"Several of the headline gains are within one standard deviation, e.g., Flux-schnell at K=7 (0.775±0.004 vs. 0.779±0.004) and SD3.5-Large at K=3 (0.815±0.002 vs. 0.817±0.002) in Table 2. Since the central claim is that GraPE improves all 13 models, the paper should provide paired significance tests or clearly restrict the claim to gains that are statistically distinguishable. The current 'improvements up to 3 points' phrasing overstates the evidence for the strongest models.","section":"Table 2 and Tables 10–11"}],"minor_comments":[{"comment":"The limitations paragraph says 'over 10 different models', but §4.1 lists 13 models; please correct the inconsistency.","section":"§5"},{"comment":"With `Ie0 ← Ig` and a plan of n edits, the loop `for k = 1, ..., n-1` executes only n-1 edits; either loop to n or index the edits from 0.","section":"Algorithm 1"},{"comment":"The T2I-Compbench and Flickr-Bench evaluations use 100-prompt subsets; please state how the prompts were sampled and, if feasible, report results on the full benchmarks or multiple sampling seeds.","section":"§4.1"},{"comment":"No inter-annotator agreement measure is reported for the human study; because Q2 and Q3 are subjective, please include an agreement metric such as Cohen's kappa.","section":"Table 4"},{"comment":"The column header 'GraPE PixEdit' is ambiguous; clarify that the two numeric columns show the base model and GraPE-with-PixEdit, respectively.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The central risk is the judge/planner self-consistency: GPT-4o both writes the corrective plans and scores the final images. This is not a fitted-parameter circularity, but it is a real correctness risk for the paper's main claim. If the authors add independent final-image evaluation (human or a different MLLM) and address the RPG comparison, the paper could be a strong contribution; I would not support acceptance without that validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a solid, useful pipeline paper whose headline numbers are more likely optimistic than wrong. The generate-plan-edit idea is not brand new—SLD and LMD/RPG already do LLM-planned refinement—but the object-centric atomic edit decomposition plus a pixel-space editor is a genuinely different combination, and the sweep across 13 generators with two editors is real work. The thing to know before citing the DSG numbers: GPT-4o is both the planner and the judge, and no human ever scores the final image. That makes me trust the direction of the results more than the magnitudes.\n\nWhat is actually new: a training-free, modular post-hoc correction loop that decomposes corrective steps into atomic object-level edits, executed by an external editor; a planner prompt that separates textual/image element analysis, error identification, and feedback generation; and PixEdit, a PixArt-Sigma-based editor trained on the Seed-Edit real subset plus the AURORA data mixture, benchmarked on AURORA-BENCH and MagicBrush. The ablations (object-centric planner vs naive planner) and the human plan-quality study are useful, and the cost/runtime table is honest.\n\nSoft spots, in order of severity. First, the evaluation circularity is real: the human study in Sec 4.4 asks about the initial image (Q1), the hypothetical effect of the plan (Q2), and plan atomicity (Q3); none of it grades the actual GraPE output. The supplementary's own breakdown—about 12.5 of 100 initially erroneous cases end 'completely correct' after editing—sits awkwardly next to DSG scores above 0.85 for strong models. A human final-image rating or an independent judge (Qwen-VL, for instance) would settle this. Second, the RPG baseline in Table 2 is reported only on samples where RPG produced valid plans (over 50% fail per Supp. Section 8), so that comparison is not on a common evaluation set; it should be flagged in the main text. Third, many gains for already-strong models are within one standard error (e.g., DALL·E 3 at K=7: 0.728 to 0.737); the consistent upward trend across all 13 models is the real evidence, not any single cell.\n\nBottom line: this deserves a serious referee. The framework is plausible, the evidence is broad, and the central claim holds up conditional on independent final-image evaluation. If the authors add that, I'd take the numbers at face value. I'd cite it for the modular pipeline idea and the PixEdit training recipe.","headline":"Solid modular post-hoc correction pipeline with a real evaluation gap: GPT-4o is both planner and judge, so expect the headline gains to shrink under independent scoring.","tokens_in":24155,"tokens_out":2827,"would_cite":true,"duration_ms":26199,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's central claim is that complex text-to-image prompts are better handled by generating a rough draft, planning object-level corrections with a multimodal LLM, and executing them as a sequence of atomic edits.","keywords":["generate-plan-edit","compositional text-to-image generation","multimodal LLM planning","image editing","training-free inference","diffusion models","prompt fidelity","object-centric edit plans"],"falsifier":"Take a random sample of ConceptMix K=7 prompts, run the base generator and GraPE with the same generator, and have independent human raters judge which image better follows the prompt; the central claim fails if human preference does not track the reported GPT-4o QA improvements.","tokens_in":23106,"feed_emoji":"🎨","tokens_out":5202,"duration_ms":45220,"temperature":0.7,"pith_summary":"This paper claims that the best way to follow a complex, multi-step text prompt is not to generate the image in one pass, but to generate a rough draft, ask a multimodal language model to list the specific object-level mistakes, and then fix them one by one with an off-the-shelf image editor. The authors argue this generate-plan-edit loop is training-free and modular, so any text-to-image model and any editing model can be paired at inference time. If the claim is right, the practical route to better prompt adherence is compute at inference time rather than new training runs, and weaker generators can be brought much closer to stronger ones. The paper reports gains on three benchmarks across thirteen generators, including up to roughly twenty percentage points for smaller models.","feed_headline":"Plan-then-edit lifts prompt fidelity of any text-to-image model","feed_subtitle":"Any generator plus an off-the-shelf editor can be combined at inference time to follow complex prompts better.","key_machinery":"The load-bearing object is the edit plan produced by the planner. The planner is prompted to decompose its assessment into four named stages -- textual elements, image elements, mistake identification, and feedback generation -- and the feedback instructions are constrained to be atomic and object-centric. This decomposition is what lets a general-purpose editor fix a complex prompt through a series of simple, composable edits. The second piece of machinery is PixEdit, a text-guided editor built on PixArt-Sigma with a T5-XXL text encoder and trained on reasoning-centric editing data, which the paper shows outperforms other editors inside the GraPE loop.","core_discovery":"GraPE's central discovery is that a sequential, object-centric edit plan can repair a partially faithful generation. Given a text prompt $T$ and an initial image $I_g$ from any diffusion model, a multimodal LLM produces a structured plan: extract the objects and attributes mentioned in the text, extract what is actually present in the image, identify the discrepancies, and emit a list of atomic corrective instructions $T_{e_1},\\ldots,T_{e_n}$. An editing model then executes these instructions in order, $I_{e_{k+1}} = E(I_{e_k}, T_{e_{k+1}})$, to produce the final image. The authors show this decomposition improves prompt following for every one of the thirteen generators tested, with larger absolute gains on weaker models, and that the average plan length grows with prompt complexity, meaning the planner recognises when more corrective steps are needed.","pith_inferences":["Because the loop is modular, the same planner could be run iteratively until the planner reports no remaining mistakes, converting a fixed plan into a convergence loop; the paper does not test this.","The evaluation design leaves open a self-consistency risk: if the judge and the planner share the same visual priors, the reported gains may partly reflect agreement with GPT-4o's own editing taste rather than human-visible fidelity.","The error analysis isolates the editor as the main bottleneck, which suggests that investing in better compositional editors -- or adding an RL feedback loop on failed edits -- may yield larger gains than improving the planner.","A similar generate-plan-edit loop could extend to other generative modalities, like video or 3D scenes, wherever a 'planner' can inspect an intermediate output and an 'editor' can apply atomic corrections."],"forward_implications":["Any existing text-to-image model can be improved at inference time by appending a planner and an editor, with no retraining of the generator.","Weak generators gain more than strong ones, so the performance gap between smaller and state-of-the-art models narrows under GraPE.","Performance on compositional prompts rises with the number of editing steps, giving a direct compute-to-fidelity trade-off.","The gains are not limited to compositional benchmarks; the paper reports improvements on Flickr-Bench prompts as well.","The planner's mistake analysis can itself serve as an evaluation signal, correlating with the LLMScore metric."],"supporting_citations":[{"why":"T2I-CompBench provides the compositional benchmark with six categories used to evaluate GraPE's prompt-following improvements.","marker":"[27]"},{"why":"ConceptMix supplies the controllable-complexity benchmark whose K-value drives prompt difficulty and whose QA questions serve as the evaluation metric.","marker":"[59]"},{"why":"InstructPix2Pix established the training strategy of adding input-image channels that PixEdit and the editing pipeline build on.","marker":"[6]"},{"why":"AURORA contributes the reasoning-centric editing dataset used for PixEdit's fine-tuning and serves as a strong editing baseline inside GraPE.","marker":"[30]"},{"why":"PixArt-Sigma is the backbone diffusion model for PixEdit, bringing a stronger base and a T5-XXL text encoder that improves compositional editing.","marker":"[11]"},{"why":"Davidsonian Scene Graph grounds the yes/no QA scoring used on T2I-CompBench and Flickr-Bench, the evaluation framework for GraPE's headline results.","marker":"[14]"},{"why":"LLMScore is the metric the planner-as-evaluator is compared against, establishing the planner's potential as a scoring tool.","marker":"[36]"},{"why":"SLD is the post-hoc self-correction baseline that GraPE is compared against in the supplementary, positioning GraPE against existing correction approaches.","marker":"[57]"}],"fun_headline_variants":["Plan-then-edit lifts fidelity of any T2I model","Training-free plan-edit improves all text-to-image models","MLLM plans corrective edits to fix T2I generations","Generate, plan, edit: a recipe for faithful T2I","Any diffusion generator improves with a planned edit"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All headline gains are measured by GPT-4o answering yes/no questions about whether the image matches the prompt, and the same model also wrote the edit plans; the human study rates plan quality rather than the correctness of the final images.","fun_headline_variants_meta":{"raw":{"variants":["Plan-then-edit lifts fidelity of any T2I model","Training-free plan-edit improves all text-to-image models","MLLM plans corrective edits to fix T2I generations","Generate, plan, edit: a recipe for faithful T2I","Any diffusion generator improves with a planned edit"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000397,"raw_usage":{"total_tokens":2143,"prompt_tokens":1077,"completion_tokens":1066,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":983}},"tokens_in":693,"tokens_out":1066,"duration_ms":8963,"temperature":1.0,"reasoning_tokens":983,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:01:21.503942+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of ConceptMix K=7 prompts, run the base generator and GraPE with the same generator, and have independent human raters judge which image better follows the prompt; the central claim fails if human preference does not track the reported GPT-4o QA improvements.","supporting_citations":[{"cited_title":"T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation, 2023","cited_arxiv_id":null,"evidence_quote":"T2I-CompBench provides the compositional benchmark with six categories used to evaluate GraPE's prompt-following improvements."},{"cited_title":"Conceptmix: A compositional image generation benchmark with controllable difficulty, 2024","cited_arxiv_id":null,"evidence_quote":"ConceptMix supplies the controllable-complexity benchmark whose K-value drives prompt difficulty and whose QA questions serve as the evaluation metric."},{"cited_title":"Learning action and reasoning-centric image editing from videos and simulations, 2024","cited_arxiv_id":null,"evidence_quote":"AURORA contributes the reasoning-centric editing dataset used for PixEdit's fine-tuning and serves as a strong editing baseline inside GraPE."},{"cited_title":"Pixart- σ: Weak-to-strong training of diffu- sion transformer for 4k text-to-image generation, 2024","cited_arxiv_id":null,"evidence_quote":"PixArt-Sigma is the backbone diffusion model for PixEdit, bringing a stronger base and a T5-XXL text encoder that improves compositional editing."},{"cited_title":"Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image gener- ation, 2024","cited_arxiv_id":null,"evidence_quote":"Davidsonian Scene Graph grounds the yes/no QA scoring used on T2I-CompBench and Flickr-Bench, the evaluation framework for GraPE's headline results."},{"cited_title":"LLMScore: Unveiling the power of large language models in text-to-image synthesis evaluation","cited_arxiv_id":null,"evidence_quote":"LLMScore is the metric the planner-as-evaluator is compared against, establishing the planner's potential as a scoring tool."}],"review_version":1}