{"id":"308be970-fec4-4f19-8247-5c7974a036c0","arxiv_id":"2411.13019","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A training-free pipeline uses text queries to complete occluded objects into transparent RGBA images, and it wins human preference over three prior methods on a new 553-class benchmark.","lead":"This paper builds a text-guided system that reconstructs hidden parts of objects in photos by chaining pretrained models for segmentation, occlusion detection, and inpainting. It reports that people prefer its completions over three existing methods on a new set of 2,379 images covering 553 object categories.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The open-world claim rests on an unverified transfer: InstaOrderNet's binary occlusion decisions (Eq. 2) are applied to open-vocabulary masks and morphological background segments, with no reported accuracy or oracle ablation; a wrong ordering directly corrupts the inpainted completion.","rationale":"The reader's weakest assumption identifies the same load-bearing premise: InstaOrderNet's pairwise occlusion decisions must transfer from its training distribution to every segment the pipeline feeds it. I agree and would sharpen the point further. The framework is modular, so the inpainting and prompt stages can be excellent while the final output is still wrong if Eq. (2) mislabels an occluder. No experiment in the paper isolates this module: the ablations vary prompt selection and background-segment inclusion, but they never replace InstaOrderNet with ground-truth occlusion labels or measure its per-pair accuracy on the new benchmark. The quantitative CLIP metric is partly circular and the human study lacks significance tests, but the human preference result is the main evidence and it is not undermined by those secondary issues. If the proposed oracle-ablation shows that InstaOrderNet errors do not change outputs, the concern is resolved; if it shows a large gap, the open-world claim needs direct validation of the occlusion module. This does not move the verdict because the paper already merits a conditional verdict on the grounds the reader gave, but it identifies the specific experiment that would convert the condition into a firm acceptance.","tokens_in":15899,"tokens_out":5364,"duration_ms":58973,"concrete_test":"On 50–100 images stratified across VG, COCO-A, LAION, and free images, run the full pipeline twice: once with Eq. (2) masks from InstaOrderNet and once with Mocc hand-annotated by three raters for the same target segments, keeping the inpainting and prompt stages fixed. Compare LPIPS plus a small forced-choice human preference between the two variants. If the oracle-vs-predicted outputs are nearly identical, InstaOrderNet transfer is not load-bearing; if they diverge materially, the open-world claim requires direct per-category validation of Eq. (2) before acceptance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—training-free amodal completion for arbitrary text-queried objects—requires the occlusion mask Mocc in Eq. (2) to be correct. Mocc is built from binary decisions of InstaOrderNet applied to every segment in S∪B, including masks produced by Grounded-SAM and morphological background blobs that InstaOrderNet was not trained to interpret. The paper reports no accuracy or agreement for InstaOrderNet on this segment inventory, and it does not ablate the occlusion-order module against an oracle. Because inpainting is only meaningful where Mocc marks true occluders, an erroneous binary decision either leaves the occluder's pixels untouched (missing the occluded region) or masks out visible target pixels that then get hallucinated. The supplementary failure table attributes the 4.1% complete failures to undetectable occluded objects, not to occlusion-order errors, so this potential failure mode is not disentangled. Thus the open-world generalization is only as strong as the unverified transfer of a pairwise-depth-order model to open-vocabulary and background segments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free framework for open-world amodal appearance completion. Given an image and a natural-language query, the pipeline uses LISA to obtain a visible mask, Grounded-SAM plus morphological background partitioning to enumerate candidate segments, InstaOrderNet to determine which segments occlude the target (Eq. 2), CLIP to select an inpainting prompt (Eq. 4), and Stable Diffusion inpainting with iterative refinement to synthesize the occluded appearance, finally outputting an RGBA layer. The authors introduce a new evaluation dataset of 2379 images spanning 553 classes and report a human preference study (Table 2), quantitative metrics (Table 3), and ablations (Table 4).","tokens_in":16062,"tokens_out":5112,"duration_ms":45937,"significance":"If the results hold, this is a useful step toward open-world amodal completion: it moves beyond closed category sets, integrates off-the-shelf models without fine-tuning, and produces compositable RGBA outputs. The new benchmark with natural occlusions and the human-preference evaluation are valuable assets. However, the central empirical claim rests on a preference study without statistical inference, and the occlusion-analysis module is not validated on the pipeline's own segment inventory; these issues need to be addressed before the claims are fully supported.","major_comments":[{"comment":"The headline result that 'Ours' is preferred (41.86% overall vs. 27.95% for Pix2gestalt) is reported without confidence intervals or significance tests. Given that each image appears to be rated by only three participants (per Fig. 5), the 13.9-point gap could be within noise for a subset of images; the paper should provide per-image bootstrap or subject-level confidence intervals and a paired test (e.g., Wilcoxon across images).","section":"§4.3, Table 2"},{"comment":"The occluder mask is built from binary decisions of InstaOrderNet applied to every segment in S∪B, including masks produced by Grounded-SAM and morphological background blobs. No accuracy or agreement measure is reported for InstaOrderNet on this segment inventory, and there is no oracle ablation replacing it with ground-truth occlusion order. Because an incorrect binary decision either leaves an occluder unpainted or masks visible target pixels that then get hallucinated, the open-world claim is only as strong as this unverified transfer. The supplementary failure analysis (Sec. 7) attributes 4.1% complete failures to undetectable objects, not to occlusion-order errors, so this failure mode is not disentangled.","section":"§3.2, Eq. (2)"},{"comment":"The CLIP score metric in Table 3 is partly circular. The evaluation uses the ground-truth class label as the query Q, and Eq. (4) selects the inpainting prompt from T∪{Q} with CLIP; the same label is then matched against the output with CLIP. This inflates the reported CLIP scores, particularly for 'Ours', and makes the cross-method comparison on this metric difficult to interpret. A fairer protocol would blind prompt selection to the evaluation label or use a distinct text encoder for evaluation.","section":"§3.3 vs. §4.2"}],"minor_comments":[{"comment":"The morphological operations (erosion and dilation structuring element sizes, number of passes) are not specified; please report them in the supplementary configuration details for reproducibility.","section":"§3.1, Eq. (1)"},{"comment":"The human study should state how many participants rated each image, whether each image was rated by the same set of three participants, and whether the text query was shown to participants; Fig. 5 implies three raters per image, but this is not stated in the main text.","section":"§4.3"},{"comment":"The symbols '!' and '#' are not defined in the table caption; the row 'T ∪ Q #' is ambiguous about whether background segments are included or excluded.","section":"Table 4"},{"comment":"There are several typos: 'an segmentation process' should be 'a segmentation process', 'our method of consider' should be 'our method to consider', and 'evalution' should be 'evaluation' in §4.4.","section":"§3.1 and §4.4"},{"comment":"The sentence 'In our evalution dataset, the Q prompt achieves the highest CLIP score' should explicitly point to Table 4 and clarify that the T∪Q row with background segments is the chosen configuration despite having a slightly lower CLIP score.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper promises code and dataset release after acceptance. Given that the evaluation dataset is central to the claims and the paper relies on many pre-trained components, I would encourage the editor to require the release as part of the revision. There is no obvious citation or novelty concern beyond the incremental framing of 'reasoning amodal completion.'"},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new integration—text-query, training-free amodal completion with RGBA output—backed by a human preference study that's more careful than most in this area. The 553-class evaluation set is a real contribution, and the authors are honest about failure modes. But the paper overreaches slightly in its open-world claim: the pipeline's only non-trivial learned module for occlusion, InstaOrderNet, is applied to open-vocabulary masks and morphology-derived background blobs with zero direct validation. The CLIP score is partly circular, and the preference study lacks significance tests or confidence intervals. Still, the central idea holds together well enough to warrant serious review.\n\nWhat's actually new: the combination of LISA-style reasoning segmentation, Grounded-SAM context masks, InstaOrderNet for pairwise ordering, background blobs as potential occluders, and progressive inpainting with auto-generated prompts is not in any prior work. PD-MC is category-limited, Pix2gestalt is trained and text-free, and LISA doesn't complete occlusions. The RGBA output enabling downstream editing and 3D is a nice practical touch. The benchmark construction from four sources with annotator-provided classes is a useful asset, provided the data is released.\n\nWhere it's soft: first, the occlusion-ordering module is load-bearing. Eq. (2) unions any segment InstaOrderNet marks as occluding, and an error in that binary decision either leaves occluder pixels untouched or erases visible target pixels that then get hallucinated. The paper reports no accuracy for InstaOrderNet on the segment inventory, no oracle ablation, and the failure analysis doesn't separate ordering errors from undetectable objects. That's a real gap, though the strong preference results suggest the module is usually right. Second, the CLIP relevance metric is circular: the same query label selects the prompt (Eq. 4) and is the evaluation text. The difference over baselines is small, and the authors themselves call the appearance metrics 'for reference only.' Third, the 41.86% vs 27.95% preference margin is convincing at face value, but no intervals or tests are given, and Fleiss' kappa around 0.32 is only 'fair.' Fourth, code and datasets are promised post-acceptance—for a system whose value is mostly empirical, that's a meaningful reproducibility risk.\n\nBottom line: this is a promising system paper, not a closed benchmark. It deserves a serious referee. Revisions should require open code/data, significance testing, and an occlusion-order ablation (or at least per-source accuracy of InstaOrderNet on the pipeline's masks). If those check out, it's an accept.","headline":"A genuinely new integration for text-query amodal completion with a reasonable human study, but the occlusion-ordering step is the least-evidenced link in the chain.","tokens_in":16673,"tokens_out":2830,"would_cite":true,"duration_ms":27746,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Text plus image reconstructs hidden objects across any category","keywords":["amodal completion","open-world","reasoning amodal completion","text-guided segmentation","occlusion ordering","inpainting","RGBA output","human preference study"],"falsifier":"Take the pipeline's 2,379 evaluation images, run InstaOrderNet alone over the same segment inventory the pipeline uses, and compare its pairwise occlusion labels with human judgments; if its accuracy on novel classes or background segments is near chance, then the occluder mask is initialized from wrong pixels and the claimed completions cannot be attributed to correct occlusion reasoning.","tokens_in":15630,"feed_emoji":"🖼️","tokens_out":6993,"duration_ms":58502,"temperature":0.7,"pith_summary":"This paper tries to show that a user can take an ordinary photo, type a phrase naming any object—specific like 'polar bear' or abstract like 'what is the mammal in this image'—and receive a complete, transparent RGBA version of that object with its occluded parts filled in, with no additional training. The central claim is that segmentation, occlusion ordering, prompt selection, and iterative inpainting can be chained from off-the-shelf components to achieve this in the open world, beyond any fixed category set. If the claim holds, amodal completion stops being a closed-set recognition problem and becomes a language-driven image reconstruction service usable for editing, 3D reconstruction, and augmented reality. The paper supports the claim with a 2,379-image, 553-class evaluation set and a human preference study in which its outputs are chosen 41.86% of the time overall against 27.95% for the strongest baseline.","feed_headline":"Text plus image reconstructs hidden objects across any category","feed_subtitle":"A text query alone completes occluded objects; the method wins 41.9% human preference.","key_machinery":"The load-bearing mechanism is the occluder mask $M_{\\text{occ}}$, built by InstaOrderNet's pairwise occlusion decisions over every object mask and morphological background segment (Eq. 2), with boundary-aware dilation when the target touches the image edge (Eq. 3). This mask tells the inpainting model which pixels are hidden by other things and therefore must be reconstructed. The other carrier is the prompt-selection step (Eq. 4), which uses CLIP to choose the descriptor from the image tags or the user query that best matches the visible target, so abstract queries still yield a concrete inpainting condition. The pipeline then iterates masked inpainting with adaptive termination.","core_discovery":"The paper's core discovery is that reasoning-based amodal completion can be decomposed into four frozen components that need no training: a vision-language model turns the text query into a visible mask for the target object; an open-set tagging and detection chain inventories every nameable object while morphological operations partition ambiguous background into segments; an occlusion-ordering network assigns each segment a binary 'occludes the target' label to form the occluder mask; and an inpainting diffusion model, guided by a CLIP-selected prompt, iteratively fills the masked region and outputs an RGBA layer. The authors argue that because the query, rather than a predefined class list, selects the target, the framework generalizes across 553 object classes and produces completions that human raters prefer to existing closed-set and diffusion-based baselines. The central output is not just an inpainted image but a layerable transparent element ready for downstream composition.","pith_inferences":["Editorial inference: the paper never measures InstaOrderNet's own accuracy on its segment inventory, so a natural extension is to ablate the pipeline by replacing the occlusion-order network with human-annotated occlusion labels; the gap would reveal how much of the final quality rests on that one component.","Editorial inference: the same chain could be applied to video frames, but temporal consistency of completed appearances would need an explicit constraint that the current single-image pipeline does not provide.","Editorial inference: completions could serve as pseudo-ground-truth for distilling a smaller, faster end-to-end model, since the current engine runs several large pre-trained models per object.","Editorial inference: the morphological handling of unlabeled background regions is what lets the method treat foliage and blur as occluders, a setting where closed-set baselines fail; a targeted stress test on heavy-background occlusion would make this contribution stand out."],"forward_implications":["A text query plus an image becomes enough to recover a full amodal appearance for objects from categories never seen in training.","Abstract or context-dependent queries, such as 'the mammal in this image', produce completions without requiring the user to know the class name.","RGBA outputs let the completed object be composited directly into image editing, novel-view synthesis, and 3D reconstruction workflows.","Because each module is frozen and replaceable, the whole framework improves automatically when better segmentation, occlusion-ordering, or inpainting models appear.","The new dataset and preference protocol provide a common benchmark for open-world amodal appearance completion."],"supporting_citations":[{"why":"Supplies the vision-language model that turns the text query into the visible mask of the target object.","marker":"[14]"},{"why":"Provides the automatic annotation framework that inventories all nameable objects in the image.","marker":"[27]"},{"why":"Supplies the open-set image tagger whose tags drive both object detection and inpainting prompt selection.","marker":"[10]"},{"why":"Open-set detector used, with SAM, to generate the object mask set for occlusion analysis.","marker":"[20]"},{"why":"Segment Anything produces the pixel-level masks for objects and background.","marker":"[12]"},{"why":"InstaOrderNet computes the pairwise occlusion order that determines which segments occlude the target.","marker":"[16]"},{"why":"CLIP both selects the inpainting prompt by visual-text similarity and scores class relevance in evaluation.","marker":"[26]"},{"why":"Stable Diffusion v2 inpainting fills the occluded region at each iterative step.","marker":"[28]"},{"why":"PD-MC is the progressive mixed-context baseline and the source of the background-isolation and iterative inpainting strategy.","marker":"[35]"},{"why":"Pix2gestalt is the supervised amodal completion baseline that the method must beat in the preference study.","marker":"[23]"}],"fun_headline_variants":["Training-free amodal completion from any text query","Open-world amodal completion via a text prompt, no training","Text query alone completes hidden objects across any category","Reasoning-based amodal completion for arbitrary objects","No training needed: text query completes occluded objects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result rests on the occlusion-ordering network's binary pairwise decisions being trustworthy for every segment the pipeline feeds it, including masks from open-vocabulary detectors and morphological background blobs, even though the paper never reports how often those decisions are correct on its own segment inventory.","fun_headline_variants_meta":{"raw":{"variants":["Training-free amodal completion from any text query","Open-world amodal completion via a text prompt, no training","Text query alone completes hidden objects across any category","Reasoning-based amodal completion for arbitrary objects","No training needed: text query completes occluded objects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001152,"raw_usage":{"total_tokens":4750,"prompt_tokens":894,"completion_tokens":3856,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":3781}},"tokens_in":510,"tokens_out":3856,"duration_ms":73536,"temperature":1.0,"reasoning_tokens":3781,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:56:01.176205+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the pipeline's 2,379 evaluation images, run InstaOrderNet alone over the same segment inventory the pipeline uses, and compare its pairwise occlusion labels with human judgments; if its accuracy on novel classes or background segments is near chance, then the occluder mask is initialized from wrong pixels and the claimed completions cannot be attributed to correct occlusion reasoning.","supporting_citations":[{"cited_title":"Lisa: Reasoning segmentation 6 via large language model","cited_arxiv_id":null,"evidence_quote":"Supplies the vision-language model that turns the text query into the visible mask of the target object."},{"cited_title":"Open-set image tagging with multi-grained text su- pervision","cited_arxiv_id":null,"evidence_quote":"Supplies the open-set image tagger whose tags drive both object detection and inpainting prompt selection."},{"cited_title":"Grounding dino: Marrying dino with grounded pre-training for open-set object detection","cited_arxiv_id":null,"evidence_quote":"Open-set detector used, with SAM, to generate the object mask set for occlusion analysis."},{"cited_title":"Segment any- thing","cited_arxiv_id":null,"evidence_quote":"Segment Anything produces the pixel-level masks for objects and background."},{"cited_title":"Instance-wise occlusion and depth orders in natural scenes","cited_arxiv_id":null,"evidence_quote":"InstaOrderNet computes the pairwise occlusion order that determines which segments occlude the target."},{"cited_title":"Learn- ing transferable visual models from natural language super- vision","cited_arxiv_id":null,"evidence_quote":"CLIP both selects the inpainting prompt by visual-text similarity and scores class relevance in evaluation."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion v2 inpainting fills the occluded region at each iterative step."},{"cited_title":"Amodal com- pletion via progressive mixed context diffusion","cited_arxiv_id":null,"evidence_quote":"PD-MC is the progressive mixed-context baseline and the source of the background-isolation and iterative inpainting strategy."},{"cited_title":"pix2gestalt: Amodal segmentation by synthesizing wholes","cited_arxiv_id":null,"evidence_quote":"Pix2gestalt is the supervised amodal completion baseline that the method must beat in the preference study."}],"review_version":1}