{"id":"841c9896-f56a-44a7-a1de-a6cd2656f918","arxiv_id":"2412.13173","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MOPD improves openable part detection and motion parameter prediction by fusing perceptual grouping and geometric priors from foundation models into a two-decoder transformer with a motion-aware optimal transport matching cost.","lead":"This paper introduces MOPD, a two-stage transformer framework that detects openable parts such as doors, drawers, and lids in images and predicts how they move. It combines features from a segment-anything model and a surface-normal estimator, plus a motion-aware matching cost, and reports higher accuracy than prior methods on the OPDMulti benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 1.2-point +MAO improvement is fragile: checkpoint selection on the validation split plus single-run reporting leaves the key motion gain potentially within run-to-run noise.","rationale":"The reader's weakest_assumption identified the validation-split checkpoint selection and lack of error bars as the key risk, and I agree that this is the most load-bearing concern. The architecture is coherent, the detection improvement is large, and the ablation study gives some internal support to the role of the geometric encoder. However, the motion accuracy gain is the weaker part of the central claim: it is small, reported from a single run, and the model selection procedure directly optimizes the reporting metric on the reporting split. The paper does not state that a separate test split was used, and Fig. 3 refers to the 'val split' for qualitative results. Because the +MAO margin is only 1.2 points and the ablation table shows similar-sized differences between model variants, the claimed motion improvement cannot be distinguished from checkpoint-selection bias or seed noise without a multi-seed, held-out evaluation. This does not invalidate the paper, but it does mean the central claim is not yet fully established. I therefore retain the reader's CONDITIONAL verdict rather than moving to ACCEPT or REJECT; the proposed concrete test would settle whether the concern actually lands.","tokens_in":9685,"tokens_out":7057,"duration_ms":68598,"concrete_test":"Run the OPDMulti and MOPD training pipelines (same code, same schedule) with at least 5 seeds each, and evaluate on an official test split or, if none exists, on a held-out subset never used for checkpoint selection. Report mean +/- standard deviation for PDet, +M, +MA, and +MAO. If the mean +MAO gap is not larger than the pooled standard deviation, or if the gap shrinks below about 1 point, the claimed 1.2% motion improvement is not established. Additionally, compare checkpoint selection on +MAO versus on PDet to quantify selection bias on the reported split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim's motion component rests on the +MAO gap between OPDMulti (16.0) and MOPD with Optimal Transport (17.2) in Table 1. Section 4.2 states that 'the best checkpoint is selected based on validation set performance (using the +MAO metric),' and Fig. 3 explicitly labels results on the 'val split.' If the quantitative numbers in Table 1 are also computed on this validation split, then the reported +MAO gain is not an unbiased estimate: the model was explicitly chosen to maximize this metric on the same images used for the headline comparison. This is compounded by single-run evaluation with no error bars. A 1.2-point gap is within typical seed-to-seed variation for a Transformer segmentation model, and the paper's own ablations in Table 3 show +MAO values of 15.9, 16.5, 16.6, 16.7, and 16.1 across variants — a spread comparable to the claimed improvement. The detection gain (+4.9 PDet) is larger and more likely to survive scrutiny, but the motion-parameter portion of the central claim is the load-bearing part and the least secure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MOPD, a Mask2Former-based two-stage framework for openable part detection from RGB images. It augments the standard OPD encoder with a perceptual grouping encoder (EfficientSAM) and a geometric understanding encoder (DSINE), uses separate semantic-segmentation and motion decoders, and introduces an optimal-transport assignment cost that includes origin and axis motion terms. Experiments on the OPDMulti dataset report that MOPD improves over OPDMulti by 4.9 points in part-detection mAP and 1.2 points in +MAO motion-parameter accuracy, with additional ablations on encoder choices, optimal transport, and efficiency.","tokens_in":9951,"tokens_out":7141,"duration_ms":57922,"significance":"If the reported gains are stable, MOPD is a useful step for openable part detection: it demonstrates that foundation-model priors can be injected into both detection and motion branches, and the two-decoder design cleanly separates mask/type prediction from motion regression. The paper makes code and models publicly available, reports efficiency numbers, and includes several controlled ablations. The main significance is conditional on the robustness of the numbers: the detection gain is substantial, but the motion-parameter gain is small and the current evidence does not rule out checkpoint-selection or run-to-run noise.","major_comments":[{"comment":"The headline +MAO gain of 1.2 points (16.0 to 17.2) is not robust as reported. Section 4.2 states that the best checkpoint is selected on validation-set performance using the +MAO metric, Fig. 3 labels results on the 'val split', and the paper does not state that Table 1 uses a separate test split or multiple seeds. The spread of +MAO values across Tables 2 and 3 (15.9, 16.1, 16.5, 16.6, 16.7) is comparable to the claimed 1.2-point improvement, so the motion-parameter portion of the central claim may be within run-to-run variation. Please report results on a held-out test split, provide multiple-seed means with standard deviations, and clarify which split underlies Table 1.","section":"Section 4.2, Table 1, Fig. 3"},{"comment":"The ablation results are internally inconsistent with the claim that the perceptual grouping encoder contributes to motion accuracy. Removing the perceptual encoder yields +MAO of 16.7, which is higher than the full model's 16.6, while the +M metric drops by only 0.7 points (36.1 to 35.4). Table 2 also shows +MAO of 16.5 when the geometric encoder is replaced with Omnidata. These numbers do not clearly support the conclusion in Section 4.3 that both feature encoders improve motion parameter prediction; the paper should reconcile them or temper the component-level claims.","section":"Table 3, Section 4.3"},{"comment":"The claim that the proposed optimal-transport motion assignment 'significantly' improves motion accuracy is not well supported by the reported single-run results. The OT variant improves +MAO from 16.6 to 17.2 (0.6 points) but decreases +MA from 20.7 to 20.1, and no error bars are given. Given the small magnitude and the conflicting +MA direction, the contribution of the motion match cost should be demonstrated with repeated runs or a larger margin before it is presented as a key contribution.","section":"Section 4.4, Table 1"}],"minor_comments":[{"comment":"The segmentation loss equation contains a typo: 'λcls + Lcls' should presumably be 'λclsLcls', since a bare λcls term is not a loss.","section":"Section 3.5"},{"comment":"In the motion loss equation, Lo is used for both the motion-origin loss and the pose loss, making the formula ambiguous; use distinct symbols such as Lorg and Lpose.","section":"Section 3.5"},{"comment":"The origin-match-cost equation has a vector-valued right-hand side while a cost should be scalar; it should be written with a norm, e.g., ||(Opred - Ogt) × Igt|| / Ldiag, and Ldiag should be defined as the ground-truth object diagonal.","section":"Section 3.6"},{"comment":"The row for MOPD(Optimal Transport) is missing a separator between the +M and +MA values ('37.720.1'); this should be corrected.","section":"Table 1"},{"comment":"The caption text is garbled: 'In the third row. We frozen EfficientSAM and DSINE.' should be a complete sentence describing the frozen-weights variant.","section":"Table 2 caption"},{"comment":"The text uses 'FNN layers' in Section 3.2 but 'FFN' in Section 3.4; the acronym should be consistent (FFN is the standard feedforward network abbreviation).","section":"Section 3.2, Section 3.4"},{"comment":"The phrase 'valid dataset' should be 'validation dataset', and it would help to state explicitly whether the quantitative table uses the same split as the qualitative figure.","section":"Fig. 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The core architectural idea is reasonable and the detection gain is promising, but the motion-parameter claim rests on single-run, validation-selected numbers. I would ask the authors to clarify the evaluation split and add multi-seed or test-set results before accepting. The self-citations to related work are not excessive and do not affect my assessment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering contribution to openable part detection, but the headline claim is mixed. The 4.9-point detection gain on OPDMulti looks credible. The 1.2-point +MAO motion gain does not.\n\nThe genuinely new part is combining two foundation-model priors with a Mask2Former-style detector: EfficientSAM features for perceptual grouping, DSINE surface-normal features for motion, with semantic and motion prediction split across two transformer decoders. To my knowledge this specific combination is not in prior OPD work, and the motion-aware optimal transport matching cost (axis and origin) is a reasonable idea. The ablations are fairly complete: alternative encoders (SAM, Omnidata), frozen vs fine-tuned, and efficiency numbers. The qualitative figures are persuasive that the model uses the priors.\n\nWhere I get uneasy is the evidence for the motion gain. The best checkpoint is chosen on the validation split using the +MAO metric, and Table 1 appears to be on that same split. Single-run numbers without error bars, plus checkpoint selection on the reporting split, make a 1.2-point gap hard to trust. The paper's own ablations show +MAO values spanning 15.9 to 16.7 across variants — the claimed 1.2-point win over OPDMulti is about as large as the noise floor in these tables. Also, the OT gain in Table 1 is inconsistent: MOPD(OT) improves +M and +MAO, but +MA drops from 20.7 to 20.1. That inconsistency deserves an explanation. And in Table 3, removing the perceptual encoder slightly improves +MAO (16.7 vs 16.6), so the contribution of that encoder to motion is unclear. These are fixable issues: multiple seeds, error bars, a split not used for checkpoint selection, and a direct OT ablation in the same table.\n\nCitation pattern seems fine; no circularity. The code and models are promised, but I did not verify they run.\n\nBottom line: the detection result is probably real; the motion-parameter gain is plausible but under-supported. This paper deserves a serious referee, and I would accept it conditionally after the robustness evidence is added.","headline":"Useful incremental OPD paper; the detection gain is credible, but the 1.2-point motion gain is fragile due to single-run results and checkpoint selection on the reporting split.","tokens_in":10483,"tokens_out":2689,"would_cite":true,"duration_ms":25006,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a two-stage Transformer framework with separate decoders for part detection and motion prediction, each fused with a foundation-model prior (perceptual grouping from a lightweight SAM, geometric surface norms from a…","keywords":["openable part detection","articulated objects","Transformer","motion parameter prediction","perceptual grouping","geometric priors","optimal transport","foundation model priors"],"falsifier":"Run the same training protocol (OPDReal pretraining, OPDMulti fine-tuning, 60000 steps, checkpoint selection on validation +MAO) at least ten times with different random seeds, and evaluate on a held-out test split never used for checkpoint selection. If the mean detection mAP gain over OPDMulti falls below 4.9 minus its standard error, or if the +MAO gain is within one standard deviation of zero, the central superiority claim would fail. A simpler check: re-running the released code on the same split should reproduce the reported numbers within a small tolerance; otherwise the claim is not reproducible as stated.","tokens_in":9512,"feed_emoji":"🚪","tokens_out":5853,"duration_ms":48736,"temperature":0.7,"pith_summary":"The paper tries to establish that openable part detection—identifying drawers, doors, and lids in a single RGB image and predicting how each opens—benefits from a two-stage design that keeps object detection and motion prediction in separate decoder branches, each enriched by a task-specific foundation-model prior. The first stage fuses perceptual grouping features (from a lightweight SAM variant) through cross-attention to sharpen detection; the second fuses surface-normal geometric features (from a monocular normal estimator) to ground the predicted motion axis and origin in physical geometry. A motion-aware matching cost added to the optimal-transport assignment aligns predicted parts to ground-truth parts not just by box overlap but by how well the motion origin and axis agree. If the central claim is right, the result is a framework that beats the previous multi-object openable-part detector on its own benchmark by 4.9% mAP in detection and 1.2% in the strictest motion metric, and that transfers to in-the-wild scenes.","feed_headline":"Two-stage model tops openable-part detection by 4.9% mAP","feed_subtitle":"Separate decoders for part and motion, plus motion-aware matching, beat the prior benchmark on openable-part detection.","key_machinery":"The argument hinges on two structural choices. First, the model splits detection and motion prediction into two Transformer decoder stacks (built on Mask2Former): a semantic segmentation decoder that fuses the OPD query features with perceptual grouping features from an EfficientSAM encoder via cross-attention, and a motion decoder that fuses the same query features with geometric features (surface normals) from a DSINE encoder. Second, training uses an optimal-transport assignment whose cost augments the usual class/mask cost with two motion terms: an origin match cost that penalizes drift of the predicted origin along the ground-truth axis, scaled by object diagonal length, and an axis match cost that penalizes the angle between predicted and ground-truth axes. These costs are what carry the claimed accuracy gain in motion prediction.","core_discovery":"On the paper's own terms, the discovery is that openable part detection improves when perceptual and geometric reasoning are separated into two decoder stages, each conditioned on a distinct foundation-model prior, and when the training-time assignment of predictions to ground-truth parts includes motion-aware costs. The perceptual grouping encoder (EfficientSAM) supplies segmentation priors that reduce confusion between distinct objects, while the geometric understanding encoder (DSINE) supplies surface-normal priors that pull predicted motion origins and axes closer to the true articulation geometry. Combined with an optimal-transport matching cost that penalizes origin drift along the ground-truth axis and axis angular error, this lifts part detection mAP from 32.9 to 37.8 and the strict +MAO motion metric from 16.0 to 17.2 on OPDMulti, with qualitative gains on in-the-wild images.","pith_inferences":["The paper reports single-run results on a validation split used for checkpoint selection; a natural next test is multi-seed evaluation on a truly held-out split, since the claimed margins (1–5 points) may be optimistic under selection bias.","The authors observe that geometric features also help part detection because pixels sharing a surface normal tend to belong to the same part; this suggests deeper fusion of normal maps into the detection branch could yield further gains than the current cross-attention layer.","The same two-stage recipe—grouping prior for detection, geometric prior for regression—could transfer to other affordance tasks such as graspable-part detection or handle detection in cluttered scenes.","The origin-match cost only penalizes origin drift along the ground-truth axis; adding a cost on the prismatic translation direction could improve +MAO further on sliding parts."],"forward_implications":["A single RGB image suffices to tell a robot where a drawer is and along which axis to pull it, removing the need for depth sensors in routine articulated-object manipulation.","Separating detection from motion prediction in the decoder prevents the two prediction heads from interfering, a design pattern that transfers to other joint detection-and-regression tasks.","The motion-aware optimal-transport cost can be dropped into any detector that regresses axes or origins, not just openable parts.","Because the perceptual and geometric encoders are swappable (SAM, EfficientSAM, Omnidata, DSINE), the framework is a general receptacle for foundation-model priors.","Higher precision at low recall means a robot can run at a high confidence threshold and act on few false positives."],"supporting_citations":[{"why":"The baseline it must beat; supplies the OPDMulti dataset, metrics, and the single-decoder architecture that MOPD extends.","marker":"[21]"},{"why":"Mask2Former is the base architecture whose transformer decoders MOPD re-organizes into two stages.","marker":"[4]"},{"why":"EfficientSAM provides the perceptual grouping encoder whose segmentation priors are fused into the detection branch.","marker":"[25]"},{"why":"DSINE provides the geometric understanding encoder whose surface-normal priors are fused into the motion branch.","marker":"[2]"},{"why":"OPD defines the single-view openable part detection task and supplies the OPDReal dataset used for pre-training.","marker":"[9]"}],"fun_headline_variants":["Two-stage model with foundation priors tops part detection by 4.9 mAP","Foundation priors lift openable-part detection to 37.8 mAP","Perceptual and geometric priors boost part detection by 4.9 mAP","Segmentation and normal priors split stages to beat part detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains rest on the assumption that the OPDMulti validation split, used to pick the best checkpoint via the +MAO metric, is a stable and unbiased estimate of true performance; all results are single runs with no error bars, so the 1–5 percentage point margins could shrink or vanish if the split is unrepresentative or the runs are not reproducible.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage model with foundation priors tops part detection by 4.9 mAP","Foundation priors lift openable-part detection to 37.8 mAP","Perceptual and geometric priors boost part detection by 4.9 mAP","Segmentation and normal priors split stages to beat part detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000559,"raw_usage":{"total_tokens":2638,"prompt_tokens":905,"completion_tokens":1733,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1650}},"tokens_in":521,"tokens_out":1733,"duration_ms":12842,"temperature":1.0,"reasoning_tokens":1650,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:20:36.238985+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training protocol (OPDReal pretraining, OPDMulti fine-tuning, 60000 steps, checkpoint selection on validation +MAO) at least ten times with different random seeds, and evaluate on a held-out test split never used for checkpoint selection. If the mean detection mAP gain over OPDMulti falls below 4.9 minus its standard error, or if the +MAO gain is within one standard deviation of zero, the central superiority claim would fail. A simpler check: re-running the released code on the same split should reproduce the reported numbers within a small tolerance; otherwise the claim is not reproducible as stated.","supporting_citations":[{"cited_title":"In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)","cited_arxiv_id":null,"evidence_quote":"DSINE provides the geometric understanding encoder whose surface-normal priors are fused into the motion branch."},{"cited_title":"In: European Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"OPD defines the single-view openable part detection task and supplies the OPDReal dataset used for pre-training."}],"review_version":1}