{"id":"a85143fb-a140-4a85-998f-82d80a76f196","arxiv_id":"2507.14820","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A keypoint-based 6-DoF grasp network trained end-to-end with a probabilistic PnP layer and a confidence map beats prior KGN variants on grasp success rate in simulation and on a small real-robot test.","lead":"KGN-Pro adds a differentiable probabilistic Perspective-n-Point layer to a keypoint-based grasp detector, so 3D grasp supervision can flow back into 2D keypoint predictions. The paper reports higher grasp success rates than KGN, KGNv2, CenterGrasp, and Contact-GraspNet on synthetic scenes and on a small real-robot test.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) omits the confidence weights w_i from the likelihood exponent that Eq. (6) defines, so the probabilistic PnP term may not use the confidence map as claimed; reported gains could come only from the second re-projection term in Eq. (7).","rationale":"The reader identified the importance-weighted Monte Carlo estimator as the weakest assumption. I agree that the estimator is the critical technical risk, but the more specific and more checkable issue is the internal inconsistency between Eq. (3) and Eq. (6): the printed importance sampling term omits the confidence weights that Eq. (6) places inside the likelihood. If the code follows the printed Eq. (3), the confidence map would not influence the probabilistic integral term L_pred, undermining the paper's central claim that confidence-weighted probabilistic PnP transmits 3D supervision end-to-end. This is not an attack on the authors; it is a concrete correctness question that code inspection or a targeted ablation would settle. The reader also noted unreported K and proposal q(y); those are real but secondary, since they affect the accuracy of a term that may already be mis-specified. Secondary concerns include the absence of baseline comparisons in the physical experiments and the omission of Direct3D numerical results, but those affect the strength of the empirical claim rather than the validity of the mechanism. The paper's synthetic results are internally plausible, and the method may work as intended; the CONDITIONAL verdict remains appropriate until the implementation is verified against the equations. My agreement is 'partial' because the reader focused on estimator variance/bias, while I emphasize the specific missing weights and the need for code verification rather than an assumed failure of the estimator.","tokens_in":11865,"tokens_out":5299,"duration_ms":55337,"concrete_test":"Obtain the released code (or ask the authors) and inspect the forward/backward implementation of L_pred from Eq. (3). Check specifically whether the exponent in the Monte Carlo sum contains ||w_i E_i(y_j)||² or ||E_i(y_j)||². If the weights are absent, retrain KGN-Pro on the same split with the weighted form and with the unweighted form, and compare Table I and Fig. 4; if the two runs differ materially, the published numbers do not reflect the stated mechanism. Also report K and the exact q(y) (initial distribution, optimization steps) to assess estimator variance in the 6-DoF pose space.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism is the KL loss in Eq. (2), approximated by Monte Carlo importance sampling in Eq. (3) and combined with the confidence-weighted likelihood in Eq. (6). As printed, Eq. (3) computes the log-likelihood integral using exp(−1/2 Σ_i ||E_i(y_j)||²), with no w_i factors, whereas Eq. (6) defines p(X|y) ∝ exp(−1/2 Σ_i ||w_i E_i(y)||²). If the implementation follows the printed Eq. (3), the confidence map does not affect the L_pred term at all; only the second term in Eq. (7), the weighted re-projection error at the matched ground-truth pose, receives confidence information. That would mean the method is not 'modeling the weighted sum of squared re-projection errors probabilistically' end-to-end as claimed; the probabilistic PnP integral is unweighted, and the confidence branch is trained only through a deterministic re-projection loss. In addition, the Adaptive Multiple Importance Sampling proposal q(y), the sample count K, and the procedure for optimizing q are never specified, so even the unweighted integral's accuracy is unverifiable. This gap is load-bearing because every reported improvement over KGN/KGNv2 is attributed to this 3D supervision pathway; if the weights are absent from L_pred, the contribution of confidence weighting to the success-rate gains is unsupported. The mismatch may be a typographical omission, but it must be resolved before the central claim is accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes KGN-Pro, a 6-DoF grasp prediction network that preserves the 2D keypoint representation of prior KGN methods and adds a 2D confidence map plus a probabilistic PnP layer. The central claim is that by modeling the weighted sum of squared re-projection errors as a probabilistic object and minimizing a KL divergence between the predicted pose distribution and a nearest-neighbor-matched ground-truth target, the network can be trained end-to-end with 3D supervision, yielding higher grasp success rates than KGN, KGNv2, CenterGrasp, and Contact-GraspNet in simulation and on a physical robot.","tokens_in":12283,"tokens_out":5953,"duration_ms":62502,"significance":"If the central claim holds, the confidence-weighted probabilistic PnP formulation is a potentially generalizable upgrade for keypoint-based 6-DoF grasping, and the paper provides a synthetic benchmark plus a small physical validation. The reported improvements over prior keypoint-based methods at tight precision tolerances are encouraging. However, the paper does not release code or data, reports no variance, and contains a direct inconsistency between Eq. (3) and Eq. (6) that affects whether the confidence map actually participates in the probabilistic PnP loss. The central claim is therefore not yet established as stated.","major_comments":[{"comment":"There is a direct inconsistency between the likelihood used in the Monte Carlo estimate and the likelihood used in the confidence-weighted formulation. Equation (3) defines L_pred as log(1/K Σ_j exp(-1/2 Σ_i ||E_i(y_j)||^2) / q(y_j)), with no confidence weights w_i in the exponent, while Eq. (6) defines p(X|y) ∝ exp(-1/2 Σ_i ||w_i E_i(y)||^2). If the implementation follows Eq. (3), the confidence map does not affect the L_pred term at all, and confidence information reaches the network only through the second term in Eq. (7); the central claim that KGN-Pro 'models the weighted sum of squared re-projection errors probabilistically' (abstract and Section IV-C) is then unsupported. Please correct Eq. (3) to include w_i if that matches the implementation, or explicitly state that the probabilistic term is unweighted and revise the claims accordingly.","section":"III, Eq. (3); IV-C, Eq. (6)"},{"comment":"The Adaptive Multiple Importance Sampling estimator is underspecified: the paper does not state the number of samples K, the parametric form of the proposal q(y), or the procedure used to optimize q(y). Since Eq. (4) relies on backpropagating importance-weighted samples to approximate the gradient of L_pred, the accuracy of this Monte Carlo estimate and its gradient cannot be verified from the manuscript; please provide these details or an ablation showing the estimator's stability.","section":"III, Eq. (3)-(4)"},{"comment":"Direct3D is plotted in Fig. 4 but is absent from Table I, and no variance or confidence intervals are reported for any of the simulated success rates. The claim of a 'clear advantage' over KGNv2 at tight tolerances cannot be statistically evaluated without repeated runs or confidence intervals; please report means and standard deviations over multiple seeds or trials, and add Direct3D to Table I for consistency with Fig. 4.","section":"V-B, Table I"},{"comment":"The abstract claims 'grasp cover rate and success rate' as the headline metrics, but 'cover rate' is never defined in the paper. The physical experiment defines GSR and SCR (Scene Completion Rate), and it is unclear whether 'cover rate' refers to SCR or to some other measure; please define the metric precisely and use consistent terminology throughout.","section":"Abstract; V-C"}],"minor_comments":[{"comment":"The loss weights are given as λ_H, λ_S, λ_O, λ_KL in Eq. (8), but the implementation text says λ_Y=1, λ_O=1, λ_J=1, λ_KL=0.1; please align the subscript notation and state which symbol multiplies which loss.","section":"V-A, Eq. (8)"},{"comment":"Equation (1) introduces σ_i, but σ_i disappears in Eq. (6) and is not mentioned again; please clarify whether σ_i is absorbed into the confidence weights w_i or set to a constant.","section":"III, Eq. (1)"},{"comment":"The table header and rows are visually confusing: the 'Class' column mixes object names (Cylinder, Sphere, etc.) with scene numbers (No.1 to No.5); please restructure the table so that single-object and multi-object results are presented as separate blocks with clear row labels.","section":"V-C, Table II"},{"comment":"Please add a statement on code and data availability, or provide them as supplementary material; the project website is listed, but no repository or dataset link appears in the text.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a borderline case: the core idea is appealing and the empirical results are suggestive, but the Eq. (3)/(6) inconsistency is exactly the kind of load-bearing issue that must be resolved before publication. I would not reject outright; a major revision with the corrected equations, missing estimator details, and strengthened evaluation reporting could make this acceptable. I also note that the lack of code/data and the undefined 'cover rate' will likely draw criticism from readers, and the authors should address these points head-on."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible and useful adaptation of probabilistic PnP to keypoint-based grasping, but the printed math has a load-bearing inconsistency that has to be resolved before the central claim holds.\n\nWhat's new: KGN-Pro takes the differentiable probabilistic PnP idea from EPro-PnP and applies it to multi-candidate grasp prediction, adding a confidence map and nearest-neighbor matching to handle multiple grasp poses. That combination is new in the grasp domain, and it is a natural next step after KGNv2's scale-prediction workaround. The synthetic evaluation sweeps precision tolerances systematically, and the physical experiments cover both single- and multi-object scenes. Credit where due: the paper clearly motivates the shift from 2D-only supervision to direct 3D supervision, and the confidence-weighting idea is sensible.\n\nSoft spots: the equations do not match. Eq. (3) gives the Monte Carlo estimate of L_pred using exp(-1/2 Σ ||E_i(y_j)||²) with no w_i, while Eq. (6) defines p(X|y) with w_i multiplying each E_i. If the implementation follows the printed text, the confidence map never enters the probabilistic PnP term; it only appears in the deterministic re-projection term in Eq. (7). That directly undercuts the paper's claim that it models the weighted sum of squared reprojection errors probabilistically. This may be a typo, but it is load-bearing, and the authors need to fix it and restate what the confidence map actually does. Beyond that: no code or data, no variance across seeds, no Direct3D numerical results despite the legend showing Direct3D, and the cover rate promised in the abstract is never defined. The Monte Carlo sample count K and the AMIS proposal optimization are unspecified, so the accuracy of the integral estimate is unverifiable. The physical experiments are small and use mostly primitive-shaped objects, so they support feasibility rather than superiority.\n\nBottom line: the approach is plausible and the reported gains over KGNv2 are meaningful on the synthetic benchmark, but the equation inconsistency plus missing artifacts make this a conditional accept. I would send it to peer review with a clear instruction to reconcile the equations and release code and data.","headline":"A credible extension of KGN whose central printed equation drops the confidence weights, so the headline claim needs a fix before it can be trusted.","tokens_in":12773,"tokens_out":2746,"would_cite":false,"duration_ms":29712,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Probabilistic PnP lets keypoint grasp networks learn end to end from 3D error, and KGN-Pro shows grasp success rises with it.","keywords":["6-DoF grasping","keypoint-based grasping","probabilistic PnP","end-to-end learning","confidence map","RGB-D grasp prediction","robotic manipulation","3D supervision"],"falsifier":"Compute the variance of the Monte Carlo estimate in Eq. (3) over repeated runs with a fixed network and varying sample count $K$; if increasing $K$ does not stabilize or improve training, or if replacing the learned confidence weights with uniform weights does not change grasp success, the probabilistic-PnP-with-confidence mechanism is not what is driving the reported gains.","tokens_in":11700,"feed_emoji":"🤖","tokens_out":5054,"duration_ms":49702,"temperature":0.7,"pith_summary":"This paper claims that the main bottleneck in keypoint-based 6-DoF robotic grasping is the non-differentiable Perspective-n-Point (PnP) pose solver, which forces earlier keypoint networks to train on 2D keypoints alone. KGN-Pro replaces that solver with a probabilistic PnP layer, turning the weighted sum of squared reprojection errors into a likelihood over poses and matching this pose distribution to ground-truth grasps. A learned 2D confidence map weights each keypoint's contribution, so 3D supervision flows back through the probabilistic layer to the 2D keypoint and confidence predictors, enabling fully end-to-end training. If correct, this makes richer 3D supervision available to any keypoint-based grasp or pose network without giving up the speed and fine-grained control of the 2D representation.","feed_headline":"Probabilistic PnP lets keypoint grasp networks learn in 3D","feed_subtitle":"A confidence-weighted keypoint network turns 3D pose error into training signal, beating prior grasp keypoint methods.","key_machinery":"The key mechanism is the probabilistic PnP layer, adapted from object-pose estimation: the reprojection likelihood is proportional to the exponential of the negative confidence-weighted sum of squared reprojection errors, and the posterior $p(y|X)$ is matched to a target distribution $t(y)$ via a KL divergence loss. The normalizing constant in that loss is estimated by adaptive multiple importance sampling with an optimized proposal $q(y)$, so gradients flow back through the pose distribution to the keypoint and confidence extractors. Nearest-neighbor matching selects the ground-truth grasp that defines $t(y)$, which turns the multi-modal grasp-prediction problem into a well-posed distribution-matching problem.","core_discovery":"KGN-Pro's central claim is that modeling the weighted sum of squared reprojection errors probabilistically transmits 3D supervision directly to 2D keypoint predictions and enables end-to-end learning for 6-DoF grasping. The network outputs a keypoint map (center heatmap, sub-pixel offsets, center-to-keypoint offsets) plus a per-keypoint 2D confidence map. The probabilistic PnP layer converts the resulting 2D-3D correspondences into a pose distribution $p(y|X)$, with likelihood $p(X|y) \\propto \\exp\\left(-\\tfrac12 \\sum_i \\|w^{2D}_i E_i(y)\\|^2\\right)$; a KL-divergence loss against a target distribution centered at the nearest matching ground-truth grasp provides the 3D supervision. The paper reports grasp success rates of about 96% in single-object scenes under loose tolerance and about 80% under the strictest 1 cm, 10° tolerance, consistently above KGN and KGNv2 and clearly above point-cloud methods, with physical experiments exceeding 90% single-object success.","pith_inferences":["In our reading, the same probabilistic-PnP-plus-confidence pattern could be lifted from grasping to category-level object pose estimation and visual servoing, where keypoint correspondences are also noisy and multi-modal.","Because the confidence map is trained only through the KL loss, an ablation that masks or randomizes the confidence weights would reveal how much of the gain comes from weighting versus from the 3D supervision itself; the paper does not isolate these two factors.","We suspect the unreported number of importance samples $K$ and the choice of proposal $q(y)$ interact with the claimed gains: at small $K$, the estimator in Eq. (3) may add noise rather than signal, so reporting variance diagnostics would make the mechanism testable.","One untested consequence is that the method should degrade gracefully as depth noise increases, since the confidence weights can learn to suppress bad correspondences; a noise-sweep experiment would confirm this."],"forward_implications":["KGN-Pro's probabilistic PnP converts 3D grasp poses into training signal for 2D keypoint and confidence predictions, removing the non-differentiable bottleneck that limited KGN and KGNv2.","The learned 2D confidence map improves pose accuracy by down-weighting unreliable keypoints during reprojection, which the paper credits for the gains under strict 1 cm/10° tolerances.","Because the representation stays 2D, the method retains the efficiency and fine-grained grasping of the KGN line while outperforming point-cloud methods like CenterGrasp and Contact-GraspNet, especially on small and cluttered objects.","Physical experiments with household objects show overall single-object success above 90% and scene completion rates between 80% and 100% across camera angles, indicating the simulation-trained model transfers to a real robot."],"supporting_citations":[{"why":"Supplies the probabilistic PnP formulation (likelihood proportional to exponentiated negative weighted reprojection error) that KGN-Pro adapts to grasping.","marker":"[30]"},{"why":"Provides the end-to-end backpropagation-through-PnP mechanism that makes 3D gradients flow to the 2D predictors.","marker":"[35]"},{"why":"The keypoint-grasp baseline whose non-differentiable PnP limitation KGN-Pro removes.","marker":"[10]"},{"why":"The KGNv2 baseline that predicts scale; KGN-Pro's results are compared against it in simulation and physical tests.","marker":"[11]"},{"why":"Gives the adaptive multiple importance sampling used to approximate the normalizing constant in the KL loss.","marker":"[31]"},{"why":"CenterGrasp, the point-cloud reconstruction baseline that KGN-Pro outperforms and whose failures motivate the 2D representation.","marker":"[45]"},{"why":"Contact-GraspNet, the direct point-cloud 6-DoF baseline used in the comparison.","marker":"[26]"},{"why":"Supports the claim that direct regression of 3D geometric numbers is inferior to residual or probabilistic formulations.","marker":"[13]"}],"fun_headline_variants":["Probabilistic PnP gives keypoint grasp networks true 3D supervision","KGN-Pro: probabilistic 3D supervision boosts grasp success","Confidence-weighted keypoints make grasp learning end-to-end in 3D","Probabilistic PnP turns 3D pose error into training signal for grasps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The end-to-end training pipeline rests on the assumption that the finite-sample Monte Carlo estimate of the pose-distribution normalizing constant in Eq. (3) is accurate and low-variance enough that the KL loss gives useful gradients to the keypoint and confidence branches; the paper does not report the number of samples $K$ used.","fun_headline_variants_meta":{"raw":{"variants":["Probabilistic PnP gives keypoint grasp networks true 3D supervision","KGN-Pro: probabilistic 3D supervision boosts grasp success","Confidence-weighted keypoints make grasp learning end-to-end in 3D","Probabilistic PnP turns 3D pose error into training signal for grasps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00061,"raw_usage":{"total_tokens":2886,"prompt_tokens":1035,"completion_tokens":1851,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":1767}},"tokens_in":651,"tokens_out":1851,"duration_ms":14174,"temperature":1.0,"reasoning_tokens":1767,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:46:32.967151+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the variance of the Monte Carlo estimate in Eq. (3) over repeated runs with a fixed network and varying sample count $K$; if increasing $K$ does not stabilize or improve training, or if replacing the learned confidence weights with uniform weights does not change grasp success, the probabilistic-PnP-with-confidence mechanism is not what is driving the reported gains.","supporting_citations":[{"cited_title":"Epro- pnp: Generalized end-to-end probabilistic perspective-n-points for monocular object pose estimation,","cited_arxiv_id":null,"evidence_quote":"Supplies the probabilistic PnP formulation (likelihood proportional to exponentiated negative weighted reprojection error) that KGN-Pro adapts to grasping."},{"cited_title":"End-to-end learnable geometric vision by backpropagating pnp optimization,","cited_arxiv_id":null,"evidence_quote":"Provides the end-to-end backpropagation-through-PnP mechanism that makes 3D gradients flow to the 2D predictors."},{"cited_title":"Keypoint-graspnet: Keypoint- based 6-dof grasp generation from the monocular rgb-d input,","cited_arxiv_id":null,"evidence_quote":"The keypoint-grasp baseline whose non-differentiable PnP limitation KGN-Pro removes."},{"cited_title":"Kgnv2: Separating scale and pose prediction for keypoint-based 6-dof grasp synthesis on rgb-d input,","cited_arxiv_id":null,"evidence_quote":"The KGNv2 baseline that predicts scale; KGN-Pro's results are compared against it in simulation and physical tests."},{"cited_title":"Adaptive multiple importance sampling,","cited_arxiv_id":null,"evidence_quote":"Gives the adaptive multiple importance sampling used to approximate the normalizing constant in the KL loss."},{"cited_title":"Centergrasp: Object-aware implicit representation learning for simultaneous shape reconstruction and 6-dof grasp estimation,","cited_arxiv_id":null,"evidence_quote":"CenterGrasp, the point-cloud reconstruction baseline that KGN-Pro outperforms and whose failures motivate the 2D representation."},{"cited_title":"Contact- graspnet: Efficient 6-dof grasp generation in cluttered scenes,","cited_arxiv_id":null,"evidence_quote":"Contact-GraspNet, the direct point-cloud 6-DoF baseline used in the comparison."},{"cited_title":"Glace: Global local accelerated coordinate encoding,","cited_arxiv_id":null,"evidence_quote":"Supports the claim that direct regression of 3D geometric numbers is inferior to residual or probabilistic formulations."}],"review_version":1}