{"id":"5bc4bd6d-ab10-4624-9f77-8fe97bfda5a0","arxiv_id":"2507.11211","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A vision-guided, optimization-based planner that refines coarse paths into fine, collision-free trajectories lets two robot arms transport objects in cluttered, partially unmodeled environments.","lead":"The paper combines two cameras with an optimization-based controller to plan safe paths for two robot arms carrying a shared object, refining the plan as the cameras see more of a cluttered scene. The intended payoff is reliable object transport in warehouses or homes where the map and target location are not known in advance.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Collision safety in Eq. (6) rests on a learned proxy trained from an incomplete point-cloud model; a single unperceived or mis-segmented obstacle makes the hard constraint blind, so the robustness claim is not yet established.","rationale":"The reader's weakest assumption correctly identifies the load-bearing point: the hard collision constraint is only as reliable as the perceived and learned environment model. My stress test sharpens that point by separating two failure routes, perception incompleteness and classifier false negatives, and by noting that no clearance margin or dense geometric verification is provided. This does not contradict the reader's CONDITIONAL verdict; it explains why the condition is necessary. The paper is a systems letter with a plausible engineering contribution, and the real-world multi-arm demonstration is real evidence of feasibility, but the central robustness claim cannot be accepted as validated until the collision scores are checked against exact geometry on the actual recorded trajectories and the perception pipeline is stress-tested with occluded or thin obstacles. I therefore keep the verdict unchanged rather than moving it.","tokens_in":10632,"tokens_out":12432,"duration_ms":176700,"concrete_test":"Run a controlled Drake simulation with the published Scenario I setup, but place a thin opaque rod (diameter ~2 cm) in the robot's swept path such that it is outside the initial field of view of both the stationary and eye-in-hand cameras and therefore absent from the first point-cloud update. Execute the full coarse-to-fine MPC pipeline and record the minimum clearance between any robot link or the workpiece and the rod over the entire dense trajectory. If the minimum clearance is below a pre-specified safety margin (or the rod is contacted), the hard constraint of Eq. (6) was protecting only against obstacles that had already been perceived, and the robustness claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the hard constraint SCORE(z(s_k)) = 0 in Eq. (6), with the kernel-perceptron score refined in Section IV-C, is a faithful certificate of whole-body collision freedom on the executed trajectory. That requirement has two unsecured parts. First, the point-cloud processing in Section IV-B filters with robot-geometry spheres, segments with DBSCAN, and constructs occlusion polytopes; any obstacle that is missed, merged, or represented too coarsely is simply absent from the support-vector set and can never activate the constraint. Second, the active-learning update samples a tiny fraction of the 14-DoF closed-chain configuration manifold, so even for a fully perceived obstacle the learned SCORE can contain false negatives. The paper imposes no clearance margin and checks constraints only at collocation points, so the advertised 'robust, whole-body collision avoidance under uncertainty' is not certified by the mathematics as written. The Scenario I and II runs show one successful pipeline, but they do not quantify the failure modes that would falsify the robustness claim; the manuscript's own deferral of pseudocode (Sections IV-B, IV-C, IV-D) makes those failure modes currently uncheckable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes an MPC-based coarse-to-fine motion planning framework for two manipulators cooperatively transporting a rigid object through cluttered, initially unknown scenes. The planner uses a B-spline transcription of the combined joint/object state, enforces closed-chain and terminal-pose constraints, and incorporates a vision-based cost to drive an eye-in-hand camera toward informative views. The environment is represented by fusing stationary and eye-in-hand point clouds, applying geometric filtering, DBSCAN segmentation, and occlusion-polytope construction, while collision avoidance is encoded as hard constraints using a refined kernel-perceptron collision score with per-link-group support vectors and an active-learning update. The claims are evaluated with one static-clutter scenario and one human-intrusion scenario on a dual Franka Emika platform, with supplementary simulations in Drake.","tokens_in":10927,"tokens_out":5079,"duration_ms":68151,"significance":"If substantiated, the framework would be a useful integration of perception-driven MPC, learned collision constraints, and visibility-aware exploration for high-dimensional closed-chain manipulation, and the real dual-arm demonstration is commendable. Strengths include the explicit closed-chain formulation, the use of B-spline collocation with time scaling, the group-wise kernel and pruning refinements to DiffCo, and the deployment on real hardware with CasADi/IPOPT. However, the central claim of robust whole-body collision avoidance under uncertainty is not yet supported by the evidence: the evaluation is largely qualitative, no baselines or repeated trials are reported, and the safety-critical hard constraint rests on a learned surrogate whose completeness and accuracy are unquantified. The visibility evaluation is also partly circular because the reported score is the same kernel-similarity objective that the MPC maximizes.","major_comments":[{"comment":"The hard constraint SCORE(z(s_k)) = 0 in Eq. (6) can only protect against obstacles that have been captured by the point-cloud pipeline of Section IV-B and converted into support vectors. Any obstacle that is missed, merged, or represented too coarsely never enters the support-vector set and hence cannot activate the constraint. In addition, the active-learning exploration step samples only a finite subset of the 14-DoF closed-chain configuration space, so even fully perceived obstacles may have false-negative regions in the learned score, and constraints are enforced only at collocation points with no clearance margin. As written, the paper does not certify 'robust, whole-body collision avoidance under uncertainty.' Please provide held-out collision-classification accuracy (false-negative rate versus clearance), an analysis of coverage of the closed-chain manifold, and a sensitivity study of the pruning and weight-reset parameters.","section":"IV-C, Eq. (6)-(10)"},{"comment":"The experimental section reports a single representative run per scenario with no repeated trials, no error bars, no comparison against baseline planners (e.g., the original DiffCo, a sampling-based planner, or an MPC without the visibility cost), and no quantitative success/failure statistics. The Abstract and Section I claim that the framework is 'experimentally validated' as robust, but the evidence is not commensurate with that claim. Please add multiple runs per scenario, report success rates, minimum distances to obstacles along the executed trajectory, planning and update times, and at least one baseline comparison.","section":"V (Scenarios I and II)"},{"comment":"The top plot of Fig. 8 shows the visibility score computed as the kernel similarity to the 'visibility support vectors' of Section IV-D, which is exactly the quantity that the vision cost Cvis in Eq. (6) maximizes in the MPC. Reporting the increase of this score is therefore partly a check that the optimizer minimized its own objective, not independent evidence that exploration succeeded. Please evaluate visibility with an external metric, such as AprilTag detection success, target-pose estimation error from the raw point clouds, or pixel coverage of the target, and report target-localization accuracy at the end of exploration.","section":"IV-D and Fig. 8"},{"comment":"The manuscript repeatedly states that pseudocode 'will be released on Github upon acceptance,' leaving unspecified the active-learning reset rule, the Gram-matrix pruning thresholds, the biased-sampling distribution, the per-group support-vector limits, and the construction of 'visibility support vectors.' These are central algorithmic contributions, so as submitted the method is not fully reproducible and the identified failure modes cannot be independently checked. Please include complete pseudocode, parameter values, and ablation choices in the manuscript or an attached supplement.","section":"IV-B, IV-C, IV-D"}],"minor_comments":[{"comment":"The 'dexterity' cost term ||q(s_i) - (q_min + q_max)/2||^2 penalizes distance from the center of the joint range, but the prose says it steers agents clear of joint limits; the relationship between this cost and the hard limit constraints z_min <= c_i <= z_max should be clarified.","section":"IV-A, Eq. (6)"},{"comment":"The text says the top plot of Fig. 8 shows 'the visibility score defined in (6),' but Eq. (6) defines the optimal control problem, not the visibility score; please give the visibility score an explicit definition and equation number in Section IV-D.","section":"V-B"},{"comment":"The notation x is used for the Cartesian end-effector pose in Eq. (2) and for configuration vectors in the kernel definition in Section IV-C; the paper notes the conflict, but the repeated reuse of the symbol is still confusing to the reader.","section":"IV-C"},{"comment":"There are several typos and grammatical issues, including 'Agent Dexterity: :' in the bullet list of Section IV-A, 'which not only expands the dimensionality...' fragment in the Introduction, and inconsistent comma use throughout.","section":"I and IV-A"},{"comment":"The sentence introducing GPMP2 cites reference [11], which is a survey; please cite the original GPMP2 paper so that readers can locate the method.","section":"II"}],"recommendation":"major_revision","confidential_remarks":"The core integration is plausible and the real-hardware demonstration is a strength, but the evidence base is too thin for the advertised robustness claims. The safety issue is load-bearing: the hard constraint is only as good as the learned support vectors and the perception completeness, neither of which is quantified. I would like to see the requested experiments and full algorithmic details before considering acceptance. There is also a fit concern: the contribution is largely an engineering integration of prior MPC and DiffCo ideas, so the novelty should be more crisply isolated and benchmarked against those prior components."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: this is a competent systems integration paper. What's new is the specific combination: dual-camera point cloud processing with incremental convex-hull and occlusion-polytope construction, a B-spline transcribed MPC that includes a learned kernel-perceptron collision constraint per link group, and a visibility cost that drives eye-in-hand camera poses. That combination is not in the cited literature, and the hardware results with two Franka arms show the pipeline works end-to-end.\n\nThe paper does several things well. The B-spline transcription with closed-chain constraints is clean. Splitting the DiffCo collision classifier into per-segment groups with separate support vectors is a sensible fix for the 'everything averaged together' problem, and the planar two-link comparison in Fig. 5 makes the point concretely. The visibility support vector idea is clever: it turns the problem of 'which camera pose sees the target' into a kernel similarity cost the MPC can optimize online.\n\nThe soft spots are real, though. The central claim is 'robust, whole-body collision avoidance under uncertainty,' but the experiments are single-run demonstrations without baselines, error bars, or repeated trials. The hard constraint SCORE(z(s_k))=0 is only as good as the learned support-vector set, which is trained from whatever the point-cloud pipeline currently sees. A missed or mis-segmented obstacle simply never enters the constraint. The paper does not impose a clearance margin, and checks collisions only at collocation points. That is a fundamental limitation of any perception-based planner, so the concern is not fatal, but it means the robustness claim should be scoped to 'obstacles that have been perceived and correctly segmented.' Right now the paper oversells it.\n\nThere is also circularity in the evaluation: the visibility score shown as evidence of exploration success is the same cost that the MPC optimizes. The task itself -- successfully placing the object -- is the real metric, and it would be easy to report. The absence of code or pseudocode until acceptance also keeps the failure modes uncheckable.\n\nWho is this for? Robotics practitioners working on cooperative manipulation with partial observability. It deserves peer review as a systems letter, but it needs a revision that adds quantitative validation, a baseline comparison, and concrete statements about what the planner can and cannot guarantee. As written, I would not cite it as evidence of robust collision avoidance, only as an example of an integrated MPC-perception pipeline.\n\nRecommendation: send to review, but require the experimental and code/reproducibility gaps to be addressed.","headline":"A competent systems integration paper for vision-guided MPC with real hardware, but the robustness claims outrun the quantitative evidence.","tokens_in":11435,"tokens_out":3048,"would_cite":false,"duration_ms":37549,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shrinking-horizon MPC with dual-camera perception and a refined kernel-perceptron collision detector lets two arms carry an object through cluttered, initially unmodeled scenes.","keywords":["model predictive control","coarse-to-fine planning","multi-arm manipulation","collision avoidance","kernel perceptron","B-spline trajectory optimization","dual-camera perception","closed-chain kinematics"],"falsifier":"Place a solid, reachable obstacle between the arms and the target but hide it from both cameras at start (for example, just behind a larger box), then run the transport task. If the arms collide with the hidden obstacle before any perception update reveals it, the claim of whole-body safety in unmodeled environments collapses to safety against perceived obstacles only.","tokens_in":10402,"feed_emoji":"🤖","tokens_out":9622,"duration_ms":100325,"temperature":0.7,"pith_summary":"This paper presents a motion planner for a team of robot arms that must carry a rigid object through a cluttered room whose contents and exact target location are initially unknown. The planner starts from rough partial observations, produces a feasible global trajectory, and then continually refines the trajectory as new point-cloud data arrive from a stationary camera and a camera mounted on one arm's hand. The central claim is that this coarse-to-fine model predictive control (MPC) scheme, with a vision-cost term that steers the hand camera toward informative viewpoints and a kernel-based collision score enforced as a hard constraint, achieves whole-body collision avoidance and target-aware motion generation under uncertainty. Real-time experiments on a two-arm platform are used to support the claim, including a simulated human-intrusion case that triggers safety-aware replanning.","feed_headline":"MPC planner moves joint-held objects through unseen clutter","feed_subtitle":"A dual-camera B-spline MPC replans from coarse to fine, adding collision constraints as visibility improves.","key_machinery":"The load-bearing mechanism is the pair formed by the B-spline trajectory transcription and the learnable collision score constraint. The full 21-dimensional state of both arms plus the object pose is encoded as a B-spline curve $z(s)$, so velocity and acceleration control points enter the optimization directly. The collision constraint is a refined kernel-perceptron proxy detector: the robot is split into separate collision groups, each with its own support vectors and a forward-kinematics kernel, and the optimization enforces $\\mathrm{SCORE}_{g_i}(z(s_k)) = 0$ for each group at collocation points. A differentiable visibility cost $C_{\\mathrm{vis}}$, built from reachable and unobstructed eye-in-hand camera poses, pulls the hand camera toward viewpoints that reveal the target, while the point-cloud pipeline (geometric filtering, convex-hull segmentation, occlusion polytopes) feeds the support-vector updates that keep the hard constraint current.","core_discovery":"The paper claims that cooperative transportation of a jointly held object in cluttered, unmodeled environments can be handled by a shrinking-horizon MPC in which the trajectory is encoded as B-splines and the environment is represented by a differentiable proxy collision score. At the core, the planner minimizes a cost over acceleration, joint-limit margin, duration, and a differentiable visibility score, subject to closed-chain kinematic constraints and the hard collision constraint $\\mathrm{SCORE}(z(s_k)) = 0$ at every collocation point. The perception module fuses a stationary and an eye-in-hand depth camera, filters out the robot's own geometry, segments obstacles into convex hulls and occlusion polytopes, and continuously updates the support vectors of a kernel-perceptron collision detector so the hard constraint reflects newly seen obstacles. The authors report real-time applicability, whole-body collision avoidance, and dynamic replanning in experiments with two redundant 7-DoF cooperative robots, including exploratory eye-in-hand motion that resolves target visibility and a replanning mode triggered by a simulated human intrusion.","pith_inferences":["The coarse-to-fine mechanism is not tied to two-arm transport: the same shrinking-horizon MPC with endpoint tolerance could let any manipulator reach into shelves or bins where the goal pose is only partially known, using the slack variable $\\epsilon_{\\mathrm{fp}}$ to represent goal uncertainty explicitly.","Because collision supports are learned independently per link group, a pre-trained boundary for the lower links might transfer across tasks that only change the wrist or gripper, potentially cutting retraining effort.","A testable extension is to add a penalty for trajectories that pass through still-unobserved regions rather than treating them as free space; that would directly sharpen the paper's weakest point, which is that the hard constraint only sees perceived obstacles.","The differentiable visibility score could serve outside planning as an objective for online view planning or for benchmarking active-perception systems, since it gives a smooth proxy for how reachable and unobstructed a camera pose is when looking at the target."],"forward_implications":["The planner runs online at fixed replan intervals on a single workstation, so no pre-mapped environment is needed before the two arms start moving.","The differentiable visibility cost induces active exploration: the eye-in-hand camera is steered to resolve target pose, turning initial partial observations into precise placement.","The per-group collision scores plus active-learning updates let the hard constraints track slow scene changes, as shown by the human-intrusion scenario that triggers safety-aware evasive replanning.","Enforcing $h_{\\mathrm{obj}}(q_{R1}) = h_{\\mathrm{obj}}(q_{R2}) = x_{\\mathrm{obj}}$ keeps the two arms coordinated during transport, and the near-identical independent pose estimates reported confirm closed-chain consistency.","Whole-body avoidance is enforced at collocation points, so not only the end effector but all link groups of both arms participate in the collision constraint."],"supporting_citations":[{"why":"supplies the kernel-perceptron proxy collision detection method that this work refines with per-link-group kernels and active-learning updates.","marker":"[12]"},{"why":"supplies the B-spline-based continuous trajectory transcription used to encode the joint and object state trajectory.","marker":"[14]"},{"why":"the prior MPC formulation with B-spline collocation on which the shrinking-horizon planner is built.","marker":"[16]"},{"why":"provides the rigid-body dynamics simulation environment used to model and validate the experiments.","marker":"[17]"},{"why":"DBSCAN is the density-based clustering step that segments filtered point clouds into convex hulls for obstacle representation.","marker":"[18]"},{"why":"supplies the nonlinear optimization framework in which the MPC is formulated and code-generated.","marker":"[20]"},{"why":"provides the interior-point solver that solves the nonlinear program at each replanning step.","marker":"[21]"}],"fun_headline_variants":["MPC vision planner threads joint-held objects through clutter","Coarse-to-fine MPC replans robot object transport in real time","Dual-camera B-spline MPC moves objects safely in unseen clutter","Real-time MPC with vision refines paths through cluttered scenes","Cooperative robots use MPC to carry objects through clutter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Collision safety depends on perception completeness: every real obstacle must appear in the processed point cloud and be represented in the support-vector set, because the hard collision constraint can only reject configurations that the perceived geometry makes visible.","fun_headline_variants_meta":{"raw":{"variants":["MPC vision planner threads joint-held objects through clutter","Coarse-to-fine MPC replans robot object transport in real time","Dual-camera B-spline MPC moves objects safely in unseen clutter","Real-time MPC with vision refines paths through cluttered scenes","Cooperative robots use MPC to carry objects through clutter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1332,"prompt_tokens":874,"completion_tokens":458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":372}},"tokens_in":490,"tokens_out":458,"duration_ms":5796,"temperature":1.0,"reasoning_tokens":372,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:13:52.913139+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Place a solid, reachable obstacle between the arms and the target but hide it from both cameras at start (for example, just behind a larger box), then run the transport task. If the arms collide with the hidden obstacle before any perception update reveals it, the claim of whole-body safety in unmodeled environments collapses to safety against perceived obstacles only.","supporting_citations":[{"cited_title":"Diffco: Autodifferentiable proxy collision detection with multiclass labels for safety-aware trajectory optimization,","cited_arxiv_id":null,"evidence_quote":"supplies the kernel-perceptron proxy collision detection method that this work refines with per-link-group kernels and active-learning updates."},{"cited_title":"Continuous Trajectory Optimization via B-splines for Multi-jointed Robotic Systems","cited_arxiv_id":"2212.10500","evidence_quote":"supplies the B-spline-based continuous trajectory transcription used to encode the joint and object state trajectory."},{"cited_title":"A mpc-based approach for motion planning on redundant manipulators in human robot collaboration,","cited_arxiv_id":null,"evidence_quote":"the prior MPC formulation with B-spline collocation on which the shrinking-horizon planner is built."},{"cited_title":"A density-based algorithm for discovering clusters in large databases with noise,","cited_arxiv_id":null,"evidence_quote":"DBSCAN is the density-based clustering step that segments filtered point clouds into convex hulls for obstacle representation."},{"cited_title":"CasADi – A software framework for nonlinear optimization and optimal control,","cited_arxiv_id":null,"evidence_quote":"supplies the nonlinear optimization framework in which the MPC is formulated and code-generated."},{"cited_title":"Adaptive barrier update strategies for nonlinear interior methods,","cited_arxiv_id":null,"evidence_quote":"provides the interior-point solver that solves the nonlinear program at each replanning step."}],"review_version":1}