{"id":"17166cd1-659c-4337-811b-d13b239db56c","arxiv_id":"2505.10219","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A modular ATACOM safety layer is added to robot foundation models pi0 and OCTO, yielding provably safe actions with minimal performance loss.","lead":"Robot foundation models get a safety filter that restricts their actions to a mathematically safe set, preventing collisions and joint-limit violations without retraining. The paper tests this filter on two robot platforms and reports safer execution with roughly unchanged success rates.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Formal safety guarantee applies only to the sphere-constraint manifold; the paper never proves the spheres conservatively bound the full robot geometry, so physical collision avoidance is not actually implied.","rationale":"After reading the paper and the reader's verdict, I agree with the conditional assessment but locate the load-bearing weakness slightly differently. The ATACOM theory is sound under Assumptions 1 and 2, but the paper's implementation does not demonstrate that the constraint functions actually encode the physical safe set. The reader focused on perception (missing or misperceived obstacles), which is an acknowledged limitation. My concern is more fundamental: even with perfect perception, the robot's own geometry is approximated by a finite set of spheres. For the forward-invariance guarantee to imply collision avoidance, these spheres must conservatively enclose the entire robot. The paper says the spheres are placed 'at critical areas' and 'approximate' the hull, but never proves or verifies that the union of spheres contains the full manipulator mesh. Without this, the safety layer's guarantee is only about an artificial sphere-constraint set; a collision through an uncovered segment such as the forearm between spheres is not ruled out. This is a gap in the central claim, and it is not mentioned in the Limitations section. A second, related gap is that the bounding-box distance constraint is non-smooth, while Assumption 2 requires C1 constraints; the paper does not explain how ATACOM handles the non-differentiable points. These issues do not invalidate the idea, but they mean the current experimental evidence does not substantiate 'formal safety guarantees' for the physical robot. The paper should either provide a conservative covering proof or an exact mesh-based signed distance field, and address constraint smoothness, or qualify the claim accordingly. Thus the verdict remains CONDITIONAL: the approach is promising and the empirical results are suggestive, but the transfer of the safety guarantee to the physical system is not yet established.","tokens_in":12537,"tokens_out":7779,"duration_ms":75151,"concrete_test":"Use the sphere configuration from the pick-and-place experiments and compute the signed distance from every vertex of the Franka mesh to the union of spheres. If any vertex is outside all spheres, the cover is not conservative; then run the filtered policy in simulation with a thin obstacle placed near that exposed link and check whether all sphere-obstacle distances remain positive while the robot mesh intersects the obstacle. If such a collision occurs, the formal guarantee does not transfer to physical safety.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that ATACOM provides formal safety guarantees for generalist policies (abstract). The inherited theory [6] guarantees forward invariance of the constraint manifold defined by constraint functions g(x). In the implemented safety layer (Section 2.1, Appendix A.2), the robot is represented by a finite set of spheres 'at critical areas' (Section 2.1) that 'approximate' the hull (Appendix A.2). For the formal guarantee to imply physical collision avoidance, the union of these spheres must be a superset of the robot's entire physical volume, so that the safe set defined by g(x) is a subset of the actual collision-free set. The paper does not state or prove this covering property, nor does it provide the sphere parameters or a completeness check. If any link or segment of the arm protrudes outside the spheres, the safety layer can certify a state as safe while the physical robot is in contact with an obstacle. This is not a perception error; it is an unvalidated modeling step in the safety layer itself. Additionally, the bounding-box distance constraint (Eq. 5) is not continuously differentiable everywhere due to clipping and the norm, while Assumption 2 requires C1 constraints; the paper does not discuss how the non-smoothness is handled in the ATACOM projection. Both gaps mean the experimental demonstrations do not actually establish the stated formal safety guarantee for the physical robot.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a modular safety layer based on ATACOM that is placed after an arbitrary robot foundation model (RFM) to enforce known safety constraints. The safety layer projects actions onto the tangent space of a constraint manifold, providing forward invariance and input-to-state stability guarantees inherited from prior work [6]. The authors evaluate the approach with two RFMs: π0 on a Franka pick-and-place task with visual bounding-box constraints, and OCTO on a dynamic air hockey hitting task. Experiments show that the safety filter reduces constraint violations while maintaining task success. The paper also describes a semi-automated visual constraint generation pipeline using SAM 2, though bounding boxes are hand-drawn in the experiments.","tokens_in":12790,"tokens_out":5881,"duration_ms":54376,"significance":"If the formal guarantees are valid and the implementation matches the theory, this is a meaningful contribution to safe deployment of robot foundation models: it offers a way to obtain safety without safety-specific fine-tuning or large safe datasets. The paper demonstrates flexibility by integrating with two different foundation models and two task types, including a real-world air hockey deployment. Strengths include the modular architecture, the explicit statement of assumptions, and the empirical reduction of collisions. However, the formal guarantee is imported from prior work and is not re-established for the specific implemented constraints, which is the main limitation addressed below.","major_comments":[{"comment":"The claimed formal safety guarantee applies to forward invariance of the constraint manifold defined by g(x), but the implemented collision-avoidance constraints use a finite set of spheres that \"cover the robot's hull at critical areas\" without stating or proving that the union of these spheres is a superset of the robot's physical volume. If any link or segment protrudes outside the spheres, the safety layer can declare a state safe while the physical robot is in contact with an obstacle. The manuscript does not provide the sphere parameters or a coverage check, so the experimental demonstrations do not establish the stated formal guarantee for physical collision avoidance.","section":"Section 2.1 and Appendix A.2"},{"comment":"Assumption 2 in Section 2 requires the constraint functions g(x) to be C1. The bounding-box constraint g(x) = -||α bbx - p||, with p = clip(...), is not continuously differentiable on the boundaries of the clipping operation and where the argument of the norm is zero. The manuscript does not explain how ATACOM handles this non-smoothness or why the differentiability assumption can be relaxed. As the forward-invariance guarantee in [6] relies on differentiability of the constraints, this gap undermines the formal claim for the visual constraints.","section":"Section 2.2, Eq. (2), and Appendix A.2, Eq. (5)"},{"comment":"The abstract and Section 2.1 assert formal safety guarantees for generalist policies, but the relevant theorem from [6] is never stated and its hypotheses are not verified for the concrete constraints (Eqs. 3-6 and 7-14). In particular, the regularity of the constraint manifold and the consistency of Assumption 1 with the actual robot dynamics are taken for granted. The authors should state the theorem and confirm that each implemented constraint satisfies its conditions.","section":"Section 2.1"}],"minor_comments":[{"comment":"The sentence 'We draw the bounding boxes ourselves to obtain reliable bounding boxes for every evaluation run' contradicts the earlier description of an automatic constraint generation pipeline; if the boxes are manually drawn, the 'semi-automated' claim should be clarified.","section":"Section 2.2"},{"comment":"The empirical results are reported without the number of evaluation runs, error bars, or statistical tests; for example, Figure 3 does not specify the number of trajectories used for the success-rate and safety-rate histograms. The authors should report these details.","section":"Section 3.1, Figure 3"},{"comment":"Equation (2) uses mismatched norm symbols ('∥' and 'p∥') and the notation pi vs p is inconsistent with Appendix A.2's pj; this should be corrected.","section":"Section 2.2, Eq. (2)"},{"comment":"The real-world air hockey deployment is described only qualitatively ('reasonable performance'); the manuscript would benefit from quantitative metrics, such as number of episodes, violation rate, and success rate, for the real-world experiments.","section":"Section B.3"}],"recommendation":"major_revision","confidential_remarks":"The paper's safety guarantees are inherited from [6], which shares authors with this submission. While this is not circular in itself, the authors should make clear in the revision which parts of the theory are novel to this paper and which are re-used. The editor may also wish to consider whether the limited statistical reporting in the experiments meets the journal's standards."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something simple and useful: it puts ATACOM, an existing safety filter, behind two existing robot foundation models (pi0 and OCTO) and shows that the combination avoids collisions without fine-tuning for safety. The empirical work is decent: in pick-and-place and air hockey, the filter eliminates constraint violations while keeping success rates roughly level, and they even deploy on a real Kuka. The idea of treating safety as a post-filter rather than a training objective is the right instinct, and I buy that it can work for known, geometrically expressible hazards.\n\nBut the 'formal safety guarantee' in the abstract is overstated relative to what is actually shown. Two gaps stand out. First, the safety layer models the robot with a few spheres, and the theory guarantees forward invariance only w.r.t. the sphere-defined safe set. The paper says the spheres 'cover' or 'approximate' the hull, but never states or proves that the union of spheres contains the full robot volume, nor does it give the sphere parameters or a completeness check. Without that, a state can be certified safe w.r.t. the spheres while a link is touching the obstacle. Second, the bounding-box distance constraint (Eq. 5) uses clipping and a norm, so it is not C1 as Assumption 2 requires; the paper does not explain how ATACOM handles the non-smoothness. These don't kill the empirical contribution, but they do mean the experiments don't establish the claimed formal guarantee for the physical robot.\n\nThere are lesser issues: the 'automatic' visual constraint pipeline is actually hand-drawn boxes (the paper admits this in Section 2.2), and the plots lack error bars or any statistical comparison. The formal guarantee is cited from Liu et al. [6], which shares authors; that's acceptable since the theory is published, but the present paper adds no new theory.\n\nOverall, I think this is a legitimate, well-scoped empirical study that points in a useful direction. The right audience is people working on safe deployment of large policies, not theorists. It deserves review, but a referee should push for a clear statement of the sphere-covering assumption and a treatment of the non-smooth constraint. I'd recommend accept with major revisions if those points are addressed.","headline":"A useful empirical recipe for adding a safety filter to robot foundation models, but the formal guarantee is narrower than the abstract claims.","tokens_in":13379,"tokens_out":2460,"would_cite":false,"duration_ms":22487,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By projecting robot foundation-model actions onto the tangent space of a constraint manifold, a modular safety layer provides formal safety guarantees without safety-specific fine-tuning.","keywords":["robot foundation models","generalist policies","safety","ATACOM","constraint manifold","vision-language-action models","inductive biases"],"falsifier":"Run the pick-and-place setup with a distractor that the instance-segmentation-based bounding-box generator does not segment, and check whether the robot arm collides with it; if it does, the guarantee applies only to the declared constraints. A second check is to shift a detected bounding box a few centimeters from the true obstacle and observe whether the arm enters the real object, showing that the formal guarantee assumes exact constraint evaluation.","tokens_in":12318,"feed_emoji":"🤖","tokens_out":6923,"duration_ms":62858,"temperature":0.7,"pith_summary":"The paper tries to establish that robot foundation models do not need to learn safety from demonstrations. A modular safety layer placed after the policy can enforce formal safety guarantees by projecting actions onto the tangent space of a constraint manifold. If this is right, generalist policies such as π0 and OCTO can be deployed in cluttered or dynamic environments without safety-specific fine-tuning, and safety holds even under disturbances. This matters because current behavior-cloned policies provide no formal safety guarantee and would need impractically large amounts of safe data to handle novel obstacles. The experiments support the claim on a quasi-static pick-and-place task and a dynamic air-hockey hitting task.","feed_headline":"One safety layer gives robot foundation models formal safety","feed_subtitle":"The ATACOM layer blocks unsafe actions, keeping task success rates intact across pick-and-place and air hockey.","key_machinery":"The central object is the ATACOM decomposition $a_{\\text{safe}} = a_{\\text{drift}}(s) + a_{\\text{err}}(s) + B(s)a_{\\text{RFM}}$, built from constraints $0 \\ge g(x)$ with $g \\in C^1$. The constraint manifold is the set where all constraints are active; the tangent-space basis $B(s)$ maps the foundation model's action into a direction that keeps the robot on that manifold, the drift term compensates the change of $g$ from the system's affine dynamics, and the error term reactively returns the state to the safe set after disturbances. This decomposition is what lets the paper claim forward invariance and input-to-state stability while leaving the policy weights untouched.","core_discovery":"The paper's central claim is that coupling any robot foundation model with ATACOM, a safety layer that acts on the tangent space of a known constraint manifold, converts a purely behavior-cloned policy into one that is safe by construction. The layer takes the policy's action $a_{\\text{RFM}}$ and outputs $a_{\\text{safe}} = a_{\\text{drift}}(s) + a_{\\text{err}}(s) + B(s)a_{\\text{RFM}}$, where the drift term compensates the system's own dynamics, the error term corrects constraint violations under disturbances, and $B(s)$ morphs the action onto the tangent space. Under mild assumptions this yields forward invariance of the safe set and input-to-state stability, so safety is guaranteed rather than learned. The experiments with π0 on Franka pick-and-place and OCTO on air hockey show constraint violations drop to zero without a meaningful loss in success rate, and the safety layer runs at a higher frequency than the policy itself.","pith_inferences":["The paper leaves implicit that safety data become unnecessary in principle: any future RFM that outputs actions for a control-affine system could be wrapped with the same filter, so safe deployment stops scaling with the policy's dataset.","A testable extension is to replace the hand-drawn bounding boxes with open-vocabulary grounding so the filter can handle new object classes at deployment; the paper's formal guarantee would then depend on perception accuracy, which it does not quantify.","The architecture suggests a broader trade-off: safety as an architectural projection may be more data-efficient than safety as a training objective, but the cost is that every hazard must be declared explicitly as a constraint."],"forward_implications":["A generalist policy that has never been fine-tuned for safety can be deployed with formal guarantees, as long as its actions are filtered by the tangent-space projection.","Safety enforcement can run at 50–60 Hz even when the foundation policy acts at 12.5–15 Hz, so the layer does not add latency to the policy's decisions.","Across both studied tasks, the safety layer keeps task success rates comparable to the unfiltered policy while eliminating constraint violations.","Because the layer requires no safety demonstrations, expert data collection can remain safe; the existing demonstration data only needs to teach task competence."],"supporting_citations":[{"why":"Introduces ATACOM's constraint-manifold action space, the mechanism the safety layer is built on.","marker":"[5]"},{"why":"Supplies the formal proofs of forward invariance and input-to-state stability that ground the safety guarantee.","marker":"[6]"},{"why":"The π0 vision-language-action flow model used as the foundation policy in the pick-and-place experiments.","marker":"[4]"},{"why":"The OCTO generalist policy fine-tuned for the air hockey task and compared with and without the safety layer.","marker":"[3]"},{"why":"The instance-segmentation model used to derive bounding-box safety constraints from camera images.","marker":"[7]"},{"why":"Supplies the teleoperation data-collection protocol and expert demonstrations used to fine-tune π0.","marker":"[18]"},{"why":"Provides the air hockey task, expert policy, and benchmark setup used to generate fine-tuning data and evaluate hitting success.","marker":"[19]"},{"why":"The physics simulator where the OCTO air hockey policy is fine-tuned and evaluated.","marker":"[20]"},{"why":"The reinforcement-learning safety-alignment baseline that this paper contrasts with its architectural approach.","marker":"[8]"}],"fun_headline_variants":["Formal safety for robot foundation models with ATACOM","ATACOM layer enforces safety in any robot foundation model","Geometric inductive bias guarantees safe robot actions","Safe by construction: robot foundation models get ATACOM","One safety layer removes need for demonstration-heavy safety training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Safety constraints must be known in advance, written down as smooth functions, and perceived correctly; if a hazard is missing from those constraints, the guarantee protects only the modeled safe set, not the actual danger.","fun_headline_variants_meta":{"raw":{"variants":["Formal safety for robot foundation models with ATACOM","ATACOM layer enforces safety in any robot foundation model","Geometric inductive bias guarantees safe robot actions","Safe by construction: robot foundation models get ATACOM","One safety layer removes need for demonstration-heavy safety training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000232,"raw_usage":{"total_tokens":1498,"prompt_tokens":959,"completion_tokens":539,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":461}},"tokens_in":575,"tokens_out":539,"duration_ms":5079,"temperature":1.0,"reasoning_tokens":461,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:13:56.273754+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pick-and-place setup with a distractor that the instance-segmentation-based bounding-box generator does not segment, and check whether the robot arm collides with it; if it does, the guarantee applies only to the declared constraints. A second check is to shift a detected bounding box a few centimeters from the true obstacle and observe whether the arm enters the real object, showing that the formal guarantee assumes exact constraint evaluation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces ATACOM's constraint-manifold action space, the mechanism the safety layer is built on."},{"cited_title":"Black, N","cited_arxiv_id":null,"evidence_quote":"The π0 vision-language-action flow model used as the foundation policy in the pick-and-place experiments."},{"cited_title":"Ghosh, H","cited_arxiv_id":null,"evidence_quote":"The OCTO generalist policy fine-tuned for the air hockey task and compared with and without the safety layer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The instance-segmentation model used to derive bounding-box safety constraints from camera images."},{"cited_title":"Khazatsky, K","cited_arxiv_id":null,"evidence_quote":"Supplies the teleoperation data-collection protocol and expert demonstrations used to fine-tune π0."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the air hockey task, expert policy, and benchmark setup used to generate fine-tuning data and evaluate hitting success."},{"cited_title":"Todorov, T","cited_arxiv_id":null,"evidence_quote":"The physics simulator where the OCTO air hockey policy is fine-tuned and evaluated."}],"review_version":1}