{"id":"aa7ae029-f3d6-4cce-b54b-0a2b37eec351","arxiv_id":"2505.12744","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A 7B multimodal model can control a robot by predicting each next gripper pose in natural language, trained from only 65 dialogues plus simulator reinforcement learning, and it claims strong generalization to new settings and real robots.","lead":"This paper trains a 7B vision-language model to solve robot manipulation tasks by reasoning out loud in text about the next gripper pose, using just 65 human-guided example dialogues and reinforcement learning in a simulator. The authors report high success rates, zero-shot transfer to new environments, and real-robot results with far less data than standard robot-learning models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No experiment isolates whether the 'system-2' reasoning text is causal for action prediction; the claimed reasoning-driven advantages (generalization, sim-to-real, interpretability) rest on an untested link.","rationale":"The reader's weakest assumption focuses on the privileged state information and its dependence on accurate point-cloud segmentation. I agree that this is a real system boundary and that the comparisons to VLA models are apples-to-oranges, but I do not think it is the single most load-bearing concern. The core novelty of the paper is the claim that LMM reasoning ('system-2 level reasoning') directly drives next-goal pose prediction and yields advantages in generalizability, sim-to-real transfer, and interpretability. That claim requires the generated reasoning text to be causally connected to the action outputs. The paper provides no ablation or controlled experiment isolating this link. SFT trains the model to imitate teacher reasoning, and GRPO optimizes only terminal task success, so the model can in principle learn the correct action distribution while the <think> tokens are a byproduct. Fig. 8 shows qualitative examples of failure detection and correction, but it does not demonstrate that the same actions would not occur without those reasoning steps. If the reasoning is epiphenomenal, the method reduces to a state-conditioned pose predictor with a chat interface, and the stated advantages driven by system-2 reasoning are unsupported. This is more damaging than the perception boundary, because the perception issue is an acknowledged limitation with a plausible mitigation (better point-cloud completion), whereas a missing causal link invalidates the paper's central contribution. The concrete test I propose directly settles this by neutralizing the reasoning tokens and measuring success; a significant drop would vindicate the reasoning claim, while no drop would require reinterpreting the results. I therefore keep the verdict CONDITIONAL, adding a specific required ablation to the conditions already set by the reader.","tokens_in":22569,"tokens_out":5700,"duration_ms":62333,"concrete_test":"Run inference with the reasoning tokens neutralized: for each of the four MetaWorld zero-shot tasks (or the SIMPLER tasks), sample N=50 trials with the normal model, and another N=50 with the same model but with the <think> segment replaced by a fixed string (e.g., 'No reasoning') while keeping the ACTION parsing identical. If success rates do not drop significantly (e.g., within error bars), the reasoning is not causally load-bearing. As a complementary probe, fine-tune a variant where the model is trained to output the correct ACTION but a deliberately wrong reasoning text; if it acquires similar success rates, the action head learns independently of the reasoning.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"ReasonManip outputs a <think> reasoning block followed by an ACTION pose, and the pose is parsed and executed. The central claim is that system-2 reasoning drives the model's performance and interpretability. For this to hold, the reasoning must be causally linked to the action tokens. However, nothing in the training objective enforces such a link: SFT (Eq. 2) trains on teacher-generated reasoning, and GRPO (Eq. 3) rewards only binary task success, not reasoning quality. The model could learn to produce correct poses while generating fluent but post-hoc rationalizations. The paper's only evidence for the reasoning's role is qualitative (Sec. 4.5 and Fig. 8), showing that the model sometimes detects failures and corrects them. No ablation removes, corrupts, or decouples the reasoning tokens. Under such an ablation, if success rates remain unchanged, the 'system-2 reasoning' contribution is not load-bearing, and the claimed advantages (exceptional OOD generalization, sim-to-real transfer via unified language, transparent interpretability) would instead be attributable to the privileged state input and action-space design. This is the weakest link in the central argument.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ReasonManip, a 7B large multimodal model that performs tabletop robot manipulation by predicting next-goal gripper poses in natural language using a novel axis-based rotation representation. The training pipeline consists of (i) collecting 65 multi-round reasoning dialogues from Qwen2.5-VL-72B with human guidance, (ii) supervised fine-tuning of Qwen2.5-VL-7B, and (iii) Group Relative Policy Optimization (GRPO) inside the SIMPLER simulator with a binary success reward. The authors claim that the resulting model exhibits system-2 reasoning behaviors, achieves high out-of-distribution generalization to MetaWorld, transfers to a real UR5 robot, and offers transparent interpretability. Experiments compare ReasonManip with RT-1-X, Octo, OpenVLA, and pi0 on SIMPLER, MetaWorld, and a real-world setup, reporting success rates that generally favor ReasonManip, alongside a small ablation of the axis representation versus Euler angles.","tokens_in":22770,"tokens_out":5457,"duration_ms":61430,"significance":"If the claims were fully supported, the paper would introduce a compelling paradigm: using language as a unified action and reasoning interface for manipulation, with extreme data efficiency (65 demonstrations), zero-shot transfer across simulators, and interpretable step-by-step control. The axis-based rotation representation and the GRPO-in-the-loop training recipe are potentially valuable. The paper also honestly states some limitations in Appendix F. However, the central attribution of the reported success to system-2 reasoning is currently unsubstantiated: the comparisons are confounded by privileged numeric state input, the reasoning tokens are never ablated, and the OOD claims are weakened by per-task GRPO training. The ideas are interesting and the topic is timely, but the evidence as presented does not yet support the strong conclusions.","major_comments":[{"comment":"The performance comparison against VLA baselines is confounded because ReasonManip receives privileged quantitative scene information (object centers, bounding-box dimensions, PCA-derived axes, and gripper pose) as text at every dialogue turn, whereas RT-1-X, Octo, OpenVLA, and pi0 receive only images. The reported success-rate advantages therefore cannot be attributed to reasoning or the language interface; a model that simply maps state vectors to goal poses might achieve comparable results. The paper should include a controlled comparison where baselines receive an equivalent state representation (e.g., as additional input tokens or via the same perception module), or an ablation in which ReasonManip is evaluated without textual scene information.","section":"Sec. 3.2, Tables 1-3"},{"comment":"The central claim that system-2 reasoning drives performance is not supported by causal evidence. The SFT loss in Eq. (2) trains on teacher-generated reasoning, and the GRPO loss in Eq. (3) rewards only binary task success, never the quality or causal role of the reasoning text. The model could learn to emit correct actions while producing fluent but post-hoc rationalizations. The qualitative examples in Sec. 4.5 and Fig. 8 illustrate failure detection and self-correction, but they do not show that the reasoning tokens are load-bearing. An ablation that removes, corrupts, or decouples the <think> tokens—for instance, training an action-only variant or replacing reasoning text with fixed filler during inference—should be performed. Without such an ablation, the paper's stated three advantages (OOD generalization, sim-to-real transfer, interpretability) cannot be causally credited to the reasoning process.","section":"Sec. 3.4, Sec. 4.5, Eq. (3)"},{"comment":"The out-of-distribution generalization claim is weakened by the fact that GRPO is trained per task. Appendix A states: 'We use the GRPO to train for each of the specific tasks, since we observe severe task conflict based on the limited quantity of SFT training samples.' The MetaWorld tasks in Table 2 are described as 'resembling those in SIMPLER' (Grasp, Move, Stack, Close), i.e., they are the same task types across different simulators and embodiments, not unseen task categories. The abstract's wording 'exceptional generalizability to out-of-distribution environments, objects, and tasks' is therefore not supported for the 'tasks' component. Please clarify what is claimed to generalize and provide evidence for generalization to genuinely novel task types, or adjust the claims accordingly.","section":"Appendix A, Table 2"},{"comment":"None of the reported success rates include confidence intervals or statistical significance tests, which matters because several headline differences are small relative to the trial counts (50 trials per SIMPLER task, 30 per MetaWorld task, 10 per real-world task). For example, Table 2 shows Move improving from 23.3% to 30.0% and Close from 20% to 40% after GRPO; with 30 trials, these differences are within plausible sampling noise. Similarly, Table 1 reports many values in the 30-45% range with no error bars. The authors should report per-trial variability (e.g., bootstrap confidence intervals over episodes) and, where possible, the number of independent evaluation seeds.","section":"Tables 1-4"}],"minor_comments":[{"comment":"There are naming typos: 'Deekseek-R1' should be 'DeepSeek-R1' and 'GPRO' should be 'GRPO'.","section":"Sec. 2"},{"comment":"The caption of Figure 3 is incomplete; the bottom-left subplots showing Euler angles are not described in the caption, and the figure contains dense inline text that is hard to read. Please expand the caption and simplify the figure.","section":"Fig. 3"},{"comment":"The notation in Eq. (3) is inconsistent: 'at.i' appears in the policy ratio but 'at,i' in the loss sum; also the definition of clip(·) is written in a compressed way. Please clean up the typography.","section":"Eq. (3)"},{"comment":"Please report the GRPO hyperparameters more completely, including the group size K, the number of update epochs per task, and whether rewards include any shaping beyond the binary success flag.","section":"Appendix A"},{"comment":"The real-world experiments would benefit from reporting the accuracy of the support-query segmentation module, since the method's failures are attributed in Appendix F to incomplete point clouds. Without this, it is unclear whether the 40-50% real-world success rates are limited by perception, reasoning, or control.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to attract attention because it combines recent LMM reasoning techniques with robot manipulation and reports strong numbers. The main risk is that the central causal claim (reasoning drives the gains) is not testable from the current experiments, and the baselines are compared under unequal information. The suggested ablations and controlled comparisons are feasible within the paper's scope and should be required before publication. I would also encourage the authors to release code, the 65 dialogues, and the GRPO training configuration, as the data efficiency claim depends heavily on the exact data collection protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"ReasonManip is a clever paper with a real idea buried under an over-claimed title. The new bit is the next-goal formulation with an axis-based rotation representation, plus a two-stage SFT+GRPO recipe that gets a 7B model to solve tabletop tasks from 65 dialogues. That data-efficiency number is striking, and the axis-vs-Euler ablation (Table 5) gives me some confidence the representation is doing real work. The paper is also reasonably honest about its failure modes in Appendix F.\n\nThe problems are with the empirical claims. The baselines are not comparable: ReasonManip receives quantitative scene states as text, while RT-1-X, OpenVLA, and pi0 get only images. You are effectively comparing a state-based planner to a vision-based policy. And ReasonManip outputs sparse goal poses, so it is executing far fewer decisions per episode than the dense-action baselines; control frequency alone can explain part of the gap. The zero-shot generalization section is also softened by the fact that GRPO is per-task: the model is fine-tuned on SIMPLER tasks that resemble the MetaWorld tasks, so this is cross-environment, not cross-task, generalization. On top of that, success rates are reported without error bars on 10-50 trials.\n\nThe weakest link, though, is the causal role of the reasoning. The stress-test note is right: GRPO rewards binary task success, not reasoning quality. Nothing forces the <think> text to be causally connected to the ACTION tokens. The model could be producing fluent post-hoc rationalizations while a separate part of the network computes the pose. The paper offers only qualitative examples of failure detection and no ablation that removes, corrupts, or decouples the reasoning tokens. Without that, the 'system-2 reasoning' claim is unsupported, and the observed performance could be attributed to the privileged state input and the action-space design.\n\nThese are all fixable. The axis representation and the data pipeline are worth publishing; the evaluation just needs to be rebuilt. I would send this to peer review, but I would ask for a reasoning ablation, image-only baselines that get the same state information (or a version of ReasonManip without text state), and dense-action control for all methods. The paper is a serious piece of work, but the central claim currently rests on an untested link.","headline":"The paper has a genuinely useful idea in the axis-based pose representation and a striking data-efficiency result, but the central claim that system-2 reasoning drives performance is not actually tested.","tokens_in":23309,"tokens_out":2746,"would_cite":false,"duration_ms":28486,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ReasonManip shows that a 7B multimodal model can drive direct robot manipulation by predicting each next gripper pose in language after being fine-tuned on only 65 teacher dialogues and reinforced with GRPO in simulation.","keywords":["robot manipulation","large multimodal models","reasoning","GRPO","reinforcement learning","sim-to-real transfer","spatial reasoning","vision-language-action"],"falsifier":"Run ReasonManip on the same SIMPLER tasks while corrupting only the supplied scene state — add Gaussian noise to object positions, or remove part of an object's point cloud so its PCA axes become unstable — while leaving the visual observation unchanged. If success collapses in proportion to the corruption, the policy is carried by the external numbers rather than by the model's spatial reasoning; the paper's own Appendix F names incomplete point clouds as a failure condition. A complementary check is to attempt tasks on symmetric objects such as a perfect cube or sphere, where the PCA axis representation is ambiguous or degenerate, and count the resulting failures.","tokens_in":22373,"feed_emoji":"🤖","tokens_out":14467,"duration_ms":129770,"temperature":0.7,"pith_summary":"This paper claims that a large multimodal model can control a robot manipulator directly by reasoning out each next gripper pose in natural language, rather than by emitting actions through a separate decoder or action head. To make that possible, it reformulates manipulation as multi-round dialogues in which the model receives quantitative scene state — each object part's center, bounding box, and three PCA-derived orientation axes — and must output the next gripper position, orientation, and open/close command. The resulting 7B model, ReasonManip, is fine-tuned on just 65 human-guided dialogues generated by a 72B teacher, then improved by GRPO trial-and-error in the SIMPLER simulator. The authors report zero-shot transfer to unseen MetaWorld tasks and to a real UR5 robot, with success rates that beat RT-1-X, OpenVLA, and pi0 and that barely change when the camera viewpoint moves. If right, this means robot manipulation can become data-efficient, interpretable, and broadly generalizable when posed as text-based spatial reasoning, provided an external perception layer faithfully supplies the numbers the model reasons over.","feed_headline":"65-dialogue model beats RT-1-X, OpenVLA, and pi0","feed_subtitle":"A 7B model that reasons each gripper pose in words transfers zero-shot from simulation to a real robot","key_machinery":"The load-bearing object is the axis-based rotation representation; the load-bearing mechanism is the two-stage training loop. Each object part is encoded as a 12-tuple: Cartesian center $(x,y,z)$, bounding-box dimensions $(l,w,h)$, and three PCA-derived orthogonal unit vectors — the longitudinal axis (largest eigenvector), the normal axis (smallest eigenvector), and the binormal axis given by their cross product. Gripper poses are described the same way, with the rotation matrix reconstructed from the axis vectors by a fixed formula, which turns orientation reasoning from opaque Euler angles into vector arithmetic the model can perform in text. The training loop first supervises a 7B LMM on 65 multi-round dialogues produced by a 72B teacher under human guidance, then runs GRPO inside SIMPLER with binary task success as reward, rewarding text that checks collisions, detects misalignment, and corrects poses before every executed action.","core_discovery":"ReasonManip establishes that next-goal gripper poses can be produced as plain text through explicit, step-by-step spatial reasoning, provided the spatial action space is re-encoded from Euler angles into three orthogonal unit vectors (longitudinal, binormal, and normal axes) computed from point-cloud PCA of each object part. On this formulation, high-level task decomposition, mid-level goal analysis, and low-level pose estimation all happen in a single dialogue pass in one unified language format. Fine-tuned on 65 teacher dialogues and then refined with GRPO inside SIMPLER, the 7B model reaches an average 45.4% success across seven SIMPLER tasks, transfers zero-shot to unseen MetaWorld environments (96.7% on grasping, where RT-1-X, Octo-base, and OpenVLA all score 0%), keeps near-identical success when the camera view changes, and drives a real UR5 arm at 40–50% success on three tasks, outperforming even real-world-fine-tuned OpenVLA and pi0. The authors attribute this generalization to the mathematical, appearance-invariant form of the task and to GRPO, which they argue mitigates SFT overfitting and induces written reasoning behaviors such as collision checks, failure detection, and self-correction before each action.","pith_inferences":["The practical ceiling of this paradigm sits in the perception front-end, not the reasoning model: because the policy receives, rather than computes, the quantitative state, the reported successes presuppose accurate segmentation and PCA, so point-cloud completion or better pose estimation might buy more robot-level robustness than a larger language model would.","The formulation is naturally limited to tasks decomposable into a short sequence of discrete absolute poses, such as pick, place, push, and close; continuous or trajectory-sensitive skills such as wiping or pouring would need the hybrid scheme the authors sketch, switching between deliberate next-goal reasoning and high-frequency direct generation.","Since GRPO is run per-task rather than jointly (the authors note severe task conflict with only 65 samples), a meaningful stress test would be training on all tasks together and measuring how much of the out-of-distribution transfer survives.","An independent ablation would isolate the sources of the gain: applying the same dialogue-and-axis format to an open 7B VLA without GRPO would show how much of the sim-to-real transfer comes from the representation itself versus from reinforcement-driven exploration."],"forward_implications":["A single language interface can carry all levels of control: task decomposition, goal analysis, and concrete pose estimation happen in one reasoning pass, so no separate planner or action decoder is required.","Manipulation policies can be trained with far less data — 65 dialogues plus simulator reinforcement — than the millions of demonstrations typically used for VLA models.","The axis representation is itself a performance lever: reformatting all orientations as Euler angles in the same training setup drops lift-coke-can success from 96.7% to 42.0% in the paper's comparison.","Policies that reason over quantitative state rather than raw appearance transfer zero-shot across simulators and to a real robot, and their success is nearly unchanged under large camera-view changes.","Every executed action is accompanied by a written rationale, so failures can be traced back to a specific reasoning step rather than an opaque action tensor."],"supporting_citations":[{"why":"Supplies the SIMPLER simulator where seed trajectories are collected, GRPO trial-and-error runs, and all seven benchmark tasks are evaluated.","marker":"[25]"},{"why":"The 72B Qwen2.5-VL teacher that, under human guidance, generates the 65 multi-round seed dialogues used for supervised fine-tuning.","marker":"[2]"},{"why":"Introduces GRPO, the group-relative policy optimization used in the second training stage to incentivize reasoning and improve out-of-distribution generalization.","marker":"[33]"},{"why":"Provides MetaWorld, the unseen environment used for the zero-shot out-of-distribution transfer evaluation.","marker":"[48]"},{"why":"OpenVLA, the 7B vision-language-action baseline that ReasonManip is compared against in SIMPLER, MetaWorld, and the real-robot experiments.","marker":"[22]"},{"why":"π0, the VLA baseline fine-tuned on real-world trajectories that ReasonManip outperforms on the three real-robot tasks.","marker":"[3]"},{"why":"RT-1-X, the 55B VLA baseline used across the SIMPLER, MetaWorld, and real-world comparisons.","marker":"[4, 31]"},{"why":"ECoT, the embodied chain-of-thought reasoning method whose separate action head ReasonManip is designed to replace.","marker":"[49]"}],"fun_headline_variants":["ReasonManip: 7B model reasons robot moves in plain English, wins","Text-based reasoning beats RL: 65 dialogues transfer zero-shot","Why say Euler when you can say coordinates? 7B robot model excels","From sim to real with words alone: 7B model outperforms","GRPO-trained 7B writes its own reasoning, beats OpenVLA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that accurate, complete quantitative scene state — every object part's position, bounding-box size, and three PCA-derived orientation axes, plus the gripper pose — is supplied to the model in text at every dialogue turn; the paper itself (Appendix F) acknowledges failure when point clouds are incomplete, and if this perception layer is noisy or wrong, the reasoning and the executed actions fail regardless of the language model's quality.","fun_headline_variants_meta":{"raw":{"variants":["ReasonManip: 7B model reasons robot moves in plain English, wins","Text-based reasoning beats RL: 65 dialogues transfer zero-shot","Why say Euler when you can say coordinates? 7B robot model excels","From sim to real with words alone: 7B model outperforms","GRPO-trained 7B writes its own reasoning, beats OpenVLA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000653,"raw_usage":{"total_tokens":3077,"prompt_tokens":1116,"completion_tokens":1961,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":732,"completion_tokens_details":{"reasoning_tokens":1863}},"tokens_in":732,"tokens_out":1961,"duration_ms":14572,"temperature":1.0,"reasoning_tokens":1863,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:27:27.516568+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ReasonManip on the same SIMPLER tasks while corrupting only the supplied scene state — add Gaussian noise to object positions, or remove part of an object's point cloud so its PCA axes become unstable — while leaving the visual observation unchanged. If success collapses in proportion to the corruption, the policy is carried by the external numbers rather than by the model's spatial reasoning; the paper's own Appendix F names incomplete point clouds as a failure condition. A complementary check is to attempt tasks on symmetric objects such as a perfect cube or sphere, where the PCA axis representation is ambiguous or degenerate, and count the resulting failures.","supporting_citations":[],"review_version":1}