{"id":"22bd0778-b6b5-457b-9c57-d1f16e7b5482","arxiv_id":"2506.17811","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RoboMonkey shows that test-time sampling with Gaussian perturbation and a VLM-based action verifier improves the success rate of vision-language-action models on manipulation tasks.","lead":"RoboMonkey is a method for improving existing robot control models at deployment time by sampling many candidate actions, perturbing them, and using a trained vision-language verifier to pick the best one. The paper reports large gains on out-of-distribution tasks, but the evidence is based on small real-world trials and no released code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Closed-loop gains are not isolated from sampling: no random/majority-voting baseline over the same candidate set, so the 25%/9% improvements may be due to sampling diversity rather than the learned verifier or its RMSE-to-expert labels.","rationale":"The reader's weakest assumption—that RMSE-to-expert preference labels assume the expert action is optimal—is real and related, but the more immediate load-bearing gap is that the closed-loop evaluation never isolates the verifier from the sampling procedure. If random selection or majority-voting selection over the same candidate pool performs as well as RoboMonkey, then neither the learned verifier nor its RMSE proxy is responsible for the reported gains; the improvements would be an artifact of sampling diversity. Appendix C's action-error comparison is insufficient because it uses the same RMSE metric that defines the training labels, so it cannot validate the proxy against task success. The paper does provide credible supporting pieces: the SGLang-based serving engine is a concrete systems contribution, the preference-versus-RMSE regression ablation in Appendix F shows relative robustness, and the LIBERO fine-tuning results in Table 1 are a positive sign. However, the absence of a closed-loop sampling-selection baseline leaves the central causal claim underdetermined. The appropriate disposition remains CONDITIONAL, exactly as the reader concluded: the framework is promising, but the headline improvements should not be attributed to verification until the selection rule is ablated. I therefore do not change the reader's verdict, but I sharpen the reason: the missing baseline, not just the label proxy, is the load-bearing issue.","tokens_in":16762,"tokens_out":6768,"duration_ms":80031,"concrete_test":"Run the SIMPLER in-distribution suite (Section 5.2, same four tasks and seeds) with four closed-loop selection rules over the same candidate pool produced by Algorithm 1: (1) RoboMonkey's learned verifier, (2) random selection among the K Gaussian-sampled actions, (3) the majority-voted Gaussian mean action with majority gripper state, and (4) best-of-N among the original N base-policy samples. If rules (2) or (3) match or exceed RoboMonkey's 47.5% average success within error bars, the improvement is not caused by verification; if RoboMonkey clearly beats both, the concern is resolved. To also test the RMSE proxy directly, repeat rule (1) with a verifier trained on trajectory-return preference labels on a held-out LIBERO split; if the RMSE verifier is no worse, the proxy is adequate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that RoboMonkey's learned verifier is what delivers the 25% OOD and 9% SIMPLER gains—is never isolated from the sampling procedure. In all closed-loop experiments (Sections 5.2–5.3), RoboMonkey is compared only to single-action greedy OpenVLA and to V-GPS; there is no comparison to random selection, majority-voting selection, or best-of-N over the same Gaussian-perturbed candidate pool. Since the candidate pool is built by sampling N actions from the base policy and then sampling K actions from a Gaussian fit to those N samples (Algorithm 1), any selection rule over that pool—even random selection—can improve over a single greedy action when the base policy's samples are diverse but individually noisy. Appendix C compares against random and majority voting only on RMSE-to-expert action error on Bridge V2 states, not on closed-loop success; action error is exactly the training proxy and may not track task success under distribution shift. Relatedly, the verifier is trained with preference labels defined as lower RMSE to the single expert action (Section 4.2), so if the expert action is not uniquely optimal or if deployment states are OOD, the verifier may rank actions that imitate the expert above actions that actually succeed. The headline OOD gains—including the banana-in-basket task where greedy OpenVLA is 0%—therefore cannot be attributed to verification until the selection rule is ablated. This is the load-bearing uncertainty: the 25%/9% improvements may be a property of sampling-and-any-selection, not of the learned verifier or its RMSE proxy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies test-time scaling for vision-language-action (VLA) models. It first presents an analysis on Bridge V2 in which action error, measured by RMSE to the expert action, decreases as the number of sampled actions grows, and claims this follows an exponentiated power law across OpenVLA, CogACT, Octo, and SpatialVLA. It then introduces RoboMonkey, a generate-then-verify framework: at deployment it samples \\hat{N} actions from a VLA, fits a Gaussian over translation and rotation components, uses majority voting for the gripper, samples \\hat{K} candidates from that proposal distribution, and selects the action with the highest score from a fine-tuned LLaVA-7B verifier. The verifier is trained on synthetic pairwise preferences whose labels are derived from RMSE to the expert action. Experiments report a 25% absolute improvement on four out-of-distribution real-world WidowX tasks, a 9% improvement on four in-distribution SIMPLER tasks, a 6.7% improvement on LIBERO-Long after fine-tuning, and a scalable synthetic-data pipeline. The paper also contributes an SGLang-based serving engine for batched OpenVLA sampling.","tokens_in":17072,"tokens_out":5539,"duration_ms":55580,"significance":"If the central attribution claim holds, this is a useful and timely contribution: it provides one of the first characterizations of inference-time scaling for VLA policies, offers a fully synthetic pipeline for training action verifiers without human preference labels, and demonstrates consistent gains across simulation and hardware. The efficiency analysis is also a genuine strength, since test-time scaling is often impractical; the reported batching and Gaussian-perturbation results make the framework deployable at roughly 1.5 Hz. The paper is generally clearly written and the experimental scope, spanning two embodiments and three evaluation settings, is substantially broader than many concurrent works. However, the headline improvements are not yet cleanly attributed to the learned verifier: the closed-loop experiments lack selection-rule ablations over the same candidate pool, the simulation results are reported as best-over-sweep, and the real-world results have small sample sizes without variance estimates. These issues are fixable within the scope of the manuscript, so the work is promising but needs revision before the core claims are fully supported.","major_comments":[{"comment":"The closed-loop results do not isolate the contribution of the learned verifier. In all closed-loop evaluations, RoboMonkey is compared with single-action OpenVLA and V-GPS, but there is no random-selection, majority-voting, or best-of-N baseline over the same candidate pool generated by Algorithm 1. Appendix C does compare these selection rules, but only on RMSE-to-expert action error over Bridge V2 states, which is exactly the proxy used to train the verifier (Section 4.2) and may not track closed-loop success under distribution shift. Since Section 3 shows that sampling alone, even random sampling, reduces RMSE relative to greedy decoding, the reported 9% SIMPLER gain and 25% real-world gain cannot yet be attributed to verification. Please add closed-loop controls for random selection, majority voting, and best-of-N over the identical proposal distribution, at least on the SIMPLER tasks and ideally on the real-world tasks.","section":"Sections 5.2-5.3 and Appendix C"},{"comment":"The reported SIMPLER results are selected as the best over \\hat{N} in {5,9} and \\hat{K} in {8,16,32} for each task, while the V-GPS baseline is also swept and reported at its best; this turns the headline comparison into a best-case comparison rather than a comparison of fixed deployment configurations. Additionally, Section 5.3 reports 10 trials per real-world task with no error bars or confidence intervals. With 10 binary trials, a one-trial difference is 10 percentage points, so the per-task differences in Figure 3 are not established as robust. Please report either a single fixed configuration or all swept configurations, and provide per-task standard errors or confidence intervals for the real-world results.","section":"Appendix B.2 and Section 5.3"},{"comment":"The paper's first contribution, the inference-time scaling law, is not supported by the reported fit statistics. The text says the relationship follows an 'exponentiated power law' and gives the model log(e) ≈ log(a) + b log(k), but no fitted values of a and b, no R² values, no residual diagnostics, and no confidence intervals are reported for OpenVLA, CogACT, Octo, or SpatialVLA. The term 'exponentiated power law' is also inconsistent with the stated log-linear functional form, which is an ordinary power law. Please provide the fitted parameters and goodness-of-fit for each model and sampling method, or soften the claim to a qualitative observation.","section":"Section 3 and Figure 1"},{"comment":"The verifier is trained and evaluated against RMSE to the single expert action, which is an imitation proxy. The ablation in Appendix F claims that preference-based learning 'generalizes better in OOD settings,' but this is tested only on the same RMSE metric, so it does not establish robustness of closed-loop success. Since the entire pipeline rests on this labeling rule, the OOD success results do not disambiguate verifier quality from proposal-distribution quality. Please add at least one closed-loop comparison (for example on SIMPLER) between a verifier trained on RMSE-to-expert labels and a verifier trained on an alternative objective, or provide a direct analysis showing that the RMSE-based labels are necessary for the reported success-rate gains.","section":"Sections 4.2, 4.3, and Appendix F"}],"minor_comments":[{"comment":"Use consistent terminology: the model e = a k^b should be called a power law, not an 'exponentiated power law'.","section":"Section 3"},{"comment":"The claim that task performance 'grows nearly log-linearly with synthetic dataset size' is based on four points without error bars or a fitted curve; please show the fitted trend or state this as an observation.","section":"Section 5.5 and Figure 6"},{"comment":"Please clarify the OpenVLA baseline average in the SIMPLER comparison: the text reports RoboMonkey at 47.5% and a 9% average improvement, but the exact OpenVLA average is not stated in the main text.","section":"Section 5.2"},{"comment":"The notation lists at as [Δx, Δy, Δz, Δu, Δv, Δw, g]' and states both the state and action spaces are 7-dimensional; please clarify whether the state also includes the gripper as a binary state or only as an action component.","section":"Section 2 and Table of Notation"},{"comment":"There are formatting artifacts in the title and abstract ('T est-Time', 'V erification'); these should be cleaned in the camera-ready version.","section":"Abstract and Title"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a strong systems contribution, but the core attribution claim—that the learned verifier, rather than the sampling procedure, produces the reported gains—is not yet demonstrated in closed-loop evaluation. The missing selection-rule ablations are standard and within scope, and I would be supportive of a revised version that adds them, reports fixed configurations rather than best-over-sweep results, and provides variance information for the real-world trials."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a genuine attempt to bring test-time scaling to VLA control, and the synthetic preference pipeline is a real contribution. But the headline 25%/9% gains are not cleanly attributed to the verifier, because the closed-loop experiments never compare against random selection, majority voting, or best-of-N over the same candidate pool. Appendix C only reports RMSE-to-expert action error on Bridge V2 states—the exact metric the verifier was trained on—so it cannot support the closed-loop claim.\n\nWhat's new: first systematic scaling-law characterization for VLA action error, a cheap Gaussian-perturbation sampling method that gets most of the benefit of repeated sampling, and a scalable pipeline that turns an imitation dataset into 20M synthetic action preferences without human labels. The Appendix F ablation showing preference learning beats direct RMSE regression under distribution shift is a good sign that relative comparisons generalize better than point regression.\n\nThe real-robot OOD evaluation is valuable: four tasks with unseen objects and instructions, 10 trials each, consistent improvements over OpenVLA and V-GPS. That said, 10 trials per task with no error bars is thin. In simulation, they report the best result per task over hyperparameter sweeps, which inflates the average. The LIBERO-Long table honestly shows several tasks with no gain, but the paper doesn't discuss what limits transfer.\n\nThe scaling law itself is mostly order statistics: with an oracle that picks the minimum-RMSE sample, the expected minimum drops as you sample more, and that alone produces a power-law-looking curve. The paper cites the LLM-monkeys power-law work but doesn't separate the trivial component from anything VLA-specific. Rewriting that section to acknowledge this would make the actual contribution—verifier design and synthetic data—stand out more.\n\nThe verifier is trained on RMSE to a single expert action. If the expert action isn't optimal at a state, the verifier learns to imitate the expert rather than judge task success. The OOD results suggest the relative-preference formulation helps, but the assumption deserves explicit discussion and ideally a multi-action-label robustness test.\n\nNo code or data release is mentioned. Given the paper's systems contributions (SGLang support, synthetic data pipeline), releasing the verifier checkpoints and data-generation code is important for reproducibility.\n\nWho it's for: researchers working on inference-time scaling for robot foundation models, VLA deployment, or action verification. It deserves a serious referee, but with major revisions. I'd send it to review and ask for closed-loop selection ablations, error bars on real-world numbers, and code release. The core idea is likely right, but the paper currently overstates how much the learned verifier—rather than sampling diversity—drives the gains.","headline":"A useful generate-and-verify wrapper for VLAs, but the headline gains are not cleanly attributed to the verifier—needs closed-loop baselines, error bars, and a more honest framing of the scaling law.","tokens_in":17662,"tokens_out":3959,"would_cite":false,"duration_ms":39488,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sampling many robot actions and verifying the best one lifts real-world task success by 25 percentage points.","keywords":["test-time scaling","vision-language-action models","action verification","preference learning","robotic manipulation","inference-time scaling laws","synthetic preference data","generate-then-verify"],"falsifier":"Take a task with a deliberately imperfect demonstration, such as a teleoperator who slips and corrects, train the verifier with the paper's RMSE-to-expert labels, and compare closed-loop rollouts against a verifier trained on outcome-based labels from successful and failed trials. If the outcome-trained verifier wins, or if the closest-to-expert candidate consistently fails while a more distant candidate succeeds, then the expert-action proxy, not sampling and verification alone, is carrying the reported gains.","tokens_in":16554,"feed_emoji":"🤖","tokens_out":12222,"duration_ms":105048,"temperature":0.7,"pith_summary":"This paper argues that a robot policy that emits one action per observation leaves useful performance on the table: repeatedly generating candidate actions and then picking the best one with a learned verifier makes the same policy substantially more reliable. The authors first show that action error falls as an exponentiated power law as the number of candidate samples grows, across several generalist policies. They then build RoboMonkey, a wrapper that samples a small batch of actions, perturbs them with a Gaussian fit and majority voting to form a cheap proposal distribution, and scores the candidates with a vision-language-model verifier trained on synthetic pairwise comparisons. In their experiments the wrapper raises success on out-of-distribution real-robot tasks by 25 percentage points and on in-distribution simulation by 9 percentage points.","feed_headline":"Sample-and-verify lifts real-robot task success by 25 points","feed_subtitle":"On hard out-of-distribution tasks the wrapper raises an off-the-shelf policy from 35% to 60%.","key_machinery":"The load-bearing mechanism is the generate-then-verify loop with a learned proxy verifier. Formally, RoboMonkey samples $\\hat N$ actions from a VLA with positive temperature, fits $\\mathcal{N}(\\mu,\\Sigma)$ to the six continuous action deltas, sets the gripper by majority vote, samples $\\hat K$ refined candidates from that Gaussian, and executes $\\arg\\max_{\\tilde a} R_\\phi(\\tilde a,s,I)$. The reward model $R_\\phi$ is trained with a modified Bradley-Terry objective on synthetic preference pairs $(\\text{winner},\\text{loser})$ whose labels come from whether a candidate is closer in RMSE to the recorded expert action, with a margin term $\\alpha\\,|\\Delta^*-\\hat\\Delta|^2$ that helps the verifier separate clearly different actions. The observed relation $\\log(\\text{error})\\approx\\log(a)+b\\log(k)$ is the phenomenon the machinery exploits: because error falls predictably with more samples, a cheap proposal distribution plus a verifier can harvest that gain without expensive repeated calls to the base policy.","core_discovery":"The paper's central discovery is that test-time compute can be traded for control quality in vision-language-action models: when an oracle always picks the candidate closest to the expert action, action error decreases regularly with the number of samples, with $\\log(\\text{error})\\approx\\log(a)+b\\log(k)$ across several off-the-shelf policies. RoboMonkey converts this observation into a deployable loop. At each step it samples a handful of actions, fits a Gaussian over their translation and rotation deltas, takes the gripper state by majority vote, draws a larger candidate set from that Gaussian, and lets a 7-billion-parameter VLM-based verifier choose the action. The verifier is learned on automatically generated comparisons whose labels are derived from RMSE to the expert demonstration, and the paper reports that scaling this synthetic preference dataset yields roughly log-linear task improvement. The upshot is that pairing this wrapper with existing policies yields 25 percentage points absolute improvement on out-of-distribution real-robot tasks, 9 points on in-distribution simulation, and 7 points when both policy and verifier are fine-tuned on a new robot benchmark.","pith_inferences":["An implicit corollary is that the same wrapper could be tuned to a compute budget: given the fitted power-law exponent and the verifier cost, there is an optimal number of candidates per second of latency, and the paper's latency analysis reports the trade-off without deriving the optimum.","Because preference labels are RMSE to the recorded expert action, a natural extension is to label synthetic pairs with actual task success from rollouts or a task reward, which would remove the dependence on demonstration optimality.","The verifier currently scores single-step actions; a testable extension is to score multi-step action chunks or whole trajectories, which speaks directly to long-horizon benchmarks where local action error is not the right criterion."],"forward_implications":["The base policy does not need to be retrained: sampling and verification turn any off-the-shelf generalist VLA into a stronger closed-loop controller.","More synthetic comparison data produces better verification, and downstream success grows roughly log-linearly with that data.","Gaussian perturbation delivers most of the benefit of full policy resampling at a fraction of the latency, making the gains available near 1.5 Hz on a single GPU.","On a new robot setup, fine-tuning the verifier together with the policy adds about 7 percentage points over fine-tuning the policy alone.","The learned verifier is markedly less sensitive to visual and semantic distribution shift than the base policy, which is why the largest gains appear on out-of-distribution tasks."],"supporting_citations":[{"why":"supplies the open generalist policy whose actions are sampled, perturbed, and rescored at deployment.","marker":"[2]"},{"why":"provides the demonstration tuples used to synthesize the action-comparison labels for verifier training.","marker":"[11]"},{"why":"establishes the repeated-sampling scaling-law view that the paper adapts from language models to robot actions.","marker":"[22]"},{"why":"one of the generalist policies whose error-vs-sample curve is used to demonstrate the power law.","marker":"[33]"},{"why":"another generalist policy whose error-vs-sample curve supports the cross-model scaling claim.","marker":"[34]"},{"why":"the third additional policy on which the inference-time scaling law is plotted.","marker":"[35]"},{"why":"supplies the grouped pairwise-preference training recipe that the verifier's objective follows.","marker":"[36]"},{"why":"provides the vision-language backbone and reward-head architecture used to build the action verifier.","marker":"[38, 39]"},{"why":"defines the in-distribution simulation benchmark on which the 9-point gain is measured.","marker":"[41]"},{"why":"the value-guided re-ranking baseline whose reported results the paper compares against.","marker":"[42]"}],"fun_headline_variants":["RoboMonkey: 25-pt real-robot gain via sample-and-verify","Test-time sampling + verification: 25-point OOD gain for VLA policies","Sampling and verification: 25% absolute boost for robot policies","RoboMonkey: test-time sampling with VLM verification lifts success 25%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the recorded expert action is the best action at each state, because every synthetic preference label is just closeness to that one demonstration; if the human demonstration is suboptimal or the deployment state was never demonstrated, the verifier is trained to imitate rather than to succeed.","fun_headline_variants_meta":{"raw":{"variants":["RoboMonkey: 25-pt real-robot gain via sample-and-verify","Test-time sampling + verification: 25-point OOD gain for VLA policies","Sampling and verification: 25% absolute boost for robot policies","RoboMonkey: test-time sampling with VLM verification lifts success 25%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001083,"raw_usage":{"total_tokens":4576,"prompt_tokens":1039,"completion_tokens":3537,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":655,"completion_tokens_details":{"reasoning_tokens":3450}},"tokens_in":655,"tokens_out":3537,"duration_ms":25769,"temperature":1.0,"reasoning_tokens":3450,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:00:30.218938+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a task with a deliberately imperfect demonstration, such as a teleoperator who slips and corrects, train the verifier with the paper's RMSE-to-expert labels, and compare closed-loop rollouts against a verifier trained on outcome-based labels from successful and failed trials. If the outcome-trained verifier wins, or if the closest-to-expert candidate consistently fails while a more distant candidate succeeds, then the expert-action proxy, not sampling and verification alone, is carrying the reported gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the demonstration tuples used to synthesize the action-comparison labels for verifier training."}],"review_version":1}