{"id":"d7d9553b-7ce2-45c3-be1c-8ee3c6e4aab9","arxiv_id":"2509.14380","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An LLM/VLM coaching loop that generates curricula and reward functions enabled MARL agents to learn coordinated gate passing, seesaw balancing, and bimanual pot lifting, with one policy transferred to real quadrupeds.","lead":"CRAFT uses large language and vision models to automatically design training curricula and reward functions for multi-robot coordination, then trains the robots with reinforcement learning. The framework succeeded on simulated quadruped navigation and bimanual lifting, and one navigation policy transferred to real robots, suggesting a path to reducing human reward engineering in robotics.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The VLM success gate is uncalibrated: false-positive evaluations can advance the curriculum prematurely, and the paper reports no agreement between VLM judgments and ground-truth success.","rationale":"The reader's weakest_assumption identifies the VLM-based evaluation and advice loop as the fragile premise. I agree, and I sharpen it to a specific, testable failure mode: the VLM success gate in Section IV-C is used to decide whether to advance to the next subtask or refine the reward. The paper provides illustrative examples of VLM evaluations but no quantitative calibration against ground truth. Because the curriculum progression is monotonic once a subtask is marked successful, a false positive cannot be undone by later refinement, making it more dangerous than a false negative. The reported maxima over five trials and the permissive definition of 'effective curricula' in Table I further mean that a noisy gate could explain the results without reliable coaching. Other concerns, such as missing strong baselines and best-seed selection, are secondary and already noted by the reader. The concrete test I propose—measuring VLM-gate accuracy and comparing against a ground-truth-gated run—would settle whether the coaching signal is actually doing the causal work. This does not move the verdict: the reader's CONDITIONAL assessment already captures the need for such validation, and the paper's positive results (ablations, hardware transfer) are not undermined by the missing measurement alone. The condition should be made explicit: the paper should report VLM evaluation accuracy and, ideally, a ground-truth-gated ablation.","tokens_in":18879,"tokens_out":5450,"duration_ms":49505,"concrete_test":"Collect the exact VLM evaluation prompts, snapshot images, and state trajectories used during the five trials (or regenerate them from the released code), and label every evaluation with the environment's ground-truth success outcome. Query the evaluation VLM under the original settings and compute the confusion matrix, especially the false-positive rate (VLM says Success when ground truth says Failure). Then rerun CRAFT on one environment, e.g., Quadruped Gate, replacing the VLM gate with the ground-truth success signal while keeping all other modules fixed. If the false-positive rate exceeds about 10%, or if the ground-truth-gated run achieves comparable or better success, the paper's claim that VLM coaching is necessary for CRAFT's success is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim treats CRAFT's coaching pipeline as the cause of the learned coordination, but the pipeline's internal decision to advance from subtask k to k+1 (or to trigger reward refinement) is made by the evaluation VLM in Section IV-C. That gate is never validated against the environment's ground-truth success condition. The paper reports no false-positive rate, no false-negative rate, and no inter-rater consistency for the VLM, and it does not ablate the gate by replacing VLM judgments with the ground-truth success signal. Section IV-C itself concedes VLMs 'lack a native understanding of 3D motion,' and Section VI concedes trial variability from foundation-model stochasticity. The dangerous failure mode is a false positive: if a failed rollout is declared successful, the curriculum advances prematurely and the reward-refinement loop never corrects the reward. A false negative only wastes an iteration, and the paper's J=3 refinement budget plus final best-policy selection can absorb that cost. Because the headline success rates (90%, 60%, 100%) are maxima over five trials, and Table I defines an 'effective' curriculum as any nonzero success, a noisy gate that occasionally passes bad policies could produce the reported numbers without reliable coaching. This is the least secure link in the causal chain from foundation-model feedback to successful coordination.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents CRAFT, a framework in which LLMs generate a curriculum and reward functions for multi-robot MARL tasks, and a VLM evaluates trained policies and provides advice that is used to iteratively refine rewards. The authors evaluate CRAFT on two multi-quadruped navigation tasks and one bimanual manipulation task, report success-rate advantages over several baselines, and demonstrate transfer of a learned quadruped policy to physical hardware. The central claim is that foundation-model coaching enables learning of coordination behaviors that are infeasible with standard MARL training and hand-specified rewards.","tokens_in":19289,"tokens_out":4751,"duration_ms":41745,"significance":"If the reported effects are real, CRAFT is a valuable step toward reducing human effort in curriculum design and reward engineering for multi-robot coordination, and the real-world transfer result is a useful existence proof. The paper's strengths include a clearly described pipeline, full prompts and example outputs in the appendix, and the demonstration that LLM/VLM-generated rewards can support successful MARL training in three nontrivial tasks. However, the evaluation as reported does not yet establish that the coaching loop is the cause of the success: the key internal gate is unvalidated, and the headline numbers are selected maxima.","major_comments":[{"comment":"The headline success rates (90%, 60%, 100%) are the maximum values over five experiments, and Fig. 6 shows only the top-three of five runs; no per-seed averages, error bars, or distributions are provided. Because the comparison against baselines uses this best-case selection, the claim in Q1 that CRAFT 'achieves the highest success rate on every environment' is not supported by the reported evidence. Please report mean and variance over all five seeds, or otherwise show that the advantage is not driven by one lucky run.","section":"Section V-B, Fig. 6 and Table I"},{"comment":"The VLM-based success gate that advances the curriculum is never validated against the environment's ground-truth success condition. The paper reports no false-positive rate, false-negative rate, or agreement statistics between the VLM's judgment and the simulator's success flag, and it does not ablate the gate by substituting ground-truth success. Since a false-positive VLM judgment can move training to the next subtask before the current behavior is actually learned, the internal decision-making of CRAFT is a load-bearing but unmeasured component; please add a calibration study or an oracle-gate ablation.","section":"Section IV-C and Section IV-D"},{"comment":"The 'effective curricula ratio' is defined as the fraction of curricula that achieve any nonzero success rate; under this definition, a curriculum with a single successful episode out of 100 is counted as effective. This metric is too weak to support the Q3 conclusion that VLM-guided reward refinement 'improves curricula reliability,' because reliability should be reflected in the full success-rate distribution rather than in a binary nonzero threshold. Please report per-curriculum summary statistics or use a threshold that reflects meaningful success.","section":"Section V-B, Table I"}],"minor_comments":[{"comment":"There are several typos in the prompt text, e.g., 'chage' for 'change' and 'Varaibles' for 'Variables'; please proofread the appendix.","section":"Appendix A.1, Prompt 1"},{"comment":"The example in Fig. 3 shows a single refinement; the paper states that J=3 iterations were sufficient, but it does not report the number of refinement iterations actually needed per subtask across runs. Please include this information or clarify.","section":"Section IV-D, Fig. 3"},{"comment":"The description of no_refinement should specify how many reward resamples are allowed and whether the resampling uses any information from the failed reward beyond stochasticity.","section":"Section V-A2"},{"comment":"The hardware section mentions 'action component-wise scaling' but does not describe how the scaling was determined; please provide details or a reference.","section":"Section V-C"},{"comment":"The sentence 'its unreliable curriculum generation strongly degrades its average performance shown in Fig. 6' refers to an average, but Fig. 6 plots top-three curves; please reconcile the terminology.","section":"Section V-B and Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a robotics venue, but the evaluation reporting needs attention before it can be accepted. The missing validation of the VLM success gate is the key technical risk; if the authors can provide a calibration analysis or an oracle-gate ablation, the central claim would be substantially strengthened. The choice of maximum and top-three statistics is a further concern that should be addressed in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely integrated system that uses LLMs to generate a subtask curriculum and reward code, a VLM to evaluate rollouts, and a second VLM to advise reward rewrites, all in one closed loop. It demonstrates learning on three coordination tasks, with a hardware transfer on the quadruped gate task. That is a real contribution, and the paper is worth a serious look. But the evidence is presented in a flattering way, and the VLM success gate is never validated. What is new: individual pieces exist, such as LLM reward design, VLM reward models, and LLM curriculum generation, but the closed-loop integration is new, and the ablation structure (no-curriculum, no-refinement) targets the right questions. The appendix includes full prompts and worked examples, which helps reproducibility of the method in spirit. The hardware validation is a plus. Soft spots: first, reporting. Figure 6 plots the top three of five seeds, and Table I reports maximum success rates. Maxima over five runs can flatter a stochastic pipeline. Means, medians, and per-seed curves would be stronger. The effective-curricula ratio helps, but it is a loose metric. Second, the VLM success gate is uncalibrated. The decision to advance subtasks or trigger reward refinement depends on the evaluation VLM, but there is no false-positive or false-negative rate, and no comparison with the simulator's ground-truth success. A false positive advances the curriculum prematurely and is not self-correcting; false negatives are absorbed by the refinement budget and final best-policy selection. This is a legitimate concern, not a fatal flaw, because the final success rates are measured against ground truth, but without gate validation the claim that CRAFT works because of accurate VLM evaluation is under-supported. Third, the claim that no prior RL method solves these tasks without demonstrations is not tested against strong baselines. Env reward and example reward are weak; a hand-designed curriculum or another curriculum method would be more convincing. Fourth, no code is released, and the appendix's placeholder github_link is not adequate. I would want code before fully trusting the numbers. Also worth noting: the example reward provided in the prompt is already a dense, reasonable reward, so the framework is automatic only after receiving a human-designed template. That does not invalidate the claim, but it does temper the framing. Who this is for: researchers working on foundation models for RL, reward design, and multi-robot coordination. It is a solid systems paper. I would send it to peer review and ask for per-seed results, VLM agreement analysis, code, and a stronger baseline before accepting.","headline":"A credible closed-loop LLM/VLM coaching framework for multi-robot coordination that deserves peer review, but the VLM success gate is unvalidated and the headline numbers are maxima over five seeds.","tokens_in":763,"tokens_out":776,"would_cite":true,"duration_ms":33150,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that a foundation-model coaching loop of LLM-generated curricula, LLM-generated rewards, and VLM-guided reward refinement can teach teams of robots long-horizon coordination tasks that standard multi-agent reinforcement…","keywords":["multi-agent reinforcement learning","curriculum learning","large language models","vision-language models","reward design","robot coordination","reward refinement","quadruped navigation"],"falsifier":"Count, on a sample of 100 saved rollouts, how often the evaluation VLM's success or failure verdict disagrees with a ground-truth check of whether the task (gate crossed, or pot lifted above 0.1 m with tilt under 30 degrees) was actually completed; if disagreement exceeds about 10%, the loop's premise is violated.","tokens_in":18706,"feed_emoji":"🤖","tokens_out":6707,"duration_ms":52106,"temperature":0.7,"pith_summary":"The paper argues that foundation models can act as an automated coach for multi-robot coordination, replacing the human-designed curricula and hand-calibrated rewards that multi-agent reinforcement learning usually needs. CRAFT generates a curriculum of subtasks with a language model, writes a reward function for each subtask in executable code, and then refines those rewards through a visual-language-model evaluation and advice loop until the learned policy succeeds. The claim is supported by experiments in three long-horizon coordination benchmarks where CRAFT reaches up to 90% success in Quadruped Gate, 60% in Quadruped Seesaw, and 100% in Two Arm Lift, while the environment rewards alone often fail completely. A zero-shot transfer of one learned policy to physical quadruped robots achieved 65% success. If the claim holds, it offers a path to teach teams of robots new coordination skills without expert reward engineering.","feed_headline":"AI coach teaches robot teams to coordinate without hand-built rewards","feed_subtitle":"CRAFT splits coordination into subtasks and refines rewards with AI visual feedback, reaching 100% success.","key_machinery":"The load-bearing mechanism is the closed coaching loop made of five modules: a curriculum LLM that decomposes the task and refines several candidate curricula into one; a reward-generation LLM that turns each subtask description into executable Python reward code with named reward components; a policy-evaluation VLM that judges rollouts from snapshot images and state metrics; an advice VLM that reads failure reasons and plots of per-component reward learning curves and proposes concrete changes; and a refine LLM that rewrites the reward accordingly. Sequential subtasks are trained by initializing from the previous policy, resetting the exploration noise and the value network to avoid loss of plasticity. The working parts are the curriculum, the executable dense rewards, and the VLM-guided refinement loop acting together.","core_discovery":"The central discovery is that an LLM-VLM \"coaching\" loop can convert a single natural-language task description into a trainable curriculum plus rewards, and that this loop, rather than any single component, is what makes complex coordination learnable. In the paper's own framing, curriculum learning changes the problem from optimizing one long-horizon reward to optimizing a sequence of rewards, and CRAFT is a method for automatically producing that sequence. The authors report that without the curriculum (even with CRAFT's final rewards) and without the VLM-based refinement, success rates fall substantially, and that the environment-provided rewards almost never succeed.","pith_inferences":["A natural extension the authors do not pursue is to use the learned per-component reward curves to detect when a task has multiple equally good coordination strategies; the VLM advice could then be prompted to preserve diversity rather than converge to a single behavior.","CRAFT's reliance on snapshot images for evaluation could be stress-tested by swapping the VLM judge for a scripted success detector; if success stays high, the visual pipeline is redundant, and if it collapses, visual feedback is the key ingredient.","The same loop could be applied to asymmetric roles (where agents have different capabilities) by letting the curriculum LLM name role-specific subtasks instead of a shared team reward, which the paper's team-reward framing does not address.","The reported 65% real-world transfer suggests that the gap to robust deployment is mostly perceptual and control calibration; a promising next step is to close that gap by fine-tuning the transferred policy with a few real-world rollouts using the same VLM evaluation loop."],"forward_implications":["If CRAFT works as reported, multi-robot coordination tasks that currently require expert curriculum design can be tackled from a language prompt alone, reducing human effort to environment setup and state-variable definitions.","The same coaching loop should apply to other long-horizon multi-agent problems beyond the three benchmarks, since the curriculum and reward modules only need the environment description and a set of interpretable state variables.","Real-world transfer at 65% success without fine-tuning suggests that policies trained in simulation by CRAFT can serve as initializations for on-robot learning with modest additional data.","The failure analysis suggests that standard dense environment rewards can lock policies into local optima (e.g., grasping without lifting); CRAFT's iterative reward refinement is a way to escape such optima automatically.","Because CRAFT logs per-component reward curves, the refinement loop doubles as a diagnostic tool: it identifies which reward term is too weak or too dominant, providing human-readable explanations of training failures."],"supporting_citations":[{"why":"Establishes the pattern of having an LLM generate reward functions in executable code, which CRAFT reuses for each subtask.","marker":"[13]"},{"why":"Shows vision-language models can serve as zero-shot reward and evaluation signals from visual observations, the basis of CRAFT's VLM evaluation and advice modules.","marker":"[14]"},{"why":"Demonstrates LLM-based automatic curriculum design for robot skills, which CRAFT extends from single-robot to multi-robot coordination.","marker":"[20]"},{"why":"Provides the cooperative multi-agent PPO algorithm with decentralized execution used to train all CRAFT policies.","marker":"[29]"},{"why":"Supplies the Quadruped Gate and Quadruped Seesaw benchmark environments used to evaluate CRAFT.","marker":"[39]"},{"why":"Supplies the bimanual manipulation simulation environment used for the Two Arm Lift task.","marker":"[40]"},{"why":"Motivates the practice of resetting the value network during sequential subtask training to avoid loss of plasticity.","marker":"[26]"}],"fun_headline_variants":["AI coach auto-splits robot tasks for coordination","No hand-coded rewards: LLM-VLM coach trains robot teams","CRAFT: LLM and VLM coach robots through subtask curriculum","LLM-VLM coach transforms task description into coordinated robot skills","Robot teams learn coordination from AI coach with auto rewards"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline's reliability rests on the VLM consistently recognizing failure from snapshot images and state metrics and on the advice VLM proposing reward changes that actually improve learning; the paper itself notes that foundation models lack native 3D understanding and that trial outcomes vary due to stochasticity.","fun_headline_variants_meta":{"raw":{"variants":["AI coach auto-splits robot tasks for coordination","No hand-coded rewards: LLM-VLM coach trains robot teams","CRAFT: LLM and VLM coach robots through subtask curriculum","LLM-VLM coach transforms task description into coordinated robot skills","Robot teams learn coordination from AI coach with auto rewards"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00065,"raw_usage":{"total_tokens":2970,"prompt_tokens":921,"completion_tokens":2049,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":1963}},"tokens_in":537,"tokens_out":2049,"duration_ms":13170,"temperature":1.0,"reasoning_tokens":1963,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:50:46.000282+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Count, on a sample of 100 saved rollouts, how often the evaluation VLM's success or failure verdict disagrees with a ground-truth check of whether the task (gate crossed, or pot lifted above 0.1 m with tilt under 30 degrees) was actually completed; if disagreement exceeds about 10%, the loop's premise is violated.","supporting_citations":[{"cited_title":"Eureka: Human-level reward design via coding large language models,","cited_arxiv_id":null,"evidence_quote":"Establishes the pattern of having an LLM generate reward functions in executable code, which CRAFT reuses for each subtask."},{"cited_title":"Vision-language models are zero-shot reward models for reinforce- ment learning,","cited_arxiv_id":null,"evidence_quote":"Shows vision-language models can serve as zero-shot reward and evaluation signals from visual observations, the basis of CRAFT's VLM evaluation and advice modules."},{"cited_title":"Curricullm: Automatic task curricula design for learning complex robot skills using large language models,","cited_arxiv_id":null,"evidence_quote":"Demonstrates LLM-based automatic curriculum design for robot skills, which CRAFT extends from single-robot to multi-robot coordination."},{"cited_title":"Mqe: Unleashing the power of interaction with multi-agent quadruped envi- ronment,","cited_arxiv_id":null,"evidence_quote":"Supplies the Quadruped Gate and Quadruped Seesaw benchmark environments used to evaluate CRAFT."},{"cited_title":"Decentralized Navigation of a Cable-Towed Load using Quadrupedal Robot Team via MARL","cited_arxiv_id":"2503.18221","evidence_quote":"Motivates the practice of resetting the value network during sequential subtask training to avoid loss of plasticity."}],"review_version":2}