{"id":"10814987-897d-4b6c-9985-3809f76207b7","arxiv_id":"2608.10232","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FACT is a causal world-action model that predicts future video and task progress from the executed action, uses masked failure rollouts as consequence supervision, and improves manipulation success while reducing success-biased future hallucination.","lead":"FACT trains a robot's world model to predict the outcome of the exact action it picked, using both successful demonstrations and failed rollouts as honest training signals. The method improves manipulation success and reduces overly optimistic future predictions under bad actions in simulation and on real bimanual robots.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage-2 world predictions are trained on ground-truth actions but deployed on Stage-1 sampled actions; no DAgger/action-noise correction is given, so value-scoring and predicted-future metrics may not transfer to deployment.","rationale":"The reader's weakest assumption identifies the same mechanism I consider most load-bearing: the world branch is teacher-forced on ground-truth actions at training time but conditioned on Stage-1 predicted actions at inference. This gap is not a minor implementation detail; it directly affects the two most novel deployment behaviors, candidate scoring and failure-aware future prediction. The paper's diagnostics for hallucination and value traces use the executed dataset action as the condition, so they verify the world branch under the training distribution, not under the deployment distribution induced by Stage-1. The Limitations section omits this issue, and no corrective scheme (action noise, DAgger, or otherwise) is present. The central conceptual contribution, using failure rollouts as consequence supervision while masking the action imitation loss, remains plausible and is supported by the ablations; the gap is fixable and does not require rejecting the paper. Other concerns, such as missing error bars, unreleased code, and the simplicity of the progress target, are real but less directly tied to the method's defining inference-time interface. Because the reader already issued a CONDITIONAL verdict and my concern strengthens that condition rather than overturning it, I leave the verdict unchanged.","tokens_in":14815,"tokens_out":6961,"duration_ms":72165,"concrete_test":"Run a deployment-faithful evaluation: on held-out real-robot states, take N=4 Stage-1 sampled action candidates, score them with Stage 2 using the predicted actions, and compare (i) value-ranked success and (ii) future-frame PSNR/SSIM against the same evaluation with ground-truth actions in the action-conditioning slot. If metrics are statistically equivalent, the gap is benign; if value-ranked success or failure-future PSNR drops materially (e.g., more than 5 points) under predicted actions, then the train/inference action-condition shift is real, and the paper should either inject action noise into the ground-truth condition during training or add a DAgger-style correction before claiming deployment-grade scoring and hallucination reduction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (4) trains the value and future branches conditioned on the clean ground-truth action slot G, while Sec. 3.3's two-stage inference fills G with Stage-1 predicted actions. No action-conditioning noise, DAgger-style relabeling, or regularization addresses this train/inference shift. The mismatch is load-bearing because the strongest deployment claims are exactly the ones that use this branch: optional candidate scoring (Eq. 10) and reduced success-biased hallucination under bad actions. The PSNR/SSIM diagnostics (Table 4, Figure 5) and value traces (Figure 8) condition on executed dataset actions, not on Stage-1 samples, so they do not close the gap. If Stage-1 actions drift from the training action manifold, the value head can be miscalibrated for candidate ranking and the future branch can hallucinate for the same reason the paper aims to fix. The Limitations section (Sec. 5) does not mention this gap. This does not invalidate the conceptual contribution, but it makes the reported 92% scored success and the deployment-oriented hallucination claims conditional on an unverified assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FACT, a causal World-Action Model that first generates an action chunk and then predicts future video and task-progress value conditioned on the executed action. A teacher-forcing causal mask separates action imitation from world prediction, so failure rollouts can supervise the future and value branches while their action imitation loss is masked. The learned progress value can optionally rank sampled action candidates at inference. Experiments in RoboTwin simulation and five real-world bimanual tasks compare FACT with VLA and WAM baselines, ablate video co-training and the causal mask, measure future-prediction PSNR/SSIM on success and failure rollouts, and report failure-data scaling and value traces.","tokens_in":15001,"tokens_out":6643,"duration_ms":66195,"significance":"If the central claims hold, FACT makes a useful conceptual contribution: it converts failed rollouts into action-conditioned consequence supervision without corrupting imitation, and it provides a value interface with a clear compute/reliability tradeoff. The controlled ablations (video co-training, causal mask, failure masking) support the main design choices, and the held-out future-prediction metrics directly test the hallucination-reduction claim. However, the strongest deployment claims—optional candidate scoring and reduced hallucination under the model's own actions—are conditional on an unverified train/inference consistency assumption, and the success-rate comparisons lack statistical support. The paper would be a solid contribution after these gaps are addressed.","major_comments":[{"comment":"The value and future branches are trained conditioned on the clean ground-truth action slot G, but two-stage inference fills G with Stage-1 predicted actions. No action-noise augmentation, DAgger-style relabeling, or regularization is described to cover this shift, and the Limitations section does not mention it. This is load-bearing because optional candidate scoring (Eq. (10)) and the deployment-oriented hallucination claims rely on value and future predictions at Stage-1 sampled actions. Tables 4 and 8 and Figure 5 condition on executed dataset actions, not on Stage-1 samples, so they do not close the gap. Please provide evidence that predicted actions stay on the training action manifold, or add corrective training; without this, the reported 92% scored success and the reduced-hallucination claim are conditional on an unverified assumption.","section":"Section 3.2, Eq. (4), and Sec. 3.3."},{"comment":"Success-rate tables report point estimates without confidence intervals or significance tests. Each real-world cell averages only 20 trials, and many comparisons (e.g., Ours 82% vs. Ours w/ failure 89% on seen tasks; Ours w/ failure 87.5% vs. Motus 87.8% in simulation) are separated by amounts that could easily arise from sampling noise. Please report per-task confidence intervals or trial-level data and run paired significance tests for the main comparisons and ablations, including the claim that scoring alone does not help (Ours + scoring 79% vs. Ours 82%).","section":"Tables 1-3."},{"comment":"The failure-aware value target depends on the failure-onset indicator 1_fail(t+H), but the paper does not specify how failure onset is determined or what happens when it is 'not available' (Algorithm 2, line 6). Since lambda_fail=1 makes the target zero for every window that reaches the onset, the exact value target depends on this annotation protocol. Please describe the labeling procedure and report sensitivity of the results to it.","section":"Eq. (6) and Algorithm 2."},{"comment":"The hallucination diagnostic compares a success-only checkpoint with a failure-co-trained checkpoint; these differ both in the presence of failure data and in training-data size. To attribute the PSNR/SSIM improvement to failure-aware causal supervision, please add a control trained on the same failure data without the causal mask (or with the action imitation loss unmasked) and report future-prediction metrics for that variant. The policy ablation 'Ours w/ failed-action loss' shows that such a control exists, but it is not measured on the Table 4 metric.","section":"Section 4.3, Table 4."}],"minor_comments":[{"comment":"The notation G_T is defined as cumulative progress up to time T, but T is also used as episode length; since p_t = t/T under the uniform-reward convention, please state that G_T > 0 and clarify the episode indexing.","section":"Section 3.1, Eq. (3)."},{"comment":"With lambda_fail=1, the failure target is clipped to 0 for any window after failure onset, making the value target a hard floor rather than a gradual penalty; Appendix H says value scores are unclipped at inference, so please clarify how negative scores arise if the training target is clipped to [0,1].","section":"Section 3.3, Eq. (6), and Appendix H."},{"comment":"The ablation 'Ours + scoring' is not marked as optional; please clarify whether it also uses N=4 so the comparison with 'Ours w/ failure + scoring' is at matched compute.","section":"Section 4.2, Table 2."},{"comment":"The failure-data scaling plot reports only average success over three tasks; please add per-task results and error bars so the monotonic improvement can be assessed.","section":"Section 4.3, Figure 6."},{"comment":"Several cited sources are arXiv preprints dated 2026, including the main baselines; if final versions exist, the references should be updated.","section":"References."}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and within scope for a robotics or machine-learning venue. The required revision should focus on closing the train/inference action-conditioning gap—either through additional training (action noise, DAgger-style relabeling) or through deployment-conditioned diagnostics—and on adding statistical support to the success-rate tables. I would not reject on the basis of the action-conditioning mismatch because it is fixable with additional experiments, but it is load-bearing for the paper's strongest claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the short version: FACT is a genuinely useful contribution to the world-action model line. It reorders the prediction sequence—action first, then future video and task-progress value—and uses a teacher-forced clean action condition so that failure rollouts can supervise the future and value branches without being treated as imitation targets. That's a clean idea, and the paper backs it up with controlled ablations and real-world experiments.\n\nWhat's new: the causal action-first order, the mask that prevents the noisy predicted action from attending to the clean action, and the failure-masked co-training objective. The ablations are the strongest part: removing video co-training, removing the causal mask, or letting failed actions update the policy all hurt, and the failure-data scaling curve is monotonic. The PSNR/SSIM diagnostic on held-out futures is a nice way to show reduced success-biased hallucination on failure rollouts while leaving success-rollout futures nearly unchanged.\n\nThe soft spots are real but not disqualifying. The success-rate tables report point estimates from 20 trials per real-world task with no confidence intervals or significance tests, so the gaps between 89% and 92% are less certain than they look. More importantly, the value and future branches are trained conditioned on the clean action slot (ground-truth or executed actions in the data), while at inference Stage 2 conditions on Stage-1 predicted actions. There is no DAgger-style relabeling, action noise, or explicit regularization to cover that shift. The paper's Limitations section doesn't mention it. This could make the optional candidate scoring and the deployment-time hallucination claims optimistic if predicted actions drift from the training action manifold. The value traces in Figure 8 are taken from real rollouts and do show the expected drop, which is evidence that the shift isn't catastrophic, but the concern should be addressed.\n\nAlso, no code or data is released, which makes the empirical claims harder to verify. That's a common weakness in this area, but worth noting.\n\nWho's this for? Anyone working on world-action models, VLA policies with predictive objectives, or learning from failed demonstrations. It deserves a serious referee: the method is novel, the experiments are mostly well controlled, and the open questions are addressable. I'd recommend sending it to review with a request for error bars and some treatment of the action-conditioning distribution shift—either action noise during training or a DAgger-style fine-tuning step. That would make the deployment claims substantially stronger.\n\nBest.","headline":"A solid method paper that turns failed rollouts into consequence supervision; the train/inference action-conditioning gap is real but not fatal.","tokens_in":15569,"tokens_out":3081,"would_cite":true,"duration_ms":29847,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A failure-aware world model that predicts what each action causes turns failed rollouts into training data, lifting real-world task success from 82% to 89% (92% with optional candidate scoring).","keywords":["World-Action Models","Robot Foundation Models","Manipulation","Failure-aware training","Future prediction","Task-progress value","Causal attention mask","Bimanual manipulation"],"falsifier":"Run two-stage inference with increasingly perturbed Stage-1 actions while scoring with the learned value: if value scores and success stay flat as task outcomes worsen, the progress value is not truly reading action quality; if they drop monotonically with action noise, the action-conditioned causal training is doing the claimed work.","tokens_in":14572,"feed_emoji":"🤖","tokens_out":13276,"duration_ms":114531,"temperature":0.7,"pith_summary":"FACT aims to establish that failed robot rollouts can be turned into structured supervision for a world model — a model that predicts how the scene evolves under robot actions — instead of being discarded. The key move is causal ordering: the model first produces an action chunk, then predicts the future video and an action-conditioned task-progress value (a score estimating how much of the task would be completed after that action). During training, success demonstrations supervise action, future, and value, while failure rollouts keep only the future and value losses, with the progress target lowered for actions that enter failure. On five real-world bimanual tasks, the paper reports that failure-aware training raises average success from 82% to 89%, and optional best-of-4 candidate scoring (choosing the action with the highest predicted progress) reaches 92%; on held-out variants the corresponding numbers are 67% to 77% and 82%. This matters because prior world-action models train mostly on successful demonstrations and therefore tend to hallucinate successful futures under bad actions, while FACT gives the model a concrete reason to know what wrong behavior leads to.","feed_headline":"Training on failed rollouts lifts robot success to 89 percent","feed_subtitle":"A causal world model predicts the consequence of each action, converting robot mistakes into supervision.","key_machinery":"The load-bearing mechanism is the teacher-forced action-conditioned causal mask over the token sequence $[P, A, G, V, I]$ in a shared causal diffusion transformer. $P$ is the observation prefix, $A$ is the noisy predicted action, $G$ is the clean ground-truth action, $V$ is the value token, and $I$ is the future-video segment; attention is arranged so that $V$ and $I$ attend to $G$ while $A$ cannot see $G$. This separates what to imitate from what to predict. All three outputs are trained with the same flow-matching denoising objective (a generative loss that learns to map noise to clean targets), and a lightweight action adapter adds robot-specific capacity to the shared video backbone. The value target is normalized progress $p_t = G_t/G_T \\in [0,1]$, with a clip-down penalty $\\lambda_{\\text{fail}}$ applied to windows whose executed action enters failure, so the value head encodes action quality. At inference, Stage 1 denoises an action chunk, that chunk fills the $G$ slot, and Stage 2 denoises value and optionally future video, making best-of-$N$ action scoring a direct use of the learned value.","core_discovery":"The paper's central claim is that an action-conditioned causal interface turns failure rollouts into valid future targets: the world branch predicts what the executed action actually leads to, while the action branch is protected from imitating failed behavior. Concretely, FACT supervises success data with the full loss $w_a\\mathcal{L}_a + w_v\\mathcal{L}_v + w_I\\mathcal{L}_I$ and failure rollouts with $w_v\\mathcal{L}_v + w_I\\mathcal{L}_I$, using a teacher-forced clean action token as the condition for value and future-video prediction. The value target is normalized task progress clipped down by a failure penalty, so the value head learns to distinguish good from bad actions. Empirically the failure-aware model reaches 87.5% average success across 50 simulated bimanual tasks, 89% on seen real-world tasks, and 77% on unseen variants, with optional value-based candidate scoring raising the real-world numbers to 92% and 82%. On held-out future-prediction samples, failure-aware training raises failure-rollout PSNR from 19.51 to 25.92 while leaving successful-rollout PSNR nearly unchanged, which the paper reads as reduced success-biased future hallucination.","pith_inferences":["Because the value and future branches are trained on clean executed actions but deployed on Stage-1 predicted actions, the reported gains may depend on how well the policy's predicted actions stay on the demonstration action manifold; adding action noise during training would be a direct test and a likely fix.","The same action-conditioned consequence interface could support online correction loops that re-label bad rollouts or reinforcement learning from negative experience, using the predicted progress drop as a shaped reward; the paper names this direction but does not implement it.","In principle the failure-aware value could be learned in settings without action labels, such as human egocentric video, by first inferring actions and then predicting consequences, though the paper does not evaluate that transfer."],"forward_implications":["Failure rollouts become reusable supervision: real-world seen-task success rises from 82% to 89%, and unseen variants from 67% to 77%.","The progress value is a usable ranking signal only after consequence training: best-of-4 scoring reaches 92% on seen tasks, whereas scoring without failure data stays at 79%.","Failure-aware training makes futures under bad actions realistic, raising failure-rollout PSNR from 19.51 to 25.92 while leaving successful futures unchanged, which reduces success-biased hallucination.","The benefit scales with failure data: on a three-task simulation subset, success grows monotonically from 32.7% with no failure rollouts to 57.3% when failure data is fully included.","Video co-training itself is a strong regularizer: removing it drops real-world seen-task success from 82% to 58%."],"supporting_citations":[{"why":"Supplies the pretrained video-diffusion backbone whose weights initialize FACT's shared transformer.","marker":"[51]"},{"why":"The strongest world-action-model baseline whose simulation and real-world success rates FACT is compared against.","marker":"[7]"},{"why":"The large-scale policy baseline used as the reference upper performance level FACT approaches without robot pretraining.","marker":"[4]"},{"why":"The video-model policy baseline that exemplifies training on successful demonstrations, the success-biased behavior FACT targets.","marker":"[6]"},{"why":"The prior causal world-model formulation whose action/future ordering FACT inverts into action-then-consequence.","marker":"[5]"},{"why":"Provides the flow-matching objective used to train action, value, and future-video tokens jointly.","marker":"[49]"},{"why":"Motivates the inference-time candidate scoring interface that the learned value head plugs into.","marker":"[50]"},{"why":"Supplies the 50-task bimanual simulation benchmark used for the main simulation comparison.","marker":"[41]"},{"why":"A flow-based vision-language-action baseline included in the comparison tables.","marker":"[3]"}],"fun_headline_variants":["Causal model turns robot mistakes into 89% success","Training on failures slashes success bias in robot actions","Failure rollouts teach robot model to reach 89% success","Robot world model learns from bad actions, hits 89%","FACT model: failure data lifts real-world success to 89%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The value and future branches are trained on the exact actions that were executed, but at inference they are fed the model's own predicted actions, and no training-time correction explicitly closes that gap.","fun_headline_variants_meta":{"raw":{"variants":["Causal model turns robot mistakes into 89% success","Training on failures slashes success bias in robot actions","Failure rollouts teach robot model to reach 89% success","Robot world model learns from bad actions, hits 89%","FACT model: failure data lifts real-world success to 89%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000616,"raw_usage":{"total_tokens":2883,"prompt_tokens":992,"completion_tokens":1891,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":1806}},"tokens_in":608,"tokens_out":1891,"duration_ms":12746,"temperature":1.0,"reasoning_tokens":1806,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:11:06.163544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run two-stage inference with increasingly perturbed Stage-1 actions while scoring with the learned value: if value scores and success stay flat as task outcomes worsen, the progress value is not truly reading action quality; if they drop monotonically with action noise, the action-conditioned causal training is doing the claimed work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates the inference-time candidate scoring interface that the learned value head plugs into."}],"review_version":1}