{"id":"8e97816c-fa65-4985-add8-d3239a6667db","arxiv_id":"2508.09032","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ST-VLA combines depth maps with projected keypoint traces to enhance spatial-temporal understanding in vision-language-action models, reporting a 4 point success rate gain over SpatialVLA in SimplerEnv.","lead":"This paper introduces a visual prompt for robot control models: it overlays tracked keypoint trajectories onto depth maps, giving the model both spatial and temporal context in a single image. The authors report small success-rate gains over two prior VLA methods in the SimplerEnv benchmark after fine-tuning on only 52 trajectories.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 4% mean-SR gain over SpatialVLA is within the resolution of the evaluation: with ~11 episodes per task, a handful of successes separates the methods, and no seeds, error bars, or consistent aggregate numbers are provided.","rationale":"The reader's weakest assumption identifies tracker and depth-estimator reliability as the fragile component, and the authors themselves acknowledge that traces disappear during slow gripper rotation. That is a plausible failure mechanism and worth flagging. However, the more load-bearing condition for the central claim is statistical: the reported 4% improvement is within the noise floor of an ~11-episode-per-task, four-task evaluation with no repeated seeds, and the configuration was selected on the same benchmark. Even a perfectly reliable trace pipeline would not establish the benefit under this protocol. I therefore partially agree with the reader, and the appropriate verdict remains conditional: accept only if the authors provide consistent aggregate numbers, error bars from repeated seeds, a fixed evaluation protocol, and ideally code. The paper is a reasonable integration of existing components, and the qualitative demonstration in Figure 6 is suggestive, but the quantitative claim is not yet robust.","tokens_in":11002,"tokens_out":6257,"duration_ms":70220,"concrete_test":"Run a preregistered comparison of ST-VLA and SpatialVLA on the same four SimplerEnv tasks with at least 5 independent fine-tuning seeds, 30 evaluation episodes per task per seed, and a fixed protocol using buffer 30 and Variant C; report mean SR with 95% confidence intervals and per-task episode counts. If the mean SR difference is not at least 5 points and the interval excludes zero, the headline 4% gain cannot be distinguished from noise. Additionally, recompute TraceVLA's mean SR from Table I and correct the reported relative improvement if the Figure 4 value is a typo.","verdict_should_be":"UNCHANGED","load_bearing_attack":"ST-VLA's headline claim rests on mean Success Rate over exactly four SimplerEnv tasks. In Table I, SpatialVLA values are multiples of 9.1, implying about 11 episodes per task, so one episode changes a task's SR by 9.1 points and the four-task mean by about 2.3 points. The reported +4.0 mean SR therefore corresponds to roughly two additional successful episodes total across the benchmark. No repeated seeds, confidence intervals, or per-episode breakdowns are given, so the effect is not statistically resolvable. This is compounded by configuration selection: buffer size 30 and trace-rendering Variant C were chosen after inspecting results on these same four tasks (Tables II and III), making the reported gain a selected maximum rather than an unbiased estimate. The numerical basis is also internally inconsistent: averaging Table I's TraceVLA SR column gives 22.7%, not the 19.9% used in Figure 4, which changes the claimed '19%' improvement to about 15 percentage points; Table II lists Stack blocks GCS as 90.0 at buffer 30 while Tables I and III list 90.9. The paper's own Figure 5 admits that during slow gripper rotation, consecutive frames are nearly identical and traces become short or absent, which is a plausible mechanism for the ST-VLA SR drop on Put carrot relative to SpatialVLA. But the more general problem is that even if traces are perfectly reliable, the current protocol cannot distinguish a 4% improvement from sampling noise. Until repeated-seed results are provided, the central claim that Spatial Traces improves VLA performance should be treated as a hypothesis rather than a measured effect.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ST-VLA, a modification of a Vision-Language-Action (VLA) model in which keypoint traces computed by CoTracker are overlaid onto a depth map estimated by ZoeDepth, and the resulting depth-traces embedding is added to the RGB observation embedding before action prediction. The model is initialized from SpatialVLA/PaliGemma2 and fine-tuned with LoRA on only 52 Bridge trajectories. On four SimplerEnv tasks, the authors report a mean success rate of 38.1% for ST-VLA versus 34.1% for SpatialVLA and 19.9% for TraceVLA, and they present ablations over buffer size (Q2), trace rendering variant (Q3), and fine-tuning attribution (Q4). The central claim is that overlaying visual traces on depth maps simultaneously improves spatial and temporal understanding with very little fine-tuning data.","tokens_in":11361,"tokens_out":5140,"duration_ms":56295,"significance":"If the reported improvement is robust, the method is a simple and practically attractive visual-prompting addition to existing VLA models: it uses off-the-shelf components, requires only a small amount of fine-tuning data, and the paper includes several useful control experiments. The strengths are the clear architectural idea, the explicit hyperparameters and training setup, and the attempt to isolate the contribution of traces from fine-tuning in Q4. However, the current evidence is not sufficient to support the headline claim. There are no error bars or repeated-seed results, the effect size is comparable to one or two successful episodes on this evaluation grid, and the paper contains internally inconsistent aggregate numbers. The contribution is therefore interesting and worth pursuing, but the empirical case needs substantial revision before the claimed gains can be accepted.","major_comments":[{"comment":"The aggregate numbers are internally inconsistent. Averaging the TraceVLA column in Table I gives a mean GCS of 59.1% and a mean SR of 22.7%, but Figure 4 reports 51.1% and 19.9% for TraceVLA. If Table I is correct, the reported improvement over TraceVLA shrinks from about 18 percentage points to about 15 percentage points in SR; if Figure 4 is correct, Table I is wrong. Such an inconsistency undermines the reliability of the headline comparison and must be resolved by the authors.","section":"VI (Table I vs. Figure 4)"},{"comment":"No repeated seeds or confidence intervals are reported, and the evaluation is too coarse to resolve the claimed effect. In Table I, per-task success rates are multiples of 9.1, which implies about 11 episodes per task; one additional successful episode changes a task's SR by 9.1 points and the four-task mean by about 2.3 points. The reported +4.0 mean SR over SpatialVLA therefore corresponds to roughly two successful episodes total. Without multiple seeds or a per-episode breakdown, the central claim is statistically indistinguishable from sampling noise.","section":"V-C and VI (Tables I and IV)"},{"comment":"The headline configuration appears to have been selected on the same four evaluation tasks used for the final comparison. Buffer size 30 and rendering variant C were chosen after inspecting results on these tasks in Tables II and III, making the reported 4% gain a selected maximum rather than an unbiased estimate. The authors should report results for all configurations across multiple seeds, or reserve a validation set or held-out tasks for configuration selection, and disclose the selection procedure.","section":"VI, Q2/Q3 (Tables II and III)"},{"comment":"The attribution experiment is not defined precisely enough to support the conclusion that the gain 'stems from the trace-based visual prompting' rather than from fine-tuning. The rows 'Base model', 'Finetuned', and 'Traces 0-shot' do not state which base model is used, whether 'Finetuned' means fine-tuning without traces, and whether 'Traces 0-shot' uses the same CoTracker/ZoeDepth pipeline without any fine-tuning. A 2x2 comparison (fine-tune or not, traces or not) with explicit definitions is needed to justify the Q4 claim.","section":"VI, Q4 (Table IV)"}],"minor_comments":[{"comment":"The phrase 'increased for 4%' should be 'increased by 4%'; the same wording appears in Section I and the conclusion.","section":"Abstract and I"},{"comment":"The training objective is written as 'arg max ... MSE ...', but maximizing MSE is the opposite of minimizing a prediction error; this should be 'arg min' with the appropriate sign.","section":"III-A"},{"comment":"The symbol P_D is used for two different operations: predicting a depth map from an observation (line 3) and predicting depth embeddings from a rendered depth-traces map (line 5). Use distinct names to avoid confusion.","section":"III-B (Algorithm 1)"},{"comment":"The 'Stack blocks' GCS for buffer size 30 is listed as 90.0 in Table II but as 90.9 in Tables I and III; the discrepancy should be reconciled.","section":"VI, Q2 (Table II)"},{"comment":"The text states that variant B was excluded from testing, but Figure 7 illustrates all three variants; clarify whether B is shown only for illustration.","section":"VI, Q3 (Figure 7)"},{"comment":"The text says 'we kept consistent random seeds in all experiments', but no seed values are reported; listing the seeds would improve reproducibility.","section":"V-C"}],"recommendation":"major_revision","confidential_remarks":"The idea is simple and potentially useful, but the current version is not publishable in its present empirical form. The most serious issues are the internal numerical inconsistency and the lack of statistical resolution for a gain of a few percentage points. If the authors can provide repeated-seed results and correct the reported aggregates, the contribution may become acceptable; otherwise the claims should be substantially scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Max, quick take on arXiv:2508.09032. The idea is simple: overlay keypoint traces onto depth maps and feed the combined image to a VLA model. That combination hasn't been tried before as far as the citations show, and the paper's ablations on how to render the traces (depth of the tracked object vs. depth of the manipulator at that time) are genuinely useful. The authors also run a control showing that fine-tuning alone on the 52 Bridge trajectories hurts performance, which strengthens the claim that the visual prompting, not the fine-tuning, is driving any gain.\n\nThe soft spots are real and they cluster around the evaluation. Only four SimplerEnv tasks. With values that appear to step in units of ~9.1% per task, that's roughly 11 episodes per task, so one success changes a task's SR by ~9 points and the four-task mean by ~2.3 points. The reported +4% over SpatialVLA is about two successful episodes. No seeds, no error bars, no per-episode counts. On top of that, the numbers are internally inconsistent: TraceVLA's mean GCS/SR from Table I (59.1/22.7) don't match Figure 4 (51.1/19.9), and Table II has Stack Blocks at 90.0 while Tables I and III say 90.9. The authors also picked buffer size 30 and rendering variant C after looking at results on these same four tasks, so the headline number is a selected maximum, not an unbiased estimate.\n\nNone of this kills the method. It's a sensible incremental contribution, and the failure analysis in Figure 5 (traces vanish during slow gripper rotation) shows the authors are thinking about when it works. But the central claim that Spatial Traces improves VLA performance should be treated as a hypothesis until there are repeated seeds and consistent numbers. I'd send it to peer review because the idea is worth circulating and the ablations are well done, but I'd ask for a proper evaluation protocol first. For the reading group, it's a maybe — the visualization trick is easy to grasp, but the evidence isn't there to convince a skeptic. I wouldn't cite it in my own work until the numbers firm up.","headline":"Useful low-cost visual prompting trick, but the headline gain is within the noise of a four-task evaluation with no error bars and internally inconsistent numbers.","tokens_in":11889,"tokens_out":1991,"would_cite":false,"duration_ms":20180,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Spatial Traces overlays keypoint motion on predicted depth maps and reports a 4% mean success-rate gain over SpatialVLA in SimplerEnv with only 52 training trajectories.","keywords":["vision-language-action models","spatial-temporal understanding","visual prompting","depth maps","keypoint trajectories","robot manipulation","few-shot fine-tuning","SimplerEnv"],"falsifier":"On the same SimplerEnv tasks and seeds, run ST-VLA with the trace overlay corrupted (for example, random trace positions) and with the depth channel replaced by a constant, keeping the identical 52-trajectory fine-tuning; if mean success does not drop from 38.1% toward the SpatialVLA level of 34.1%, the depth-trace prompt is not the operative ingredient. A complementary check is to remove only the trace overlay while keeping depth, which should reproduce the drop reported in the paper's Table IV.","tokens_in":10843,"feed_emoji":"🤖","tokens_out":6901,"duration_ms":67706,"temperature":0.7,"pith_summary":"This paper tries to show that a Vision-Language-Action (VLA) model can gain spatial and temporal awareness at once by overlaying tracked keypoint motion onto predicted depth maps. Prior methods either supply depth or motion traces, but not both; the proposed ST-VLA combines them into a single visual prompt. If the claim holds, robot manipulation policies can improve without large fine-tuning datasets, since the reported gains come from only 52 real-robot trajectories. The central evidence is a mean success rate of 38.1% in SimplerEnv, against 34.1% for SpatialVLA and 19.9% for TraceVLA, with mean goal-condition success rising to 64.8%.","feed_headline":"Depth maps stamped with motion traces lift robot task success by 4%","feed_subtitle":"Overlaying tracked motion on depth maps lifts mean success by 4% using just 52 training trajectories.","key_machinery":"The load-bearing object is the depth-trace composite: a predicted depth map onto which the 2D trajectories of tracked key points are painted, each trace pixel taking the depth of the nearest object in the current frame. This single image carries both where objects are (depth) and where the manipulator has been (trajectory), so the model can read spatial structure and motion history from one visual prompt. The trace predictor CoTracker produces keypoint sequences over the observation buffer, ZoeDepth supplies the metric depth map, and the Ego3D positional encoder, originally from SpatialVLA, embeds the composite so that adding it to the SigLIP observation embedding injects spatial-temporal information into the policy.","core_discovery":"The paper claims that projecting keypoint trajectories onto depth maps gives a VLA policy both spatial structure and motion history from one image, and that this improves manipulation performance. Concretely, ST-VLA tracks key points over a buffer of up to 30 previous observations with CoTracker, estimates a depth map with ZoeDepth, and paints each trace onto the depth map so that trace pixels take the depth of the nearest object in the current frame. This depth-trace image is encoded with the Ego3D positional encoder from SpatialVLA and summed with the SigLIP observation embedding before being fed to a PaliGemma2-based VLA model. After fine-tuning on 52 Bridge trajectories with LoRA, ST-VLA reaches 38.1% mean success rate in SimplerEnv versus 34.1% for SpatialVLA and 19.9% for TraceVLA. Ablations show the gain comes from the trace prompting itself, that longer trace history (30 frames) is more reliable, and that rendering traces at object depth beats rendering them at the manipulator's historical depth.","pith_inferences":["Because the method only changes the visual input, the same depth-trace overlay could plausibly be applied to other VLA architectures that consume RGB or RGB-D images, not only the PaliGemma2-based SpatialVLA used here.","The identified failure mode (short or absent traces during slow gripper rotation) suggests that an adaptive trace generator that lengthens traces when motion between frames is small could close the remaining gap in rotation-heavy tasks.","If depth estimation improves, the approach might benefit disproportionately, since trace pixels inherit their values from the current depth map; uncertain depth may be a main ceiling on the reported 38.1% result.","The 52-trajectory result implies that for spatial-temporal VLA fine-tuning the bottleneck is the format of the visual prompt rather than data volume; that is testable by comparing trace-based prompting against equal fine-tuning budgets on other task suites."],"forward_implications":["On SimplerEnv Bridge tasks, adding spatial traces raises mean success rate to 38.1% from 34.1% for SpatialVLA and from 19.9% for TraceVLA, with mean goal-condition success at 64.8%.","The improvement is achieved with a minimal fine-tuning set: 52 Bridge trajectories, 1,969 steps, and two hours of LoRA training on a single GPU.","The trace prompting itself, not the Bridge fine-tuning, drives the gain: direct fine-tuning without traces lowers mean success by 10%, while zero-shot traces lower it by only 6.8%.","Longer interaction history helps: a 30-image observation buffer gives more consistent results than 7 or 15, especially on spatially demanding tasks such as stacking blocks.","Trace rendering method matters: assigning trace pixels the depth of the nearest object in the current frame outperforms assigning them their historical manipulator depth."],"supporting_citations":[{"why":"SpatialVLA supplies the base model, the Ego3D positional encoder, and the primary spatial baseline whose mean success ST-VLA exceeds.","marker":"[16]"},{"why":"TraceVLA provides the RGB trace-prompting baseline and the temporal-prompting idea that Spatial Traces extends to depth maps.","marker":"[21]"},{"why":"CoTracker generates the keypoint trajectories that are projected onto the depth map, carrying the temporal information.","marker":"[30]"},{"why":"ZoeDepth supplies the predicted depth maps in which the traces are embedded, carrying the spatial information.","marker":"[31]"},{"why":"SimplerEnv is the evaluation environment where the reported GCS and SR metrics are measured.","marker":"[33]"},{"why":"The Bridge dataset is the source of the 52 real-robot trajectories used for the minimal fine-tuning.","marker":"[36]"},{"why":"PaliGemma2 is the vision-language backbone whose checkpoint initializes the VLA model.","marker":"[32]"}],"fun_headline_variants":["Depth-trace images fuse spatial and temporal cues for VLA policies","Keypoint trails on depth maps boost robot success by 4%","Overlaying tracked motion on depth lifts VLA task performance","ST-VLA traces keypoint motion on depth maps for better control","Depth maps with motion history improve VLA manipulation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method only helps when the off-the-shelf keypoint tracker and depth estimator that generate its visual prompt give informative signals; when consecutive frames barely change, traces become short or absent and the claimed spatial-temporal benefit disappears.","fun_headline_variants_meta":{"raw":{"variants":["Depth-trace images fuse spatial and temporal cues for VLA policies","Keypoint trails on depth maps boost robot success by 4%","Overlaying tracked motion on depth lifts VLA task performance","ST-VLA traces keypoint motion on depth maps for better control","Depth maps with motion history improve VLA manipulation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1802,"prompt_tokens":904,"completion_tokens":898,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":812}},"tokens_in":520,"tokens_out":898,"duration_ms":9578,"temperature":1.0,"reasoning_tokens":812,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:30:18.152057+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the same SimplerEnv tasks and seeds, run ST-VLA with the trace overlay corrupted (for example, random trace positions) and with the depth channel replaced by a constant, keeping the identical 52-trajectory fine-tuning; if mean success does not drop from 38.1% toward the SpatialVLA level of 34.1%, the depth-trace prompt is not the operative ingredient. A complementary check is to remove only the trace overlay while keeping depth, which should reproduce the drop reported in the paper's Table IV.","supporting_citations":[{"cited_title":"Cotracker: It is better to track together,","cited_arxiv_id":null,"evidence_quote":"CoTracker generates the keypoint trajectories that are projected onto the depth map, carrying the temporal information."},{"cited_title":"Evaluating real-world robot manipulation policies in simulation,","cited_arxiv_id":null,"evidence_quote":"SimplerEnv is the evaluation environment where the reported GCS and SR metrics are measured."},{"cited_title":"Bridgedata v2: A dataset for robot learning at scale,","cited_arxiv_id":null,"evidence_quote":"The Bridge dataset is the source of the 52 real-robot trajectories used for the minimal fine-tuning."}],"review_version":2}