{"id":"7d90290f-372a-4810-9c45-76ba627755ca","arxiv_id":"2608.08749","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A value-scored hierarchical memory, initialized offline and refined by online rollouts, improves frozen vision-language-action policies on long-horizon manipulation tasks by a few percentage points.","lead":"This paper adds a memory system to pretrained robot policies that learns which past moments to remember from training demonstrations and real robot practice. It reports small success-rate gains on two long-horizon manipulation benchmarks while keeping the underlying policy frozen.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Value estimator's training signal and necessity are unspecified, so the claimed value-guided memory evolution is not yet established.","rationale":"We read the paper as claiming that a value-guided memory writer, initialized offline and refined online, is what improves a frozen VLA policy. The causal chain is: queries from the frozen encoder -> learned value estimator -> high-value and transition writing -> retrieval -> gated cross-attention -> better actions. The least secure link is the value estimator, because the paper never specifies an objective for F_phi. Without a loss or a demonstration that V_t tracks the claimed events, the memory banks could be operating as generic retrieval. The reported gains are also reported without variance, so even the 'improves' part is uncertain; however, the reader's conditional verdict already captures this. Our proposed ablation directly tests whether the value signal is necessary. If it is not, the headline claim about learning what to remember is unsupported. If it is, the missing loss should still be reported. We therefore keep the verdict CONDITIONAL and agree with the reader's weakest assumption.","tokens_in":4498,"tokens_out":6418,"duration_ms":72454,"concrete_test":"Run an ablation on LiberoLong-10 (and ideally on RMBench SwapBlocks) in which F_phi's learned V_t is replaced by a fixed recency-only score for writing to the elite and transition banks, while keeping retrieval, bank structure, and gated cross-attention exactly identical. If success rates remain within one standard error of the reported '+ Online' numbers, the value-guided component is not load-bearing; if they drop materially, the concern is resolved. Report at least 3 seeds with confidence intervals.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that Q_t = E_theta(o_t) carries enough information for F_phi to output a writing score V_t that aligns with grasp success, contact loss, and task-stage transitions, and that temporal changes |V_t - V_{t-1}| segment experience correctly. Section 3.1 defines F_phi, but Section 3.2 gives no loss function or training target: 'trajectory outcomes' are invoked, yet the paper never states how episode-level success is converted into per-timestep value supervision, nor how the memory read/write/injection modules are trained. Consequently, the reported gains in Table 1 could be produced by any history mechanism—the value signal might be recency, similarity, or novelty in disguise. No ablation replaces the value-guided writer with a fixed or heuristic writer, and no analysis shows V_t or its temporal differences peak at annotated grasp/contact/stage events. If value guidance is not the active ingredient, the paper's stated contribution collapses to retrieval-augmented context, which is not the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OnEvoMemory, a value-guided memory module that augments a frozen pretrained vision-language-action (VLA) policy for long-horizon robot manipulation. The memory consists of three banks: a short-term FIFO buffer, an elite experience bank, and a transition bank. Reading is performed by retrieving a union of top-k similar and recently written entries from the long-term banks, which are injected via gated cross-attention into the policy's action-query representations. Writing is guided by a learned value estimator F_phi that outputs a writing score V_t, a retrieval key k_t, and a stored representation v_t; high-value experiences go to the elite bank and large temporal value differences go to the transition bank. The module is first initialized by replaying offline demonstrations, then refined through one round of online rollouts using both successful and unsuccessful trajectories, while the base policy remains frozen. Experiments on LiberoLong-10 and two RMBench tasks report higher success rates with offline memory initialization and further gains after online evolution.","tokens_in":4687,"tokens_out":2517,"duration_ms":28524,"significance":"If the reported results are robust, the paper's central contribution is a plausible new mechanism for adapting memory selection to a frozen VLA policy without modifying the policy itself. The architecture is clearly described at a high level, and the decision to keep the policy frozen during online adaptation is a practically valuable design choice that addresses a real deployment constraint. The paper also deserves credit for explicitly acknowledging the moderation of its gains and the limited absolute success rates on RMBench, which is more transparent than is typical. However, the significance is currently limited because the experimental evidence is thin, the value-guided writing mechanism is not fully specified, and no comparison is made against the memory baselines cited in the related work. The claim that value guidance rather than generic retrieval is the active ingredient is not yet established.","major_comments":[{"comment":"The training procedure for the value estimator and memory modules is not specified. The text says that 'action supervision and trajectory outcomes jointly initialize the memory prior and train the memory reading, writing, and injection modules,' but no loss function, no per-timestep value target, and no optimization objective are given. Since the entire writing mechanism depends on V_t aligning with grasp success, contact loss, and task-stage transitions, the absence of a concrete training objective makes the method non-reproducible and leaves open the possibility that V_t encodes recency or similarity rather than task-relevant value.","section":"Section 3.2"},{"comment":"The experimental evidence consists of a single table with no variance, no number of seeds, no number of evaluation episodes, and no statistical test. The LiberoLong-10 gains are 2.4 points offline and 1.6 points online, which could easily be within rollout noise. The RMBench numbers are based on only 10 online adaptation trajectories and 0% baseline success; without confidence intervals or multiple seeds, the claimed improvements are not convincingly supported.","section":"Table 1"},{"comment":"The online evaluation protocol is underspecified and potentially circular. The paper states that 20 rollout trajectories per task on LiberoLong-10 and 10 per task on RMBench are collected for online adaptation, but it does not state whether the reported '+ Online' success rates are measured on fresh evaluation episodes or on the same rollouts used for adaptation. If evaluation is performed on the adaptation rollouts, the online improvement would be partly built into the reported numbers. This must be clarified and, if necessary, corrected with fresh-episode evaluation.","section":"Section 4"},{"comment":"The paper criticizes existing memory mechanisms such as BPP, MemER, MemoryWAM, EventVLA, KEMO, and WeaveLA, but it never compares against any of them. Without at least one related baseline evaluated under the same base policy and evaluation protocol, the paper's claim that OnEvoMemory improves over 'existing memory mechanisms' is unsupported. A minimum requirement is to run a representative baseline, such as a fixed similarity-based retrieval or a manually defined keyframe selector, under identical conditions.","section":"Section 2 and Section 4"},{"comment":"There is no ablation that isolates the value-guided writing mechanism. A natural control would replace the learned value score V_t with a random score, a recency score, or a fixed heuristic (e.g., writing every k-th timestep), while keeping the memory banks and retrieval unchanged. Without such an ablation, the reported gains could be produced by the retrieval-augmented context alone, which would reduce the central contribution to a retrieval system rather than value-guided memory evolution.","section":"Section 3.1"}],"minor_comments":[{"comment":"The notation (V_t, k_t, v_t) = F_phi(Qtilde_t, A_t) is introduced but the roles of k_t and v_t are not fully elaborated; in particular, it is unclear whether k_t is derived from Qtilde_t, from A_t, or from both, and whether v_t is a learned projection of either. The paper should state the dimensionalities and the exact inputs to the retrieval key and stored representation.","section":"Section 3.1"},{"comment":"The phrase 'trajectory outcomes' is used repeatedly but never defined operationally. It should be stated explicitly whether an outcome is a binary episode success, a sparse reward, a dense reward, or a combination, and how this outcome is assigned to individual timesteps for updating F_phi.","section":"Section 3.2"},{"comment":"The paper does not report how many evaluation episodes were run per task for each setting, nor the variance across tasks on LiberoLong-10 beyond the average. Reporting per-task success rates with confidence intervals would substantially strengthen the empirical claims.","section":"Section 4"},{"comment":"The sentence 'The additional gains from online adaptation may be attributed to the unsuccessful rollouts collected during interaction' is speculative; the paper provides no analysis separating the contribution of successful versus unsuccessful rollouts. A small experiment using only successful rollouts, or only unsuccessful rollouts, would make this attribution concrete.","section":"Section 4"},{"comment":"There are minor formatting issues, including the inconsistent use of 'OnEvoMemory' with and without a space in the abstract and introduction, and the reference to 'QwenOFT' cites two papers [3,11] that are about fine-tuning VLAs and a codebase, respectively, rather than a model named QwenOFT; the authors should cite the actual base model or clarify its provenance.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is in scope for a robotics venue and addresses a worthwhile problem, but the current manuscript is closer to a workshop-level extended abstract. The central mechanism is underspecified, and the experiments are too thin to support the headline claim. The authors should be encouraged to resubmit after adding a concrete training formulation, a proper evaluation protocol with fresh episodes and variance, at least one baseline comparison, and an ablation of the value-guided writer. I do not see evidence of deliberate misrepresentation, but the circularity risk in the online evaluation must be resolved explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the thing you need to know: OnEvoMemory is a plausible memory module for frozen VLA policies, but the paper leaves the value-guided part underspecified and the evaluation is too thin to carry the claim.\n\nWhat's genuinely new is the combination of a hierarchical memory (short-term buffer, elite bank, transition bank) with a learned value estimator that decides what to write, plus offline initialization from demonstrations and online refinement from rollouts. Keeping the base policy frozen is a real practical advantage. The writing is clear, the motivation (tracking completed subtasks, grasping and contact events) makes sense, and the authors are honest that the gains are moderate and RMBench rates are still low.\n\nThe soft spots are substantial, though. The biggest is that Section 3.1 defines the value estimator F_phi, but Section 3.2 never gives its loss or training signal. Saying 'action supervision and trajectory outcomes jointly initialize' isn't enough. Without knowing how episode success becomes per-timestep value targets, the reported gains could come from any retrieval mechanism. The natural ablation—swap the value-guided writer with a fixed or heuristic writer—is missing. No baseline comparison with MemoryWAM, KEMO, HAMLET, EventVLA, or similar memory policies, even though they're cited. The stress-test note is right on this.\n\nThe numbers are also reported loosely. Table 1 has no variance, no seed count, and no statistical tests. On RMBench you collect 10 trajectories per task, but report 14% success; that's not an integer unless you averaged over multiple seeds, which isn't stated. Small absolute gains (0 to 14% on SwapBlocks) could be one or two lucky rollouts. I'd want to see error bars and per-seed counts.\n\nI don't think the central idea is wrong; it's just not yet supported. The paper reads like a preliminary preprint that needs a full version with ablations, baselines, and proper experimental detail. Who is this for? People working on memory-augmented VLA policies will want to know about it, and the frozen-policy setup is relevant to anyone deploying pretrained policies in long-horizon tasks. I'd send it to peer review, but with the expectation of major revision. A serious reviewer could quickly turn this into a solid paper by demanding the missing loss function and a couple of ablations.","headline":"Plausible memory module for frozen VLA policies, but the unspecified value training and thin, seed-free evaluation leave the central claim unproven.","tokens_in":5211,"tokens_out":2692,"would_cite":false,"duration_ms":27907,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A value-guided memory module, seeded from demonstrations and refined by successful and failed robot rollouts, claims to make a frozen pretrained policy better at long-horizon manipulation.","keywords":["long-horizon manipulation","vision-language-action models","value-guided memory","online memory evolution","hierarchical memory","robot rollouts","frozen policy adaptation","offline demonstration initialization"],"falsifier":"Run the same experiments with the value-guided writer ablated: fill the elite and transition banks with random or uniformly sampled experiences while keeping retrieval and gated cross-attention identical. If the reported success-rate gains on LiberoLong-10 and RMBench largely persist, then the value estimator's scoring is not the cause and the central claim about learned memory selection collapses.","tokens_in":4289,"feed_emoji":"🤖","tokens_out":9261,"duration_ms":87317,"temperature":0.7,"pith_summary":"OnEvoMemory claims that a pretrained vision-language-action robot policy can become better at long-horizon manipulation without any modification to the policy itself, by adding a memory module that learns what to remember. The module keeps a short-term raw buffer, an elite bank of high-value experiences, and a transition bank of moments where trajectory value changes sharply; a learned value estimator decides what goes into each bank. Offline demonstrations initialize the writer, and successful plus failed online rollouts refine it. On the paper's benchmarks, offline memory initialization raises average success on LiberoLong-10 from 86.2% to 88.6%, and one round of online memory evolution raises it further to 90.2%, while two RMBench tasks move from 0% success to 14% and 10%. If true, the point matters because memory selection is usually hand-designed or outsourced to external models, whereas this work tries to learn it from interaction outcomes while keeping the policy frozen.","feed_headline":"Learned memory lifts frozen robot policy from 86% to 90%","feed_subtitle":"Demonstrations seed the memory, failed rollouts refine it, and the pretrained action policy never changes.","key_machinery":"The central mechanism is the action-conditioned value estimator $(V_t, k_t, v_t) = F_\\phi(\\tilde{Q}_t, A_t)$, operating on action-query representations $Q_t = E_\\theta(o_t)$ from the frozen VLA. Its writing score $V_t$ places high-value experiences into an elite bank and large temporal value changes $|V_t - V_{t-1}|$ into a transition bank, while a fixed-capacity FIFO short-term buffer preserves recent context; retrieval returns top-k similar plus recently written items from each bank, and gated cross-attention fuses that context into the frozen action decoder. This single learned writer replaces fixed temporal windows, hand-written event rules, and external models for deciding what history is worth keeping.","core_discovery":"The central discovery is that memory selection for long-horizon manipulation can be learned from trajectory outcomes and injected into a frozen policy. In the paper's architecture, action-query representations $Q_t = E_\\theta(o_t)$ from the frozen VLA are read against three memory banks through gated cross-attention; a value estimator $F_\\phi$ scores each experience and routes high-value items to an elite bank and large temporal value changes to a transition bank, while retrieval combines top-k similarity with recently written items. The paper reports that this structure improves the base policy on LiberoLong-10 from 86.2% to 88.6% with offline-initialized memory and to 90.2% after online evolution, and that it gives the first successes on RMBench's SwapBlocks and SwapT, moving both from 0% to 14% and 10%. The authors interpret these results as evidence that a value-guided writer can recognize task-stage transitions and keep the policy from repeating completed subtasks without changing the pretrained action decoder.","pith_inferences":["The design implies a testable decomposition: replace the value-guided writer with random or heuristic writing while keeping the banks and retrieval fixed; if the gains persist, the value estimator is not the active ingredient.","Because only the memory modules are updated, OnEvoMemory could in principle be attached to other token-based VLA backbones; the paper's numbers alone do not show whether the learned value signal transfers across backbones.","The one-round online protocol leaves open whether additional rollout rounds continue to improve memory or saturate; a multi-round experiment would map the evolution curve.","Since the transition bank records jumps in value, the same writer could be re-purposed to auto-label demonstration keyframes, turning OnEvoMemory into a data-curation tool rather than only a policy add-on."],"forward_implications":["Attaching OnEvoMemory to a frozen VLA can improve long-horizon success without retraining the policy, making memory adaptation a separable axis from policy adaptation.","Both successful and unsuccessful rollouts carry usable supervision for memory selection; failure trajectories are not discarded but are used to revise overvalued experiences.","Large temporal changes in the learned value score can mark task-stage boundaries, grasp success, and contact loss, giving the policy a way to track progress without external event detection.","One round of online memory evolution with ten to twenty trajectories per task already moves success rates upward, suggesting memory quality can be improved cheaply relative to policy fine-tuning."],"supporting_citations":[{"why":"Supplies the base QwenOFT vision-language-action policy that stays frozen while only memory modules update.","marker":"[3,11]"},{"why":"Defines LiberoLong-10, the ten-task long-horizon benchmark whose average success rates are the paper's main quantitative result.","marker":"[6]"},{"why":"Defines the two RMBench tasks, SwapBlocks and SwapT, where the base policy scores 0% and memory produces the first nonzero successes.","marker":"[1]"}],"fun_headline_variants":["Evolving memory writes success into frozen policy","Online rollouts teach robot memory, raising score","Frozen VLA policy sees gains via learned memory","Value-guided memory bank boosts robot performance","Robot memory evolves, frozen policy improves"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method works only if the frozen policy's action-query vectors carry enough information about task state and outcome quality that the learned value estimator can tell which past moments are worth keeping and when the task stage changed.","fun_headline_variants_meta":{"raw":{"variants":["Evolving memory writes success into frozen policy","Online rollouts teach robot memory, raising score","Frozen VLA policy sees gains via learned memory","Value-guided memory bank boosts robot performance","Robot memory evolves, frozen policy improves"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1264,"prompt_tokens":877,"completion_tokens":387,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":320}},"tokens_in":493,"tokens_out":387,"duration_ms":4734,"temperature":1.0,"reasoning_tokens":320,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:25:55.810644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same experiments with the value-guided writer ablated: fill the elite and transition banks with random or uniformly sampled experiences while keeping retrieval and gated cross-attention identical. If the reported success-rate gains on LiberoLong-10 and RMBench largely persist, then the value estimator's scoring is not the cause and the central claim about learned memory selection collapses.","supporting_citations":[{"cited_title":"In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S","cited_arxiv_id":null,"evidence_quote":"Defines LiberoLong-10, the ten-task long-horizon benchmark whose average success rates are the paper's main quantitative result."}],"review_version":1}