{"id":"d414d1f5-e320-4532-a3eb-952962375bb3","arxiv_id":"2505.08189","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A dual-system framework where an RL agent handles familiar subtasks and a VLM planner/executor handles unfamiliar ones improves task success and speed in Crafter and Housekeep.","lead":"This paper combines a fast reinforcement learning agent with a slow vision-language model that plans and executes unfamiliar subtasks, calling the combination a dual-system framework. It reports higher success rates and faster completion times than baselines in the Crafter and Housekeep game environments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VLM-judged proficiency scores in the Memory Space (Sec. 4.1) are unvalidated; if p_v does not track true RL competence, the routing that drives DSADF's claimed generalization gains has no support.","rationale":"The reader's weakest assumption correctly identifies the unvalidated proficiency scoring in Sec. 4.1 as the most load-bearing point. The central claim is that DSADF significantly outperforms other RL agents on unseen tasks, and the mechanism proposed for this is the dual-system routing driven by p_v. If p_v is not a faithful estimate of the RL agent's competence, then the routing rule in Eq. (5) is arbitrary, and the claimed benefit of the adaptive coordination is not established. Notably, the paper's own component ablations (Variations I-IV) do not isolate the effect of the proficiency values; they remove entire modules, so they cannot show that the specific scoring mechanism contributes. A secondary issue is that the OOD comparisons in Tables 4-5 are against VLM agents rather than the RL baselines used in-domain, which further complicates the claim of superiority over RL agents; however, the proficiency concern is more fundamental because it threatens the method's scientific contribution even if the empirical numbers are reproduced. Since the framework otherwise appears coherent and the empirical results, while lacking statistical detail, are suggestive, a conditional acceptance requiring validation of the proficiency scoring is appropriate. The proposed correlation and oracle-router test would directly settle whether the memory mechanism is sound.","tokens_in":27214,"tokens_out":9797,"duration_ms":100949,"concrete_test":"Freeze the trained RL agent and measure its actual per-subtask success rate on each u_v in M over many episodes. Independently obtain p_v = VLM(u_v, h_v) using the paper's protocol, then compute the rank correlation (e.g., Spearman) between predicted and true proficiency. Next, compare full DSADF against an oracle router that assigns tasks using the measured true success rates with the same threshold T, while keeping all other components identical. If p_v is uncorrelated with true proficiency, or if oracle routing does not outperform VLM-judged routing, the memory-based allocation mechanism is not validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The system's adaptive routing depends on proficiency values p_v = VLM(u_v, h_v) computed in Sec. 4.1 (Memory Space). These values are produced by asking a vision-language model to judge the RL agent's competence on each rewarded action, yet the paper provides no calibration against measured task success, no definition of the evaluation prompt or output scale, and no sensitivity analysis for the threshold T used in Eq. (5). If p_v is arbitrary, the partition into G_RL and G_VLM misallocates tasks, and the central claim that dual-system coordination (rather than simply adding a VLM) drives the reported OOD gains is unsupported. The ablations show each main component matters, but they never test whether the proficiency-based routing itself is sound; the same OOD performance could arise from a VLM executing most subtasks while the memory mechanism contributes little or even harms efficiency. This is load-bearing because the framework's novelty rests on adaptive task allocation, and no evidence demonstrates that a VLM can reliably assess RL proficiency from the provided history.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DSADF (Dual-System Adaptive Decision Framework), which combines a goal-conditioned RL agent (System 1) with a vision-language model (System 2) for decision-making in long-horizon tasks. System 2 decomposes high-level goals into short-horizon subtasks, maintains a memory space of proficiency scores p_v = VLM(u_v, h_v), routes each subtask to either the RL agent or the VLM auxiliary performer based on whether p_v exceeds a threshold T, and generates progressive rewards for RL training. The framework is evaluated in the Crafter and Housekeep environments, reporting improvements in task success rate, completion time, completion rate, survival rate, and average object success rate on both in-domain and out-of-distribution tasks, compared to several RL baselines and VLM-as-agent baselines. Ablations isolate the contributions of the planner, auxiliary performer, RL agent, and self-reflection component.","tokens_in":27457,"tokens_out":4841,"duration_ms":45964,"significance":"If the reported results are reliable, DSADF offers a concrete architecture for adaptive coordination between RL agents and foundation models, operating on a plausible interpretation of Kahneman's dual-process theory. The paper has clear strengths: the framework is described in detail, ablations cover the main components, and evaluation spans two environments with in-domain and OOD tasks. The significance is, however, tempered by two unresolved issues: the proficiency-based routing mechanism is not validated, and the empirical claims rely on point estimates without statistical support. These weaknesses currently prevent the paper from establishing its central claim that adaptive task allocation, rather than simply adding a VLM, drives the observed generalization gains. The contribution is nonetheless sufficiently interesting and potentially useful to warrant a major revision.","major_comments":[{"comment":"The proficiency value p_v = VLM(u_v, h_v) is the linchpin of the routing mechanism, but the paper provides no evidence that this value tracks the RL agent's true competence. There is no definition of the evaluation prompt, no description of the output scale, and no calibration against measured success rates on held-out short-horizon tasks. Because the same VLM both scores proficiency and serves as auxiliary performer, the reported OOD gains could arise from the VLM executing most subtasks rather than from the dual-system allocation. I request a calibration study (e.g., correlation between p_v and the RL agent's rollout success rate on the corresponding subtask) and a sensitivity analysis of the threshold T in Eq. (5). Without these, the central mechanism of the framework is unsupported.","section":"§4.1, Eq. (5)"},{"comment":"All performance tables and figures report point estimates of TSR, execution time, or AOSR without confidence intervals, standard errors, or significance tests, despite the text repeatedly claiming 'significant' improvements. For example, Table 3 reports DSADF exceeding baselines by 3-17 percentage points on in-domain tasks, and Table 4 reports improvements of 0-68 percentage points on OOD tasks, but no measure of variability is given even though Figure 3 states the mean TSR is averaged over 60 independent runs. I request error bars or confidence intervals for the reported means and a statistical test (e.g., bootstrap or paired test across seeds/runs) for the key comparisons, especially DSADF versus the best baseline in Tables 3, 4, and 10.","section":"§5.3.1, §5.3.2, Tables 3-10"},{"comment":"The task numbering and categorization are inconsistent across the paper, making it difficult to verify which results correspond to which claims. Section 5.2 describes 14 tasks with specific categories, but the appendix lists only Tasks 1-13; Table 4 uses Tasks 5-7, Table 5 uses Tasks 8-10, Table 6 and Table 7 use overlapping labels (Table 7 refers to 'Tasks 4-6' as OOD while Task 4 is an in-domain task), and Table 10 uses Tasks 11-13. Additionally, in the appendix Task 9 is described as 'Craft iron sword' but its step list is 'Make wood pickaxe → find stone → mine stone → make stone sword.' Please align the numbering, descriptions, and table labels so that the empirical evaluation is unambiguous.","section":"§5.2, Appendix A.1.1, Tables 4-10"},{"comment":"The opening of Section 5.3.2 states that DSADF 'outperforms other RL agents in both generalization and efficiency on unseen tasks,' but the baselines in Table 4 are vision-language model agents (LLaVA-1.5, MiniGPT-4, Qwen-2.5-VL) and combinations with GPT-4o, not RL agents. The comparison to RL agents appears only on in-domain tasks in Table 3. I request OOD results for RL baselines (for example, the ELLM- or APT-pretrained sparse-reward agent) or a revised claim that restricts the OOD comparison to foundation-model agents.","section":"§5.3.2, Table 4"}],"minor_comments":[{"comment":"The sentence 'The knowledge conveyed by Otext includes the object coordinates from the goal list G and the corresponding action subspace of the object' is repeated verbatim; please remove the duplicate.","section":"§3.1"},{"comment":"The cosine similarity expression contains typos: 'cos(E_tran,E_g)' should be 'cos(E_trans,E_g)' and the closing parenthesis is missing in the condition 'cos( Etran,E g ) >β'.","section":"§4.3, Eq. (9)"},{"comment":"The objectives are labeled 'Object 1', 'Object 2', 'Object 3'; these should be 'Objective 1', etc., since 'Object' is misleading.","section":"§3.1, §5.3"},{"comment":"Table 3 contains typographical errors ('Reweard', '3375,4'), and Figure 3 reports means over 60 runs without error bars; adding shaded confidence regions would improve interpretability.","section":"Table 3, Figure 3"},{"comment":"The sentence 'Does each scenario consist of a single room...' should read 'Each scenario consists of a single room...'.","section":"§5.4"},{"comment":"The entry 'ed reflection instruction' is incomplete; the notation e_d used in Eqs. (4) and (6) should be defined clearly in the notational table.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is readable and the core idea is plausible, but the empirical evidence currently does not support the strong 'significant' claims due to missing statistical rigor and inconsistent task definitions. The proficiency-validation issue is the most serious scientific gap and should be addressed directly, ideally with a calibration experiment and threshold sensitivity analysis. I would recommend sending the paper back for major revision rather than rejecting it, as the framework has potential and the requested experiments appear feasible within the scope of the authors' setup."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"DSADF is a dual-system framework that routes subtasks between an RL agent and a VLM based on proficiency scores stored in a memory space. That routing idea is the actual novelty, and the paper shows large empirical gains over RL baselines and VLM-as-agent baselines in Crafter and Housekeep. The method is coherent, the progressive reward design is sensible, and the ablations demonstrate that each main component contributes. The comparison set is appropriate.\n\nThe soft spots are real. The proficiency values p_v = VLM(u_v, h_v) in Sec. 4.1 are produced by the same VLM that later executes tasks, and there is no calibration against measured success, no prompt details, no output scale, and no sensitivity analysis for the threshold T. If p_v does not track the RL agent's actual competence, the memory-based routing is arbitrary, and the paper's claim that adaptive task allocation drives the gains has no direct support. The ablations remove whole components but never test the routing itself; a fixed or random split might do just as well. That is a load-bearing gap for the central novelty.\n\nThe experimental reporting is also thinner than it should be. The text says 'significantly outperforms' but gives no variance or significance tests (the figures show mean TSR over 60 runs but no error bars). Task numbering conflicts across sections: Section 5.2 describes Tasks 5 and 8 as long OOD tasks, but Table 4 uses Tasks 5-7, Table 5 uses Tasks 8-10, and Table 7 labels OOD tasks as Tasks 4-6 with values that match Tasks 5-7 in Table 4. The survival-rate arrow in Table 4 points the wrong way. No code is provided. These are fixable but they matter.\n\nWho this is for: people working on RL plus foundation-model coordination will want to know this framework. It is not a breakthrough, but it is a serious candidate template for fast/slow hybrid policies. It deserves a serious referee. I would send it to review, with the expectation of major revision: validate p_v against task success, add error bars and significance tests, fix the task labels, and ideally release code. The core idea is worth engaging with.","headline":"A coherent RL+VLM dual-system framework with large but under-supported empirical claims; the unvalidated proficiency routing is the main soft spot.","tokens_in":27939,"tokens_out":3558,"would_cite":false,"duration_ms":32042,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-system framework lets a vision-language model plan and a fast reinforcement-learning agent execute, claiming stronger generalization on unseen tasks than either component alone.","keywords":["reinforcement learning","vision language model","System 1 and System 2","decision-making","generalization","long-horizon tasks","task allocation","progressive reward"],"falsifier":"Shuffle or randomize the proficiency values stored in memory while keeping everything else fixed, then run the same unseen-task evaluations; if success rates stay at the reported levels, the memory-routed allocation is not the source of the gain and the claimed mechanism fails the test.","tokens_in":27047,"feed_emoji":"🧠","tokens_out":5383,"duration_ms":49487,"temperature":0.7,"pith_summary":"The paper argues that a single reinforcement-learning agent cannot efficiently generalize to long-horizon or unseen tasks, so it builds a two-part decision system: a fast RL agent handles familiar short subtasks, and a slow vision-language model plans, decomposes goals, and steps in on unfamiliar subtasks. The central move is to store per-subtask proficiency scores in a memory space and use them to route each subtask to whichever component is better suited. The paper reports that this dual-system framework, called DSADF, outperforms RL baselines and VLM-only agents on both known and unseen tasks in the Crafter and Housekeep environments, while completing tasks faster.","feed_headline":"Fast RL plus slow VLM beats both on unseen game tasks","feed_subtitle":"Paper claims a memory-routed dual system lifts success on unseen tasks and cuts completion time.","key_machinery":"The load-bearing mechanism is the Memory Space M, a list of short-horizon task subspaces storing pairs of actions and proficiency values, where each proficiency is produced by the vision-language model from the action and its history. A threshold T decides assignment: subtasks whose proficiency meets or exceeds T go to the RL agent, and the rest go to the VLM. Around this routing sits the planner's chain-of-thought decomposition with a self-reflection step, and a three-part progressive reward (initial-target reward, sub-goal reward, and proximity reward) that turns a sparse long-horizon reward into dense single-step signals; the memory is updated after training with new proficiency scores.","core_discovery":"On the paper's own terms, the discovery is that the bottleneck to generalization in goal-conditioned RL is not the RL policy alone but the absence of a mechanism for deciding when the policy can be trusted. DSADF solves this by letting a VLM decompose a long-horizon goal into a list of short-horizon subtasks, scoring the RL agent's past rewarded actions for proficiency, and routing each subtask to the RL agent only when the score meets a threshold; otherwise the VLM executes it as an auxiliary performer. The paper claims this routing, together with a progressive reward made of sparse, sub-goal, and proximity rewards, lets the RL agent learn single-step skills quickly and still complete multi-step unseen tasks, with measured success rates above those of RL baselines and VLM agents.","pith_inferences":["Editorial: the memory scores are produced by the VLM from descriptions of past successes, so the framework's benefit is only as good as that scoring; a cheaper proxy for proficiency, such as measured success statistics, could replace it and would be a natural test of whether VLM scoring is essential.","Editorial: if the threshold T were tuned per task, the system could shift work adaptively as the RL agent improves; the paper treats T as a fixed hyperparameter, leaving this online adaptation implicit.","Editorial: the dual-system layout may transfer to continual learning settings, where each new task family first goes through System 2 and is gradually absorbed into System 1's memory as proficiency rises."],"forward_implications":["If DSADF generalizes as claimed, RL agents can be trained on a few short tasks and still complete longer, unseen tasks by handing unfamiliar steps to a VLM.","Task success on known tasks would improve because the RL agent trains on dense, sub-goal-level rewards instead of sparse final rewards, shortening training time.","Deployment cost becomes controllable: the VLM only acts on subtasks the memory says are not proficient, so most steps run on the fast RL policy.","The memory space turns every deployment into a chance to update proficiency scores, so the system can adapt to new environments by reassigning subtasks.","The same routing idea could apply beyond video games to embodied or household tasks where reliable low-level control and high-level common-sense reasoning must coexist."],"supporting_citations":[{"why":"Provides the ELLM baseline and the goal-conditioned RL reward design that DSADF builds on.","marker":"[17]"},{"why":"Supplies the fast-and-slow dual-process theory that motivates the two-system architecture.","marker":"[22]"},{"why":"Supplies the chain-of-thought prompting used by the planner to decompose long-horizon goals.","marker":"[23]"},{"why":"Provides the large vision-language model (GPT-4o) used as System 2 in the main experiments.","marker":"[41]"},{"why":"Supplies the CLIP-based module that converts visual observations into textual descriptions.","marker":"[61]"},{"why":"Supplies the SentenceBERT text encoder used in the proximity reward to compare goal and transition embeddings.","marker":"[69]"},{"why":"Defines the Crafter environment used for in-domain and out-of-domain generalization tests.","marker":"[70]"},{"why":"Defines the Housekeep environment used to test environmental generalization without explicit targets.","marker":"[71]"},{"why":"Provides the Qwen-2.5-VL model used as the auxiliary performer in the DSADF framework.","marker":"[78]"},{"why":"Provides the LINVIT baseline that combines LLMs with RL and is compared against in the experiments.","marker":"[36]"}],"fun_headline_variants":["Dual-system RL-VLM router lifts unseen-task success","Fast RL, slow VLM: A routing win on unseen games","Memory-routed dual system speeds adapt to new tasks","When RL hesitates, VLM steps in: DSADF aces new goals","Balancing intuition and reasoning improves RL generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole design assumes that the proficiency scores a vision-language model assigns to each small task truly reflect how well the reinforcement-learning agent can do that task, because these scores alone decide which side handles what.","fun_headline_variants_meta":{"raw":{"variants":["Dual-system RL-VLM router lifts unseen-task success","Fast RL, slow VLM: A routing win on unseen games","Memory-routed dual system speeds adapt to new tasks","When RL hesitates, VLM steps in: DSADF aces new goals","Balancing intuition and reasoning improves RL generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000524,"raw_usage":{"total_tokens":2539,"prompt_tokens":962,"completion_tokens":1577,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":1492}},"tokens_in":578,"tokens_out":1577,"duration_ms":11001,"temperature":1.0,"reasoning_tokens":1492,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:00:23.955612+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Shuffle or randomize the proficiency values stored in memory while keeping everything else fixed, then run the same unseen-task evaluations; if success rates stay at the reported levels, the memory-routed allocation is not the source of the gain and the claimed mechanism fails the test.","supporting_citations":[{"cited_title":"In: International Conference on Machine Learning, pp","cited_arxiv_id":null,"evidence_quote":"Provides the ELLM baseline and the goal-conditioned RL reward design that DSADF builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the fast-and-slow dual-process theory that motivates the two-system architecture."},{"cited_title":"In: ICML, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP-based module that converts visual observations into textual descriptions."},{"cited_title":"In: Empirical Methods in Natural Language Processing (2019)","cited_arxiv_id":null,"evidence_quote":"Supplies the SentenceBERT text encoder used in the proximity reward to compare goal and transition embeddings."},{"cited_title":"ICLR (2022)","cited_arxiv_id":null,"evidence_quote":"Defines the Crafter environment used for in-domain and out-of-domain generalization tests."},{"cited_title":"In: European Conference on Computer Vision, pp","cited_arxiv_id":null,"evidence_quote":"Defines the Housekeep environment used to test environmental generalization without explicit targets."}],"review_version":1}