{"id":"eade87b6-c8a8-494c-ae40-315b3b6a8206","arxiv_id":"2607.09493","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Selectively persisting task specs, data schemas, tool configs, and output constraints (discarding reasoning traces) yields 96% agentic task completion versus 79% with no memory and 71% with full history.","lead":"Agentic LLM systems usually start every session from scratch, forcing users to re-specify rules, schemas, and tools. This paper shows that selectively keeping four kinds of reusable configuration—and sharing those workspaces—raises task completion from 79% to 96% while cutting tokens and time.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The 96% completion claim rests on a proprietary, non-reproducible enterprise evaluation whose single selective-memory failure already exposes the manual four-category schema summary as incomplete.","rationale":"The Reader correctly flags the manual four-category decomposition and the single join failure as the weakest assumption, and correctly notes that enterprise data/code unavailability plus single-model testing keep confidence only MODERATE. My concern is the same load-bearing point, sharpened: the 96% figure itself is not re-runnable, while the public 12/12 result only validates the zero-token path under perfect schema compatibility. That does not overturn the architecture’s practical value or the interesting full-history degradation finding, so the verdict stays CONDITIONAL rather than REJECT; it simply makes the contingency more precise. No mathematical circularity exists; the risk is empirical non-reproducibility of the headline numbers.","tokens_in":13008,"tokens_out":565,"duration_ms":5629,"concrete_test":"Release the 24 enterprise task definitions (or a sanitized schema-compatible subset) plus the exact Mtask/Mdata/Mtools/Moutput snapshots used for the selective condition; re-run the three-way ablation with an open model (e.g., Llama-3.1-70B or GPT-4o) under the same four-criterion automated checks used in the public replication. If selective completion falls below ~85% or the ordering collapses, the 96% claim does not generalize.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central ordering (selective 96% > no-memory 79% > full-history 71%) and the 14\times zero-token claim are driven by the enterprise ablation (Table 1, n=24 recurring tasks, §6.2). That evaluation uses 24 proprietary files, Claude Opus 4 only, and a four-criterion pass/fail rubric scored by two blinded raters (κ=0.91). The public replication (Table 2) shows 100% only under zero-token refresh on schema-compatible V2 data; it does not re-run the multi-turn generation ablation that produces the 96%/79%/71% numbers. The paper itself records the single selective-memory failure as a cross-file join missed by the Mdata summary (§6.2) and states in Limitations that the four-category split is manually designed and limited to stable tabular schemas. Thus the headline percentages are not independently re-runnable, and the architecture’s sufficiency is already contradicted by its own failure mode. Without public enterprise artifacts or multi-model re-runs, the strongest claim remains contingent on an unreproducible proprietary corpus.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes shared selective persistent memory for agentic LLM systems that generate code via multi-turn tool use. It retains four structured categories of reusable context—task specifications (Mtask), data schemas (Mdata), tool configurations (Mtools), and output constraints (Moutput)—while discarding session-specific reasoning traces, tool logs, and intermediate files. Memory is packaged in shareable workspaces with role-based access control. A complementary data-injection contract enables zero-token refresh of generated artifacts when new data is schema-compatible. The architecture is implemented in a deployed collaborative platform with multi-connector data access (CSV, SQL, REST, MCP), git-versioned artifacts, and draft isolation. Across 24 enterprise recurring-generation tasks, selective memory reports 96% completion versus 79% with no memory and 71% with full history (Table 1); zero-token refresh yields large task-time reductions; a public-dataset replication reports 12/12 zero-token successes (Table 2).","tokens_in":13315,"tokens_out":1387,"duration_ms":25937,"significance":"If the central ordering and efficiency claims hold, the work is a useful systems contribution: it treats context management—not model scale—as the primary lever for recurring enterprise agentic workflows, and it documents that naive full-history persistence can actively harm completion (consistent with lost-in-the-middle). Strengths include a clear four-category decomposition with explicit selective forgetting, a practical zero-token data-injection contract, a deployed multi-connector platform with git versioning, a blinded dual-rater protocol (κ=0.91) with Fisher’s exact tests, failure-mode analysis (format non-compliance vs. trace anchoring), and a public-dataset check of zero-token refresh. These are concrete, falsifiable systems results rather than purely conceptual claims.","major_comments":[{"comment":"§6.2 and Table 1: The headline result (selective 96% > no-memory 79% > full-history 71%, n=24) is the load-bearing empirical claim, but it rests entirely on proprietary enterprise files, a single model (Claude Opus 4), and a four-criterion pass/fail rubric. The public replication (Table 2, §6.3) validates zero-token refresh under schema-compatible V2 data (12/12, 0 LLM tokens) and shows full history again underperforming no-memory, yet it does not re-run multi-turn initial/recurring generation under the three memory conditions with the same task families and human rubric. Independent verification of the generation-quality ordering therefore remains limited. A public multi-turn ablation (even on the four public datasets with fixed task specs) or multi-model re-runs would substantially strengthen the central claim.","section":null},{"comment":"§3.2–3.5 and Limitations: The architecture’s sufficiency axiom—that reusable agentic context decomposes into exactly the four manual categories, with a simple column-subset compatibility check for zero-token refresh—is already strained by the paper’s own evidence. The single selective-memory enterprise failure is a cross-file join missed by the Mdata summary (§6.2); Limitations state the decomposition is manually designed and that zero-token refresh is limited to structured tabular data with stable schemas. Without an ablation of which categories drive the gains (e.g., Mtask-only vs. Mdata-only vs. full selective) or a clearer failure taxonomy (joins, schema drift, multi-source semantics), the claim that selective memory is the right intermediate between no memory and full history remains only partially tested.","section":null},{"comment":"§6.1–6.2 protocol: Completion is defined as dual-rater pass on render correctness, data fidelity, format compliance, and completeness. That is appropriate for a systems paper, but the enterprise tasks are drawn from three use-case families with uneven n (UC1=10, UC2=8, UC3=6), and UC3 shows equal completion (83%) with only turn reduction. The paper should report per-use-case confidence intervals or exact counts, and clarify how many of the 24 selective successes were pure zero-token re-renders (stated as 18/24 schema-compatible) versus actual LLM re-generation under selective memory, so readers can separate architectural reuse from generation-quality gains.","section":null}],"minor_comments":[{"comment":"§6.5 / Limitations: The user study (N=12) is correctly flagged as underpowered for item-level Likert inference; move any strong usability claims to exploratory language and report effect sizes or raw distributions if space allows.","section":null},{"comment":"Figure 1 and Eq. (1)–(3): The composition operators ⊕ and workspace tuple W are clear, but a short note on token budgets for Mdata (~500 tokens claimed) versus actual measured prompt sizes in Table 1 (3.4K input) would help readers reconcile summary size with full composed prompts.","section":null},{"comment":"§2 Related Work: Voyager, MemGPT, and Assistants API are appropriately positioned; a one-sentence contrast with recent agent memory work that persists skills or trajectories (beyond the cited set) would sharpen novelty without lengthening the section much.","section":null},{"comment":"Table 3 / Figure 2: Token reductions (97× enterprise, 946× public) are striking; state the tokenizer used for enterprise counts as well (public uses cl100k_base) and whether tool-call overhead is included.","section":null},{"comment":"Appendix A: The composed-prompt example is helpful; ensure it is referenced from §3.4 so readers find it.","section":null},{"comment":"Minor consistency: abstract and §6 report 14× task-time reduction and 97× token reduction; keep units (wall-clock vs. tokens; enterprise vs. public) aligned in every restatement.","section":null}],"recommendation":"major_revision","confidential_remarks":"The main scientific risk is not conceptual novelty of selective memory but over-reliance on a non-reproducible enterprise corpus for the 96/79/71 ordering. For a serious venue this is fixable with a public multi-turn ablation and/or category ablations; without that, the paper reads more as a strong systems/demo report than a fully verifiable empirical claim. Scope fit is good for applied AI / agent systems tracks."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The punchline is empirical and practical: on their recurring artifact-generation tasks, keeping only four structured buckets (task specs, data schemas, tool configs, output constraints) and throwing away reasoning/tool traces gets 96% completion versus 79% with no memory and 71% with full history. Full history is worse than starting over. That matches lost-in-the-middle and is the result worth remembering. The zero-token refresh (data-injection contract so programs never hardcode data) is the other clean piece; it succeeds 12/12 on public schema-compatible updates and drives the 14× time claim when no model call is needed.\n\nWhat is actually new is the packaged architecture, not any single idea. They decompose reusable context, discard traces on purpose, wrap it in shareable workspaces with RBAC, and enforce a runtime injection contract so artifacts re-render without re-invocation. The platform is real: git-versioned drafts, multi-connector (CSV/SQL/REST/MCP), summary-driven generation (~0.5K tokens vs raw). Ablation design is careful—blinded dual raters κ=0.91, Fisher p-values, failure modes named (format non-compliance for no-memory, trace anchoring for full history, one cross-file join miss for selective). Public replication and the small user study support the efficiency numbers (97× token cut when the model is called).\n\nSoft spots are real and the paper mostly owns them. The 96%/79%/71% numbers sit on 24 proprietary enterprise files and Claude Opus 4 only; outsiders cannot re-run the central multi-turn ablation. Public data validates zero-token refresh, not that full ordering. The four-category split is manual; their own single selective failure already shows the schema summary can miss join semantics. N is small, no multi-model, no error bars. Stress-test concern is fair on reproducibility of the headline percentages, but it does not make the work incoherent—the direction of the result is consistent and the design is falsifiable.\n\nThis is for people building multi-session agentic tooling for dashboards, reports, and collaborative workspaces. It is solid systems practice with an honest limitations section, not a foundational theory paper. I would send it to peer review; a serious editor should not desk-reject it. Worth a systems reading-group slot for the selective-vs-full finding and the zero-token pattern.","headline":"Selective memory beats both cold start and full history on agentic artifact tasks, and full history actively hurts—the useful finding is the ordering, not a new science of memory.","tokens_in":13951,"tokens_out":603,"would_cite":true,"duration_ms":16212,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Keeping only four reusable context types—and throwing away session traces—raises agentic LLM task completion to 96 percent, while saving full history actually hurts.","keywords":["agentic LLM systems","selective persistent memory","shared workspaces","zero-token data refresh","data schemas","tool configurations","context management","token efficiency"],"falsifier":"Run the same 24 recurring enterprise tasks (and the four public-dataset replications) under three conditions—no memory, full history, selective memory—and measure blinded pass/fail completion; if selective memory no longer outperforms both baselines, or if zero-token refresh fails on schema-compatible data in more than a negligible fraction of trials, the central claim fails.","tokens_in":13855,"feed_emoji":"💾","tokens_out":930,"duration_ms":13920,"temperature":0.7,"pith_summary":"Agentic LLM systems that write code and use tools currently start every session from scratch, forcing users to restate domain rules, data shapes, tool setups, and output contracts. This paper argues that the useful knowledge from a session can be reduced to four compact categories (task specifications, data schemas, tool configurations, and output constraints) while the multi-turn reasoning and tool logs should be discarded. When that selective memory is shared across users in versioned workspaces and paired with a contract that injects data only at runtime, recurring dashboards and reports can be refreshed with zero LLM calls. In enterprise and public-data tests the approach reaches 96 percent completion, cuts turns and tokens sharply, and outperforms both no memory and full-history baselines; full history actively degrades results by anchoring the agent to stale paths.","feed_headline":"Selective memory hits 96% where full history fails","feed_subtitle":"Four reusable context types beat both no memory and full chat logs for agentic code work","key_machinery":"Shared selective persistent memory: a four-category workspace store (M_task, M_data, M_tools, M_output) composed into the next session’s prompt, plus a runtime data-injection contract that lets generated programs re-render on new data without any model call.","core_discovery":"Shared selective persistent memory—retaining task specifications, data schemas, tool configurations, and output constraints while discarding session-specific reasoning traces—achieves 96 percent task completion on recurring agentic artifact-generation tasks versus 79 percent with no memory and 71 percent with full history; a complementary zero-token data-refresh contract then eliminates LLM re-invocation for schema-compatible updates.","pith_inferences":["The same four-category cut could be applied to agentic coding assistants and multi-agent orchestrators that currently keep full threads, potentially reducing both cost and path-dependence.","Automatic classifiers that label which spans of a session belong to the four reusable categories would remove the manual-design bottleneck the authors flag as a limitation.","If agent-level private memory (successful tool sequences, error heuristics) were added on top of workspace memory, multi-session planning quality might improve without re-introducing the stale-trace problem.","Schema-compatibility checking could be relaxed to support streaming or semi-structured sources by adding light structural embeddings rather than exact column subsets."],"forward_implications":["Recurring enterprise dashboards and reports can be refreshed with one click and zero LLM tokens whenever schemas stay compatible.","Workspaces become shareable templates: a colleague loads the selective memory, swaps in their own data, and the artifact renders immediately under role-based access.","Full conversation history should be treated as harmful default context for agentic code generation rather than a free continuity boost.","Token cost of data representation can drop two orders of magnitude by injecting statistical schemas instead of raw tables.","Git-backed draft isolation plus selective memory lets teams iterate risk-free and roll back without re-invoking the model."],"fun_headline_variants":["Selective memory hits 96% as full history drops to 71%","Shared selective memory: 96% vs 79% none and 71% full history","Four reusable contexts beat no-memory and full logs at 96%","Discard session traces; keep schemas to reach 96% completion","Selective persistent memory tops full history for agentic code"],"cache_read_input_tokens":128,"weakest_assumption_plain":"A hand-designed four-category split plus a simple “new columns may be added, none may be missing” schema check is enough to capture reusable context and keep zero-token refresh correct across artifact types and data sources.","fun_headline_variants_meta":{"raw":{"variants":["Selective memory hits 96% as full history drops to 71%","Shared selective memory: 96% vs 79% none and 71% full history","Four reusable contexts beat no-memory and full logs at 96%","Discard session traces; keep schemas to reach 96% completion","Selective persistent memory tops full history for agentic code"]},"model":"grok-4.5","effort":"low","cost_usd":0.00514,"raw_usage":{"total_tokens":1428,"prompt_tokens":857,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":51400000,"prompt_tokens_details":{"text_tokens":857,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":492,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":857,"tokens_out":79,"duration_ms":4122,"temperature":1.0,"reasoning_tokens":492,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T02:37:38.136160+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same 24 recurring enterprise tasks (and the four public-dataset replications) under three conditions—no memory, full history, selective memory—and measure blinded pass/fail completion; if selective memory no longer outperforms both baselines, or if zero-token refresh fails on schema-compatible data in more than a negligible fraction of trials, the central claim fails.","supporting_citations":[],"review_version":1}