{"id":"1d28e524-bab9-4558-b5a9-88aae6730d2f","arxiv_id":"2605.22794","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"MOSS performs source-level self-rewriting in agent systems using failure-anchored pipelines and container-based verification, raising OpenClaw mean score from 0.25 to 0.61 in one cycle.","lead":"MOSS lets autonomous agents rewrite their own source code when they encounter recurring failures. A generalist reader might care because this moves agent adaptation from fragile text prompts into deterministic code changes.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Verification replays only past failures; does not certify absence of new structural failures post source rewrite.","rationale":"The reader's weakest_assumption matches the load-bearing methodological gap exactly. Because the full manuscript was unavailable to the initial reader, the UNVERDICTED verdict already reflects inability to assess soundness; the verification concern identified above is the precise reason the empirical result cannot yet be accepted at face value. No stronger internal inconsistency or formal gap is visible from the supplied abstract.","tokens_in":1764,"tokens_out":382,"duration_ms":28705,"concrete_test":"After the reported cycle, execute the promoted container on a disjoint test suite of 20+ OpenClaw tasks (or extended interaction traces) never seen in the failure batch; measure grader scores. If mean score falls below the original 0.25 on any task or if new structural errors appear, the replay-only verification is shown to be insufficient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline empirical claim (0.25 → 0.61 on OpenClaw) rests on the assertion that a source-level edit, produced by an external coding agent and accepted after replay, is safe and effective. The described pipeline curates a batch of prior failures, runs the candidate image against that batch in ephemeral workers, and promotes on user consent. Because the batch is finite and drawn from past traces, any new failure mode introduced by the rewrite (altered hook ordering, changed state invariants, different dispatch paths, or interactions outside the recorded scenarios) remains untested. Source-level changes are Turing-complete and affect the harness itself, so the set of reachable behaviors expands; replaying only known failures therefore provides no coverage argument for the complement. The abstract supplies no additional checks (static analysis, property-based testing, or live shadow traffic) that would close this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents MOSS, a system enabling self-evolution in autonomous agent systems via source-level code rewriting rather than limiting changes to text-based artifacts like prompts or workflows. The approach involves curating batches of production failures, delegating code modifications to an external coding agent, verifying candidates by replaying failures in ephemeral workers, and promoting changes with user consent and rollback mechanisms. The central empirical result is an increase in the four-task mean grader score on the OpenClaw benchmark from 0.25 to 0.61 following a single evolution cycle without human intervention.","tokens_in":1917,"tokens_out":450,"duration_ms":40136,"significance":"If the empirical result and the safety of the source-level edits can be substantiated, this work would be significant for the field of autonomous agents by addressing a class of structural failures that text-mutable methods cannot reach. The argument that source-level adaptation is Turing-complete and deterministic is a conceptual strength. The use of a pluggable external coding agent and health-probe-gated rollback are practical contributions. However, the current presentation provides insufficient detail to evaluate these claims.","major_comments":[{"comment":"Abstract: The claim of lifting the mean grader score from 0.25 to 0.61 is presented without controls, variance estimates, multiple independent runs, or comparisons to text-only baselines, rendering the central empirical result unevaluable from the provided information.","section":"Abstract"},{"comment":"the described multi-stage pipeline: Verification by replaying a finite batch of past failures on ephemeral trial workers does not address the possibility of new structural failures introduced by the source rewrite (e.g., altered hook ordering or state invariants outside the replayed scenarios). This is load-bearing for the assertion that the edit is safe for promotion.","section":"the described multi-stage pipeline"}],"minor_comments":[{"comment":"Abstract: The four tasks comprising the OpenClaw mean and the definition of the grader are not specified, which would aid interpretation of the reported scores.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments. We address the two major points below and indicate the changes we will make.","responses":[{"response":"We agree the abstract requires more context. In revision we will add a reference to the experimental protocol in Section 4, include a text-only baseline comparison, and explicitly note that the reported result is from a single production cycle without variance estimates from repeated independent runs. The latter cannot be supplied from the existing data.","revision_made":"partial","referee_comment":"[Abstract] Abstract: The claim of lifting the mean grader score from 0.25 to 0.61 is presented without controls, variance estimates, multiple independent runs, or comparisons to text-only baselines, rendering the central empirical result unevaluable from the provided information."},{"response":"This observation is correct. Replay verification only confirms behavior on the curated failure batch. The promotion step relies on the health-probe-gated rollback to handle any new structural issues post-deployment. We will expand the pipeline description to state this limitation explicitly and clarify the role of rollback in the safety argument.","revision_made":"yes","referee_comment":"[the described multi-stage pipeline] the described multi-stage pipeline: Verification by replaying a finite batch of past failures on ephemeral trial workers does not address the possibility of new structural failures introduced by the source rewrite (e.g., altered hook ordering or state invariants outside the replayed scenarios). This is load-bearing for the assertion that the edit is safe for promotion."}],"tokens_in":1447,"tokens_out":340,"duration_ms":27989,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"Hi colleague,\n\nThe main thing to know about MOSS is that it builds a deterministic pipeline for agents to edit their own source code based on production failures, then promotes the change via container swap after replay verification. This is a step past the usual prompt or skill file tweaks.\n\nThe paper does a clean job laying out why source-level changes matter: they can reach routing, hooks, and invariants that text artifacts cannot touch, and the changes are deterministic rather than dependent on model compliance. The architecture keeps MOSS in charge of stages and verdicts while delegating the actual code edit to an external coding CLI, and the ephemeral trial workers plus health-probe rollback are practical choices for deployment.\n\nThe soft spots sit in the evidence and the verification claim. The abstract reports one lift on OpenClaw from 0.25 to 0.61 but gives no variance, multiple runs, or text-only baseline, so the size of the effect is hard to judge. More importantly, the verification replays only a curated batch of prior failures; that leaves any new structural problems created by the rewrite untested, and the paper does not add static checks or broader testing to close the gap. The stress-test note on this point holds up from the abstract.\n\nThis is for people working on long-running agent systems who want to explore deeper self-modification. It has enough of a system description and a focused claim to deserve peer review, though it will need stronger experiments and a tighter argument on safety before the central result lands.","headline":"MOSS has a concrete pipeline for source-level agent self-rewriting with container swaps, but the verification only replays past failures and the single reported result has no controls or baselines.","tokens_in":2405,"tokens_out":392,"would_cite":false,"duration_ms":29621,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Autonomous agents can rewrite their own source code to fix structural failures that text changes cannot reach.","keywords":["self-evolving agents","source code rewriting","autonomous agent systems","failure-driven adaptation","container deployment","agentic self-improvement"],"falsifier":"Live deployment of a promoted change that produces new structural failures absent from the original system on the same tasks.","tokens_in":2663,"feed_emoji":"🤖","tokens_out":647,"duration_ms":25506,"temperature":0.7,"pith_summary":"Current self-evolving agents modify only text artifacts such as prompts and workflows, leaving code-level elements like routing, hooks, and state invariants unreachable. MOSS performs evolution directly at the source level through a deterministic pipeline that curates failure evidence, delegates modifications to an external coding tool, verifies candidates by replaying failures on trial workers, and promotes changes with rollback safeguards. The approach is presented as Turing-complete and deterministic, independent of base-model compliance or context length. On the OpenClaw benchmark it raises mean grader score from 0.25 to 0.61 after one cycle with no human intervention. If the claim holds, recurring structural failures become fixable without waiting for external updates.","feed_headline":"Agents rewrite their own source code to fix structural failures","feed_subtitle":"MOSS lifts OpenClaw mean score from 0.25 to 0.61 in one autonomous cycle using failure batches and trial verification.","key_machinery":"Source-level rewriting pipeline anchored to failure batches, verified on ephemeral trial workers, and promoted with consent-gated container swap.","core_discovery":"MOSS performs self-rewriting at the source level on production agentic substrates. Each evolution is anchored to an automatically curated batch of production-failure evidence and proceeds through a deterministic multi-stage pipeline; code modification is delegated to a pluggable external coding-agent CLI while MOSS retains stage ordering and verdicts. Candidates are verified by replaying the batch against the candidate image in ephemeral trial workers, then promoted via user-consent-gated, in-place container swap with health-probe-gated rollback.","pith_inferences":["The same pipeline could be applied to other containerized agent systems by swapping the coding CLI.","Hybrid use with text-layer methods might address both code and prompt-level issues in one loop.","Repeated cycles could accumulate improvements that compound across multiple failure types.","Rollback mechanisms might need extension if verification batches miss rare edge cases."],"forward_implications":["Structural failures in routing, hook ordering, and dispatch become reachable for autonomous repair.","Evolution occurs deterministically through code rather than probabilistic text generation.","Changes remain stable under long-context drift because they reside in source rather than prompts.","A single cycle of failure-driven rewriting can raise task performance without external intervention."],"fun_headline_variants":["MOSS enables source-level rewriting in autonomous agents","Agents fix structural failures by rewriting their code","Production failure batches guide MOSS code evolution","Trial verification promotes source changes in agent systems","MOSS reaches 0.61 score on OpenClaw after source rewriting"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Replaying a curated batch of past failures on ephemeral trial workers is sufficient to certify that a source change will not introduce new structural failures during live operation.","fun_headline_variants_meta":{"raw":{"variants":["MOSS enables source-level rewriting in autonomous agents","Agents fix structural failures by rewriting their code","Production failure batches guide MOSS code evolution","Trial verification promotes source changes in agent systems","MOSS reaches 0.61 score on OpenClaw after source rewriting"]},"model":"grok-4.3","cost_usd":0.005073,"raw_usage":{"total_tokens":2428,"prompt_tokens":744,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":50728000,"prompt_tokens_details":{"text_tokens":744,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1621,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":744,"tokens_out":63,"duration_ms":22027,"temperature":1.0,"reasoning_tokens":1621,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T16:42:27.930807+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Live deployment of a promoted change that produces new structural failures absent from the original system on the same tasks.","supporting_citations":[],"review_version":2}