{"id":"256c7aa5-2dcc-4ca5-917b-d3ba8643dc4b","arxiv_id":"2605.28053","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"RW-TTT enables batched serving for request-owned TTT state via owner/version tagging and selective batching of compatible phases, reaching 274.61 tok/s and 9.31x speedup on one GPU with eight InPlace-TTT streams while preserving RULER behavior.","lead":"RW-TTT is a serving system that tags each LLM decode step with its request owner, version, and READ/WRITE effect to safely batch only compatible operations while committing updates only to the owning request. This delivers 9.31x higher throughput than sequential execution for request-owned test-time training state under fixed memory.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"Correctness of RW-TTT batching depends on owner/version/READ-WRITE tags identifying all non-conflicts, verified only by paper's empirical checks rather than exhaustive or formal argument.","rationale":"The reader's weakest_assumption directly names the load-bearing gap. Full text does not appear to add formal verification or exhaustive enumeration beyond the reported checks, so the concern remains unchanged.","tokens_in":1674,"tokens_out":313,"duration_ms":17644,"concrete_test":"Take the smallest non-trivial case (2–3 concurrent InPlace-TTT streams) and exhaustively enumerate all legal batching decisions under the paper's tagging rules; for each schedule, compare final per-request state and output tokens against a reference sequential execution. Any divergence falsifies completeness of the conflict detection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that selective batching + owner-only commit never corrupts request-owned TTT state (fast weights, deltas, etc.). The paper tags each decode step and batches only compatible phases, then reports that it 'passes owner/version checks' and preserves RULER behavior. This leaves open whether the tagging rules are complete: e.g., whether concurrent READ/WRITE phases on different versions can still produce observable divergence when streams interleave at the level of individual matrix multiplies or when version increments interact with batched GEMM kernels. No model-checked invariant, no exhaustive enumeration of small stream counts, and no reduction to a known serializable schedule is supplied.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents RW-TTT, a batched serving system for LLMs that perform request-owned test-time training (e.g., fast weights or low-rank deltas). It tags each decode step with owner, version, and READ/WRITE effect, batches only compatible phases, and commits updates only to the owning request. On one GPU with eight InPlace-TTT streams it reports 274.61 aggregate tok/s (9.31x over sequential serving, 3.44x over per-stream replicas under identical memory budget), while preserving RULER benchmark behavior and passing owner/version checks.","tokens_in":1824,"tokens_out":345,"duration_ms":26268,"significance":"If the batching rules are sound, the work enables substantially higher throughput for serving adaptive per-request TTT models without state corruption, addressing a practical barrier to deploying such methods at scale under fixed memory constraints.","major_comments":[{"comment":"Abstract: the reported throughput (274.61 tok/s) and speedups (9.31x, 3.44x) are stated without any description of experimental setup, number of trials, error bars, hardware configuration details, or how the compatibility rules were validated beyond the owner/version checks.","section":"Abstract"},{"comment":"Abstract: the central correctness claim (selective batching plus owner-only commit never corrupts request-owned state) rests solely on passing owner/version checks and RULER preservation; no formal invariant, model-checked argument, or exhaustive enumeration of interleaving scenarios at the level of batched GEMM operations is supplied.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the comments. We address each major comment point by point below.","responses":[{"response":"The abstract is written for brevity. Full experimental details—including the single-GPU configuration, eight InPlace-TTT streams, and validation via owner/version checks plus RULER—are provided in the Experiments section. We will revise the abstract to include a concise statement of the hardware setup and a pointer to the evaluation section for trials, error bars, and compatibility validation.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the reported throughput (274.61 tok/s) and speedups (9.31x, 3.44x) are stated without any description of experimental setup, number of trials, error bars, hardware configuration details, or how the compatibility rules were validated beyond the owner/version checks."},{"response":"Correctness follows from the explicit per-step tagging (owner, version, READ/WRITE) and the deterministic selective-batching rules that only combine compatible phases while restricting commits to the owner. These rules are validated by the owner/version checks (which would surface any corruption) and by unchanged RULER behavior. The current manuscript does not supply a formal invariant or model-checked argument; we will add an expanded discussion of the phase-compatibility invariants in the revision.","revision_made":"partial","referee_comment":"[Abstract] Abstract: the central correctness claim (selective batching plus owner-only commit never corrupts request-owned state) rests solely on passing owner/version checks and RULER preservation; no formal invariant, model-checked argument, or exhaustive enumeration of interleaving scenarios at the level of batched GEMM operations is supplied."}],"tokens_in":1300,"tokens_out":374,"duration_ms":32637,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper solves a practical barrier in LLM serving: how to batch decode steps when each request maintains its own changing TTT state like fast weights or deltas. They tag every step with its owner, version, and read/write effect, batch only compatible phases, and commit writes only to the matching owner. On one GPU running eight InPlace-TTT streams this yields 274.61 aggregate tokens per second, 9.31 times sequential execution and 3.44 times separate replicas at the same memory budget, while RULER scores stay the same.\n\nWhat is new is the concrete read-write TTT serving formulation plus the owner/version/READ-WRITE tagging rules that decide batch eligibility. Earlier serving work assumed static shared weights, so this directly targets the corruption risk that appears once state becomes request-owned.\n\nThe paper does a solid job turning the problem into an engineering fix and showing measurable speedups without obvious behavior change on their benchmark. The numbers are presented as direct measurements, not derived from fitted parameters.\n\nThe soft spot is the completeness of the compatibility rules. The validation consists of owner/version matching plus unchanged RULER results. That leaves open whether the tags catch every interleaving case at the level of batched GEMM kernels or version increments. No formal invariant, no exhaustive small-case enumeration, and no reduction to serializable schedules are mentioned, so the claim that selective commit prevents corruption rests on empirical checks rather than exhaustive argument. Experimental details on setup, variability, and edge cases would also need to be clearer in the full text.\n\nThis is for inference-systems people who want to add per-request adaptation to production serving. A reader working on that exact constraint would get concrete value from the tagging approach and the reported gains.\n\nIt deserves peer review because it identifies a real deployment obstacle and supplies a working mitigation with throughput evidence, even if the correctness argument could be strengthened.","headline":"RW-TTT gives a tagging scheme to batch per-request TTT updates safely enough to get 9x throughput on eight streams, but the safety rests on owner/version checks and one benchmark rather than stronger validation.","tokens_in":2282,"tokens_out":476,"would_cite":false,"duration_ms":37794,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"RW-TTT batches TTT decode steps across requests by tagging each with owner, version, and read/write effect, then commits updates only to the matching owner.","keywords":["test-time training","batched LLM serving","request-owned state","fast weights","inference optimization"],"falsifier":"An execution trace in which two requests pass the owner/version checks yet one request's final output differs from the output obtained by running the same requests sequentially.","tokens_in":2593,"feed_emoji":"","tokens_out":573,"duration_ms":24539,"temperature":0.7,"pith_summary":"Test-time training adapts an LLM by updating request-specific state such as fast weights during generation. Standard batching assumes shared static weights and therefore either runs requests serially or risks corrupting per-request state. The paper formulates the resulting constraint as read-write TTT serving and solves it with a tagging scheme that identifies compatible phases for simultaneous execution. On one GPU the method delivers 274.61 aggregate tokens per second across eight InPlace-TTT streams while matching sequential behavior on the RULER benchmark.","feed_headline":"Batched TTT serving hits 9.31x speedup on one GPU","feed_subtitle":"Owner and version tags let eight request streams share compute while each keeps its own fast-weight state.","key_machinery":"Owner/version/READ-WRITE tagging scheme that restricts batch formation to non-conflicting phases and restricts commits to the matching owner.","core_discovery":"RW-TTT tags every decode step with its owner identifier, version counter, and READ or WRITE effect; it forms batches only from mutually compatible phases and commits each update exclusively to the owning request's state, thereby restoring safe batching without altering the underlying TTT algorithm.","pith_inferences":["The same tagging discipline could be applied to other request-owned state updates such as online learning of low-rank adapters.","The approach may reduce the need for separate model replicas in multi-tenant serving environments that support stateful inference.","If the checks scale to larger batch sizes, the method could change the economics of deploying TTT-augmented models on shared hardware."],"forward_implications":["Eight concurrent fast-weight TTT streams run at 9.31 times the throughput of sequential execution under identical memory limits.","Per-stream replica replication is no longer required to achieve isolation, freeing memory that can be used for longer contexts or more streams.","Correctness on long-context tasks is preserved when the tagging rules are followed."],"fun_headline_variants":["RW-TTT batches TTT by owner version and effect","Request-owned TTT state supports batched serving","Version checks ensure safe multi-stream TTT","274 tok/s from RW-TTT on eight InPlace-TTT streams"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The owner and version tags together with selective commit are sufficient to prevent any cross-request state corruption.","fun_headline_variants_meta":{"raw":{"variants":["RW-TTT batches TTT by owner version and effect","Request-owned TTT state supports batched serving","Version checks ensure safe multi-stream TTT","274 tok/s from RW-TTT on eight InPlace-TTT streams"]},"model":"grok-4.3","cost_usd":0.005129,"raw_usage":{"total_tokens":2449,"prompt_tokens":580,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":51287000,"prompt_tokens_details":{"text_tokens":580,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1806,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":580,"tokens_out":63,"duration_ms":13888,"temperature":1.0,"reasoning_tokens":1806,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T13:44:00.143815+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An execution trace in which two requests pass the owner/version checks yet one request's final output differs from the output obtained by running the same requests sequentially.","supporting_citations":[],"review_version":1}