{"id":"987a942c-d0a0-4c12-b7c2-e4832519c00b","arxiv_id":"2608.01247","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RestoreKV generates a small learned restore cache in a single LoRA pass, recovering up to 35 accuracy points over query-agnostic eviction baselines under a 5% KV budget.","lead":"RestoreKV is a plug-in for KV cache compression that trains a few 'restore tokens' to summarize what gets evicted, recovering much of the lost accuracy when memory is tight. It works with existing eviction methods, adds negligible one-time cost, and keeps question answering and decoding unchanged afterward.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed restore-token count (n=8) limits capacity as context length grows; SCBench gains are much smaller than 4-11K gains, suggesting a long-context capacity bottleneck.","rationale":"The paper's key premise is that a single shared mechanism can generate a context-conditioned restore cache that compensates for eviction loss. The mechanism's output capacity is fixed by n=8 restore tokens per layer. As context length grows, the information that eight tokens can carry relative to the context shrinks linearly. The SCBench results provide the only evidence at ~100K scale and show much smaller gains than 4-11K benchmarks: at r=0.05, a 1.5-point average improvement versus a 34.6-point improvement on RULER-4K. This is consistent with a capacity bottleneck. If confirmed, the method would need n to scale with T, which would increase the one-time cache-construction overhead and complicate the 'less than 0.5% overhead' claim. The proposed test, varying n at fixed total KV budget on SCBench, would distinguish a capacity bottleneck from other limits such as LoRA adaptation or training-data distribution. The main results on the paper's four core benchmarks (RULER-4K, QASPER, QuALITY, LongHealth) remain credible, so the CONDITIONAL verdict is appropriate. However, the paper's broader claim that the benefit extends to very long contexts, which rests on the SCBench numbers, should be explicitly conditioned on the outcome of this scaling test. The reader's weakest assumption about generalization across context lengths is well aligned with this concern, but the reader did not identify the fixed n as the specific mechanism; hence partial agreement.","tokens_in":16550,"tokens_out":24892,"duration_ms":220357,"concrete_test":"On the SCBench ~104K suite, evaluate RestoreKV with n=8, 16, 32, and 64 at r=0.1 and r=0.05, each time reducing the retained-context budget so the total number of KV pairs stays at floor(r*T*L*H). If accuracy at n=32 or n=64 exceeds n=8 by more than, say, 5 points on average, the fixed n=8 is a capacity bottleneck at long context. If gains are flat across n, the limitation is not restore-cache size but another part of the mechanism, such as LoRA adaptation or training-data coverage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The shared restoration mechanism has a fixed capacity: n=8 restore tokens per layer produce nLH KV pairs, independent of context length T. Thus the relative information capacity of the restore cache, n/T, shrinks linearly with T. The SCBench results (Appendix B.2) are consistent with this: at ~104K tokens and r=0.05, RestoreKV improves KVzip from only 24.1 to 25.6 (a 1.5-point gain), whereas on RULER-4K the same budget ratio yields a 34.6-point gain (38.2 to 73.2). If this capacity bottleneck is real, then in the very long-context regime where KV eviction is most needed, the method's ability to compensate for lost information is sharply reduced. The paper's central claim, that RestoreKV 'substantially reduces compression-induced degradation,' is strongly supported at 4-11K contexts but not at 100K scale. Moreover, n being a fixed hyperparameter means the method would require tuning n as a function of T to maintain effectiveness, which would change the overhead profile and weaken the 'shared mechanism' premise. The paper's Table 4 shows that on 4K contexts, n=16 is worse than n=8, so the optimal n is not monotonic; this makes the long-context behavior genuinely uncertain.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RestoreKV, a plug-in for query-agnostic KV cache eviction. After context prefill, n=8 learned restore tokens attend to the full KV cache in a single LoRA-adapted pass, producing nLH context-conditioned KV pairs; these are concatenated with the base evictor's retained pairs under the same total KV budget. The restore-token embeddings and LoRA adapters are trained offline by self-distillation from the frozen full-cache model on 6.2k examples, with the budget ratio sampled uniformly. At inference the adapters are disabled after cache construction, so query processing and decoding use the original frozen backbone. Experiments on Qwen3-0.6B/4B/8B and Llama-3.1-8B-Instruct across RULER-4K, QASPER, QuALITY, and LongHealth (plus LongBench and SCBench in the appendix) report large gains at tight budgets, e.g., RULER-4K KVzip from 38.2 to 73.2 at r=0.05 on Qwen3-4B, and 59 of 60 paired budget-matched improvements over five base eviction methods.","tokens_in":16808,"tokens_out":9254,"duration_ms":81737,"significance":"If the main results hold, RestoreKV is a practically valuable complement to selection-based eviction: it keeps the base importance scorer and eviction rule unchanged, adds negligible one-time cache-construction overhead, and improves five different eviction methods across four backbones. The paper's ablations are a genuine strength: Table 2 isolates attention-side LoRA adaptation as the primary source of recovery, Table 3 controls for full-context conditioning, Appendix A.2 rules out the RoPE-offset confound, Appendix C.5 reports seed sensitivity, and Appendix A.3 tests training-inference evictor mismatch. The self-distillation target is an external full-cache behavior, so the evaluation is not circular. However, the fixed-size restore cache scales poorly to very long contexts: at roughly 104K tokens the reported SCBench gains are far smaller than at 4-11K, which limits the scope of the paper's central claim that compression-induced degradation is substantially reduced.","major_comments":[{"comment":"The long-context evidence in Appendix B.2 does not support the generalization claim made there. Because the restore cache has size nLH independent of context length T, its share of the total KV budget is n/(rT); at an average context of 104K tokens and r=0.05 this is about 0.15% of the budget, versus about 3.9% at the 4K RULER context. The reported SCBench averages are correspondingly small: 29.4 to 29.7 at r=0.4, 31.6 to 33.1 at r=0.2, 30.0 to 32.4 at r=0.1, and 24.1 to 25.6 at r=0.05, compared with a 34.6-point gain on RULER-4K at r=0.05. The statement that 'the benefit of the restore cache is not limited to short contexts' is technically true, but the benefit shrinks sharply with T, and the abstract's claim of 'substantially reduces compression-induced degradation' is not established at roughly 100K tokens. The authors should either scale n with context length (analyzing the resulting budget trade-off), provide a capacity analysis, or substantially temper the long-context claim.","section":"Appendix B.2; Eq. (3), Eq. (6)"},{"comment":"The non-monotonic effect of the restore-token count prevents the obvious remedy of simply increasing n for long contexts. At r=0.05, accuracy is 65.3 for n=1, 70.4 for n=2, 68.5 for n=4, 73.2 for n=8, and 69.1 for n=16, so the optimal n is not monotonic and n=16 is worse than n=8. This means the capacity bottleneck identified above cannot be resolved by increasing n without further study, and the paper should either report how n interacts with longer contexts (for example, an n=16 run at 32K or 104K) or explain the non-monotonicity.","section":"Table 4"},{"comment":"The headline empirical claims rest on single-seed runs. Table 1 and Figure 3 report seed-0 results for all settings, and Appendix C.5 reports seed statistics only for RULER-4K on Qwen3-4B. Since one of the 60 paired settings in Table 1 is a regression (SnapKV on QuALITY at r=0.1, -1.2), the consistency claim would be materially stronger if seed variance were reported for at least one additional benchmark, or if the single-seed nature of the main table were stated more prominently in Section 4.","section":"Section 4.2; Appendix C.5"}],"minor_comments":[{"comment":"The notation p_full_i and p_res_i is clear to a specialist, but the equations would benefit from an explicit statement that both distributions are over the vocabulary at answer position i and are conditioned on the same gold answer prefix y_{<i}.","section":"Section 3.3, Eq. (7)-(8)"},{"comment":"Please specify how the base evictor recomputes per-layer and per-head budgets when the total budget is reduced from B to B-nLH; if the allocation is proportional, the floor/ceil handling should be stated so that the claim of an unchanged eviction rule is precise.","section":"Section 3.2, Eq. (6)"},{"comment":"The abstract describes 'four long-context benchmarks' while the longest of these is about 11K tokens (LongHealth); the roughly 104K-token SCBench results appear only in the appendix. The context-length coverage of the headline claim should be stated more precisely.","section":"Abstract; Section 4.1"},{"comment":"The training mixture includes generic instructions such as 'summarize' and 'explain' alongside source-provided queries; a sentence on how well the generic-instruction examples transfer to the downstream benchmarks would be useful.","section":"Appendix C.1"},{"comment":"The paper provides a project page but no code or data release; releasing the training-data construction scripts and the evaluation harness would materially aid reproducibility.","section":"Reproducibility"},{"comment":"The overhead percentage is described relative to 'end-to-end compression time' in the text and 'compression time' in the caption; clarify whether this includes the full prefill or only the scoring/eviction phase.","section":"Figure 6(a)"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RestoreKV is a solid, well-scoped plug-in for query-agnostic KV eviction. The idea is simple: after prefill, a few learned restore tokens attend to the full KV cache through a LoRA pass, producing a small context-conditioned restore cache that takes slots in the existing KV budget. The base evictor is untouched, and the adapters are disabled afterward. The experimental story is convincing at the context lengths they emphasize. On RULER-4K with Qwen3-4B at a 5% budget, they move KVzip from 38.2 to 73.2, and the improvement pattern is consistent across four backbones, four benchmarks, and five base evictors. The ablations are genuinely informative: the RoPE-offset control rules out a positional artifact, the full-context conditioning ablation isolates the value of seeing the full cache, and the component ablations point to attention-side adaptation as the main driver. The seed sensitivity in the appendix is a nice touch.\n\nThe soft spots are mostly about completeness. No code or data is released, which matters for a method whose value is in exact numbers and easy integration. The main table reports single-seed runs, though the appendix seed check is reassuring. There is a citation gap for KVzip+ (the reference list has KVzap but not KVzip+), which should be fixed.\n\nThe bigger substantive concern is capacity. The restore cache uses a fixed n=8 tokens per layer, so the number of generated KV pairs is independent of context length. The SCBench results at ~104K tokens show much smaller gains: +1.5 at r=0.05 versus +34.6 on RULER-4K. That is consistent with a capacity bottleneck in the very regime where KV eviction matters most. The paper notes the SCBench gains but does not discuss this scaling limitation, and the fact that n=16 is no better than n=8 at 4K makes it unclear whether simply increasing n fixes it. This should be stated as a limitation and investigated before the method is sold as a long-context solution.\n\nOverall, the central claim holds for 4-11K contexts, which is where the paper's evidence is strongest. A serious referee should engage with it; the missing code/data and the long-context capacity question are exactly what a revision should address.","headline":"A well-engineered, budget-matched restore cache gives large gains at 4-11K contexts; the fixed-capacity mechanism looks weaker at 100K, and missing code/data hold it back from full reproducibility.","tokens_in":17387,"tokens_out":2357,"would_cite":true,"duration_ms":19779,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RestoreKV shows that a small, context-conditioned 'restore cache' added to any query-agnostic KV eviction method recovers most of the accuracy lost under aggressive compression.","keywords":["KV cache eviction","query-agnostic compression","learned restore tokens","LoRA adaptation","self-distillation","long-context LLM inference","RULER benchmark","KVzip"],"falsifier":"Evaluate RestoreKV at a retention ratio of r=0.01, below the minimum ratio sampled during training (0.025), on RULER-4K with Qwen3-4B. The paper's claim that one shared mechanism supports multiple budgets implies it should still improve over KVzip at this out-of-range ratio; if the improvement vanishes or turns negative, the mechanism only works inside the trained ratio range.","tokens_in":16313,"feed_emoji":"🔄","tokens_out":8414,"duration_ms":63870,"temperature":0.7,"pith_summary":"This paper argues that the standard formulation of query-agnostic KV cache eviction—choosing which original key-value pairs to keep—is incomplete. It introduces RestoreKV, which supplements the retained pairs with a small, context-conditioned 'restore cache' generated by a few learned tokens that read the full cache once before eviction. The central claim is that while the information lost by eviction is specific to each context, the mechanism that produces a compact complement of that information can be shared across contexts. If that claim holds, any existing eviction method can be plugged into RestoreKV under the same total memory budget and recover most of the accuracy it loses at tight budgets, with negligible one-time cost and no added query-time latency.","feed_headline":"Learned restore cache recovers accuracy lost by aggressive KV eviction","feed_subtitle":"Eight restore tokens lift RULER accuracy from 38.2 to 73.2 at a 5% KV budget.","key_machinery":"The central mechanism is the restore pass: a set of $n=8$ learnable restore-token embeddings that are processed, immediately after context prefill and before eviction, through one forward pass of the frozen backbone with LoRA adapters enabled. Because the restore tokens have causal access to the full KV cache, each outputs a KV pair per layer and head, forming a 'restore cache' of $nLH$ pairs. Budget matching is the identity that carries the argument: the total budget $B=\\lfloor r T L H\\rfloor$ is split so the restore cache occupies $nLH$ slots and the unchanged base evictor fills the remaining $B-nLH$ slots, preserving the query-time KV memory exactly. Training uses symmetric KL self-distillation against the full-cache model, and the LoRA adapters are disabled for all queries and decoding, so the only added cost is a single forward pass over eight positions at cache-construction time.","core_discovery":"RestoreKV's core discovery is that a single shared, learned transformation can turn the full KV cache of any context into a small set of restore KV pairs that compensate for what eviction removes. Concretely, eight learnable restore-token embeddings are appended after the context; in one pass through the frozen backbone equipped with LoRA adapters, they attend to all context key-value pairs and produce $nLH$ new context-conditioned KV pairs. These restore pairs occupy a reserved slice of the fixed KV budget, while the base evictor fills the remaining slots with original context pairs exactly as it would without RestoreKV. The whole system is trained offline by self-distillation: the restored-cache student is optimized to match the frozen full-cache teacher's answer-token distributions, with budget ratios sampled uniformly from $U(0.025,0.25)$. The paper reports that this recovers a large fraction of the accuracy lost under aggressive eviction—for example, raising KVzip from 38.2 to 73.2 on RULER-4K at a 5% KV budget on Qwen3-4B—and that the gain comes primarily from the attention-side LoRA adaptation that generates the context-conditioned restore cache, not from generic information stored in the restore-token embeddings.","pith_inferences":["RestoreKV suggests a general design principle for compression: instead of only selecting what to keep, reserve a small fraction of the budget for a learned summary of what was discarded; the same principle could apply to prompt summarization or embedding distillation where a shared reconstruction mechanism replaces per-instance optimization.","Because the restore tokens attend to the full cache, RestoreKV's overhead should grow with context length in absolute terms, but the reported 0.04 s at 32K implies the method can remain worthwhile at 100K+ contexts; a direct measurement of restore-pass latency at SCBench lengths would confirm this scaling.","The training-inference evictor mismatch results suggest that the restore mechanism adapts to the specific error pattern of the evictor it was trained with; a natural extension is to train RestoreKV jointly with a family of evictors or with an adversarial evictor so a single checkpoint remains effective when the eviction rule changes at deployment.","Because the restore cache is generated from the full cache before eviction, it can capture distributed patterns across many tokens that no selection-based method could retain; this hints that RestoreKV may excel on tasks where answers depend on global context rather than a few salient tokens, a hypothesis testable by comparing retrieval-heavy versus aggregation-heavy task categories within RULER."],"forward_implications":["Any query-agnostic evictor—KVzip, KVzip+, ContrastKV, SnapKV, H2O, or Fast KVzip—can be upgraded by training one RestoreKV checkpoint per model and evictor, improving 59 of 60 paired budget-matched settings on Qwen3-4B.","At a 5% KV budget, RestoreKV raises KVzip on RULER-4K from 38.2 to 73.2, and RestoreKV+ reaches 86.4 RULER accuracy at 16× compression on the KVPress Benchmark with KVzip+.","The one-time cache-construction cost is small: about 0.03–0.04 s (under 0.5% of compression time) and 84 MB (0.4% peak memory) on a 32K context with Llama-3.1-8B-Instruct, with no added query-time latency or memory.","The gains are driven by the attention-side LoRA adaptation that generates context-conditioned restore states, not by the learnable embeddings themselves; fixed embeddings with LoRA recover 96% of the full improvement.","The method generalizes to contexts much longer than the training data: on SCBench's ~104K-token tasks, RestoreKV improves KVzip at all four tested budget ratios despite being trained on contexts up to 15K tokens."],"supporting_citations":[{"why":"Provides KVzip, the primary base eviction method whose query-agnostic formulation and KVPress evaluation protocol RestoreKV builds on and compares against.","marker":"Kim et al. 2025"},{"why":"Supplies KVzip+ and KVzap, the base methods used for the 16× KVPress result and the learned-scorer compatibility test.","marker":"Jegou and Jeblick 2026"},{"why":"ContrastKV, one of the five base eviction methods RestoreKV is evaluated against to show plug-in generality.","marker":"Chen et al. 2026"},{"why":"LoRA is the parameter-efficient adaptation mechanism used to generate the restore cache with only 0.4% trainable parameters.","marker":"Hu et al. 2022"},{"why":"Self-distillation from the frozen full-cache teacher is the training objective that shapes the restore cache.","marker":"Hinton, Vinyals, and Dean 2015"},{"why":"Attention Matching is the synthesized-cache baseline compared on accuracy versus construction time.","marker":"Zweiger et al. 2026"}],"fun_headline_variants":["Restore tokens rescue KV cache eviction: 38→73 RULER at 5% budget","Eight restore tokens fix aggressive KV eviction, lifting RULER by 35 points","RestoreKV: shared learned restore cache recovers evicted accuracy","One pass, eight tokens: restore cache beats eviction loss","RestoreKV: 59 of 60 gains under tight KV budgets via learned restore"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The one shared set of restore-token embeddings and LoRA adapters, trained on about 6.2 thousand examples from three text corpora, transfers across unseen contexts, across retention ratios, and to contexts about seven times longer than anything seen in training, without any per-context fitting. If that transfer fails, RestoreKV would need per-context optimization and would lose its efficiency advantage.","fun_headline_variants_meta":{"raw":{"variants":["Restore tokens rescue KV cache eviction: 38→73 RULER at 5% budget","Eight restore tokens fix aggressive KV eviction, lifting RULER by 35 points","RestoreKV: shared learned restore cache recovers evicted accuracy","One pass, eight tokens: restore cache beats eviction loss","RestoreKV: 59 of 60 gains under tight KV budgets via learned restore"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00049,"raw_usage":{"total_tokens":2506,"prompt_tokens":1135,"completion_tokens":1371,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":751,"completion_tokens_details":{"reasoning_tokens":1265}},"tokens_in":751,"tokens_out":1371,"duration_ms":9545,"temperature":1.0,"reasoning_tokens":1265,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:10:04.548584+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate RestoreKV at a retention ratio of r=0.01, below the minimum ratio sampled during training (0.025), on RULER-4K with Qwen3-4B. The paper's claim that one shared mechanism supports multiple budgets implies it should still improve over KVzip at this out-of-range ratio; if the improvement vanishes or turns negative, the mechanism only works inside the trained ratio range.","supporting_citations":[{"cited_title":"C ontrast KV : Robust KV Cache Eviction via Contrastive Signal Fusion for Multi-Query Generalization","cited_arxiv_id":null,"evidence_quote":"ContrastKV, one of the five base eviction methods RestoreKV is evaluated against to show plug-in generality."}],"review_version":1}