{"id":"2cd4c2ad-1d92-4883-b0af-a5d9ed4cc91f","arxiv_id":"2607.06519","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Frequency-guided inter-layer KV sharing with logit-aware head routing nearly matches full-cache long-context accuracy at about 3.9× lower peak KV memory.","lead":"FreqDepthKV compresses long-context LLM key-value caches by sharing low-frequency structure across neighboring layers while keeping sparse high-frequency residuals for heads that matter. If the reported accuracy and 3.9× memory savings hold, it is a practical systems lever for cheaper long-context serving.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The strongest claim rests on an unverifiable experimental stack: unnamed base model, no code/data, and a polluted reference list undermine Table 1 as evidence.","rationale":"I agree with CONDITIONAL and with the high correctness_risk / moderate confidence. The method (DCT depth factorization + three-mode logit routing, Eqs. 1–3) is clearly described and the ablation table is internally coherent. The load-bearing soft spot for the *strongest claim*, however, is not primarily the fixed-prefill routing assumption the reader highlighted—though that is a genuine limitation admitted in §7—but the fact that Table 1 cannot be audited at all. Without a named model, code, seeds, or clean bibliography, the near-parity numbers are not yet scientific evidence. That is why I only partially agree with the reader's weakest_assumption framing: routing stability matters for the *design* claim, but provenance is what currently props up the *performance* claim. Verdict stays CONDITIONAL; it should not move to ACCEPT until the concrete reproduction test passes, nor to REJECT solely on design grounds, because the technical idea remains a reasonable extension of MiniCache if the numbers can be verified.","tokens_in":12791,"tokens_out":636,"duration_ms":8045,"concrete_test":"Require authors to name the exact base model (architecture, parameter count, checkpoint) and release a minimal reproduction package that regenerates Table 1 row 'FreqDepthKV' vs 'Full KV' and 'MiniCache' on one public LongBench subset under the stated 32k prefill; if the EM/F1 gap to Full KV exceeds ~1 point or MiniCache is not reproduced within ~0.5 points of the paper's 56.6/61.0, the strongest claim does not hold as stated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that FreqDepthKV nearly matches Full KV (58.3 vs 58.7 EM, 63.0 vs 63.4 F1, etc.) at 3.9× compression (Table 1). That claim is only as strong as the experimental provenance. The paper never names the base long-context decoder, gives no architecture/size/checkpoint, provides no code or data release, reports no error bars or multi-seed runs, and tunes the compression budget on a held-out LongBench subset without disclosing the selected budgets for baselines. The reference list is heavily polluted with off-topic PMC/sensor/biology entries and the header claims NeurIPS 2023 while the arXiv stamp is 2026. Under those conditions, the Table 1 deltas (including the MiniCache comparison that is the main depth-sharing foil) cannot be treated as established facts; they are uncheckable numbers attached to a plausible method description. The reader's weakest_assumption (fixed prefill probe sufficiency) is a real design limitation the paper itself flags in Future Work, but it is secondary: even a perfect routing probe would not make the headline numbers trustworthy without a named model and reproducible setup.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes FreqDepthKV, an inference-time KV-cache compression method that stacks adjacent-layer keys/values, applies a fixed DCT depth transform (Eq. 1), and stores a shared low-frequency component plus sparse high-frequency residuals. An online prefill probe routes each head to shared-depth, residual-depth, or exact mode by minimizing a reconstruction-aware attention-logit loss plus memory penalty (Eq. 2–3). On a 32k prefill setup the method reports 58.3 EM / 63.0 F1 / 32.5 ROUGE-L / 48.1 pass@1, nearly matching Full KV while reducing peak KV memory to 6.2 GB (3.9×) and raising throughput to 70.4 tokens/s (Table 1), with ablations in Table 2 attributing gains to the frequency factorization, residuals, and routing.","tokens_in":13203,"tokens_out":982,"duration_ms":9888,"significance":"If the empirical claims hold, the work is a useful systems contribution: it refines MiniCache-style depth sharing with frequency residuals and logit-aware head routing, remains training-free, and is complementary to token eviction and quantization. The design is concrete (DCT blocks, three cache modes, residual scoring) and the ablations isolate components. Significance is currently limited by incomplete experimental provenance—the base model is unnamed, budgets and variance are underspecified, and the reference list is unreliable—so the headline Table 1 deltas cannot yet be treated as established facts.","major_comments":[{"comment":"§4 Experiments never names the base long-context decoder (architecture, size, or checkpoint), only “the same base long-context decoder with a 32k-token prefill window.” Without this, Table 1’s Full-KV and baseline numbers (including the MiniCache comparison that is the main depth-sharing foil) are not reproducible or comparable to the literature, so the central claim that FreqDepthKV nearly matches Full KV at 3.9× cannot be verified.","section":null},{"comment":"Table 1 / §4: no error bars, multi-seed runs, or multi-model results are reported, and compression budgets are “tuned on a held-out subset of LongBench” without disclosing the selected budgets or residual rates r_b,h for FreqDepthKV or baselines. Aggregate EM/F1/ROUGE/pass@1 and systems metrics are therefore hard to interpret as stable gains rather than single-run outcomes under an opaque budget protocol.","section":null},{"comment":"§3, Eq. (2) and Future Work: the load-bearing assumption that a one-shot prefill probe over |P|=128 positions (recent tokens, document boundaries, high-entropy rows), using key-induced logit reconstruction, is a sufficient proxy for which heads must keep residual/exact storage throughout decoding is asserted but not stress-tested. The paper itself notes routing is fixed after prefill; without generation-time re-routing experiments or failure cases on multi-step/needle tasks, the claim that task answers remain correct under aggressive residual sparsity is only weakly supported.","section":null}],"minor_comments":[{"comment":"Header claims “37th Conference on Neural Information Processing Systems (NeurIPS 2023)” while the arXiv stamp is 2026; this should be corrected.","section":null},{"comment":"References contain many off-topic PMC/sensor/biology/fluid-dynamics entries unrelated to KV cache compression; the bibliography needs a thorough cleanup.","section":null},{"comment":"Figure 1 is described as summarizing routing patterns in §4, but the caption only restates the high-level idea; a figure that actually shows head-mode assignments would help.","section":null},{"comment":"Notation: B is used both for block size and (implicitly) for the DCT basis F_B; residual rate r_b,h and the exact split of “first coefficient group” vs remaining groups should be stated more precisely.","section":null},{"comment":"Systems metrics (tokens/s, TTFT, peak GB) lack hardware/software stack details (GPU, batch size, kernel fusion), which limits interpretability of the 70.4 tokens/s and 6.2 GB claims.","section":null}],"recommendation":"major_revision","confidential_remarks":"The combination of an unnamed base model, no code/data, a polluted reference list, and a mismatched NeurIPS 2023 header raises a provenance concern beyond ordinary missing details. I would treat the current Table 1 numbers as provisional until the experimental stack is fully specified. Methodologically the idea is plausible and worth a revision cycle; I would not reject on novelty alone if the authors can name the model, release budgets/seeds, and clean the bibliography."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing to know: this is a systems paper that takes MiniCache-style depth sharing and makes it frequency-structured—fixed DCT over adjacent-layer KV, keep a shared low-frequency piece, store sparse high-frequency residuals only where an online prefill probe says attention logits would move. Heads get routed to shared, residual, or exact. That design is clear, incremental, and actually new relative to uniform layer merge and pure token eviction.\n\nWhat it does well is the framing and the ablations. They measure routing on key-induced logit reconstruction rather than only token saliency, which is the right failure mode for needle/retrieval/code. Table 2 moves in the expected directions when you strip residuals, routing, exact mode, or the DCT factorization. Future Work admits the fixed prefill probe is a real limit. The method is also honestly positioned as complementary to SnapKV-style eviction and KIVI-style quant.\n\nThe soft spots are mostly provenance, not math. The base model is never named—no size, architecture, or checkpoint. No error bars, no multi-seed, no code/data. Budgets are tuned on held-out LongBench then frozen without reporting the chosen knobs per baseline. The reference list is badly polluted with off-topic PMC/sensor/biology entries, and the header says NeurIPS 2023 while the stamp is 2026. Under those conditions the headline deltas (58.3 vs 58.7 EM, 3.9×, 70.4 tok/s) cannot be treated as established facts; they are numbers attached to a readable algorithm. The fixed-probe assumption is secondary and they already flag it.\n\nWho this is for: people who already care about KV systems and MiniCache-class depth compression. A serious referee should see it if the authors fix identity, reproducibility, and citations. I would not cite the numbers yet. Engage the method idea; do not trust the leaderboard until the stack is named and checkable.","headline":"Plausible MiniCache extension with a clean frequency+routing idea, but Table 1 is uncheckable until they name the model and clean the paper.","tokens_in":13797,"tokens_out":504,"would_cite":false,"duration_ms":31851,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"FreqDepthKV compresses long-context LLM key-value caches by sharing low-frequency depth components across layers while keeping sparse high-frequency residuals that protect retrieval and reasoning.","keywords":["KV cache compression","long-context LLM inference","depth-frequency factorization","attention head routing","high-frequency residuals","reconstruction-aware loss","inference-time compression"],"falsifier":"On a long needle-retrieval or multi-step code task, if the prefill probe routes heads to shared-depth mode yet later decoding produces wrong answers that full KV gets right—and restoring residual or exact storage for those same heads recovers correctness—then the prefill logit proxy fails as a stand-in for end-to-end task fidelity.","tokens_in":13693,"feed_emoji":"💾","tokens_out":985,"duration_ms":21223,"temperature":0.7,"pith_summary":"Long-context language model inference is bottlenecked by the memory and bandwidth of key-value caches, yet aggressive compression often erases the layer-specific evidence that retrieval and multi-step reasoning need. This paper introduces FreqDepthKV, an inference-time method that factorizes adjacent-layer caches into shared low-frequency depth components and sparse high-frequency residuals, then uses a lightweight prefill probe to assign each attention head to shared-depth, residual-depth, or exact mode according to how compression would change attention logits. The policy adapts to prompt structure without retraining. On long-context QA, needle retrieval, summarization, and code generation with a 32k-token prefill, it closely matches full-cache accuracy while cutting peak KV memory to 6.2 GB (about 3.9\times compression) and raising decoding throughput to 70.4 tokens/s. A sympathetic reader cares because many long prompts hide decisive evidence in a few token-head-layer interactions that average-error depth sharing would discard.","feed_headline":"KV cache cut 3.9× while matching full long-context accuracy","feed_subtitle":"Shared low-frequency depth components plus sparse residuals protect the evidence retrieval needs.","key_machinery":"Depth-frequency factorization with reconstruction-aware head routing: adjacent-layer KV states are transformed with a fixed DCT basis into shared low-frequency components and sparse high-frequency residuals; a prefill probe routes each head to shared-depth, residual-depth, or exact mode by minimizing a memory-penalized attention-logit reconstruction loss.","core_discovery":"FreqDepthKV establishes that inter-layer KV redundancy is frequency-structured: sharing low-frequency depth components while selectively preserving sparse high-frequency residuals—and exact entries for reconstruction-sensitive heads—lets inference-time compression nearly match full KV accuracy on long-context QA, summarization, and code tasks at roughly 3.9× lower peak KV memory and higher throughput.","pith_inferences":["Generation-time re-routing, which the paper only sketches as future work, may be necessary when relevant evidence shifts mid-decode in multi-turn or long-generation settings.","Assigning lower bit-width to shared low-frequency coefficients than to sparse residuals could cut bandwidth further without changing the routing logic.","The same logit-sensitivity probe could identify heads that are chronically non-compressible across models, guiding architecture choices toward more cache-friendly layers.","Prompts with diffuse evidence and weak document boundaries may force more exact-mode heads, shrinking the compression advantage relative to retrieval-heavy workloads."],"forward_implications":["Aggressive depth compression can retain retrieval and code accuracy when high-frequency residuals are kept for logit-sensitive heads.","A prefill-only reconstruction-aware router adapts compression to prompt structure without model retraining.","Depth-frequency sharing stacks with token eviction and quantization for joint sequence, precision, and depth savings.","Mixed layer-block sizes (tighter near model boundaries) balance memory and quality better than uniform blocks.","At 32k prefill, peak KV memory can fall to 6.2 GB with 70.4 tokens/s throughput while closely matching full-KV task scores."],"fun_headline_variants":["FreqDepthKV shares low-freq depth for 3.9× KV cut matching full accuracy","Frequency-guided depth sharing compresses KV 3.9× with near-full long-context scores","Inter-layer low-freq KV components plus residuals enable 3.9× memory savings","Adaptive head modes keep reconstruction-critical evidence under 3.9× compression","Sparse high-freq residuals protect retrieval as FreqDepthKV halves-plus KV size"],"cache_read_input_tokens":3584,"weakest_assumption_plain":"A one-shot prefill probe over a small set of query positions is assumed to be a stable enough proxy for which heads must keep residual or exact caches so that answers stay correct throughout decoding without re-routing.","fun_headline_variants_meta":{"raw":{"variants":["FreqDepthKV shares low-freq depth for 3.9× KV cut matching full accuracy","Frequency-guided depth sharing compresses KV 3.9× with near-full long-context scores","Inter-layer low-freq KV components plus residuals enable 3.9× memory savings","Adaptive head modes keep reconstruction-critical evidence under 3.9× compression","Sparse high-freq residuals protect retrieval as FreqDepthKV halves-plus KV size"]},"model":"grok-4.5","effort":"low","cost_usd":0.00706,"raw_usage":{"total_tokens":1764,"prompt_tokens":785,"num_sources_used":0,"completion_tokens":99,"cost_in_usd_ticks":70600000,"prompt_tokens_details":{"text_tokens":785,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":880,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":785,"tokens_out":99,"duration_ms":8713,"temperature":1.0,"reasoning_tokens":880,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T00:10:34.383023+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a long needle-retrieval or multi-step code task, if the prefill probe routes heads to shared-depth mode yet later decoding produces wrong answers that full KV gets right—and restoring residual or exact storage for those same heads recovers correctness—then the prefill logit proxy fails as a stand-in for end-to-end task fidelity.","supporting_citations":[],"review_version":2}