{"id":"369db839-4cef-4120-b974-e4368c593c77","arxiv_id":"2607.04593","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"SAE concept-overlap grouping of visual tokens yields content-adaptive pruning/merging that preserves VLM accuracy better than attention or similarity baselines at moderate reduction.","lead":"TORINO cuts visual tokens in vision-language models by grouping image patches that share the same Sparse Autoencoder concepts, then pruning or merging them without retraining. It keeps accuracy high at moderate compression and adapts how many tokens each image keeps.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"CLS-trained SAE transfer to patches is load-bearing and only partially validated; moderate-regime gains may not generalize beyond this dictionary.","rationale":"The reader correctly isolates the CLS-to-patch SAE transfer as the weakest assumption supporting the strongest claim. The paper is otherwise careful: it reports the spatial-SAE gap, the extreme-compression loss to PruneSID, and the necessity of the SAE versus raw embeddings (Table 2). No internal inconsistency appears; the concern is empirical support for the key inductive bias. Because the moderate-regime gains are real under the public CLS SAE and the limitation is already partially quantified, the verdict remains CONDITIONAL rather than REJECT. Shipping code plus a full-patch SAE re-run would move it toward ACCEPT. I agree with the reader’s identification of the load-bearing point and do not adjust the verdict.","tokens_in":25327,"tokens_out":582,"duration_ms":5165,"concrete_test":"Train (or obtain) a Matryoshka BatchTopK SAE on all spatial patch tokens of ImageNet-1K at the same block 22 of CLIP ViT-L/14@336, freeze it, and re-run the exact dynamic TORINO-P/M pipelines of Table 1 (ε=64, (k,δ)∈{(1,1),(2,2),(3,3)}) on LLaVA-1.5-7B. If relative scores at the ~217- and ~129-token tiers fall by more than ~2 pp versus the CLS SAE, the transfer assumption fails and the moderate-regime claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (moderate-compression superiority of concept-overlap grouping) rests on the assumption that a Matryoshka BatchTopK SAE trained solely on CLIP ViT-L/14 CLS activations at block 22 yields monosemantic latents that faithfully group spatial patch tokens of the same encoder (Sec. 3.1, 4.1, 4.5). The paper’s own spatial-SAE ablation (two random patches per image) shows large drops that widen with compression (Table 4: TORINO-M −2.7 / −7.7 / −19.6 pp at B=192/128/64; Table 10 dynamic), so the CLS dictionary is not interchangeable with a patch-trained one. Late-layer bidirectional mixing is offered as justification, but no direct monosemanticity or concept-coverage metrics are reported for patch activations under the CLS SAE. Without that validation, the reported 98.7 % / 97.1 % relative scores at ~62–78 % reduction could be an artifact of this particular public dictionary rather than a general property of SAE concept overlap.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"TORINO is a training-free, plug-and-play visual token reduction method for VLMs that projects patch embeddings (CLIP ViT-L/14@336, block 22) through a frozen Matryoshka BatchTopK SAE into a sparse concept space, builds a token graph by δ-overlap of top-k active latents, and reduces each connected component by either peak-activation pruning (TORINO-P) or log-size-rescaled merging (TORINO-M). A fixed-budget variant adds truncation of the largest groups and padding by peak activation. On LLaVA-1.5-7B/13B across nine VLMEvalKit benchmarks the method reports content-adaptive retention (~217 / ~129 / ~47 tokens) with relative macro-scores of 98.7% (TORINO-M) and 97.1% (TORINO-P) at moderate compression, outperforming Random, FOLDER, PruneSID and PruMerge in that regime; an SAE-vs-raw ablation (−5.2 pp), expansion-factor Pareto, latency isolation and qualitative failure modes are also provided.","tokens_in":25667,"tokens_out":1134,"duration_ms":9173,"significance":"If the moderate-regime gains hold under broader SAE training, the work supplies a concrete instance of pragmatic interpretability: monosemantic SAE latents are used not only for analysis but as the decision basis for inference-time compression. The content-adaptive budget (G_dyn emerging from concept groups without an explicit complexity score), the unified prune/merge framework, the matched dynamic-vs-fixed comparison, and the public-dictionary reproducibility are genuine strengths. The contribution is incremental rather than foundational—limited to LLaVA + one CLIP SAE—but it is a clean, well-documented step that other VLM token-reduction lines can build on.","major_comments":[{"comment":"Sec. 3.1, 4.1 and 4.5 (and Tables 4, 10): the central claim that concept-overlap grouping is superior rests on a Matryoshka BatchTopK SAE trained only on CLS activations at block 22. The paper’s own spatial-SAE ablation (two random patches) shows large, compression-widening drops (−2.7 / −7.7 / −19.6 pp for TORINO-M at B=192/128/64). Late-layer mixing is offered as justification, yet no monosemanticity scores, concept-coverage statistics or reconstruction metrics are reported for patch activations under the CLS SAE. Without that validation the 98.7 % / 97.1 % relative scores could be dictionary-specific rather than a general property of SAE concept overlap; either a full-patch SAE or direct patch-level monosemanticity numbers are needed to underwrite the claim.","section":"Sec. 3.1, 4.1, 4.5; Tables 4, 10"},{"comment":"Table 1 and Sec. 4.2: at the most aggressive tier (~47 tokens, ↓92 %) PruneSID leads (92.8 % vs TORINO-P 91.1 %). The abstract and introduction frame TORINO as achieving “favorable efficiency-accuracy trade-offs” without regime qualification. The moderate-regime advantage is real and should be stated as the primary result; the extreme-compression ranking should be reported with equal prominence so that the contribution is not overstated.","section":"Table 1, Sec. 4.2, Abstract"}],"minor_comments":[{"comment":"Eq. (10): the log-size rescale 1+log|G_g| is taken from ToMe without an ablation against plain averaging or other sublinear factors; a one-line comparison would strengthen the design choice.","section":"Eq. (10), Sec. 3.3"},{"comment":"Fig. 3 and latency column of Table 1: no error bars or multi-seed statistics are given for either the per-image G_dyn distributions or the isolated reduction-module timings; even a short note on variance would help.","section":"Fig. 3, Table 1"},{"comment":"Sec. 4.1: the three (k,δ) pairs are presented as inducing distinct average budgets, yet no sensitivity sweep around those discrete points is shown; a brief grid would clarify robustness.","section":"Sec. 4.1"},{"comment":"Typographical inconsistencies appear throughout (LLaV A / LLaVA, fac ¸ade, T´el´ecom); a single pass of copy-editing would remove them.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The novelty claim (“first SAE-based framework for visual token reduction”) appears accurate on the cited literature. The work is a solid systems paper whose main risk is over-generalization from a single public CLS SAE; once the authors either retrain on patches or supply patch-level monosemanticity evidence, the moderate-regime result is publishable. Scope is appropriate for a CV / multimodal venue."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: TORINO is a clean plug-and-play reducer that groups visual tokens by shared top-k SAE latents (δ-overlap connected components), then prunes or merges inside groups, so the retained count emerges from image content without an explicit complexity score. That is a genuine new application of pragmatic interpretability, not just another attention or cosine baseline.\n\nWhat they do well is the empirical package. On LLaVA-1.5-7B/13B they beat FOLDER, PruneSID, and PruMerge at moderate compression (~62–78% tokens removed): TORINO-M hits 98.7% relative at ~217 tokens, TORINO-P 97.1% at ~129. They match dynamic vs fixed-budget averages, isolate reduction latency, ablate SAE vs raw embeddings (−5.2 pp), sweep expansion factor, and show the expected small-object failure mode. Relative macro-average of ratios is a fair aggregator. The math is elementary graph + log-size merge; citations cover ToMe, PruMerge, VisionZip, PruneSID, and the Matryoshka BatchTopK SAE they freeze. No circularity.\n\nThe soft spot is real but bounded. The load-bearing dictionary is a public Matryoshka BatchTopK SAE trained only on CLS activations at block 22. Their own spatial-SAE ablation (two random patches) drops hard and widens with compression. Late-layer mixing is offered as justification, yet they never report monosemanticity or concept-coverage numbers for patch activations under the CLS SAE. So the 98.7/97.1 numbers could be partly dictionary-specific rather than a pure property of “concept overlap.” Extreme compression also loses to PruneSID, which they acknowledge. Free parameters (k, δ, ε, layer, B) are standard for this genre.\n\nThis is for people who already ship VLMs and care about serving cost, and for anyone watching interpretability tools move into functional decisions. The central moderate-regime claim holds under the stated setup; the transfer gap is the main thing a referee should press. I would send it to peer review. Engage if you work on token reduction or SAE reuse; otherwise skim the tables and the ablation.","headline":"Solid training-free token reduction that uses SAE concept overlap for content-adaptive prune/merge; moderate-regime gains look real, but rest on a CLS-trained dictionary whose patch transfer is only partially checked.","tokens_in":26256,"tokens_out":560,"would_cite":true,"duration_ms":5411,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Visual tokens can be reduced by grouping patches that share the same sparse-autoencoder concepts, adapting compression to each image without fine-tuning the VLM.","keywords":["token reduction","vision-language models","sparse autoencoders","concept overlap","visual tokens","adaptive compression","plug-and-play"],"falsifier":"Run the identical top-k overlap pipeline on raw patch embeddings instead of SAE latents at a matched average budget of roughly 47 tokens; if the reported multi-point accuracy gap vanishes, the SAE is not supplying the claimed semantic basis.","tokens_in":26227,"feed_emoji":"🧩","tokens_out":513,"duration_ms":14015,"temperature":0.7,"pith_summary":"Vision-language models are slowed by the hundreds of visual patch tokens they feed into the language model. Prior reduction methods score tokens by attention or raw similarity and never name what each patch means. TORINO projects every patch into a frozen sparse autoencoder concept space, links patches that share active latents, and then prunes or merges inside each concept group. The number of groups is set by the image itself, so plain scenes collapse more aggressively than busy ones. Across nine multimodal benchmarks the method keeps nearly full accuracy at moderate compression rates while remaining a pure inference-time plug-in.","feed_headline":"SAE concept groups shrink VLM visual tokens safely","feed_subtitle":"Patches that share active latents merge; simple images compress harder than complex ones, no fine-tuning.","key_machinery":"Concept overlap: two tokens are joined if they share at least δ of their top-k active SAE features; connected components of that graph become the groups that are then pruned or merged to a single primary token.","core_discovery":"Measuring agreement among active SAE latents produces semantically coherent groups of visual tokens; reducing each group to one representative (by peak activation or log-rescaled merge) removes redundancy more accurately than attention- or similarity-based baselines in the moderate-reduction regime, and the retained token count emerges automatically from image content rather than a fixed budget.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["TORINO groups VLM tokens by shared SAE concepts then prunes or merges","Concept overlap via SAE latents adapts visual token counts to image complexity","Shared active SAE latents form groups for redundancy-free VLM token reduction","TORINO reduces VLM visual tokens by concept-overlap pruning or merge","SAE concept agreement yields coherent token groups without fixed budgets"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The method assumes that a sparse autoencoder trained only on class-token activations still supplies faithful monosemantic concepts for grouping the encoder’s spatial patch tokens.","fun_headline_variants_meta":{"raw":{"variants":["TORINO groups VLM tokens by shared SAE concepts then prunes or merges","Concept overlap via SAE latents adapts visual token counts to image complexity","Shared active SAE latents form groups for redundancy-free VLM token reduction","TORINO reduces VLM visual tokens by concept-overlap pruning or merge","SAE concept agreement yields coherent token groups without fixed budgets"]},"model":"grok-4.5","effort":"low","cost_usd":0.005206,"raw_usage":{"total_tokens":1409,"prompt_tokens":764,"num_sources_used":0,"completion_tokens":97,"cost_in_usd_ticks":52060000,"prompt_tokens_details":{"text_tokens":764,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":548,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":764,"tokens_out":97,"duration_ms":5034,"temperature":1.0,"reasoning_tokens":548,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T16:43:29.756466+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the identical top-k overlap pipeline on raw patch embeddings instead of SAE latents at a matched average budget of roughly 47 tokens; if the reported multi-point accuracy gap vanishes, the SAE is not supplying the claimed semantic basis.","supporting_citations":[],"review_version":1}