{"id":"e877f853-3698-414f-a070-b81208754ec9","arxiv_id":"2608.08832","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"The Visual Token Codec compresses ViT intermediate features by entropy-coding patch tokens on their native grid instead of a flattened sequence, reducing bitrate by 15.7x to 37.4x at 90% of uncompressed performance over a VTM-based baseline.","lead":"A new compression codec for Vision Transformer features splits the few global tokens from the many patch tokens and encodes patch tokens on their original 2D grid, capturing spatial redundancy that flattened token sequences miss. On DINOv2 and SAM3, it needs far less bitrate to preserve downstream accuracy than a standard video-codec baseline, which matters for edge-cloud deployment of large vision models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table II's 15.7x-37.4x multipliers are computed at a 90%-of-bypass operating point adopted only because BD metrics failed; for DINOv2 Cls the VTC entry is an inequality, so the headline magnitudes are not robust.","rationale":"The reader identified the same weakness: the 90%-of-bypass comparison is a fallback for failed BD metrics, and the DINOv2 Cls entry is not a measured crossing. I agree. This is the single most load-bearing concern because the paper's headline contribution is a quantitative bitrate reduction; the architectural mechanism is separately validated by its layout ablation, so the mechanism itself is not in doubt. Without a stable evaluation protocol, the exact '15.7x-37.4x' claim cannot be verified from the reported data. The concern is not fatal: VTC's gains are visible across tasks and the ablation supports the mechanism. It argues for keeping the verdict at CONDITIONAL rather than ACCEPT, and for the authors to provide denser rate points, error bars, and outside baselines. Because this matches the reader's assessment and does not change the recommended conditional verdict, verdict_should_be is UNCHANGED.","tokens_in":19957,"tokens_out":6293,"duration_ms":67896,"concrete_test":"Re-run the task-specific evaluation with a denser, common set of rate points for VTC and LaMoFC-VTM, including QP/lambda values below the current minimum, so that BD-Rate/BD-Acc can be computed over an overlapping range. Then recompute the reported multipliers at 90%, 95%, and 99% of bypass performance for all four tasks. If the relative gains remain within a small band of the Table II values and BD metrics agree with the ordering, the concern is resolved; if the multipliers move substantially or the ordering changes, the headline should be rephrased as threshold-dependent rather than a single 15.7x-37.4x range.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central qualitative claim is well supported: the layout ablation in Fig. 9 shows dual-path coding beats all-1D and all-2D arrangements under fixed transforms, and the rate curves in Fig. 5 show VTC dominating the baselines. The load-bearing weak spot is the quantitative headline in Sec. IV-B1 and Table II. The paper states that BD metrics cannot be computed reliably because the curves have limited overlap, then switches to 'bitrate needed to reach 90% of bypass.' That is a defensible fallback, but it makes the 15.7x-37.4x numbers protocol-dependent. In particular, the DINOv2 Cls entry is '≤0.013 bpfp': the rate-accuracy curve is nearly horizontal there, so 0.013 is the lowest tested rate rather than a measured crossing at the 90% target. Changing the threshold to 95% or 99%, or adding denser low-rate points, could shift the multipliers substantially. Since all compared baselines are from the same group and no error bars are reported, the exact magnitude of the gain is the least secure part of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Visual Token Codec (VTC), a learned codec for compressing intermediate ViT features in split-inference and feature-storage settings. The key design is a dual-path architecture that codes the few global tokens with a lightweight factorized prior and codes the patch tokens on their native 2D grid with a spatial-channel context entropy model (SCCTX from ELIC), instead of flattening all tokens into a pseudo-image. VTC also adds feature-matching supervision on deeper ViT blocks and a variable-rate mechanism. The authors evaluate on DINOv2 (classification, semantic segmentation, reconstruction) and SAM3 (object detection, instance segmentation) and report that at 90% of uncompressed-feature performance, VTC reduces bitrate by 15.7x–37.4x relative to LaMoFC-VTM. They additionally provide intermediate-layer rate-distortion trade-offs, a backbone-scale study, and ablations showing that the dual-path layout outperforms all-1D and all-2D layouts under fixed transforms.","tokens_in":20221,"tokens_out":3706,"duration_ms":43154,"significance":"If the quantitative claims hold, the paper makes a useful contribution to ViT feature coding: it identifies a structural prior (patch-token spatial locality) that prior pseudo-image codecs miss, and it validates the design with extensive experiments on two modern backbones and five tasks, using substantially larger test sets than earlier feature-coding studies. The ablation in Fig. 9 is well designed and supports the qualitative central claim that layout matters independently of transform capacity. The intermediate-layer deployment analysis in Sec. IV-C and Table III is a valuable addition for practical bandwidth/computation trade-offs. The manuscript is also transparent about where BD metrics could not be computed. However, the headline bitrate-reduction magnitudes rest on a post-hoc operating point and on single-point ratios without uncertainty quantification, so the exact multipliers are less secure than the qualitative ranking.","major_comments":[{"comment":"The headline 15.7x–37.4x multipliers are computed at a 90%-of-bypass operating point adopted only after BD metrics were abandoned because the rate curves have limited overlap. For the DINOv2 Cls row, Table II reports VTC as '≤0.013 bpfp', which is the lowest tested rate on a nearly horizontal accuracy curve rather than a measured crossing at the 90% target. This makes the claimed multipliers protocol-dependent: the DINOv2 Cls entry is an inequality, and if a 95% or 99% threshold were used, or if denser low-rate points were added, the reported gains could shift substantially. Please report the DINOv2 Cls entry explicitly as an inequality, either exclude it from the headline range or provide denser low-rate measurements, and state that the remaining multipliers are single-point estimates rather than BD-derived averages.","section":"Sec. IV-B1, Table II"},{"comment":"All compared baselines (LaMoFC-VTM, VQFC, and the uncompressed bypass) come from the same research group or the authors' previous work, and no uncertainty is reported for any rate-performance point. Since the quantitative magnitude is the main contribution beyond the qualitative ablation, the paper should report error bars or multiple evaluation runs for the Table II bitrate estimates, and ideally include an independent off-the-shelf learned image codec (for example, applying ELIC to the flattened pseudo-image layout) as an additional baseline. Such a baseline would also directly test whether the gains come from the layout prior rather than from codec choice.","section":"Sec. IV-A4 and Sec. IV-B1"},{"comment":"The intermediate-layer analysis is informative, but the BD metrics in Table III are computed relative to a single hand-selected operating point (Input=9, Spv=12, fa=fs=2), and rows with '–' in BD-rate are not explained. Because the choice of reference row substantially determines the reported BD-Rate percentages, the paper should clarify how sensitive the conclusions are to this reference choice and should report the full rate-performance curves for the key rows, not only in the appendix text but as accessible data, so that readers can compare operating points directly.","section":"Sec. IV-C, Table III"}],"minor_comments":[{"comment":"The 1:1 weighting between global-token and patch-token distortion is described as an empirical choice, but no ablation or sensitivity study is provided; a small experiment varying this weighting would strengthen the claim that the choice is robust.","section":"Sec. III-E"},{"comment":"The notation '≤0.013' in the DINOv2 Cls row should be explained in the caption, since it denotes an upper bound from the lowest tested rate rather than a measured crossing point.","section":"Table II caption"},{"comment":"Rows 4–5 are first discussed in Sec. IV-C but are later used as codec-transform ablations in Sec. IV-E; adding a forward pointer in the Table III discussion would avoid confusion.","section":"Table III"},{"comment":"The text states that ImageNet BD-rate is omitted because the curves are nearly horizontal; please clarify whether BD-rate is undefined or simply not reported, and whether the flatness is due to the accuracy range or the rate range.","section":"Table IV"},{"comment":"The all-1D, all-2D, and dual-path curves are visually close at low bitrates; consider reporting the BD-mIoU or BD-rate differences between the three layouts numerically to make the ablation comparison easier to assess.","section":"Fig. 9"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest about the limitations of its evaluation protocol, and the central qualitative claim is well supported by the layout ablation. My main concern is that the headline 15.7x–37.4x numbers are presented as a firm result even though they rest on a post-hoc 90% threshold and on single-point ratios without error bars, and all learned baselines share authors with this paper. I would ask the authors to densify the low-rate points, report the DINOv2 Cls entry as an inequality, and add at least one independent baseline before reconsidering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is straightforward and, I think, correct: ViT patch tokens live on a 2D grid with real spatial structure, and flattening them into a sequence for entropy coding throws that away. Separating global tokens (CLS/REG) onto a lightweight factorized path and coding patch tokens on their native grid with a spatial-channel context model is a sensible, natural design. The controlled ablation in Fig. 9 is the best evidence—with transforms fixed, dual-path beats all-1D and all-2D by a clear margin. That is the real contribution, and it holds up.\n\nWhat else is good: the paper is thorough. It covers four downstream tasks across DINOv2-S/B/L and SAM3, includes reconstruction and intermediate-layer analyses, reports FLOPs for transmission- vs storage-oriented splits, and gives wall-clock timings. The authors are also honest about their own measurement problems—they explicitly say BD metrics are unreliable due to limited curve overlap, and they report the DINOv2 classification entry as an inequality (≤0.013 bpfp). That transparency earns credit.\n\nNow the soft spots, in proportion. The 15.7x–37.4x multipliers are the headline, but they are computed at a 90%-of-bypass threshold that was adopted because the curves didn't overlap enough for standard BD metrics. For DINOv2 classification the rate-accuracy curve is nearly flat, so VTC's table entry is the lowest tested rate, not a measured crossing at 90% of bypass. Change the threshold to 95% or add denser low-rate points and those multipliers will move. All compared baselines come from the same research group, there are no error bars, and no code is released. None of this undermines the qualitative mechanism, but it does mean the exact magnitude of the gain is the least secure part of the paper.\n\nOne more point: the 1:1 weighting between global and patch MSE is acknowledged as a choice, not derived. That's fine, but it's a free parameter that could matter for classification-heavy vs segmentation-heavy transfer.\n\nWho is this for? Anyone working on coding for machines, feature compression, or distributed inference of vision transformers. The paper would benefit from a revision that strengthens the operating-point protocol, adds at least one independent baseline (ideally from another group), and releases code. I'd send it to peer review—the central claim is new, credible, and well-supported by the ablation, even if the headline numbers need to be presented with more caution.","headline":"The dual-path split of global vs patch tokens is a clean, well-validated idea, but the headline bitrate multipliers rest on a fragile operating point chosen after BD metrics failed.","tokens_in":20806,"tokens_out":1690,"would_cite":true,"duration_ms":18630,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that VTC, a dual-path token codec, cuts the bitrate needed to preserve 90 percent of uncompressed ViT feature performance by 15.7x–37.4x compared with a VTM-based baseline.","keywords":["visual token codec","ViT feature coding","spatial-channel context entropy model","patch-token grid","feature-matching supervision","variable-rate coding","DINOv2","SAM3"],"falsifier":"Run both codecs at many more rate points on DINOv2 classification and find the actual bitrate at which each reaches 90 percent of bypass accuracy; if VTC's required bitrate is above 0.013 bits per feature point, or the gap to LaMoFC-VTM is smaller than 28.4x, the headline saving does not hold. As a second check, retrain the flattened-token layout with the same nonlinear transforms and feature-matching loss; if it matches VTC's curve, the 2D grid context is not the source of the gains.","tokens_in":19735,"feed_emoji":"🧩","tokens_out":10296,"duration_ms":95020,"temperature":0.7,"pith_summary":"The paper argues that when a Vision Transformer is split across devices, the intermediate token features exchanged between nodes can be compressed much more aggressively than current methods do, because patch tokens still sit on their original 2D image grid and retain strong local spatial correlation. Existing ViT feature codecs flatten all tokens into an $L\\times C$ pseudo image, so their entropy models mostly exploit sequence-axis dependence and leave grid redundancy untouched. VTC instead routes a few global summary tokens through a lightweight factorized 1D codec and codes the many patch tokens on their native $H\\times W$ grid with a spatial–channel context entropy model. The paper reports that at 90% of uncompressed-feature (bypass) performance, VTC needs 15.7x–37.4x less bitrate than the LaMoFC-VTM baseline across SAM3 detection and instance segmentation and DINOv2 classification and semantic segmentation. A feature-matching loss and a variable-rate module make the codec usable at intermediate layers and over 65 rate points in a single model.","feed_headline":"Patch-grid codec keeps 90% of ViT performance at 37.4x lower bitrate","feed_subtitle":"Separate paths for global and patch tokens exploit 2D spatial structure that flattened layouts discard.","key_machinery":"The central object is the dual-path token layout: a short 1D path for global summary tokens and a 2D $H\\times W$ grid path for patch tokens. The grid path uses a Spatial-Channel Context (SCCTX) entropy model, which partitions the latent into channel groups and checkerboard spatial positions, then predicts conditional Gaussian parameters for each position from hyperprior side information and from previously decoded spatial and channel contexts; this is the component that harvests the local 2D redundancy. Two auxiliary mechanisms carry the deployment claims: feature-matching supervision, which pushes reconstructed tokens through the next $j$ frozen ViT blocks and adds mean-squared error on those deeper features to the rate–distortion loss, and a variable-rate module bank that conditions the codec on a quantization parameter selecting one of 65 Lagrange multipliers, letting one model span multiple bitrates.","core_discovery":"The central claim is that ViT patch tokens carry exploitable local spatial correlation on their original 2D grid, and the right codec design is to stop treating all tokens as one flattened sequence. VTC's discovery is the dual-path layout: global tokens are coded with a factorized prior on a 1D path, while patch tokens are reshaped back to the $H\\times W$ grid and coded by a spatial–channel context entropy model that predicts each quantized latent from hyperprior side information plus already-decoded spatial and channel neighbors. Because this layout matches the entropy model to the token statistics, VTC reaches 90% of bypass performance at 0.009–0.013 bits per feature point where the LaMoFC-VTM baseline needs 0.164–0.498, giving the 15.7x–37.4x savings. In the multi-task setting, one compressed bitstream serves both segmentation and reconstruction, and compressing layer 9 with feature-matching supervision at layer 12 is the best tested rate–distortion operating point for ADE20K segmentation. The paper also demonstrates that larger backbones are not automatically better under a pixel-normalized bitrate metric.","pith_inferences":["If the patch-grid prior is the main source of savings, similar dual-path layouts should transfer to other dense-grid ViT features beyond DINOv2 and SAM3; that is testable by running VTC's token split on such features without retraining the codec.","The flat DINOv2 classification rate–accuracy curve hints that a classification-only codec could spend nearly all bits on global tokens and almost nothing on patch detail, which would make dense tasks the real constraint on bitrate.","Because the supervision layer shapes the rate–distortion trend more than the compressed layer does, a practical design rule would be to pick the feature-matching layer according to the downstream task head rather than the transmission point.","The reported multipliers are measured against a VTM-based baseline; an independent learned baseline that also flattens tokens but uses the same feature-matching supervision would isolate how much of the gain is the 2D layout itself."],"forward_implications":["At 90% of uncompressed-feature performance, VTC needs 0.009–0.013 bits per feature point versus 0.164–0.498 for LaMoFC-VTM, translating to 15.7x–37.4x lower bitrate across the four task-specific settings.","Dual-path coding beats both flattening all tokens into one sequence and padding global tokens into the 2D grid; removing the codec's nonlinear transform blocks collapses performance by 15.51 BD-mIoU in the ablation.","Feature-matching supervision largely determines the rate–distortion behavior: curves sharing the same supervision layer follow similar trajectories, and compressing layer 9 with supervision at layer 12 is the best ADE20K segmentation operating point among the tested configurations.","A single VTC model with a QP-conditioned module bank covers 65 rate points, so rate adaptation does not require retraining per bitrate.","A single multi-task bitstream from DINOv2-B/16-reg4 supports both semantic segmentation and subjective reconstruction; at LPIPS=0.484 VTC uses 17.7x less bitrate than LaMoFC-VTM."],"supporting_citations":[{"why":"Supplies the Spatial-Channel Context (SCCTX) entropy model that codes patch tokens on the 2D grid.","marker":"[13]"},{"why":"Supplies the factorized prior and scale hyperprior entropy models used for the global-token path and the patch hyperprior.","marker":"[12]"},{"why":"Defines the LaMoFC benchmark and the VTM-based baseline that VTC's bitrate savings are measured against, plus the bits-per-feature-point metric.","marker":"[7]"},{"why":"Provides the DINOv2 backbones and register-enhanced features used for classification, segmentation, and reconstruction experiments.","marker":"[2]"},{"why":"Provides the SAM3 image encoder whose tokens are compressed for detection and instance segmentation.","marker":"[5]"},{"why":"Motivates the separate 1D path by explaining the global and register tokens that carry non-patch semantics.","marker":"[8]"},{"why":"Is the VTM codec used inside the LaMoFC-VTM baseline that compresses flattened token pseudo-images.","marker":"[11]"},{"why":"Is the VQFC baseline whose weak-spatial-redundancy assumption VTC directly counters.","marker":"[10]"}],"fun_headline_variants":["Spatial-aware codec hits 37x bitrate cut for ViT features","ViT token codec exploits 2D grid for 15-37x bitrate savings","Split global/patch paths shave 37x bitrate off ViT features","Patch-grid coding delivers 90% ViT quality at 37x less bits","Dual-path codec maps tokens to grid, cutting bits 15-37x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline savings rest on the assumption that the comparison point—the bitrate at which a codec reaches 90 percent of the uncompressed feature's task performance—is a well-defined and fair measure on every task, and on DINOv2 classification the accuracy curve is nearly flat with bitrate, so the reported figure is an upper bound at the lowest tested rate rather than a directly measured crossover.","fun_headline_variants_meta":{"raw":{"variants":["Spatial-aware codec hits 37x bitrate cut for ViT features","ViT token codec exploits 2D grid for 15-37x bitrate savings","Split global/patch paths shave 37x bitrate off ViT features","Patch-grid coding delivers 90% ViT quality at 37x less bits","Dual-path codec maps tokens to grid, cutting bits 15-37x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00029,"raw_usage":{"total_tokens":1745,"prompt_tokens":1040,"completion_tokens":705,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":594}},"tokens_in":656,"tokens_out":705,"duration_ms":7179,"temperature":1.0,"reasoning_tokens":594,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:23:39.398417+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both codecs at many more rate points on DINOv2 classification and find the actual bitrate at which each reaches 90 percent of bypass accuracy; if VTC's required bitrate is above 0.013 bits per feature point, or the gap to LaMoFC-VTM is smaller than 28.4x, the headline saving does not hold. As a second check, retrain the flattened-token layout with the same nonlinear transforms and feature-matching loss; if it matches VTC's curve, the 2D grid context is not the source of the gains.","supporting_citations":[{"cited_title":"ELIC: Efficient Learned Image Compression with Unevenly Grouped Space- Channel Contextual Adaptive Coding,","cited_arxiv_id":null,"evidence_quote":"Supplies the Spatial-Channel Context (SCCTX) entropy model that codes patch tokens on the 2D grid."},{"cited_title":"Vision transformers need registers,","cited_arxiv_id":null,"evidence_quote":"Motivates the separate 1D path by explaining the global and register tokens that carry non-patch semantics."},{"cited_title":"VVC and VTM 11.0: The Versatile Video Coding standard and its reference software,","cited_arxiv_id":null,"evidence_quote":"Is the VTM codec used inside the LaMoFC-VTM baseline that compresses flattened token pseudo-images."},{"cited_title":"Transform-Free Feature Coding via Entropy-Constrained Vector Quantization,","cited_arxiv_id":null,"evidence_quote":"Is the VQFC baseline whose weak-spatial-redundancy assumption VTC directly counters."}],"review_version":1}