{"id":"afcbb304-5a6b-48d4-b17f-f7863146f04d","arxiv_id":"2501.04329","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"EAC adaptively selects subsets of quantized latent features for multiple machine vision tasks and uses tiny per-task adapters, improving machine task accuracy at lower bitrates without retraining the base codec.","lead":"EAC is a compression add-on that sends only task-relevant parts of the neural code to machine vision networks while still allowing full reconstruction for humans. It is worth reading because learned compression is becoming a bottleneck for systems that store and analyze visual data at scale.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Independent binary mask predictors do not guarantee a partition of the latent, so the claimed bitrate savings and human-vision parity are not internally supported.","rationale":"The reader's weakest assumption concerns mean-filling of unselected latents. My concern is more specific and upstream: the masks themselves must form a partition for the rate and human-vision claims to hold. The paper never specifies a mechanism for this, and independent binary predictors with Gumbel-Softmax do not produce partitions. This is an internal-consistency issue, not a disagreement with community consensus. It is load-bearing because both the claimed bitrate saving and the 'similar to baseline' human PSNR depend on the union of masks equalling the full latent exactly once. The concern is testable by inspecting the predictor implementation or the conference version. If a partition is enforced, the central claims remain plausible; if not, the claims as written are not supported. The reader's medium risk is justified; the verdict stays CONDITIONAL pending this clarification.","tokens_in":27700,"tokens_out":6874,"duration_ms":71929,"concrete_test":"Check the released code or the conference version [25] for the mask-generation layer. Specifically, determine whether the n masks come from a single categorical Gumbel-Softmax over n choices (one-hot, sum=1) or from n independent binary Gumbel-Softmax predictors. If the latter, run the trained predictors on VOC2007 and compute the element-wise sum of all masks; if any entry is 0 or >1, the bitrate accounting and the human-vision parity claim are invalid. Also verify whether mask_n is defined as the complement of the union of masks 1..n-1; if not, the partition property is absent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that the n binary masks in Section III-A form a partition of the quantized latent y-hat, i.e., every element is selected by exactly one mask. This is required twice: (i) for the machine-vision bitrate, R(y-hat_i) for i=1..n must sum to R(y-hat) without duplicates; (ii) for human vision, summing all reconstructed subsets must recover the full y-hat, which the 'Human Vision Results' paragraph invokes to claim parity with the baseline codec. However, the paper describes each mask as produced by an independent predictor using a Gumbel-Softmax/Max module (Section III-C, Fig. 1), and the training losses in Eq. (3) and Eq. (4) contain no term enforcing disjointness or full coverage. Nothing prevents two predictors from selecting the same high-information latent (inflating rate) or all predictors from ignoring some element (degrading the human reconstruction below baseline despite the paper's assertion). The word 'partition' is asserted but not implemented in the described architecture, so the central claims of >33% bitrate savings and of maintaining human vision quality rest on an unstated and unenforced structural condition.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Efficient Adaptive Compression (EAC), a plug-in module for existing learned image and video codecs (Ballé2018, Cheng2020, DVC, FVC) to serve multiple downstream machine vision tasks (classification, segmentation, detection, action recognition, video object segmentation) while preserving human-viewing quality. The core idea is to partition the quantized latent into n subsets via binary masks generated from hyperprior statistics (and, for video, reference frames), transmit only the subsets needed by each task, and fill unselected entries with the predicted mean μ. A second contribution is a lightweight task-specific adapter, trained with the task network frozen, that adapts the reconstructed frame for each task. Training is two-stage: predictors are trained with rate–distortion–task losses, then adapters are trained alone. Experiments on VOC, COCO, ILSVRC2012, UCF101, and DAVIS report bitrate savings over the base codecs plus task-network pipelines and claim parity with the base codecs for human viewing.","tokens_in":27911,"tokens_out":9271,"duration_ms":80925,"significance":"If the results hold, EAC offers a simple and broadly applicable way to redirect established compression architectures toward machine tasks, with a very small parameter overhead (adapter ~0.2% of the task network) and a clean separation of codec and task-network tuning. The adapter-based delta-tuning is a sensible, efficient strategy. However, the central mechanism's correctness depends on an unstated and unenforced partition property of the learned masks, and the experimental evidence is presented entirely through unreadable RD subfigures without tables, error bars, or code release. The claimed 33% bitrate savings and human-vision parity cannot currently be verified. The idea is promising and the architectural modifications are local, so the paper is a candidate for major revision rather than rejection.","major_comments":[{"comment":"The paper describes the adaptive compression module as partitioning the quantized latent ŷ into n subsets via binary masks, and claims that for human vision 'we add all reconstructed latent features ... obtaining full latent feature ŷ' (Section III-A). This is only correct if the masks are a partition (disjoint and covering). However, each mask is produced by an independent predictor (with Gumbel-Softmax/Max) and the losses in Eqs. (3)–(4) contain no term enforcing disjointness or full coverage. Nothing prevents two predictors from selecting the same latent elements (inflating total rate) or all predictors from omitting some elements (degrading the human reconstruction). Please add an explicit partition constraint (e.g., a single softmax over n+1 classes, or a regularizer) and, in the experiments, report the overlap/coverage statistics of the learned masks.","section":"Section III-A, Fig. 1, Eqs. (3)–(4)"},{"comment":"The paper does not state how the binary masks are encoded and transmitted. If the masks are regenerated at the decoder, the predictor must be deterministic and the paper should say so; if they are encoded, their bitrate must be added to the reported savings. In addition, because the masks may overlap, the sum R(ŷ1)+...+R(ŷn) is not guaranteed to equal R(ŷ), so the claimed bitrate savings are not well defined. Please clarify the mask transmission scheme and provide a breakdown of bitrate: mask overhead versus latent subvectors.","section":"Section III-A, Section IV-C"},{"comment":"The central quantitative claims, including 'saves more than 33% bit-rate' at 0.79 mIoU and at 0.67 mAP@0.5, are supported only by RD subfigures with unreadable labels and no numerical tables. No error bars, standard deviations, or number of runs are given, and no code or data are released. The 33% figure cannot be checked from the text. Please add tables with the actual bpp, task-metric, and PSNR values for at least the key operating points, and report variance or standard deviations across at least three runs.","section":"Section IV-C, Fig. 5"},{"comment":"The claim that human-vision performance is maintained is circular: it asserts that because the same pre-trained codec and the 'full latent feature' are used, performance is similar to the baseline. This is only true under the partition assumption of Comment 1, and even then, the full-latent reconstruction would be identical to the baseline only if the masks perfectly reconstruct ŷ, which is not guaranteed given that unselected elements are filled with μ for the machine branches. Moreover, no numerical PSNR values are reported; the paper should compare PSNR (or MS-SSIM) of 'Ours' and the baseline at matched bitrates.","section":"Section IV-C, 'Human Vision Results'"},{"comment":"The paper states that a cumulative objective over T frames mitigates error propagation caused by transmitting only subsets of motion and residual latents in video. No ablation is provided: there is no experiment varying T, no comparison with single-frame training, and no measurement of reconstruction drift over time. Please either add evidence for this claim or soften the claim to what the experiments actually support.","section":"Section III-E, Eq. (4)"}],"minor_comments":[{"comment":"'It is worse noting' should be 'It is worth noting' (Section IV-C). Section III-C misspells 'predictor' as 'Preditor'; Section III-D misspells 'reconstructed' as 'reconsturcted'; Section III-C has 'hyperperior' for 'hyperprior'.","section":"Section IV-C"},{"comment":"The figure captions contain garbled Unicode tokens (e.g., '/uni00000013/...') that obscure the figure content; these artifacts must be cleaned in the final version.","section":"Figures 5–8"},{"comment":"Eq. (4) has an unbalanced parenthesis in the displayed formula; please fix the typesetting.","section":"Eq. (4)"},{"comment":"The notation is inconsistent: 'Ballé2018' and 'Cheng2020' are used both as abbreviations and as reference names (e.g., in captions and text). Use 'Ballé et al. [15]' and 'Cheng et al. [16]' consistently.","section":"Notation"},{"comment":"The paper claims to outperform VTM 'across most bit-rates' but does not specify the bitrate range or whether VTM is used in intra-only mode for image comparison; please state the exact configuration.","section":"Section IV-B"},{"comment":"Table I gives parameter counts but does not specify what is included in 'Adapter in EAC (NIC)' and 'Adapter in EAC (NVC)'; please clarify the adapter architecture and parameter count.","section":"Table I"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an extension of the authors' conference paper (ICMH-Net). Most of the comparisons are against the authors' own prior work, and the only external baselines are VTM, BPG, and x.265. The partition issue and lack of numerical tables are the main technical blockers. If the authors can fix the partition mechanism and provide transparent experimental numbers, the paper could be suitable for this journal. The lack of code and data is a concern for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper extends the authors' earlier ICMH-Net from a single machine-vision task on images to multiple tasks (classification, segmentation, detection, action recognition, video segmentation) on both images and video, and adds a parameter-efficient adapter so the task network stays frozen. That combination is genuinely new, and the adapter idea is sensible: the parameter counts in Table I are striking, and the ablations in Fig. 8 suggest both the adaptive-selection module and the adapter contribute. The scope of experiments, across four codecs and five task networks, is appropriate for a systems paper. If the results are real, this is a useful drop-in module for practical analytics pipelines.\n\nThat \"if\" is where the soft spots are. The most serious one is structural: the paper says the latent is \"partitioned\" into n subsets by binary masks, and for human vision the subsets are summed to recover the full latent. But each mask is produced by an independent predictor with a Gumbel-Softmax/Max module, and the losses in Eq. (3) and (4) contain no term enforcing disjointness or full coverage. Nothing stops two masks from selecting the same element, or all masks from ignoring some element. If masks overlap, the bitrate for multiple tasks includes duplicate transmissions, which is not accounted for in the reported savings. If masks do not cover the full latent, summing them does not recover the original ŷ, so the claim that human vision is maintained by \"adding all reconstructed latent features\" is not internally supported. The human-vision section later says they \"use the full latent feature for human vision,\" which suggests they may transmit the entire ŷ directly rather than via subsets; if so, the partition issue does not affect human parity, but the text in Section III-A contradicts that reading. This needs to be clarified or fixed.\n\nThe other weaknesses are more conventional. All quantitative claims appear only as RD curves; there are no numeric tables, no error bars, and no code or data. The 33% bitrate savings figure cannot be checked from the text. Key implementation details (the value of n, λ, the predictor architecture, T in Eq. (4)) are deferred to the conference version, which is not self-contained. The conclusion claims superiority over DeepSVC, but no DeepSVC comparison appears in the experiments. These are fixable in revision but currently leave the central empirical claims unverifiable.\n\nI would send this to peer review. The idea is plausible, the evaluation is broad, and the adapter mechanism is a nice contribution. But I would make the authors enforce or explicitly justify the partition property, provide numeric tables and error bars, and release code. A referee should not have to take the claimed savings on faith.","headline":"A plausible incremental extension of the authors' ICMH-Net to multi-task and video settings, but the described architecture doesn't actually enforce the latent partition that the bitrate-savings and human-parity claims rest on.","tokens_in":28446,"tokens_out":4010,"would_cite":true,"duration_ms":39904,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a lightweight add-on, EAC, can partition a neural codec's latent features into task-specific subsets so that multiple machine vision tasks run at over 33% lower bitrate while human-viewable reconstruction quality is…","keywords":["neural image compression","neural video compression","machine vision","latent subset selection","binary masks","task-specific adapter","parameter-efficient fine-tuning","human-machine vision"],"falsifier":"Take a trained EAC (NIC) model and replace its learned binary masks with random binary masks of the same density on a detection benchmark: if mAP@0.5 stays within a small margin of the reported level, then the mask predictor is not the source of the task accuracy. Alternatively, replace the μ-filling of unselected latent entries with zeros: if task accuracy is unchanged, the aggregation step's dependence on the hyperprior mean is not load-bearing.","tokens_in":27450,"feed_emoji":"🧠","tokens_out":4963,"duration_ms":44540,"temperature":0.7,"pith_summary":"The paper proposes Efficient Adaptive Compression (EAC), a lightweight add-on that makes existing neural image and video codecs serve both human viewers and multiple machine vision tasks at once. The core claim is that a codec's quantized latent features can be partitioned, by learned binary masks, into task-specific subsets, so that each machine task receives only the bits it needs while human vision can still request the full latent. The authors report that this saves more than 33% bitrate over Ballé2018-based baselines at matched segmentation and detection accuracy, and that even with classic neural codecs the results beat the modern standard codec VTM across most bitrates. A second component, a task-specific adapter trained with parameter-efficient delta-tuning, lets downstream networks (classification, segmentation, detection, action recognition, video object segmentation) be adapted cheaply without retraining them. If correct, existing learned codecs can be upgraded for machine-centric pipelines without redesigning them.","feed_headline":"One module lets codecs serve machines with 33% fewer bits","feed_subtitle":"EAC selects task-relevant latent subsets so existing image and video codecs keep human quality while feeding segmentation and detection.","key_machinery":"The adaptive compression module: it multiplies the quantized latent ŷ elementwise by n binary masks (generated from hyperprior μ, σ and, in video, reference-frame features via Gumbel-softmax), losslessly transmits only the selected subsets, then reconstructs and aggregates them, filling unselected elements with the predicted mean μ to form task-specific latents. The task-specific adapter is a small residual-block network inserted before a frozen task network; it projects the reconstructed image or frame (plus, for video, several reconstructed reference frames) into a feature map that nudges the task network toward the correct output. These two pieces together carry the argument: mask selection buys the bitrate savings, and μ-filling plus the adapter preserves task accuracy.","core_discovery":"EAC claims that the information a downstream vision task needs is a subset of the quantized latent, and that subset can be identified from the hyperprior statistics (μ, σ) — plus, for video, from reference frames — by a small predictor network that outputs binary masks. The selected subset is transmitted and decoded; the unselected positions are filled with the predicted mean μ before reconstruction. For the i-th task, reconstructed subsets are summed and unselected entries filled with μ to yield a task-specific reconstruction. The full latent is transmitted only when a high-quality human-viewable reconstruction is required. With codec weights frozen, only the masks and a tiny adapter are trained, so the scheme is a drop-in extension of Ballé2018, Cheng2020, DVC, and FVC.","pith_inferences":["The μ-filling trick suggests a general recipe for any autoencoder with a tractable prior: unselected latent entries can be replaced by their conditional mean at no extra bit cost, and testing it on other modalities (audio, point clouds, medical images) would show whether the recipe is codec-specific.","The mask predictor learns a task-relevance map over latent channels and positions; equipping it with uncertainty estimates could let a system decide per-image whether to request the full latent, an implicit rate-control policy the paper does not explore.","A natural stress test is distribution shift: since masks are predicted from hyperprior statistics, images whose statistics differ from training (e.g., night scenes, medical scans) may need more subsets or a fallback to full transmission, a condition the paper does not report.","The comparison to VTM is at the system level (codec plus frozen task network); a more complete machine-vision benchmark would tune VTM's bitrate allocation for task features, a direction the paper leaves open."],"forward_implications":["Any existing neural codec with a hyperprior (e.g., Ballé2018, Cheng2020) can be wrapped by EAC to serve multiple machine tasks without retraining the codec, so the reported savings should transfer to newer codecs that expose latent statistics.","Machine-vision clients can operate at a fraction of the bitrate by receiving only their task's subset; the same bitstream can still be upgraded to a full human-quality reconstruction when needed, enabling progressive transmission.","For video, using reference frames as additional mask input improves the selection, so temporal redundancy is exploited for machine tasks the same way it is for human viewing.","Because only the adapter (about 0.2M parameters) is optimized, adapting a codec to a new task costs orders of magnitude less than full fine-tuning of the task network.","The two-stage training (predictors first, then adapter) lets the mask and the adapter be optimized separately, so the scheme is compatible with frozen codecs and frozen task networks."],"supporting_citations":[{"why":"Ballé2018 scale-hyperprior NIC supplies the hyperprior statistics (μ, σ) and the pre-trained image codec that EAC wraps for image tasks.","marker":"[15]"},{"why":"Cheng2020 attention-based NIC is the second image codec EAC integrates with, providing a stronger baseline and latent structure.","marker":"[16]"},{"why":"DVC, the first end-to-end NVC, is one of the two video backbones EAC extends to machine tasks.","marker":"[17]"},{"why":"FVC, the feature-space video codec, is the other NVC backbone EAC integrates with.","marker":"[18]"},{"why":"ICMH-Net, the authors' conference version, contributes the predictor design for binary-mask generation that EAC generalizes to multiple tasks.","marker":"[25]"},{"why":"The parameter-efficient delta-tuning survey motivates the adapter strategy EAC adopts, keeping the task network frozen while training only a small adapter.","marker":"[14]"}],"fun_headline_variants":["Adaptive codec picks latent subsets for machine tasks","Drop-in module lets codecs serve machine vision without retraining","EAC: task-specific latent selection for human and machine vision","Frozen codec weights, learned masks: adaptive compression for all"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme rests on the assumption that the information a downstream vision task needs lives in a subset of the codec's latent features that can be predicted from the hyperprior statistics (plus reference frames in video), and that filling the discarded positions with the average value μ leaves reconstructions accurate enough for the task.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive codec picks latent subsets for machine tasks","Drop-in module lets codecs serve machine vision without retraining","EAC: task-specific latent selection for human and machine vision","Frozen codec weights, learned masks: adaptive compression for all"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00105,"raw_usage":{"total_tokens":4422,"prompt_tokens":965,"completion_tokens":3457,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":3387}},"tokens_in":581,"tokens_out":3457,"duration_ms":28151,"temperature":1.0,"reasoning_tokens":3387,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:35:26.842551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained EAC (NIC) model and replace its learned binary masks with random binary masks of the same density on a detection benchmark: if mAP@0.5 stays within a small margin of the reported level, then the mask predictor is not the source of the task accuracy. Alternatively, replace the μ-filling of unselected latent entries with zeros: if task accuracy is unchanged, the aggregation step's dependence on the hyperprior mean is not load-bearing.","supporting_citations":[{"cited_title":"Variational image compression with a scale hyperprior,","cited_arxiv_id":null,"evidence_quote":"Ballé2018 scale-hyperprior NIC supplies the hyperprior statistics (μ, σ) and the pre-trained image codec that EAC wraps for image tasks."},{"cited_title":"Learned image com- pression with discretized gaussian mixture likelihoods and attention modules,","cited_arxiv_id":null,"evidence_quote":"Cheng2020 attention-based NIC is the second image codec EAC integrates with, providing a stronger baseline and latent structure."},{"cited_title":"Dvc: An end-to-end deep video compression framework,","cited_arxiv_id":null,"evidence_quote":"DVC, the first end-to-end NVC, is one of the two video backbones EAC extends to machine tasks."},{"cited_title":"Fvc: A new framework towards deep video compression in feature space,","cited_arxiv_id":null,"evidence_quote":"FVC, the feature-space video codec, is the other NVC backbone EAC integrates with."},{"cited_title":"Icmh-net: Neural image compres- sion towards both machine vision and human vision,","cited_arxiv_id":null,"evidence_quote":"ICMH-Net, the authors' conference version, contributes the predictor design for binary-mask generation that EAC generalizes to multiple tasks."},{"cited_title":"Parameter-efficient fine-tuning of large- scale pre-trained language models,","cited_arxiv_id":null,"evidence_quote":"The parameter-efficient delta-tuning survey motivates the adapter strategy EAC adopts, keeping the task network frozen while training only a small adapter."}],"review_version":1}