{"id":"1a503d4c-a811-4b4f-a3da-ea285a32bfcf","arxiv_id":"2412.03841","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"LL-ICM jointly trains a neural image codec and a vision-language-guided diffusion restorer, claiming large bit-rate savings for low-level machine vision tasks.","lead":"This paper introduces LL-ICM, a learned image codec trained together with a multi-task image restoration network so a single compressed stream can support tasks like deraining, dehazing, and denoising. It reports large bit-rate savings at equal perceptual quality, but the evaluation compares against baselines whose restoration network was not retrained, so the headline gains are inflated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BD-rate gain conflates codec and downstream adaptation: LL-ICM's IR-SDE is fine-tuned jointly while baselines use off-the-shelf IR-SDE, so the claimed 22.65% advantage may not reflect codec performance.","rationale":"The paper's proposed RPO formulation and joint-training idea are interesting, and the use of VLM features is plausible. However, the only quantitative evidence for the central claim is the comparison in Table 2. The asymmetry in downstream adaptation is not a minor implementation detail: it directly changes the object being measured. A reviewer cannot tell from the present data whether LL-ICM's codec is better than MLIC at preserving information useful for restoration, or whether the fine-tuned IR-SDE is simply more tolerant of LL-ICM's particular artifacts. Because the abstract states a single aggregate BD-rate number without the per-task context and no code/weights are provided, the risk of the conclusion being an artifact is high. The proposed test—matching downstream fine-tuning across all codecs—directly separates these explanations. If the test favors LL-ICM, the paper's contribution stands; if not, the contribution reduces to 'a jointly fine-tuned restoration model helps,' which is a different and much weaker claim. Given the reader already identified this issue and recommended conditional acceptance, my stress test does not change the verdict; it reinforces the need for the matched-baseline experiment before the work can be judged correct.","tokens_in":8367,"tokens_out":3728,"duration_ms":86251,"concrete_test":"Fine-tune the same IR-SDE (with the same DA-CLIP conditioning and training data as in stage 2) separately on compressed outputs of each baseline codec—Balle2018, Cheng2020, ECM, and MLIC—using identical loss weights and training iterations, then recompute the BD-rate, BD-LPIPS, and BD-QAlign tables (Table 2) for all six tasks. If LL-ICM's advantage remains at the same magnitude when every baseline also has an adapted downstream model, the headline claim is supported; if the advantage shrinks or disappears, the reported gains were an artifact of unmatched downstream fine-tuning. As a simpler complementary check, run LL-ICM with the frozen off-the-shelf IR-SDE (no stage-2 adaptation) against all baselines with the same frozen IR-SDE; if the codec alone does not win, the 'encoding ability' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that jointly optimizing a codec with low-level vision tasks yields large BD-rate savings—rests on a comparison that is not matched on the downstream model. In the Testing Setting, the authors state that 'to ensure the fairness of the comparison, we also use IR-SDE as the LL vision model to generate enhance outputs of the compressed images from these codecs.' But the Training Setting describes a second stage in which 'the LL task is then trained jointly with the codec and optimize it jointly.' Thus LL-ICM's enhanced output is produced by an IR-SDE that has been fine-tuned on LL-ICM's compressed images, whereas the baselines' enhanced outputs come from an off-the-shelf IR-SDE that has never seen those codecs' artifacts. Table 2's BD-rate, BD-LPIPS, and BD-QAlign numbers therefore measure a joint system (adapted codec + adapted restoration) against a baseline system (fixed codec + generic restoration), so the reported 22.65% BD-rate reduction and per-task gains (e.g., −96.07% for inpainting) cannot be attributed to the codec's 'encoding ability' or to mutual adaptation per se; the fine-tuned restoration model can absorb part or all of the apparent gain. The abstract's phrasing 'achieve 22.65% BD-rate reductions over the state-of-the-art methods' suggests a codec-level result, which this protocol does not establish. A matched control is required before the headline claim is credible.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LL-ICM, an image compression framework for low-level (LL) machine vision tasks. It extends a learned codec (MLIC) with a frozen CLIP feature extractor, a DA-CLIP encoder that predicts degradation type and caption, and an IR-SDE diffusion network that enhances the compressed image. In a two-stage training procedure, the codec is first trained for compression and then jointly fine-tuned with the LL restoration model under a rate-perception objective. The method is evaluated on six LL tasks (deraining, dehazing, raindrop removal, denoising, deshadowing, and inpainting) using LPIPS, Q-Align, and LIQE, with BD-rate, BD-LPIPS, and BD-QAlign metrics compared against Balle2018, Cheng2020, ECM, and MLIC. The paper claims a 22.65% BD-rate reduction over state-of-the-art codecs.","tokens_in":8643,"tokens_out":7280,"duration_ms":66637,"significance":"If the reported results were properly controlled, the paper would fill a genuine gap: most ICM work targets high-level tasks, while real-world images often need low-level enhancement after compression. The rate-perception formulation (Eq. 1) is a sensible extension of RDO, and the idea of using a VLM to condition one universal restorer is timely and practically appealing. The inclusion of both full-reference (LPIPS) and no-reference (Q-Align, LIQE) quality metrics is a strength, and the paper explicitly addresses the rate-perception trade-off rather than only reconstruction fidelity. However, the central quantitative claim is currently not supported because the baseline comparison confounds codec performance with downstream restorer adaptation, and the headline aggregate BD-rate number is undefined. The contribution is therefore promising but requires a re-evaluation under matched conditions before the claims can be accepted.","major_comments":[{"comment":"The comparison behind Table 2 is not matched on the downstream model. In the Training Setting, the paper states that in the second stage 'the LL task is then trained jointly with the codec and optimize it jointly,' so the IR-SDE used for LL-ICM is fine-tuned on LL-ICM's compressed images. In contrast, the Testing Setting states that for all baseline codecs 'we also use IR-SDE as the LL vision model to generate enhance outputs,' meaning the baselines are paired with an off-the-shelf IR-SDE that has not been adapted to their compression artifacts. The BD-rate, BD-LPIPS, and BD-QAlign values in Table 2 therefore compare a jointly adapted codec+restorer system against a fixed codec+generic restorer system. The headline claim in the Abstract (22.65% BD-rate reduction) and the per-task gains such as -96.07% for inpainting cannot be attributed to the codec's 'encoding ability' or to mutual adaptation per se, because the fine-tuned restorer could absorb a substantial part of the apparent gain. A matched control is required: for each baseline codec, fine-tune the same IR-SDE on its compressed images (or, alternatively, evaluate LL-ICM with the off-the-shelf IR-SDE), and report the comparison under equal restorer adaptation.","section":"Experiment (Training Setting / Testing Setting)"},{"comment":"The 22.65% BD-rate reduction claimed in the Abstract is not defined anywhere in the experiments. Table 2 reports per-task BD-rate values with Balle2018 as the anchor, and the text also mentions '7.70%-49.23% coding gains across various LL tasks compared to state-of-the-art methods,' but no formula or table entry shows how the 22.65% aggregate is computed or which baseline is considered state-of-the-art for that number. Since this is the central quantitative claim, the paper should specify the averaging procedure (e.g., over tasks, over rate points, versus MLIC) and provide the corresponding aggregate table entry.","section":"Abstract and Experiment (Table 2)"},{"comment":"The experimental setup for inpainting is underspecified. Table 1 lists 'inpainting' with sizes '29901+100' and references RePaint and CelebaHQ-256, but the text does not describe how the degraded input for inpainting is generated (e.g., mask type, mask ratio, or whether the model sees the mask). This matters because the largest claimed gain (-96.07% BD-rate in Table 2) is for inpainting; without the exact task protocol the result cannot be reproduced or interpreted. The formatting of Table 1 also makes dataset-task assignments ambiguous (e.g., the rows for denoising and inpainting are garbled).","section":"Experiment (Table 1 and Testing Setting)"},{"comment":"The LPIPS values reported in Fig. 6 appear to contradict the qualitative claim. For the deraining example, MLIC (LPIPS 0.13) and ECM (LPIPS 0.09) achieve lower (better) LPIPS than LL-ICM (LPIPS 0.19), and for raindrop removal LL-ICM (LPIPS 0.30) is worse than Cheng2020 (LPIPS 0.07) and ECM (LPIPS 0.15). If these LPIPS values are computed on the enhanced images that are shown, the figure undermines the statement that 'our method preserves more intricate details and texture'; if they are computed elsewhere, the caption should say so. The paper should reconcile this inconsistency or choose qualitative examples consistent with the quantitative results.","section":"Qualitative Comparison (Fig. 6)"}],"minor_comments":[{"comment":"The section contains a duplicated paragraph: the passage beginning 'Not all LL tasks yield consistent results' appears twice with different subfigure references (one mentions Fig. 4(g)/(h), the other Fig. 4(e)/(f)), while the Fig. 4 caption only lists subfigures (a)-(f). The text should be merged and the figure references made consistent.","section":"Rate-perception Performance Result"},{"comment":"There are several typos and inconsistencies: 'Exsiting' should be 'Existing', 'LL-LCM' appears where 'LL-ICM' is meant, 'debluring' should be 'deblurring', and the codec name is written both as 'Balle2018' and 'Balle'. These should be corrected.","section":"Throughout"},{"comment":"Hyperparameters for the second-stage joint training are not reported: the learning rate, the loss weights α, β, and γ in Eq. (5) for stage 2 are not given, and only stage-1 β values for Lr are listed. This information is necessary for reproducibility.","section":"Experiment (Training Setting)"},{"comment":"No code, trained models, or evaluation scripts are released. Given the reliance on multiple pretrained components (MLIC, CLIP, DA-CLIP, IR-SDE) and the complex two-stage training, releasing the implementation would be important for reproducibility and for verifying the claimed gains.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the proposed framework is interesting, but the evaluation confound is serious and the headline aggregate number is undefined. I do not see evidence of deliberate misreporting; the issue appears to be an oversight in designing the control condition. With a matched restorer-adaptation control, a clarified aggregate metric, and the inpainting protocol specified, the paper could become a solid contribution. The duplicated paragraph and figure inconsistencies should also be fixed in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline result here — 22.65% BD-rate savings from a single codec serving multiple restoration tasks — does not survive a close read of the evaluation. The idea itself is new and worth taking seriously: this is the first ICM framework aimed at low-level vision, and it sensibly couples a learned codec (MLIC) with a VLM-conditioned diffusion restorer (CLIP + DA-CLIP + IR-SDE). The rate-perception formulation is a reasonable reframing, and the evaluation spans six tasks, which is broader than prior ICM work. I give credit for that.\n\nThe problem is the baseline protocol. The authors state that they use the same off-the-shelf IR-SDE for all anchor codecs, but LL-ICM's IR-SDE is fine-tuned jointly with its own codec in stage two. So the comparison measures an adapted codec plus an adapted restorer against fixed codecs plus a generic restorer. The apparent gains — including the per-task numbers like -96% for inpainting — cannot be attributed to the codec's encoding ability. The stress-test note is right: this is an evaluation confound, not algebraic circularity. The claim would be credible with a matched control, e.g., fine-tuning the restorer for each anchor, or freezing one restorer for all systems.\n\nThere are smaller issues. No error bars, no code or weights, and the relationship between the abstract's 22.65% and the per-task BD-rates is never explained (averaging? which anchor? which metric?). The write-up also has duplicated denoising paragraphs and references to subfigures (g)/(h) that don't exist. Those are minor, but they add noise.\n\nIf the matched-baseline experiment still shows a large advantage, this is a solid contribution. As it stands, the quantitative claim is unsubstantiated. That said, the idea is original and the fix is clear, so I would send it to peer review rather than desk-reject. A good referee should demand the matched control, the missing reproducibility details, and a cleaner account of the headline number.","headline":"Novel idea, unfair baseline: the 22.65% BD-rate claim conflates codec performance with jointly fine-tuned restoration, so the headline should be read as unsubstantiated until matched controls are run.","tokens_in":9231,"tokens_out":2239,"would_cite":false,"duration_ms":21176,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One neural codec jointly trained with a vision-language-guided diffusion restorer can compress for six restoration tasks at once, saving 22.65% BD-rate over state-of-the-art codecs.","keywords":["image compression for machines","low-level vision","image restoration","vision-language model","rate-perception optimization","neural image codec","diffusion model","BD-rate"],"falsifier":"Retrain or fine-tune the baseline restoration model separately for each baseline codec, keeping the same total training effort as LL-ICM, and then recompute the BD-rate differences; if the savings shrink substantially, the reported 22.65% gain is an artifact of the unfair control.","tokens_in":1452,"feed_emoji":"🖼️","tokens_out":2051,"duration_ms":71325,"temperature":0.7,"pith_summary":"This paper sets out to extend image compression for machines from high-level tasks such as detection and segmentation to low-level restoration tasks such as deraining, dehazing, and denoising. It claims that a codec and a downstream restoration model can be trained as one unit, so that the encoder learns what to keep and what to discard for later enhancement. To make one codec serve many tasks, the framework uses a frozen vision-language model to extract general features and a diffusion network to turn those features into restored images. If the claim holds, a single bitstream can support multiple restoration tasks with large bitrate savings, and the reported experiments put those savings at 22.65% BD-rate over state-of-the-art codecs.","feed_headline":"One codec, six restoration tasks: 22.65% bitrate saved","feed_subtitle":"A vision-language-guided diffusion restorer trains jointly with a neural codec, letting one bitstream serve deraining, dehazing, and more.","key_machinery":"The load-bearing mechanism is the joint loss $\\mathcal{L} = \\alpha \\mathcal{L}_{\\text{dist}} + \\beta \\mathcal{L}_r + \\gamma \\mathcal{L}_{\\text{task}}$, in which $\\mathcal{L}_{\\text{dist}}$ measures compressed-image fidelity, $\\mathcal{L}_r$ measures bitrate, and $\\mathcal{L}_{\\text{task}}$ is the diffusion loss that drives the enhanced image toward the ideal image. What makes the framework general is the frozen vision-language extractor that produces feature embeddings from the compressed image; a controller then predicts the degradation type and a caption, and these guide the diffusion restorer. The codec is first trained alone and then fine-tuned jointly with the restoration network, which is the step that lets the encoder allocate bits to what restoration can fix rather than to what a human would want to see.","core_discovery":"The central claim is that rate-perception optimization, rather than classic rate-distortion optimization, is the right objective for compressing images that will later be restored. The paper defines the distortion term as the error between the enhanced image and an ideal high-quality image, not between the compressed and original image. Under this objective the proposed LL-ICM couples the MLIC neural codec with a frozen vision-language feature extractor, a controller that predicts degradation type and caption, and an IR-SDE diffusion restorer, all trained jointly. The authors report that one trained LL-ICM model handles six low-level tasks and achieves 22.65% BD-rate reductions over the compared codecs while also improving LPIPS and Q-Align scores.","pith_inferences":["The abnormal denoising curves suggest a testable extension: jointly optimized codecs should learn to allocate fewer bits to high-frequency noise, and measuring the bit allocation directly would show where the savings come from.","The paper measures perceptual quality with LPIPS and Q-Align, but a human study on the restored outputs would test whether the BD-rate savings translate into subjectively preferred images.","Because the vision-language embedding is frozen, the framework could in principle be applied to codecs other than MLIC, and the main empirical question is how much of the gain comes from the joint training versus the backbone codec choice.","A practical deployment would use the no-reference metrics in the loop, since ideal high-quality references are often missing in the field; the paper's use of Q-Align and LIQE points toward this but does not close the loop with an online adaptation scheme."],"forward_implications":["One trained LL-ICM codec can replace several task-specific codec-plus-restoration pipelines, since the same bitstream supports dehazing, deraining, raindrop removal, deshadowing, denoising, and inpainting.","If the reported BD-rate savings hold, deploying a joint codec for a restoration-driven pipeline costs about a fifth less bitrate at equal perceptual quality compared with the best compared anchors.","The rate-perception objective implies that the encoder can discard visually unpleasant but restoration-irrelevant content, which classic rate-distortion training would waste bits preserving.","The vision-language conditioning makes the framework extensible: new low-level tasks can be added to the same codec as long as the diffusion restorer can use the generalized embedding.","In some tasks such as denoising, perceptual quality can decrease as bitrate increases under ordinary codecs, a phenomenon the paper attributes to the codec preserving noise; LL-ICM still outperforms the anchors in that regime."],"supporting_citations":[{"why":"provides the backbone neural codec that LL-ICM builds on and fine-tunes jointly","marker":"[3]"},{"why":"supplies the frozen vision-language feature extractor whose generalized embeddings guide restoration across tasks","marker":"[4]"},{"why":"supplies the controller that maps the generalized features to degradation type and image caption","marker":"[5]"},{"why":"provides the diffusion restoration network and its loss for the downstream enhancement","marker":"[6]"},{"why":"serves as a traditional block-based codec baseline in the comparison","marker":"[14]"},{"why":"serves as the anchor learned-codec baseline for the BD-rate and BD-LPIPS calculations","marker":"[15]"},{"why":"serves as an additional learned-codec baseline","marker":"[16]"},{"why":"defines the LPIPS perceptual metric used for full-reference quality evaluation","marker":"[17]"},{"why":"defines the Q-Align no-reference quality metric used in the BD-Q-Align calculation","marker":"[18]"},{"why":"defines the BD-rate computation used to quantify coding gains","marker":"[20]"}],"fun_headline_variants":["Vision-language-guided codec cuts bitrate 22.65% for restoration","One codec, six low-level tasks, 22.65% BD-rate saving","LL-ICM: Rate-perception optimization beats rate-distortion for restoration","Codec and restorer trained jointly with VLM guidance saves 22.65% bitrate","For low-level vision, compress with rate-perception not rate-distortion"],"cache_read_input_tokens":11136,"weakest_assumption_plain":"The evaluation assumes that applying the same off-the-shelf IR-SDE to all baseline codecs is a fair control, even though LL-ICM's restoration network is fine-tuned jointly with its codec.","fun_headline_variants_meta":{"raw":{"variants":["Vision-language-guided codec cuts bitrate 22.65% for restoration","One codec, six low-level tasks, 22.65% BD-rate saving","LL-ICM: Rate-perception optimization beats rate-distortion for restoration","Codec and restorer trained jointly with VLM guidance saves 22.65% bitrate","For low-level vision, compress with rate-perception not rate-distortion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000791,"raw_usage":{"total_tokens":3486,"prompt_tokens":948,"completion_tokens":2538,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":2432}},"tokens_in":564,"tokens_out":2538,"duration_ms":16901,"temperature":1.0,"reasoning_tokens":2432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:00:44.747789+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain or fine-tune the baseline restoration model separately for each baseline codec, keeping the same total training effort as LL-ICM, and then recompute the BD-rate differences; if the savings shrink substantially, the reported 22.65% gain is an artifact of the unfair control.","supporting_citations":[{"cited_title":"Learning transferable visual models from natural language supervision,","cited_arxiv_id":null,"evidence_quote":"supplies the frozen vision-language feature extractor whose generalized embeddings guide restoration across tasks"},{"cited_title":"Video Compression Beyond VVC: Quantitative Analysis of Intra Coding Tools in Enhanced Compression Model (ECM)","cited_arxiv_id":"2404.07872","evidence_quote":"serves as a traditional block-based codec baseline in the comparison"},{"cited_title":"Learned image compres- sion with discretized gaussian mixture likelihoods and attention modules,","cited_arxiv_id":null,"evidence_quote":"serves as an additional learned-codec baseline"},{"cited_title":"The unrea- sonable effectiveness of deep features as a perceptual metric,","cited_arxiv_id":null,"evidence_quote":"defines the LPIPS perceptual metric used for full-reference quality evaluation"}],"review_version":1}