{"id":"602f867e-5617-49fd-bd61-95ffeb463004","arxiv_id":"2411.09308","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A transformer-based model predicts the minimum detectable distortion for machine vision and uses it to reduce video coding bitrate by about 30% without measured loss in object-detection accuracy.","lead":"This paper trains a Vision Transformer to predict the Just Recognizable Difference (JRD), the smallest distortion a machine-vision system can notice, using only the original image. The predicted JRD guides a video codec to spend fewer bits on background regions while preserving object-detection accuracy, achieving reported bitrate savings of about 30% versus VVC.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Background-QP assumption is the load-bearing risk: the 29.58% BDBR saving is not isolated from QPb choice, is tested with one detector, and may not transfer to full-image detection.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern I find: the coding gain from the proposed VCM depends on the untested assumption that background CTUs can be aggressively compressed without hurting object detection. This concern is central because the headline result of 29.58% bit-rate reduction is an application claim, not just a prediction-accuracy claim. The paper's evidence is consistent with the claim on its own test setup, but it does not isolate the background-QP component, does not specify QPb, and does not test detector generality. A concrete sweep of QPb across multiple detectors would settle whether the saving is a robust property of JRD-guided coding or an artifact of one detector and one split. Because the reader already conditioned acceptance on this and related reproducibility issues, my analysis does not move the verdict; it reinforces the conditional status. I see no reason to call for rejection: the prediction-accuracy numbers and ablations are internally consistent, and the method is a plausible extension of prior work. The concern is a missing generality check on the application claim, not a demonstrated error.","tokens_in":17107,"tokens_out":12793,"duration_ms":111087,"concrete_test":"Fix the object CTU QPs to the predicted JRDs and sweep the background QP QPb over a range (e.g., QP_object+0, +5, +10, +15, +20) on a held-out COCO-image test set, using at least two detectors (Faster R-CNN and one transformer-based detector such as DETR). Plot mAP@0.5 vs bit rate and compute BDBR against VVC all-intra for each QPb and each detector. If the BDBR savings or the mAP floor changes substantially across detectors, or if mAP drops monotonically as QPb increases, then the Section II-E background assumption is not generally valid and the 29.58% claim is specific to the chosen detector/test setting. Also compare against a version with QPb = QP_object to isolate the background contribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central coding-gain claim (29.58% BDBR savings vs VVC all-intra at equal mAP) rests on the Section II-E strategy of encoding background CTUs with a QP larger than the object CTUs' predicted JRD. This is load-bearing because the bit-rate saving is substantially the background bit budget, yet the paper never specifies QPb, never ablates it, and evaluates mAP with a single detector (Faster R-CNN) on a single split of the JRD dataset. Faster R-CNN uses context outside the annotated box, so aggressive background compression can remove cues the detector relies on; preserving object CTUs does not by itself guarantee detection accuracy. Moreover, the JRD ground truth in [37] was collected on cropped object images, so transferring object-wise JRD thresholds to full-image CTU coding is an extrapolation not validated by the reported mAP numbers alone. The reported BDBR savings therefore rest on an unquantified and untested assumption. No confidence intervals are provided, so the magnitude is also uncertain. This is not an accusation of misreporting; it is a missing generality check on the key application claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DT-JRD, a ViT-based multi-class classifier that predicts object-wise Just Recognizable Difference (JRD) directly from the original image, avoiding the multi-pass binary-classification search of prior JRD predictors. Training uses Gaussian Distribution-based Soft Labels (GDSL), and the predicted JRD is used to guide VVC all-intra coding by encoding object CTUs at the predicted JRD QP and background CTUs at a larger QP. The authors report a JRD prediction MAE of 5.574, a 13.1% improvement over BC-JRD, and a 29.58% BDBR reduction against VVC all-intra at matched mAP for Faster R-CNN object detection.","tokens_in":17348,"tokens_out":9947,"duration_ms":134974,"significance":"If confirmed, the single-pass prediction is a practical advantage over prior multi-pass JRD predictors, and the GDSL soft-label formulation is a simple, transferable idea for ordinal classification near perceptual thresholds. The paper includes comparisons to two baselines, ablations of learning strategy, output features, and position embedding, and a wide set of accuracy metrics (MAE, PSNR, SSIM, bit rate, R2). The main uncertainty is not the JRD prediction itself but the coding-gain evaluation, which currently rests on an unspecified background-QP choice and a nonstandard mAP protocol; these issues are fixable with additional experiments and therefore warrant major revision rather than rejection.","major_comments":[{"comment":"The mAP is computed using the detections of Faster R-CNN on the original images as ground truth rather than the dataset's human annotations. This protocol measures only consistency with the uncompressed detector output and can hide detection failures on objects that the detector already misses in the original image. Since the central coding claim is \"maintaining object detection accuracy,\" please recompute mAP against COCO ground-truth boxes, or justify the pseudo-ground-truth choice and show that the BDBR conclusions are unchanged.","section":"Section III-A.3"},{"comment":"The background CTU QP, denoted QPb, is a free parameter of the proposed VCM, but the paper never states its value, how it was chosen, or how sensitive the 29.58% BDBR saving is to it. Because background CTUs dominate the bit budget, the reported coding gain could be driven by QPb rather than by the predicted JRD. Please specify the QPb values used, add an ablation over QPb (including a foreground-only baseline without any background QP increase), and report a foreground/background bit-rate decomposition.","section":"Section II-E and Section III-D"},{"comment":"The coding experiments use a single detector (Faster R-CNN), a single random 8:1:1 split, and no confidence intervals or repeated runs. Since the JRD ground truth was collected on cropped object images, transferring object-wise JRD thresholds to full-image CTU-level coding is an extrapolation. Please add at least one additional detector (e.g., YOLO or DETR), evaluate on multiple splits, and report mean and standard deviation for MAE and BDBR; otherwise the generality of the 29.58% claim is not established.","section":"Section III-D and Section III-A.1"},{"comment":"The reported MAE values are drawn from different evaluation sets (validation in Table I, category-specific and overall test values in Table II, and ablation values in Table III), but the table captions and surrounding text do not consistently say which set each number refers to. Please state the dataset split explicitly for every table and metric, and clarify that the abstract's 5.574 is the Table III ablation value while the full-test overall EA in Table II is 5.57 after rounding.","section":"Section II-D, Table I, and Section III-B"}],"minor_comments":[{"comment":"The column order in Table II is confusing because the header repeats \"EA\" and the text refers to \"the first five columns\" and \"the last two columns.\" Please label the columns explicitly as Small, Medium, Large, Person, Car, EA, and E[27,51].","section":"Table II"},{"comment":"The QP settings are inconsistent: Section III-A.3 states that VVC all-intra uses QP ∈ {25, 27, 29, 31, 33}, while Section III-D states that the QP of the original VVC all-intra is from 28 to 35. Please unify the settings and report the exact operating points used for each BDBR calculation.","section":"Section III-A.3 and Section III-D"},{"comment":"The sentence \"The predicted JRD by the DT-JRD is averagely larger than the ground truth JRD for about 1.67\" should be tied to the QP-offset analysis; please clarify whether this bias contributes to the bit-rate savings and whether it affects the mAP comparison.","section":"Section III-D"},{"comment":"The checkmark-only notation in Table III makes the configurations hard to read; please use explicit Yes/No entries or separate rows for each configuration so that the reader can identify exactly which modules are active in each ablation.","section":"Table III"},{"comment":"No code, pretrained models, or evaluation scripts are released. Given the need for replication of the VCM results, a public release would strengthen the paper.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for IEEE Transactions on Multimedia. The JRD prediction contribution appears sound, but the headline coding gain of 29.58% currently depends on an unspecified background QP and a nonstandard mAP protocol, both of which are fixable with additional experiments. I would not reject on the prediction results; the requested revisions are about making the application claim reproducible and general."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a credible, useful paper for the VCM subfield, and its main prediction claim is probably right. What is actually new: DT-JRD predicts JRD from a single original image with a ViT, where the prior state of the art (EL-JRD, BC-JRD) required generating N compressed versions and running binary classifiers N times. That is a real simplification, and the numbers support it: MAE drops from 6.24 to 5.61 on the test set (about 13%), and the improvement is consistent across object sizes and categories. The GDSL loss and DAFT fine-tuning are small adaptations of known ideas, but they are sensible and the ablation table suggests each contributes. Credit where due: the experiments are fairly extensive within the JRD dataset, and the paper is honest about comparing to the two earlier methods.\n\nThe main soft spot is the coding-gain claim. The 29.58% BDBR saving over VVC all-intra comes from the Section II-E scheme that encodes background CTUs with QPb larger than the predicted object JRD. The paper never says what QPb is, never ablates it, and tests only Faster R-CNN on one split. Since the JRD labels were collected on cropped object images, applying them to full-image CTU allocation is an extrapolation; aggressive background compression could hurt detectors that use context outside the box. That makes the headline coding number conditional, not wrong.\n\nAlso: no error bars, one random split, no code release, and a small internal inconsistency — the abstract quotes MAE 5.574, Table II gives 5.61. Table III's checkmarks are hard to read. These are mechanical reproducibility issues, not signs of fraud.\n\nWho this is for: people working on VCM, machine-oriented compression, or perceptual thresholds for machine vision. They will get a solid baseline and a clear direction. I would send this out for peer review; the prediction model deserves referee time. The revision needs to pin down QPb, add variance estimates or multiple splits, release code, and fix the MAE discrepancy. If that happens, this is a solid TMM-level paper.","headline":"A credible single-image JRD predictor for VCM, with a coding-gain claim that is plausible but not yet pinned down because the background-QP choice is unstated and untested.","tokens_in":17891,"tokens_out":2812,"would_cite":true,"duration_ms":26017,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single forward pass on the original image can predict the distortion level at which machine vision starts to fail, and using that prediction to guide the VVC codec saves about 29.58% of the bit rate while preserving object-detection…","keywords":["Just Recognizable Difference","Video Coding for Machines","object detection","Vision Transformer","Gaussian soft labels","VVC","bit-rate reduction","multi-class classification"],"falsifier":"Run the DT-JRD VCM pipeline on a benchmark where background context matters for detection—for instance, small or partially occluded objects whose surroundings provide essential cues—and compare the mAP@0.5 versus bit-rate curve against VVC all-intra; if the curve dips below VVC at matched bit rates, the background-QP assumption fails.","tokens_in":16900,"feed_emoji":"🤖","tokens_out":7669,"duration_ms":74608,"temperature":0.7,"pith_summary":"The paper tries to establish that the Just Recognizable Difference (JRD)—the smallest amount of compression distortion that makes a machine-vision system's output change—can be predicted directly from a single original image, and that the prediction is accurate enough to drive a Video Coding for Machines (VCM) system. The proposed DT-JRD model casts JRD prediction as a 64-class classification task solved by a fine-tuned Vision Transformer, instead of the previous approach of running many binary classifiers on many pre-compressed versions of the same image. On the Object-Wise JRD dataset the model reports a mean absolute error of 5.574, about 13.1% better than the best previous predictor. When the predicted JRD is used to set per-block quantization in the Versatile Video Coding (VVC) standard, the authors report an average 29.58% bit-rate reduction at matched object-detection accuracy. If true, this turns a costly multi-pass search into a one-pass estimate and gives VCM a practical way to spend bits only where the machine vision task needs them.","feed_headline":"Predicting machines' distortion tolerance cuts video bit rate ~30%","feed_subtitle":"Single-pass estimate of the just-recognizable difference saves ~30% bit rate without hurting detection.","key_machinery":"The load-bearing mechanism is the DT-JRD network itself: a Vision Transformer (ViT-Large/32) that takes only the original object image and outputs a probability distribution over 64 VVC distortion levels. Three design choices carry the result: (1) the Distortion-Aware Fine-Tuning (DAFT) strategy, which freezes the patch embedding, class token, position embedding, and final layer norm while training the transformer blocks and classification head, so the pretrained model adapts without overfitting on the small JRD dataset; (2) patch-token average pooling rather than the class token for the classification head, because pooled patch features carry more JRD-relevant distortion information; and (3) the Gaussian Distribution-based Soft Label (GDSL) loss, which replaces one-hot labels with a Gaussian centered at the true JRD and standard deviation σ=3, effectively augmenting labels with neighboring distortion levels and relaxing the classification boundary.","core_discovery":"The core discovery is that JRD is a learnable, content-dependent threshold, and a transformer trained with soft Gaussian labels can predict it in one forward pass. The paper shows that neighboring distortion levels around the ground-truth JRD are visually and semantically close, so instead of treating only the exact JRD class as correct, the loss assigns partial probability to nearby classes according to a Gaussian centered at the true JRD with standard deviation σ = 3. Combined with a Distortion-Aware Fine-Tuning strategy and 2D-interpolated position embeddings for 384×384 inputs, this brings the predicted-JRD error down to 5.574 MAE. Feeding the predicted JRD into a VVC all-intra encoder as a per-object quantization constraint—background coding tree units get a larger QP than object regions—yields an average 29.58% bit-rate reduction at matched mAP@0.5 and 26.84% at mAP@0.75 compared with unmodified VVC, while the detector's accuracy is maintained.","pith_inferences":["An untested extension is to feed the predicted JRD into a learned image or video codec as an adaptive quality constraint rather than mapping it to VVC QP; the paper only demonstrates the VVC path.","The background-QP assumption is validated on one detector and one data split; on detectors that use context outside the bounding box, the reported 29.58% saving may shrink because the detector would notice the coarser background.","The GDSL idea is not JRD-specific: any ordered-label prediction task where nearby classes are semantically close, such as bitrate ladders or difficulty ratings, could benefit from the same Gaussian soft-label loss."],"forward_implications":["A single source image is enough to predict JRD, so the expensive N-times compression-and-classification loop of earlier predictors can be dropped.","Because the predicted JRD is about 1.67 QP steps above the ground truth on average, the measured bit-rate savings are conservative and could grow with a better-calibrated predictor.","Per-CTU quantization guided by predicted JRD transfers the gain to any standard encoder: the paper demonstrates a 29.58% bit-rate saving at matched mAP@0.5 over VVC all-intra.","The same predicted-JRD interface can be plugged into other coding pipelines without changing the decoder or the downstream object detector."],"supporting_citations":[{"why":"Defines the JRD concept and supplies the EL-JRD ensemble predictor, the first of the two baselines DT-JRD must beat.","marker":"[33]"},{"why":"Provides the Object-Wise JRD dataset construction and the BC-JRD binary-classifier baseline that DT-JRD improves on by 13.1%.","marker":"[37]"},{"why":"Releases the 29,218-object JRD dataset used to train, validate, and test DT-JRD.","marker":"[46]"},{"why":"Introduces the Vision Transformer architecture whose pretrained encoder blocks and position embeddings DT-JRD adapts.","marker":"[38]"},{"why":"Specifies the Faster R-CNN detector used to measure the mAP values in the VCM coding experiments.","marker":"[1]"},{"why":"Defines the VVC standard used as the base codec and bit-rate baseline in the coding experiments.","marker":"[6]"}],"fun_headline_variants":["Transformer learns machines' loss threshold, cuts video bit rate 30%","JRD transformer predicts tolerable distortion, trims 30% video bit rate","Machine-vision codec uses transformer to cut bits 30% at same accuracy","Single-pass JRD estimate enables 30% bit-rate cut for machine tasks","Deep transformer spots machine distortion limit, reduces bit rate 30%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a detector's accuracy on an object depends only on the content inside its bounding box, so the surrounding background can be compressed much more aggressively; the paper tests this with one detector and one data split rather than establishing it as a general property of machine vision.","fun_headline_variants_meta":{"raw":{"variants":["Transformer learns machines' loss threshold, cuts video bit rate 30%","JRD transformer predicts tolerable distortion, trims 30% video bit rate","Machine-vision codec uses transformer to cut bits 30% at same accuracy","Single-pass JRD estimate enables 30% bit-rate cut for machine tasks","Deep transformer spots machine distortion limit, reduces bit rate 30%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000794,"raw_usage":{"total_tokens":3529,"prompt_tokens":1011,"completion_tokens":2518,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":2419}},"tokens_in":627,"tokens_out":2518,"duration_ms":16900,"temperature":1.0,"reasoning_tokens":2419,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:47:28.484741+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the DT-JRD VCM pipeline on a benchmark where background context matters for detection—for instance, small or partially occluded objects whose surroundings provide essential cues—and compare the mAP@0.5 versus bit-rate curve against VVC all-intra; if the curve dips below VVC at matched bit rates, the background-QP assumption fails.","supporting_citations":[{"cited_title":"Just recognizable distortion for machine vision oriented image and video coding,","cited_arxiv_id":null,"evidence_quote":"Defines the JRD concept and supplies the EL-JRD ensemble predictor, the first of the two baselines DT-JRD must beat."},{"cited_title":"Learning to predict object-wise just recognizable distortion for image and video compression,","cited_arxiv_id":null,"evidence_quote":"Provides the Object-Wise JRD dataset construction and the BC-JRD binary-classifier baseline that DT-JRD improves on by 13.1%."},{"cited_title":"Object- wise just recognizable distortion dataset,","cited_arxiv_id":null,"evidence_quote":"Releases the 29,218-object JRD dataset used to train, validate, and test DT-JRD."},{"cited_title":"Overview of the versatile video coding (vvc) standard and its applications,","cited_arxiv_id":null,"evidence_quote":"Defines the VVC standard used as the base codec and bit-rate baseline in the coding experiments."}],"review_version":1}