{"id":"439cb28f-888c-4ac3-92a6-e1e4732bb6f6","arxiv_id":"2506.09795","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A concatenation of VCA complexity residuals and SSIM fed into an XGBoost regressor reaches PLCC 0.787 on the VQA Grand Challenge test set, though the method is not truly reduced-reference.","lead":"This paper combines handcrafted complexity features and SSIM in an XGBoost model to predict video quality scores for the VQA Grand Challenge. The authors report competitive correlation with human ratings while claiming a lightweight, reduced-reference design, but the method actually requires full reference pixels for SSIM.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SSIM feature requires full reference frames at inference, so the claimed reduced-reference setting is not realized and the central applicability claim is unsupported.","rationale":"The reader's verdict identifies the same load-bearing weakness: the method is described as reduced-reference but Eq. 7 requires the full reference video to compute SSIM. The full text confirms this: Section 3.5 explicitly defines SSIM_i as the similarity between reference and distorted frames, and the architecture diagram in Fig. 1 shows SSIM computed from 'Ref vs Test'. The abstract's phrase 'compute SSIM values from the test video' is ambiguous, but the formal definition and diagram remove that ambiguity. The result is that the 8-dimensional feature vector z in Eq. 9 cannot be produced at inference time under the claimed reduced-reference constraint. This is not an external disagreement about consensus; it is an internal inconsistency between the method definition and the stated operating mode. The Section 4.6 limitation discusses only reference complexity features, not the SSIM requirement, so the paper does not honestly flag this gap. Because the reduced-reference mode is the central contribution and the motivation for the claimed deployment scenarios, the central claim is not well-supported as written. The empirical PLCC numbers might be accurate for a full-reference variant, but the paper does not present the method in those terms or compare it fairly in that category. Therefore the reader's reject verdict stands without modification. No additional concern is needed: this single issue is sufficient, since it changes the central claim from a practical RR system to an FR system or to an unverified 7-dimensional variant.","tokens_in":6938,"tokens_out":2259,"duration_ms":27438,"concrete_test":"Reimplement the test-set inference under two input regimes: (a) only the compact reference VCA summary vector x̄ from Eq. 4 plus the test video, so that μ_SSIM cannot be computed; and (b) full reference frames, so that Eq. 7 can be evaluated. If the reported PLCC of 0.787 is attainable only in regime (b), the published result belongs to a full-reference system and the reduced-reference claim is unsupported. If a comparable PLCC is obtained in regime (a), the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is a lightweight reduced-reference VQA method requiring only partial reference-side information. However, Section 3.5 (Eq. 7) defines the SSIM feature as the average similarity between reference and distorted frames, and Fig. 1 labels the input as 'SSIM (Ref vs Test)'. Computing SSIM requires pixel-level local statistics from the pristine reference video at inference time, not merely compact VCA features. Thus the fused vector z = [r | μ_SSIM] in Eq. 9 cannot be formed under the advertised reduced-reference constraint. If the reference video is unavailable at inference, the method degrades to the 7-dimensional residual vector alone, and the reported PLCC of 0.787 is not the PLCC of the described RR system. Conversely, if the reference frames are available, the system is full-reference and must be benchmarked and described as such. The Section 4.6 limitation statement only acknowledges needing reference complexity features and omits this SSIM requirement, which does not resolve the inconsistency. This is not merely a framing issue: it changes the deployment scenario (streaming monitoring, edge encoding) that motivates the paper, and it means the strongest empirical claim is attached to a method that differs from the proposed RR contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a reduced-reference video quality assessment model that combines seven VCA complexity features with a mean SSIM value, feeds the residual between reference and distorted feature averages plus SSIM into an XGBoost regressor, and reports SROCC 0.832, PLCC 0.787, KROCC 0.632, and RMSE 0.862 on the VQA Grand Challenge test set. The authors claim the system requires only partial reference-side information, is lightweight and interpretable, and outperforms PSNR while approaching VMAF and P.1204.3.","tokens_in":7053,"tokens_out":4705,"duration_ms":44404,"significance":"If the reduced-reference claim were actually realized, the approach would be practically relevant for streaming monitoring and adaptive encoding, since it is simple, fast, and does not require deep learning or GPU inference. The comparison against PSNR, VMAF, and P.1204.3 is transparent, and the SHAP analysis provides a useful interpretability check. However, as written, the central contribution is not supported: the SSIM feature requires the full reference video at inference time, so the evaluated system is full-reference rather than reduced-reference. The information-theoretic justification is asserted rather than derived and is not used by the model. The performance gap to VMAF and P.1204.3 is also larger than the text suggests, and the evaluation lacks uncertainty quantification, so the empirical claims are weaker than presented.","major_comments":[{"comment":"The SSIM feature μ_SSIM is defined as the average SSIM between reference and distorted frames, and Fig. 1 labels the input 'SSIM (Ref vs Test)'. Computing SSIM requires pixel-level statistics from the pristine reference video at inference time. Therefore the fused vector z=[r | μ_SSIM] in Eq. (9) cannot be formed under the advertised reduced-reference constraint. Section 4.6 acknowledges only the need for reference complexity features and omits this SSIM requirement. This is load-bearing: the reported PLCC of 0.787 applies to a system that uses the full reference, not to the claimed RR system, and the deployment scenarios in the abstract and conclusion are exactly those where a full reference is typically unavailable.","section":"3.5, Eq. (7), Fig. 1, Sec. 4.6"},{"comment":"The information-theoretic equations are asserted without proof and are not used by the regression. Equation (6) is also not correct in the stated form: for Gaussian feature distributions, the KL divergence equals (1/2)(Δμ)^T Σ^{-1} (Δμ), which reduces to (1/2)||r||_2^2 only for identity covariance. Equation (8) is presented as a known result, but the cited reference [21] does not establish that SSIM is a lower bound on normalized mutual information. Since these equations are presented as the grounding for the residual and SSIM features, they need to be correctly derived and justified or removed from the paper.","section":"3.4-3.5, Eqs. (6), (8)"},{"comment":"The train/validation/test protocol is underspecified: the paper states that 360 training videos and 198 test videos are used, but it does not describe how the validation set for Optuna was constructed, whether it was disjoint from the test set, or whether hyperparameters were selected on the same test set. The reported SROCC, PLCC, KROCC, and RMSE values are single-run point estimates with no confidence intervals, bootstrap error bars, or significance tests, making it impossible to assess whether the observed difference from PSNR is reliable.","section":"4.1, 4.4"},{"comment":"The claim that the method 'approaches' the performance of VMAF and P.1204.3 is not supported by Table 2: PLCC is 0.787 versus 0.901 for VMAF and 0.935 for P.1204.3, a gap of 0.11-0.15, and SROCC is 0.832 versus 0.905 and 0.925. The qualitative language overstates the results and should be revised to a factual statement of the gap.","section":"4.4, Table 2, Conclusion"}],"minor_comments":[{"comment":"The abstract says SSIM is computed 'from the test video', while Eq. (7) computes SSIM between reference and distorted frames; please align the descriptions.","section":"Abstract, 3.5"},{"comment":"The text first says '31 open-sourced contents' and then says '20 open-sourced videos' are used for training; clarify the dataset split and the role of the remaining 11 contents.","section":"4.1"},{"comment":"Hyperparameter names such as 'n estimators' and 'colsample bytree' should be formatted as 'n_estimators' and 'colsample_bytree', and the search range should be reported as a complete sentence.","section":"4.2"},{"comment":"'P1204 3' should be 'P.1204.3', and the corresponding reference [14] should be cited at the table caption.","section":"Table 2"},{"comment":"The runtime comparison lacks implementation details such as software version, CPU model, measured overhead, and the breakdown between VCA and SSIM computation; report actual timings rather than only a relative fraction.","section":"4.5"},{"comment":"The paper has no code or data availability statement; given that the challenge dataset is public, a code release would help reproducibility.","section":"Reproducibility"}],"recommendation":"reject","confidential_remarks":"For the editor: the manuscript is a short challenge paper with a simple and potentially useful lightweight pipeline, but the central reduced-reference claim is invalid as written because the SSIM feature requires the full reference video. Re-framing the method as full-reference would remove the stated contribution, while removing the SSIM term would change the method and likely reduce performance. The information-theoretic framing is unsupported and should not influence acceptance. I recommend rejection, though the authors could resubmit a substantially revised version with a genuinely reduced-reference formulation and proper uncertainty quantification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front: the method is not reduced-reference as advertised, and the reported PLCC of 0.787 belongs to a full-reference system. The SSIM feature in Eq. 7 is computed between reference and distorted frames, which means the full reference video is required at inference. The figure confirms it. So the streaming-monitoring story they sell is not what they built.\n\nWhat is actually new: the specific combination of VCA residual features, SSIM, and XGBoost for VQA is a reasonable engineering blend. It is lightweight, interpretable, and runs fast. The empirical result beats PSNR on their test set, though it is well below VMAF and P.1204.3. The SHAP analysis is a useful addition.\n\nThe soft spots are real and load-bearing. The SSIM issue is not a framing quibble; it changes the deployment scenario. The information-theoretic framing in Eqs. 6 and 8 is decorative—those formulas are asserted without proof and the XGBoost regression never uses them. The evaluation is thin: one test-set number, no error bars, underspecified train/validation/test protocol, and no comparisons with other reduced-reference or no-reference metrics. No code or data is shipped. The limitation section admits needing reference complexity features but omits that SSIM needs the full reference, so the reduced-reference claim survives only in name.\n\nTo be fair, the rest of the method is internally coherent, and the paper is reasonably organized. The citation pattern is mostly appropriate, including self-citations to the authors' VCA tool, which is legitimate. But the central contribution is misrepresented as reduced-reference when it is actually full-reference. The result could be salvaged by reframing as full-reference and benchmarking against VMAF/PSNR in that category, or by replacing SSIM with a genuinely reduced-reference structural feature and re-evaluating.\n\nThis is a workshop-grade paper that needs major revision. The reader who cares about lightweight VQA for streaming might get a useful idea from the feature combination, but should not rely on the reduced-reference claim. A serious referee could help the authors fix the framing and add the missing comparisons, so I would not desk-reject it. But acceptance in the current form would be a mistake.","headline":"The reduced-reference claim breaks because the SSIM feature needs the full reference at inference; the actual result is for a full-reference system, and the rest is a competent but thin engineering exercise.","tokens_in":7667,"tokens_out":3413,"would_cite":false,"duration_ms":42475,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A compact 8-vector of complexity residuals and SSIM, fed to XGBoost, predicts video quality with a PLCC of 0.787—outperforming PSNR and approaching VMAF and P.1204.3.","keywords":["video quality assessment","reduced-reference","Video Complexity Analyzer","SSIM","XGBoost regression","spatio-temporal features","perceptual quality prediction","streaming quality monitoring"],"falsifier":"Run the challenge test set with the SSIM component removed, keeping only the seven VCA residuals; if the Pearson correlation falls below PSNR-Y's 0.677, the structural term is load-bearing and the method is effectively near-full-reference rather than reduced-reference.","tokens_in":6621,"feed_emoji":"🎬","tokens_out":7080,"duration_ms":67386,"temperature":0.7,"pith_summary":"This paper tries to show that video quality can be predicted with a small, interpretable set of handcrafted features instead of deep networks or full-reference pixels. It extracts seven spatio-temporal complexity values per frame with the Video Complexity Analyzer, averages them over time, and computes residual differences between reference and test videos. Adding the mean SSIM between reference and test frames yields an 8-dimensional vector that an XGBoost regressor maps to a subjective quality score. On the VQA Grand Challenge test set the model reaches a Pearson correlation of 0.787, beating PSNR and approaching VMAF and P.1204.3 at roughly one-third of VMAF's runtime. The appeal is that streaming or edge systems could estimate perceived quality cheaply, with interpretable features and no GPU.","feed_headline":"8 features predict video quality near VMAF's accuracy at 1/3 cost","feed_subtitle":"A residual vector from complexity stats plus SSIM beats PSNR and lands at 0.787 PLCC on the VQA Grand Challenge test set.","key_machinery":"The load-bearing object is the fused feature vector $z = [r \\mid \\mu_{\\text{SSIM}}] \\in \\mathbb{R}^8$, where $r = \\bar{x} - \\bar{\\hat{x}}$ is the difference between temporally averaged VCA feature vectors and $\\mu_{\\text{SSIM}}$ is the frame-averaged SSIM. VCA supplies seven handcrafted per-frame features—luma texture energy, gradient magnitude, luminance, and chroma complexity terms—that act as empirical embeddings of source and channel statistics. The identity doing the work is $r \\approx$ information loss: with Gaussian feature distributions, $\\frac{1}{2}\\|r\\|_2^2$ approximates the KL divergence between reference and distorted distributions, and SSIM is invoked as a lower bound on normalized mutual information. XGBoost, a gradient-boosted ensemble of decision trees, supplies the nonlinear regression $f(z) \\to$ quality score, and SHAP analysis identifies the luma-energy residual $r_E$ as the most influential feature.","core_discovery":"This paper establishes that predictions of perceived video quality can be carried by an eight-dimensional vector whose entries are temporally averaged Video Complexity Analyzer features, their residuals between reference and test videos, and the mean SSIM between reference and test frames. The residual vector $r$ is interpreted as a proxy for information loss because, under Gaussian assumptions, $\\frac{1}{2}\\|r\\|_2^2$ approximates the Kullback–Leibler divergence between reference and distorted feature statistics, while $\\mu_{\\text{SSIM}}$ is treated as an estimate of normalized mutual information. XGBoost then learns the empirical mapping from this compressed mutual-information approximation to subjective scores. On the VQA Grand Challenge test set the model reaches a PLCC of 0.787 and SROCC of 0.832, outperforming PSNR-Y (0.677) and approaching VMAF (0.901) and P.1204.3 (0.935) at roughly one-third of VMAF's computational time.","pith_inferences":["A truly reduced-reference deployment would need to drop the SSIM term, since it requires reference pixels; if SSIM is removed, the model reduces to a 7-dimensional residual regressor, and its edge over PSNR may shrink—an implicit limitation of the paper's reduced-reference framing.","The residual-as-KL-divergence proxy suggests a saturation prediction: quality estimates should flatten as residual magnitudes grow, a shape that could be inspected directly in XGBoost's learned trees.","Because luma-energy residual dominates importance, distortions that preserve DCT energy (semantic edits, temporal aliasing) may escape the model; adding motion or semantic features is a testable extension.","The same residual-fusion recipe could be applied to other feature families, such as learned embeddings or VMAF's sub-metrics, to test whether the complexity residual plus structural similarity is the general mechanism."],"forward_implications":["Video streaming pipelines can monitor perceptual quality in near real time: the method needs no GPU and runs in about one-third of VMAF's time, close to PSNR's runtime.","Adaptive bitrate controllers can rank candidate encodings by predicted quality using only VCA statistics and SSIM, making per-chunk decisions feasible.","Engineers get interpretable diagnostics: SHAP analysis shows which complexity dimension (notably luma energy residual) drives a quality drop, pointing to the cause of degradation.","The approach extends to HDR and SDR content within the challenge's BT.2020/PQ gamut, indicating the handcrafted features transfer across dynamic range."],"supporting_citations":[{"why":"Defines the Video Complexity Analyzer that supplies the seven-dimensional spatio-temporal features used in the residual vector.","marker":"[7]"},{"why":"Introduces the XGBoost gradient-boosted tree algorithm that regresses the fused feature vector to a quality score.","marker":"[8]"},{"why":"Defines the Structural Similarity Index used as the mean SSIM component of the feature vector.","marker":"[4]"},{"why":"VMAF is the full-reference metric whose accuracy the method approaches while running substantially faster.","marker":"[5]"},{"why":"PSNR-Y is the classical metric the proposed method outperforms on correlation with subjective scores.","marker":"[3]"},{"why":"P.1204.3 is the bitstream-based standard that represents the high-accuracy baseline the method approaches.","marker":"[14]"},{"why":"Provides the information-theoretic link treating SSIM as an estimate of normalized mutual information.","marker":"[21]"},{"why":"Describes the HDR/SDR dataset used for training and testing in the VQA Grand Challenge.","marker":"[22]"}],"fun_headline_variants":["8-feature XGBoost nears VMAF at one-third the compute","XGBoost on 8 features rivals VMAF at 1/3 the time","Lightweight VQA: 8 features predict quality near VMAF at 1/3 cost","8 features beat PSNR, approach VMAF, at 1/3 cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing structural signal is the SSIM term, which is computed between reference and distorted frames; if only compact reference features are transmitted, SSIM cannot be computed and the model degrades to the seven-dimensional residual vector alone.","fun_headline_variants_meta":{"raw":{"variants":["8-feature XGBoost nears VMAF at one-third the compute","XGBoost on 8 features rivals VMAF at 1/3 the time","Lightweight VQA: 8 features predict quality near VMAF at 1/3 cost","8 features beat PSNR, approach VMAF, at 1/3 cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001224,"raw_usage":{"total_tokens":5027,"prompt_tokens":937,"completion_tokens":4090,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":3993}},"tokens_in":553,"tokens_out":4090,"duration_ms":28201,"temperature":1.0,"reasoning_tokens":3993,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:40:11.941742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the challenge test set with the SSIM component removed, keeping only the seven VCA residuals; if the Pearson correlation falls below PSNR-Y's 0.677, the structural term is load-bearing and the method is effectively near-full-reference rather than reduced-reference.","supporting_citations":[{"cited_title":"Transcoding Quality Prediction for Adaptive Video Streaming,","cited_arxiv_id":null,"evidence_quote":"Defines the Video Complexity Analyzer that supplies the seven-dimensional spatio-temporal features used in the residual vector."},{"cited_title":"On the Computation of PSNR for a Set of Images or Video,","cited_arxiv_id":null,"evidence_quote":"Introduces the XGBoost gradient-boosted tree algorithm that regresses the fused feature vector to a quality score."},{"cited_title":"Dataset The Image and Video Engineering (LIVE) lab at the Univer- sity of Texas at Austin (sponsored by Amazon Prime Video) created the dataset provided for this challenge","cited_arxiv_id":null,"evidence_quote":"Defines the Structural Similarity Index used as the mean SSIM component of the feature vector."},{"cited_title":"The proposed method fuses handcrafted low-level complexity fea- tures extracted via VCA with structural quality cues obtained from SSIM statistics","cited_arxiv_id":null,"evidence_quote":"VMAF is the full-reference metric whose accuracy the method approaches while running substantially faster."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PSNR-Y is the classical metric the proposed method outperforms on correlation with subjective scores."},{"cited_title":"Perceptual visual quality metrics: A survey,","cited_arxiv_id":null,"evidence_quote":"P.1204.3 is the bitstream-based standard that represents the high-accuracy baseline the method approaches."},{"cited_title":"Image information and vi- sual quality,","cited_arxiv_id":null,"evidence_quote":"Provides the information-theoretic link treating SSIM as an estimate of normalized mutual information."}],"review_version":1}