Rate-Distortion Optimization with Non-Reference Metrics for UGC Compression

Antonio Ortega; Balu Adsumilli; Eduardo Pavez; Neil Birkbeck; Samuel Fern\'andez-Mendui\~na; Xin Xiong

REVIEW 3 major objections 5 minor 43 references

Rate-Distortion Optimization with Non-Reference Metrics for UGC Compression

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A linearized no-reference metric makes block-wise rate-distortion optimization practical for noisy UGC.

desk verdict A practical, well-executed method for injecting no-reference metric gradients into encoder RDO; the main gap is the unmeasured trust in the first-order surrogate at low rates. read the letter →

arxiv 2505.15003 v1 pith:GAOKTDT6 submitted 2025-05-21 eess.IV eess.SP

Samuel Fernández-Menduiña , Xin Xiong , Eduardo Pavez , Antonio Ortega , Neil Birkbeck , Balu Adsumilli This is my paper · ORCID

classification eess.IVeess.SP

keywords rate-distortionoptimizationno-referencequalityassessmentuser-generatedcontentcompressiongradientlinearizationtransform-domainRDOBRISQUEARNIQAVSFA

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Conventional encoders optimize a full-reference distortion such as sum of squared errors, which drives the codec to preserve noise and artifacts in user-generated content. This paper proposes replacing that distortion with a no-reference quality metric, linearized around the uncompressed input via a first-order Taylor expansion, so that the resulting cost is additive and can be evaluated block-wise in the transform domain. The linear term measures how the quantization error aligns with the gradient of the no-reference metric, and a sum-of-squared-errors term is added to prevent large deviations at low bitrates. In AVC experiments on noisy UGC images and videos, the method reports average bitrate savings of more than 30 percent relative to SSE-RDO when quality is measured by the target metric (for example -58.94% for BRISQUE on UGC images), with no decoder change and an encoder runtime increase of 2-4 percent.

What carries the argument

The central object is the linearized no-reference metric (LNRM) cost $d(x,\hat{x}) = \nabla b(x)^\top(\hat{x}-x) + \tau\|\hat{x}-x\|_2^2$. The gradient $\nabla b(x)$ is computed once by automatic differentiation; the first term scores the alignment of the reconstruction error with the direction of maximum change of the metric, and the second term with weight $\tau$ keeps the reconstruction close to the input when the linearization is unreliable. In the transform domain this becomes $d(z_i,\hat{z}_i(\theta_i)) = t_i(x)^\top(\hat{z}_i(\theta_i)-z_i) + \tau\|\hat{z}_i(\theta_i)-z_i\|_2^2$ with $t_i(x)=U^\top\nabla b_i(x)$, which is what makes block-wise RDO practical. The Lagrange multiplier is set as $\lambda = \tau\, c\,2^{(QP-12)/3}$, mirroring SSE-RDO, and the normalized regularizer $\tilde{\tau}=2\|\nabla b(x)\|_2/(\sqrt{n_p}\,\Delta)$ equalizes worst-case LNRM and SSE terms under uniform quantization.

What would settle it

At very low bitrates (large quantization steps), encode UGC content with LNRM-RDO and compare the resulting target NRM against SSE-RDO; the bitrate where the savings first turn negative marks the breakdown point of the linearization, and if that point lies inside the practical operating range, the method's central assumption fails. A complementary check is to compare LNRM-RDO against a version that recomputes the gradient on each P frame after reconstruction; savings differences between the two expose staleness of the fixed gradient.

Watch

Extended reading notes

Core claim

The paper's central claim is that the distortion $d(x,\hat{x}) = \nabla b(x)^\top(\hat{x}-x) + \tau\|\hat{x}-x\|_2^2$, obtained by linearizing a no-reference metric $b$ around the uncompressed input $x$, can serve as the distortion term in rate-distortion optimization. Because the gradient $\nabla b(x)$ is computed once on the input, the cost separates into per-block terms and can be evaluated in the transform domain using an orthogonal transform and Parseval's identity. This removes the need to encode, decode, and evaluate the metric for every coding option, which had made NRM-based RDO impractical. The paper shows experimentally that with this cost function, AVC encoders allocate bits to reduce the target NRM rather than to reproduce input noise, achieving BD-rate savings of $-58.94\%$ (BRISQUE-RDO), $-57.26\%$ (ARNIQA-RDO), and $-34.53\%$ (VSFA-RDO) on UGC content, with the SSE regularization parameter and Lagrange multiplier derived analogously to SSE-RDO.

Load-bearing premise

The load-bearing premise is that a first-order Taylor expansion of the no-reference metric around the uncompressed input stays accurate enough over the range of quantization errors encountered at the operating bitrates, so that a single gradient computed on the input predicts which reconstruction errors the metric will reward or penalize.

Editorial extensions

If this is right

The same linearization can be plugged into any differentiable no-reference metric, so an encoder can be targeted at a specific quality measure without retraining the codec.
Because only the RDO cost is modified, the compressed bitstream remains standard-compliant and existing decoders work unchanged.
The approach carries over to codecs with richer RDO option sets, such as VVC and AV1, where the block-wise cost has more decisions to guide.
Lower values of $\tau$ favor the target NRM (larger bitrate savings in that metric), while higher $\tau$ preserves fidelity to the input; the operator can choose the operating point.
The method also improves target-NRM coding efficiency on pristine content (for example KODAK), so it is not limited to noisy UGC.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to refresh the gradient periodically on P frames rather than computing it once on the I frame; if inter-coded residual errors make the fixed gradient stale, savings on longer GOPs would grow with refreshing.
The reported savings are measured by the same NRM used inside the cost function; gains measured by a different metric or by subjective opinion may be smaller, so the claim should not be read as a general perceptual-quality win.
The same linearization recipe could be applied to full-reference metrics that are not block-additive, opening a route to feature-preserving or machine-vision-oriented RDO without per-option decoding.
The authors suggest training learned codecs with the linearized cost as a training loss; an alternative is to test whether end-to-end training against the true NRM outperforms the linearized surrogate, which would show whether the approximation is the limiting factor.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a solid methods paper with a genuine new idea. The linearized-NRM distortion (Eq. 13) is a clean way to make any differentiable no-reference metric usable in block-wise transform-domain RDO, and the authors back it with explicit expressions for tau and lambda, a sensible SSE regularizer, and a transparent experimental setup. The reported savings are measured on the target metric, and the paper is honest that the best results for each metric come from the RDO targeting that metric (Fig. 4). That is the point of the method, not a hidden bug.

The derivation is largely sound; the only issue I see is a typo in Proposition 3.1's proof, where 'maximum quantization error is n_p Delta/2' should read sqrt(n_p) Delta/2. Equation (14) is consistent with the correct version, so this is cosmetic.

The experiment set is credible: 85 UGC frames, 24 KODAK, 85 video GOPs, standard errors on RD curves, and three alpha settings. The runtime numbers (2-4% encoder overhead, no decoder change) make the contribution practical. The ablations across alpha show the NRM term is doing real work, and the PSNR changes are small, which suggests the method isn't just throwing away fidelity.

The soft spots: first, the paper never directly checks how well the linearized cost correlates with the true NRM difference over the operating QP range. The authors themselves note the linearization is only reliable at high bitrates, yet the biggest reported savings occur at the weakest regularization (alpha = tau/2). A simple scatter plot of LNRM cost vs. actual b(hat x)-b(x) would settle whether the surrogate is trustworthy where it matters. Second, the evaluation is single-metric: BD-rate savings are computed on the same NRM being optimized. That's a stated limitation, not an oversight, but it means we don't know if the gains transfer to subjective quality or to a held-out metric. Third, for P frames the gradient is computed on the uncompressed frame, and the reconstruction error after inter prediction can be far from the linearization point; the paper does not isolate this.

None of these sink the paper. The central mechanism is plausible and the empirical evidence is consistent with it. I would send it to a serious referee, with the request to ask for a surrogate-vs-truth correlation analysis and a non-selected alpha operating point.

Who this is for: anyone working on perceptual/quality-aware RDO for UGC, especially codec-side quality optimization. It deserves a proper peer review.

Referee Report

3 major / 5 minor

Summary. The paper proposes LNRM-RDO, a rate-distortion optimization (RDO) cost for block-based video/image coding that linearizes a no-reference quality metric (NRM) around the uncompressed input, adds an SSE regularization term, and derives expressions for the regularization parameter and the Lagrangian multiplier. The cost is evaluated block-wise in the transform domain, enabling AVC encoding with modified RDO while leaving the decoder unchanged. Experiments with BRISQUE, ARNIQA, and VSFA on KODAK and YouTube-UGC content report bitrate savings of more than 30% over SSE-RDO when the target NRM is used as the evaluation metric, with an encoder runtime increase of 2–4%.

Significance. If the linearization of the NRM is sufficiently accurate over the operating range, the method offers a practical and elegant way to incorporate non-reference metrics into block-level RDO without iterative encode-decode loops. The derivation is clean, the authors are transparent about the α grid and report standard errors on the RD curves, and the complexity analysis is concrete (one gradient computation per frame, 2–4% encoder overhead). The main caveat is that the same NRM is used both to construct the optimization cost and to evaluate the outcome, so the reported gains are metric-specific; the paper does not directly measure how well the linearized cost approximates the true NRM difference, which is central to interpreting the savings.

major comments (3)

[§3.2–3.3, §4.1] The manuscript acknowledges in Sec. 3.3 that the Taylor expansion is only reliable at high bitrates and may lead to spurious results as rate decreases, yet the largest BD-rate savings in Tables 1 and 2 are reported at the weakest SSE regularization (α=τ̃/2 for images, and the only video row uses α=τ̃), where the linear term dominates. No experiment isolates the accuracy of the linearization: the paper does not report the correlation between the LNRM cost in Eq. (13) and the true NRM difference b(x̂)−b(x) over the set of candidate coding options at the operating QPs. Since the reported savings are measured with the same NRM that defines the gradient, a direct validation of the surrogate would establish whether the gains reflect genuine improvement in the target metric or an artifact of optimizing a potentially inaccurate proxy. I ask for a scatter plot or correlation analysis at a representative QP, and ideally a comparison of the coding decisions made by LNRM-RDO against an oracle that evaluates the true NRM for each option.
[§3.3, Proposition 3.1] The proof of Proposition 3.1 states that "The maximum quantization error is n_p Δ/2," which should be √n_p Δ/2 because the l2 norm of a vector whose n_p entries each attain the maximum magnitude Δ/2 is √n_p Δ/2. Equation (14) is consistent with the corrected value, so this appears to be a typo in the proof text, but it should be fixed for correctness.
[§4.1, Tables 1 and 2] The headline claim of >30% savings is obtained by selecting, for each target metric, the best of three α values ({2, 1, 1/2}). While reporting all α values is transparent, the paper does not specify how α would be chosen in practice for a given metric across different content, nor does it demonstrate that the savings are robust to α within the tested range. Please add a statement on the intended selection rule for α (e.g., fixed per metric, content-adaptive) and, if possible, show the BD-rate variation across the α grid for each dataset to assess sensitivity.

minor comments (5)

[§2.1, Eq. (1)] The notation Θ∈N^{n_b} is imprecise; Θ is a set of vectors, so it should be Θ ⊂ N^{n_b} (or Θ ⊆ {1,...,n_r}^{n_b}) to avoid the appearance that Θ is a natural number.
[§3.3, Eq. (15)] The symbol c in Eq. (15) is used without restating that it is the same constant as in Eq. (4); a brief parenthetical clarification would help the reader connect the two equations.
[§4.1, Fig. 5] The colorbar in Fig. 5(b) for the BRISQUE gradient ranges from −5 to 5, but the gradient values are not described; please specify the normalization or the units of the displayed gradient.
[§4.1, video configuration] The text states that for P frames the modified RDO is used only for block partitioning; please clarify how intra blocks inside P frames are handled (e.g., are they optimized with the same LNRM cost or with SSE?).
[§3.3, α selection] The paper says "we vary the value of α to explore different trade-offs," but only three values are tested; a sentence justifying the choice of these particular multipliers (e.g., based on the normalization in Proposition 3.1) would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the linearized NRM is an approximation evaluated against the true NRM, and the reported savings are empirical outcomes rather than identities.

full rationale

The paper's central derivation defines a surrogate distortion d(x, x̂) = ∇b(x)ᵀ(x̂−x) + τ‖x̂−x‖² as a first-order Taylor approximation of the target NRM difference b(x̂)−b(x). This is an approximation with stated assumptions, not a restatement of the target metric: the evaluation in Sec. 4 measures the true BRISQUE, ARNIQA, and VSFA scores of decoded content, not the surrogate cost. Because the surrogate can fail to track the true metric, the BD-rate savings are empirically testable and not forced by construction. The cross-metric results reinforce this: BRISQUE-RDO improves BRISQUE by −58.94% while slightly worsening ARNIQA (+0.72%), and ARNIQA-RDO improves ARNIQA by −57.26% while slightly worsening BRISQUE (+0.69%). If the optimization were equivalent to the evaluation by definition, such divergence would not occur. The regularization parameter τ and the scale α are presented as heuristics, and the paper reports results for all tested α values (2τ̃, τ̃, τ̃/2) rather than hiding a fitted parameter as a prediction. The self-citations to prior UGC-compression work (refs. [4], [9]) support the motivational premise but are not load-bearing for the derivation, which rests on standard Taylor expansion, Cauchy–Schwarz, and external rate-distortion theory. The absence of a direct correlation check between LNRM and the true NRM difference is a correctness/robustness limitation, not circularity. The paper is self-contained and benchmarked against an external SSE-RDO baseline on public datasets.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central derivation introduces one hand-tuned scaling α and relies on the standard uniform-quantization noise model and differentiability of the NRM. The τ̃ formula (Eq. 14) is derived from a worst-case normalization heuristic, not from data. The paper reports all α values, so the parameter is disclosed.

free parameters (2)

alpha (SSE regularization scaling) = 2, 1, 1/2 (best results at 1/2 for images, 1 for VSFA video)
Multiplier on the normalized τ̃ in d = LNRM + α τ̃ SSE; grid of three values tested, and the abstract's >30% savings use the best per metric. This is the main hand-chosen parameter.
Lagrangian constant c = unspecified
Standard codec constant in λ = τ c 2^{(QP-12)/3} (Eq. 15); its value is not given in the paper, so exact λ values are not reproducible.

assumptions (5)

domain assumption The NRM is differentiable with respect to input pixels and the autodiff gradient is stable and meaningful
Section 3.2 defines the gradient ∇b(x); Section 3.5 computes it with PyTorch autodiff. BRISQUE is a hand-crafted feature model, so a differentiable implementation is required but not described. Instability or non-smoothness would break the Taylor expansion.
domain assumption Uniform quantization noise has zero mean and variance Δ²/12
Used in Section 3.3 to derive the Lagrange multiplier λ = τ c 2^{(QP-12)/3}; also the derivation of τ̃ uses the worst-case per-sample quantization error Δ/2.
domain assumption Block-wise independence of RDO (each block's reconstruction depends only on its own θ_i)
Standard in video coding; used in Eq. (3) and (11) to decompose the global problem into per-block decisions.
standard math Orthogonal transform preserves the l2 norm and the linear term, so the cost can be evaluated in the transform domain
Section 3.4 uses Parseval's theorem with orthogonal transform U to rewrite d(zi, ẑi).
domain assumption The first-order Taylor approximation of the NRM is adequate at operating rates; the SSE term with hand-tuned α can rescue inaccurate regimes
Section 3.3 acknowledges the approximation degrades at low bitrates and adds SSE regularization. This is the load-bearing modeling assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Rate-Distortion Optimization with Non-Reference Metrics for UGC Compression." pith.science (2026). https://pith.science/paper/GAOKTDT6

@misc{pith2026250515003,
  author       = {Pith},
  title        = {Pith review of: Rate-Distortion Optimization with Non-Reference Metrics for UGC Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GAOKTDT6}},
  note         = {Machine review of arXiv:2505.15003}
}

read the original abstract

Service providers must encode a large volume of noisy videos to meet the demand for user-generated content (UGC) in online video-sharing platforms. However, low-quality UGC challenges conventional codecs based on rate-distortion optimization (RDO) with full-reference metrics (FRMs). While effective for pristine videos, FRMs drive codecs to preserve artifacts when the input is degraded, resulting in suboptimal compression. A more suitable approach used to assess UGC quality is based on non-reference metrics (NRMs). However, RDO with NRMs as a measure of distortion requires an iterative workflow of encoding, decoding, and metric evaluation, which is computationally impractical. This paper overcomes this limitation by linearizing the NRM around the uncompressed video. The resulting cost function enables block-wise bit allocation in the transform domain by estimating the alignment of the quantization error with the gradient of the NRM. To avoid large deviations from the input, we add sum of squared errors (SSE) regularization. We derive expressions for both the SSE regularization parameter and the Lagrangian, akin to the relationship used for SSE-RDO. Experiments with images and videos show bitrate savings of more than 30\% over SSE-RDO using the target NRM, with no decoder complexity overhead and minimal encoder complexity increase.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 42 canonical work pages

[1]

Rate-Distortion Optimization with Non-Reference Metrics for UGC Compression

INTRODUCTION Non-professional video, also known as user-generated content (UGC), plays a central role in platforms like YouTube and Tik- Tok [1]. UGC is often noisy due to amateur production, sub- par equipment, and prior compression. Once uploaded, service providers usually re-encode these videos at different qualities and resolutions for streaming [2]. ...

work page Pith review arXiv 2025
[2]

Rate-distortion optimization Letxbe the input image withn p pixels and ˆx(θθθ)its compressed version using parametersθ θθ= [θ 1 θ2

PRELIMINARIES 2.1. Rate-distortion optimization Letxbe the input image withn p pixels and ˆx(θθθ)its compressed version using parametersθ θθ= [θ 1 θ2 . . . θnb ]∈Θ, whereΘ∈ Nnb is the set of all possible operating points andn b the number of blocks in the image. Assume every entry ofθθθtakes values in the set {1, . . . , nr}, wheren r denotes the number o...

work page
[3]

RDO with NRMs We denote a given NRM asb(·)

TOW ARDS RDO WITH LINEARIZED NRMS 3.1. RDO with NRMs We denote a given NRM asb(·). In general,b(·)will be a computa- tional metric. We assume lower values mean higher quality. Direct bit allocation to optimize an NRM leads to: θθθ⋆ = arg min θθθ∈Θ b(ˆx(θθθ))−b(x) +λ nbX i=1 ri(ˆxi(θθθ)).(5) Unlike SSE, the proposed distortion termb(ˆx(θθθ))−b(x)can be neg...

work page
[4]

=τ np∆2/12. 3.4. Transform domain evaluation LetUbe an orthogonal transform [29], and define the gradient in the transform domain byt i(x) =U ⊤∇bi(x), fori= 1, . . . , nb. Then, we can re-write (13) block-wise in transform domain as d(zi, ˆzi(θi)) =t i(x)⊤(ˆzi(θi)−z i) +τ∥ ˆzi(θi)−z i∥2 2.(16) withz i =U ⊤xi and ˆzi(θθθ)being the compressed version ofz i ...

work page
[5]

We compare our re- sults with A VC using SSE-RDO

EMPIRICAL EV ALUA TION To guarantee that the compressed image converges to the input as bitrate increases, we use 4:4:4 A VC baseline1. We compare our re- sults with A VC using SSE-RDO. To compress the color channels, we add3to the QP value of the luma channel. We run our experiments on an Intel(R) CPU E5-2667 with a NVIDIA Geforce RTX 3090 (24GB VRAM). F...

work page 2000
[6]

More negative values are better, the best value is in boldface for a given metric

using BRISQUE-RDO (B-RDO) and ARNIQA-RDO (A-RDO). More negative values are better, the best value is in boldface for a given metric. Each method optimizes the metric they target. More regularization improves SSE but worsens the target metric. Method PSNR[%]VSFA[%]BRISQUE[%] VSFA-RDO,2˜τ0.47−24.26−2.71 VSFA-RDO,˜τ1.62−34.53−1.42 Table 2: BD-rate saving for...

work page
[7]

CONCLUSION In this paper, we tackled the challenge of compressing UGC videos by incorporating a non-reference quality metric (NRM) as the distor- tion term in rate-distortion optimization (RDO). Leveraging a Taylor expansion, we derived a distortion term that can be efficiently evalu- ated block-wise in the transform domain while capturing the percep- tua...

work page
[8]

YouTube UGC dataset for video compression research,

Y . Wang, S. Inguva, and B. Adsumilli, “YouTube UGC dataset for video compression research,” inProc. IEEE Intl. Work. on Mult. Signal Process.IEEE, 2019, pp. 1–5

work page 2019

Show all 43 references

[9]

A survey on quality of experience of HTTP adaptive streaming,

M. Seufert, S. Egger, M. Slanina, T. Zinner, et al., “A survey on quality of experience of HTTP adaptive streaming,”IEEE Comms. Surv. & Tutor ., vol. 17, no. 1, pp. 469–492, 2014

2014
[10]

Rate-distortion optimization for video compression,

G. J. Sullivan and T. Wiegand, “Rate-distortion optimization for video compression,”IEEE Signal Process. Mag., vol. 15, no. 6, pp. 74–90, 1998

1998
[11]

Compression of user generated content using denoised refer- ences,

E. Pavez, E. Perez, X. Xiong, A. Ortega, and B. Adsumilli, “Compression of user generated content using denoised refer- ences,” inProc. IEEE Int. Conf. Image Process.IEEE, 2022, pp. 4188–4192

2022
[12]

Multiscale struc- tural similarity for image quality assessment,

Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale struc- tural similarity for image quality assessment,” inProc. Asilo- mar Conf. on Signals, Sys. & Comput.IEEE, 2003, vol. 2, pp. 1398–1402

2003
[13]

Towards perceptually-optimized compression of user generated content (UGC): Prediction of UGC rate-distortion category,

S. Ling, Y . Baveye, P. Le Callet, J. Skinner, and I. Katsavouni- dis, “Towards perceptually-optimized compression of user generated content (UGC): Prediction of UGC rate-distortion category,” inProc. IEEE Intl. Conf. on Mult. and Expo. IEEE, 2020, pp. 1–6

2020
[14]

Rate distortion opti- mization over large scale video corpus with machine learning,

S. John, A. Gadde, and B. Adsumilli, “Rate distortion opti- mization over large scale video corpus with machine learning,” arXiv preprint arXiv:2008.12408, 2020

2008 arXiv
[15]

Predicting the quality of compressed videos with pre-existing distortions,

X. Yu, N. Birkbeck, Y . Wang, C. G. Bampis, et al., “Predicting the quality of compressed videos with pre-existing distortions,” IEEE Trans. Image Process., vol. 30, pp. 7511–7526, 2021

2021
[16]

Rate- distortion optimization with alternative references for UGC video compression,

X. Xiong, E. Pavez, A. Ortega, and B. Adsumilli, “Rate- distortion optimization with alternative references for UGC video compression,” inProc. IEEE Int. Conf. Acoust., Speech, and Signal Process., 2023, pp. 1–5

2023
[17]

Asymptotic perfor- mance of vector quantizers with a perceptual distortion mea- sure,

J. Li, N. Chaddha, and R. M. Gray, “Asymptotic perfor- mance of vector quantizers with a perceptual distortion mea- sure,”IEEE Trans. Inform. Theory, vol. 45, no. 4, pp. 1082– 1091, 1999

1999
[18]

Automatic differentiation in Pytorch,

A. Paszke, S. Gross, S. Chintala, G. Chanan, et al., “Automatic differentiation in Pytorch,” 2017

2017
[19]

Lagrange multiplier selection in hybrid video coder control,

T. Wiegand and B. Girod, “Lagrange multiplier selection in hybrid video coder control,” inProc. IEEE Int. Conf. Image Process.2001, vol. 2, pp. 542–545, IEEE

2001
[20]

Overview of the H.264/A VC video coding standard,

T. Wiegand, G. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H.264/A VC video coding standard,”IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560– 576, July 2003

2003
[21]

No-reference image quality assessment in the spatial domain,

A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,”IEEE Trans. Image Process., vol. 21, no. 12, pp. 4695–4708, 2012

2012
[22]

Arniqa: Learning distortion manifold for image quality as- sessment,

L. Agnolucci, L. Galteri, M. Bertini, and A. Del Bimbo, “Arniqa: Learning distortion manifold for image quality as- sessment,” inProc. IEEE/CVF Wint. Conf. on Apps. of Comp. Vis., 2024, pp. 189–198

2024
[23]

Quality assessment of in-the- wild videos,

D. Li, T. Jiang, and M. Jiang, “Quality assessment of in-the- wild videos,” inProc. ACM Intl. Conf. on Mult., 2019, pp. 2351–2359

2019
[24]

Kodak lossless true color image suite,

E. Kodak, “Kodak lossless true color image suite,”URL http://r0k. us/graphics/kodak, 1993

1993
[25]

Overview of the versatile video coding (VVC) standard and its applications,

B. Bross, Y .-K. Wang, Y . Ye, S. Liu, et al., “Overview of the versatile video coding (VVC) standard and its applications,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 10, pp. 3736–3764, 2021

2021
[26]

A technical overview of A V1,

J. Han, B. Li, D. Mukherjee, C.-H. Chiang, et al., “A technical overview of A V1,”Proc. of the IEEE, vol. 109, no. 9, pp. 1435– 1462, 2021

2021
[27]

Generalized Lagrange multiplier method for solving problems of optimum allocation of resources,

H. Everett III, “Generalized Lagrange multiplier method for solving problems of optimum allocation of resources,”Opera- tions research, vol. 11, no. 3, pp. 399–417, 1963

1963
[28]

Rate-distortion methods for image and video compression,

A. Ortega and K. Ramchandran, “Rate-distortion methods for image and video compression,”IEEE Signal Process. Mag., vol. 15, no. 6, pp. 23–50, Nov. 1998

1998
[29]

Feature- preserving rate-distortion optimization in image coding for ma- chines,

S. Fern ´andez-Mendui˜na, E. Pavez, and A. Ortega, “Feature- preserving rate-distortion optimization in image coding for ma- chines,” inProc. Intl. Works. on Mult. Signal Process., 2024, pp. 1–6

2024
[30]

The dis- parity between optimal and practical Lagrangian multiplier es- timation in video encoders,

D. J. Ringis, Vibhoothi, F. Piti ´e, and A. Kokaram, “The dis- parity between optimal and practical Lagrangian multiplier es- timation in video encoders,”Front. in Signal Process., vol. 3, pp. 1205104, 2023

2023
[31]

Im- age database TID2013: Peculiarities, results and perspectives,

N. Ponomarenko, L. Jin, O. Ieremeiev, V . Lukin, et al., “Im- age database TID2013: Peculiarities, results and perspectives,” Sig. proc.: Image comms., vol. 30, pp. 57–77, 2015

2015
[32]

High-resolution source coding for non-difference distortion measures: multidimen- sional companding,

T. Linder, R. Zamir, and K. Zeger, “High-resolution source coding for non-difference distortion measures: multidimen- sional companding,”IEEE Trans. Inform. Theory, vol. 45, no. 2, pp. 548–561, Mar. 1999

1999
[33]

Defense against adversarial attacks on no-reference image quality models with gradient norm regularization,

Y . Liu, C. Yang, D. Li, J. Ding, and T. Jiang, “Defense against adversarial attacks on no-reference image quality models with gradient norm regularization,” inProc. IEEE/CVF Conf. on Comp. Vis. and Patt. Recog., 2024, pp. 25554–25563

2024
[34]

Rate-distortion theory,

T. Berger, “Rate-distortion theory,”Wiley Encyclopedia of Telecommunications, 2003

2003
[35]

Asymptotically efficient quantizing,

H. Gish and J. Pierce, “Asymptotically efficient quantizing,” IEEE Trans. Inform. Theory, vol. 14, no. 5, pp. 676–683, 1968

1968
[36]

The discrete cosine transform,

G. Strang, “The discrete cosine transform,”SIAM review, vol. 41, no. 1, pp. 135–147, 1999

1999
[37]

Hierarchical training of deep neural networks using early exiting,

Y . Sepehri, P. Pad, A. C. Y ¨uz¨ug¨uler, P. Frossard, and L. A. Dunbar, “Hierarchical training of deep neural networks using early exiting,”IEEE Trans. on Neural Nets. and Learn. Sys., pp. 1–15, 2024

2024
[38]

Uhd-iqa benchmark database: Pushing the bound- aries of blind photo quality assessment,

V . Hosu, L. Agnolucci, O. Wiedemann, D. Iso, and D. Saupe, “Uhd-iqa benchmark database: Pushing the bound- aries of blind photo quality assessment,”arXiv preprint arXiv:2406.17472, 2024

2024 arXiv
[39]

A study on the quantiza- tion scheme in H. 264/A VC and its application to rate control,

S. Ma, W. Gao, D. Zhao, and Y . Lu, “A study on the quantiza- tion scheme in H. 264/A VC and its application to rate control,” inProc. Pac. Rim Conf. on Mult., Part III 5. Springer, 2004, pp. 192–199

2004
[40]

5th Challenge on Learned Image Compression dataset,

“5th Challenge on Learned Image Compression dataset,” On- line, June 2022

2022
[41]

Calculation of average PSNR differences be- tween RD-curves,

G. Bjontegaard, “Calculation of average PSNR differences be- tween RD-curves,”ITU SG16 Doc. VCEG-M33, 2001

2001
[42]

Fast-VQA: Efficient end-to-end video quality assessment with fragment sampling,

H. Wu, C. Chen, J. Hou, L. Liao, et al., “Fast-VQA: Efficient end-to-end video quality assessment with fragment sampling,” inEuro. Conf. on Compt. Vis.Springer, 2022, pp. 538–554

2022
[43]

Nonlinear transform coding,

J. Ball ´e, P. A. Chou, D. Minnen, S. Singh, et al., “Nonlinear transform coding,”IEEE Journal of Sel. Top. in Sig. Process., vol. 15, no. 2, pp. 339–353, 2020

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Rate-Distortion Optimization with Non-Reference Metrics for UGC Compression

INTRODUCTION Non-professional video, also known as user-generated content (UGC), plays a central role in platforms like YouTube and Tik- Tok [1]. UGC is often noisy due to amateur production, sub- par equipment, and prior compression. Once uploaded, service providers usually re-encode these videos at different qualities and resolutions for streaming [2]. ...

work page Pith review arXiv 2025

[2] [2]

Rate-distortion optimization Letxbe the input image withn p pixels and ˆx(θθθ)its compressed version using parametersθ θθ= [θ 1 θ2

PRELIMINARIES 2.1. Rate-distortion optimization Letxbe the input image withn p pixels and ˆx(θθθ)its compressed version using parametersθ θθ= [θ 1 θ2 . . . θnb ]∈Θ, whereΘ∈ Nnb is the set of all possible operating points andn b the number of blocks in the image. Assume every entry ofθθθtakes values in the set {1, . . . , nr}, wheren r denotes the number o...

work page

[3] [3]

RDO with NRMs We denote a given NRM asb(·)

TOW ARDS RDO WITH LINEARIZED NRMS 3.1. RDO with NRMs We denote a given NRM asb(·). In general,b(·)will be a computa- tional metric. We assume lower values mean higher quality. Direct bit allocation to optimize an NRM leads to: θθθ⋆ = arg min θθθ∈Θ b(ˆx(θθθ))−b(x) +λ nbX i=1 ri(ˆxi(θθθ)).(5) Unlike SSE, the proposed distortion termb(ˆx(θθθ))−b(x)can be neg...

work page

[4] [4]

=τ np∆2/12. 3.4. Transform domain evaluation LetUbe an orthogonal transform [29], and define the gradient in the transform domain byt i(x) =U ⊤∇bi(x), fori= 1, . . . , nb. Then, we can re-write (13) block-wise in transform domain as d(zi, ˆzi(θi)) =t i(x)⊤(ˆzi(θi)−z i) +τ∥ ˆzi(θi)−z i∥2 2.(16) withz i =U ⊤xi and ˆzi(θθθ)being the compressed version ofz i ...

work page

[5] [5]

We compare our re- sults with A VC using SSE-RDO

EMPIRICAL EV ALUA TION To guarantee that the compressed image converges to the input as bitrate increases, we use 4:4:4 A VC baseline1. We compare our re- sults with A VC using SSE-RDO. To compress the color channels, we add3to the QP value of the luma channel. We run our experiments on an Intel(R) CPU E5-2667 with a NVIDIA Geforce RTX 3090 (24GB VRAM). F...

work page 2000

[6] [6]

More negative values are better, the best value is in boldface for a given metric

using BRISQUE-RDO (B-RDO) and ARNIQA-RDO (A-RDO). More negative values are better, the best value is in boldface for a given metric. Each method optimizes the metric they target. More regularization improves SSE but worsens the target metric. Method PSNR[%]VSFA[%]BRISQUE[%] VSFA-RDO,2˜τ0.47−24.26−2.71 VSFA-RDO,˜τ1.62−34.53−1.42 Table 2: BD-rate saving for...

work page

[7] [7]

CONCLUSION In this paper, we tackled the challenge of compressing UGC videos by incorporating a non-reference quality metric (NRM) as the distor- tion term in rate-distortion optimization (RDO). Leveraging a Taylor expansion, we derived a distortion term that can be efficiently evalu- ated block-wise in the transform domain while capturing the percep- tua...

work page

[8] [8]

YouTube UGC dataset for video compression research,

Y . Wang, S. Inguva, and B. Adsumilli, “YouTube UGC dataset for video compression research,” inProc. IEEE Intl. Work. on Mult. Signal Process.IEEE, 2019, pp. 1–5

work page 2019