Pith. sign in

REVIEW 3 major objections 5 minor 72 references

Video Decomposition Prior: A Methodology to Decompose Videos into Layers

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single training-free optimization decomposes a video into layers, and dehazing, relighting, and object segmentation emerge from that decomposition.

desk verdict The FlowRGB-based test-time video decomposition is a real and useful extension of DoubleDIP-style internal learning, but the relighting formulation is just gamma correction in log form and the headline numbers need code, error bars, and a synthetic control before the 'emergence' story can be trusted. read the letter →

arxiv 2412.04930 v2 pith:BBJDZUV3 submitted 2024-12-06 cs.CV cs.LG

Gaurav Shrivastava , Ser-Nam Lim , Abhinav Shrivastava This is my paper · ORCID
classification cs.CVcs.LG
keywords videodecompositioninference-timeoptimizationunsupervisedobjectsegmentationdehazingrelightinglayerededitingopticalflowdeepimageprior
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to establish that video dehazing, relighting, and unsupervised object segmentation can be solved as one problem: decomposing a video into layered RGB images with opacity maps, using only the test video itself. No paired input and ground-truth corpus is collected or trained on; the framework optimizes two small convolutional U-Nets directly on the query clip, one reading frames and one reading color-coded optical-flow images. The paper's central proposal is a logarithmic decomposition for relighting in which the well-lit frame, a transmission map, and a trainable gamma factor are separated, and the relighting behavior emerges from optimizing this reconstruction. On the reported benchmarks the recovered layers beat trained baselines by standard similarity measures (27.92 PSNR on SDSD relighting, 24.83 PSNR on REVIDE dehazing; PSNR is peak signal-to-noise ratio) and reach 81.1 Jaccard overlap on DAVIS-16 unsupervised segmentation. If the claim holds, a single framework offers an alternative to collecting task-specific paired video data wherever such data is physically difficult or impossible to capture.

What carries the argument

The load-bearing object is a pair of shallow convolutional U-Nets optimized on a single video: $\mathrm{RGBnet}$ maps each frame $X_t$ to an RGB layer, and $\alpha$-net maps the RGB visualization of the forward optical flow, $F^{\mathrm{RGB}}_{t\to t+1}$, to an opacity or transmission map. These two networks are tied together by a task-specific reconstruction equation—the logarithmic gamma decomposition of Eq. 5 for relighting, Eq. 14's scattering model for dehazing, and Eq. 13's $\alpha$-blending for segmentation—along with a perceptual-plus-$\ell^1$ reconstruction loss and an optical-flow warp loss. The mechanism that selects a meaningful layer is the deep image prior: convolutional U-Nets favor outputs with natural, recurrent patch statistics, so among the many decompositions that can rebuild the input, the optimization is pulled toward the physically plausible well-lit or haze-free layer.

What would settle it

Build a synthetic low-light video by applying a known gamma to a well-lit video, then run the optimization: if the recovered relit layer and gamma match the known values across random initializations, the claim is supported. A sharper negative test is to craft a video whose true scene layer is highly repetitive while another factor satisfying Eq. 5 is even more patch-recurrent; if the optimization returns that alternative factor, the emergent relighting is an artifact of the prior rather than a validated physical decomposition.

Watch

Extended reading notes

Core claim

The paper's central claim, stated on its own terms, is that a proper decomposition formulation is itself a prior: from just the input frames and their forward optical flow, optimizing a reconstruction loss over layered RGB and opacity outputs produces the desired edited layers directly, with no task-specific training data. For relighting the reconstruction is logarithmic, $\log(X^{\mathrm{in}}_t)=\frac{1}{\gamma}\left(\log(1\oslash A_t)+\log(X^{\mathrm{out}}_t)\right)$, with $\gamma^{-1}$ trainable; for dehazing it is the scattering equation $X_t=\alpha\odot \mathrm{Clr}(X_t)+(1-\alpha)\odot A_t$ with the airlight map fixed; for segmentation it is a two-layer $\alpha$-blend with motion and mask regularizers. In each case the paper reports that the intermediate layer of interest—the relit frame, the haze-free frame, or the foreground mask—beats the compared baselines; for dehazing and relighting those baselines include models trained on the benchmark's own data, while the segmentation comparison is against other inference-time and unsupervised methods.

Load-bearing premise

The method's bet is that when many splits of a video into a scene layer and a lighting or transmission layer all reconstruct the input equally well, the networks' built-in preference for natural image statistics, together with the allowed range of the trainable gamma, picks the physically correct well-lit or haze-free layer rather than just any arbitrary factor that happens to reconstruct the input.

Editorial extensions

If this is right

  • Videos where paired clean footage cannot be captured—identical motion and camera angle in both corrupted and clean versions—can still be dehazed, relit, and segmented by optimizing on the test clip alone.
  • Because the decomposition yields editable RGB layers with opacity maps, edits applied to one layer on any keyframe (stickers, stylization, background swap) propagate coherently through the video via optical-flow warping and the learned masks.
  • Relighting emerges from the logarithmic decomposition without a relighting-specific loss, so the framework needs no relighting ground truth at all.
  • Feeding $\alpha$-net the flow's RGB image instead of the frame or noise is what stabilizes the transmission maps; the reported ablation gap identifies motion as the key information source for dehazing and relighting.
  • Setting the number of layers to two or three switches the same framework between foreground and background separation and multi-object decomposition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the logarithmic factorization is as general as it appears, the same $\log X = \gamma^{-1}(\log(1/A)+\log X_{\mathrm{out}})$ structure should transfer to other multiplicative image-formation models, such as intrinsic image decomposition, shadow removal, or color-correction tasks where components multiply rather than add.
  • A sharper test of the paper's emergence claim than benchmark averages would be seed-to-seed stability: re-optimizing the same video from many random initializations and measuring whether the recovered gamma and relit layer are identical, or whether equally good reconstructions land on different factors.
  • The paper's own limitation list—flow-estimator quality, fixed layer count, $2N$ networks, and test-time cost—points to a natural extension: adaptive layer discovery and a fast amortized initialization that predicts good starting weights instead of optimizing from scratch.
  • Real benchmarks like REVIDE and SDSD cannot reveal how often the prior selects the physically correct factor; a synthetic suite with known ground-truth layers would quantify that directly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VDP, an inference-time optimization framework that decomposes a video into RGB layers and opacity maps using two U-Nets, one operating on frames and one on optical-flow RGB. The framework is applied to unsupervised video object segmentation, dehazing, and relighting. The relighting formulation introduces a logarithmic decomposition based on gamma correction and claims that relighting emerges from the optimization. The paper reports state-of-the-art results on SDSD, REVIDE, and DAVIS-16, along with ablations demonstrating the importance of FlowRGB inputs.

Significance. If the claims hold, VDP would be a valuable training-free alternative for multiple video tasks. The UVOS component, in particular, is well-structured, with ablations that quantify the contribution of each loss term. However, the central 'emergent relighting' claim is not supported by the manuscript's evidence, and the quantitative evaluation lacks statistical rigor. The paper would benefit from synthetic validation and a more careful framing of the relighting contribution.

major comments (3)
  1. [Sec. 4, Eq. (5)] Equation (5) is a logarithmic rearrangement of the standard gamma correction in Eq. (4): exponentiating and rearranging Eq. (5) yields Xout = A ⊙ Xin^γ, which is exactly Eq. (4). Thus the decomposition does not introduce a new physical model, and the excellent relighting results are not an 'emergent property' of the optimization but a consequence of fitting a gamma-correction model whose output layer is defined to be the relit frame. The paper should either retract the 'emergent' claim or identify a property that goes beyond gamma correction.
  2. [Sec. 4 and Sec. 10] The relighting objective is underdetermined: for a fixed input X, any candidate relit layer Xout can be paired with a transmission map A = exp(γ log X − log Xout) that satisfies Eq. (5) exactly, so the reconstruction loss does not prefer the physically correct relit video over other factorizations. The only selection mechanism is the U-Net's patch-recurrence prior and the tunable range of γ−1, but this mechanism is asserted in Sec. 10 without validation. To support the central claim, the paper needs a synthetic control experiment with known ground-truth degradation (e.g., a well-exposed video artificially darkened with a known gamma, then recovered), together with a characterization of the distribution of recovered solutions over initializations. Tables 1, 3, and 7 report single PSNR/SSIM values with no error bars or significance tests, so the reported 27.92 dB on SDSD does not establish that the method recovers the true relit layer rather than an arbitrary factorization.
  3. [Tables 1, 3, and 7] The quantitative claims are supported only by averaged PSNR/SSIM numbers computed over the test sets, without per-sequence breakdowns, standard deviations, or significance tests. Phrases such as 'significant margin' in Sections 4 and 6 are therefore unsupported. Please provide variance or per-sequence results, report the number of test sequences, and, if possible, make the code available to allow independent verification.
minor comments (5)
  1. [Abstract] There is a typo: 'We addresses tasks' should be 'We address tasks'.
  2. [Sec. 4] The text says 'we change Eqn. 4 as follows' but Eq. (5) is a reparameterization rather than a change; the relationship should be stated explicitly to avoid overclaiming novelty.
  3. [Sec. 5, Eq. (8)] The constraint ∑_i M_i = J is not enforced by any loss term; the paper should clarify how this condition is guaranteed, especially when L > 2.
  4. [Sec. 6] The dehazing model in Eq. (14) is also underdetermined, and the same identifiability concern applies; please address this in the limitations.
  5. [Sec. 8] The statement that 'consistent quantitative results' were observed across random initializations would be more convincing with actual numbers.

Circularity Check

2 steps flagged · score 3.0 of 10

Relighting is built into Eq. 5 (a log rewrite of gamma correction), but external benchmarks keep the empirical core non-circular.

  1. self definitional [Sec. 4 (Eqs. 4-5) and Sec. 10 (Intuition for Dehazing/Relighting Tasks)]
    "For the relighting task, we begin by defining a manifold for the relit frame, which is based on the gamma correction formulation as delineated in Eqn. 5. ... When we employ the U-Net for optimization, its output (termed as `relit` frame) reliably gravitates towards a natural image marked by repetitive patches."

    Eq. 5 is algebraically the same statement as Eq. 4: log(Xin) = (1/gamma)(log(1./A) + log(Xout)) rearranges to Xout = A .* Xin^gamma. The `relit frame` Xout is therefore defined, by construction, as a gamma-corrected factor of the input; any desired Xout can be paired with a transmission map A that makes reconstruction exact. Consequently, the claimed `emergent` relighting is not a consequence derived from the decomposition objective alone, but is encoded in the chosen decomposition equation and selected by the U-Net's untested inductive bias. The external SDSD benchmark keeps the empirical result meaningful, but the `emergent property` wording overstates what the equations force.

  2. renaming known result [Abstract and Sec. 1 contributions]
    "We introduce a new formulation of the logarithmic decomposition of a video to perform a relighting task. This new formulation leads relighting of video sequence as an emergent property."

    The `logarithmic decomposition` is a log-domain rewrite of standard gamma correction (Eq. 4), which the paper itself cites as the traditional model; taking logs of both sides does not add a new constraint. Presenting it as a new formulation that `leads` to relighting renames the known gamma-correction operation rather than deriving relighting from an independent prior. This is a presentational rename; the benchmark comparisons still provide external evidence.

full rationale

No load-bearing self-citation is present: the citations to Shrivastava et al. (2023) and (2024) are background references for test-time optimization and for the same paper itself, not uniqueness theorems or premises that force the results. The UVOS objective (Eqs. 6-12) and dehazing model (Eq. 14) are standard compositional formulations evaluated against external ground-truth benchmarks (DAVIS, REVIDE, SDSD), so those empirical outcomes are not circular. The main circularity-adjacent issue is the relighting claim: Eq. 5 is a logarithmic rearrangement of Eq. 4, so the relit layer is definitionally a gamma-corrected factor, and the reconstruction loss is underdetermined. The reported PSNR/SSIM improvements are nevertheless measured against independent ground truth, giving the central empirical claim independent content. The score is moderate (3) rather than high because the underdetermination and definitional encoding affect the interpretation of the `emergent` claim but do not by themselves fabricate the benchmark numbers.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The method rests on standard compositing, scattering, and gamma models, a pretrained flow estimator, and an unformalized deep image prior assumption. No new physical entity is introduced; FlowRGB is a standard colorized visualization of optical flow, and layers and transmission maps are standard quantities.

free parameters (3)
  • gamma^{-1} (trainable relighting exponent) = optimized in (0,1) per video
    Sec. 4 and Fig. 2 treat gamma^{-1} as a trainable scalar; the relit output is sensitive to its fitted value.
  • Number of layers L = 2 for UVOS, 3 in one multi-layer example
    L is chosen before optimization (Sec. 5, Sec. 9.3, Sec. 13); decomposition quality depends on it.
  • Loss weights lambda = lambda_rec=1, lambda_Fsim=0.001, lambda_layer=1, lambda_warp=0.01, lambda_Mask=0.01 for UVOS; lambda_rec=1…
    Hand-set in Sec. 8.1; no sensitivity analysis is reported.
assumptions (7)
  • domain assumption Atmospheric scattering model X_t = alpha * Clr(X_t) + (1 - alpha) * A_t (Eq. 14)
    Dehazing assumes hazy video follows this scattering model, with airlight fixed via the Bahat and Irani method.
  • domain assumption Gamma correction model X_out = A_t * (X_in)^gamma (Eq. 4)
    Relighting is modeled as per-pixel gamma correction with a spatially varying factor; Eq. 5 is a logarithmic rearrangement of this model.
  • domain assumption Alpha-blending compositing X_t = sum_i M_i * f_RGB^i(X_t) (Eqs. 6 and 13)
    Segmentation and layer edits assume video frames are weighted sums of layer colors.
  • ad hoc to paper Deep image prior patch recurrence selects natural layers
    Sec. 10 argues U-Net outputs converge toward natural image statistics, which is used to justify that the arbitrary decomposition factor is the clean or relit frame; no formal identifiability guarantee is given.
  • ad hoc to paper VGG embeddings of FlowRGB capture motion grouping
    The flow similarity loss in Eq. 9 assumes cosine similarity between VGG features of colorized flow separates different motions; this is asserted without derivation.
  • domain assumption Pretrained RAFT optical flow is accurate enough
    Sec. 14 uses RAFT for all flow estimates; the paper itself lists flow estimator accuracy as a limitation.
  • standard math Pretrained VGG perceptual features are a valid reconstruction metric
    The perceptual loss in Eq. 2 uses ImageNet-trained VGG embeddings, a standard choice in the literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Video Decomposition Prior: A Methodology to Decompose Videos into Layers." pith.science (2026). https://pith.science/paper/BBJDZUV3

@misc{pith2026241204930,
  author       = {Pith},
  title        = {Pith review of: Video Decomposition Prior: A Methodology to Decompose Videos into Layers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBJDZUV3}},
  note         = {Machine review of arXiv:2412.04930}
}
read the original abstract

In the evolving landscape of video enhancement and editing methodologies, a majority of deep learning techniques often rely on extensive datasets of observed input and ground truth sequence pairs for optimal performance. Such reliance often falters when acquiring data becomes challenging, especially in tasks like video dehazing and relighting, where replicating identical motions and camera angles in both corrupted and ground truth sequences is complicated. Moreover, these conventional methodologies perform best when the test distribution closely mirrors the training distribution. Recognizing these challenges, this paper introduces a novel video decomposition prior `VDP' framework which derives inspiration from professional video editing practices. Our methodology does not mandate task-specific external data corpus collection, instead pivots to utilizing the motion and appearance of the input video. VDP framework decomposes a video sequence into a set of multiple RGB layers and associated opacity levels. These set of layers are then manipulated individually to obtain the desired results. We addresses tasks such as video object segmentation, dehazing, and relighting. Moreover, we introduce a novel logarithmic video decomposition formulation for video relighting tasks, setting a new benchmark over the existing methodologies. We observe the property of relighting emerge as we optimize for our novel relighting decomposition formulation. We evaluate our approach on standard video datasets like DAVIS, REVIDE, & SDSD and show qualitative results on a diverse array of internet videos. Project Page - https://www.cs.umd.edu/~gauravsh/video_decomposition/index.html for video results.

Figures

Figures reproduced from arXiv: 2412.04930 by the authors.

Figure 1
Figure 1. Visual representation of video edits obtained using VDP. The first row demonstrates a foreground manipulation example that leverages an object mask as the decomposition guide. This object mask is obtained by performing Unsupervised Video Object Segmentation (UVOS), which is a downstream task of video de￾composition and is achieved using our proposed framework. Our approach effectively separates the foreground object… view at source ↗
Figure 2
Figure 2. VDP for relighting a video sequence. In this pipeline, the input video frame t is fed into a shallow U-Net denoted by f (1) RGB. While the flow-RGB is given as input to a separate shallow U-Net network denoted by f (1) α . The intermediate output of f (1) RGB is a re-lit version of the input frame t. While the output of f (1) α is transmission maps or Tmap denoted by 1/At. It is important to note that γ −1 is also t… view at source ↗
Figure 3
Figure 3. Qualitative evaluation on Video Relighting benchmark: We compare the re-lit result of our method with the baselines on the SDSD (58) dataset. We compare our method against both the image and video baselines. ZeroDCE++ (26) method is a image-based baseline while SDSD (58) and Stablellve (65) are video baselines. Please note that we have utilized the pretrained models of all the baselines to obtain qualitative results… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Our Framework: In this figure, we present the pipeline for the task of decomposing the video into two different components. For the above configuration, the input video frame t is fed into two different shallow U-Nets denoted by f (1) RGB and f (2) RGB respectively. We…
Figure 5
Figure 5. Figure 5: Qualitative evaluation on VOS benchmark: We compare the mask from our method with the baselines on DAVIS-16 (35) dataset. We compare our method against the baselines DS (64) and DyStab (62). Please note the above masks (exception: DoubleDIP) for baselines are pre￾compu…
Figure 6
Figure 6. Figure 6: Edits propagation: First column in the figure represents the input frame to be edited. Second column represents the edits performed. Next columns in Fig. (a) represent the qualitative comparison of propagated edits with baselines, while Fig. (b) represents propagated e…
Figure 7
Figure 7. Figure 7: Qualitative evaluation on Video Dehazing benchmark: We compare the dehazed result of our method with the baselines on the REVIDE (67) dataset. We compare our method against both the image and video baselines. MSBDN (11) method is an image-based baseline while EDVR (59)…
Figure 8
Figure 8. Figure 8: Comparison with pretrained image segmentation model: The examples presented in the figure above are samples from the MoCA dataset (Lamdouar et al., 2020). In comparison to MaskRCNN (He et al., 2017), a pretrained image segmentation model trained on the COCO dataset, ou…
Figure 9
Figure 9. Figure 9: (a) Segmentation in variable lighting conditions throughout the scene. We compare the mask from our method with the baselines Deformable sprites (Ye et al., 2022) and MG (Yang et al., 2021a) on frames of three video sequences. It can be observed from the above figure m…
Figure 10
Figure 10. Figure 10: Ablation Study: We study the effect of using only a subset of the loss function for the video object segmentation task. Low-lit Input Ours w/ Flow Ours w/ Gaussian Noise Ours w/ RGB Frame Hazy Input Ours w/ Flow Ours w/ Gaussian Noise Ours w/ RGB Frame (a) (b) [PITH_…
Figure 11
Figure 11. Figure 11: Ablation Study:This figure presents the results of an ablation study where we analyzed the impact of using FlowRGB, Gaussian Noise, and RGB frame as input to the α-net. Row (a) shows a qualitative comparison for relighting tasks, where we evaluated three different set…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 56 canonical work pages

  1. [1]

    zero-shot

    Michal Irani Assaf Shocher, Nadav Cohen. "zero-shot" super-resolution using deep internal learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    work page 2018

  2. [2]

    Blind dehazing using internal patch recurrence

    Yuval Bahat and Michal Irani. Blind dehazing using internal patch recurrence. In 2016 IEEE International Conference on Computational Photography (ICCP), pp.\ 1--9. IEEE, 2016

    work page 2016

  3. [3]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In European Conference on Computer Vision, pp.\ 707--723. Springer, 2022

    work page 2022

  4. [4]

    Blind super-resolution kernel estimation using an internal-gan

    Sefi Bell-Kligler, Assaf Shocher, and Michal Irani. Blind super-resolution kernel estimation using an internal-gan. Advances in Neural Information Processing Systems, 32, 2019

    work page 2019

  5. [5]

    It’s moving! a probabilistic model for causal motion segmentation in moving camera videos

    Pia Bideau and Erik Learned-Miller. It’s moving! a probabilistic model for causal motion segmentation in moving camera videos. In European Conference on Computer Vision, pp.\ 433--449. Springer, 2016

    work page 2016

  6. [6]

    Hierarchical video prediction using relational layouts for human-object interactions

    Navaneeth Bodla, Gaurav Shrivastava, Rama Chellappa, and Abhinav Shrivastava. Hierarchical video prediction using relational layouts for human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 12146--12155, 2021

    work page 2021

  7. [7]

    Object segmentation by long term analysis of point trajectories

    Thomas Brox and Jitendra Malik. Object segmentation by long term analysis of point trajectories. In European conference on computer vision, pp.\ 282--295. Springer, 2010

    work page 2010

  8. [8]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 6299--6308, 2017

    2017

Show all 72 references
  1. [9]

    Unsupervised learning from video to detect foreground objects in single images

    Ioana Croitoru, Simion-Vlad Bogolin, and Marius Leordeanu. Unsupervised learning from video to detect foreground objects in single images. In Proceedings of the IEEE International Conference on Computer Vision, pp.\ 4335--4343, 2017

    2017

  2. [10]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database . In CVPR09, 2009

    2009

  3. [11]

    Multi-scale boosted dehazing network with dense feature fusion

    Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, and Ming-Hsuan Yang. Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 2157--2167, 2020

    2020

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

    2010 arXiv

  5. [13]

    Exploiting geometric constraints on dense trajectories for motion saliency

    Muhammad Faisal, Ijaz Akhter, Mohsen Ali, and Richard Hartley. Exploiting geometric constraints on dense trajectories for motion saliency. arXiv preprint arXiv:1909.13258, 3 0 (4), 2019

    1909 arXiv

  6. [14]

    Video segmentation by non-local consensus voting

    Alon Faktor and Michal Irani. Video segmentation by non-local consensus voting. In BMVC, number 7, pp.\ 8, 2014

    2014

  7. [15]

    Single image dehazing

    Raanan Fattal. Single image dehazing. ACM transactions on graphics (TOG), 27 0 (3): 0 1--9, 2008

    2008

  8. [16]

    Geodict: an integrated gazetteer

    Jacques Fize, Gaurav Shrivastava, and Pierre Andr \'e M \'e nard. Geodict: an integrated gazetteer. In Proceedings of Language, Ontology, Terminology and Knowledge Structures Workshop (LOTKS 2017), 2017

    2017

  9. [17]

    double-dip

    Yosef Gandelsman, Assaf Shocher, and Michal Irani. " double-dip": Unsupervised image decomposition via coupled deep-image-priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 11026--11035, 2019

    2019

  10. [18]

    Single image haze removal using dark channel prior

    Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, 33 0 (12): 0 2341--2353, 2010

    2010

  11. [19]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll \'a r, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp.\ 2961--2969, 2017

    2017

  12. [20]

    Robust interpolation of correspondences for large displacement optical flow

    Yinlin Hu, Yunsong Li, and Rui Song. Robust interpolation of correspondences for large displacement optical flow. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 481--489, 2017

    2017

  13. [21]

    Layered neural atlases for consistent video editing

    Yoni Kasten, Dolev Ofri, Oliver Wang, and Tali Dekel. Layered neural atlases for consistent video editing. ACM Transactions on Graphics (TOG), 40 0 (6): 0 1--12, 2021

    2021

  14. [22]

    Motion trajectory segmentation via minimum cost multicuts

    Margret Keuper, Bjoern Andres, and Thomas Brox. Motion trajectory segmentation via minimum cost multicuts. In Proceedings of the IEEE international conference on computer vision, pp.\ 3271--3279, 2015

    2015

  15. [23]

    Primary object segmentation in videos based on region augmentation and reduction

    Yeong Jun Koh and Chang-Su Kim. Primary object segmentation in videos based on region augmentation and reduction. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 7417--7425. IEEE, 2017

    2017

  16. [24]

    Imagenet classification with deep convolutional neural networks

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012

    2012

  17. [25]

    Betrayed by motion: Camouflaged object discovery via motion segmentation

    Hala Lamdouar, Charig Yang, Weidi Xie, and Andrew Zisserman. Betrayed by motion: Camouflaged object discovery via motion segmentation. Asian Conference on Computer Vision, 2020

    2020

  18. [26]

    Learning to enhance low-light image via zero-reference deep curve estimation

    Chongyi Li, Chunle Guo, and Chen Change Loy. Learning to enhance low-light image via zero-reference deep curve estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. doi:10.1109/TPAMI.2021.3063604

    2021

  19. [27]

    Layered neural rendering for retiming people in video

    Erika Lu, Forrester Cole, Tali Dekel, Weidi Xie, Andrew Zisserman, David Salesin, William T Freeman, and Michael Rubinstein. Layered neural rendering for retiming people in video. arXiv preprint arXiv:2009.07833, 2020 a

    2009 arXiv

  20. [28]

    Omnimatte: Associating objects and their effects in video

    Erika Lu, Forrester Cole, Tali Dekel, Andrew Zisserman, William T Freeman, and Michael Rubinstein. Omnimatte: Associating objects and their effects in video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4507--4515, 2021

    2021

  21. [29]

    Zero-shot video object segmentation with co-attention siamese networks

    Xiankai Lu, Wenguan Wang, Jianbing Shen, David Crandall, and Jiebo Luo. Zero-shot video object segmentation with co-attention siamese networks. IEEE transactions on pattern analysis and machine intelligence, 44 0 (4): 0 2228--2242, 2020 b

    2020

  22. [30]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65 0 (1): 0 99--106, 2021

    2021

  23. [31]

    Narasimhan and S.K

    S.G. Narasimhan and S.K. Nayar. Chromatic framework for vision in bad weather. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662), volume 1, pp.\ 598--605 vol.1, 2000. doi:10.1109/CVPR.2000.855874

  24. [32]

    Object segmentation in video: a hierarchical variational approach for turning point trajectories into dense regions

    Peter Ochs and Thomas Brox. Object segmentation in video: a hierarchical variational approach for turning point trajectories into dense regions. In 2011 international conference on computer vision, pp.\ 1583--1590. IEEE, 2011

    2011

  25. [33]

    Fast object segmentation in unconstrained video

    Anestis Papazoglou and Vittorio Ferrari. Fast object segmentation in unconstrained video. In Proceedings of the IEEE international conference on computer vision, pp.\ 1777--1784, 2013

    2013

  26. [34]

    Swapping autoencoder for deep image manipulation

    Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei Efros, and Richard Zhang. Swapping autoencoder for deep image manipulation. Advances in Neural Information Processing Systems, 33: 0 7198--7211, 2020

    2020

  27. [35]

    Perazzi, J

    F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool , M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Computer Vision and Pattern Recognition, 2016

    2016

  28. [36]

    Across scales and across dimensions: Temporal super-resolution using deep internal learning

    Liad Pollak Zuckerman, Eyal Naor, George Pisha, Shai Bagon, and Michal Irani. Across scales and across dimensions: Temporal super-resolution using deep internal learning. In European Conference on Computer Vision (ECCV). Springer, 2020

    2020

  29. [37]

    Competitive collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation

    Anurag Ranjan, Varun Jampani, Lukas Balles, Kihwan Kim, Deqing Sun, Jonas Wulff, and Michael J Black. Competitive collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation. In Proceedings of the IEEE/CVF conference on computer vis...

    2019

  30. [38]

    Valorcarn-tetis: Terms extracted with biotex

    Mathieu Roche, Maguelonne Teisseire, and Gaurav Shrivastava. Valorcarn-tetis: Terms extracted with biotex. 2017

    2017

  31. [39]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp.\ 234--241. Springer, 2015

    2015

  32. [40]

    Recognizing actions using object states

    Nirat Saini, Bo He, Gaurav Shrivastava, Sai Saketh Rambhatla, and Abhinav Shrivastava. Recognizing actions using object states. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality, 2022

    2022

  33. [41]

    Layered depth images

    Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. Layered depth images. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pp.\ 231--242, 1998

    1998

  34. [42]

    Increasing space-time resolution in video

    Eli Shechtman, Yaron Caspi, and Michal Irani. Increasing space-time resolution in video. In Computer Vision—ECCV 2002: 7th European Conference on Computer Vision Copenhagen, Denmark, May 28--31, 2002 Proceedings, Part I 7, pp.\ 753--768. Springer, 2002

    2002

  35. [43]

    Space-time super-resolution

    Eli Shechtman, Yaron Caspi, and Michal Irani. Space-time super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 0 (4): 0 531--545, 2005

    2005

  36. [44]

    Diverse Video Generation

    Gaurav Shrivastava. Diverse Video Generation. PhD thesis, University of Maryland, College Park, 2021

    2021

  37. [45]

    Advance Video Modeling Techniques for Video Generation and Enhancement Tasks

    Gaurav Shrivastava. Advance Video Modeling Techniques for Video Generation and Enhancement Tasks. PhD thesis, University of Maryland, College Park, 2024

    2024

  38. [46]

    Diverse video generation using a gaussian process trigger

    Gaurav Shrivastava and Abhinav Shrivastava. Diverse video generation using a gaussian process trigger. arXiv preprint arXiv:2107.04619, 2021

    2021 arXiv

  39. [47]

    Video prediction by modeling videos as continuous multi-dimensional processes

    Gaurav Shrivastava and Abhinav Shrivastava. Video prediction by modeling videos as continuous multi-dimensional processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  40. [48]

    Video dynamics prior: An internal learning approach for robust video enhancements

    Gaurav Shrivastava, Ser-Nam Lim, and Abhinav Shrivastava. Video dynamics prior: An internal learning approach for robust video enhancements. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  41. [49]

    Video decomposition prior: Editing videos layer by layer

    Gaurav Shrivastava, Ser-Nam Lim, and Abhinav Shrivastava. Video decomposition prior: Editing videos layer by layer. In The Twelfth International Conference on Learning Representations, 2024

    2024

  42. [50]

    Object level grouping for video shots

    Josef Sivic, Frederik Schaffalitzky, and Andrew Zisserman. Object level grouping for video shots. International Journal of Computer Vision, 67 0 (2): 0 189--210, 2006

    2006

  43. [51]

    Pyramid dilated deeper convlstm for video salient object detection

    Hongmei Song, Wenguan Wang, Sanyuan Zhao, Jianbing Shen, and Kin-Man Lam. Pyramid dilated deeper convlstm for video salient object detection. In Proceedings of the European conference on computer vision (ECCV), pp.\ 715--731, 2018

    2018

  44. [52]

    Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 8934--8943, 2018

    2018

  45. [53]

    Object segmentation by long term analysis of point trajectories

    T.Brox and J.Malik. Object segmentation by long term analysis of point trajectories. In European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science. Springer, Sept. 2010. URL http://lmb.informatik.uni-freiburg.de/Publications/2010/Bro10c

    2010

  46. [54]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In European conference on computer vision, pp.\ 402--419. Springer, 2020

    2020

  47. [55]

    Learning motion patterns in videos

    Pavel Tokmakov, Karteek Alahari, and Cordelia Schmid. Learning motion patterns in videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 3386--3394, 2017

    2017

  48. [56]

    Deep image prior

    Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 9446--9454, 2018

    2018

  49. [57]

    Representing moving images with layers

    John YA Wang and Edward H Adelson. Representing moving images with layers. IEEE transactions on image processing, 3 0 (5): 0 625--638, 1994

    1994

  50. [58]

    Seeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment

    Ruixing Wang, Xiaogang Xu, Chi-Wing Fu, Jiangbo Lu, Bei Yu, and Jiaya Jia. Seeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 9700--9709, 2021

    2021

  51. [59]

    Edvr: Video restoration with enhanced deformable convolutional networks

    Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. Edvr: Video restoration with enhanced deformable convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.\ 0--0, 2019

    2019

  52. [60]

    Self-supervised video object segmentation by motion grouping

    Charig Yang, Hala Lamdouar, Erika Lu, Andrew Zisserman, and Weidi Xie. Self-supervised video object segmentation by motion grouping. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 7177--7188, 2021 a

    2021

  53. [61]

    Unsupervised moving object detection via contextual information separation

    Yanchao Yang, Antonio Loquercio, Davide Scaramuzza, and Stefano Soatto. Unsupervised moving object detection via contextual information separation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 879--888, 2019 a

    2019

  54. [62]

    Dystab: Unsupervised object segmentation via dynamic-static bootstrapping

    Yanchao Yang, Brian Lai, and Stefano Soatto. Dystab: Unsupervised object segmentation via dynamic-static bootstrapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 2826--2836, 2021 b

    2021

  55. [63]

    Anchor diffusion for unsupervised video object segmentation

    Zhao Yang, Qiang Wang, Luca Bertinetto, Weiming Hu, Song Bai, and Philip HS Torr. Anchor diffusion for unsupervised video object segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 931--940, 2019 b

    2019

  56. [64]

    Deformable sprites for unsupervised video decomposition

    Vickie Ye, Zhengqi Li, Richard Tucker, Angjoo Kanazawa, and Noah Snavely. Deformable sprites for unsupervised video decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 2657--2666, 2022

    2022

  57. [65]

    Learning temporal consistency for low light video enhancement from single images

    Fan Zhang, Yu Li, Shaodi You, and Ying Fu. Learning temporal consistency for low light video enhancement from single images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 4967--4976, 2021 a

    2021

  58. [66]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 586--595, 2018

    2018

  59. [67]

    Learning to restore hazy video: A new real-world dataset and a new method

    Xinyi Zhang, Hang Dong, Jinshan Pan, Chao Zhu, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Fei Wang. Learning to restore hazy video: A new real-world dataset and a new method. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 9...

    2021

  60. [68]

    Motion-attentive transition for zero-shot video object segmentation

    Tianfei Zhou, Shunzhou Wang, Yi Zhou, Yazhou Yao, Jianwu Li, and Ling Shao. Motion-attentive transition for zero-shot video object segmentation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 13066--13073, 2020

    2020

  61. [69]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  62. [70]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  63. [71]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  64. [72]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.