REVIEW 7 cited by
Looking Backward: Streaming Video-to-Video Translation with Feature Banks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This paper introduces StreamV2V, a diffusion model that achieves real-time streaming video-to-video (V2V) translation with user prompts. Unlike prior V2V methods using batches to process limited frames, we opt to process frames in a streaming fashion, to support unlimited frames. At the heart of StreamV2V lies a backward-looking principle that relates the present to the past. This is realized by maintaining a feature bank, which archives information from past frames. For incoming frames, StreamV2V extends self-attention to include banked keys and values and directly fuses similar past features into the output. The feature bank is continually updated by merging stored and new features, making it compact but informative. StreamV2V stands out for its adaptability and efficiency, seamlessly integrating with image diffusion models without fine-tuning. It can run 20 FPS on one A100 GPU, being 15x, 46x, 108x, and 158x faster than FlowVid, CoDeF, Rerender, and TokenFlow, respectively. Quantitative metrics and user studies confirm StreamV2V's exceptional ability to maintain temporal consistency.
Forward citations
Cited by 7 Pith papers
-
Unified Video Dense Prediction from Disjoint Data
A single video backbone predicts eight dense scene tasks from separate single-task datasets via latent distillation from diffusion-based specialists, with no co-annotated data or pseudo-labels.
-
LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing
LiveEdit distills a bidirectional video foundation model into a unidirectional streaming editor via three-stage training plus mask caching to reach 12.66 FPS with stable edits.
-
Importance-Aware OBS Pruning for Diffusion Models
Injecting spatial importance maps (e.g., CFG delta) into the OBS Hessian improves subject preservation in pruned diffusion models at high sparsity, but gains over the baseline are small and without error bars.
-
Streaming Drag-Oriented Interactive Video Manipulation: Drag Anything, Anytime!
DragStream enables real-time drag, deform, and rotate edits on autoregressively generated videos without retraining, by correcting latent drift and selectively filtering context features.
-
AnchorSync: Global Consistency Optimization for Long Video Editing
By jointly editing sparse anchor frames and interpolating with flow and edge guidance, AnchorSync produces temporally consistent edits on videos longer than previous diffusion methods could handle.
-
Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
Sparse VideoGen accelerates video diffusion transformers by about 2.3x with only small quality loss by classifying attention heads into spatial and temporal sparse patterns and using hardware-friendly layouts.
-
Human Motion Video Generation: A Survey
A comprehensive survey with a five-phase pipeline model for human motion video generation, covering over 200 papers and adding a new benchmark comparison of nine pose-guided methods.
Discussion (0). Continue with ORCID to comment.