pith. sign in

arxiv: 2604.03118 · v2 · pith:C2JAM3NKnew · submitted 2026-04-03 · 💻 cs.CV · eess.IV

Salt: Self-Consistent Distribution Matching with Cache-Aware Training for Fast Video Generation

Xingtong Ge , Yi Zhang , Yushi Huang , Dailan He , Xiahong Wang , Bingqi Ma , Guanglu Song , Yu Liu
show 1 more author
Jun Zhang
This is my paper

Pith reviewed 2026-05-13 19:35 UTC · model grok-4.3

classification 💻 cs.CV eess.IV
keywords video generationdiffusion distillationdistribution matchinglow-NFE inferenceKV cacheautoregressive generationself-consistency
0
0 comments X

The pith

Salt distills video models to 2-4 steps by regularizing the endpoint consistency of consecutive denoising updates and conditioning on KV cache states.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that distribution matching distillation produces sharp video samples but allows drift when denoising updates are composed into full rollouts at very low step counts. It introduces self-consistent regularization that forces partial trajectories to match at their endpoints, reducing accumulated errors in motion and detail. For autoregressive real-time models, the same idea is extended by treating the KV cache as an explicit conditioning signal during training and adding a feature alignment loss that pulls low-quality outputs toward high-quality references. Experiments confirm the approach works on both standard diffusion backbones and cache-based autoregressive setups without changing inference speed or memory layout. If the regularization holds, low-budget video generation could reach the sharpness previously available only at much higher compute cost.

Core claim

Self-Consistent Distribution Matching Distillation (SC-DMD) explicitly regularizes the endpoint-consistent composition of consecutive denoising updates so that multi-step rollouts avoid drift, while Cache-Distribution-Aware training treats the KV cache as a quality-parameterized condition and adds cache-conditioned feature alignment to steer low-quality autoregressive outputs toward high-quality references, yielding higher-quality video at 2-4 NFEs across tested non-autoregressive and autoregressive architectures.

What carries the argument

Self-Consistent Distribution Matching Distillation (SC-DMD) that enforces endpoint consistency across consecutive denoising updates, together with cache-conditioned feature alignment that uses the KV cache as a conditioning variable.

If this is right

  • Low-NFE video quality improves on non-autoregressive backbones such as Wan 2.1.
  • Autoregressive real-time models such as Self Forcing gain quality while remaining compatible with existing KV-cache mechanisms.
  • Sharp, mode-seeking samples are recovered without the conservative smoothing typical of trajectory consistency distillation.
  • The method adds no extra inference cost or memory overhead beyond the original backbone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same endpoint-consistency idea could be tested on image or audio generation tasks that also rely on multi-step sampling.
  • Cache-aware alignment might extend naturally to streaming or online generation where the cache state evolves over time.
  • Combining the regularization with other acceleration methods such as step-size scheduling could be checked for additive gains.

Load-bearing premise

That enforcing endpoint consistency on composed denoising updates will prevent drift in full rollouts and that cache-conditioned feature alignment will reliably improve quality without creating new inconsistencies.

What would settle it

Quantitative comparison of motion consistency and perceptual sharpness metrics on identical prompts at 2-4 NFEs between Salt and baseline distribution matching distillation, checking whether trajectory drift or over-smoothing visibly decreases.

Figures

Figures reproduced from arXiv: 2604.03118 by Bingqi Ma, Dailan He, Guanglu Song, Jun Zhang, Xiahong Wang, Xingtong Ge, Yi Zhang, Yu Liu, Yushi Huang.

Figure 1
Figure 1. Figure 1: Compositionality deficit of DMD. First, middle, and last frames from 4- /8-/16-step DMD students (rows, top to bottom) on: (a) “...a spaceman wearing a red wool knitted motorcycle helmet...” and (b) “...a large stack of vintage televisions all showing different programs...museum gallery.” Increasing the number of denoising steps degrades rather than improves quality: the 16-step model loses the knitted hel… view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of training trajectories for few-step distillation meth [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of Salt for autoregressive video generation. Left: A step count K ∈ {8, 4, 2} is sampled to define the few-step denoising trajectory. Middle: Conditioned on the current KV cache, text, and noise, the student generator Gθ denoises the current chunk. A self-consistency (SC) loss LSC regularizes the endpoint discrepancy between a direct update and a composed two-step update. Right: During mixed-step … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on texture-rich and high-dynamic scenes. trained on a 4-point grid; DMD-8, trained on a denser 8-point grid; and SC￾DMD, which uses the same 8-point grid as DMD-8 but adds the shortcut self￾consistency loss. Simply increasing the training grid density does not solve the problem: compared with DMD-4, DMD-8 drops from 84.39 to 84.05 in Quality and from 82.78 to 82.54 in Total, while al… view at source ↗
Figure 5
Figure 5. Figure 5: Cross-step consistency under the same seed and prompt. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 1
Figure 1. Figure 1: Displacement-normalized local semigroup defect on the test-time 4-step infer￾ence path. For each adjacent inference interval (ts, te), we compare the direct endpoint x (1) te = Ψ ts→te θ (xts ) against the composed endpoint x (2) te = Ψ tm→te θ (Ψ ts→tm θ (xts )), where tm is the corresponding intermediate timestep from the finer training grid. Lower is bet￾ter. SC-DMD achieves a lower overall local semigr… view at source ↗
Figure 2
Figure 2. Figure 2: More qualitative comparisons between the Causal Forcing [47] baseline and our method. Our method shows consistent advantages in both visual quality and se￾mantic consistency. Compared with the baseline, our results better preserve subject identity, object geometry, and scene composition across frames, while also produc￾ing smoother motion progression. The reading-girl example highlights reduced seman￾tic/i… view at source ↗
read the original abstract

Distilling video generation models to extremely low inference budgets (e.g., 2--4 NFEs) is crucial for real-time deployment, yet remains challenging. Trajectory-style consistency distillation often becomes conservative under complex video dynamics, yielding an over-smoothed appearance and weak motion. Distribution matching distillation (DMD) can recover sharp, mode-seeking samples, but its local training signals do not explicitly regularize how denoising updates compose across timesteps, making composed rollouts prone to drift. To overcome this challenge, we propose Self-Consistent Distribution Matching Distillation (SC-DMD), which explicitly regularizes the endpoint-consistent composition of consecutive denoising updates. For real-time autoregressive video generation, we further treat the KV cache as a quality parameterized condition and propose Cache-Distribution-Aware training. This training scheme applies SC-DMD over multi-step rollouts and introduces a cache-conditioned feature alignment objective that steers low-quality outputs toward high-quality references. Across extensive experiments on both non-autoregressive backbones (e.g., Wan~2.1) and autoregressive real-time paradigms (e.g., Self Forcing), our method, dubbed \textbf{Salt}, consistently improves low-NFE video generation quality while remaining compatible with diverse KV-cache memory mechanisms. Project page: https://xingtongge.github.io/Salt

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces Salt for distilling video generation models to low NFEs (2-4). It proposes Self-Consistent Distribution Matching Distillation (SC-DMD) that adds explicit regularization on the endpoint-consistent composition of consecutive denoising updates to reduce drift in composed rollouts, and Cache-Distribution-Aware training that treats the KV cache as a quality-conditioned input, applies SC-DMD over multi-step autoregressive rollouts, and adds a cache-conditioned feature alignment loss to steer outputs toward high-quality references. Experiments on non-autoregressive backbones (e.g., Wan 2.1) and autoregressive paradigms (e.g., Self Forcing) report consistent quality gains at low NFEs while remaining compatible with diverse KV-cache mechanisms.

Significance. If the added regularization demonstrably closes the composition gap for high-dimensional video dynamics and the cache alignment improves quality without new inconsistencies, the work would meaningfully extend distribution-matching distillation to practical real-time video generation. The compatibility with both non-autoregressive and autoregressive KV-cache setups, plus the promise of open-sourced code, would strengthen its utility for deployment.

major comments (2)
  1. [§3.1] §3.1 (SC-DMD formulation): the central claim that explicit endpoint-consistent regularization prevents drift in low-NFE rollouts is load-bearing, yet the manuscript provides no derivation showing that the added term closes the composition gap beyond the local signals already present in standard DMD; without this or an ablation isolating the regularization's effect on accumulated error over timesteps, the improvement over baseline DMD remains unverified for complex motions.
  2. [§4.3] §4.3 (Cache-Distribution-Aware training): the cache-conditioned feature alignment is asserted to steer low-quality outputs toward references without introducing new inconsistencies, but the reported experiments contain no direct metric (e.g., temporal consistency or endpoint mismatch) quantifying whether the alignment term creates fresh drift or artifacts in autoregressive rollouts, which is required to support the claim for real-time paradigms.
minor comments (2)
  1. [Abstract] The abstract and §1 could more precisely state the exact quantitative metrics (e.g., FVD, CLIP score) and NFE settings used to claim 'consistent improvements'.
  2. [§3.2] Notation for the KV-cache conditioning in Eq. (X) is introduced without an explicit diagram showing how the cache state is injected into the feature alignment loss.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive comments. We address each major comment below and have revised the manuscript accordingly to strengthen the presentation and empirical support.

read point-by-point responses

  1. Referee: [§3.1] §3.1 (SC-DMD formulation): the central claim that explicit endpoint-consistent regularization prevents drift in low-NFE rollouts is load-bearing, yet the manuscript provides no derivation showing that the added term closes the composition gap beyond the local signals already present in standard DMD; without this or an ablation isolating the regularization's effect on accumulated error over timesteps, the improvement over baseline DMD remains unverified for complex motions.

    Authors: We appreciate this observation. In the revised manuscript we have added an explicit derivation in Section 3.1 (and expanded in Appendix A) showing that the endpoint-consistent regularization term penalizes discrepancies between the composed multi-step trajectory and the direct endpoint mapping, thereby addressing the composition gap that is invisible to the per-step local signals of standard DMD. We have also inserted a targeted ablation in Section 4.2 that isolates the regularization's contribution by measuring accumulated temporal error over long rollouts on complex motion sequences, confirming a measurable reduction in drift relative to baseline DMD. revision: yes

  2. Referee: [§4.3] §4.3 (Cache-Distribution-Aware training): the cache-conditioned feature alignment is asserted to steer low-quality outputs toward references without introducing new inconsistencies, but the reported experiments contain no direct metric (e.g., temporal consistency or endpoint mismatch) quantifying whether the alignment term creates fresh drift or artifacts in autoregressive rollouts, which is required to support the claim for real-time paradigms.

    Authors: We agree that direct quantification is necessary. In the revised Section 4.3 we now report temporal consistency (optical-flow-based frame-to-frame coherence) and endpoint mismatch metrics on autoregressive rollouts. These measurements show that the cache-conditioned feature alignment improves fidelity to high-quality references while keeping both consistency and endpoint error at or below the levels observed with the unaligned baseline, supporting the claim that no new drift is introduced. revision: yes

Circularity Check

0 steps flagged

No circularity: new regularization terms and training scheme introduced independently

full rationale

The paper proposes SC-DMD as an explicit regularization of endpoint-consistent composition of denoising updates on top of standard DMD, plus a cache-conditioned feature alignment objective for autoregressive rollouts. These are framed as novel additions to address drift in low-NFE video generation, without any equations or claims reducing to self-citations, fitted parameters renamed as predictions, or ansatzes smuggled from prior author work. The derivation chain builds on established distribution matching principles with independent methodological content that does not collapse by construction to its inputs. No load-bearing steps exhibit self-definitional loops or uniqueness imported from overlapping citations.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based on abstract only; no explicit free parameters, new axioms, or invented entities are detailed beyond standard assumptions in diffusion distillation.

axioms (1)
  • domain assumption Distribution matching distillation recovers sharp mode-seeking samples from teacher models
    Invoked as the basis for using DMD to address over-smoothing in consistency distillation.

pith-pipeline@v0.9.0 · 5578 in / 1178 out tokens · 50960 ms · 2026-05-13T19:35:26.286285+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PARE: Pruning and Adaptive Routing for Efficient Video Generation

    cs.CV 2026-05 unverdicted novelty 6.0

    PARE applies structure-aware head pruning and timestep/content-conditioned block routing to compress video DiTs, reducing per-step compute while preserving quality on Wan2.1-14B.

  2. One-Forcing: Towards Stable One-Step Autoregressive Video Generation

    cs.CV 2026-05 unverdicted novelty 5.0

    One-Forcing augments DMD with a GAN loss to enable stable one-step causal autoregressive video generation, reporting a VBench score of 83.76 as SOTA among one-step methods.