REVIEW 4 major objections 4 minor 31 references
PromptTea: Let Prompts Tell TeaCache the Optimal Threshold
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper argues that cache reuse in video diffusion should be governed by the complexity of the text prompt, and that complexity can be estimated from text-embedding similarity, yielding automatic thresholds and a 2.79x speedup on Wan2.1…
desk verdict The input-output refit and dynamic CFG cache are real wins; the prompt-complexity thresholding story is much weaker than the title suggests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the PCA complexity coefficient and its induced threshold: $R = \mathrm{Sim}_C/(\mathrm{Sim}_C+\mathrm{Sim}_S+\epsilon)$, $S = 1/(1+e^{-k(R-0.5)})$, and $\delta_{\mathrm{PCA}} = S\,\delta_{\max} + (1-S)\,\delta_{\min}$. This chain turns the input prompt into a number that controls how often cached features are reused, with no per-prompt tuning. The second mechanism is the polynomial feature expansion $X = g(x,t)$ that includes powers and products of the timestep-embedding relative difference $x$ and the timestep $t$; a linear regression on these features supplies the accumulated-output-difference estimate. The third is DynCFGCache's per-timestep codebook, which gives the CFG path the same threshold-and-accumulate logic instead of FFT-based fixed-interval reuse.
What would settle it
On Wan2.1, take two prompts with nearly identical $R$ but very different measured motion in the generated videos, grid-search the cache threshold for each, and check whether the true best thresholds differ; if they do, $R$ is not the right driver and the threshold mapping fails.
Extended reading notes
Core claim
PromptTea's central claim is that scene complexity—roughly, how much motion and how many interacting subjects the prompt calls for—is measurable from the prompt embedding itself, and that this measurement should set the cache threshold. The paper defines a complexity coefficient $R = \frac{\mathrm{Sim}_C}{\mathrm{Sim}_C+\mathrm{Sim}_S+\epsilon}$ from cosine similarities to curated complex and simple prompt sets, then maps it through a steep sigmoid to produce a threshold $\delta_{\mathrm{PCA}}$ inside model-specific bounds. It also claims that TeaCache's predictor is mis-specified: TEMNI combines text, timestep, and noise, but controlled experiments show seed noise barely changes output differences while text dominates, so the paper predicts output differences from timestep-embedding differences and the scalar timestep through 4th-order polynomial features, cutting prediction MSE from $84.51\times10^{-4}$ to $2.85\times10^{-4}$ on CogVideoX1.5 and similarly on the other models. Finally, it replaces uniform CFG reuse with DynCFGCache, which accumulates a per-timestep codebook of conditional/unconditional differences and reuses only when the accumulation stays below a threshold.
Load-bearing premise
The argument depends on the premise that a single scalar—how similar a prompt's embedding is to two fixed lists of simple and complex prompts—monotonically determines the cache threshold that best balances speed and quality.
Editorial extensions
If this is right
- On Wan2.1, PromptTea reaches 2.79x speedup while raising PSNR to 23.0 dB, above both TeaCache-slow (17.50 dB at 1.93x) and TeaCache-fast (16.81 dB at 2.64x).
- On all three backbones the speedup and quality are simultaneously better than the fixed-threshold baseline: 1.75x on CogVideoX1.5, 2.56x on HunyuanVideo, and 2.79x on Wan2.1.
- The polynomial reconstruction alone lifts speedup from 1.93x to 2.67x at threshold 0.2 with better PSNR and SSIM, and from 2.64x to 3.39x at threshold 0.3.
- Adding DynCFGCache pushes speedup further, to 3.45x at the 0.2 threshold without hurting the improved quality, and to 3.85x at the 0.3 threshold.
- HunyuanVideo cannot use the CFG module because its text-guidance distillation merges conditional and unconditional computation, so the PCA-TeaCache changes carry the full gain there.
Reading between the lines
- An implication the paper leaves implicit is that the same prompt-embedding complexity ratio could calibrate per-layer or per-block cache budgets, or even the number of denoising steps, not just the single global threshold.
- A natural next test, not run in the paper, is to grid-search the optimal threshold for a larger prompt distribution and check directly whether it is a single-valued function of $R$.
- Because DynCFGCache's codebook is prompt-independent and per-model, it could be precomputed once per backbone, making the CFG-side speedup available without any per-video computation.
- A testable consequence of the paper's complexity definition is that prompts describing intricate but static scenes should still receive the same threshold as other prompts with the same $R$; if scene motion is the real driver, this prediction would fail.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PromptTea, a training-free caching acceleration method for DiT-based text-to-video generation. It consists of three components: PCA, which estimates prompt complexity from text embeddings and maps it to an adaptive cache-reuse threshold; an input-output relationship reconstruction (R) that replaces TeaCache's TEMNI feature with timestep embeddings plus polynomial features; and DynCFGCache (D), which extends FasterCache's CFGCache by dynamic difference accumulation. Experiments on CogVideoX1.5, HunyuanVideo, and Wan2.1 report speedups of 1.75x, 2.56x, and 2.79x respectively over the uncached baseline, with quality measured by VBench2, LPIPS, SSIM, and PSNR. The central claim is that prompt-complexity-aware threshold selection improves the speed-quality tradeoff over fixed-threshold caching.
Significance. If the central claim holds, PromptTea addresses a practical bottleneck in video diffusion inference without retraining or quantization, and its reported speedups on three open models would be useful to practitioners. The manuscript's strengths include a released code repository, use of official baseline settings, reporting of both FLOPs and wall-clock latency, and ablations that separately quantify the contributions of the relationship reconstruction and DynCFGCache modules. The weakest point is that the PCA contribution, which is the title component and the paper's main novelty, is not isolated in the ablation: no fixed-threshold baseline is run at the mean of the thresholds that PCA actually assigns, so the data in Table 2 do not demonstrate that adaptive thresholding helps over a carefully chosen fixed threshold. The evaluation also uses a small prompt sample without error bars, and the prompt-complexity mapping rests on one anecdotal figure. These gaps are fixable with additional experiments and analysis, so the paper is a candidate for major revision rather than rejection.
major comments (4)
- [Table 2] The ablation does not isolate the PCA module. Comparing PromptTea(0.1,0.23) with TeaCache(0.2)+R+D (3.45x, VBench2 0.4729, PSNR 21.61) and TeaCache(0.3)+R+D (3.85x, VBench2 0.3669, PSNR 19.82) shows that the full method is slower than the fixed threshold 0.2 with R+D and has lower VBench2, although higher pixel metrics. Because PCA only modulates thresholds within [0.1,0.23], the missing comparison is a fixed-threshold baseline with R+D at the mean of the thresholds actually assigned by Eq. 7 (for example, delta=0.15 or the per-prompt mean of delta_PCA). Without reporting TeaCache(delta_mean)+R+D, the data do not establish that prompt-adaptive thresholding contributes over simply choosing a more conservative fixed threshold; all reported gains over TeaCache-fast could come from the R and D modules alone.
- [Section 'Prompt Complexity Aware', Eqs. (5)-(7)] The central assumption that the optimal cache threshold is a monotone function of R is supported only by Fig. 4, which shows one simple and one complex prompt, and by the cluster visualization in Fig. 6. The paper does not report a scatter plot of R versus the empirically optimal threshold, a correlation coefficient, or a held-out validation of the monotonic mapping. In addition, R=SimC/(SimC+SimS+epsilon) is not guaranteed to lie in [0,1] unless all cosine similarities are nonnegative and their sum is positive; without clipping or normalization, S in Eq. 6 and hence delta_PCA in Eq. 7 can fall outside [delta_min, delta_max]. Please specify how R and S are normalized or clipped, and release or fully specify the 100 simple and 100 complex prompts used to construct the reference embedding sets.
- [Section 'Metrics' and Table 1] The quantitative evaluation uses 5 prompts per dimension over VBench2's 16 axes (80 prompts) and reports no seeds or error bars. Many comparative numbers in Table 1 are close, such as VBench2 0.4710 versus 0.4480 for PromptTea versus TeaCache-fast on Wan2.1, and the FLOPs and latency figures are single-run measurements without variance. Please report mean and standard deviation over at least three seeds and make the sampled evaluation prompts public or list them, so the differences in Table 1 can be assessed for statistical significance.
- [Appendix, 'More Details on Input-Output Relationship Reconstruction' and Eq. (8)] The description 'We use 100 x 50 groups of (X, y) data' is ambiguous, and the manuscript does not state whether the same prompts used to fit the polynomial regression are also used when evaluating the corresponding model in Tables 1 and 2. Please clarify the training/evaluation split for the linear regression, report fitting error and generalization error separately, and state whether the fitted coefficients are shared across models or fitted per model. Without this information, part of the reported quality gain of the R module could be an artifact of fitting and evaluating on the same prompts.
minor comments (4)
- [Figures 2, 4, and Table 1] There are typos in labels and captions: 'timpstep' in Figs. 2 and 9 should be 'timestep', 'Threshould' in Fig. 4 should be 'Threshold', and 'W AN2.1' in the Table 1 header has a stray space.
- [Section 'Prompt Complexity Aware'] The paragraph beginning 'Based on the above findings, we propose to evaluate scenario complexity...' appears twice verbatim; one copy should be removed.
- [Eqs. (3), (9), and (10)] The reuse conditions are written inconsistently: Eq. (3) uses '<= deltaTea <=' while Eqs. (9) and (10) use '<= deltaPCA <' and '<= deltaCFG <'. Please make the boundary convention uniform and state whether the reuse condition is strict at the threshold.
- [Eq. (8)] If X is intended to contain all fourth-order monomials in x and t, the term x^2 t^2 is missing from the list; if the omission is intentional, the text should say so, since the current phrase '4th-order polynomial features' is inaccurate.
Circularity Check
No circularity: the PCA threshold is a defined heuristic mapping prompt embeddings to a threshold, the polynomial regressor is standard supervised fitting, and the main evaluation is against external baselines and benchmarks.
full rationale
The paper's derivation chain is not circular. The complexity coefficient R (Eq. 5) is computed from cosine similarities to hand-collected simple/complex prompt sets, and the adaptive threshold δ_PCA is defined by the sigmoid interpolation in Eqs. 6-7. This is a closed-form heuristic definition, not a prediction of an independently measured 'optimal threshold'; the paper never fits δ_PCA to an external per-prompt optimum and then claims to predict it. The word 'optimal' is an unsupported strength claim, but that is a validation gap, not a circular reduction. The input-output relationship in Eq. 8-9 is a supervised polynomial regression: the model l is fitted to (X, y) pairs from 100 prompts and then applied at inference to estimate output differences. Using a fitted model to predict the same kind of quantity it was trained on is standard supervised learning, not 'fitted input called prediction'; the paper's downstream quality metrics (PSNR, SSIM, LPIPS, VBench2) are external and not used as training targets. DynCFGCache's codebook (Eq. 10) is a calibrated per-timestep statistic from 100 prompts and is compared against a hand-set threshold; again, this is a fixed decision rule, not a tautology. The VBench-2.0 reference includes a coauthor of the present paper, but it is a public, externally used benchmark suite, so the self-citation is not load-bearing. The absence of a matched fixed-threshold ablation (e.g., a fixed threshold equal to the mean of δ_PCA over the evaluation prompts) and the lack of error bars are legitimate experimental weaknesses, but they do not make any claimed result equivalent to its inputs by construction.
Assumptions & free parameters
free parameters (4)
- sigmoid stretch k =
50 (CogVideoX1.5), 200 (HunyuanVideo), 50 (Wan2.1)
- threshold bounds (delta_min, delta_max) =
(0.2,0.3), (0.1,0.15), (0.1,0.23) per model
- CFG threshold delta_CFG =
0.02 for CogVideoX1.5 and Wan2.1
- polynomial regression coefficients theta_PT =
fitted via sklearn LinearRegression on 100 prompts x 50 timesteps per model
assumptions (3)
- ad hoc to paper Prompt complexity as measured by average embedding distance and cosine similarity to hand-picked prompt sets correlates monotonically with the optimal cache threshold.
- domain assumption Timestep embeddings are a sufficient signal for cache reuse decisions once augmented with text-derived thresholds and polynomial features.
- domain assumption The fitted linear model generalizes to unseen prompts without retuning.
Cite this review
Pith. "Pith review of PromptTea: Let Prompts Tell TeaCache the Optimal Threshold." pith.science (2026). https://pith.science/paper/UHDBACB5
@misc{pith2026250706739,
author = {Pith},
title = {Pith review of: PromptTea: Let Prompts Tell TeaCache the Optimal Threshold},
year = {2026},
howpublished = {\url{https://pith.science/paper/UHDBACB5}},
note = {Machine review of arXiv:2507.06739}
}
read the original abstract
Despite recent progress in video generation, inference speed remains a major bottleneck. A common acceleration strategy involves reusing model outputs via caching mechanisms at fixed intervals. However, we find that such fixed-frequency reuse significantly degrades quality in complex scenes, while manually tuning reuse thresholds is inefficient and lacks robustness. To address this, we propose Prompt-Complexity-Aware (PCA) caching, a method that automatically adjusts reuse thresholds based on scene complexity estimated directly from the input prompt. By incorporating prompt-derived semantic cues, PCA enables more adaptive and informed reuse decisions than conventional caching methods. We also revisit the assumptions behind TeaCache and identify a key limitation: it suffers from poor input-output relationship modeling due to an oversimplified prior. To overcome this, we decouple the noisy input, enhance the contribution of meaningful textual information, and improve the model's predictive accuracy through multivariate polynomial feature expansion. To further reduce computational cost, we replace the static CFGCache with DynCFGCache, a dynamic mechanism that selectively reuses classifier-free guidance (CFG) outputs based on estimated output variations. This allows for more flexible reuse without compromising output quality. Extensive experiments demonstrate that our approach achieves significant acceleration-for example, 2.79x speedup on the Wan2.1 model-while maintaining high visual fidelity across a range of scenes.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Blattmann, A.; Dockhorn, T.; Kulal, S.; Mendelevitch, D.; Kilian, M.; Lorenz, D.; Levi, Y.; English, Z.; Voleti, V.; Letts, A.; et al. 2023. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127
arXiv 2023
-
[4]
Chen, L.; Meng, Y.; Tang, C.; Ma, X.; Jiang, J.; Wang, X.; Wang, Z.; and Zhu, W. 2025. Q-dit: Accurate post-training quantization for diffusion transformers. In Proceedings of the Computer Vision and Pattern Recognition Conference, 28306--28315
work page 2025
-
[5]
Chen, P.; Shen, M.; Ye, P.; Cao, J.; Tu, C.; Bouganis, C.-S.; Zhao, Y.; and Chen, T. 2024. -DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers. arXiv preprint arXiv:2406.01125
arXiv 2024
-
[6]
DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437
arXiv 2024
-
[7]
Dhariwal, P.; and Nichol, A. 2021. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 8780--8794
2021
-
[8]
Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840--6851
2020
Show all 31 references
-
[9]
S.; and Xie, T
Kahatapitiya, K.; Liu, H.; He, S.; Liu, D.; Jia, M.; Zhang, C.; Ryoo, M. S.; and Xie, T. 2024. Adaptive caching for faster video generation with diffusion transformers. arXiv preprint arXiv:2411.02397
2024 arXiv
-
[10]
Kong, W.; Tian, Q.; Zhang, Z.; Min, R.; Dai, Z.; Zhou, J.; Xiong, J.; Li, X.; Wu, B.; Zhang, J.; et al. 2024. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603
2024 arXiv
-
[11]
Li, M.; Cai, T.; Cao, J.; Zhang, Q.; Cai, H.; Bai, J.; Jia, Y.; Li, K.; and Han, S. 2024. Distrifusion: Distributed parallel inference for high-resolution diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7183--7193
2024
-
[12]
S.; Li, L.; Yang, S.; Wang, Y.; Cheng, M.-M.; and Yang, J
Li, S.; Hu, T.; Khan, F. S.; Li, L.; Yang, S.; Wang, Y.; Cheng, M.-M.; and Yang, J. 2023. Faster diffusion: Rethinking the role of unet encoder in diffusion models. CoRR
2023
-
[13]
Lin, B.; Ge, Y.; Cheng, X.; Li, Z.; Zhu, B.; Wang, S.; He, X.; Ye, Y.; Yuan, S.; Chen, L.; et al. 2024. Open-Sora Plan: Open-Source Large Video Generation Model. arXiv preprint arXiv:2412.00131
2024 arXiv
-
[14]
Liu, F.; Zhang, S.; Wang, X.; Wei, Y.; Qiu, H.; Zhao, Y.; Zhang, Y.; Ye, Q.; and Wan, F. 2025. Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model. In Proceedings of the Computer Vision and Pattern Recognition Conference, 7353--7363
2025
-
[15]
Lv, Z.; Si, C.; Song, J.; Yang, Z.; Qiao, Y.; Liu, Z.; and Wong, K.-Y. K. 2024. Fastercache: Training-free video diffusion model acceleration with high quality. arXiv preprint arXiv:2410.19355
2024 arXiv
-
[16]
Ma, X.; Fang, G.; and Wang, X. 2024. Deepcache: Accelerating diffusion models for free. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 15762--15772
2024
-
[17]
McInnes, L.; Healy, J.; and Melville, J. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426
2018 arXiv
-
[18]
Meng, C.; Rombach, R.; Gao, R.; Kingma, D.; Ermon, S.; Ho, J.; and Salimans, T. 2023. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14297--14306
2023
-
[19]
Peebles, W.; and Xie, S. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 4195--4205
2023
-
[20]
Ramesh, A.; Dhariwal, P.; Nichol, A.; Chu, C.; and Chen, M. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2): 3
2022 arXiv
-
[21]
L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.; Salimans, T.; et al
Saharia, C.; Chan, W.; Saxena, S.; Li, L.; Whang, J.; Denton, E. L.; Ghasemipour, K.; Gontijo Lopes, R.; Karagol Ayan, B.; Salimans, T.; et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing syste...
2022
-
[22]
Sauer, A.; Lorenz, D.; Blattmann, A.; and Rombach, R. 2024. Adversarial diffusion distillation. In European Conference on Computer Vision, 87--103. Springer
2024
-
[23]
Sohl-Dickstein, J.; Weiss, E.; Maheswaranathan, N.; and Ganguli, S. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, 2256--2265. pmlr
2015
-
[24]
Song, Y.; and Ermon, S. 2019. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32
2019
-
[25]
Wan, T.; Wang, A.; Ai, B.; Wen, B.; Mao, C.; Xie, C.-W.; Chen, D.; Yu, F.; Zhao, H.; Yang, J.; Zeng, J.; Wang, J.; Zhang, J.; Zhou, J.; Wang, J.; Chen, J.; Zhu, K.; Zhao, K.; Yan, K.; Huang, L.; Feng, M.; Zhang, N.; Li, P.; Wu, P.; Chu, R.; Feng, R.; Zhang, S.; Sun, S.; Fang, ...
2025 arXiv
-
[26]
Wang, X.; Zhang, S.; Zhang, H.; Liu, Y.; Zhang, Y.; Gao, C.; and Sang, N. 2023. Videolcm: Video latent consistency model. arXiv preprint arXiv:2312.09109
2023 arXiv
-
[27]
Yang, Z.; Teng, J.; Zheng, W.; Ding, M.; Huang, S.; Xu, J.; Yang, Y.; Hong, W.; Zhang, X.; Feng, G.; et al. 2024. CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer. arXiv preprint arXiv:2408.06072
2024 arXiv
-
[28]
A.; Shechtman, E.; and Wang, O
Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; and Wang, O. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 586--595
2018
-
[29]
Zhao, X.; Jin, X.; Wang, K.; and You, Y. 2024. Real-time video generation with pyramid attention broadcast. arXiv preprint arXiv:2408.12588
2024 arXiv
-
[30]
Zheng, D.; Huang, Z.; Liu, H.; Zou, K.; He, Y.; Zhang, F.; Zhang, Y.; He, J.; Zheng, W.-S.; Qiao, Y.; and Liu, Z. 2025. VBench-2.0 : Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness. arXiv preprint arXiv:2503.21755
2025 arXiv
-
[31]
Zheng, Z.; Peng, X.; Yang, T.; Shen, C.; Li, S.; Liu, H.; Zhou, Y.; Li, T.; and You, Y. 2024. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404
2024 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.