{"id":"8494b8e7-db5f-480b-ac11-dbbf40690820","arxiv_id":"2506.14404","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"CSVC optimizes text prompts via vision-language-model feedback to steer frozen video diffusion editors toward causally consistent facial counterfactuals such as aging, gender change, beard addition, and baldness.","lead":"Researchers propose CSVC, a prompt-optimization framework that steers black-box text-to-video diffusion editors to produce counterfactual facial videos (e.g., aging a person, changing gender) that respect a user-supplied causal graph. The method uses vision-language-model feedback to iteratively refine target prompts, improving attribute-change success with modest quality loss.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Causal-faithfulness claim is not actually measured: the reported metrics check only the intervened attribute and non-causal minimality, never the non-intervened causal variables.","rationale":"The reader's weakest assumption identifies a real and related issue: VLM-based metrics are unvalidated proxies, and the same model family is used for steering and evaluation. My concern is more direct and even more load-bearing: even if the VLM judgments were perfectly truthful, the presented metrics would not detect violations of the causal graph, because non-intervened causal variables are never scored. Eq. 4 checks only the intervened attribute; Eq. 5 explicitly filters out all DAG variables; LPIPS is global and attribute-agnostic. Consequently the evidence in Table 1 does not establish causal faithfulness, only that the target attribute is more often judged present. This gap is concrete and testable by scoring all four variables per intervention with VLM and human labels. Since the paper otherwise presents a coherent and reproducible method, the right response is the same conditional acceptance the reader recommends, with the additional required condition that causal propagation and exclusion restrictions be directly evaluated. I therefore keep the verdict unchanged, while noting that the condition is more specific than the reader's proxy-validation concern.","tokens_in":21331,"tokens_out":4884,"duration_ms":55775,"concrete_test":"Run the existing 67-video set and, for every generated counterfactual, query the VLM with four binary questions covering all DAG variables (age changed? gender changed? beard changed? baldness changed?), using the target counterfactual prompt to define correct answers for intervened variables and the factual prompt for non-intervened variables. Report a per-intervention confusion matrix for CSVC versus initial prompts. Separately, have human raters label the same questions on a 50-video subset. If, e.g., beard interventions change gender or age as often as the initial-prompt baseline, or if age/gender interventions do not increase beard/bald change rates, the causal-faithfulness claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CSVC produces 'causally faithful video counterfactuals' (Abstract, Section 5.2). For a Pearl-style counterfactual under the assumed DAG (age, gender, beard, baldness), faithfulness requires two things: (i) intervening on an upstream variable propagates to downstream variables, and (ii) intervening on a downstream variable leaves upstream variables unchanged. The paper's only causal metrics do not verify either. Effectiveness (Eq. 4) asks the VLM a multiple-choice question about the intervened attribute only; for a beard intervention it checks whether a beard appears, not whether age or gender changed. Minimality (Eq. 5) explicitly instructs the VLM to 'exclude DAG variables' from the descriptions (Listing 9), so it erases all causal information by design; LPIPS is attribute-agnostic and rises with any edit. Thus Table 1's effectiveness columns and VLM-Min cannot support the causal-faithfulness claim. The ambiguity in Section 5.1 ('four multiple-choice questions, each targeting a variable from the causal graph') does not resolve this: the reported table shows accuracy per intervened variable only, and no results are given for non-intervened variables. The causal-decoupling prompt (Listing 4) even instructs the optimizer to drop upstream references for beard/bald interventions, so a beard intervention that also changes gender or age would score identically on Eq. 4 and could score better on Eq. 5. The framework may therefore be optimizing a single VLM-judged attribute rather than enforcing the graph structure. This is a correctness risk for the headline claim, independent of the additional circularity concern that the optimizer and evaluator are VLM-based.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CSVC, a black-box prompt-optimization framework for LDM-based video editing that generates counterfactual videos from a user-specified causal graph. It first uses GPT-4 in-context learning to convert factual prompts into counterfactual prompts for age, gender, beard, and baldness interventions; then iteratively refines those prompts with TextGrad-style textual gradients derived from VLM feedback on generated frames. Experiments with FLATTEN, Tune-A-Video, and TokenFlow on 67 CelebV-Text videos compare CSVC against initial prompts and LLM paraphrasing on VLM-based effectiveness, VLM-based minimality, LPIPS, DOVER, FVD, and CLIP temporal consistency.","tokens_in":21605,"tokens_out":6778,"duration_ms":66620,"significance":"The framework is genuinely model-agnostic, requires no fine-tuning or internal access, and the implementation is made available; applying TextGrad to causal counterfactual video editing is a reasonable and novel idea. The reported effectiveness improvements are consistent across all three editing systems, which suggests the prompt-steering mechanism has practical value. However, the central claim of causal faithfulness is not supported by the current evaluation, and the VLM-based metrics lack human validation and statistical grounding; the contribution's significance will depend on whether the authors can supply those missing pieces.","major_comments":[{"comment":"The causal-faithfulness claim is not tested by the reported effectiveness metric. Effectiveness(α) asks only about the intervened attribute α, so for a beard intervention it checks whether a beard is present, not whether age or gender changed; for a downstream intervention such as do(beard), the DAG requires upstream variables (age, gender) to remain unchanged, and for an upstream intervention such as do(age), downstream variables should respond, but neither requirement is ever measured. The text in Section 5.1 saying four multiple-choice questions target variables from the causal graph is not reflected in Table 1, which gives one accuracy per intervened variable; please report per-intervention accuracy on all four variables, or equivalently conditional causal checks, for each editor.","section":"Section 5.2, Eq. (4), Table 1"},{"comment":"The VLM minimality metric erases exactly the causal variables whose invariance should be tested. Listing 9 removes age, gender, beard, and hair/baldness from both descriptions, so a counterfactual that changes gender during a beard intervention receives the same VLM-Min as one that does not; LPIPS in Table 1 measures any pixel-level change and cannot separate causal from non-causal edits. Minimality should be computed separately for non-causal attributes, which should be preserved, and for non-intervened causal variables, which should be preserved for downstream interventions and may change for upstream interventions.","section":"Section 4.4, Eq. (5), Listing 9"},{"comment":"No confidence intervals, variance estimates, or significance tests are reported. Many of the headline differences are small relative to the sample size (e.g., TokenFlow age effectiveness 0.940 vs 0.909, VLM-Min 0.768 vs 0.773, DOVER 0.786 vs 0.787), and with 67 videos a few mislabeled VLM answers could change the ranking; the state-of-the-art causal effectiveness claim needs paired tests, confidence intervals, or at least per-sample results.","section":"Section 5.2, Table 1"},{"comment":"The evaluation is partially optimized in a closed loop with the same VLM family used for the loss. The textual loss (Eq. (2) and Listing 3) and the minimality metric (Eq. (5)) are both implemented with GPT-4o, so the reported VLM-Min may reflect the optimizer's success at satisfying GPT-4o's textual preferences rather than measurable visual minimality; effectiveness uses LLaVA-NeXT, which is an independent model but still unvalidated. Since no human study or established benchmark calibrates either metric, the numbers in Table 1 should be treated as proxy scores, not as causal ground truth; please provide human validation on a subsample or an agreement analysis between VLMs and human raters.","section":"Sections 4.2 and 4.4"}],"minor_comments":[{"comment":"The paper repeatedly calls the method state-of-the-art, but Table 1 compares only three editors and two prompt baselines; please qualify this claim or add stronger comparative baselines.","section":"Abstract and Section 1"},{"comment":"The VLM loss outputs text, not a scalar; the paper should state explicitly that the loss is a textual feedback signal consumed by TextGrad rather than a numeric loss.","section":"Section 4.2, Eq. (2)"},{"comment":"There is a naming inconsistency between GPT-4 and GPT-4o across Section 5.1, Figure 2, and the Additional implementation details; please unify the model names.","section":"Section 5.1 and Appendix"},{"comment":"The minimality prompt removes hair, hairstyle, color, and style even though these are not variables in the assumed DAG; this makes the prompt broader than the causal graph and should be explained or corrected.","section":"Listing 9"},{"comment":"The Limitations section notes that no temporal consistency loss is added; given that the paper claims temporal consistency is preserved, a temporal evaluation of the edited attribute itself (e.g., per-frame beard presence) would strengthen the claim.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"I see the causal-faithfulness issue as binding rather than cosmetic: if the authors cannot provide per-variable causal checks and human validation, the paper may not be acceptable. I do not see grounds for outright rejection if those additions are made, because the prompt-steering mechanism is reproducible and shows consistent improvements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe punchline: the method is a neat prompt-refinement trick, but the paper's central claim—causally faithful counterfactuals—is not actually measured. The effectiveness metric checks only whether the intervened attribute changed; the minimality metric explicitly excludes all causal-graph variables from the description (Listing 9). So a beard intervention that also flips gender or age scores the same on both metrics. None of the reported numbers verify that non-intervened causal variables stayed put.\n\nWhat's genuinely new: encoding a small DAG (age, gender, beard, baldness) into initial prompts and then using TextGrad with a VLM-based loss to iteratively refine those prompts, all in a black-box, training-free manner. That combination is not in the literature they cite, and the gains are consistent across three editors with released code. The causal-decoupling heuristic—telling the optimizer to drop upstream references when intervening on downstream variables—is a small but useful trick for getting a woman-with-beard edit to succeed.\n\nWhere it gets soft: the evaluation. Effectiveness (Eq. 4) is a multiple-choice VLM question about the intervened attribute only. The paper says they created four questions per prompt targeting each DAG variable, but Table 1 reports accuracy only for the intervened variable; no results are shown for whether age or gender changed during a beard intervention. That is the load-bearing piece of the causal claim, and it's missing. On top of that, both the optimizer and the evaluator are unvalidated VLMs (GPT-4 for optimization and minimality, LLaVA-NeXT for effectiveness). They are different models, so it isn't a single closed loop, but without human ground truth the reported improvements may just be VLM prompt preferences. No significance tests, no error bars, and the 'state-of-the-art' claim appears without comparing to existing counterfactual video methods like D'Artagnan.\n\nThese are fixable: report accuracy on non-intervened DAG variables, add a small human study, and run error bars. The method itself is sound enough to referee; the claim just needs to match the evidence.\n\nRecommendation: accept for peer review with the expectation of major revision, and tell the authors to either weaken the causal-faithfulness language or add the missing checks.","headline":"A genuinely new prompt-steering framework for video counterfactuals, but the causal-faithfulness claim is not actually measured by the reported metrics.","tokens_in":22217,"tokens_out":2483,"would_cite":true,"duration_ms":25675,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Prompt-level causal steering makes video diffusion editors produce causally faithful counterfactuals without retraining.","keywords":["counterfactual video generation","latent diffusion models","video editing","causal graph","prompt optimization","vision-language model feedback","textual gradient descent","causal effectiveness"],"falsifier":"Have independent human annotators label whether the target attribute changed and whether unrelated attributes stayed unchanged on the same CSVC-generated counterfactual videos; if human agreement with the paper’s VLM effectiveness scores is near chance, or if swapping the VLM judge for a different model family reverses the reported advantage over unsteered prompts, the central claim collapses.","tokens_in":21095,"feed_emoji":"🎬","tokens_out":11201,"duration_ms":93601,"temperature":0.7,"pith_summary":"This paper tries to establish that the causal fidelity of video counterfactual generation can be controlled entirely through text prompts, without touching the diffusion model's weights or internals. It introduces CSVC, a framework that encodes a user-supplied causal graph into an initial counterfactual prompt, generates a video edit, asks a vision-language model to critique how well the edit matches the intended intervention, and then uses that natural-language criticism as a “textual gradient” to refine the prompt, iterating until the VLM is satisfied. The paper’s claim is that this loop steers latent diffusion editors toward out-of-distribution counterfactual outcomes that respect the graph, and even deliberately breaks strong causal edges (for instance, adding a beard to a woman) when a causal-decoupling instruction is added. On 67 real facial videos with interventions on age, gender, beard, and baldness, CSVC reports higher causal effectiveness than unsteered or paraphrased prompts across three video editing systems, while keeping visual quality and temporal consistency roughly unchanged. A sympathetic reader cares because this would make any existing black-box video editor causally aware simply by optimizing its text input.","feed_headline":"Video counterfactuals stay causally faithful with prompt-only steering","feed_subtitle":"No retraining or model access: a causal graph plus VLM feedback steers black-box editors to faithful what-if videos.","key_machinery":"The load-bearing object is the editable text prompt $P$, treated as the only handle on the latent diffusion model’s behavior. The framework couples two black boxes—any prompt-based video editor $f(V,P)$ as the counterfactual generator and a VLM as the critic—connected by a VLM-based counterfactual loss (Eq. 2) and updated by TextGrad’s textual gradient descent (Eq. 3), where natural-language criticisms replace numeric gradients. The causal-decoupling instruction—“if beard or bald appears in target interventions, do not include references to age or gender”—is the mechanism that lets a prompt intentionally violate strong causal edges by omitting upstream variables, operationalizing Pearl-style graph mutilation. The whole construction rests on the equivalence between video editing and Pearl’s abduction–action–prediction loop, with DDIM inversion, the prompt intervention, and DDIM sampling playing the three roles.","core_discovery":"The central claim is that prompt optimization can act as causal steering for video diffusion. Concretely, the paper identifies the video editing pipeline with Pearl’s abduction–action–prediction loop—DDIM inversion is abduction, the edited text prompt is the action, DDIM sampling is prediction—so that choosing the right counterfactual prompt is the entire intervention. CSVC builds the initial prompt from a causal DAG, then minimizes a VLM-based loss $\\mathcal{L}=\\mathrm{VLM}(V'_{\\mathrm{frame}},\\text{evaluation instruction},P)$ by applying TextGrad’s textual gradient descent, $P'=\\mathrm{LLM}(\\text{criticisms},P)$, where the “gradient” is natural-language feedback saying which attributes failed to render. A causal-decoupling instruction tells the VLM to drop references to upstream variables when intervening on downstream ones, which the paper interprets as graph mutilation and which produces deliberately out-of-distribution counterfactuals such as a woman with a beard. The reported results are that this steering raises VLM-measured causal effectiveness for age, gender, beard, and baldness interventions across FLATTEN, Tune-A-Video, and TokenFlow, with LPIPS, DOVER, FVD, and CLIP-temporal scores close to the baselines, which the paper summarizes as state-of-the-art causal effectiveness on real facial videos.","pith_inferences":["Because the same model family (GPT-4) generates the initial prompts, supplies the textual gradients, and scores effectiveness, part of the reported improvement may reflect the judge preferring prompts that resemble its own recommended language; a decisive test would use humans or a different VLM family to score the same generated videos.","The causal-decoupling instruction is a general recipe: any DAG edge can be severed by telling the VLM to ignore upstream variables, so the method could extend naturally to other strong correlations (for example, expression, hair color, or age-related attributes) beyond the four studied here.","The paper’s stated limitations say no loss is added for temporal consistency and only static facial attributes are intervened on, so the “video counterfactual” claim currently covers attribute-level changes on faces rather than actions, motion, or event-level what-if scenarios.","The per-video optimization loop calls the VLM/LLM multiple times per video, so applying CSVC at scale would carry API cost and latency that the paper does not quantify; a batch or one-shot prompt-prediction variant would be a natural extension."],"forward_implications":["Any prompt-based video editor, including closed commercial systems, can be given causal awareness without fine-tuning or internal access, by wrapping it in the CSVC optimization loop.","Causal knowledge becomes an input the user can specify as a DAG plus a few natural-language prompt examples, so producing counterfactual videos needs no causal-model training data.","Prompt-level textual gradients can substitute for weight-level gradients in diffusion models for at least some semantic interventions, widening the scope of black-box controllable generation.","The causal-decoupling trick provides a reusable recipe for generating counterfactuals that violate learned correlations (such as a woman with a beard), which naive LLM paraphrasing fails to achieve.","Effectiveness gains come with small increases in LPIPS and FVD, so the method trades a little distribution fidelity for stronger intervention fidelity, a trade-off the paper reports as consistent."],"supporting_citations":[{"why":"Supplies TextGrad textual gradient descent, the exact mechanism used to update counterfactual prompts from VLM criticisms.","marker":"Yuksekgonul et al. 2025"},{"why":"Provides the abduction-action-prediction counterfactual framework and graph mutilation that the paper maps onto DDIM inversion, prompt intervention, and DDIM sampling.","marker":"Pearl 2009"},{"why":"Defines causal effectiveness as an axiomatic counterfactual property that CSVC's VLM accuracy metric operationalizes.","marker":"Monteiro et al. 2023"},{"why":"Supplies the causal graph over facial attributes and the effectiveness/minimality evaluation criteria adapted to video counterfactuals.","marker":"Melistas et al. 2024"},{"why":"CelebV-Text is the source of the 67 real facial text-video pairs used for evaluation.","marker":"Yu et al. 2023"},{"why":"FLATTEN is one of the three black-box video editing systems used as the counterfactual generator.","marker":"Cong et al. 2024"},{"why":"Tune-A-Video is the one-shot video editing system used as a black-box generator.","marker":"Wu et al. 2023b"},{"why":"TokenFlow is the zero-shot video editing system used as a black-box generator.","marker":"Geyer et al. 2024"},{"why":"LLaVA-NeXT is the VLM used in the effectiveness metric (Eq. 4).","marker":"Li et al. 2024"}],"fun_headline_variants":["Prompt-only steering keeps video counterfactuals causal","Causal graph plus VLM steers black-box video editors","OOD video what-ifs without model access or retraining","CSVC: causal prompt tweaks for faithful video edits","Steer video diffusion with causal prompts and VLM feedback"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a vision-language model’s natural-language judgment of whether an intervention succeeded is a valid measure of causal fidelity, even though the same model family both proposes the prompt refinements and scores the outcome, with no human validation of the metric.","fun_headline_variants_meta":{"raw":{"variants":["Prompt-only steering keeps video counterfactuals causal","Causal graph plus VLM steers black-box video editors","OOD video what-ifs without model access or retraining","CSVC: causal prompt tweaks for faithful video edits","Steer video diffusion with causal prompts and VLM feedback"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1436,"prompt_tokens":1102,"completion_tokens":334,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":718,"completion_tokens_details":{"reasoning_tokens":253}},"tokens_in":718,"tokens_out":334,"duration_ms":3855,"temperature":1.0,"reasoning_tokens":253,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:52:06.122007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent human annotators label whether the target attribute changed and whether unrelated attributes stayed unchanged on the same CSVC-generated counterfactual videos; if human agreement with the paper’s VLM effectiveness scores is near chance, or if swapping the VLM judge for a different model family reverses the reported advantage over unsteered prompts, the central claim collapses.","supporting_citations":[],"review_version":1}