{"id":"b706f25a-6cc1-45aa-b87f-4fdd4f7e6fb3","arxiv_id":"2411.19415","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"An attention-modulated overshooting sampler for rectified flow models improves text rendering accuracy on SD3 and Flux without retraining or added inference cost.","lead":"The paper introduces AMO, a training-free modification to the standard Euler sampler for rectified flow text-to-image models that improves how accurately models render text in images. It works by briefly over-simulating the flow ODE in text-relevant regions and then re-injecting noise, which corrects accumulated sampling errors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline percentages are not reproducible from Table 2: the claimed 35.9% improvement on Flux does not match any reported metric, so the central quantitative claim is currently unsupported.","rationale":"I read the paper in good faith: the overshooting construction is clever, the ablation study supports that both overshooting and noise compensation are necessary, and the method is training-free with negligible runtime overhead. The most load-bearing concern, however, is not the perfect-velocity assumption highlighted by the reader. Even if the learned velocity is imperfect, the method could still improve text rendering empirically through noise injection and error correction; the marginal-preservation argument is an idealized limit and the authors themselves note that the discretized SDE is less stable than the overshooting update. The more direct threat to the central claim is that the abstract's headline numbers are not recoverable from the paper's own Table 2, and the human evaluation is small enough that the Flux improvement may be within sampling noise. This is an internal reporting inconsistency rather than a dispute with external consensus, and it can be settled by recomputation and confidence intervals. The paper remains conditionally acceptable if the authors clarify the metric behind 35.9% and report statistical uncertainty, so the reader's verdict is unchanged.","tokens_in":15859,"tokens_out":10267,"duration_ms":93102,"concrete_test":"Recompute all relative improvements from Table 2 and, using the released per-prompt human evaluation data, construct paired bootstrap or exact binomial confidence intervals for the correct-rate difference between Euler and AMO on Flux and SD3. Also identify the metric and step count that produces the 35.9% figure. If no reported metric or subset yields 35.9%, or if the Flux correct-rate 95% confidence interval includes zero, the central quantitative claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims a 32.3% and 35.9% improvement in text rendering accuracy on SD3 and Flux. From Table 2, the SD3 correct-rate increase from 32.5% to 43.0% is indeed a 32.3% relative improvement. However, for Flux, the correct rate rises from 74.0% to 82.5%, which is only an 11.5% relative increase; OCR-A rises from 0.313 to 0.381 (+21.7%), and OCR-F rises from 0.458 to 0.494 (+7.9%). No metric listed in the paper produces a 35.9% improvement. If the figure comes from an unreported subset, such as the 20-step runs in Figure 5, or from a different definition, that definition is not given. This matters because the abstract's central quantitative claim is not verifiable from the paper's own results. In addition, the human correct-rate numbers rest on 100 prompts with no confidence intervals or inter-annotator agreement; for Flux, the 8.5-point correct-rate gap has a binomial standard error of roughly 4.4 points, so the headline improvement may not be statistically significant. The paper should state exactly which metric and step count yields the 35.9% figure and provide uncertainty estimates before the central claim is accepted.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free stochastic sampling modification for pretrained rectified flow text-to-image models. At each denoising step, the Euler update is replaced by an 'overshooting' step that advances the ODE further than the target time and then adds noise whose coefficients are chosen to preserve the marginal law. An attention-modulated variant (AMO) scales the overshooting strength per image patch using cross-attention with text tokens, in order to reduce over-smoothing outside text regions. The paper derives an SDE limit for the overshooting sampler, shows its equivalence to a Langevin-corrected Euler step, and reports human and OCR evaluations on SD3, Flux, and AuraFlow claiming substantial text-rendering gains without extra training or inference cost.","tokens_in":122,"tokens_out":6466,"duration_ms":119937,"significance":"If the central claims hold, this would be a useful and cheap inference-time improvement for a real bottleneck in text-to-image generation, and the attention-modulation mechanism is a sensible way to localize stochastic correction. The paper has notable strengths: the SDE-limit derivation in Appendix A.1 is algebraically sound; the Fokker-Planck argument in Appendix A.2 connecting the overshooting sampler to a Langevin term is standard and correctly executed; the attention-weighted elementwise noise compensation is a neat practical trick; and the code is publicly released. However, the headline quantitative claim for Flux is not reproducible from the reported table, the human evaluation lacks uncertainty estimates and is too small to support the main Flux gap, and the overshooting strength is tuned per model on the same evaluation benchmarks. These issues are load-bearing for the abstract-level claims and must be fixed before the empirical conclusions can be accepted.","major_comments":[{"comment":"The abstract states that AMO demonstrates a 35.9% improvement in text rendering accuracy on Flux, but no metric in Table 2 yields that number. The correct rate rises from 74.0% to 82.5%, which is an 11.5% relative improvement; OCR-A rises 21.7%; OCR-F rises 7.9%. If the 35.9% figure comes from an unreported subset, such as the 20-step runs in Figure 5, the exact metric and step count must be stated and the underlying numbers reported. As written, the paper's central quantitative claim for Flux is not verifiable from its own results.","section":"Abstract; Table 2"},{"comment":"The human correct-rate evaluation uses 100 prompts with at least two raters per image, but no confidence intervals, inter-annotator agreement, or per-prompt breakdown are provided. For Flux, the 8.5-percentage-point gap between 74.0% and 82.5% has an approximate standard error of 5.8 percentage points under independent binomial sampling with n=100, so the difference is not statistically significant at the 5% level. The paper should report interval estimates and agreement statistics, and soften the claim of 'notably surpassing' Euler accordingly.","section":"Section 5, Table 2; Appendix A.3"},{"comment":"The overshooting strength c is a per-model hyperparameter (c=2 for Flux, c=1 for SD3 and AuraFlow in Table 4) and is selected on the same benchmark set used for evaluation: Figure 7 shows CR, OCR-F, and OCR-A versus c on Flux and the text says c=2 is chosen because performance plateaus there. This creates a selection-on-the-evaluation-set concern for the reported gains. The authors should either tune c on a separate validation split and report held-out numbers, or demonstrate that the qualitative conclusions are insensitive to c over a range fixed before evaluation.","section":"Section 5.2, Table 4, Figure 7"},{"comment":"The marginal-preservation guarantee and the score identity in Eq. (23) assume that the learned velocity v(Z_t,t) equals the true conditional expectation E[X_1-X_0 | X_t]. The paper's motivation is precisely that text rendering is difficult because the model's velocity is imperfect in text regions; if the dominant failure is model bias rather than Euler discretization error, the Langevin correction in Eq. (8) targets the wrong marginal and the coefficients a and b in Eq. (4) are not guaranteed to correct the actual error. This does not invalidate the empirical method, but the mechanism claim should be qualified and tested, for example by measuring velocity error or by comparing against a model with reliable text rendering.","section":"Section 3.1, Eq. (3); Appendix A.2, Lemma A.2"}],"minor_comments":[{"comment":"The text refers to 'GraphControl', but the model and the cited reference are 'GlyphControl'.","section":"Section 5.3 and Figure 8"},{"comment":"The conclusion contains the duplicated phrase 'Overshooting sampler sampler'.","section":"Section 6"},{"comment":"The figure would be much more informative with numerical values and error bars; currently it only shows bars for three metrics at three step counts, and the text uses it to support the low-step-count advantage.","section":"Figure 5"},{"comment":"The timestep schedule is not fully specified, in particular whether t0 is exactly 0 or a small epsilon; near t=0 the coefficients in Eq. (4) have a 1/o form, so the practical handling of the first step should be stated.","section":"Appendix A.3 and Algorithm 1"},{"comment":"The attention mask m_i is listed as an input to the overshooting update, but the paper should clarify whether it reuses cross-attention activations from the velocity evaluation or requires an extra forward pass, since the claim of negligible overhead depends on this.","section":"Algorithm 1, Eq. (9)"}],"recommendation":"major_revision","confidential_remarks":"The method is interesting and the theory is mostly sound, but the abstract's Flux improvement number is unsupported by the reported table and the human evaluation is underpowered for the main claim. The per-model tuning of c on the evaluation benchmark also needs explicit handling. These issues are fixable within the manuscript's scope, so major revision rather than rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. The core idea is clever: for rectified flow, overshoot the ODE step and then add noise to compensate, so the marginal is preserved. The authors show the limit SDE is Euler plus an extra Langevin term, and the attention mask that localizes overshooting to text patches is a practical fix for the over-smoothing that global overshooting causes. That part is genuinely new and the derivation is careful.\n\nWhat's solid: the SDE limit in Appendix A.1 is algebraically correct, and the Fokker-Planck equivalence in Section 3.2 is standard but correctly applied. Lemma A.2 derives the score for the linear interpolation, and the coefficients a and b are solved, not fitted. So the marginal-preservation claim is a genuine mathematical property, not a tautology. I believe the method works as described: adding overshooting in text regions does improve text rendering in the qualitative examples, and the effect is plausible.\n\nThe soft spots are mostly in the evaluation. The headline claim of a 35.9% improvement on Flux is not reproducible from the paper's own numbers. The human correct rate goes from 74.0% to 82.5%, an 11.5% relative gain; OCR-A goes up 21.7%; OCR-F up 7.9%. No metric in Table 2 gives 35.9%. It might be from the 20-step results in Figure 5, but those numbers are not reported. That is a reporting error that must be fixed. Second, the human evaluation covers only 100 prompts, with no confidence intervals or inter-annotator agreement; for Flux the 8.5-point gap has a binomial standard error around 4.4 points, so even the 82.5 vs 74.0 difference is not clearly significant. Third, the overshooting strength c is tuned separately for each model (2 for Flux, 1 for SD3 and AuraFlow) on the same benchmarks, which invites overfitting. Fourth, the only baseline is Euler; they don't compare to other stochastic flow samplers, so we can't separate the benefit of stochasticity from the benefit of this particular construction.\n\nThe deeper caveat: the derivation assumes the learned velocity is the true conditional expectation. Where that fails — which is exactly where text rendering fails — the Langevin correction targets the model's own marginal, not the true one. The fact that it still helps empirically is interesting, but it means the transfer to other models is not guaranteed. This is a caveat, not a fatal flaw.\n\nWho should read it: people working on sampling for flow models and on training-free ways to improve text-to-image. It deserves a serious referee, but the review must insist on a corrected abstract, uncertainty estimates, and at least one non-Euler baseline. I'd send it to review, not desk-reject, but with a clear request for major revision.","headline":"A clever and correctly derived stochastic sampler for rectified flow whose headline Flux improvement doesn't match any reported number — a real but fixable problem.","tokens_in":16657,"tokens_out":4590,"would_cite":false,"duration_ms":36668,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that a training-free overshooting sampler, alternating an over-simulated ODE step with matched noise injection, acts as a Langevin correction and improves text rendering accuracy on SD3, Flux, and AuraFlow without extra…","keywords":["text rendering","rectified flow","training-free sampling","Langevin dynamics","attention modulation","text-to-image generation","stochastic sampler","overshooting"],"falsifier":"Compare AMO at $c=2$ with Euler on a rectified-flow model whose velocity has been deliberately biased in text regions, e.g. a model fine-tuned to produce misspelled text; if correct-text rates do not improve or worsen, the Langevin correction is chasing the wrong marginal. A more direct test on a synthetic flow with known $v$ would compare the empirical distribution of overshot samples $\\hat Z_o$ to $oX_1+(1-o)X_0$ and reveal any mismatch in the coefficient formula $b=\\sqrt{(1-s)^2-a^2(1-o)^2}$.","tokens_in":15669,"feed_emoji":"✍️","tokens_out":7507,"duration_ms":59949,"temperature":0.7,"pith_summary":"The paper tries to fix misspelled or garbled text in images produced by rectified-flow text-to-image models, without retraining or extra inference cost. It claims that alternating an overshot ODE step with a matched noise-injection step is mathematically equivalent to adding a Langevin dynamics term that pushes samples back toward the correct marginal distribution, countering errors accumulated by successive Euler steps. To avoid over-smoothing, it modulates the overshooting strength per image patch using cross-attention between text tokens and image patches, calling the combined sampler AMO. On SD3, Flux, and AuraFlow, the reported correct-text rate rises by 32.3% and 35.9% relative on the two stronger models, with FID and CLIP scores staying roughly level. A reader should care because this type of fix can be dropped into any pretrained flow model rather than requiring new training.","feed_headline":"Training-free sampler cuts AI text errors by a third","feed_subtitle":"A noise-correction step at inference boosts correct text in SD3 and Flux images, with no extra training cost.","key_machinery":"The central object is the overshooting sampler update: from $\\tilde Z_t$ at time $t$, compute $\\hat Z_o = \\tilde Z_t + (o-t)v(\\tilde Z_t,t)$ with $o = t + (1+c)\\epsilon$, then set $\\tilde Z_s = a\\hat Z_o + b\\xi$ for $\\xi\\sim N(0,I)$, with $a = s/o$ and $b = \\sqrt{(1-s)^2 - a^2(1-o)^2}$. In the $\\epsilon\\to0$ limit this becomes the SDE in the core discovery, i.e. the learned ODE plus a Langevin step controlled by $c$. The attention modulation replaces the scalar $c$ with $c\\,m_{h,w}$, where $m_{h,w}$ is a normalized cross-attention map between the text tokens and each image patch, so that overshooting acts strongly on text-related regions and weakly elsewhere.","core_discovery":"The central claim is that the overshooting update, where one Euler step lands at time $o = s + c\\epsilon$ and a coefficient-matched noise re-introduction returns to $s$, preserves the marginal law of the rectified flow in the small-step limit. In that limit the update becomes the SDE $dZ_t = ((1+c)v(Z_t,t) - c Z_t/t)\\,dt + \\sqrt{2c(1-t)/t}\\,dW_t$, which equals the original ODE plus a Langevin correction term whose step size is $c(1-t)/t$. Because successive Euler steps make the particle distribution drift from $\\rho_t$, the Langevin term acts as a denoising correction that improves text rendering; attention modulation then scales $c$ per patch using $m_{h,w}$, the normalized cross-attention weight between text tokens and image patch $(h,w)$, confining the correction to text areas. In the paper's own terms, 'AMO demonstrates a 32.3% and 35.9% improvement in text rendering accuracy on SD3 and Flux without compromising overall image quality or increasing inference cost.'","pith_inferences":["Beyond the paper: the Langevin-correction mechanism should apply to any ODE-based generative model, not just rectified-flow text rendering, because the derivation only requires the marginal-preservation identity and the score formula $\\nabla\\log\\rho_t=(tv(x,t)-x)/(1-t)$.","Beyond the paper: the attention mask $m_{h,w}$ could be reused or approximated with a single attention layer, and AMO could be evaluated on a hand-keypoint benchmark to quantify the appendix's anecdotal improvement in hands and body structure.","Beyond the paper: if the dominant text error is model bias rather than discretization drift, the correction will be limited; a synthetic test with a deliberately mis-specified velocity would separate the two error sources."],"forward_implications":["On SD3, Flux, and AuraFlow, AMO raises the correct-text rate in human evaluation relative to Euler: 43.0% vs 32.5% on SD3, 82.5% vs 74.0% on Flux, and 3.0% vs 1.0% on AuraFlow.","The gains persist at 20, 50, and 100 sampling steps, with larger relative improvements at the smaller step counts.","FID and CLIP scores stay roughly level or improve slightly, so the text correction does not trade away overall image quality.","The sampler is training-free and model-agnostic for rectified flow: it only requires access to the velocity network and the cross-attention maps.","Ablations show both noise compensation and attention modulation are necessary; overshooting alone without the matched noise step collapses the marginal and yields a 0% correct-text rate."],"supporting_citations":[{"why":"Defines rectified flow and the marginal-preserving ODE that the overshooting sampler modifies.","marker":"[19]"},{"why":"Supplies the Fokker-Planck / Langevin derivation that connects the overshooting SDE to marginal correction.","marker":"[33]"},{"why":"Prior stochastic sampler with noise injection; used as reference and source of the Langevin correction idea.","marker":"[14]"},{"why":"Stable Diffusion 3, the first evaluated model and motivation for text-rendering difficulty in RF transformers.","marker":"[7]"},{"why":"Flux, the second evaluated state-of-the-art RF model.","marker":"[15]"},{"why":"AuraFlow, the third evaluated RF model where AMO also improves OCR metrics.","marker":"[8]"},{"why":"Mask TextSpotter v3, the OCR model used to compute OCR accuracy and F-measure.","marker":"[17]"},{"why":"TextDiffuser, a finetuned text-rendering model used as benchmark baseline and prompt source.","marker":"[5]"},{"why":"GlyphControl, a finetuned text-rendering model used for human-evaluation comparison.","marker":"[22]"}],"fun_headline_variants":["Training-free sampler sharpens AI text by over 30%","Overshooting noise step enhances AI image text at no cost","AMO sampler improves AI text rendering without extra training","New sampler reduces AI text errors by 32% with zero training","Attention-modulated overshooting enhances AI text for free"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The derivation assumes the learned velocity field matches the true conditional expectation, so that the overshot sample follows the correct flow marginal; in text regions, where the model is imperfect, this assumption is exactly what is least safe.","fun_headline_variants_meta":{"raw":{"variants":["Training-free sampler sharpens AI text by over 30%","Overshooting noise step enhances AI image text at no cost","AMO sampler improves AI text rendering without extra training","New sampler reduces AI text errors by 32% with zero training","Attention-modulated overshooting enhances AI text for free"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000962,"raw_usage":{"total_tokens":4133,"prompt_tokens":1018,"completion_tokens":3115,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":634,"completion_tokens_details":{"reasoning_tokens":3032}},"tokens_in":634,"tokens_out":3115,"duration_ms":19760,"temperature":1.0,"reasoning_tokens":3032,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:11:48.261369+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare AMO at $c=2$ with Euler on a rectified-flow model whose velocity has been deliberately biased in text regions, e.g. a model fine-tuned to produce misspelled text; if correct-text rates do not improve or worsen, the Langevin correction is chasing the wrong marginal. A more direct test on a synthetic flow with known $v$ would compare the empirical distribution of overshot samples $\\hat Z_o$ to $oX_1+(1-o)X_0$ and reveal any mismatch in the coefficient formula $b=\\sqrt{(1-s)^2-a^2(1-o)^2}$.","supporting_citations":[{"cited_title":"Elucidating the design space of diffusion-based generative models","cited_arxiv_id":null,"evidence_quote":"Prior stochastic sampler with noise injection; used as reference and source of the Langevin correction idea."},{"cited_title":"Scaling rectified flow trans- formers for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion 3, the first evaluated model and motivation for text-rendering difficulty in RF transformers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Flux, the second evaluated state-of-the-art RF model."},{"cited_title":"Aura Flow Pipeline Documentation , 2023","cited_arxiv_id":null,"evidence_quote":"AuraFlow, the third evaluated RF model where AMO also improves OCR metrics."},{"cited_title":"Mask textspotter v3: Segmentation proposal network for robust scene text spotting","cited_arxiv_id":null,"evidence_quote":"Mask TextSpotter v3, the OCR model used to compute OCR accuracy and F-measure."},{"cited_title":"Textdiffuser: Diffusion models as text painters","cited_arxiv_id":null,"evidence_quote":"TextDiffuser, a finetuned text-rendering model used as benchmark baseline and prompt source."}],"review_version":1}