Pith. sign in

REVIEW 5 major objections 6 minor 102 references

DETONATE: A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that replacing the scalar preference term in Direct Preference Optimization with a kernelized embedding similarity, paired with Rényi divergence, turns text-to-image safety alignment into a structural property of latent…

desk verdict The kernel term that is supposed to make this 'geometry-aware' is a constant in the implemented training loop, so the paper's central claim is not supported by its own code. read the letter →

arxiv 2506.14903 v1 pith:RGGIH6EF submitted 2025-06-17 cs.CV

Renjith Prasad , Abhilekh Borah , Hasnat Md Abdullah , Chathurangi Shyalika , Gurpreet Singh , Ritvik Garimella , Rajarshi Roy , Harshul Surana
show 4 more authors
Nasrin Imanpour Suranjana Trivedy Amit Sheth Amitava Das
This is my paper · ORCID
classification cs.CV
keywords text-to-imagealignmentdirectpreferenceoptimizationkernelmethodsRényidivergencesafetysocialbiasbenchmarklatent-spaceevaluationdiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that alignment for text-to-image models should be treated as a structural property of latent geometry, not as surface-level behavior. It proposes DPO-Kernels, which replaces the scalar preference ratio of Direct Preference Optimization with a kernelized similarity over prompt and image embeddings and swaps the KL regularizer for Rényi or Wasserstein divergences. To test this, it builds DETONATE, about 100K chosen/rejected image pairs drawn from hate-speech prompts across race, gender, and disability, and introduces the Alignment Quality Index (AQI), a cluster-separability metric for latent activations. A sympathetic reader would care because the paper offers a concrete way to bake safety into the model's representations rather than filtering outputs after the fact.

What carries the argument

The central object is the DPO-Kernels objective (Eq. 1). It maximizes a kernelized preference score equal to $\log\frac{\pi(y_+|x)}{\pi(y_-|x)}$ plus $\gamma$ times $\log\frac{\kappa(e_x,e_{y_+})}{\kappa(e_x,e_{y_-})}$, where $\kappa$ is an RBF, polynomial, or wavelet kernel over prompt and image embeddings, and subtracts $\alpha$ times a divergence between policy and reference denoising-error distributions for chosen versus rejected samples. The divergence can be KL, Wasserstein, or Rényi. The paper also introduces the Alignment Quality Index (AQI), a diagnostic combining a Davies-Bouldin score and a Dunn index on pooled UNet activations to measure cluster separability of safe and unsafe generations. These components carry the argument: the kernel term supplies semantic locality, the divergence shapes gradient stability, and AQI provides the latent-space evidence.

What would settle it

Take a random sample of DETONATE pairs and have fresh human annotators independently label image hatefulness; if their agreement with the released chosen and rejected labels drops well below the reported Cohen's kappa of 0.86, then the preference pairs and the reported safety gains rest on unreliable ground truth.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a kernelized preference score, which combines the log-likelihood ratio with a kernel ratio between prompt and chosen/rejected image embeddings, trained under a divergence-regularized diffusion objective, yields safety alignment that is both behaviorally and representationally stronger than scalar DPO, DDPO, and SAFREE. The paper reports that RBF + Rényi achieves the best results on both backbones, with Toxicity at 0.12/0.11 and AQI at 0.80/0.80 on SD-XL and SD-v1.5, and that AQI shows safe and unsafe generations become geometrically separable in UNet activations after training. The paper's stated conclusion is that alignment is best achieved through structural regularization in representation space, not post hoc filtering.

Load-bearing premise

The load-bearing premise is that the DETONATE chosen and rejected labels are correct, since all preference training and every measured safety improvement inherits whatever errors the VLM-plus-human annotation pipeline made.

Editorial extensions

If this is right

  • If the central claim is correct, safety alignment can be achieved by training-time structural regularization, making inference-time filters like SAFREE unnecessary or at most complementary.
  • Rényi divergence as a regularizer appears to produce sharper alignment boundaries than KL, so future alignment methods should treat divergence choice as a first-class design decision.
  • AQI can expose latent misalignment even when outputs look safe, offering a diagnostic for alignment faking and a way to audit models before deployment.
  • DETONATE provides a public preference-based benchmark spanning race, gender, and disability, enabling direct comparison of future alignment techniques.
  • Spectral analysis with Heavy-Tailed Self-Regularization suggests kernel-divergence pairs trade off expressivity against generalization, so the best safety score may not be the most generalizable configuration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the method's validation depends entirely on DETONATE labels, an independent re-annotation study would be the cleanest test: any drop in label agreement would directly bound how much of the reported toxicity and AQI gains are genuine.
  • The kernelized preference objective is written over generic text and image embeddings, so the same loss could plausibly transfer to text-to-video or multimodal reward models, though the paper only demonstrates it on diffusion-based text-to-image models.
  • AQI is presented as an evaluation metric, but its layer-wise activation pooling could also serve as a training-time early-stopping signal, letting practitioners stop once safe and unsafe clusters separate; the paper does not test this.
  • The comparison across RBF, polynomial, and wavelet kernels suggests kernel bandwidth and polynomial degree interact with divergence choice, so a systematic sweep over those hyperparameters could show whether RBF + Rényi's lead is robust or specific to the chosen settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes DPO-Kernels, a kernelized extension of Direct Preference Optimization for text-to-image diffusion models, combining a log-likelihood preference ratio with a kernel embedding similarity term and a diffusion denoising regularizer. It also introduces DETONATE, a benchmark of approximately 100K chosen/rejected image pairs derived from hate-speech prompts across race, gender, and disability axes, and proposes the Alignment Quality Index (AQI), a latent-space cluster-separability metric. The authors report that RBF+Rényi outperforms all baselines on toxicity, CMMD, CLIP score, and AQI, and claim that the method achieves structural, geometry-aware alignment rather than surface-level filtering. The central mechanism is that the kernelized embedding term in Eq. (1) provides semantically localized gradients during training.

Significance. If the method worked as described, the paper would offer three useful contributions: a large preference dataset for safety alignment in T2I generation, a kernelized preference objective with alternative divergences, and an intrinsic latent-space diagnostic. DETONATE and the code release are stated as public, and the pseudocode in Appendix I is a reproducibility strength that makes the training procedure concrete. However, the implemented kernel term is constant with respect to the policy parameters, so the claimed gradient mechanism is not actually optimized. The AQI metric is a reasonable idea, but its use as evidence for the proposed method is compromised by its direct alignment with the stated objective and by the absence of any reliability analysis. As presented, the empirical claims are not supported, and the central theoretical framing does not match the implementation.

major comments (5)
  1. [Section 3, Eq. (1), Pseudocode I] The kernelized embedding term gamma * log(kappa(e_x, e_y+) / kappa(e_x, e_y-)) is presented as part of the optimized objective, but in the training loop the chosen and rejected images are converted to numpy uint8 arrays and passed through a frozen image_embedding_model; the resulting embeddings are constants with respect to the UNet parameters theta. Consequently, this term contributes only a per-pair scalar to the logit inside the log-sigmoid loss. It cannot introduce gradient directions that localize updates around semantically proximate regions, as claimed in Fig. 3, Table 1, and Section 5. Any observed improvements could be produced by the standard DPO/log-likelihood and denoising regularization terms alone, or by a per-pair reweighting effect. This is an internal inconsistency between the proposed objective and its implementation, and it invalidates the central claim of geometry-aware alignment.
  2. [Section 3, Eq. (1), Table 1] Equation (1) is asserted rather than derived from DPO or RLHF; no derivation shows that maximizing this composite objective corresponds to a valid preference optimization problem or to RKHS risk minimization. Table 1 applies polynomial, RBF, and wavelet kernels to scalar log-ratios rather than to embedding vectors; for example, the 'RBF' formula exp(-(log(pi(y+|x)/pi(y-|x)))^2 / (2 sigma^2)) is not the Gaussian kernel over embeddings, and the polynomial expression (log ratio + c)^d is not guaranteed to be positive or to define a valid kernel. The paper needs a formal statement of the kernel space and a derivation of the objective before the kernelized preference score can be treated as theoretically grounded.
  3. [Section 2 and Appendix B.3] The dataset statistics are internally inconsistent. Section 2 states approximately 25K prompts and approximately 100K curated image pairs, with around 25K pairs from each of the three categories Race, Gender, and Disability; 3 x 25K equals 75K, not 100K. Moreover, since each prompt yields one chosen and one rejected image, 25K prompts would imply 25K pairs, not 100K. The benchmark scale and the pair-generation procedure need to be clarified and corrected.
  4. [Table 2 (and Table 7)] Every metric in Table 2 is reported as a single scalar with no error bars, no number of evaluation samples, no seeds, and no significance tests. Differences such as CLIP score 0.410 versus 0.395 across variants are likely within noise. The statements that DPO-Kernel variants 'consistently outperform all baselines' and that RBF+Renyi is 'best-in-class' are not supported without variance estimates or paired statistical tests.
  5. [Section 4 and Section 5] AQI measures latent cluster separability of safe versus unsafe activations, while the proposed objective in Eq. (1) explicitly rewards a high ratio kappa(e_x, e_y+) / kappa(e_x, e_y-), which is a similarity-based separation between chosen and rejected images relative to the prompt. Thus the reported AQI improvements are partly a direct measure of the training objective's effect, not an independent diagnostic of structural alignment. The paper does not discuss or control for this circularity, and it should.
minor comments (6)
  1. [Section 2 and Fig. 2] The text mentions 'four protected axes' but only three axes (Race, Gender, Disability) are used; Appendix B.3 also says 'four thematic categories' while listing three. Please make the count consistent.
  2. [Appendix D vs. Eq. (1)] Appendix D defines Rényi and Wasserstein divergences for distributions pi over outputs y, whereas Eq. (1) uses a divergence D applied to denoising error vectors err_theta(y+) and err_ref(y+); the relationship between these two uses is never defined.
  3. [Section 5.1 and Table 10] The main text says Wavelet + Wasserstein has alpha-hat = 3.64 and exceeds the threshold, but Table 10 lists Wavelet + Rényi as 3.84 and Wavelet + Wasserstein as 3.64, while Section H.2 states that Wavelet + Wasserstein yields 3.84. These numbers are inconsistent and should be reconciled.
  4. [Pseudocode I] The pseudocode uses fixed weights 0.5 for the embedding term and the KL terms, but Eq. (1) has free hyperparameters gamma and alpha; the mapping between the equation and the implementation is unspecified.
  5. [References] Several references are incomplete or placeholders, including 'Author Unknown' [61], anonymous entries with arXiv IDs such as 2404.XXXX [7-9], and a self-citation [96] to the same work; these need to be resolved before submission to a journal.
  6. [Tables 2 and 7] Table 2 and Table 7 report the same evaluation results; keeping both is redundant and should be consolidated.

Circularity Check

2 steps flagged · score 6.0 of 10

The kernelized embedding term in Eq. 1 is constant w.r.t. the policy in the implemented training loop, so the claimed kernel-driven AQI gains reduce by construction to standard DPO with static per-pair weights; the FAQ also relies on a self-citation.

  1. other [Sec. 3 Eq. (1) and Appendix I Pseudocode (dpo_kernel_loss)]
    "max π Ex,y+,y− κ[ log(pθ(y+|x)/pθ(y−|x)) + γ log(κ(ex, ey+)/κ(ex, ey−)) ] − α·[D[errθ(y+)∥errref(y+)]−D[errθ(y−)∥errref(y−)]] (1) ... win_np = [img.squeeze(0).permute(1, 2, 0).cpu().numpy().astype(np.uint8) for img in win_imgs]; lose_np = [img.squeeze(0).permute(1, 2, 0).cpu().numpy().astype(np.uint8) for img in lose_imgs]; ... win_emb = image_embedding_model.encode_images(win_np); lose_emb = image_embedding_model.encode_images(lose_np); ... log_emb_diff = torch.log((kernel_win + 1e-10) / (kernel_lose + 1e-10))"

    In the implemented training loop the chosen/rejected images are detached to numpy uint8 before embedding, so ey+ and ey−, and hence κ(ex,ey+)/κ(ex,ey−), are constants with respect to the UNet parameters θ. The kernelized embedding term in Eq. (1) therefore contributes only a fixed per-pair offset to the logit inside logsigmoid; it cannot produce the 'localized gradients' or 'geometry-aware' updates claimed in Sec. 3 and Fig. 3. Kernel-dependent results in Table 2 (e.g., RBF vs Wavelet AQI/toxicity differences) are not attributable to the proposed mechanism: by construction the objective reduces to standard DPO with static sample weights, and the kernel choice is a fixed input, not a learned geometric predictor.

  2. self citation load bearing [Sec. 8 FAQ ('What is the performance of DPO-Kernel under adversarial prompting scenarios?') and References [96]]
    "Across the DETONATE benchmark [96], which spans a broad spectrum of social axes (e.g., race, gender, disability), DPO-Kernel consistently outperforms baseline alignment techniques including standard DPO [77], DDPO [78], and SAFREE [79], achieving higher alignment precision and significantly reduced rate of unsafe completions under adversarial attacks. ... [96] Anonymous. Dpo-kernels and the detonate benchmark: Geometry-aware preference optimization for safer text-to-image generation, 2025. Under review at NeurIPS 2025."

    Reference [96] is the present manuscript itself. The FAQ invokes it as the authority for DETONATE's scope and for DPO-Kernel's empirical superiority, but a paper citing itself under review provides no independent evidence. The claim is not machine-checked, code-reproduced in an external artifact, or otherwise verified outside the paper's own fitted results, so this self-citation is load-bearing in the FAQ's argument and adds no support beyond the same manuscript's tables.

full rationale

Most of the paper's data-construction and evaluation pipeline is not classically circular: DETONATE labels come from VLM-plus-human annotation, and AQI is an extrinsic cluster metric computed on held-out prompts. The central circularity is in the optimization claim. Equation (1) presents γ log κ(ex,ey+)/κ(ex,ey−) as a learnable, geometry-aware preference term, but the Appendix I pseudocode detaches the chosen/rejected images via .cpu().numpy().astype(np.uint8) before embedding, so the term is constant with respect to the UNet parameters. The kernelized preference score therefore reduces by construction to standard DPO with static per-pair offsets, and kernel-dependent AQI/toxicity results cannot be attributed to the proposed kernel mechanism. The FAQ additionally leans on reference [96], which is the present manuscript, as authority for DETONATE and DPO-Kernel's superiority, a self-citation that adds no independent support. Because the central mechanism reduces to a constant input, but the benchmark construction and the external toxicity/CLIP/CMMD metrics retain independent content, the score is 6 rather than higher.

Assumptions & free parameters 7 free parameters · 5 assumptions · 1 invented entities

The central claims rest on a substantial set of unreported free parameters, an asserted rather than derived objective, and an unvalidated geometric evaluation metric. The dataset labeling assumption is load-bearing because all preference pairs and AQI clusters are built on it.

free parameters (7)
  • β (DPO temperature) = β=2000 (SD1.5), β=5000 (SD-XL)
    Tuned to yield strongest results; controls the log-probability ratio term in the preference loss.
  • γ (kernel embedding similarity weight)
    Balances log-ratio and embedding similarity terms in Eq. 1; value not reported.
  • α (denoising regularizer weight)
    Weights the diffusion denoising divergence term in Eq. 1; value not reported.
  • σ (kernel bandwidth)
    Controls locality for RBF and wavelet kernels; stated as sensitive but not reported.
  • d and c (polynomial degree and offset)
    Shape the polynomial kernel nonlinearity and offset; not reported.
  • Rényi order parameter
    Controls sensitivity of Rényi divergence; not reported.
  • AQI γ
    Weight in the convex combination of Davies-Bouldin and Dunn indices; not reported.
assumptions (5)
  • domain assumption Alignment is encoded in local topology of embeddings rather than likelihood.
    Foundational premise for the kernelized embedding term; asserted in Sec. 3 with intuition but no empirical or theoretical validation.
  • domain assumption Explicit visual hate can be reliably annotated by LLaVA VLMs plus human review.
    Chosen and rejected pairs in DETONATE depend on this; reported Cohen's kappa of 0.86/0.89 does not guarantee correctness on borderline images.
  • ad hoc to paper The composite objective in Eq. 1 is a valid preference optimization loss.
    No derivation from DPO or RLHF; the loss is asserted and then used for all experiments.
  • domain assumption Heavy-tailed spectral exponent alpha is a valid generalization proxy.
    Used to claim generalization properties; cited from Martin et al., but no formal bound is derived here.
  • domain assumption AQI computed from penultimate UNet activations captures alignment quality.
    Assumed without external validation; the paper does not show AQI against independent human judgments.
invented entities (1)
  • Alignment Quality Index (AQI)
    purpose: New evaluation metric to score latent-space separability of safe and unsafe image activations.
    Not validated against external human judgments or independent safety outcomes; used to adjudicate the method it was designed to justify.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DETONATE: A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization." pith.science (2026). https://pith.science/paper/RGGIH6EF

@misc{pith2026250614903,
  author       = {Pith},
  title        = {Pith review of: DETONATE: A Benchmark for Text-to-Image Alignment and Kernelized Direct Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGGIH6EF}},
  note         = {Machine review of arXiv:2506.14903}
}
read the original abstract

Alignment is crucial for text-to-image (T2I) models to ensure that generated images faithfully capture user intent while maintaining safety and fairness. Direct Preference Optimization (DPO), prominent in large language models (LLMs), is extending its influence to T2I systems. This paper introduces DPO-Kernels for T2I models, a novel extension enhancing alignment across three dimensions: (i) Hybrid Loss, integrating embedding-based objectives with traditional probability-based loss for improved optimization; (ii) Kernelized Representations, employing Radial Basis Function (RBF), Polynomial, and Wavelet kernels for richer feature transformations and better separation between safe and unsafe inputs; and (iii) Divergence Selection, expanding beyond DPO's default Kullback-Leibler (KL) regularizer by incorporating Wasserstein and R'enyi divergences for enhanced stability and robustness. We introduce DETONATE, the first large-scale benchmark of its kind, comprising approximately 100K curated image pairs categorized as chosen and rejected. DETONATE encapsulates three axes of social bias and discrimination: Race, Gender, and Disability. Prompts are sourced from hate speech datasets, with images generated by leading T2I models including Stable Diffusion 3.5 Large, Stable Diffusion XL, and Midjourney. Additionally, we propose the Alignment Quality Index (AQI), a novel geometric measure quantifying latent-space separability of safe/unsafe image activations, revealing hidden vulnerabilities. Empirically, we demonstrate that DPO-Kernels maintain strong generalization bounds via Heavy-Tailed Self-Regularization (HT-SR). DETONATE and complete code are publicly released.

Figures

Figures reproduced from arXiv: 2506.14903 by the authors.

Figure 1
Figure 1. Text-to-Image Alignment under Hateful Prompts: A Visual and Latent Space Com￾parison across Race, Disability, and Gender Axes. This figure compares four models—Vanilla SD-XL [1], SD-XL + DDPO [2], SAFREE [3], and SD-XL + DPO-Kernel (Ours)—under toxic prompt conditions. Visually (left), DPO-Kernel generates respectful and non-provocative images, neutralizing bias where prior methods fail. For gender prompts, red mask… view at source ↗
Figure 2
Figure 2. DETONATE Benchmark Curation Framework. The pipeline has three stages: (i) Collection: Prompts are extracted from public hate speech datasets and filtered across four protected axes—race, gender, and disability—using keyword heuristics (cf. Sec. 2). (ii) Image Generation: Prompts are processed through SoTA T2I models (SD-XL, SD-3.5 Large, and Midjourney) to generate ten diverse images each. (iii) Annotation: Images a… view at source ↗
Figure 3
Figure 3. Effect of Kernelization on DPO Loss Landscapes. Each subplot visualizes the induced alignment surface for a given kernel choice in the DPO-Kernels framework. Vanilla DPO exhibits a flat linear preference slope with no geometric structure. Polynomial Kernels introduce global curvature, capturing higher-order interactions. RBF Kernels localize gradients around semantically proximate regions, creating a rugged surface … view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Oscillatory patterns of KL, Rényi, and Wasserstein divergences over training, highlight￾ing their differing sensitivity to evolving align￾ment dynamics. We evaluate two alternatives: Rényi diver￾gence [53], which amplifies contrast in high￾sensitivity regimes via its o…
Figure 5
Figure 5. Figure 5: Illustration of our AQI. We visualize the activations of 800 samples generated by SD-XL after DPO-Kernel (RBF+KL) alignment, having 400 safe (green) and 400 unsafe (red) instances. We show the calculation of DBS (left) using the distance between safe-centroid (triangle…
Figure 6
Figure 6. Figure 6: AQI heatmap across alignment axes for DPO-Kernel variants. Darker shades de￾note higher scores; yellow borders highlight the top three. Wavelet variants excel, particu￾larly on Race and Disability. Quantitative Results: We evaluate DPO-Kernel variants on SD-XL and SD-v…
Figure 7
Figure 7. Figure 7: Generalization vs. overfitting trade-off [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Overview of the proposed DPO-Kernel method applied to diffusion models. The model [PITH_FULL_IMAGE:figures/full_fig_p040_8.png]
Figure 9
Figure 9. Figure 9: Contour plots with gradient descent fields for various kernel functions. Each subplot [PITH_FULL_IMAGE:figures/full_fig_p043_9.png]
Figure 10
Figure 10. Figure 10: AQI heatmap across alignment axes for DPO-Kernel variants. Darker shades denote higher [PITH_FULL_IMAGE:figures/full_fig_p047_10.png]
Figure 11
Figure 11. Figure 11: Latent Geometry of Alignment Across Social Axes and Epochs. This figure visualizes the evolution of 3D latent representations for aligned (green) and misaligned (red) generations across three key social alignment axes—Race (Row 1), Gender (Row 2), and Disability (Row …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

102 extracted references · 53 canonical work pages

  1. [1]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    arXiv 2023

  2. [2]

    Ddpo: Denoising diffusion policy optimization

    Eric Wallace, Simran Arora, Eric Zelikman, Colin Raffel, and Tatsunori Hashimoto. Ddpo: Denoising diffusion policy optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  3. [4]

    Alignment faking in language models.arXiv preprint arXiv:2412.14093, 2024

    Yao Fu, Taneli Mielikäinen, Xiao Liu, Leo Gao, Dan Roth, Denny Zhou, and Xiang Lisa Li. Alignment faking in language models.arXiv preprint arXiv:2412.14093, 2024

    arXiv 2024

  4. [5]

    Facing reality? law enforcement and the challenge of deepfakes, 2024

    EUROPOL. Facing reality? law enforcement and the challenge of deepfakes, 2024. Accessed: 2025-01-12

    2024

  5. [6]

    More speech and fewer mistakes, 2025

    Joel Kaplan. More speech and fewer mistakes, 2025. Accessed: 2025-01-12

    2025

  6. [7]

    Prompt-noise optimization for safe text-to-image generation, 2024

    Anonymous. Prompt-noise optimization for safe text-to-image generation, 2024. arXiv preprint arXiv:2404.XXXX

    2024

  7. [8]

    Posi: Prompt optimization with safety intent via reinforcement learning, 2024

    Anonymous. Posi: Prompt optimization with safety intent via reinforcement learning, 2024. arXiv preprint arXiv:2403.XXXX

    2024

  8. [9]

    Embedding sanitizer: Prompt-level harm suppression for diffusion models, 2024

    Anonymous. Embedding sanitizer: Prompt-level harm suppression for diffusion models, 2024. arXiv preprint arXiv:2402.XXXX

    2024

Show all 102 references
  1. [10]

    Beyond accuracy: Behavioral testing of nlp models with checklist

    Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Behavioral testing of nlp models with checklist. InACL, 2020

    2020

  2. [11]

    Training language models to follow instructions with human feedback.arXiv preprint arXiv:2203.02155, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, et al. Training language models to follow instructions with human feedback.arXiv preprint arXiv:2203.02155, 2022

    2022 arXiv

  3. [12]

    Diffusion-dpo: Direct preference optimization for text-to-image models

    Eric Wallace et al. Diffusion-dpo: Direct preference optimization for text-to-image models. In NeurIPS, 2024

    2024

  4. [13]

    Safetydpo: Modular alignment of diffusion models with ai feedback, 2024

    Arjun Singh et al. Safetydpo: Modular alignment of diffusion models with ai feedback, 2024. arXiv preprint arXiv:2405.XXXX

    2024

  5. [14]

    Safety-constrained direct preference optimization for diffusion models, 2025

    Anonymous. Safety-constrained direct preference optimization for diffusion models, 2025. arXiv preprint arXiv:2502.XXXX

    2025

  6. [15]

    Rankdpo: Scaling preference optimization with synthetic image rankings, 2024

    Anonymous. Rankdpo: Scaling preference optimization with synthetic image rankings, 2024. arXiv preprint arXiv:2403.XXXX

    2024

  7. [16]

    Imagereward: Open-source visual reward models for image generation, 2023

    Yuheng Xu et al. Imagereward: Open-source visual reward models for image generation, 2023. arXiv preprint arXiv:2304.05977

    2023 arXiv

  8. [17]

    Visionreward: General-purpose reward modeling for visual content, 2024

    Anonymous. Visionreward: General-purpose reward modeling for visual content, 2024. arXiv preprint arXiv:2402.XXXX

    2024

  9. [18]

    Safety and ethics in the era of generative ai

    Rishi Bommasani, Percy Liang, Yuntao Wu, et al. Safety and ethics in the era of generative ai. arXiv preprint arXiv:2306.03772, 2023

    2023 arXiv

  10. [21]

    Concept steerers: Sparse monosemantic intervention for safety in diffusion models,

    Anonymous. Concept steerers: Sparse monosemantic intervention for safety in diffusion models,

  11. [22]

    A mathematical framework for transformer circuits, 2022

    Nelson Elhage, Tom Henighan, Neel Nanda, Catherine Olsson, Nicholas Schiefer, Andy Jones, Ben Mann, Jacob Steinhardt, and Chris Olah. A mathematical framework for transformer circuits, 2022. Transformer Circuits Thread

    2022

  12. [23]

    From languages to geographies: Towards evaluating cultural bias in hate speech datasets

    Manuel Tonneau, Diyi Liu, Samuel Fraiberger, Ralph Schroeder, Scott Hale, and Paul Röttger. From languages to geographies: Towards evaluating cultural bias in hate speech datasets. In Yi-Ling Chung, Zeerak Talat, Debora Nozza, Flor Miriam Plaza-del Arco, Paul Röttger, Aida Mos...

    2024

  13. [24]

    Constructing interval variables via faceted rasch measurement and multitask deep learning: a hate speech application

    Chris J Kennedy, Geoff Bacon, Alexander Sahn, and Claudia von Vacano. Constructing interval variables via faceted rasch measurement and multitask deep learning: a hate speech application. arXiv preprint arXiv:2009.10277, 2020

    2009 arXiv

  14. [25]

    2024.https://www.midjourney.com/home

    Midjourney. 2024.https://www.midjourney.com/home

    2024

  15. [26]

    SD-3.5 Large. 2024. https://stability.ai/news/ introducing-stable-diffusion-3-5

    2024

  16. [27]

    Alignbench: Evaluating and advancing alignment for language models

    Yuxuan Wang et al. Alignbench: Evaluating and advancing alignment for language models. arXiv preprint arXiv:2312.14047, 2023

    2023 arXiv

  17. [28]

    Lima: Less is more for alignment.arXiv preprint arXiv:2305.11206, 2023

    Andy Zhou, Nathanael Schärli, Le Hou, et al. Lima: Less is more for alignment.arXiv preprint arXiv:2305.11206, 2023

    2023 arXiv

  18. [29]

    Gpt-4 technical report, 2023

    OpenAI. Gpt-4 technical report, 2023. arXiv preprint arXiv:2303.08774

    2023 arXiv

  19. [30]

    Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36, 2024

    2024

  20. [31]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  21. [32]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    2024

  22. [33]

    Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    Yuntao Bai, Sherry Kadavath, Sandipan Kundu, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862, 2022

    2022 arXiv

  23. [35]

    Scaling direct preference optimization for fine-grained reward specification.arXiv preprint arXiv:2310.12036, 2023

    Steven Gao, Ramin Rafailov, Eric Zelikman, et al. Scaling direct preference optimization for fine-grained reward specification.arXiv preprint arXiv:2310.12036, 2023

    2023 arXiv

  24. [36]

    Discovering latent knowledge in language models without supervision.arXiv preprint arXiv:2310.02690, 2023

    Farzan Farnia, Guillaume Deletang, Victoria Krakovna, et al. Discovering latent knowledge in language models without supervision.arXiv preprint arXiv:2310.02690, 2023

    2023 arXiv

  25. [37]

    MIT Press, 2002

    Bernhard Schölkopf and Alexander J Smola.Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002

    2002

  26. [38]

    Support-vector networks

    Corinna Cortes and Vladimir Vapnik. Support-vector networks. InMachine Learning, vol- ume 20, pages 273–297. Springer, 1995

    1995

  27. [39]

    Preference learning with gaussian processes

    Wei Chu and Zoubin Ghahramani. Preference learning with gaussian processes. InProceedings of the 22nd International Conference on Machine Learning (ICML), pages 137–144, 2005

    2005

  28. [40]

    Optimizing search engines using clickthrough data

    Thorsten Joachims. Optimizing search engines using clickthrough data. InProceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 133–142, 2002. 12

    2002

  29. [41]

    Manifold regularization: A geometric framework for learning from labeled and unlabeled examples.Journal of Machine Learning Research, 7(Nov):2399–2434, 2006

    Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples.Journal of Machine Learning Research, 7(Nov):2399–2434, 2006

    2006

  30. [42]

    Representation learning: A review and new perspectives.IEEE transactions on pattern analysis and machine intelligence, 35(8):1798– 1828, 2013

    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives.IEEE transactions on pattern analysis and machine intelligence, 35(8):1798– 1828, 2013

    2013

  31. [43]

    Learning with kernels: support vector machines, regularization, optimization, and beyond.MIT Press, 2001

    Bernhard Schölkopf and Alexander Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond.MIT Press, 2001

    2001

  32. [44]

    Nonlinear component analysis as a kernel eigenvalue problem

    Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. InNeural Computation, volume 10, pages 1299–1319, 1998

    1998

  33. [45]

    Marc G. Genton. Classes of kernels for machine learning: A statistics perspective.Journal of Machine Learning Research, 2:299–312, 2001

    2001

  34. [46]

    Support vector machines with polynomial kernels

    Sariel Har-Peled, Dan Roth, and Louis Zimak. Support vector machines with polynomial kernels. InProceedings of the 14th Annual Conference on Computational Learning Theory (COLT), pages 406–421, 2002

    2002

  35. [47]

    Regularization theory and neural network architectures.Neural Networks, 6(5):613–627, 1995

    Federico Girosi. Regularization theory and neural network architectures.Neural Networks, 6(5):613–627, 1995

    1995

  36. [48]

    Smola, and Klaus-Robert Müller

    Bernhard Schölkopf, Alexander J. Smola, and Klaus-Robert Müller. Support vector kernels. Advances in Kernel Methods: Support Vector Learning, pages 109–144, 1997

    1997

  37. [49]

    Wavelet support vector machine.Expert Systems with Applications, 36(7):10170–10173, 2009

    Linlin Zhang and Shuang Wang. Wavelet support vector machine.Expert Systems with Applications, 36(7):10170–10173, 2009

    2009

  38. [50]

    Wavelet-based kernel function and its application in support vector regression.Information Sciences, 179(23):4070–4081, 2009

    Zhenwei Shi, Zhenming Wang, and Jie Yang. Wavelet-based kernel function and its application in support vector regression.Information Sciences, 179(23):4070–4081, 2009

    2009

  39. [51]

    Gonzalez, Richard E

    Rafael C. Gonzalez, Richard E. Woods, and Steven L. Eddins.Digital Image Processing using MATLAB. McGraw-Hill, 2012

    2012

  40. [52]

    On information and sufficiency.The Annals of Mathematical Statistics, 22(1):79–86, 1951

    Solomon Kullback and Richard A Leibler. On information and sufficiency.The Annals of Mathematical Statistics, 22(1):79–86, 1951

    1951

  41. [53]

    On measures of entropy and information

    Alfréd Rényi. On measures of entropy and information. InProceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 547–561, 1961

    1961

  42. [54]

    L. V . Kantorovich. On the translocation of masses.C.R. (Doklady) Acad. Sci. URSS (N.S.), 37:199–201, 1942

    1942

  43. [55]

    Springer, 2009

    Cédric Villani.Optimal Transport: Old and New. Springer, 2009

    2009

  44. [56]

    R ’enyi differential privacy of the subsampled gaussian mechanism

    Tim Van Erven, Ziyu Bu, and James Zou. R ’enyi differential privacy of the subsampled gaussian mechanism. InProceedings of the 38th International Conference on Machine Learning (ICML), pages 10615–10625, 2021

    2021

  45. [57]

    Learning with a wasserstein loss

    Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso Poggio. Learning with a wasserstein loss. InAdvances in Neural Information Processing Systems (NeurIPS), pages 2053–2061, 2015

    2015

  46. [58]

    Improved training of wasserstein gans

    Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. InAdvances in Neural Information Processing Systems (NeurIPS), pages 5767–5777, 2017

    2017

  47. [59]

    Deceptive alignment is easy in large language models.arXiv preprint arXiv:2312.06683, 2023

    Samuel Greenblatt, Shibani Santurkar, et al. Deceptive alignment is easy in large language models.arXiv preprint arXiv:2312.06683, 2023

    2023 arXiv

  48. [60]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Xiang Lisa Li, Yao Fu, Sheng Shen, Barret Zoph, Xinyun Chen, Shuran Zhang, Sen Zhao, et al. Universal and transferable adversarial attacks on aligned language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 13

    2023

  49. [61]

    Mechanistic interpretability of safety fine-tuning in llms

    Author Unknown. Mechanistic interpretability of safety fine-tuning in llms. InAdvances in Neural Information Processing Systems, 2024. NeurIPS

    2024

  50. [62]

    G-eval: Nlg evaluation using gpt-4 with better human alignment.arXiv preprint arXiv:2305.13283, 2023

    Fangzhou Liu, Shuyang Liu, Yujia Zheng, Yixin Cao, Lemao Li, Lidong Bing, Lei Li, Karan Sinha, Yizhong Wang, Chris Callison-Burch, et al. G-eval: Nlg evaluation using gpt-4 with better human alignment.arXiv preprint arXiv:2305.13283, 2023

    2023 arXiv

  51. [63]

    On the limitations of toxicity classifiers in detoxifying language models.arXiv preprint arXiv:2402.03509, 2024

    Lisa Jiang, Ethan Perez, Kevin Lee, Deep Ganguli, Jimmy Ba, Colin Raffel, and He He. On the limitations of toxicity classifiers in detoxifying language models.arXiv preprint arXiv:2402.03509, 2024

    2024 arXiv

  52. [66]

    On lines and planes of closest fit to systems of points in space.Philosophical Magazine, 2(11):559–572, 1901

    Karl Pearson. On lines and planes of closest fit to systems of points in space.Philosophical Magazine, 2(11):559–572, 1901

    1901

  53. [67]

    Visualizing data using t-sne

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. InJournal of Machine Learning Research, volume 9, pages 2579–2605, 2008

    2008

  54. [68]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  55. [69]

    Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811, 2025

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811, 2025

    2025 arXiv

  56. [70]

    Rethinking fid: Towards a better evaluation metric for image generation

    Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9307–9...

    2024

  57. [71]

    CLIPScore: a reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: a reference-free evaluation metric for image captioning. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6516–6528, 2021

    2021

  58. [72]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Christopher Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InProceedings of the International Conference...

    2021

  59. [73]

    Safree: Steering away from unsafe concepts in text-to-image and text-to- video generation

    Jae Sung Yoon et al. Safree: Steering away from unsafe concepts in text-to-image and text-to- video generation. InCVPR, 2024

    2024

  60. [74]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  61. [75]

    Adafactor: Adaptive learning rates with sublinear memory cost

    Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. InInternational Conference on Machine Learning, pages 4596–4604. PMLR, 2018

    2018

  62. [76]

    Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data.Nature Communica- tions, 12(1):4122, 2021

    Charles H Martin, Tongsu Peng, and Michael W Mahoney. Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data.Nature Communica- tions, 12(1):4122, 2021

    2021

  63. [77]

    Direct preference optimization: Your language model is secretly a reward model.arXiv preprint arXiv:2305.18290, 2023

    Ramin Rafailov, Eric Zelikman, Steven Gao, and Tatsunori B Hashimoto. Direct preference optimization: Your language model is secretly a reward model.arXiv preprint arXiv:2305.18290, 2023

    2023 arXiv

  64. [78]

    Diffusion-dpo: Enhancing preference optimization for text-to-image alignment

    Ziqi Wang, Yixuan Zhang, Xinyang Chen, Zhen Liu, Jianwei Sun, Yelong Shen, Rui Zhang, and Jie Tang. Diffusion-dpo: Enhancing preference optimization for text-to-image alignment. arXiv preprint arXiv:2311.07654, 2023. 14

    2023 arXiv

  65. [79]

    Safree: Safety alignment in text-to-image diffusion via reward-guided editing.arXiv preprint arXiv:2312.00704, 2023

    Linxi He, Yichong Gao, Ruisi Wang, Haotian Liu, Weize Zhang, Lijuan Wang, and Lei Zhou. Safree: Safety alignment in text-to-image diffusion via reward-guided editing.arXiv preprint arXiv:2312.00704, 2023

    2023 arXiv

  66. [80]

    Latentguard: Contrastive safety filtering for text-to-image models, 2024

    Anonymous. Latentguard: Contrastive safety filtering for text-to-image models, 2024. arXiv preprint arXiv:2401.XXXX

    2024

  67. [81]

    Steerdiff: Safety steering in latent space for diffusion models, 2024

    Anonymous. Steerdiff: Safety steering in latent space for diffusion models, 2024. arXiv preprint arXiv:2402.XXXX

    2024

  68. [82]

    A mathemati- cal framework for transformer circuits.Transformer Circuits Thread, 2022

    Nelson Elhage, Tom Henighan, Neel Nanda, Catherine Olsson, et al. A mathemati- cal framework for transformer circuits.Transformer Circuits Thread, 2022. https:// transformer-circuits.pub/2022/framework/index.html

    2022

  69. [83]

    Beyond accuracy: Behavioral testing of nlp models with checklist

    Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Behavioral testing of nlp models with checklist. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902–4912, 2020

    2020

  70. [84]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. InInternational conference on machine learning, pages 3519–3529. PMLR, 2019

    2019

  71. [85]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. InAdvances in Neural Information Processing Systems (NeurIPS), volume 30, 2017

    2017

  72. [86]

    A cluster separation measure.IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2):224–227, 1979

    David L Davies and Donald W Bouldin. A cluster separation measure.IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2):224–227, 1979

    1979

  73. [87]

    Well-separated clusters and optimal fuzzy partitions.Journal of Cybernetics, 4(1):95–104, 1974

    Joseph C Dunn. Well-separated clusters and optimal fuzzy partitions.Journal of Cybernetics, 4(1):95–104, 1974

    1974

  74. [88]

    Generalized diversity- based learning for multiple kernel clustering.Advances in Neural Information Processing Systems, 29, 2016

    Jaedeok Lee, Yuan Lin, Wei Chen, Jaime G Carbonell, and Eric P Xing. Generalized diversity- based learning for multiple kernel clustering.Advances in Neural Information Processing Systems, 29, 2016

    2016

  75. [89]

    Unsupervised metric learning for kernel embedded clustering

    Lifeng Jiang, Zhuowen Tu, and Alan Yuille. Unsupervised metric learning for kernel embedded clustering. InProceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2940–2947, 2011

    2011

  76. [90]

    Multiple kernel learning algorithms.Journal of Machine Learning Research, 12:2211–2268, 2011

    Mehmet Gönen and Ethem Alpaydın. Multiple kernel learning algorithms.Journal of Machine Learning Research, 12:2211–2268, 2011

    2011

  77. [91]

    Dropout: A simple way to prevent neural networks from overfitting.Journal of Machine Learning Research, 15(1):1929–1958, 2014

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting.Journal of Machine Learning Research, 15(1):1929–1958, 2014

    1929

  78. [92]

    Mixed precision training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Greg Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. InInternational Conference on Learning Representations (ICLR), 2018

    2018

  79. [93]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Informa- tion Processing Systems, volume 30, 2017

    2017

  80. [94]

    Using the nyström method to speed up kernel machines

    Christopher KI Williams and Matthias Seeger. Using the nyström method to speed up kernel machines. InAdvances in neural information processing systems, volume 13, pages 682–688, 2001

    2001

  81. [95]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. InAdvances in Neural Information Processing Systems, volume 20, pages 1177–1184, 2007

    2007

  82. [96]

    Dpo-kernels and the detonate benchmark: Geometry-aware preference optimiza- tion for safer text-to-image generation, 2025

    Anonymous. Dpo-kernels and the detonate benchmark: Geometry-aware preference optimiza- tion for safer text-to-image generation, 2025. Under review at NeurIPS 2025. 15

    2025

  83. [97]

    Kosmos-2: Grounding multimodal large language models to the world.ArXiv, abs/2306, 2023

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world.ArXiv, abs/2306, 2023

    2023

  84. [98]

    jina-clip-v2: Multilingual multimodal embeddings for text and images, 2024

    Andreas Koukounas, Georgios Mastrapas, Bo Wang, Mohammad Kalim Akram, Sedigheh Eslami, Michael Günther, Isabelle Mohr, Saba Sturua, Scott Martens, Nan Wang, and Han Xiao. jina-clip-v2: Multilingual multimodal embeddings for text and images, 2024. 16 8 Frequently Asked Question...

    2024

  85. [100]

    Log-likelihood ratio term: log π(y +|x) π(y −|x), capturing preference under the model’s genera- tive distribution

  86. [101]

    Kernelized embedding similarity:A structured preference function defined over kernel- induced similarity κ(ex,e y± ) using RBF [47], polynomial [40], or wavelet kernels [49], which encode local semantic proximity, global nonlinear interactions, or multi-scale abstraction respectively

  87. [102]

    chosen” (safe) and “rejected

    Diffusion-specific regularization:A divergence term (KL [ 52], Rényi [53], or Wasser- stein [55]) applied to the denoising error distributions of the aligned and reference models, ensuring stable preference propagation across the diffusion trajectory. This triadic formulation ...

  88. [103]

    The reference model typically represents the model’s state before preference alignment or a snapshot of a trusted, pre-aligned model

    Model Setup:Two UNet architectures are utilized: a trainable policy model, denoted asUNet( ϵθ), and a frozenreference UNet, ϵref. The reference model typically represents the model’s state before preference alignment or a snapshot of a trusted, pre-aligned model

  89. [104]

    Noisy Latent Generation:For each training instance, consisting of a preferred image y+ 0 and a rejected image y− 0 , corresponding noisy latents ( y+ t , y− t ) are generated. This is achieved by adding a noise vector ϵ∗ ∼ N(0, I)(scaled appropriately by the diffusion schedule...

  90. [105]

    Subsequently, error vectors are computed for both models and for both preferred and rejected samples

    Noise Prediction and Error Computation:Both the policy UNet ( ϵθ) and the reference UNet (ϵref) are tasked with predicting the noise added to y+ t and y− t . Subsequently, error vectors are computed for both models and for both preferred and rejected samples. The error vector ...

  91. [106]

    ) (𝒙) Diffusion ModelGradually Denoising ………𝑋#𝑋#

    Divergence-based Regularization:The standard KL regularizer from Equation C.1 is replaced by a differential divergence term computed over these denoising error distributions. Specifically, the regularization term becomes D[errθ(y+)∥errref(y+)]−D[err θ(y−)∥errref(y−)]. Here, D ...

  92. [107]

    Pooled Embedding Computation– Compute ˆa(x)using the weighted aggregation defined above

  93. [108]

    pixel_values

    Unsupervised Clustering Index Evaluation– Assess the separability of embeddings (e.g., via metrics like Dunn Index or Davies–Bouldin Index) to quantify alignment quality. We adopt DBS to penalize cluster compactness and DI to reward inter-cluster separation, balancing average ...

  94. [2025]

    arXiv preprint arXiv:2501.XXXX. 11

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.