Pith. sign in

REVIEW 5 cited by

RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.17401 v1 pith:NDLHEM6Y submitted 2024-05-27 cs.LG cs.CVstat.ML

Litu Rout , Yujia Chen , Nataniel Ruiz , Abhishek Kumar , Constantine Caramanis , Sanjay Shakkottai , Wen-Sheng Chu This is my paper
classification cs.LGcs.CVstat.ML
keywords stylecontentrb-modulationreferencetraining-freeallowscompositioncontrol
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We propose Reference-Based Modulation (RB-Modulation), a new plug-and-play solution for training-free personalization of diffusion models. Existing training-free approaches exhibit difficulties in (a) style extraction from reference images in the absence of additional style or content text descriptions, (b) unwanted content leakage from reference style images, and (c) effective composition of style and content. RB-Modulation is built on a novel stochastic optimal controller where a style descriptor encodes the desired attributes through a terminal cost. The resulting drift not only overcomes the difficulties above, but also ensures high fidelity to the reference style and adheres to the given text prompt. We also introduce a cross-attention-based feature aggregation scheme that allows RB-Modulation to decouple content and style from the reference image. With theoretical justification and empirical evidence, our framework demonstrates precise extraction and control of content and style in a training-free manner. Further, our method allows a seamless composition of content and style, which marks a departure from the dependency on external adapters or ControlNets.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Hallo4D uses vision-language models to detect and correct spatial and temporal mistakes in AI-generated 3D and 4D content, improving consistency without retraining the base generators.

  2. PoseAlign: Sculpting Pose-Consistent Meshes via Text-Guided Deformation

    cs.GR 2026-07 conditional novelty 6.0 of 10

    Two-stage text-guided mesh deformation (Laplacian CLIP scaling + attention-shared SDS Jacobian sculpting) better preserves source pose while aligning to text than TextDeformer or MeshUp.

  3. Solving Inverse Problems with Flow-based Models via Model Predictive Control

    eess.IV 2026-01 conditional novelty 6.0 of 10

    MPC-Flow applies model predictive control to guide pretrained flow models through inverse problems, with a single-step variant that avoids backpropagation and scales to 32B-parameter models on consumer hardware.

  4. MARBLE: Material Recomposition and Blending in CLIP-Space

    cs.CV 2025-06 conditional novelty 6.0 of 10

    MARBLE performs material blending and parametric material-attribute control by manipulating CLIP image embeddings and injecting them into a specific U-Net block of a pre-trained diffusion model.

  5. Diffusion Bridge or Flow Matching? A Unifying Framework and Comparative Analysis

    cs.CV 2025-09 reject novelty 4.0 of 10

    A theoretical and empirical comparison claiming diffusion bridges have lower stochastic-optimal-control cost and greater robustness than flow matching when training data are scarce.

Pith tools