Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Text-to-image models can be taught to obey camera settings such as focal length and shutter speed while keeping the scene fixed, the paper claims.

desk verdict Clever framework for camera-intrinsic control in T2I, but the headline 'accuracy' is measured against the same simulators that generated the training data, so the physical-fidelity claim needs real-photo validation. read the letter →

arxiv 2412.02168 v3 pith:SLLPGQJP submitted 2024-12-03 cs.CV

Yu Yuan , Xijun Wang , Yichen Sheng , Prateek Chennuri , Xingguang Zhang , Stanley Chan This is my paper · ORCID
classification cs.CV
keywords generativephotographycameraintrinsicstext-to-imagegenerationsceneconsistencydimensionalityliftingdifferentialencoderdiffusionmodelscontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that text-to-image diffusion models can be made to respect physical camera intrinsics—focal length, shutter speed, bokeh blur, and color temperature—while leaving the scene itself unchanged. It argues that current state-of-the-art generators such as Stable Diffusion 3 and FLUX fail at this because they entangle camera information with scene content, so changing a lens or shutter prompt rewrites the scene. The proposed fix lifts the problem from text-to-image to text-to-video generation: the scene description builds the base content and each camera setting conditions a separate frame, with a differential camera encoder injecting both a coarse physical embedding and inter-setting differences. Trained on a small physically simulated dataset, the framework reports high trend accuracy and perceptual consistency across settings. If the claim holds, professional photography workflows gain precise camera control inside generative tools, and data-driven models gain a route toward real optical and sensor physics.

What carries the argument

Two mechanisms carry the argument. Dimensionality Lifting converts camera-controlled text-to-image generation into text-to-video generation: an invariant scene prompt establishes the base scene, and a list of camera settings conditions successive frames, exploiting video models' temporal attention for cross-frame consistency. Differential Camera Intrinsics Learning supplies the camera awareness: on the data side, a differential dataset is generated by physically based simulation—center-crop and resize for focal length, a bokeh renderer for aperture, a CMOS sensor model with Poisson-Gaussian noise for shutter speed, and a blackbody-radiation approximation for color temperature—using randomly sampled continuous settings; on the network side, a differential camera encoder concatenates a coarse, physics-derived embedding of each setting with feature differences from a frozen CLIP text encoder. The combined signal is injected into the temporal attention layers of the video model through a lightweight adapter encoder.

What would settle it

Concrete test: photograph a fixed scene with a real camera at several focal lengths (24, 35, 50, 70 mm) on a tripod, and separately at several shutter speeds, apertures, and color temperatures; then generate images from the same scene description with the same settings and compare the trend of blur, zoom, brightness, and color against the real photographs instead of simulated frames. The focal-length case is the cleanest because it can be checked directly against optical zoom and does not require specialized capture hardware beyond one zoom lens.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that camera intrinsic control can be added to a pre-trained text-to-video diffusion model without sacrificing scene identity or image quality. The method treats a sequence of frames of the same scene under different camera settings as a video, so the temporal attention of the video model does the work of keeping the scene stable while each frame responds to its own setting. A differential camera encoder then supplies two signals per frame: a coarse embedding constructed from a simplified physical model of the setting, and the difference between CLIP text features of adjacent settings, which teaches the network the meaning of changes rather than absolute values. With about 1,000 base images per task, the model reports Pearson correlations above 0.86 for bokeh, focal length, shutter speed, and color temperature trends, compared with below 0.30 for Stable Diffusion 3 and FLUX, and produces bokeh that appears depth-aware even though no depth map is given at inference.

Load-bearing premise

The quantitative results treat the physical-simulation pipeline used to build the training data as ground truth: if the bokeh renderer, center-crop focal-length model, simplified CMOS equation, and color-temperature approximation do not faithfully reproduce real camera behavior, then the reported accuracy measures agreement with the simulator, not with real optics and sensors.

Editorial extensions

If this is right

  • A user can request any value on a continuous camera-setting scale, not just preset modes, and get a photorealistic frame that reflects that value.
  • Changing the camera setting changes the photographic effect while the identity of the scene, its objects, and their layout remain stable.
  • A few thousand physically simulated image sets can teach camera physics to a pre-trained video diffusion model, reducing the need for expensive paired real captures.
  • The same conditioning path can be run with a single setting by repeating it across frames, so the method doubles as an ordinary camera-aware text-to-image generator.
  • Prompt following is largely preserved, so adding camera control does not trade away the base model's ability to follow the text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension left implicit by the paper: swap the coarse embedding and simulation module to cover other camera parameters, such as lens distortion, ISO noise, or white-balance tint.
  • A neighbouring application of the same lifting idea: any per-frame condition that can be separated from an invariant scene prompt, such as lighting direction, weather, or time of day, could get the same scene-consistent control.
  • A caveat in my reading: because the reference videos come from the same simulators used for training, the accuracy numbers are not yet evidence of fidelity to real camera physics; real-photo benchmarks would settle that.
  • The depth-free bokeh behavior suggests the model may have internalized depth cues from simulation; testing on scenes with unusual foreground-background structure would show whether that generalizes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Generative Photography, a framework for controlling camera intrinsic settings (focal length, shutter speed, bokeh, and color temperature) in text-to-image generation. The method rests on two ideas: dimensionality lifting, which reformulates multi-setting image generation as text-to-video generation so that the scene description is separate from per-frame camera settings, and differential camera intrinsics learning, which combines a physically simulated training set of image pairs with a differential camera encoder that takes coarse embeddings and inter-frame difference features. The authors report quantitative improvements over Stable Diffusion 3, FLUX, AnimateDiff, and CameraCtrl in terms of a Pearson correlation accuracy metric against simulated reference frames, LPIPS consistency, and CLIP prompt following, and they provide ablations of the differential encoder, sampling strategy, frame count, and dataset scale.

Significance. The problem is well motivated: existing text-to-image models do not respect camera intrinsics, and a framework that provided precise, scene-consistent control over focal length, shutter speed, bokeh, and color temperature would be practically valuable. The dimensionality-lifting idea is creative and is supported by clear architectural and ablative evidence that the proposed components help. The paper also ships code and detailed visual comparisons. However, the central quantitative claim of 'physical accuracy' currently rests on a metric that compares against the same simulator pipeline used to generate the training data, so the headline result is not yet grounded in real camera physics. If the authors add external validation against real captured image sets and strengthen the statistical reporting, the work would be a solid contribution.

major comments (3)
  1. [§9.1 Accuracy, §3.2.1 Differential Data] The central accuracy metric is circular with respect to the training data. In §3.2.1, training frames are produced by physically based simulation: BokehMe for bokeh, center-crop/resize for focal length, the simplified CMOS model of Eq. (4) for shutter speed, and the Fairchild empirical formulas for color temperature. In §9.1, the reference frames for the accuracy metric are generated with the same pipeline. A high CorrCoef therefore primarily measures how well the model imitates its own training-data generator, not how faithfully it reproduces real camera optics and sensor behavior. The only real-photo validation is the qualitative focal-length comparison in Fig. 8; the other three controls have no real-image grounding. The authors should either validate each simulator against real captured multi-setting image sets (e.g., using a camera with controlled settings on the same scenes) or, at minimum, explicitly temper the claims of physical accuracy to simulator fidelity.
  2. [Tables 1 and 2, §4.2 Comparisons] No error bars, confidence intervals, or statistical significance tests are reported for any of the CorrCoef, LPIPS, or CLIP values. The paper claims 'significant advantage' over baselines, but with 75 test settings and no variance estimates, the reader cannot judge whether differences such as 0.8626 versus 0.6025 (Bokeh, Table 1) are statistically meaningful. The authors should report means over independent sampling runs, standard deviations, and an appropriate significance test, or at least provide per-setting distributions.
  3. [§9.1 Accuracy, §9.2 Consistency] The trend-based CorrCoef is a weak proxy for the claimed accuracy. For bokeh, the metric is the average Laplacian blur level; for shutter speed, it is average brightness; for color temperature, it is average color. A model that applies a global Gaussian blur, a global brightness scaling, or a global color tint would achieve high correlation without any scene-consistent camera physics, and such degenerate behavior is not penalized by the consistency metric because LPIPS is also computed globally. The paper should include a more direct evaluation that checks, for example, depth-aware bokeh structure, content preservation under focal-length changes, and per-region color or brightness changes consistent with the scene.
minor comments (5)
  1. [Table 1] The column header 'Shuttle Speed' should be 'Shutter Speed'.
  2. [§4.1 Implementation Details] The training configuration states '25,000 epochs' with a batch size of 8 on 1,000 images, which is ambiguous; if this means 25,000 optimization steps, the term 'epochs' should be corrected.
  3. [§4.3 Ablation study] The phrase 'incorporating different information enhances the model’s accuracy' should read 'differential information' for clarity.
  4. [Table 1] The '—-' entries for SD3 and FLUX CLIP scores should be explained; presumably the prompts used for these models include the camera setting tokens, but this should be stated explicitly.
  5. [§7.3 Differential Data for Shutter Speed, Eq. (4)] Eq. (4) uses many symbols (ξ, QE, μ_dark, σ_read) that are defined only in the surrounding text; a table of definitions or a brief statement of the assumed values would improve reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

Quantitative 'accuracy' is measured against reference frames produced by the same physical simulators used to build the training data, so the headline result evidences simulator mimicry rather than real-camera fidelity.

  1. fitted input called prediction [Section 9.1 (Accuracy) vs. Section 3.2.1 (Physically-based Simulation) and Table 1 caption]
    "To evaluate the accuracy of the camera physics in generated images, we first simulate the reference frames of the base image under multiple camera settings, using the same scene description and corresponding camera parameters for generation. We then calculate the overall trend of camera effects within the reference frames and the overall trend of camera effects within the generated multi-frame sequence. The Pearson correlation coefficient between these two trends is computed as an accuracy metric (CorrCoef)."

    The reference frames in this metric are produced by the same Physically-based Simulation pipeline used in Section 3.2.1 to construct the differential training data: BokehMe for bokeh, FoV center-crop/resize for focal length, Eq. 4 CMOS/ISP model for shutter speed, and the Fairchild empirical RGB mapping for color temperature. Table 1's caption states: Reference refers to the results obtained from physical simulations. The CorrCoef therefore measures how well the model reproduces its own training-data generator, not how well it matches real optics and sensors. The only real-photo validation is the qualitative focal-length comparison in Fig. 8, which is not quantified and does not cover the other three controls.

  2. self definitional [Section 8.1 (Coarse Embedding) vs. Section 7.4 and Eq. 4 (simulation formulas used as ground truth)]
    "For color temperature, we estimate the ratio coefficients for the RGB channels based on the color temperature value, using a simplified version of the corresponding formula from Equation 5 to Equation 7. These coefficients are then used as the scaling factors for the coarse embedding."

    The coarse embedding is the network's direct conditioning signal for color temperature and is computed from the same empirical Kelvin-to-RGB equations (Eqs. 5-7) used to render the reference frames and training labels in Section 7.4. Similarly, the shutter-speed coarse embedding uses the target/base shutter-time ratio, matching the exposure-time scaling in Eq. 4 that defines the simulated ground truth. The model is therefore handed a hint derived from the exact transformation it is later scored against, so the color-temperature and shutter-speed CorrCoef values are partly a readout of injected coefficients rather than an independent verification of learned camera physics.

full rationale

The paper's strongest quantitative claim--accurate and consistent camera-setting control--rests on Section 9.1's Accuracy metric, which compares generated frames to reference frames synthesized by the same physical-simulation pipeline (Section 3.2.1) that generated the training data. Because both training labels and evaluation references come from BokehMe, FoV center-crop/resize, Eq. 4, and Fairchild's RGB approximation, a high CorrCoef primarily demonstrates fidelity to these simulators. Section 8.1 further injects simplified versions of the same formulas as coarse embeddings, making part of the control signal definitionally tied to the evaluation target. The paper is not wholly circular: it provides qualitative real-focal-length comparison in Fig. 8, reports scene-consistency and prompt-following comparisons against external baselines (SD3, FLUX), and the generative framework has independent engineering content. The self-citations for Eq. 4 ([12], [48]) are not load-bearing because Eq. 4 is a standard CMOS imaging model also citing external work [37], and no uniqueness-theorem self-citation chain appears. However, the headline claim of 'physical accuracy' is not established by the self-referential Accuracy metric; external validation on real captured image sets would be needed. Score 6: partial circularity, with the central quantitative 'prediction' reducing substantially to simulator mimicry by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on four domain assumptions: the physical simulators are faithful, video temporal attention yields scene consistency, CLIP text embeddings represent camera settings, and VLM captions are accurate. It uses no fitted scalars beyond the hand-designed coarse-embedding priors and sampling ranges. No new physical entities are introduced.

free parameters (3)
  • Base exposure time for shutter-speed coarse embedding = 0.2 s (average)
    Hand-chosen in Sec. 8.1 to convert target shutter speed to a brightness ratio; the shutter-speed control result depends on this assumed base.
  • Camera setting sampling ranges = focal 24-70 mm, blur 1-30, shutter 0.1-1.0 normalized, temperature 2000-10000 K
    Fixed by hand in Sec. 3.2.1; the model is only evaluated within these ranges.
  • Coarse embedding design (bokeh Gaussian proxy, FoV mask, brightness ratio, RGB ratios) = hand-crafted per task
    Sec. 8.1 injects a simplified physical prior into the network; this hand-designed signal carries much of the camera-effect information.
assumptions (4)
  • domain assumption The physics-based simulation modules (BokehMe, FoV center-crop, simplified CMOS imaging model in Eq. 4, Fairchild color-temperature approximation) faithfully reproduce real camera physics.
    Invoked in Sec. 3.2.1 to create training data and in Sec. 9.1 as the reference ground truth for the accuracy metric.
  • domain assumption Temporal attention in a pre-trained text-to-video model provides sufficient consistency across frames with different camera settings.
    Motivates dimensionality lifting in Sec. 3.1.
  • domain assumption CLIP text embeddings of camera-setting strings (e.g., '24mm lens', '3000K') carry usable semantic differences.
    Differential features in the encoder (Sec. 3.2.2) are computed as differences between these embeddings.
  • domain assumption LLaVA-generated captions accurately describe the base scenes.
    Used in the data pipeline (Sec. 3.2.1, step i) as invariant scene descriptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis." pith.science (2026). https://pith.science/paper/SLLPGQJP

@misc{pith2026241202168,
  author       = {Pith},
  title        = {Pith review of: Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SLLPGQJP}},
  note         = {Machine review of arXiv:2412.02168}
}
read the original abstract

Image generation today can produce somewhat realistic images from text prompts. However, if one asks the generator to synthesize a specific camera setting such as creating different fields of view using a 24mm lens versus a 70mm lens, the generator will not be able to interpret and generate scene-consistent images. This limitation not only hinders the adoption of generative tools in professional photography but also highlights the broader challenge of aligning data-driven models with real-world physical settings. In this paper, we introduce Generative Photography, a framework that allows controlling camera intrinsic settings during content generation. The core innovation of this work are the concepts of Dimensionality Lifting and Differential Camera Intrinsics Learning, enabling smooth and consistent transitions across different camera settings. Experimental results show that our method produces significantly more scene-consistent photorealistic images than state-of-the-art models such as Stable Diffusion 3 and FLUX. Our code and additional results are available at https://generative-photography.github.io/project.

Figures

Figures reproduced from arXiv: 2412.02168 by the authors.

Figure 1
Figure 1. This paper introduces generative photography, a new paradigm for text-to-image generation that maintains a consistent base scene while modifying only the camera settings to achieve varied photographic effects. Current state-of-the-art text-to-image generation models like Stable Diffusion 3 (SD3) [4] and FLUX [1] face two major limitations: failure to accurately interpret camera-specific settings and difficulties in … view at source ↗
Figure 2
Figure 2. (a) Existing text-to-image (T2I) models struggle to perceive physical camera settings and maintain consistency across multiple settings, even when the random seed is fixed. (b) We solve this problem by lifting camera-controlled text-to-image (T2I) generation into text-to-video (T2V) generation, thereby decoupling scene description from camera settings and achieving better scene consistency. paradigm. As shown in [P… view at source ↗
Figure 3
Figure 3. The pipeline of building differential data. (i). Vision Language Model Captioning for Base Im￾ages: To ensure that the data we generate can be used for training, we first need to collect base images and obtain their textual labels. The base images we select must be of high quality and meet specific requirements for camera generation tasks. For example, for shutter speed control, the base im￾ages should have a wide d… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: The overall architecture of differential camera encoder. levels by adjusting the captured image irradiance and con￾verting it back to RGB images using a simulated image signal processor (ISP) [37]. • Color temperature: We adjust the RGB channel ratios according to the …
Figure 5
Figure 5. Figure 5: Visual comparisons between different generative methods. Our method is capable of generating realistic camera effects for any given camera setting scale, while maintaining high scene consistency across images corresponding to different scales. Both AnimateDiff [21] and…
Figure 6
Figure 6. Figure 6: Ablation study on dataset scaling characteristic. 5. Conclusions Generative photography opens the door to a new paradigm in photography where the generated contents are consistent with the intrinsic camera physics. Our solution is based on two concepts: dimensionality …
Figure 7
Figure 7. Figure 7: The first row shows examples of base images used for constructing bokeh rendering data, featuring prominent foregrounds and distinguishable backgrounds. The second row presents depth maps extracted using the Depth Anything [71, 72] model. Horizontal FoV: \text {FoV}_\t…
Figure 8
Figure 8. Figure 8: The comparison between the reference real focal lengths and our simulated results. Note that the real-world shooting data is derived from [76], and there may be slight misalignment between images of different resolutions due to shooting conditions. We observe that exce…
Figure 9
Figure 9. Figure 9: We use a mask as the coarse embedding for focal length control. The black areas represent pixels around the edges of the frame that should not be displayed at the given focal length. For shutter speed, we roughly estimate the ratio between the target shutter time and t…
Figure 10
Figure 10. Figure 10: Visual comparisons between different generative methods on camera bokeh rendering control. Both AnimateDiff [21] and CameraCtrl [23] have been fine-tuned/trained on our data [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Visual comparisons between different generative methods on camera focal length control. Both AnimateDiff [21] and CameraCtrl [23] have been fine-tuned/trained on our data [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Visual comparisons between different generative methods on camera shutter speed control. Both AnimateDiff [21] and CameraCtrl [23] have been fine-tuned/trained on our data [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Visual comparisons between different generative methods on camera color temperature control. Both AnimateDiff [21] and CameraCtrl [23] have been fine-tuned/trained on our data [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Wonderland: Navigating 3D Scenes from a Single Image

    cs.CV 2024-12 conditional novelty 7.0 of 10

    A feed-forward pipeline reconstructs 3D Gaussian scenes from single images by regressing 3DGS directly from camera-conditioned video diffusion latents.

Reference graph

Works this paper leans on

82 extracted references · 57 canonical work pages · cited by 1 Pith paper

  1. [17]

    Camera settings as tokens: Modeling photography on latent diffusion models

    I-Sheng Fang, Yue-Hua Han, and Jun-Cheng Chen. Camera settings as tokens: Modeling photography on latent diffusion models. In SIGGRAPH Asia, 2024. 3

    work page 2024

  2. [1]

    https://github.com/black- forest- labs/ flux

    FLUX. https://github.com/black- forest- labs/ flux. 1, 2, 6, 8

  3. [2]

    https://docs.opencv.org/4.x/ dc/dc3/tutorial_py_matcher.html

    Brute-Force Matcher. https://docs.opencv.org/4.x/ dc/dc3/tutorial_py_matcher.html. 3

  4. [3]

    https://help.runwayml.com/hc/en- us/articles/ 34926468947347- Creating- with- Camera- Control- on-Gen-3-Alpha-Turbo

    Creating with camera control on gen-3 alpha turbo. https://help.runwayml.com/hc/en- us/articles/ 34926468947347- Creating- with- Camera- Control- on-Gen-3-Alpha-Turbo . 3

    work page

  5. [4]

    https://github.com/Stability-AI/ StableDiffusion

    Stable Diffusion. https://github.com/Stability-AI/ StableDiffusion. 1, 2, 3, 6, 8

    work page

  6. [5]

    https:// openai.com/index/video- generation- models- as- world-simulators/

    Video generation models as world simulators. https:// openai.com/index/video- generation- models- as- world-simulators/. 2

    work page

  7. [6]

    Lindell, and Sergey Tulyakov

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siaro- hin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, David B. Lindell, and Sergey Tulyakov. VD3D: Taming large video diffusion transformers for 3d camera con- trol. arXiv preprint arXiv:2407.12781, 2024. 3

    arXiv 2024

  8. [7]

    Stable video diffusion: Scaling la- tent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable video diffusion: Scaling la- tent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 4

    arXiv 2023

Show all 82 references
  1. [8]

    Any-resolution training for high-resolution image synthesis

    Lucy Chai, Michael Gharbi, Eli Shechtman, Phillip Isola, and Richard Zhang. Any-resolution training for high-resolution image synthesis. In European Conference on Computer Vi- sion, 2022. 5

    2022

  2. [9]

    Stanley H. Chan. Tutorial on diffusion models for imaging and vision. arXiv preprint arXiv:2403.18103, 2024. 2

    2024 arXiv

  3. [10]

    Image neural field diffusion models

    Yinbo Chen, Oliver Wang, Richard Zhang, Eli Shechtman, Xiaolong Wang, and Michael Gharbi. Image neural field diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8007–8017, 2024. 5

    2024

  4. [11]

    Boosting camera mo- tion control for video diffusion transformers

    Soon Yau Cheong, Duygu Ceylan, Armin Mustafa, Andrew Gilbert, and Chun-Hao Paul Huang. Boosting camera mo- tion control for video diffusion transformers. arXiv preprint arXiv:2410.10802, 2024. 3

    2024 arXiv

  5. [12]

    Yiheng Chi, Xingguang Zhang, and Stanley H. Chan. HDR imaging with spatially varying signal-to-noise ratios. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5724–5734, 2023. 1

    2023

  6. [13]

    Robin Courant, Nicolas Dufour, Xi Wang, Marc Christie, and Vicky Kalogeiton. E.T. the exceptional trajectories: Text-to- camera-trajectory generation with character awareness.arXiv preprint arXiv:2407.01516, 2024. 3

    2024 arXiv

  7. [14]

    Fairchild

    Mark D. Fairchild. Color Appearance Models. 2013. 5, 2

    2013

  8. [15]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Infor- mation Processing Systems, 2021. 1, 2

    2021

  9. [16]

    Problems of dataset creation for light source estimation.arXiv preprint arXiv:2006.02692, 2020

    Egor Ershov, Alexander Belokopytov, and Alex Savchik. Problems of dataset creation for light source estimation.arXiv preprint arXiv:2006.02692, 2020. 6

    2006 arXiv

  10. [18]

    Bermano, Gal Chechik, and Daniel Cohen-Or

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image gener- ation using textual inversion, 2022. 2

    2022

  11. [19]

    Concept Sliders: Lora adap- tors for precise control in diffusion models

    Rohit Gandikota, Joanna Materzy ´nska, Tingrui Zhou, Anto- nio Torralba, and David Bau. Concept Sliders: Lora adap- tors for precise control in diffusion models. arXiv preprint arXiv:2311.12092, 2023. 3

    2023 arXiv

  12. [20]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, 2014. 3

    2014

  13. [21]

    AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yao- hui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning. International Con- ference on Learning Representations , 2024. 2, ...

    2024

  14. [22]

    Hasinoff, Dillon Sharlet, Ryan Geiss, Andrew Adams, Jonathan T

    Samuel W. Hasinoff, Dillon Sharlet, Ryan Geiss, Andrew Adams, Jonathan T. Barron, Florian Kainz, Jiawen Chen, and Marc Levoy. Burst photography for high dynamic range and low-light imaging on mobile cameras.ACM Transactions on Graphics, 35(6), 2016. 6

    2016

  15. [23]

    CameraCtrl: Enabling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. CameraCtrl: Enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101, 2024. 3, 5, 6, 7, 8

    2024 arXiv

  16. [24]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. InAdvances in Neural Information Processing Systems, page 6840–6851, 2020. 1, 2

    2020

  17. [25]

    Video diffu- sion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffu- sion models. In Advances in Neural Information Processing Systems, pages 8633–8646, 2022. 4

    2022

  18. [26]

    Training-free camera control for video generation

    Chen Hou, Guoqiang Wei, Yan Zeng, and Zhibo Chen. Training-free camera control for video generation. arXiv preprint arXiv:2406.10126, 2024. 3

    2024 arXiv

  19. [27]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models.arXiv preprint arXiv: 2106.09685, 2021. 6

    2021 arXiv

  20. [28]

    Cinematographic camera diffusion model

    Hongda Jiang, Xi Wang, Marc Christie, Libin Liu, and Bao- quan Chen. Cinematographic camera diffusion model. Com- puter Graphics Forum, 43(2), 2024. 3

    2024

  21. [29]

    How far is video generation from world model: A physical law perspective

    Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: A physical law perspective. arXiv preprint arXiv: 2411.02385, 2024. 2

    2024 arXiv

  22. [30]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv: 2001.08361 ,

    2001 arXiv

  23. [31]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 3

    2019

  24. [32]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Hui- wen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007–6017, 2023. 3

    2023

  25. [33]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. An introduction to variational autoencoders. arXiv preprint arXiv:1906.02691,

    1906 arXiv

  26. [34]

    Ross, Bryan Seybold, and Lu Jiang

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos´e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birod- kar, Jimmy Yan, Ming-Chang Chiu, Krishna Somandepalli, Hassan Akbari, Yair Alon, Yong Cheng, Josh Dillon, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alons...

    2023 arXiv

  27. [35]

    Wetzstein

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hong- sheng Li, Leonidas Guibas, and Gordon. Wetzstein. Collab- orative video diffusion: Consistent multi-video generation with camera control. In arXiv preprint arXiv:2405.17414 ,

    arXiv

  28. [36]

    GLIGEN: Open-set grounded text-to-image genera- tion

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. GLIGEN: Open-set grounded text-to-image genera- tion. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 3

    2023

  29. [37]

    Salman Asif, and Zhan Ma

    Zhihao Li, Ming Lu, Xu Zhang, Xin Feng, M. Salman Asif, and Zhan Ma. Efficient visual computing with camera raw snapshots. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 46(7):4684–4701, 2024. 5, 1

    2024

  30. [38]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 5

    2023

  31. [39]

    David G. Lowe. Distinctive image features from scale- invariant keypoints. International Journal of Computer Vi- sion, 60(2):91–110, 2004. 3

    2004

  32. [40]

    Understanding diffusion models: A unified per- spective

    Calvin Luo. Understanding diffusion models: A unified per- spective. arXiv preprint arXiv:2208.11970, 2022. 2

    2022 arXiv

  33. [41]

    S.G. Mallat. A theory for multiresolution signal decompo- sition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 11(7):674–693,

  34. [42]

    CamViG: Camera aware image-to-video generation with multimodal transform- ers

    Andrew Marmon, Grant Schindler, Jos ´e Lezama, Dan Kon- dratyuk, Bryan Seybold, and Irfan Essa. CamViG: Camera aware image-to-video generation with multimodal transform- ers. arXiv preprint arXiv:2405.13195, 2024. 3

    2024 arXiv

  35. [43]

    T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08453, 2023. 3, 5

    2023 arXiv

  36. [44]

    GLIDE: Towards photorealistic image gen- eration and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image gen- eration and editing with text-guided diffusion models. In In- ternational Conference on Machine Learning,...

    2022

  37. [45]

    Neural camera simulators

    Hao Ouyang, Zifan Shi, Chenyang Lei, Ka Lung Law, and Qifeng Chen. Neural camera simulators. In IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2021. 6

    2021

  38. [46]

    BokehMe: When neural rendering meets classical rendering

    Juewen Peng, Zhiguo Cao, Xianrui Luo, Hao Lu, Ke Xian, and Jianming Zhang. BokehMe: When neural rendering meets classical rendering. In IEEE/CVF International Con- ference on Computer Vision and Pattern Recognition, 2022. 5, 1

    2022

  39. [47]

    MPIB: An mpi-based bokeh rendering framework for realistic partial occlusion effects

    Juewen Peng, Jianming Zhang, Xianrui Luo, Hao Lu, Ke Xian, and Zhiguo Cao. MPIB: An mpi-based bokeh rendering framework for realistic partial occlusion effects. InEuropean Conference on Computer Vision, 2022. 5

    2022

  40. [48]

    Xiangyu Qu, Yiheng Chi, and Stanley H. Chan. Spatially varying exposure with 2-by-2 multiplexing: Optimality and universality. IEEE Transactions on Computational Imaging, 10:261–276, 2024. 1

    2024

  41. [49]

    Learning transferable vi- sual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable vi- sual models from natural language supervision. CoRR, abs/2103.000...

    2021 arXiv

  42. [50]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Confer- ence on Machine Learning, 2021. 2

    2021

  43. [51]

    S.F. Ray. Applied Photographic Optics: Lenses and Optical Systems for Photography, Film, Video, Electronic and Digital Imaging. 2002. 5

    2002

  44. [52]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 10674–10685, 2022. 1, 2

    2022

  45. [53]

    DreamBooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500–22510, 2023. 2, 3

    2023

  46. [54]

    Sara Mahdavi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Lit, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion ...

    2022

  47. [55]

    Closed-Form factorization of latent semantics in gans

    Yujun Shen and Bolei Zhou. Closed-Form factorization of latent semantics in gans. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1532–1540,

  48. [56]

    In- terpreting the latent space of gans for semantic face editing

    Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. In- terpreting the latent space of gans for semantic face editing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

    2020

  49. [57]

    InterFaceGAN: Interpreting the disentangled face represen- tation learned by gans.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

    Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. InterFaceGAN: Interpreting the disentangled face represen- tation learned by gans.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 3

    2020

  50. [58]

    Yichen Sheng, Zixun Yu, Lu Ling, Zhiwen Cao, Xuaner Zhang, Xin Lu, Ke Xian, Haiting Lin, and Bedrich Benes. Dr. Bokeh: Differentiable occlusion-aware bokeh rendering. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4515–4525, 2024. 5

    2024

  51. [59]

    MVDream: Multi-view diffusion for 3d generation

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308.16512, 2023. 4

    2023 arXiv

  52. [60]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations ,

  53. [61]

    DimensionX: Create any 3d and 4d scenes from a single image with controllable video diffusion, 2024

    Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen, Yueqi Duan, Jun Zhang, and Yikai Wang. DimensionX: Create any 3d and 4d scenes from a single image with controllable video diffusion, 2024. 3

    2024

  54. [62]

    Designing an encoder for StyleGAN image manipulation

    Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, and Daniel Cohen-Or. Designing an encoder for StyleGAN image manipulation. ACM Transactions on Graphics, 40(4), 2021. 3

    2021

  55. [63]

    Mo- tionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tian- shui Chen, Menghan Xia, Ping Luo, and Ying Shan. Mo- tionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers , pages 1–11, 2024. 3

    2024

  56. [64]

    Andrew P. Witkin. Scale-space filtering. In Readings in Computer Vision, pages 329–332. 1987. 5, 1

    1987

  57. [65]

    Learning images across scales using adversarial training

    Krzysztof Wolski, Adarsh Djeacoumar, Alireza Javanmardi, Hans-Peter Seidel, Christian Theobalt, Guillaume Cordon- nier, Karol Myszkowski, George Drettakis, Xingang Pan, and Thomas Leimk¨ uhler. Learning images across scales using adversarial training. ACM Transactions on Graph...

  58. [66]

    Contrastive prompts improve disentanglement in text-to-image diffusion models

    Chen Wu and Fernando De la Torre. Contrastive prompts improve disentanglement in text-to-image diffusion models. arXiv preprint arXiv:2402.13490, 2024. 3

    2024 arXiv

  59. [67]

    Uncovering the disentanglement capability in text-to-image diffusion models

    Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung Bui, Tong Yu, Zhe Lin, Yang Zhang, and Shiyu Chang. Uncovering the disentanglement capability in text-to-image diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1900–1910, 2023. 3

    1900

  60. [68]

    SV4D: Dynamic 3d content generation with multi-frame and multi-view consistency.arXiv preprint arXiv:2407.17470, 2024

    Yiming Xie, Chun-Han Yao, Vikram Voleti, Huaizu Jiang, and Varun Jampani. SV4D: Dynamic 3d content generation with multi-frame and multi-view consistency.arXiv preprint arXiv:2407.17470, 2024. 4

    2024 arXiv

  61. [69]

    Cavia: Camera-controllable multi-view video diffusion with view-integrated attention

    Dejia Xu, Yifan Jiang, Chen Huang, Liangchen Song, Thorsten Gernoth, Liangliang Cao, Zhangyang Wang, and Hao Tang. Cavia: Camera-controllable multi-view video diffusion with view-integrated attention. arXiv preprint arXiv:2410.10774, 2024. 3

    arXiv

  62. [70]

    CamCo: Camera- controllable 3d-consistent image-to-video generation

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. CamCo: Camera- controllable 3d-consistent image-to-video generation. arXiv preprint arXiv:2406.02509, 2024. 3

    2024 arXiv

  63. [71]

    Depth Anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Anything: Unleashing the power of large-scale unlabeled data. In IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2024. 5, 1

    2024

  64. [72]

    Depth Anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Anything v2. arXiv preprint arXiv: 2406.09414, 2024. 5, 1

    2024 arXiv

  65. [73]

    Direct-a-Video: Customized video generation with user- directed camera movement and object motion

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Zhang Di, Xiaodong Chen, and Jing Liao. Direct-a-Video: Customized video generation with user- directed camera movement and object motion. In ACM SIG- GRAPH 2024 Conference Papers, page 12, 2024. 2

    2024

  66. [74]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 2, 3, 6

    2023

  67. [75]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. 6, 3

    2018

  68. [76]

    Zoom to learn, learn to zoom

    Xuaner Zhang, Qifeng Chen, Ren Ng, and Vladlen Koltun. Zoom to learn, learn to zoom. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 6, 1, 2

    2019

  69. [77]

    Synthetic defocus and look-ahead autofocus for casual videography

    Xuaner Zhang, Kevin Matzen, Vivien Nguyen, Dillon Yao, You Zhang, and Ren Ng. Synthetic defocus and look-ahead autofocus for casual videography. ACM Transactions on Graphics, 2019. 6 Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis ...

    2019

  70. [78]

    Introduction This supplementary material provides additional discussions and details on the construction of differential data (Section 7), network design (Section 8), evaluation metrics (Section 9), and more visual results (Section 10). To better illustrate the continuity and ...

  71. [79]

    Camera settings are sampled during training and simulated on-the-fly using physical principles, producing differential multi-frame data without pre-storing large video files

    More Details of Building Differential Data Our differential data pipeline dynamically generates training data by storing only base images and scene descriptions. Camera settings are sampled during training and simulated on-the-fly using physical principles, producing different...

    1933

  72. [80]

    We extract the camera settings for𝐹𝑟 frames using the CLIP text encoder, compute the differences, and then reshape the result into an embedding of size𝐹𝑟×𝐶×𝐻×𝑊

    More Details of Differential Camera Encoder In the Differential Camera Encoder, an important aspect is the incorporation of the differences in camera setting scales. We extract the camera settings for𝐹𝑟 frames using the CLIP text encoder, compute the differences, and then resh...

  73. [81]

    More Details of Proposed Metrics 9.1. Accuracy To evaluate the accuracy of the camera physics in generated images, we first simulate the reference frames of the base image under multiple camera settings, using the same scene description and corresponding camera parameters for ...

  74. [82]

    More Visual Results In this section, we provide additional visual results and com- parisons with other methods. Fig. 10 to Fig. 13 illustrate the visual comparisons for bokeh rendering, focal length, shutter speed, and color temperature across various generative methods. Our a...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.