Pith. sign in

REVIEW 2 major objections 7 minor 72 references

Existing VLMs systematically confuse camera translation with rotation, left with right, and object motion with camera motion; a 17-class taxonomy, atomic real-plus-synthetic benchmark, and augmented training let an 8B model beat Gemini 3.1

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 05:14 UTC pith:TV32J3E4

load-bearing objection Clean empirical paper that diagnoses VLM camera-motion failures, ships a usable atomic benchmark, and shows modest SFT beats Gemini-3.1-Pro by ~10 points while still trailing humans badly. the 2 major comments →

arxiv 2607.03043 v1 pith:TV32J3E4 submitted 2026-07-03 cs.CV

Natural Language Camera Movement Understanding

Yuwen Tan , Joey Huang , Jin Huang , Haoxiang Li , Boqing Gong This is my paper
classification cs.CV
keywords camera movement understandingvision-language modelscinematography taxonomyatomic camera motionvideo understandingACaM benchmarksupervised fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that mapping video to natural-language cinematographic descriptions is now essential for training and evaluating controllable video generators, yet current vision-language models fail the task in recurring, surprising ways. They confuse translation with rotation, reverse left and right, treat optical zoom as physical dolly, and read object motion as camera motion, while also missing moderate-intensity moves that humans notice immediately. To turn this into a standalone research problem the authors define a two-level taxonomy of 17 atomic camera movements drawn from cinematography practice, release the ACaM benchmark of both real-world and synthetically generated clips that isolate one movement per clip, and build a multi-source training corpus re-balanced by targeted augmentations (cropping for zoom, temporal reverse for dolly, flips for truck, affine rolls). Fine-tuning an 8B model on that corpus yields absolute gains of roughly 10% on real videos and 11% on synthetic videos over Gemini 3.1 Pro, yet human accuracy remains near 93–98%, leaving a clear performance gap that the authors present as an open invitation for further work.

Core claim

Vision-language models do not yet understand camera movement in natural language; they systematically mis-map the same visual cues that humans read as dolly, pan, truck, zoom, or tracking. By grounding evaluation in a 17-class cinematographic taxonomy and an atomic real-and-synthetic benchmark, and by fine-tuning on a carefully re-balanced multi-source set, an 8B VLM can surpass the strongest proprietary baseline by 10–11% absolute accuracy while remaining far below human performance.

What carries the argument

The ACaM (Atomic Camera Movement) benchmark, built on a two-level cinematographic taxonomy of 17 classes that cleanly separates translations, rotations, focal-length changes, static shots, and object-centric moves (tracking, arc). Real clips are filtered and re-verified from prior motion datasets; synthetic clips are generated via caption-then-Veo with manual reclassification and regeneration. The same taxonomy structures the 27 K instruction-tuning set whose targeted augmentations supply the supervision that produces the reported gains.

Load-bearing premise

The synthetic half of the benchmark, produced by prompting a commercial video generator and then manually accepting, reclassifying or discarding clips, is a faithful, unbiased proxy for the 17 intended atomic classes.

What would settle it

An independent multi-annotator re-label of the entire synthetic evaluation set that finds a large fraction of accepted clips still mismatched to their assigned class (especially roll direction and zoom-versus-dolly), which would invalidate both the 11% synthetic gain and the claim that ACaM cleanly isolates atomic movements.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Automatic filtering and captioning of large unannotated video collections for camera-aware training data becomes practical at higher reliability.
  • Video-generation leaderboards can score whether models obey specific cinematographic prompts without exhaustive manual review for atomic cases.
  • Object-centric and subtle-intensity movements remain the clearest remaining targets for closing the human–machine gap.
  • Once atomic recognition is reliable, compound multi-move and long-form camera sequences become the natural next evaluation target.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Geometry-based pose estimators and language models remain complementary; feeding estimated trajectories into VLM heads may specifically reduce the translation–rotation and left–right confusions documented here.
  • The same five failure modes are likely to degrade any VLM that must reason about viewpoint change, including robotics scene description and AR/VR spatial grounding.
  • The paper’s motion-specific augmentations (affine rolls, temporal reverse, progressive crops) form a reusable recipe for other long-tail spatial perception tasks beyond camera motion.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper establishes natural language camera-movement understanding as a standalone VLM task. It documents five failure modes of current VLMs (insensitivity to small motion, translation–rotation confusion, left–right errors, zoom vs. dolly, and object vs. camera motion), introduces a 17-class cinematographic taxonomy, and releases ACaM—an atomic MCQ benchmark with real-world clips (1,423 videos / 1,464 QA from CameraBench, ShotBench, MotionBench, FavorBench, CineTechBench, and curated YouTube) plus synthetic clips generated via Gemini-3-Pro captioning and Veo 3.1 with manual reclassification. A multi-source 27K instruction set with targeted augmentations (progressive crop, temporal reverse, horizontal flip, affine roll) is used to SFT Qwen3-VL-4B/8B. The 8B model reports relative gains of ~10% and ~11% over Gemini-3.1-Pro on real and synthetic class-average accuracy (Tables 5–6), with ablations (Table 7), binary-QA F1 (supplement), geometry and specialized-VLM controls, and a human baseline (≈93–98%, IAA 0.953). A large gap to humans remains.

Significance. If the reported numbers hold, the work supplies a clean task definition, a reusable atomic benchmark with human ceiling, and a strong open 8B baseline that already exceeds a leading proprietary model on both real and synthetic splits. That combination is directly useful for (i) automatic filtering/captioning of cinematographic video–text data and (ii) automatic verification of camera prompts in text-to-video systems—both bottlenecks the introduction correctly identifies. Strengths include multi-source training with explicit balancing augmentations, dual Overall/Avg reporting, geometry and prior specialized-VLM controls, intensity and LoRA ablations, and a documented human study. The residual human–model gap is framed as an open research problem rather than oversold.

major comments (2)
  1. Abstract and §1 headline the result as outperforming Gemini-3.1-Pro by “10% and 11%,” while §1 later says “relative improvements.” Tables 5–6 show absolute Overall gaps of ~4.3 / ~5.2 points and class-average (Avg) gaps of ~6.6 / ~7.9 points; the ~10% / ~11% figures match relative improvement on the Avg column only. Because this is the paper’s central quantitative claim, the abstract, intro, and table captions should state explicitly: (a) relative vs. absolute, (b) Overall vs. Avg, and (c) which column is primary. Without that, readers comparing Overall columns will understate the result, and readers taking “10%” as absolute points will overstate it.
  2. §3.3 and Fig. 8: synthetic evaluation clips are produced by Gemini-3-Pro prompt generation + Veo 3.1, with manual reclassification and a second generation pass. The paper correctly notes Veo biases (roll direction inversion; zoom collapsing to static/dolly) and heavy filtering. Because Gemini models are also evaluated on this split (Tables 5–6), the manuscript should add a short quantitative check that ranking and relative gains are not driven by residual Gemini-friendly artifacts—e.g., report Gemini vs. SFT accuracy restricted to the “accepted without reclassification” subset, or confirm that the real-world-only ranking already matches the joint ranking (which it appears to). The real-world half is independently curated and already supports the claim; making that independence explicit would remove residual doubt about the synthetic half.
minor comments (7)
  1. §2.1 / Table 2 and supplement Table 8: intensity bins (low/medium/high) for “push in” are human-labeled; a one-sentence definition of the intensity criteria (e.g., approximate FOV change or pixel shift) would aid reproducibility.
  2. Fig. 7 class distributions: “roll” remains underrepresented even after YouTube enrichment. A brief note on whether Avg accuracy is macro-averaged over the 17 classes (and how empty/near-empty cells are handled) would clarify the Avg column.
  3. §4.2 / Fig. 9: augmentation volumes (~1K zoom crop, ~2K temporal reverse, ~1.7K flip, affine roll) are stated; reporting the final per-class counts in the 27K set (or a small table) would make the balancing claim fully checkable.
  4. §5: geometry models (Mega-SaM, ViPE) cannot predict zoom/tracking/arc by design; the “–” cells are appropriate, but a footnote that their Overall/Avg are computed only over applicable classes would avoid unfair comparison.
  5. Abstract: “Gemini 3.1 Pro” vs. body “Gemini-3.1-Pro” / “Gemini-3-Pro”—normalize model naming throughout.
  6. Project page URL is given; if code, ACaM splits, and the 27K training list will be released, state that explicitly in the conclusion or reproducibility statement.
  7. Supplement Figs. 10–11 (full confusion matrices) are valuable; a one-sentence pointer in §2.2–2.3 of the main text would help readers find them.

Circularity Check

0 steps flagged

No significant circularity: empirical SFT gains and ACaM scores are measured against held-out human-verified labels, not derived from self-defined quantities or load-bearing self-citations.

full rationale

This is an empirical vision-language paper whose central claims (failure modes of existing VLMs, introduction of a 17-class cinematographic taxonomy and ACaM benchmark with real+synthetic atomic clips, construction of a multi-source 27K training set with geometric augmentations, and SFT Qwen3-VL-8B outperforming Gemini-3.1-Pro by ~10–11% overall accuracy on the held-out real and synthetic MCQ splits while remaining far below human ~93–98%) rest on direct measurement against human-verified ground-truth labels (Tables 5–6, ablation Table 7, binary F1 in supplement). Training clips are drawn from external sources (CameraBench, ShotBench, SpatialVID, MultiCamVideo, GenDoP) plus standard geometric operators (crop/resize for zoom, temporal reverse for dolly, flip for truck, affine roll); none of the reported accuracies is obtained by fitting a parameter to a subset and then “predicting” a quantity that is definitionally identical. Synthetic evaluation clips are generated via Gemini captioning + Veo 3.1 with subsequent manual reclassification/filtering, but the final labels used for scoring are human-assigned, so model rankings (including Gemini itself) are not circular with respect to the generator. The taxonomy is taken from standard cinematography literature rather than self-defined uniqueness theorems. There are no self-definitional equations, no fitted-input-called-prediction steps, no load-bearing self-citation chains, and no renaming of known results presented as first-principles derivation. The work is therefore self-contained against its external benchmarks; circularity score is zero.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

As an empirical multimodal paper the load-bearing commitments are the adopted cinematographic taxonomy, the correctness of manual and LLM-assisted labels, the fidelity of synthetic generation, and standard SFT hyper-parameters. No new physical entities or free constants are postulated; free parameters are ordinary training knobs.

free parameters (3)
  • LoRA rank = 256
    Chosen by ablation (r=64/128/256); final reported models use r=256 for the 8B model.
  • learning rate / batch size / epochs / max frames = 1e-4 / 128 / 5 / 16
    Standard SFT hyper-parameters set by the authors (1e-4, 128, 5, 16) without further search reported.
  • augmentation volumes (zoom crop, temporal reverse, horizontal flip, affine roll) = ~1k / 2k / 1.7k
    Hand-chosen quantities (≈1k, 2k, 1.7k, and additional rolls) used to re-balance the long-tail training distribution.
axioms (4)
  • domain assumption The 17-class two-level taxonomy (translations, rotations, zoom, static, tracking, arc) drawn from cinematography literature is an adequate and unambiguous label space for atomic camera movement.
    Invoked throughout Sec. 3.1 and Table 1; all evaluation and training labels are projected onto this set.
  • domain assumption Manual verification by graduate students with cinematography training, plus GPT/Gemini parsing of captions, yields sufficiently clean ground-truth labels for both training and evaluation.
    Described in Sec. 3.2, 4.1 and the supplement; residual noisy samples are acknowledged but assumed rare after filtering.
  • domain assumption Geometric heuristics that map estimated or synthetic camera poses to semantic movement labels are accurate enough for training data construction.
    Used for SpatialVID and MultiCamVideo (Sec. 4.1).
  • ad hoc to paper Atomic (single-movement) clips are the correct first step toward understanding compound camera motion.
    Explicit design choice stated in the introduction and Sec. 3; compound movements are left for future work.
invented entities (2)
  • ACaM benchmark (Atomic Camera Movement) no independent evidence
    purpose: Provide a standardized real+synthetic evaluation set with human baseline for the newly framed task.
    Constructed by the authors from six sources plus YouTube and Veo generation; first resource of its kind with both real and synthetic atomic clips.
  • Targeted camera-movement augmentation operators (progressive crop for zoom, temporal reverse for dolly, horizontal flip for truck, continuous affine roll) no independent evidence
    purpose: Re-balance the long-tail training distribution without collecting new real footage.
    Introduced in Sec. 4.2 and Fig. 9; effectiveness shown only via the authors’ own ablation.

pith-pipeline@v1.1.0-grok45 · 29641 in / 3180 out tokens · 34767 ms · 2026-07-12T05:14:55.488411+00:00 · methodology

0 comments
read the original abstract

Understanding camera movement in natural language is critical for training and evaluating video generation models, among other applications. However, we demonstrate that existing vision-language models (VLMs) fail this task in surprising ways, frequently confusing translation with rotation, left with right, and object movement with camera movement. To address these limitations, we establish natural language camera movement understanding as a standalone research task. We introduce a two-level cinematographic taxonomy and an extensive, atomic benchmark featuring both real and synthetic videos. Furthermore, we curate a large-scale, multi-source training set enhanced by targeted camera movement augmentation. Our fine-tuned VLM-8B outperforms Gemini 3.1 Pro by 10% and 11% on our benchmark's real and synthetic videos, respectively. Despite these gains, a significant gap remains relative to human performance, underscoring the need to promote and facilitate future research on natural language camera movement understanding.

Figures

Figures reproduced from arXiv: 2607.03043 by Boqing Gong, Haoxiang Li, Jin Huang, Joey Huang, Yuwen Tan.

Figure 1
Figure 1. Figure 1: Some examples of camera movements in cinematographic practice: Camera rotations (pan, tilt, roll) and translations (boom, dolly, truck). video-text pairs. Therefore, robust models capable of automatically filtering, cu￾rating, and captioning cinematographic camera movements in large-scale, unan￾notated videos are becoming increasingly relied upon to eliminate the bottle￾neck of expensive manual annotation.… view at source ↗
Figure 2
Figure 2. Figure 2: An example illustrating VLMs’ limited sensitivity to small camera movement. on real-world videos and 11% on synthetic videos over the strong proprietary model Gemini-3.1-Pro. However, a human study still exposes a substantial gap in performance between our best VLM and human, thus calling for future and more extensive research on natural language camera movement understanding. 2 Why is camera movement unde… view at source ↗
Figure 3
Figure 3. Figure 3: Examples of translation-rotation and left-right confusion. 2.2 Confusion Between Translation and Rotation Beyond fine-grained sensitivity, VLMs also frequently struggle to distinguish be￾tween camera translation and rotation. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Partial confusion matrices for Qwen3-VL-32B (left) and Gemini-3-Pro (right), illustrating translation-rotation and left-right confusion. (Full matrix in supplement) Q: What is the camera movement in this video shot? Please answer with only the letter of the correct option (A, B, C, or D). A. Truck left B. Pan right C. Push in D. Pull out Qwen3-VL-8B:C❌ Qwen3-VL-32B:C❌ Gemini-3-Pro:C ❌ Human:D ✅ As the clip… view at source ↗
Figure 5
Figure 5. Figure 5: A failure case in distinguishing optical ‘zoom out’ from physical ‘dolly out’. 2.4 Confusion Between Optical Zoom and Physical Dolly A prominent blind spot for current VLMs is in their limited ability to distin￾guish optical scaling from physical movement. Although both produce a centered expansion or contraction, their underlying mechanics differ. Camera translation introduces depth-dependent parallax tha… view at source ↗
Figure 6
Figure 6. Figure 6: Object motion challenges current VLMs’ inference about camera movement [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Dataset statistics of our ACaM evaluation benchmark. Top: Source composition and duration distribution of real-world videos. Bottom: Class-wise camera movement distribution of real-world and synthetic videos. natural language. Furthermore, our benchmark provides a more comprehensive set of categories than Cinematic2K [19] and CineTechBench [43], spanning five core dimensions: physical translations, angular… view at source ↗
Figure 8
Figure 8. Figure 8: Pipeline for generating synthetic videos of various camera movement classes. extract subsets related to camera movement and retain only evaluation questions involving a single camera movement. Similarly, for fine-grained motion under￾standing benchmarks (MotionBench [10] and FavorBench [36]), we extract the camera-related QA subsets and remove questions involving object motion. These datasets present an in… view at source ↗
Figure 9
Figure 9. Figure 9: Targeted data augmentation for distribution balancing. Top: The shift from an unbalanced 45K raw dataset to a refined 27K instruction-tuning set. Bottom: The augmentation operators used to synthesize samples for tail classes. subset of SpatialVID [38], which provides estimated camera poses, despite being highly dominated by the ‘push in’ movement. MultiCamVideo [1] is a dataset of synthetic videos rendered… view at source ↗
Figure 10
Figure 10. Figure 10: Confusion matrix of Qwen3-VL-32B on the entire synthetic evaluation set. 9 More Details of Real-World Videos Subset Construction Examples of FavorBench and MotionBench. Unlike CameraBench [21] and ShotBench [22], which focus on the dominant camera movement over an entire clip, FavorBench [36] and MotionBench [10] often require identifying the camera movement associated with a specific event or timestamp. … view at source ↗
Figure 11
Figure 11. Figure 11: Confusion matrix of Gemini-3-Pro on the entire synthetic evaluation set. camera movement labels, annotation errors, or cases where the correct cam￾era motion is difficult to determine. Several illustrative examples are shown in [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Examples of camera movement questions from MotionBench and FavorBench. 10 More Details of Synthetic Videos Subset Construction System Prompts for Gemini-3-Pro. For each clip, we provide the raw video and its corresponding camera-movement label to Gemini-3-Pro [35] and instruct it to analyze the video and generate a structured prompt for video generation. The system prompt used for caption generation is sh… view at source ↗
Figure 13
Figure 13. Figure 13: Illustrative examples of noisy or ambiguous samples in existing motion bench￾marks identified during manual verification. to form a four-option question, and the position of the correct answer is uni￾formly randomized across options to avoid potential option bias. 11 More Details of Training Set Construction Our training set is formulated as a four-choice multiple-choice question answering (MCQ) task that… view at source ↗
Figure 14
Figure 14. Figure 14: System prompt used to prompt Gemini-3-Pro to generate camera-movement captions from real-world videos, which are subsequently used by Veo 3.1 for video generation. classify the captions into three categories: single motion, combined motion, and complex motion. Only clips categorized as single motion are retained for training. The system prompt used for this classification process is shown in [PITH_FULL_I… view at source ↗
Figure 15
Figure 15. Figure 15: System prompt used to prompt Gemini-3-Pro to refine the generated captions for improved clarity [PITH_FULL_IMAGE:figures/full_fig_p027_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: System prompt used to prompt Gemini-3-Pro to generate additional prompts for underrepresented camera movements based solely on the camera-movement label [PITH_FULL_IMAGE:figures/full_fig_p028_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Successful video generation example where Veo 3.1 correctly reproduces the intended camera movement. A slow, deliberate zoom-in draws the viewer's eye from a high, rocky vantage point toward the fortified coastal city of Dubrovnik, revealing the dense cluster of terracotta rooftops nestled within ancient stone walls. The vibrant orange tiles contrast sharply against the deep blue waters of the Adriatic Se… view at source ↗
Figure 18
Figure 18. Figure 18: Example requiring reclassification. While the original video exhibits a ‘zoom in’ movement, the generated video instead shows a ‘pan right’ movement [PITH_FULL_IMAGE:figures/full_fig_p029_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Overview of the data construction pipeline, including caption generation, human reclassification, prompt refinement, and video regeneration [PITH_FULL_IMAGE:figures/full_fig_p030_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Failure cases of Veo 3.1, including abrupt frame transitions [PITH_FULL_IMAGE:figures/full_fig_p030_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Examples of generated videos with extremely subtle camera motion that ap￾pear nearly static and are filtered out during data construction. """ You are a camera movement classifier. Given a camera-movement description, decide: 1. If the description clearly corresponds to exactly one category → return "certain". 2. If the description is ambiguous, unclear, too short, or multiple labels are possible → return… view at source ↗
Figure 22
Figure 22. Figure 22: System prompt to extract camera movement from CameraBench [21]. """ You are an expert camera-motion classifier. You will categorize a camera motion description into: 1. single motion 2. double motion 3. complex motion Follow these rules: SINGLE MOTION - Static throughout. - OR single continuous motion (e.g., steady forward). - OR simple one-step change (static → forward). - Does NOT include “A while B”. D… view at source ↗
Figure 23
Figure 23. Figure 23: System prompt used to extract single movement videos from GenDoP [49] [PITH_FULL_IMAGE:figures/full_fig_p031_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Qualitative comparison of model predictions under the MCQ setting. Q: What is the camera movement in this movie shot? A. Tilt down B. Pan left C. Static D. Zoom in Please answer with only the letter of the correct option (A, B, C, or D) Qwen3-VL-32B:C❌ Gemini-3-Pro:C❌ Our SFT (4B): C Our SFT (8B): C Human:A ✅ Q: What is the camera movement in this video shot? A. Arc B. Pan right C. Static D. Boom up Pleas… view at source ↗
Figure 25
Figure 25. Figure 25: Qualitative comparison of model predictions under the MCQ setting [PITH_FULL_IMAGE:figures/full_fig_p032_25.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

72 extracted references · 20 linked inside Pith

  1. [1]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Bai,J., Xia, M., Fu, X.,Wang, X.,Mu, L., Cao, J.,Liu, Z., Hu, H.,Bai, X., Wan, P., et al.: Recammaster: Camera-controlled generative rendering from a single video. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 14834–14844 (2025)

    2025

  2. [2]

    arXiv preprint arXiv:2511.21631 (2025)

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025)

    Pith/arXiv arXiv 2025

  3. [3]

    5-vl technical report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

    Pith/arXiv arXiv 2025

  4. [4]

    Bordwell, D., Thompson, K., Smith, J.: Film art: An introduction, vol. 7. McGraw- Hill New York (2008) 16 Y. Tan et al

    2008

  5. [5]

    Brooks, T., Peebles, B., Holmes, C., DePue, W., Guo, Y., Jing, L., Schnurr, D., Taylor, J., Luhman, T., Luhman, E., Ng, C., Wang, R., Ramesh, A.: Video gener- ation models as world simulators (2024),https://openai.com/research/video- generation-models-as-world-simulators, last accessed 2026/06/29

    2024

  6. [6]

    Routledge (2016)

    Brown, B.: Cinematography: theory and practice: image making for cinematogra- phers and directors. Routledge (2016)

    2016

  7. [7]

    arXiv preprint arXiv:2506.01674 (2025)

    Du, Y., Fan, T., Nan, K., Xie, R., Zhou, P., Li, X., Yang, J., Yang, Z., Tai, Y.: Mo- tionsight: Boosting fine-grained motion understanding in multimodal llms. arXiv preprint arXiv:2506.01674 (2025)

    arXiv 2025

  8. [8]

    arXiv preprint arXiv:2505.20279 (2025)

    Fan, Z., Zhang, J., Li, R., Zhang, J., Chen, R., Hu, H., Wang, K., Qu, H., Wang, D., Yan, Z., et al.: Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction. arXiv preprint arXiv:2505.20279 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    google / models / veo/(2024), last accessed 2026/06/29

    Google: Veo.https : / / deepmind . google / models / veo/(2024), last accessed 2026/06/29

    2024

  10. [10]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Hong, W., Cheng, Y., Yang, Z., Wang, W., Wang, L., Gu, X., Huang, S., Dong, Y., Tang, J.: Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 8450–8460 (2025)

    2025

  11. [11]

    arXiv preprint arXiv:2511.21688 (2025)

    Hu, W., Lin, J., Long, Y., Ran, Y., Jiang, L., Wang, Y., Zhu, C., Xu, R., Wang, T., Pang, J.: G2 vlm: Geometry grounded vision language model with unified 3d reconstruction and spatial reasoning. arXiv preprint arXiv:2511.21688 (2025)

    arXiv 2025

  12. [12]

    arXiv preprint arXiv:2508.10934 (2025)

    Huang, J., Zhou, Q., Rabeti, H., Korovko, A., Ling, H., Ren, X., Shen, T., Gao, J., Slepichev,D.,Lin,C.H.,etal.:Vipe:Videoposeenginefor3dgeometricperception. arXiv preprint arXiv:2508.10934 (2025)

    Pith/arXiv arXiv 2025

  13. [13]

    In: European conference on computer vision

    Huang, Q., Xiong, Y., Rao, A., Wang, J., Lin, D.: Movienet: A holistic dataset for movie understanding. In: European conference on computer vision. pp. 709–727. Springer (2020)

    2020

  14. [14]

    In: Proceedings of the 33rd ACM International Conference on Multimedia

    Jia, Z., Zhang, Z., Qian, J., Wu, H., Sun, W., Li, C., Liu, X., Lin, W., Zhai, G., Min, X.: Vqa2: visual question answering for video quality assessment. In: Proceedings of the 33rd ACM International Conference on Multimedia. pp. 6751–6760 (2025)

    2025

  15. [15]

    Columbia University Press (2019)

    Keating, P.: The dynamic frame: camera movement in classical hollywood. Columbia University Press (2019)

    2019

  16. [16]

    arXiv preprint arXiv:2412.03603 (2024)

    Kong, W., Tian, Q., Zhang, Z., Min, R., Dai, Z., Zhou, J., Xiong, J., Li, X., Wu, B., Zhang, J., et al.: Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 (2024)

    Pith/arXiv arXiv 2024

  17. [17]

    Kuaishou: Kling.https://kling.ai/(2024), last accessed 2026/06/29

    2024

  18. [18]

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y.,Liu,Z.,Li,C.:LLaVA-onevision:Easyvisualtasktransfer.TransactionsonMa- chine Learning Research (2025),https://openreview.net/forum?id=zKv8qULV6n

    2025

  19. [19]

    arXiv preprint arXiv:2412.12223 (2024)

    Li, X., Wu, K., Yang, S., Qu, Y., Zhang, G., Chen, Z., Li, J., Mu, J., Hu, X., Fang, W., et al.: Can video generation replace cinematographers? research on the cinematic language of generated video. arXiv preprint arXiv:2412.12223 (2024)

    Pith/arXiv arXiv 2024

  20. [20]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Li, Z., Tucker, R., Cole, F., Wang, Q., Jin, L., Ye, V., Kanazawa, A., Holynski, A., Snavely, N.: Megasam: Accurate, fast and robust structure and motion from casual dynamic videos. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 10486–10496 (2025)

    2025

  21. [21]

    arXiv preprint arXiv:2504.15376 (2025) Natural Language Camera Movement Understanding 17

    Lin, Z., Cen, S., Jiang, D., Karhade, J., Wang, H., Mitra, C., Ling, T., Huang, Y., Liu, S., Chen, M., et al.: Towards understanding camera motions in any video. arXiv preprint arXiv:2504.15376 (2025) Natural Language Camera Movement Understanding 17

    Pith/arXiv arXiv 2025

  22. [22]

    arXiv preprint arXiv:2506.21356 (2025)

    Liu, H., He, J., Jin, Y., Zheng, D., Dong, Y., Zhang, F., Huang, Z., He, Y., Li, Y., Chen, W., et al.: Shotbench: Expert-level cinematic understanding in vision- language models. arXiv preprint arXiv:2506.21356 (2025)

    arXiv 2025

  23. [23]

    MiniMax: Hailuo.https://hailuoai.video/(2024), last accessed 2026/06/29

    2024

  24. [24]

    arXiv preprint arXiv:2503.02341 (2025)

    Mou, Z., Xia, B., Huang, Z., Yang, W., Jia, J.: Gradeo: Towards human-like evaluation for text-to-video generation via multi-step reasoning. arXiv preprint arXiv:2503.02341 (2025)

    arXiv 2025

  25. [25]

    Department of Inf

    Nielsen, J.I., Kau, E., Raskin, R.: Camera movement in narrative cinema: towards a taxonomy of functions. Department of Inf. & Media Studies, University of Aarhus (2007)

    2007

  26. [26]

    arXiv preprint arXiv:2410.21276 (2024)

    OpenAI: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    Pith/arXiv arXiv 2024

  27. [27]

    In: European Conference on Computer Vision

    Pan, L., Baráth, D., Pollefeys, M., Schönberger, J.L.: Global structure-from-motion revisited. In: European Conference on Computer Vision. pp. 58–77. Springer (2024)

    2024

  28. [28]

    In: European Conference on Computer Vision

    Rao, A., Wang, J., Xu, L., Jiang, X., Huang, Q., Zhou, B., Lin, D.: A unified framework for shot type classification based on subject centric lens. In: European Conference on Computer Vision. pp. 17–34. Springer (2020)

    2020

  29. [29]

    Data in Brief51, 109627 (2023)

    Savardi, M., Kovács, A.B., Signoroni, A., Benini, S.: Cinescale2: a dataset of cine- matic camera features in movies. Data in Brief51, 109627 (2023)

    2023

  30. [30]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Schonberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4104–4113 (2016)

    2016

  31. [31]

    arXiv preprint arXiv:2601.03267 (2025)

    Singh, A., Fry, A., Perelman, A., Tart, A., Ganesh, A., El-Kishky, A., McLaughlin, A., Low, A., Ostrow, A., Ananthram, A., et al.: Openai gpt-5 system card. arXiv preprint arXiv:2601.03267 (2025)

    Pith/arXiv arXiv 2025

  32. [32]

    Univ of California Press (1969)

    Spottiswoode, R.: A grammar of the film: An analysis of film technique. Univ of California Press (1969)

    1969

  33. [33]

    In: Proceed- ings of the Computer Vision and Pattern Recognition Conference

    Sun, K., Huang, K., Liu, X., Wu, Y., Xu, Z., Li, Z., Liu, X.: T2v-compbench: A comprehensive benchmark for compositional text-to-video generation. In: Proceed- ings of the Computer Vision and Pattern Recognition Conference. pp. 8406–8416 (2025)

    2025

  34. [34]

    Tang, Y., Guo, J., Hua, H., Liang, S., Feng, M., Li, X., Mao, R., Huang, C., Bi, J., Zhang, Z., et al.: Vidcomposition: Can mllms analyze compositions in compiled videos? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8490–8500 (2025)

    2025

  35. [35]

    arXiv preprint arXiv:2312.11805 (2023)

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., Hauth, A., Millican, K., et al.: Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)

    Pith/arXiv arXiv 2023

  36. [36]

    arXiv preprint arXiv:2503.14935 (2025)

    Tu, C., Zhang, L., Chen, P., Ye, P., Zeng, X., Cheng, W., Yu, G., Chen, T.: Favor- bench: A comprehensive benchmark for fine-grained video motion understanding. arXiv preprint arXiv:2503.14935 (2025)

    Pith/arXiv arXiv 2025

  37. [37]

    arXiv preprint arXiv:2503.20314 (2025)

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025)

    Pith/arXiv arXiv 2025

  38. [38]

    arXiv preprint arXiv:2509.09676 (2025)

    Wang, J., Yuan, Y., Zheng, R., Lin, Y., Gao, J., Chen, L.Z., Bao, Y., Zhang, Y., Zeng, C., Zhou, Y., et al.: Spatialvid: A large-scale video dataset with spatial annotations. arXiv preprint arXiv:2509.09676 (2025)

    arXiv 2025

  39. [39]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: Vggt: Visual geometry grounded transformer. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 5294–5306 (2025) 18 Y. Tan et al

    2025

  40. [40]

    arXiv preprint arXiv:2505.12098 (2025)

    Wang, J., Duan, H., Jia, Z., Zhao, Y., Yang, W.Y., Zhang, Z., Chen, Z., Wang, J., Xing, Y., Zhai, G., et al.: Love: Benchmarking and evaluating text-to-video gener- ation and video-to-text interpretation. arXiv preprint arXiv:2505.12098 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Wang, Q., Zhang, Y., Holynski, A., Efros, A.A., Kanazawa, A.: Continuous 3d perception model with persistent state. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 10510–10522 (2025)

    2025

  42. [42]

    5: Advancing open-source multimodal models in versatility, reasoning, and efficiency

    Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., et al.: Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025)

    Pith/arXiv arXiv 2025

  43. [43]

    arXiv preprint arXiv:2505.15145 (2025)

    Wang, X., Xu, S., Shan, X., Zhang, Y., Diao, M., Duan, X., Huang, Y., Liang, K., Ma, Z.: Cinetechbench: A benchmark for cinematographic technique understanding and generation. arXiv preprint arXiv:2505.15145 (2025)

    Pith/arXiv arXiv 2025

  44. [44]

    5: Empowering video mllms with long and rich context modeling

    Wang, Y., Li, X., Yan, Z., He, Y., Yu, J., Zeng, X., Wang, C., Ma, C., Huang, H., Gao, J., et al.: Internvideo2. 5: Empowering video mllms with long and rich context modeling. arXiv preprint arXiv:2501.12386 (2025)

    Pith/arXiv arXiv 2025

  45. [45]

    arXiv preprint arXiv:2505.23747 (2025)

    Wu, D., Liu, F., Hung, Y.H., Duan, Y.: Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence. arXiv preprint arXiv:2505.23747 (2025)

    Pith/arXiv arXiv 2025

  46. [46]

    arXiv preprint arXiv:2510.02423 (2025)

    Wu, H., Cai, Y., Ge, H., Chen, H., Yang, M.H., Wang, Y.: Refineshot: Rethinking cinematography understanding with foundational skill evaluation. arXiv preprint arXiv:2510.02423 (2025)

    arXiv 2025

  47. [47]

    arXiv preprint arXiv:2602.00181 (2026)

    Wu, H., Cai, Y., Li, Z., Ge, H., Sun, B., Yuan, J., Wang, Y.: Camreasoner: Re- inforcing camera movement understanding via structured spatial reasoning. arXiv preprint arXiv:2602.00181 (2026)

    Pith/arXiv arXiv 2026

  48. [48]

    arXiv preprint arXiv:2408.06072 (2024)

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al.: Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072 (2024)

    Pith/arXiv arXiv 2024

  49. [49]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Zhang,M.,Wu,T.,Tan,J.,Liu,Z.,Wetzstein,G.,Lin,D.:Gendop:Auto-regressive camera trajectory generation as a director of photography. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 18229–18239 (2025)

    2025

  50. [50]

    Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al.: Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 (2025) Natural Language Camera Movement Understanding 19 T able 8:VLM accuracy on ‘push in’vs.three levels of camera movement int...

    Pith/arXiv arXiv 2025

  51. [51]

    Slowly dolly in,

    [Cinematography]: Start with the provided camera motion. You MUST observe the video to describe the speed and rhythm of the movement (e.g., "Slowly dolly in," "A fast sweeping pan to the left," "A gradual tracking shot")

  52. [52]

    [Subject]: Identify and describe the main focal point

  53. [53]

    [Action]: Detail the specific movements or behaviors of the subject

  54. [54]

    [Context]: Describe the environment, background, and spatial setting

  55. [55]

    Subject:

    [Style & Ambiance]: Define the lighting, aesthetic style, and mood. Constraints: - OUTPUT MUST BE A SINGLE CONTINUOUS SENTENCE OR PARA- GRAPH. - DO NOT use labels like "Subject:". Output one continuous, cinematic paragraph. - The camera motion MUST be the very first part of the sentence. - Be descriptive but concise; provide enough detail for generation w...

  56. [56]

    Pan left,

    A camera motion type (e.g., "Pan left," "Dolly in")

  57. [57]

    following

    A prompt that may have physics errors OUTPUT: A corrected prompt that follows motion physics rules EXACTLY. CRITICAL MOTION PHYSICS RULES: PAN/TILT (Rotation from fixed position): - Camera DOES NOT MOVE through space - Camera ROTATES HORIZONTALLY on a fixed point - Background SWEEPS across frame - Pan left -> scene enters from LEFT, exits RIGHT - Pan righ...

  58. [58]

    Identify the specified camera motion type

  59. [59]

    Check if opening syntax matches EXACTLY -> if not, replace with correct opening

  60. [60]

    Scan the middle content for physics violations

  61. [61]

    Preserve ALL specific visual content: objects, colors, lighting, subjects, actions

  62. [62]

    Check if ending syntax matches EXACTLY -> if not, replace with correct ending

  63. [63]

    "" Fig. 15:System prompt used to prompt Gemini-3-Pro to refine the generated captions for improved clarity. 28 Y. Tan et al

    Ensure 60-80 words total CONSTRAINTS: - Output ONLY the corrected prompt, no preamble - One paragraph, 60-80 words - Use exact opening and ending syntax for the motion type - Preserve all visual details from the original prompt - Fix ONLY physics violations """ Fig. 15:System prompt used to prompt Gemini-3-Pro to refine the generated captions for improved...

  64. [64]

    Zoom", "Arc

    A camera motion type ("Zoom", "Arc", or "Roll")

  65. [65]

    The camera zooms in optically without moving through space,

    The number of unique prompts to generate Each prompt must be 60-80 words and: - Start with the exact motion syntax - Describe how the camera motion affects what appears in the frame - End with the exact motion reinforcement - Depict a UNIQUE scene REQUIRED MOTION SYNTAX: Opening: - Zoom in: "The camera zooms in optically without moving through space," - Z...

  66. [66]

    ALWAYS start with exact opening syntax

  67. [67]

    Describe how the motion affects the frame

  68. [68]

    Use motion-specific vocabulary

  69. [69]

    Subjects are passive scene elements

  70. [70]

    ALWAYS end with exact ending syntax

  71. [71]

    Never mix motion types

  72. [72]

    subject":

    Generate UNIQUE scenes OUTPUT FORMAT: Return ONLY the prompts in this exact JSON format: [ "subject": "brief scene descriptor", "prompt": "full generated prompt" ] Do NOT include any additional text or formatting. """ Fig. 16:System prompt used to prompt Gemini-3-Pro to generate additional prompts for underrepresented camera movements based solely on the ...