Pith. sign in

REVIEW 5 major objections 5 minor 68 references

EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EditIQ fully automates cinematic editing of static wide-angle stage recordings by combining large-language-model dialogue interpretation with visual saliency, and a user study rates its edits close to professional human edits.

desk verdict Solid engineering with a genuinely new LLM-based shot potential, but the evaluation doesn't back the two-pronged claim: EditIQ is statistically indistinguishable from LLM-only editing on BBC-OSD. read the letter →

arxiv 2502.02172 v1 pith:UX66YNSG submitted 2025-02-04 cs.MM cs.CVcs.HC

Rohit Girmaji , Bhav Beri , Ramanathan Subramanian , Vineet Gandhi This is my paper · ORCID
classification cs.MMcs.CVcs.HC
keywords automatedvideoeditingcinematiclargelanguagemodelsdialogueinterpretationvisualsaliencyshotselectionenergyminimizationdynamicprogramming
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EditIQ claims that a single locked-off, wide-angle camera can be turned into a full cinematic edit without a camera crew or a human editor. The system simulates a team of virtual cameras by cropping and moving within the wide frame, then decides which virtual shot to show at every moment. The decision is driven by three signals: a language model reads the transcript and proposes which person or group the audience should be watching, a saliency model highlights non-verbal actions and reactions, and speaker detection marks who is talking. These signals are combined into a shot-importance score plus cinematic penalties for jump cuts, bad framing, wrong pacing, and too-fast cuts, and the best sequence is found by dynamic programming. In a 20-participant study the resulting edits beat random, wide, speaker-only, and saliency-only baselines, match an LLM-only variant, and come close to professional human edits on the benchmark dataset.

What carries the argument

The load-bearing mechanism is a trellis graph whose nodes are the $2^n-1$ possible virtual shots (rushes) at each frame plus the master shot, and whose objective combines three unary potentials with four cinematic penalty terms. The contextual potential assigns cost $\lambda_c$ to the LLM-chosen shot and smaller costs $\lambda_c/p$ to higher-order shots containing that actor; the saliency potential assigns $\lambda_{\mathrm{sal}}$ to the actor with the highest thresholded saliency score and $\lambda_{\mathrm{sal}}/2$ to the second; the speaker potential adds $\lambda_{\mathrm{sp}}$ to the current speaker. Higher-order shot potentials are composed from 1-shot potentials by a max-like formula, and penalties for overlap, misframing, rhythm, and transitions are added. Dynamic programming over the cost matrix yields the optimal shot sequence, with the master shot fixed as a 2-second establishing shot.

What would settle it

Re-running EditIQ on held-out stage performances while sweeping the unreported blending constants and saliency threshold would settle it: if no single setting reproduces the reported advantage of EditIQ over the LLM-only and saliency-only baselines, or if the published settings cannot be reproduced without peeking at the evaluation videos, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that narrative understanding from a large language model, prompted to suggest shot choices and cut points from a character-aware transcript, can serve as a reliable guide for camera selection, and that visual saliency adds complementary information when dialogue alone misses actions and reactions. EditIQ formalizes this as an energy minimization over virtual camera rushes: each rush gets a unary cost from contextual, saliency, and speaker potentials, and transition penalties enforce cinematic continuity; dynamic programming finds the optimal path. The authors report this is the first demonstration that LLM-based dialogue interpretation can effectively drive camera selection, and the user study shows the full system generally outperforms all automated baselines, with professional human edits remaining superior on the quiz-show benchmark.

Load-bearing premise

The load-bearing premise is that the fixed numbers blending the three shot-importance signals, along with the saliency threshold, whose values the paper never reports, will keep producing good edits on new performances rather than having been tuned to the evaluated videos.

Editorial extensions

If this is right

  • For quiz-style scenes where speech guides attention, the combined LLM-plus-saliency system performs on par with the LLM-only variant; the added value of saliency shows up in theatre recordings where actions happen away from the speaker.
  • The LLM-only baseline already beats speaker-only and saliency-only editing on narrative effectiveness, scene actions, and viewing experience, indicating that dialogue understanding carries most of the editing signal.
  • Because the pipeline is fully automated and runs much faster than manual editing, about 2 minutes per 2-minute clip on a standard GPU, it is practical as an assistive tool for low-budget productions.
  • Tuning the cinematic parameters, such as minimum and maximum shot length and overlap thresholds, changes the editing style, so the same system can produce faster or calmer cuts without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the approach transfers, a single static 4K camera could replace multi-camera crews for archival-to-promotional editing of theatre, concerts, talks, and ceremonies; the main cost barrier would be the wide-angle capture rather than the editor.
  • The energy-minimization shell is agnostic to where the unary potentials come from, so audio cues such as applause, music, or laughter, or the script itself, could be added as additional potentials without changing the optimizer.
  • The saliency model's apparent advantage on human-centric scenes suggests the visual potential could eventually be trained on editor ground truth rather than gaze data, potentially closing some of the remaining gap to human edits.
  • A streaming variant with incremental LLM calls, which the authors note is future work, would make the same pipeline applicable to live event production.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. EditIQ proposes a fully automated pipeline for editing static wide-angle recordings of staged events. The method generates virtual PTZ rushes, derives unary potentials from an LLM shot-selection prompt, a video saliency model, and an active-speaker detector, combines them with cinematic penalty terms, and solves the resulting energy minimization with dynamic programming. The authors evaluate the system in a 20-participant user study on the BBC Old School dataset and 11 theatre videos, comparing against random, wide, speaker, LLM-only, saliency-only, and human-edited baselines across four attributes. The central claims are that EditIQ outperforms automated baselines and is close to professional human edits, and that combining LLM dialogue understanding with saliency cues is the key novelty.

Significance. If the empirical claims were supported, EditIQ would be a meaningful advance for automated multicamera-style editing from a single static camera: it combines dialogue-level LLM understanding with visual saliency in a principled optimization framework, is fully automated, and is described in enough detail (prompts, model architecture, parameter set, datasets) to be reproduced and extended. The paper also includes a breadth of baselines, including human edits, and evaluates four attributes. However, the statistical evidence for the specific two-pronged contribution is weak, and an internal inconsistency in the potential definitions affects the core optimization. The contribution is therefore potentially valuable but not currently established at the level claimed.

major comments (5)
  1. [§3.3, Eqs. (2)–(4)] The definitions of the contextual potential are internally inconsistent. If a single-order shot is selected, Eq. (2) assigns a cost λ_c/p to any p-order shot containing that actor. But Eq. (4) computes a higher-order potential from single-order potentials as C(a)+C(b)-|C(a)-C(b)|. For a 2-shot containing the selected actor and one actor with zero potential, this formula gives 0, not λ_c/2. Conversely, when a p-order shot is selected with p=2, Eq. (3) assigns λ_c/3 to each constituent 1-shot, and Eq. (4) then yields 2λ_c/3, not the claimed λ_c. Since these potentials enter the unary term in Eq. (8) and drive the DP solution in Eq. (7), the optimization is operating on an inconsistent objective as written. Please correct the potential definitions and re-verify the experimental results under the corrected computation.
  2. [§5.2.1, §5.2.2, Figure 5] The central two-pronged claim that adding saliency to the LLM potential improves editing is not supported by the reported statistics. On BBC-OSD, EditIQ is statistically indistinguishable from LLM-only editing on all attributes (NE p=0.6728, SA p=0.9177, ER p=0.9177, and VX reported only as comparable). On the theatre videos, the advantages over LLM-only are p=0.0909 for NE and p=0.0571 for VX, with SA and ER described as insignificantly higher. All of these are nonsignificant at the conventional 0.05 level. The conclusion that EditIQ generally outperforms other baselines therefore rests on comparisons against random, wide, speaker, and saliency-only baselines, not on a demonstrated saliency benefit. Please report effect sizes, confidence intervals, and a mixed-effects analysis that accounts for rater and video clustering.
  3. [§4.3 and Eqs. (2), (5), (6), (8)] The values of λ_c, λ_sal, λ_sp, and τ_sal are never reported, although these constants determine the relative contribution of the saliency term to the unary potential and hence the final shot sequence. Section 4.3 reports only the rhythm and overlap parameters. Because the observed EditIQ-versus-LLM differences are small and mostly nonsignificant, the undisclosed saliency weight is load-bearing: if λ_sal and τ_sal were selected on the evaluation set, the claimed advantage would not transfer. Please report all free parameters and include a sensitivity analysis over λ_sal and τ_sal to show that the qualitative results are stable.
  4. [§5.1, Materials & Methods] The evaluation has only two ratings per video-condition (20 participants, each viewing 2–3 videos), yet the reported ANOVAs and t-tests treat individual ratings as independent observations. This ignores rater effects and video-level clustering, and no inter-rater reliability measure is reported. With n=2 per cell, a single outlier rating can determine a video's mean, so the p-values are fragile. Please re-analyze the data with mixed-effects models that include random intercepts for participants and videos, and report variance components or intraclass correlation coefficients.
  5. [§4.4.4, §4.4.5, vs §3.7] The LLM-only and Saliency-only ablations are described as having cinematic corrections applied afterward, whereas EditIQ optimizes potentials and penalties jointly in Eq. (7) via dynamic programming. If the ablations do not solve the same optimization problem, the comparison conflates the contribution of the removed potential with the benefit of joint optimization. Please state whether the ablations use the same DP objective with the corresponding term removed, or re-run them in the same framework.
minor comments (5)
  1. [Eq. (7)] The objective uses -ln(U(r_t)) with U(r_t)=C(r_t)+V(r_t)+S(r_t), but C, V, and S are each zero for many rushes, which would make the logarithm undefined in the real numbers. Please specify the floor value or normalization used in the implementation.
  2. [§5.1] The design is described as '11 (video types) × 2 (user ratings/video) × 7', but the study uses 11 videos, not 11 video types; please correct the wording.
  3. [§5.2.3] The comparison of experienced versus non-experienced participants is presented without statistical tests or effect sizes; please add them or label the observation as anecdotal.
  4. [§1 and §6] The claim of being 'the first to demonstrate' LLM-guided camera selection for narrative emphasis is strong; please position it against recent LLM-based editing or camera-selection systems, or soften the claim.
  5. [Appendix B, Tables 1–2] The saliency model comparison reports only aggregate metrics; please state whether the improvements over ViNet are statistically significant and describe the training and evaluation protocol used for the modified model.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EditIQ's edits are produced by a forward pipeline and judged against external human ratings, not fitted to them.

full rationale

EditIQ's derivation chain is a forward model: an LLM maps the transcript to shot suggestions (Section 3.3), a pretrained saliency model produces per-actor scores (Section 3.4), a speaker detector provides the speaker potential (Section 3.5), and Equation (7) minimizes a sum of these unary potentials plus cinematic penalties. The evaluation in Section 5 is an independent psychophysical study comparing EditIQ to external baselines and human edits. Nothing in Equations (2)-(8) is fitted to the user ratings, and the energy function does not reduce to the evaluation labels by construction. The same-author citations (GAZED for the higher-order potential composition and ViNet for the saliency backbone) are component-level reuse: the former is an explicitly given algebraic combination (Equation (4)) and the latter is benchmarked on external datasets (Appendix B, Tables 1-2), so neither imports the central claim. The LLM-only ablation sharing the contextual potential with EditIQ is an experimental design choice, not a circular step, and the reported p-values even show that the saliency addition is not significantly better on BBC-OSD, which is opposite to a forced result. The undisclosed lambda values are a reproducibility concern, not circularity.

Assumptions & free parameters 14 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the transferability of pretrained LLM and saliency models to stage performances, on the validity of the hand-designed cinematic penalties, and on the assumption that fixed combination weights are universal. The paper does not disclose the weights or provide code, so the ledger below lists the main hand-set parameters and domain assumptions.

free parameters (14)
  • lambda_c = not reported
    Weight of LLM contextual potential in Equation (8); value never specified.
  • lambda_sal = not reported
    Weight of saliency potential in Equation (8); value never specified.
  • lambda_sp = not reported
    Weight of speaker potential in Equation (8); value never specified.
  • tau_sal = not reported
    Threshold applied to saliency maps before computing actor saliency scores (Section 3.4).
  • lambda_mis = not reported
    Misframing penalty magnitude (Section 3.6.2).
  • lambda_trans = not reported
    Transition penalty for every cut (Section 3.6.4).
  • gamma2 = not reported
    Scaling for long-hold penalty in rhythm model (Section 3.6.3).
  • mu = not reported
    Linear penalty scaling in overlap piecewise function (Section 3.6.1).
  • l = 1 second
    Minimum shot duration in rhythm penalty; set from film literature (Section 4.3).
  • m = 7 seconds
    Maximum shot duration in rhythm penalty; set to average modern film shot length (Section 4.3).
  • alpha = 0.15
    Overlap ratio below which no penalty (Section 4.3).
  • beta = 0.3
    Overlap ratio above which strong penalty (Section 4.3).
  • gamma1 = 100
    Scaling for fast-cut penalty (Section 4.3).
  • nu = 10^6
    Large penalty for severe overlap to strictly avoid jump cuts (Section 4.3).
assumptions (5)
  • standard math Dynamic programming on the shot-selection graph yields a global optimum of the energy in Equation (7).
    The cost matrix recursion is a standard Viterbi-style forward pass; relies on the first-order Markov assumption of transition penalties.
  • domain assumption The LLM (Claude 3.5 Sonnet, temperature 0) produces shot suggestions that are semantically meaningful for dialogue-driven scenes.
    No validation of LLM shot suggestions against editor decisions is provided; the user study indirectly tests the full pipeline.
  • domain assumption Saliency maps from a model trained on MVVA, Coutrot2, and ETMD datasets transfer to BBC-OSD and theatre recordings.
    The modified ViNet is evaluated on those training datasets in Appendix B but not on the test videos; transfer is assumed.
  • domain assumption The cinematic penalty functions and their thresholds encode universal editing rules.
    Parameters are drawn from film literature or set empirically; no validation that they apply to quiz and theatre content.
  • ad hoc to paper The contextual potential of a higher-order shot is the sum of its members' single-shot potentials minus the absolute difference (Equation 4), which equals twice the smaller potential.
    This formula is chosen without justification; it makes the group-shot potential equal to twice the minimum member potential, which is an arbitrary design choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues." pith.science (2026). https://pith.science/paper/UX66YNSG

@misc{pith2026250202172,
  author       = {Pith},
  title        = {Pith review of: EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UX66YNSG}},
  note         = {Machine review of arXiv:2502.02172}
}
read the original abstract

We present EditIQ, a completely automated framework for cinematically editing scenes captured via a stationary, large field-of-view and high-resolution camera. From the static camera feed, EditIQ initially generates multiple virtual feeds, emulating a team of cameramen. These virtual camera shots termed rushes are subsequently assembled using an automated editing algorithm, whose objective is to present the viewer with the most vivid scene content. To understand key scene elements and guide the editing process, we employ a two-pronged approach: (1) a large language model (LLM)-based dialogue understanding module to analyze conversational flow, coupled with (2) visual saliency prediction to identify meaningful scene elements and camera shots therefrom. We then formulate cinematic video editing as an energy minimization problem over shot selection, where cinematic constraints determine shot choices, transitions, and continuity. EditIQ synthesizes an aesthetically and visually compelling representation of the original narrative while maintaining cinematic coherence and a smooth viewing experience. Efficacy of EditIQ against competing baselines is demonstrated via a psychophysical study involving twenty participants on the BBC Old School dataset plus eleven theatre performance videos. Video samples from EditIQ can be found at https://editiq-ave.github.io/.

Figures

Figures reproduced from arXiv: 2502.02172 by the authors.

Figure 1
Figure 1. We present EditIQ, an automated video editing pipeline based on dialogue understanding using LLMs and visual understanding via video saliency. First row presents original video frames input to the pipeline, which generates multiple rushes (depicted in the next two rows). The speaker is denoted by a green arrow in the original frame and the transcript below 3rd row. LLMs are employed to analyze the scene’s narrative,… view at source ↗
Figure 2
Figure 2. EditIQ Pipeline: This fully automated pipeline takes input in the form of video and face crops + IDs and outputs the completely edited video. The various parts of the pipeline are shown in the figure, with each step operating on the outputs of the previous ones. elements, assuming that actor tracks are available. In contrast, our approach eliminates reliance on gaze data and introduces a fully automated editing pipe… view at source ↗
Figure 3
Figure 3. Dialogue understanding module to get Contextual Potential from LLM for different shots based on the transcript of a [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Saliency potential of different single-order shots for two frames in a theatre video (potential value is shown along [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: User Study Evaluation: Bar plots denoting mean user ratings for the different editing methodologies across four [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Here, we compare our modified Saliency Prediction model with state-of-the-art ViNet Model [ [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 59 canonical work pages

  1. [1]

    Sudheer Achary, Rohit Girmaji, Adhiraj Anil Deshmukh, and Vineet Gandhi

    work page

  2. [2]

    Nir Aharon, Roy Orfaig, and Ben-Zion Bobrovsky. 2022. BoT-SORT: Robust Associations Multi-Pedestrian Tracking. arXiv preprint arXiv:2206.14651 (2022)

    arXiv 2022

  3. [3]

    Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. https: //assets.anthropic.com/m/61e7d27f8c8f5919/original/Claude-3-Model-Card.pdf

    work page 2024

  4. [4]

    Ido Arev, Hyun Soo Park, Yaser Sheikh, Jessica Hodgins, and Ariel Shamir. 2014. Automatic editing of footage from multiple social cameras. ACM Transactions on Graphics (TOG) 33, 4 (2014), 81

    work page 2014

  5. [5]

    Daniel Arijon. 1976. Grammar of the film language. (1976)

    work page 1976

  6. [6]

    Max Bain, Jaesung Huh, Tengda Han, and Andrew Zisserman. 2023. WhisperX: Time-Accurate Speech Transcription of Long-Form Audio. INTERSPEECH 2023 (2023)

    work page 2023

  7. [7]

    Jinkun Cao, Jiangmiao Pang, Xinshuo Weng, Rawal Khirodkar, and Kris Kitani

    work page

  8. [8]

    Peter Carr, Michael Mistry, and Iain Matthews. 2013. Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording. In Proceedings of the 21st ACM international conference on Multimedia . 193–202

    work page 2013

Show all 68 references
  1. [9]

    Qinyao Chang and Shiping Zhu. 2021. Temporal-spatial feature pyramid for video saliency detection. arXiv preprint arXiv:2105.04213 (2021)

    2021 arXiv

  2. [10]

    Christine Chen, Oliver Wang, Simon Heinzle, Peter Carr, Aljoscha Smolic, and Markus H. Gross. 2013. Computational sports broadcasting: Automated director assistance for live sports. In Proceedings of the 2013 IEEE International Conference on Multimedia and Expo, ICME 2013, San...

    2013

  3. [11]

    Fan Chen and Christophe De Vleeschouwer. 2010. Personalized production of basketball videos from multi-sensored data under limited display resolution. Computer Vision and Image Understanding 114, 6 (2010), 667–680

    2010

  4. [12]

    Jianhui Chen, Lili Meng, and James J Little. 2018. Camera Selection for Broadcast- ing Soccer Games. In 2018 IEEE Winter Conference on Applications of Computer Vision (W ACV). IEEE, 427–435

    2018

  5. [13]

    David B Christianson, Sean E Anderson, Li-wei He, David H Salesin, Daniel S Weld, and Michael F Cohen. 1996. Declarative camera control for automatic cinematography. In AAAI/IAAI, Vol. 1. 148–155

    1996

  6. [14]

    Antoine Coutrot and Nathalie Guyader. 2015. An efficient audiovisual saliency model to predict eye positions when looking at conversations. In 2015 23rd European Signal Processing Conference (EUSIPCO) . IEEE, 1531–1535

    2015

  7. [15]

    James Cutting and Ayse Candan Simsek. 2015. Shot Durations, Shot Classes, and the Increased Pace of Popular Movies. Projections 9 (12 2015), 40–52. https: //doi.org/10.3167/proj.2015.090204

    2015

  8. [16]

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slow- fast networks for video recognition. In International Conference on Computer Vision (ICCV). 6202–6211

    2019

  9. [17]

    Vamsidhar Reddy Gaddam, Ragnhild Eg, Ragnar Langseth, Carsten Griwodz, and Pål Halvorsen. 2015. The Cameraman Operating My Virtual Camera is Artificial: Can the Machine Be as Good as a Human? ACM Trans. Multimedia Comput. Commun. Appl. 11, 4, Article 56 (June 2015), 20 pages

    2015

  10. [18]

    Quentin Galvane, Rémi Ronfard, Christophe Lino, and Marc Christie. 2015. Con- tinuity editing for 3D animation. In Twenty-Ninth AAAI Conference on Artificial Intelligence

    2015

  11. [19]

    Vineet Gandhi and Rémi Ronfard. 2015. A computational framework for vertical video editing. In 4th Workshop on Intelligent Camera Control, Cinematography and Editing. Eurographics Association, 31–37

    2015

  12. [20]

    Vineet Gandhi, Remi Ronfard, and Michael Gleicher. 2014. Multi-clip video editing from a single viewpoint. In Proceedings of the 11th European Conference on Visual Media Production (London, United Kingdom) (CVMP ’14). Association for Computing Machinery, New York, NY, USA, Art...

    2014

  13. [21]

    Matthias Grundmann, Vivek Kwatra, and Irfan Essa. 2011. Auto-directed video stabilization with robust L1 optimal camera paths. In CVPR 2011. 225–232. https: //doi.org/10.1109/CVPR.2011.5995525

    2011

  14. [22]

    Chunhui Gu, Chen Sun, David A Ross, Carl Vondrick, Caroline Pantofaru, Yeqing Li, Sudheendra Vijayanarasimhan, George Toderici, Susanna Ricco, Rahul Suk- thankar, et al. 2018. Ava: A video dataset of spatio-temporally localized atomic visual actions. In Proceedings of the IEEE...

    2018

  15. [23]

    Li-wei He, Michael F Cohen, and David H Salesin. 1996. The virtual cinematog- rapher: a paradigm for automatic real-time camera control and directing. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 217–224

    1996

  16. [24]

    Rachel Heck, Michael Wallick, and Michael Gleicher. 2007. Virtual videography. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 3, 1 (2007), 4–es

    2007

  17. [25]

    Yani Ioannou, Duncan Robertson, Roberto Cipolla, and Antonio Criminisi. 2017. Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups. In Con- ference on Computer Vision and Pattern Recognition (CVPR)

    2017

  18. [26]

    Eakta Jain, Yaser Sheikh, Ariel Shamir, and Jessica Hodgins. 2015. Gaze-Driven Video Re-Editing. ACM Trans. Graph. 34, 2, Article 21 (March 2015), 12 pages. https://doi.org/10.1145/2699644

    2015 doi

  19. [27]

    Samyak Jain, Pradeep Yarlagadda, Shreyank Jyoti, Shyamgopal Karthik, Ra- manathan Subramanian, and Vineet Gandhi. 2021. Vinet: Pushing the limits of visual modality for audio-visual saliency prediction. In IROS. IEEE, 3520–3527

    2021

  20. [28]

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu. 2023. Ultralytics YOLO. https: //github.com/ultralytics/ultralytics

    2023

  21. [29]

    Stephen Jolly, Graeme Phillipson, and Michael Evans. 2023. Old School: An 8K Multicamera Shoot to Create a Dataset for Computational Cinematography. In Proceedings of the 2023 ACM International Conference on Interactive Media Experiences Workshops (Nantes, France) (IMXw ’23). ...

    2023

  22. [30]

    Yueying Kao, Bowen Pan, Miao Xu, Jiangjing Lyu, Xiangyu Zhu, Yuanzhang Chang, Xiaobo Li, and Zhen Lei. 2023. Toward 3d face reconstruction in per- spective projection: Estimating 6dof face pose from monocular image. IEEE Transactions on Image Processing 32 (2023), 3080–3091

    2023

  23. [31]

    Bruno Korbar, Jaesung Huh, and Andrew Zisserman. 2024. Look, Listen and Recognise: character-aware audio-visual subtitling. (2024)

    2024

  24. [32]

    Bruno Korbar and Andrew Zisserman. 2022. Personalised CLIP or: how to find your vacation videos. In British Machine Vision Conference

    2022

  25. [33]

    Petros Koutras, Athanasios Katsamanis, and Petros Maragos. 2014. Predicting Eyes’ Fixations in Movie Videos: Visual Saliency Experiments on a New Eye- Tracking Database. In Engineering Psychology and Cognitive Ergonomics , Don Harris (Ed.). 183–194

    2014

  26. [34]

    Philipp Krähenbühl, Manuel Lang, Alexander Hornung, and Markus Gross. 2009. A system for retargeting of streaming video. In ACM SIGGRAPH Asia 2009 Papers (Yokohama, Japan) (SIGGRAPH Asia ’09). Association for Computing Machinery, Article 126, 10 pages

    2009

  27. [35]

    Mackenzie Leake, Abe Davis, Anh Truong, and Maneesh Agrawala. 2017. Com- putational video editing for dialogue-driven scenes. ACM Trans. Graph. 36, 4, Article 130 (July 2017), 14 pages. https://doi.org/10.1145/3072959.3073653

    2017

  28. [36]

    Christophe Lino, Mathieu Chollet, Marc Christie, and Rémi Ronfard. 2011. Com- putational model of film editing for interactive storytelling. In International Conference on Interactive Digital Storytelling . Springer, 305–308

    2011

  29. [37]

    Feng Liu and Michael Gleicher. 2006. Video retargeting: automating pan and scan. In Proceedings of the 14th ACM international conference on Multimedia . 241–250

    2006

  30. [38]

    Yufan Liu, Minglang Qiao, Mai Xu, Bing Li, Weiming Hu, and Ali Borji. 2020. Learning to predict salient faces: A novel visual-audio saliency model. InEuropean Conference on Computer Vision (ECCV) . 413–429. EditIQ: Automated Cinematic Editing IUI ’25, March 24–27, 2025, Caglia...

    2020

  31. [39]

    Billal Merabti, Marc Christie, and Kadi Bouatouch. 2016. A Virtual Director Using Hidden Markov Models. In Computer Graphics Forum , Vol. 35. Wiley Online Library, 51–67

    2016

  32. [40]

    Kyle Min and Jason J Corso. 2019. Tased-net: Temporally-aggregating spatial encoder-decoder network for video saliency detection. InInternational Conference on Computer Vision (ICCV) . 2394–2403

    2019

  33. [41]

    K. L. Bhanu Moorthy, Moneish Kumar, Ramanathan Subramanian, and Vineet Gandhi. 2020. GAZED- Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20) . ACM. https://doi.org/...

    2020

  34. [42]

    Junting Pan, Siyu Chen, Mike Zheng Shou, Yu Liu, Jing Shao, and Hongsheng Li

  35. [43]

    Junting Pan, Siyu Chen, Mike Zheng Shou, Yu Liu, Jing Shao, and Hongsheng Li. 2021. Actor-context-actor relation network for spatio-temporal action lo- calization. In Conference on Computer Vision and Pattern Recognition (CVPR) . 464–474

    2021

  36. [44]

    Yingwei Pan, Yue Chen, Qian Bao, Ning Zhang, Ting Yao, Jingen Liu, and Tao Mei. 2021. Smart director: An event-driven directing system for live broadcasting. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 17, 4 (2021), 1–18

    2021

  37. [45]

    Hyun Park, Eakta Jain, and Yaser Sheikh. 2012. 3D Social Saliency from Head-mounted Cameras. In Advances in Neural Information Processing Systems , F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (Eds.), Vol. 25. Cur- ran Associates, Inc. https://proceedings.neurips.cc...

    2012

  38. [46]

    Minglang Qiao, Yufan Liu, Mai Xu, Xin Deng, Bing Li, Weiming Hu, and Ali Borji. 2023. Joint Learning of Audio-Visual Saliency Prediction and Sound Source Localization on Multi-face Videos. 132 (2023), 2003–2025

    2023

  39. [47]

    Kranthi Kumar Rachavarapu, Moneish Kumar, Vineet Gandhi, and Ramanathan Subramanian. 2018. Watch to edit: Video retargeting using gaze. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 205–215

    2018

  40. [48]

    Abhishek Ranjan, Jeremy Birnholtz, and Ravin Balakrishnan. 2008. Improving meeting capture by applying television production principles with audio and motion detection. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). A...

    2008

  41. [49]

    Ralf Schäfer, Peter Kauff, and Christian Weissig. 2010. Ultra high resolution video production and display as basis of a format agnostic production system. In Proceedings of International Broadcast Conference (IBC 2010) , Vol. 1

    2010

  42. [50]

    Yu-Chuan Su, Dinesh Jayaraman, and Kristen Grauman. 2016. Pano2vid: Auto- matic cinematography for watching 360 videos. In Asian Conference on Computer Vision. Springer, 154–171

    2016

  43. [51]

    Kawshik Sundar, Shubham Toshniwal, Makarand Tapaswi, and Vineet Gandhi

  44. [52]

    Chengzhou Tang, Oliver Wang, Feng Liu, and Ping Tan. 2019. Joint stabilization and direction of 360 videos. ACM Transactions on Graphics (TOG) 38, 2 (2019), 1–13

    2019

  45. [53]

    Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, and Haizhou Li. 2021. Is someone speaking? exploring long-term temporal features for audio-visual active speaker detection. InProceedings of the 29th ACM international conference on multimedia. 3927–3935

    2021

  46. [54]

    Antigoni Tsiami, Petros Koutras, and Petros Maragos. 2020. Stavis: Spatio- temporal audiovisual saliency network. In Conference on Computer Vision and Pattern Recognition (CVPR). 4766–4776

    2020

  47. [55]

    In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

    Major Entity Identification: A Generalizable Alternative to Coreference Resolution. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 11679–11695

    2024

  48. [56]

    Krishnapriya Vishnubhotla, Adam Hammond, Graeme Hirst, and Saif M Moham- mad. 2024. The Emotion Dynamics of Literary Novels. ACL (2024)

    2024

  49. [57]

    Jinjun Wang, Changsheng Xu, Engsiong Chng, Hanqing Lu, and Qi Tian. 2008. Automatic composition of broadcast sports video. Multimedia Systems 14, 4 (2008), 179–193

    2008

  50. [58]

    Wenguan Wang, Jianbing Shen, Jianwen Xie, Ming-Ming Cheng, Haibin Ling, and Ali Borji. 2019. Revisiting video saliency prediction in the deep learning era. TPAMI 43, 1 (2019), 220–237

    2019

  51. [59]

    Rejin Varghese and Sambath M. 2024. YOLOv8: A Novel Object Detection Algo- rithm with Enhanced Performance and Robustness. In 2024 International Confer- ence on Advances in Data Engineering and Intelligent Computing Systems (ADICS) . 1–6. https://doi.org/10.1109/ADICS58448.202...

    2024

  52. [60]

    Saining Xie, Chen Sun, Jonathan Huang, Zhuowen Tu, and Kevin Murphy. 2018. Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. In European Conference on Computer Vision (ECCV)

    2018

  53. [61]

    Junwen Xiong, Ganglai Wang, Peng Zhang, Wei Huang, Yufei Zha, and Guangtao Zhai. 2023. CASP-Net: Rethinking video saliency prediction from an audio-visual consistency perceptual perspective. In Conference on Computer Vision and Pattern Recognition (CVPR). 6441–6450

    2023

  54. [62]

    Cha Zhang, Yong Rui, Jim Crawford, and Li-Wei He. 2008. An automated end-to- end lecture capture and broadcasting system. ACM Transactions on multimedia computing, communications, and applications (TOMM) 4, 1 (2008), 1–23

    2008

  55. [63]

    Yu-Shuen Wang, Hui-Chih Lin, Olga Sorkine, and Tong-Yee Lee. 2010. Motion- based video retargeting with optimized crop-and-warp. In ACM SIGGRAPH 2010 papers. 1–9

    2010

  56. [64]

    Zhihan Zhou, Xue Gu, Yujie Zhao, and Hao Xu. 2024. POP-CEE: Position-oriented Prompt-tuning Model for Causal Emotion Entailment. In Findings of the Associa- tion for Computational Linguistics ACL 2024 . 4199–4210. IUI ’25, March 24–27, 2025, Cagliari, Italy Rohit Girmaji, Bhav...

    2024

  57. [67]

    Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. 2018. Shufflenet: An extremely efficient convolutional neural network for mobile devices. InConference on Computer Vision and Pattern Recognition (CVPR)

    2018

  58. [2021]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Actor-context-actor relation network for spatio-temporal action localiza- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 464–474

  59. [2023]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Observation-centric sort: Rethinking sort for robust multi-object track- ing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9686–9696

  60. [2024]

    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Real Time GAZED: Online Shot Selection and Editing of Virtual Cameras from Wide-Angle Monocular Video Recordings. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision . 4108–4116

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.