Pith. sign in

REVIEW 4 major objections 6 minor 64 references

EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Periodic reference resets and recursive mask tracking make surgical pulse magnification robust.

desk verdict A competent, well-written engineering extension of FlowMag for endoscopic vascular motion magnification, but its central accuracy claim rests on a circular metric and needs independent verification. read the letter →

arxiv 2507.15292 v4 pith:QFNV6BS7 submitted 2025-07-21 eess.IV cs.AIcs.CV

An Wang , Rulin Zhou , Mengya Xu , Yiru Ye , Longfei Gou , Yiting Chang , Hao Chen , Chwee Ming Lim
show 2 more authors
Jiankun Wang Hongliang Ren
This is my paper
classification eess.IVcs.AIcs.CV
keywords vascularmotionmagnificationendoscopicsurgeryLagrangianmask-conditionedvideoeditingperiodicreferenceresettinghierarchicaltissue-awareobjecttrackingEndoVMM24
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EndoControlMag aims to make the faint pulsing of blood vessels visible in endoscopic surgical video without training a new network. The paper claims that two additions to mask-conditioned Lagrangian motion magnification achieve this: periodically resetting the reference frame so optical-flow errors cannot accumulate, and recursively tracking the vessel mask while softening the magnification strength into surrounding tissue. This matters because surgeons must infer vascular location and perfusion from subtle motion cues amid smoke, tool occlusion, camera shifts, and tissue deformation, conditions where existing global or static-mask methods drift or produce boundary artifacts. On the authors' four-surgery EndoVMM24 benchmark, both quantitative metrics and expert surgeon ratings favor EndoControlMag over prior methods.

What carries the argument

The load-bearing mechanism is the combination of two modules. Periodic Reference Resetting (PRR) splits a video into overlapping clips of length $N=4$, using the first frame of each clip as the reference for Lagrangian magnification, so the temporal distance from any frame to its reference is bounded and cumulative optical-flow error stays $O(N)$ instead of $O(T)$. Hierarchical Tissue-aware Magnification (HTM) builds the magnification map $M_t$: 1 inside the tracked vessel mask, a softened weight $W_t$ in the dilated outer region, and 0 outside; the outer weight is either normalized flow magnitude or $\exp(-\beta \cdot d)$ from the vessel boundary. The recursive inner-mask tracking keeps the magnified region aligned to the moving vessel, while the softened outer ring prevents hard boundaries between magnified and unmagnified tissue.

What would settle it

Take a clip in which a vessel is fully hidden behind an instrument for about three seconds and then reappears; annotate its true mask in the frames before and after. If EndoControlMag's output after reappearance shows the magnified pulsation centered more than one vessel radius away from the annotated vessel, or if the magnification map is visibly frozen at the pre-occlusion position, then the tracking assumption fails in precisely the occlusion regime the paper claims to handle.

Watch

Extended reading notes

Core claim

The central claim is that EndoControlMag robustly outperforms existing video motion magnification methods for endoscopic vascular visualization. It is a training-free Lagrangian pipeline: optical flow from a periodically reset reference frame is scaled by a factor $\alpha$ and backward-warped, with the scale modulated by a spatially varying magnification mask. The mask comes from a hierarchy: an inner binary mask tracks the target vessel through a pretrained long-term point tracker, and an outer ring around the vessel is softened either by normalized optical-flow magnitude (motion-based) or by exponential decay with distance (distance-based). The paper reports that this design yields lower motion and magnification errors, higher SSIM/PSNR/MUSIQ scores, and better surgeon ratings than FlowMag and other baselines, especially under occlusions, view changes, vessel deformation, and tool disturbance.

Load-bearing premise

The design depends on the vessel mask tracker staying locked onto the target while instruments, smoke, and camera motion pass through the scene; if the tracker loses the vessel for longer than about two seconds or during a very fast camera move, the magnified region no longer points at the right structure and a surgeon must reinitialize the mask by hand.

Editorial extensions

If this is right

  • If the claim holds, a surgeon can select a vessel once and get a pulsing, magnified view of it that stays aligned as the camera moves and instruments pass in front of it, without needing retraining on surgical data.
  • The PRR clip length $N=4$ implies that error growth is bounded within a short window, so the method should degrade gracefully on long procedures where a fixed-reference baseline would drift.
  • The dual softening modes give users a choice: motion-based for deformation-heavy scenes, and distance-based for smoke or instrument occlusion where optical flow is unreliable.
  • The EndoVMM24 dataset provides a reusable benchmark with four surgery types and four challenge categories, so future methods can be compared on the same hard conditions.
  • Because the method is training-free, it can be combined with improved off-the-shelf optical flow or tracking models as those improve, without changing the framework.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One step the authors leave implicit is automating the choice between motion-based and distance-based softening by monitoring optical-flow confidence or smoke indicators, which would remove the manual pre-selection and make the system closer to plug-and-play in the operating room.
  • A testable extension is to read physiological parameters such as pulse rate and relative pulse amplitude from the magnified pulsation signal; if the amplification preserves phase, the enhanced video could serve as a non-contact hemodynamic estimate rather than only a visual aid.
  • The same PRR-plus-tracked-mask recipe should transfer to other deformable, occluded scenes, such as laparoscopic assessment of bowel motility or fetal ultrasound, where subtle periodic motion matters and static references fail.
  • If tracker reliability is the bottleneck, replacing the current tracking model with one adapted to surgical appearance would likely improve robustness more than any other component of the pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes EndoControlMag, a training-free Lagrangian framework for mask-conditioned vascular motion magnification in endoscopic surgery. It introduces Periodic Reference Resetting (PRR) to bound optical-flow error accumulation by slicing videos into short overlapping clips, and a Hierarchical Tissue-aware Magnification (HTM) module that recursively tracks a vessel mask and applies either motion-based or distance-based softening to the surrounding tissue. The method is evaluated on a new EndoVMM24 dataset spanning four surgery types and four challenge categories, using image-quality metrics (SSIM, PSNR, MUSIQ), magnification-accuracy metrics (Emotion, Emag), a surgeon scoring study, and ablations.

Significance. If validated, the framework addresses a real clinical need: making subtle vascular pulsations visible during endoscopic procedures while preserving the surrounding tissue. The manuscript is clearly written, the algorithm is fully specified in pseudocode, and the authors commit to releasing code, data, and video results, which is commendable and will aid reproducibility. The use of off-the-shelf tracking and flow models without fine-tuning is a practical strength, and the ablation study on mask dilation strategies is informative. The main scientific value lies in demonstrating that reference resetting and adaptive mask softening improve robustness in dynamic surgical scenes; however, the central quantitative claim of superior magnification accuracy currently rests on metrics that are computed with the same optical flow estimator used inside the method, which creates a self-consistency risk that must be resolved before the accuracy claim can be accepted as independent evidence.

major comments (4)
  1. [§4.2.2, Eqs. (11)-(12), Tables 3 and 5] The magnification-accuracy metrics Emotion and Emag are computed with RAFT, which is the same optical flow estimator used by the base FlowMag model inside fVMM and also by the motion-based softening in Eq. (7). Since the magnified frame is synthesized so that RAFT flow from the reference to the magnified frame approximates alpha times the reference-to-current RAFT flow inside the mask, low Emotion/Emag partly reflect that the method directly optimizes what the metric measures. The reported reductions (e.g., Emotion 1.151→0.999, Emag 1.732→1.033 at ×2 in Table 3) may therefore be self-consistency artifacts rather than evidence of true motion fidelity. Please re-evaluate with an independent flow estimator (e.g., a different architecture such as GMFlow or a learned dense tracking method) or on synthetic sequences with known ground-truth motion, so that the accuracy claim is not circular.
  2. [§5.2, Fig. 5] The Hard Set evaluation compares EndoControlMag only against FlowMag, not against the other baselines (EVM, DMM, MDL-VMM, STB-VMM, Axial-VMM) that were included on the Easy Set. The abstract and introduction claim that the method 'significantly outperforms existing methods' and maintains 'robustness across challenging surgical conditions'; however, the evidence for robustness in occlusion, view change, vessel deformation, and tool disturbance is based solely on comparison with a single baseline. Please include at least the strongest learning-based baselines on the Hard Set, or explicitly justify their exclusion beyond stating that global methods would produce 'intolerable visual distortions.'
  3. [§5.1, §5.2, Abstract] The phrase 'significantly outperforms' is used repeatedly, but the quantitative tables (Table 2, Table 3) and Fig. 5 report only means and standard deviations with no statistical tests. With only 4 videos in the Easy Set and 3-8 clips per Hard Set category, the observed differences (e.g., SSIM 0.97 vs 0.96) may not be statistically reliable. Provide paired significance tests (e.g., paired t-test or Wilcoxon signed-rank test) across videos, or temper the language to 'consistently outperforms' without the statistical connotation.
  4. [§5.3, Fig. 6] The surgeon evaluation is based on only 3 surgeons, 8 videos, and a binary per-item rubric. The reported score gaps (9.75 vs 8.19 on Easy Set; 9.19/9.22 vs 7.61 on Hard Set) are presented without any measure of inter-rater agreement or significance. Given the small number of raters and items, the standard error of these averages is large, and the claim of 'strong agreement' is not supported by the data shown. Please report per-item agreement statistics (e.g., Fleiss' kappa or a simple percentage agreement) and a statistical comparison (e.g., a permutation test) or explicitly state the pilot nature of the study.
minor comments (6)
  1. [Table 2 header] The PSNR column is labeled 'PNSR' in the header; please correct the typo.
  2. [Table 5, Vessel-adaptive row] The Emotion value at ×16 is printed as '15.152±8.394' and at ×32 as '31.203±17793', which appears to be missing a decimal point (likely 31.203±17.793). Please check all entries for formatting consistency.
  3. [§5.4.1, 'Physiological Relevance' paragraph] The link between N=4 and cardiac pulsation frequency is a post-hoc justification; it is not tested independently of the ablation. Consider framing it as a hypothesis rather than a confirmed physiological explanation.
  4. [§6.2] The limitations section honestly acknowledges that the MFT tracker can lose the vessel during occlusions longer than about 2 seconds or very fast camera movements, requiring manual re-initialization. This is a useful caveat, but it should be reflected in the abstract and conclusion, where the claim of 'robustness across challenging surgical conditions' is currently stated without this qualification.
  5. [§3.1.2] The term 'training-free' may be misleading because the framework uses pretrained models (RAFT, MFT, and the FlowMag base model). The paper clarifies in §6.2 that no fine-tuning on surgical data is performed; consider defining 'training-free' in the introduction to avoid confusion.
  6. [Fig. 6(a)] The description of the scoring scheme is confusing: a 10-item binary (Yes/No) rubric is used, but the text says each item is scored per video and per magnification factor, and the final score is an average. Please clarify how the 0-10 scale is constructed so the reader can interpret the scores (e.g., is it the average number of 'Yes' answers across all 30 item-video-factor combinations?).

Circularity Check

1 steps flagged · score 6.0 of 10

Magnification-accuracy metrics replicate the FlowMag optimization objective, so the reported accuracy gains are self-consistency artifacts rather than independent verification.

  1. self definitional [Sec. 3.1.2 Eq. (3); Sec. 4.2.2 Eqs. (11)-(12)]
    "˜It = fVMM(Ir, It, PE(α)⊙ Mt), (3) ... Emotion =∥O(Ir, It)−α·O (Ir, ˜It)∥1, (11) ... For both accuracy metrics, we use RAFT [40], a state-of-the-art optical flow estimator, to compute the flow fields."

    EndoControlMag's fVMM is instantiated as FlowMag (Sec. 3.1.2), and the FlowMag method (as cited, 'backpropagating through optical flow') synthesizes the magnified frame by enforcing that the flow from reference to magnified frame equals α times the flow from reference to original frame. Eqs. (11)-(12) measure exactly that equality, and the paper computes both the metric and the motion-based mask weights with the same RAFT estimator (Eq. (7), Sec. 4.2.2). Hence low Emotion/Emag are residuals of the optimization the framework itself performs, not independent evidence of magnification accuracy. The large reported margin over baselines that do not optimize this objective is achieved by construction, and the comparison with FlowMag compares two runs of the same objective.

full rationale

The PRR and HTM designs are self-contained: clip-based reference resetting, recursive point tracking, and the two mask-softening rules are implemented from stated equations and do not secretly assume the results. The visual-quality claim is supported by standard SSIM/PSNR/MUSIQ metrics and by a blinded surgeon scoring protocol; those are independent of the method's internals and justify the robustness and quality conclusions. However, the magnification-accuracy claim is not independently verified. The paper adopts the FlowMag metrics (Eqs. 11-12) and then builds its magnified frame with FlowMag, whose optimization objective is the same alpha-flow-consistency relation; using RAFT for both the metric and the motion-based mask further couples the measurement to the method. Thus the 'significant outperformance in magnification accuracy' is in part a restatement of the optimization objective. This is a partial, not total, circularity: the PRR/HTM contributions, the new dataset, and the quality and clinical evaluations remain meaningful. Score 6.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim relies on external models (RAFT, MFT, FlowMag) and hand-set constants (N, gamma, beta). These are not independently verified on surgical data in this paper, and the paper's own limitations section acknowledges possible failure modes.

free parameters (3)
  • clip length N = 4
    Chosen from ablation on Easy Set (Table 4); differences across N are small, and the physiological justification is post hoc.
  • dilation scaling gamma = 1/15
    Scaling factor for adaptive dilation radius in Eq. 6, chosen by hand without sensitivity analysis.
  • decay rate beta = 1
    Decay rate in distance-based softening (Eq. 9), set to approximate viscoelastic damping; no tuning study.
assumptions (5)
  • standard math Lagrangian magnification via backward warping (Eq. 2) produces faithful magnified frames
    This is the standard formulation from Liu et al. and FlowMag.
  • domain assumption RAFT optical flow accurately estimates tissue displacement in endoscopic video
    Used in Eq. 7-8 for motion-based softening and in the evaluation metrics; no validation on surgical data.
  • domain assumption MFT point tracker reliably propagates the vessel mask through occlusions and view changes
    The paper's own limitations section acknowledges tracker failure under extreme occlusion and fast motion.
  • ad hoc to paper Exponential decay with beta=1 models biomechanical force attenuation in tissue
    Proposed without experimental support; the viscoelastic analogy is qualitative.
  • domain assumption FlowMag is an accurate base motion magnification model
    The framework builds on fVMM = FlowMag; if FlowMag's warping is flawed, the whole pipeline is affected.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control." pith.science (2026). https://pith.science/paper/QFNV6BS7

@misc{pith2026250715292,
  author       = {Pith},
  title        = {Pith review of: EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QFNV6BS7}},
  note         = {Machine review of arXiv:2507.15292}
}
read the original abstract

Visualizing subtle vascular motions in endoscopic surgery is crucial for surgical precision and decision-making, yet remains challenging due to the complex and dynamic nature of surgical scenes. To address this, we introduce EndoControlMag, a training-free, Lagrangian-based framework with mask-conditioned vascular motion magnification tailored to endoscopic environments. Our approach features two key modules: a Periodic Reference Resetting (PRR) scheme that divides videos into short overlapping clips with dynamically updated reference frames to prevent error accumulation while maintaining temporal coherence, and a Hierarchical Tissue-aware Magnification (HTM) framework with dual-mode mask dilation. HTM first tracks vessel cores using a pretrained visual tracking model to maintain accurate localization despite occlusions and view changes. It then applies one of two adaptive softening strategies to surrounding tissues: motion-based softening that modulates magnification strength proportional to observed tissue displacement, or distance-based exponential decay that simulates biomechanical force attenuation. This dual-mode approach accommodates diverse surgical scenarios-motion-based softening excels with complex tissue deformations while distance-based softening provides stability during unreliable optical flow conditions. We evaluate EndoControlMag on our EndoVMM24 dataset spanning four different surgery types and various challenging scenarios, including occlusions, instrument disturbance, view changes, and vessel deformations. Quantitative metrics, visual assessments, and expert surgeon evaluations demonstrate that EndoControlMag significantly outperforms existing methods in both magnification accuracy and visual quality while maintaining robustness across challenging surgical conditions. The code, dataset, and video results are available at https://szupc.github.io/EndoControlMag/.

Figures

Figures reproduced from arXiv: 2507.15292 by the authors.

Figure 1
Figure 1. Comparison of motion magnification approaches for transform￾ing an input frame It into a magnified frame ˜It with a factor α. (a) Conven￾tional methods apply global magnification uniformly to the entire image with￾out region-specific control. (b) FlowMag [31] introduces mask-conditioned magnification but relies on a fixed reference frame I0 and static mask M0 throughout the sequence, leading to mask misalignment and… view at source ↗
Figure 2
Figure 2. Our EndoControlMag for vascular motion magnification in endoscopic surgical video. (a) The overall magnification pipeline, where vascular motion in the current frame It is magnified by the video motion magnification model fVMM to produce the magnified frame ˜It . This process utilizes a corresponding reference frame Ir and a hierarchical magnification mask Mr , which is positionally encoded by the magnification fact… view at source ↗
Figure 3
Figure 3. The composition of our EndoVMM24 dataset spanning multiple surgical specialties and challenging clinical scenarios. (a) The Easy Set contains representative clips from four distinct surgical procedures (Cholecystectomy, Prostatectomy, Gastric Bypass, and Laparoscopic Distal Gastrectomy), each featuring a specific vascular structure with minimal movement relative to the camera. The vessel masks (purple overlay) indic… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of our EndoControlMag with baseline methods on a representative surgical video with magnification factor ×8. For each method, we highlight the vascular region (blue box) and non-vascular region (red box) with close-up views shown at the top of ea…
Figure 5
Figure 5. Figure 5: Quantitative performance comparison of our EndoControlMag against FlowMag [31] on the Hard Set across four surgical challenge categories. Each column represents a specific challenge: Tool Disturbance, Vessel Deformation, View Change, and Occlusion. Results are organize…
Figure 6
Figure 6. Figure 6: Clinical evaluation by expert surgeons. (a) Evaluation methodology: Three experienced surgeons assessed magnification quality using a standardized 10-item scoring rubric covering vascular and non-vascular regions, boundary clarity, consistency, noise levels, pulsation …
Figure 7
Figure 7. Figure 7: Visual comparison of mask dilation strategies for vascular motion magnification. (a-c) Fixed-radius dilation with uniform weights (2.5, 10, and 25 pixels, respectively) creates abrupt boundaries between magnified and un￾magnified regions. (d) Our distance-based softeni…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 53 canonical work pages

  1. [1]

    An overview of eulerian video motion magnification methods

    Ahmed, A.M., Abdelrazek, M., Aryal, S., Nguyen, T.T., 2023. An overview of eulerian video motion magnification methods. Computers & Graphics

    work page 2023

  2. [2]

    Pixel-wise recognition for holistic surgical scene understanding

    Ayobi, N., Rodr ´ıguez, S., P ´erez, A., Hern ´andez, I., Aparicio, N., Des- sevres, E., Pe˜na, S., Santander, J., Caicedo, J.I., Fern´andez, N., Arbel´aez, P., 2024. Pixel-wise recognition for holistic surgical scene understanding. arXiv preprint arXiv:2401.11174

    arXiv 2024

  3. [3]

    Learning-based axial video motion magnification, in: European Confer- ence on Computer Vision, Springer

    Byung-Ki, K., Hyun-Bin, O., Jun-Seong, K., Ha, H., Oh, T.H., 2025. Learning-based axial video motion magnification, in: European Confer- ence on Computer Vision, Springer. pp. 179–195

    work page 2025

  4. [4]

    Surgnet: Self- supervised pretraining with semantic consistency for vessel and instru- ment segmentation in surgical images

    Chen, J., Li, M., Han, H., Zhao, Z., Chen, X., 2023a. Surgnet: Self- supervised pretraining with semantic consistency for vessel and instru- ment segmentation in surgical images. IEEE Transactions on Medical Imaging

    work page

  5. [5]

    Control-a-video: Controllable text-to-video generation with diffusion models

    Chen, W., Ji, Y ., Wu, J., Wu, H., Xie, P., Li, J., Xia, X., Xiao, X., Lin, L., 2023b. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv preprint arXiv:2305.13840

    arXiv

  6. [6]

    Xmem: Long-term video object seg- mentation with an atkinson-shiffrin memory model, in: European Confer- ence on Computer Vision, Springer

    Cheng, H.K., Schwing, A.G., 2022. Xmem: Long-term video object seg- mentation with an atkinson-shiffrin memory model, in: European Confer- ence on Computer Vision, Springer. pp. 640–658. 15

    work page 2022

  7. [7]

    A temporal learning approach to inpainting endoscopic specularities and its e ffect on image correspondence

    Daher, R., Vasconcelos, F., Stoyanov, D., 2023. A temporal learning approach to inpainting endoscopic specularities and its e ffect on image correspondence. Medical Image Analysis 90, 102994

    work page 2023

  8. [8]

    Segstrong-c: Segmenting surgical tools robustly on non-adversarial generated corruptions–an endovis’ 24 challenge

    Ding, H., Lu, T., Zhang, Y ., Liang, R., Shu, H., Seenivasan, L., Long, Y ., Dou, Q., Gao, C., Unberath, M., 2024. Segstrong-c: Segmenting surgical tools robustly on non-adversarial generated corruptions–an endovis’ 24 challenge. arXiv preprint arXiv:2407.11906

    arXiv 2024

Show all 64 references
  1. [9]

    Di ffu- toon: High-resolution editable toon shading via di ffusion models

    Duan, Z., Wang, C., Chen, C., Qian, W., Huang, J., 2024. Di ffu- toon: High-resolution editable toon shading via di ffusion models. arXiv preprint arXiv:2401.16224

    2024 arXiv

  2. [10]

    Video mag- nification in presence of large motions, in: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pp

    Elgharib, M., Hefeeda, M., Durand, F., Freeman, W.T., 2015. Video mag- nification in presence of large motions, in: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pp. 4119–4127

    2015

  3. [11]

    Fan, W., Zheng, Z., Zeng, W., Chen, Y ., Zeng, H.Q., Shi, H., Luo, X.,

  4. [12]

    Lagrangian motion magnification with double sparse optical flow decomposition

    Flotho, P., Heiss, C., Steidl, G., Strauss, D.J., 2023. Lagrangian motion magnification with double sparse optical flow decomposition. Frontiers in Applied Mathematics and Statistics 9, 1164491

    2023

  5. [13]

    Gao, S., Feng, Y ., Yang, L., Liu, X., Zhu, Z., Doermann, D.S., Zhang, B.,

  6. [14]

    Motion prompting: Controlling video genera- tion with motion trajectories

    Geng, D., Herrmann, C., Hur, J., Cole, F., Zhang, S., Pfa ff, T., Lopez- Guevara, T., Doersch, C., Aytar, Y ., Rubinstein, M., Sun, C., Wang, O., Owens, A., Sun, D., 2024. Motion prompting: Controlling video genera- tion with motion trajectories. arXiv preprint arXiv:2412.02700

    2024 arXiv

  7. [15]

    Revisiting learning- based video motion magnification for real-time processing

    Ha, H., Hyun-Bin, O., Jun-Seong, K., Byung-Ki, K., Sung-Bin, K., Tran, L.T., Kim, J.Y ., Bae, S.H., Oh, T.H., 2024. Revisiting learning- based video motion magnification for real-time processing. arXiv preprint arXiv:2403.01898

    2024 arXiv

  8. [16]

    Image quality metrics: Psnr vs

    Hore, A., Ziou, D., 2010. Image quality metrics: Psnr vs. ssim, in: 2010 20th international conference on pattern recognition, IEEE. pp. 2366– 2369

    2010

  9. [17]

    Huang, D., Bi, Y ., Navab, N., Jiang, Z., 2023. Motion magnification in robotic sonography: enabling pulsation-aware artery segmentation, in: 2023 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), IEEE. pp. 6565–6570

    2023

  10. [18]

    Janatka, M., Marcus, H.J., Dorward, N.L., Stoyanov, D., 2020. Surgi- cal video motion magnification with suppression of instrument artefacts, in: Medical Image Computing and Computer Assisted Intervention– MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 20...

    2020

  11. [19]

    Janatka, M., Sridhar, A., Kelly, J., Stoyanov, D., 2018. Higher order of motion magnification for vessel localisation in surgical video, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, ...

    2018

  12. [20]

    Motion Magnification for Surgical Video

    Janatka, M.P., 2022. Motion Magnification for Surgical Video. Ph.D. thesis. UCL (University College London)

    2022

  13. [21]

    Musiq: Multi- scale image quality transformer, in: Proceedings of the IEEE /CVF inter- national conference on computer vision, pp

    Ke, J., Wang, Q., Wang, Y ., Milanfar, P., Yang, F., 2021. Musiq: Multi- scale image quality transformer, in: Proceedings of the IEEE /CVF inter- national conference on computer vision, pp. 5148–5157

    2021

  14. [22]

    Stb-vmm: Swin transformer based video motion magnification

    Lado-Roig ´e, R., P ´erez, M.A., 2023. Stb-vmm: Swin transformer based video motion magnification. Knowledge-Based Systems 269, 110493

    2023

  15. [23]

    Motion magnification

    Liu, C., Torralba, A., Freeman, W.T., Durand, F., Adelson, E.H., 2005. Motion magnification. ACM transactions on graphics (TOG) 24, 519– 526

    2005

  16. [24]

    Liu, Z., Lin, Y ., Cao, Y ., Hu, H., Wei, Y ., Zhang, Z., Lin, S., Guo, B.,

  17. [25]

    Deep video harmonization with color mapping consistency

    Lu, X., Huang, S., Niu, L., Cong, W., Zhang, L., 2022. Deep video harmonization with color mapping consistency. arXiv preprint arXiv:2205.00687

    2022 arXiv

  18. [26]

    Minimally invasive and robotic surgery

    Mack, M.J., 2001. Minimally invasive and robotic surgery. Jama 285, 568–572

    2001

  19. [27]

    10012–10022

    Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE /CVF international conference on computer vision, pp. 10012–10022

  20. [28]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R., 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65, 99–106

    2021

  21. [29]

    Mft: Long-term tracking of every pixel, in: Proceedings of the IEEE /CVF Winter Conference on Applica- tions of Computer Vision, pp

    Neoral, M., ˇSer`ych, J., Matas, J., 2024. Mft: Long-term tracking of every pixel, in: Proceedings of the IEEE /CVF Winter Conference on Applica- tions of Computer Vision, pp. 6837–6847

    2024

  22. [30]

    Motion magnification for endoscopic surgery, in: Medical Imaging 2014: Image- Guided Procedures, Robotic Interventions, and Modeling, SPIE

    McLeod, A.J., Baxter, J.S., de Ribaupierre, S., Peters, T.M., 2014. Motion magnification for endoscopic surgery, in: Medical Imaging 2014: Image- Guided Procedures, Robotic Interventions, and Modeling, SPIE. pp. 81– 88

    2014

  23. [31]

    Self-supervised motion magnifica- tion by backpropagating through optical flow

    Pan, Z., Geng, D., Owens, A., 2024. Self-supervised motion magnifica- tion by backpropagating through optical flow. Advances in Neural Infor- mation Processing Systems 36

    2024

  24. [32]

    Weakly supervised tempo- ral convolutional networks for fine-grained surgical activity recognition

    Ramesh, S., Dall’Alba, D., Gonzalez, C., Yu, T., Mascagni, P., Mutter, D., Marescaux, J., Fiorini, P., Padoy, N., 2023. Weakly supervised tempo- ral convolutional networks for fine-grained surgical activity recognition. IEEE Transactions on Medical Imaging 42, 2592–2602

    2023

  25. [33]

    Learning-based video motion magnification, in: Pro- ceedings of the European Conference on Computer Vision (ECCV), pp

    Oh, T.H., Jaroensri, R., Kim, C., Elgharib, M., Durand, F., Freeman, W.T., Matusik, W., 2018. Learning-based video motion magnification, in: Pro- ceedings of the European Conference on Computer Vision (ECCV), pp. 633–648

    2018

  26. [34]

    High-resolution image synthesis with latent di ffusion models, in: Pro- ceedings of the IEEE /CVF conference on computer vision and pattern recognition, pp

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B., 2022. High-resolution image synthesis with latent di ffusion models, in: Pro- ceedings of the IEEE /CVF conference on computer vision and pattern recognition, pp. 10684–10695

    2022

  27. [35]

    Ro- bust medical instrument segmentation challenge 2019

    Ross, T., Reinke, A., Full, P.M., Wagner, M., Kenngott, H., Apitz, M., Hempe, H., Filimon, D.M., Scholz, P., Tran, T.N., Bruno, P., Arbel´aez, P., Bian, G.B., Bodenstedt, S., Bolmgren, J.L., Bravo-S ´anchez, L., Chen, H.B., Gonz ´alez, C., Guo, D., Halvorsen, P., Heng, P.A., H...

    2020 arXiv

  28. [36]

    Cholec80-cvs: An open dataset with an evaluation of strasberg’s critical view of safety for ai

    R ´ıos, M.S., Molina-Rodriguez, M.A., Londo˜no, D., Guill´en, C.A., Sierra, S., Zapata, F., Giraldo, L.F., 2023. Cholec80-cvs: An open dataset with an evaluation of strasberg’s critical view of safety for ai. Scientific Data 10, 194

    2023

  29. [37]

    Multi domain learning for motion magnification, in: Proceedings of the IEEE /CVF Conference on Com- puter Vision and Pattern Recognition, pp

    Singh, J., Murala, S., Kosuru, G., 2023. Multi domain learning for motion magnification, in: Proceedings of the IEEE /CVF Conference on Com- puter Vision and Pattern Recognition, pp. 13914–13923

    2023

  30. [38]

    Univst: A unified framework for training-free localized video style transfer

    Song, Q., Lin, M., Zhan, W., Yan, S., Cao, L., Ji, R., 2024. Univst: A unified framework for training-free localized video style transfer. arXiv preprint arXiv:2410.20084

    2024

  31. [39]

    Financial and clinical outcomes associated with sur- gical bleeding complications

    Shander, A., 2007. Financial and clinical outcomes associated with sur- gical bleeding complications. Surgery 142, S20–S25

    2007

  32. [40]

    Teed, Z., Deng, J., 2020. Raft: Recurrent all-pairs field transforms for op- tical flow, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, Springer. pp. 402–419

    2020

  33. [41]

    Attention is all you need

    Vaswani, A., 2017. Attention is all you need. Advances in Neural Infor- mation Processing Systems

    2017

  34. [42]

    Bilateral video magnification filter, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Takeda, S., Niwa, K., Isogawa, M., Shimizu, S., Okami, K., Aono, Y ., 2022. Bilateral video magnification filter, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17369–17378

    2022

  35. [43]

    Wang, A., Islam, M., Xu, M., Ren, H., 2022. Rethinking surgical instru- ment segmentation: A background image can be all you need, in: Interna- tional Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 355–364

    2022

  36. [44]

    Curriculum-based aug- mented fourier domain adaptation for robust medical image segmentation

    Wang, A., Islam, M., Xu, M., Ren, H., 2023a. Curriculum-based aug- mented fourier domain adaptation for robust medical image segmentation. IEEE Transactions on Automation Science and Engineering

  37. [45]

    Phase- based video motion processing

    Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T., 2013. Phase- based video motion processing. ACM Transactions on Graphics (ToG) 32, 1–10

    2013

  38. [46]

    Easycontrol: Transfer controlnet to video di ffusion for control- lable generation and interpolation

    Wang, C., Gu, J., Hu, P., Zhao, H., Guo, Y ., Han, J., Xu, H., Liang, X., 2024a. Easycontrol: Transfer controlnet to video di ffusion for control- lable generation and interpolation. arXiv preprint arXiv:2408.13005 . 16

    arXiv

  39. [47]

    Eulermormer: Robust eu- lerian motion magnification via dynamic filtering within transformer, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp

    Wang, F., Guo, D., Li, K., Wang, M., 2024b. Eulermormer: Robust eu- lerian motion magnification via dynamic filtering within transformer, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 5345– 5353

  40. [48]

    Wang, A., Xu, M., Zhang, Y ., Islam, M., Ren, H., 2023b. S 2me: Spatial- spectral mutual teaching and ensemble learning for scribble-supervised polyp segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 35–45

  41. [49]

    Wu, H.Y ., Rubinstein, M., Shih, E., Guttag, J., Durand, F., Freeman, W.,

  42. [50]

    Tune-a-video: One-shot tuning of im- age di ffusion models for text-to-video generation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Wu, J.Z., Ge, Y ., Wang, X., Lei, S.W., Gu, Y ., Shi, Y ., Hsu, W., Shan, Y ., Qie, X., Shou, M.Z., 2023. Tune-a-video: One-shot tuning of im- age di ffusion models for text-to-video generation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7...

    2023

  43. [51]

    Image qual- ity assessment: from error visibility to structural similarity

    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P., 2004. Image qual- ity assessment: from error visibility to structural similarity. IEEE trans- actions on image processing 13, 600–612

    2004

  44. [52]

    Motion-conditioned image animation for video editing

    Yan, W., Brown, A., Abbeel, P., Girdhar, R., Azadi, S., 2023. Motion-conditioned image animation for video editing. arXiv preprint arXiv:2311.18827

    2023 arXiv

  45. [53]

    Magnification and localization of vessels in robotic surgical videos based on accuracy high-order phase-based video magnification

    Yang, Y ., Jiang, Q., 2024. Magnification and localization of vessels in robotic surgical videos based on accuracy high-order phase-based video magnification. Biomedical Signal Processing and Control 96, 106575

    2024

  46. [54]

    Adding conditional control to text-to-image diffusion models, in: Proceedings of the IEEE /CVF Inter- national Conference on Computer Vision, pp

    Zhang, L., Rao, A., Agrawala, M., 2023a. Adding conditional control to text-to-image diffusion models, in: Proceedings of the IEEE /CVF Inter- national Conference on Computer Vision, pp. 3836–3847

  47. [55]

    Tsa2: Temporal segment adaptation and aggregation for video harmonization, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp

    Xiao, Z., Zhu, Y ., Fu, X., Xiong, Z., 2024. Tsa2: Temporal segment adaptation and aggregation for video harmonization, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4136–4145

    2024

  48. [56]

    Controlvideo: Training-free controllable text-to-video generation

    Zhang, Y ., Wei, Y ., Jiang, D., Zhang, X., Zuo, W., Tian, Q., 2023b. Controlvideo: Training-free controllable text-to-video generation. arXiv preprint arXiv:2305.13077

    arXiv

  49. [57]

    Avid: Any-length video inpainting with diffu- sion model, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, pp

    Zhang, Z., Wu, B., Wang, X., Luo, Y ., Zhang, L., Zhao, Y ., Vajda, P., Metaxas, D., Yu, L., 2024. Avid: Any-length video inpainting with diffu- sion model, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, pp. 7162–7172

    2024

  50. [58]

    Localization and local motion magnification of pulsatile regions in endoscopic surgery videos, in: International Conference on Multimedia Modeling, Springer

    Zheng, H., Fan, W., Chen, Y ., Luo, X., 2024a. Localization and local motion magnification of pulsatile regions in endoscopic surgery videos, in: International Conference on Multimedia Modeling, Springer. pp. 141– 154

  51. [59]

    Video acceleration mag- nification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Zhang, Y ., Pintea, S.L., Van Gemert, J.C., 2017. Video acceleration mag- nification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 529–537

    2017

  52. [60]

    Propainter: Improv- ing propagation and transformer for video inpainting, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Zhou, S., Li, C., Chan, K.C., Loy, C.C., 2023. Propainter: Improv- ing propagation and transformer for video inpainting, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10477– 10486. 17

    2023

  53. [63]

    Makima: Tuning-free multi- attribute open-domain video editing via mask-guided attention modula- tion

    Zheng, H., Zhang, W., Lv, Z., Zhong, Y ., Dai, Y ., An, J., Shen, Y ., Li, J., Zhang, D., Tang, S., Zhuang, Y ., 2024b. Makima: Tuning-free multi- attribute open-domain video editing via mask-guided attention modula- tion. arXiv preprint arXiv:2412.19978

    arXiv

  54. [2012]

    ACM transactions on graphics (TOG) 31, 1–8

    Eulerian video magnification for revealing subtle changes in the world. ACM transactions on graphics (TOG) 31, 1–8

  55. [2021]

    IEEE Robotics and Automation Letters 6, 1567– 1573

    Robotically surgical vessel localization using robust hybrid video motion magnification. IEEE Robotics and Automation Letters 6, 1567– 1573

  56. [2022]

    Magformer: Hybrid video motion magnification transformer from eulerian and lagrangian perspectives., in: BMVC, p. 444

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.