Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking

Bing Liu; Hancheng Zhu; Jiaqi Zhao; Kunyang Sun; Rui Yao; Shenglan Li; Yong Zhou; Zhiwen Shao

REVIEW 4 major objections 7 minor 34 references

Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Self-supervised RGB-T tracking can beat supervised baselines by fusing modalities with a dynamic graph and denoising with a temporal diffusion model that treats neighboring frames as distractors.

desk verdict Solid incremental architecture, but the evaluation reporting—especially the LasHeR train/test split—needs fixing before the headline numbers can be believed. read the letter →

arxiv 2505.03507 v1 pith:3KTSVKZA submitted 2025-05-06 cs.CV

Shenglan Li , Rui Yao , Yong Zhou , Hancheng Zhu , Kunyang Sun , Bing Liu , Zhiwen Shao , Jiaqi Zhao This is my paper · ORCID

classification cs.CV

keywords self-supervisedtrackingRGB-Tdynamicgraphfusiontemporaldiffusionpseudo-labelnoisemulti-modalattentionnetworkobject

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a self-supervised RGB-T tracker can match or beat fully supervised approaches by attacking the two failure modes of pseudo-label training: fusion that drifts onto background regions, and confusion caused by objects that look like the target. The proposed GDSTrack builds a dynamic graph over RGB and thermal features, using a similarity-based adjacency matrix to keep attention on coherent object regions, then runs a diffusion model that treats the fused features of a neighboring frame as synthetic 'distractor noise' so the tracker learns to ignore similar objects. On RGBT234, LasHeR, and VTUAV, the method reports higher precision and success rates than the previous best self-supervised tracker, with the largest gain on the UAV benchmark. The practical stakes are high-quality RGB-T tracking without manual bounding-box annotations, which matters for large-scale and always-changing scenes.

What carries the argument

The engine of the paper is the interplay of two modules. The Adjacency Matrix Generator (AMG) builds a dynamic adjacency matrix $A$ from the sum of scaled dot-product similarity and cosine similarity between concatenated RGB and thermal feature sequences, thresholds it, applies softmax, and keeps the Top-K connections symmetrized; this matrix guides a two-layer Graph Attention Network (GAT) that treats every spatial location in both modalities as a graph node, producing first-layer ($f^{l1}$) and fused ($f^{S1}$) features. The Temporal Graph-Informed Diffusion (TGID) module takes $f^{S1}$ of the current frame as the diffusion input $x_0$, injects the neighboring frame's $f^{S1}$ as a non-Gaussian 'distractor' additive term $d_t$ alongside standard Gaussian noise in the forward process, and denoises with a U-Net whose intermediate layers are conditioned on the first-layer graph features via a ConFuse residual-addition module. That conditioning is what prevents information loss when the diffusion model is asked to remove the distractor-like noise.

What would settle it

Run GDSTrack on a controlled video where the target is stationary for many frames (or include a static camera sequence with no object motion) and compare the full model against the same model trained with only Gaussian noise in Eq. (20): if the neighboring-frame distractor term $d_t$ is helping, its benefit should vanish or reverse when there is no positional offset to make it look like a different, similar object. More directly, inspect the noise residual in Eq. (20): when the object does not move, the distractor term equals a shifted copy of the true target features, and the denoiser's learned residual should be measurably different from that learned on moving-object sequences.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that both dominant errors of pseudo-label-supervised RGB-T tracking can be modeled and corrected by a two-module design. The Modality-guided Dynamic Graph Fusion (MDGF) module computes a dynamic adjacency matrix from cross-modal cosine similarity and scaled dot-product attention, then uses it to guide a graph attention network so that fusion concentrates on the object's coherent regions even when the pseudo-label includes background. The Temporal Graph-Informed Diffusion (TGID) module feeds the MDGF output of the current frame into a DDIM denoising process whose added noise combines Gaussian noise with the MDGF output of a neighboring frame; because that neighboring-frame feature is offset from the target by object motion, it acts as a realistic similar-object distractor. The paper reports that GDSTrack surpasses the previous best self-supervised method S2OTFormer by 2.5% precision on RGBT234, 6.1% on LasHeR, and 15.6% on VTUAV, while remaining comparable in speed.

Load-bearing premise

The load-bearing premise is that the fused feature map of the neighboring frame really is a good stand-in for a similar-object distractor: if the tracked object barely moves or the offset regions do not look like a plausible target, the 'distractor noise' injected into the diffusion process is really a corrupted version of the target itself, and the denoiser could learn to trust incorrect noise statistics.

Editorial extensions

If this is right

Under pseudo-label supervision, GDSTrack reports higher precision and success than the previous best self-supervised tracker on RGBT234, LasHeR, and VTUAV, with the largest improvement on the UAV set.
The MDGF module alone improves fusion under noisy pseudo-labels (roughly 2-5 points of precision across datasets compared to the baseline), showing that dynamic adjacency guided by cross-modal similarity recovers object-relevant fusion without ground truth.
The TGID module adds further gains on top of MDGF (up to 2 points of precision on VTUAV), indicating that modeling similar-object noise as temporal distractor noise makes the tracker more robust to interference.
Because the tracker trains with pseudo-labels generated from multi-modal optical flow, it removes the need for manual bounding-box annotations in the training pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The distractor-noise trick is not tied to RGB-T tracking: any self-supervised task with temporal correspondences (e.g., video object segmentation, multi-object tracking) could reuse a neighboring-frame feature map as a cheap source of semantically meaningful noise for a denoiser.
The AMG adjacency matrix could be swapped into other multi-modal fusion problems (e.g., audio-visual or depth-RGB fusion) as a drop-in attention mask, since it only needs two feature tensors and no labels.
A natural test of the paper's mechanism would be to vary the frame gap used for the neighboring-frame distractor: larger gaps should create stronger 'similar object' offsets, but also increase the chance that the same object appears in very different pose, so performance should peak at some intermediate gap.
An open question the paper leaves implicit is whether the denoiser treats the distractor as clutter or as an extra appearance cue; ablating the distractor under varying motion could separate these two interpretations.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

GDSTrack is a plausible incremental contribution to self-supervised RGB-T tracking. The new part is the combination: a learned adjacency mask (AMG) guiding graph attention fusion, and a DDIM denoiser that uses a neighboring frame's fused feature as a 'distractor' to make the model robust to pseudo-label noise. Each component is borrowed, but the specific integration is new and clearly described. The authors also did a fair thing by retraining ViPT, TBSI, and AFter with the same pseudo-labels, making the comparisons more honest than simply citing published numbers. The ablation structure is clear, and code is promised. The soft spots are concentrated in the evaluation. The largest is the LasHeR protocol. The paper says 'We use LasHeR as our training dataset' and then reports LasHeR results in Table 1, without ever specifying how the 1,224 videos are split into training and test sets. If the same sequences are used for training and testing, the 6.1% PR gain over S2OTFormer is meaningless. If a split exists, the paper must state the number of training and test sequences and the splitting rule. This is not a minor omission; it is central to the main claim. Second, the TopK hyperparameter (k=256) is chosen after looking at test-set performance on GTOT and RGBT234 (Table 5). That is test-set overfitting, and it inflates the reported numbers. Third, the TGID module's benefit is small—0.2% SR on RGBT234 and LasHeR—so the paper should not overclaim its contribution. No error bars are given, which matters when the differences are a few tenths of a percent. The conceptual worry about the distractor noise in Sec 3.3 is less damaging than it first appears: in most tracking videos, object motion creates a real offset, so the neighboring fusion result does act as a distractor. But the paper does not analyze static-object cases, where the 'distractor' is just a corrupted target. That is a minor omission. Who is this for? Researchers working on RGB-T tracking or self-supervised tracking. It deserves a serious referee, because the architecture is coherent and the problem is real. I would ask for a rewritten experiments section with a clear data split, error bars, and hyperparameter selection on a validation set. If the LasHeR split cannot be clarified, the paper still makes a moderate contribution on RGBT234 and VTUAV, but not with the 6.1% headline. The citation pattern is fine; self-citations are mostly to baseline methods and previous work.

Referee Report

4 major / 7 minor

Summary. The paper introduces GDSTrack, a self-supervised RGB-T tracking method that combines a Modality-guided Dynamic Graph Fusion (MDGF) module with a Temporal Graph-Informed Diffusion (TGID) module. MDGF dynamically generates an adjacency matrix via the AMG module and uses it to guide graph attention for fusing RGB and thermal features. TGID treats the MDGF fusion output of a neighboring frame as a distractor-noise term in a DDIM-based diffusion process, aiming to improve robustness to similar-object interference. The model is trained using pseudo-labels generated from optical flow, and evaluated on GTOT, RGBT234, LasHeR, and VTUAV, reporting improvements over prior self-supervised trackers, with code released.

Significance. If the reported results are reproducible, GDSTrack would be a meaningful advance for self-supervised multi-modal tracking, where only a few methods exist. The combination of graph-based fusion and diffusion-based denoising is novel in this context, and the substantial margins on LasHeR and VTUAV suggest that the architecture addresses a real gap. The release of source code is a strength. However, the evaluation protocol for LasHeR is incompletely specified, the TopK hyperparameter appears to be tuned on test sets, and the TGID module's contribution is small relative to the paper's claims; these issues need to be resolved before the results can be fully trusted.

major comments (4)

[Section 4 (Implementation Details), Table 1] The evaluation protocol for LasHeR is not specified. The text says 'We use LasHeR as our training dataset' but gives no train/test split, so the reported PR/NPR/SR on LasHeR are unverifiable. If training and evaluation use the same sequences, the 6.1% PR gain over S2OTFormer would reflect in-distribution fitting rather than generalization. Please state the exact split (e.g., number of training/test sequences, random seed) and, if available, the standard protocol, or clarify the evaluation procedure.
[Section 4.2, Table 5 and Eq. (14)] The TopK hyperparameter in the AMG is selected based on test-set performance on GTOT and RGBT234 ('When K is set to 256, the model performs the best'), which is a post-hoc choice that inflates the reported numbers. In addition, the threshold θ in Eq. (14) is never given a value, so the AMG module is not fully reproducible. Please either use a validation split for model selection or explicitly acknowledge test-set tuning, and report θ and all other hyperparameters.
[Section 4.2, Tables 3 and 6] The ablation of the TGID module shows gains of only 0.2% SR on RGBT234 and 0.2% PR/SR on LasHeR (Table 3), which are within the range of run-to-run variation, and the diffusion component ablation in Table 6 is run with K=5 rather than the final K=256. These results do not support the claim that TGID 'significantly' improves robustness to distractor noise. Please report the ablation under the same configuration as the full model and provide an analysis of statistical significance.
[Section 3.3, Eq. (20)] The design assumes that the fused feature f_{x1}^{S1} from the neighboring frame is offset from the current object location and therefore acts as a similar-object distractor. If the object is static or the offset is small, this term is instead a corrupted version of the target, and the diffusion model may learn incorrect noise statistics. Please validate this assumption by reporting the distribution of inter-frame object displacement in the training data, or by an experiment with stationary objects.

minor comments (7)

[Abstract and throughout] The string 'VTUA V' should be 'VTUAV' (Abstract, Section 4.1, Table 1). Also, 'Y ong Zhou' in the author list should be 'Yong Zhou'.
[Section 3.2, Eq. (13)] The statement that M is shifted to [0,1] is not justified because S1 is an unscaled dot product; please clarify the normalization or define the range precisely.
[Section 4 (Implementation Details)] The number of diffusion timesteps, the noise schedule, and the value of β in Eq. (20) are not reported; please add these to the implementation details.
[Section 4.2, Table 6] The header 'Naive DM condition distractor' is ambiguous; please label the rows clearly (e.g., with check marks) to indicate which components are enabled.
[Section 4.2, Table 4] The AMG ablation shows a 0.4% SR drop on RGBT234 versus the identity matrix; the text explains this as a 'little decrease,' but the PR gain is 1.1%; please discuss this trade-off.
[Section 4 (Implementation Details)] The pseudo-label generation method is not described; the paper references USOT but does not explain how optical flow is applied to thermal images and how the two modalities' pseudo-labels are merged; please provide details or a specific citation to a prior RGB-T pseudo-label procedure.
[Section 4.1, Table 1] All experiments appear to be single runs without error bars or statistical tests; given the small performance gaps on some benchmarks (e.g., 0.8% SR on RGBT234), please report multiple runs with mean and standard deviation, or justify the absence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GDSTrack's results are empirical evaluations on held-out benchmarks, not derivations from its own definitions.

full rationale

The paper proposes an end-to-end trained tracking model. Its training losses (Eqs. 3, 7, 8, 9) supervise the network using pseudo-labels generated externally by USOT (Zheng et al., 2021) and using the model's own intermediate outputs; no loss term is defined as the quantity it claims to predict. The three headline improvements over S2OTFormer are measured on the standard RGBT234, LasHeR, and VTUAV benchmark protocols after training, so they are not forced by construction. Self-citations (Li et al., 2023; Li et al., 2024) are used only to name baseline trackers and do not serve as unverified premises. The distractor-noise construction in Eq. (20) is an inductive bias about temporal offset, not a definitional identity. The manuscript omits the LasHeR train/test split, which is a reproducibility concern, but that omission is not a circularity in the derivation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. The free parameters are standard deep learning hyperparameters, with TopK explicitly tuned on test data. The core axioms are domain assumptions about pseudo-label quality, the utility of neighboring-frame offsets, and the effectiveness of diffusion denoising for tracking features.

free parameters (4)

TopK k in AMG = 256
Selected based on best performance on GTOT and RGBT234 test sets (Table 5).
Threshold theta in AMG = not reported
Used in Eq. (14) to filter the adjacency matrix; exact value is not given.
Distractor noise mixing coefficient beta = not reported
Controls the weight of distractor noise in Eq. (20); no value is specified.
Loss weights lambda1, lambda2 = 2 and 5
Adopted from prior work (Zhao et al., 2023a) rather than tuned here.

assumptions (4)

domain assumption Optical-flow based pseudo-labels (following USOT) are accurate enough to train a self-supervised tracker.
The entire pipeline relies on these pseudo-labels as supervision; inaccuracies are the problem GDSTrack aims to solve but the method still presumes the labels contain enough signal.
domain assumption The neighboring frame fusion result offsets at the object location due to motion and can serve as a similar-object distractor.
Stated in Section 3.3, used to inject f_{x1}^{S1} as noise in Eq. (20).
domain assumption A diffusion model with modal features as conditions can effectively denoise fused tracking features.
Borrowed from GMMT (Tang et al., 2024); the paper assumes this transfers to the self-supervised setting.
domain assumption The AMG similarity mask guides graph attention to focus on coherent object regions.
The design is heuristic; no theoretical guarantee is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking." pith.science (2026). https://pith.science/paper/3KTSVKZA

@misc{pith2026250503507,
  author       = {Pith},
  title        = {Pith review of: Modality-Guided Dynamic Graph Fusion and Temporal Diffusion for Self-Supervised RGB-T Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3KTSVKZA}},
  note         = {Machine review of arXiv:2505.03507}
}

read the original abstract

To reduce the reliance on large-scale annotations, self-supervised RGB-T tracking approaches have garnered significant attention. However, the omission of the object region by erroneous pseudo-label or the introduction of background noise affects the efficiency of modality fusion, while pseudo-label noise triggered by similar object noise can further affect the tracking performance. In this paper, we propose GDSTrack, a novel approach that introduces dynamic graph fusion and temporal diffusion to address the above challenges in self-supervised RGB-T tracking. GDSTrack dynamically fuses the modalities of neighboring frames, treats them as distractor noise, and leverages the denoising capability of a generative model. Specifically, by constructing an adjacency matrix via an Adjacency Matrix Generator (AMG), the proposed Modality-guided Dynamic Graph Fusion (MDGF) module uses a dynamic adjacency matrix to guide graph attention, focusing on and fusing the object's coherent regions. Temporal Graph-Informed Diffusion (TGID) models MDGF features from neighboring frames as interference, and thus improving robustness against similar-object noise. Extensive experiments conducted on four public RGB-T tracking datasets demonstrate that GDSTrack outperforms the existing state-of-the-art methods. The source code is available at https://github.com/LiShenglana/GDSTrack.

Figures

Figures reproduced from arXiv: 2505.03507 by the authors.

**Figure 2.** The pipeline of GDSTrack model. First, the encoder obtains the search frame features that interact with the template frame features. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The pipeline of AMG. The AMG module concatenates vis [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Attribute-based evaluation on VTUAV dataset compared [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 27 canonical work pages

[1]

Scene-dependent pre- diction in latent space for video anomaly detection and anticipation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(1):224–239,

[Caoet al., 2025 ] Congqi Cao, Hanwen Zhang, Yue Lu, Peng Wang, and Yanning Zhang. Scene-dependent pre- diction in latent space for video anomaly detection and anticipation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(1):224–239,

work page 2025
[7]

Bridging search region in- teraction with template for rgb-t tracking

[Huiet al., 2023 ] Tianrui Hui, Zizheng Xun, Fengguang Peng, Junshi Huang, Xiaoming Wei, Xiaolin Wei, Jiao Dai, Jizhong Han, and Si Liu. Bridging search region in- teraction with template for rgb-t tracking. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13630–13639,

work page 2023
[9]

Learning collabora- tive sparse representation for grayscale-thermal tracking

[Liet al., 2016 ] Chenglong Li, Hui Cheng, Shiyi Hu, Xi- aobai Liu, Jin Tang, and Liang Lin. Learning collabora- tive sparse representation for grayscale-thermal tracking. IEEE Transactions on Image Processing, 25(12):5743– 5756,

work page 2016
[13]

Motion-aware self-supervised rgbt tracking with multi-modality hierarchical transform- ers.ACM Transactions on Multimedia Computing, Communications and Applications,

[Liet al., 2024 ] Shenglan Li, Rui Yao, Yong Zhou, Hancheng Zhu, Jiaqi Zhao, Zhiwen Shao, and Abdul- motaleb El Saddik. Motion-aware self-supervised rgbt tracking with multi-modality hierarchical transform- ers.ACM Transactions on Multimedia Computing, Communications and Applications,

work page 2024
[14]

Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

[Loshchilov, 2017] I Loshchilov. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

arXiv 2017
[16]

U-net: Convolutional networks for biomedical image segmentation

[Ronnebergeret al., 2015 ] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical Image Computing and Computer-Assisted Intervention–MICCAI 2015., pages 234–241. Springer,

work page 2015
[18]

S2siamfc: Self-supervised fully convolutional siamese network for visual tracking

[Sioet al., 2020 ] Chon Hou Sio, Yu-Jen Ma, Hong- Han Shuai, Jun-Cheng Chen, and Wen-Huang Cheng. S2siamfc: Self-supervised fully convolutional siamese network for visual tracking. InProceedings of the 28th ACM international conference on multimedia, pages 1948–1957,

work page 2020
[19]

Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

[Songet al., 2020 ] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

arXiv 2020

Show all 34 references

[20]

Datfuse: Infrared and visible image fusion via dual attention transformer.IEEE Trans- actions on Circuits and Systems for Video Technology, 33(7):3159–3172,

[Tanget al., 2023 ] Wei Tang, Fazhi He, Yu Liu, Yansong Duan, and Tongzhen Si. Datfuse: Infrared and visible image fusion via dual attention transformer.IEEE Trans- actions on Circuits and Systems for Video Technology, 33(7):3159–3172,

2023
[21]

Generative-based fusion mechanism for multi-modal tracking

[Tanget al., 2024 ] Zhangyong Tang, Tianyang Xu, Xiaojun Wu, Xue-Feng Zhu, and Josef Kittler. Generative-based fusion mechanism for multi-modal tracking. InProceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pages 5189–5197,

2024
[22]

Attention is all you need.Advances in neural information processing systems, 30,

[Vaswaniet al., 2017 ] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30,

2017
[23]

Graph attention networks.stat, 1050(20):10–48550,

[Velickovicet al., 2017] Petar Velickovic, Guillem Cucu- rull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks.stat, 1050(20):10–48550,

2017
[24]

Unsupervised deep tracking

[Wanget al., 2019 ] Ning Wang, Yibing Song, Chao Ma, Wengang Zhou, Wei Liu, and Houqiang Li. Unsupervised deep tracking. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 1308–1317,

2019
[25]

Mfgnet: Dynamic modality-aware filter generation for rgb-t tracking.IEEE Transactions on Multimedia, 25:4335–4348,

[Wanget al., 2023 ] Xiao Wang, Xiujun Shu, Shiliang Zhang, Bo Jiang, Yaowei Wang, Yonghong Tian, and Feng Wu. Mfgnet: Dynamic modality-aware filter generation for rgb-t tracking.IEEE Transactions on Multimedia, 25:4335–4348,

2023
[26]

A dynamic feature interaction framework for multi-task visual perception.Int

[Xiet al., 2023 ] Yuling Xi, Hao Chen, Ning Wang, Peng Wang, Yanning Zhang, Chunhua Shen, and Yifan Liu. A dynamic feature interaction framework for multi-task visual perception.Int. J. Comput. Vision, 131(11):2977–2993, July

2023
[27]

Multimodal cross-layer bilinear pooling for rgbt tracking.IEEE Transactions on Multimedia, 24:567–580,

[Xuet al., 2021 ] Qin Xu, Yiming Mei, Jinpei Liu, and Chen- glong Li. Multimodal cross-layer bilinear pooling for rgbt tracking.IEEE Transactions on Multimedia, 24:567–580,

2021
[28]

Meem: robust tracking via multiple experts using entropy minimization

[Zhanget al., 2014 ] Jianming Zhang, Shugao Ma, and Stan Sclaroff. Meem: robust tracking via multiple experts using entropy minimization. InEuropean Conference on Com- puter Vision, pages 188–203. Springer,

2014
[29]

Multi-modal fusion for end-to-end rgb-t track- ing

[Zhanget al., 2019 ] Lichao Zhang, Martin Danelljan, Abel Gonzalez-Garcia, Joost Van De Weijer, and Fahad Shah- baz Khan. Multi-modal fusion for end-to-end rgb-t track- ing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision Workshops, pages 0–0,

2019
[30]

Visible-thermal uav track- ing: A large-scale benchmark and new baseline

[Zhanget al., 2022 ] Pengyu Zhang, Jie Zhao, Dong Wang, Huchuan Lu, and Xiang Ruan. Visible-thermal uav track- ing: A large-scale benchmark and new baseline. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8886–8895,

2022
[31]

[Zhanget al., 2024 ] Lei Zhang, Jiangtao Nie, Wei Wei, and Yanning Zhang. Unsupervised test-time adaptation learn- ing for effective hyperspectral image super-resolution with unknown degeneration.IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 46(7):5008–5025,

2024
[32]

Diff-tracker: text-to-image diffusion models are unsupervised trackers

[Zhanget al., 2025 ] Zhengbo Zhang, Li Xu, Duo Peng, Hos- sein Rahmani, and Jun Liu. Diff-tracker: text-to-image diffusion models are unsupervised trackers. InEuropean Conference on Computer Vision, pages 319–337. Springer,

2025
[33]

Learning to track objects from un- labeled videos

[Zhenget al., 2021 ] Jilai Zheng, Chao Ma, Houwen Peng, and Xiaokang Yang. Learning to track objects from un- labeled videos. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 13546– 13555,

2021
[34]

Visual prompt multi-modal track- ing

[Zhuet al., 2023 ] Jiawen Zhu, Simiao Lai, Xin Chen, Dong Wang, and Huchuan Lu. Visual prompt multi-modal track- ing. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 9516–9526, 2023

2023
[2014]

Exploiting multimodal spatial-temporal patterns for video object tracking.arXiv preprint arXiv:2412.15691,

[Huet al., 2024 ] Xiantao Hu, Ying Tai, Xu Zhao, Chen Zhao, Zhenyu Zhang, Jun Li, Bineng Zhong, and Jian Yang. Exploiting multimodal spatial-temporal patterns for video object tracking.arXiv preprint arXiv:2412.15691,

2024 arXiv
[2015]

Unsupervised learning of accu- rate siamese tracking

[Shenet al., 2022 ] Qiuhong Shen, Lei Qiao, Jinyang Guo, Peixia Li, Xin Li, Bo Li, Weitao Feng, Weihao Gan, Wei Wu, and Wanli Ouyang. Unsupervised learning of accu- rate siamese tracking. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, page...

2022
[2016]

Rgb-t object tracking: Bench- mark and baseline.Pattern Recognition, 96:106977,

[Liet al., 2019 ] Chenglong Li, Xinyan Liang, Yijuan Lu, Nan Zhao, and Jin Tang. Rgb-t object tracking: Bench- mark and baseline.Pattern Recognition, 96:106977,

2019
[2017]

After: Attention-based fusion router for rgbt tracking.arXiv preprint arXiv:2405.02717,

[Luet al., 2024 ] Andong Lu, Wanyu Wang, Chenglong Li, Jin Tang, and Bin Luo. After: Attention-based fusion router for rgbt tracking.arXiv preprint arXiv:2405.02717,

2024 arXiv
[2019]

Lasher: A large-scale high-diversity benchmark for rgbt track- ing.IEEE Transactions on Image Processing, 31:392–404,

[Liet al., 2021 ] Chenglong Li, Wanlin Xue, Yaqing Jia, Zhichen Qu, Bin Luo, Jin Tang, and Dengdi Sun. Lasher: A large-scale high-diversity benchmark for rgbt track- ing.IEEE Transactions on Image Processing, 31:392–404,

2021
[2020]

High-speed tracking with kernelized correlation filters.IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 37(3):583–596,

[Henriqueset al., 2014 ] Jo˜ao F Henriques, Rui Caseiro, Pe- dro Martins, and Jorge Batista. High-speed tracking with kernelized correlation filters.IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 37(3):583–596,

2014
[2021]

Unsupervised rgb-t object tracking with attentional multi- modal feature fusion.Multimedia Tools and Applications, pages 1–19,

[Liet al., 2023 ] Shenglan Li, Rui Yao, Yong Zhou, Hancheng Zhu, Bing Liu, Jiaqi Zhao, and Zhiwen Shao. Unsupervised rgb-t object tracking with attentional multi- modal feature fusion.Multimedia Tools and Applications, pages 1–19,

2023
[2022]

An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929,

[Dosovitskiyet al., 2020 ] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition a...

2020 arXiv
[2023]

Mam- bavt: Spatio-temporal contextual modeling for robust rgb-t tracking.arXiv preprint arXiv:2408.07889,

[Laiet al., 2024 ] Simiao Lai, Chang Liu, Jiawen Zhu, Ben Kang, Yang Liu, Dong Wang, and Huchuan Lu. Mam- bavt: Spatio-temporal contextual modeling for robust rgb-t tracking.arXiv preprint arXiv:2408.07889,

2024 arXiv
[2024]

Mixformer: End-to-end tracking with it- erative mixed attention

[Cuiet al., 2022 ] Yutao Cui, Cheng Jiang, Limin Wang, and Gangshan Wu. Mixformer: End-to-end tracking with it- erative mixed attention. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13608–13618,

2022
[2025]

Unified sequence- to-sequence learning for single- and multi-modal visual object tracking,

[Chenet al., 2024 ] Xin Chen, Ben Kang, Jiawen Zhu, Dong Wang, Houwen Peng, and Huchuan Lu. Unified sequence- to-sequence learning for single- and multi-modal visual object tracking,

2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Scene-dependent pre- diction in latent space for video anomaly detection and anticipation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(1):224–239,

[Caoet al., 2025 ] Congqi Cao, Hanwen Zhang, Yue Lu, Peng Wang, and Yanning Zhang. Scene-dependent pre- diction in latent space for video anomaly detection and anticipation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(1):224–239,

work page 2025

[2] [7]

Bridging search region in- teraction with template for rgb-t tracking

[Huiet al., 2023 ] Tianrui Hui, Zizheng Xun, Fengguang Peng, Junshi Huang, Xiaoming Wei, Xiaolin Wei, Jiao Dai, Jizhong Han, and Si Liu. Bridging search region in- teraction with template for rgb-t tracking. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13630–13639,

work page 2023

[3] [9]

Learning collabora- tive sparse representation for grayscale-thermal tracking

[Liet al., 2016 ] Chenglong Li, Hui Cheng, Shiyi Hu, Xi- aobai Liu, Jin Tang, and Liang Lin. Learning collabora- tive sparse representation for grayscale-thermal tracking. IEEE Transactions on Image Processing, 25(12):5743– 5756,

work page 2016

[4] [13]

Motion-aware self-supervised rgbt tracking with multi-modality hierarchical transform- ers.ACM Transactions on Multimedia Computing, Communications and Applications,

[Liet al., 2024 ] Shenglan Li, Rui Yao, Yong Zhou, Hancheng Zhu, Jiaqi Zhao, Zhiwen Shao, and Abdul- motaleb El Saddik. Motion-aware self-supervised rgbt tracking with multi-modality hierarchical transform- ers.ACM Transactions on Multimedia Computing, Communications and Applications,

work page 2024

[5] [14]

Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

[Loshchilov, 2017] I Loshchilov. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101,

arXiv 2017

[6] [16]

U-net: Convolutional networks for biomedical image segmentation

[Ronnebergeret al., 2015 ] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical Image Computing and Computer-Assisted Intervention–MICCAI 2015., pages 234–241. Springer,

work page 2015

[7] [18]

S2siamfc: Self-supervised fully convolutional siamese network for visual tracking

[Sioet al., 2020 ] Chon Hou Sio, Yu-Jen Ma, Hong- Han Shuai, Jun-Cheng Chen, and Wen-Huang Cheng. S2siamfc: Self-supervised fully convolutional siamese network for visual tracking. InProceedings of the 28th ACM international conference on multimedia, pages 1948–1957,

work page 2020

[8] [19]

Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

[Songet al., 2020 ] Jiaming Song, Chenlin Meng, and Ste- fano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502,

arXiv 2020