Multi-Modal Fusion for End-to-End RGB-T Tracking

Abel Gonzalez-Garcia; Fahad Shahbaz Khan; Joost Van De Weijer; Lichao Zhang; Martin Danelljan

REVIEW 3 major objections 5 minor 61 references

Multi-Modal Fusion for End-to-End RGB-T Tracking

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fusing RGB and thermal features inside the tracker beats either modality alone.

desk verdict Solid end-to-end RGB-T tracking paper with a credible feature-level fusion result, but the evaluation has selection bias and the synthetic-TIR proxy is under-tested; deserves peer review with revisions. read the letter →

arxiv 1908.11714 v1 pith:JN7C2LWJ submitted 2019-08-30 cs.CV

Lichao Zhang , Martin Danelljan , Abel Gonzalez-Garcia , Joost van de Weijer , Fahad Shahbaz Khan This is my paper · ORCID

classification cs.CV

keywords RGB-Ttrackingthermalinfraredmulti-modalfusionend-to-endtrainingDiscriminativeModelPredictionsyntheticdatavisualobjectfeature-level

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a state-of-the-art single-modality RGB tracker, DiMP, can be turned into a strong RGB-thermal tracker by fusing modalities at the right internal point and training the whole network end-to-end. It compares pixel-level, feature-level, and response-level fusion and finds that feature-level fusion, feeding concatenated RGB and TIR features into both the IoU-Net and the model predictor, works best. With this configuration, plus a higher learning rate for the TIR branch, the resulting mfDiMP tracker reports an EAO of 0.391 on VOT-RGBT2019 and a 55.5% success rate on RGBT210. These results matter because RGB-T tracking is valuable in low illumination and bad weather, and the paper offers evidence that the tracking network itself can learn how to combine the two modalities instead of relying on hand-crafted fusion rules.

What carries the argument

The central object is the DiMP tracker (Discriminative Model Prediction), an end-to-end learnable architecture whose main components are a ResNet-50 feature extractor, a model predictor that learns a discriminative filter, and an IoU-Net that estimates bounding-box overlap. The paper inserts modality fusion into this machinery at three levels: pixel-level fusion concatenates the RGB and TIR images into a single four-channel input; response-level fusion sums independently computed confidence maps; and feature-level fusion runs separate feature extractors for each modality and concatenates their deep features before the IoU-Net and model predictor. The load-bearing mechanism is the feature-level concatenation combined with end-to-end fine-tuning on synthetic paired data and an asymmetric learning rate that lets the TIR branch catch up with the RGB-pretrained branch.

What would settle it

Train the same feature-level fusion architecture on real aligned RGB-T pairs instead of pix2pix-synthesized TIR, using the same training budget, and compare EAO on VOT-RGBT2019 and success rate on RGBT210; if the real-trained model does not match or exceed the synthetic-trained one, the synthetic data proxy is the weak link.

Watch

Extended reading notes

Core claim

The central claim is that the best place to fuse RGB and TIR is neither at the input pixels nor at the final response maps, but at the deep feature level, inside both the target-estimation network and the model predictor of DiMP. The paper uses separate modality-specific ResNet-50 feature extractors, concatenates the block3 and block4 features from the two modalities, and feeds the fused features to the IoU-Net and the model predictor while fine-tuning the whole architecture end-to-end. Because the TIR branch starts from weights pre-trained on RGB, the paper also increases the TIR feature extractor's learning rate by a factor of ten, and this configuration lifts VOT-RGBT2019 EAO from 0.327 for the RGB-only DiMP to 0.391, with robustness improving from 0.345 to 0.228. On RGBT210 the success rate is 55.5%, an absolute gain of 4.2% over the single-modality baseline, and the tracker outperforms prior methods on 11 of the 12 attribute categories, with thermal crossover as the sole exception.

Load-bearing premise

The argument assumes that thermal images synthesized by pix2pix from RGB, trained on 87K aligned RGB-TIR pairs, are faithful enough to real thermal imagery that end-to-end training on them transfers to real RGB-T benchmarks.

Editorial extensions

If this is right

If feature-level fusion is the right choice, future RGB-T trackers should place fusion inside the learned components rather than at the input or output.
The tenfold higher learning rate for the TIR branch suggests that RGB-pretrained initialization biases the fusion, and that symmetric learning rates leave performance on the table.
End-to-end training with synthetic paired RGB-T data transfers to real benchmarks, implying that the pix2pix-based data-generation pipeline is sufficient for training the fusion architecture.
Feature-level fusion improves robustness more than accuracy: on VOT-RGBT2019 the robustness score drops from 0.345 to 0.228 while accuracy rises from 0.586 to 0.615.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension would train the same architecture on real aligned RGB-T pairs instead of pix2pix-synthesized thermal images; if the synthetic-to-real gap is small, the reported numbers should hold or improve.
The attribute analysis shows thermal crossover as the only category where mfDiMP does not lead, which hints that the synthetic TIR generator may be weakest precisely when real thermal signatures matter most.
The asymmetric-learning-rate result suggests the fusion gain is partly a rebalancing of imbalanced initialization, so the same recipe could transfer to other RGB-pretrained trackers beyond DiMP.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Good paper to know about if you work on RGB-T tracking. It does the obvious thing that nobody had done yet: take a strong deep RGB tracker (DiMP), train it end-to-end on paired RGB/TIR data, and systematically compare where to fuse the two modalities. The main conclusion, that feature-level fusion into both the model predictor and the IoU-Net beats single-modality and earlier fusion points, looks credible: EAO goes from 0.341 for the best single-modality setup to 0.389-0.391 on VOT-RGBT2019, and the gains on RGBT210 are in the same direction. The ablations are extensive (14 configurations), and the attribute analysis is useful even if the claim "outperforms all but one" hides the fact that the one exception is thermal crossover.

The soft spots are mostly in the evaluation. The best configuration was selected from the same VOT-RGBT2019 benchmark that is then used for the state-of-the-art claim; the gap between the x1 and x10 learning-rate variants is 0.002, so picking the better one and selling the "big jump" is over-reading. There are no error bars or significance tests, even though the paper acknowledges DiMP is stochastic and averages runs. And on VOT-RGBT2019 the comparison only includes RGB trackers, so the "state-of-the-art" claim should be read narrowly; the RGBT210 comparison includes other RGB-T methods and is more convincing.

The stress-test worry about the synthetic TIR data deserves a careful answer. The paper trains the TIR branch on pix2pix-generated images and never measures the synthetic-to-real gap for the fusion task. But the single-modality fine-tuning results on real TIR benchmarks improve, so the proxy is not obviously broken. The thermal crossover weakness is suggestive but not decisive: that is precisely the regime where TIR is uninformative, so RGB-only doing better there does not sink the fusion story. The missing experiment is a capacity control, e.g. feeding the same RGB stream twice into the architecture; without that, part of the fusion gain could be extra model capacity rather than thermal complementarity. Still, that is a request for an additional ablation, not a fatal flaw.

No code or synthetic dataset is released, which will make independent replication harder. But the paper is clearly written, the method is reproducible in principle, and the empirical claims are specific enough to test. Anyone working on multimodal tracking should read it. I would send it to peer review with requests for error bars, a validation split for configuration selection, and ideally the capacity-control ablation.

Referee Report

3 major / 5 minor

Summary. The paper proposes an end-to-end RGB-T tracking framework built on the DiMP baseline. It systematically compares three fusion levels (pixel-level, feature-level, response-level) and ablates where fusion is applied in the feature extractor, the IoU-Net, and the model predictor. Because no large-scale aligned RGB-T training set exists, the authors generate a synthetic paired RGB-T dataset by applying a pre-trained pix2pix model to GOT-10k RGB videos, following their earlier TIR-tracking work [58]. On VOT-RGBT2019 the best configuration (feature-level fusion feeding both the IoU-Net and the model predictor, with a 10x learning rate on the TIR branch) achieves EAO 0.391, and on RGBT210 the final tracker reaches 78.6% precision and 55.5% success rate, reported as state-of-the-art. The paper concludes that feature-level fusion with end-to-end training exploits multi-modal complementary information and substantially improves over single-modality counterparts.

Significance. If the claims hold, the paper makes a useful contribution by showing where modality fusion matters inside a modern deep tracker and by providing a practical way to train RGB-T trackers despite the scarcity of aligned real data. The systematic ablation across fusion levels and network components is a strength, as is the use of established public benchmarks with multiple runs averaged per configuration. However, the headline result rests on an unvalidated synthetic-to-real transfer and on differences between configurations that are reported without variance or significance testing. The paper's own attribute analysis shows the weakest fusion gain precisely on the thermal-crossover attribute, which is where real TIR information should matter most. These issues leave the mechanistic claim (that the gain comes from modality complementarity rather than from added model capacity or an effectively second RGB-like stream) not fully secured. The work is nevertheless publishable if these concerns are addressed with additional experiments and more careful statistical reporting.

major comments (3)

[§5.4, Table 1] The paper selects the best of 14 configurations and reports only mean EAO/accuracy/robustness values, despite noting in §5.3 that DiMP is stochastic and that 15 runs (VOT) or 5 runs (RGBT210) were averaged. No standard deviations, confidence intervals, or significance tests are given. Consequently the difference between the selected variant (0.389) and the ×10 learning-rate variant (0.391), which is then declared the final tracker, may be within run-to-run noise. Please report per-configuration variance and perform a pairwise significance test for the key comparisons (best feature-level vs. best single-modality, and the ×10 variant vs. the plain feature-level variant).
[§4.2 and §5.1 (synthetic data) with Table 3] The central claim that feature-level fusion improves tracking by exploiting RGB-T complementarity depends on the synthetic TIR stream behaving like real thermal imagery during end-to-end training, but the paper never measures this synthetic-to-real gap for the fusion task. The evidence in Table 3 points in the opposite direction on the thermal-crossover attribute, where real TIR should be most informative: mfDiMP scores 76.5/55.2 while the RGB-only ECO scores 82.1/58.8. To secure the modality-specific interpretation, please add a control experiment that isolates the contribution of true thermal cues, for example by fine-tuning on a modest amount of real aligned RGB-T data, or by evaluating a dual-stream RGB-RGB fusion with the same architecture and capacity. Without such a control, the gains over single-modality baselines could stem from increased model capacity or from a second correlated appearance stream rather than from modality complementarity.
[§5.7 and Table 3] The paper states that mfDiMP outperforms all other trackers on all attributes but one (thermal crossover), and the conclusion section claims optimal use of information from both modalities. The thermal-crossover result is not a minor exception: it is exactly the regime in which the TIR modality should provide information unavailable in RGB, and a RGB-only tracker (ECO) outperforms the fused tracker there. This undercuts the interpretation that the feature-level fusion learns complementary thermal representations. Please analyze this attribute explicitly, report per-sequence diagnostics, and discuss whether the synthetic TIR branch is learning thermal signatures or merely RGB-derived appearance statistics.

minor comments (5)

[§5.4] 'Form Table 1' should be 'From Table 1'.
[Figure 1 caption] The caption 'Two exemplar videos from RGB modality and TIR modality on the top and bottom separately' is grammatically awkward; consider rewriting for clarity.
[§5.3] The ×10 higher learning rate for the TIR feature extractor is a free hyperparameter, but no sensitivity analysis is provided around this choice, so it is unclear how robust the 0.391 EAO result is to this setting.
[Table 3] The table header contains the merged entry 'CFnet+ RGBT' with inconsistent spacing and no clear indication of which baseline it refers to; please format the header consistently.
[§5.2] The evaluation protocol description for VOT-RGBT2019 is brief; please state the failure threshold and re-initialization lag explicitly, or cite the VOT toolkit documentation, so that readers do not need to consult an external website.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical fusion study whose claims rest on benchmark comparisons and controlled ablations, not on a derivation that reduces to its inputs.

full rationale

The paper does not perform a derivation whose conclusion is equivalent to its assumptions by construction. It adapts the existing DiMP tracker, proposes three fusion mechanisms (pixel-level, feature-level, response-level), and evaluates them on VOT-RGBT2019 and RGBT210 against single-modality baselines and published trackers. The only potentially self-referential ingredient is the synthetic RGB-T training data, which is generated using the authors' earlier method [58]. However, [58] is a separate published method for TIR data synthesis with its own validation, and the present paper's central claim—that feature-level fusion improves over single-modality tracking—is established by direct ablation in Table 1 and by comparison with external trackers in Tables 2 and 3. The choice of a higher learning rate for the TIR branch is a motivated hyperparameter, not a parameter fitted so that a renamed quantity becomes the prediction. The attribute analysis even reports a weakness on thermal crossover, which shows the evaluation is not constructed to force the claimed outcome. No equation in the paper reduces to a fitted constant, and no result is imported solely through a self-citation chain. Concerns about synthetic-to-real domain gap or test-set variant selection are validity risks, not circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper does not introduce new theoretical entities or fitted constants beyond one learning-rate hyperparameter. Its main assumptions are about the fidelity of synthetic thermal data and the alignment of the two modalities.

free parameters (1)

TIR feature extractor learning rate multiplier = 10
Selected as part of the best configuration in Sec. 5.4; no validation search or robustness analysis is reported.

assumptions (3)

domain assumption Synthetic TIR images from pix2pix are a faithful proxy for real TIR imagery in training.
The entire end-to-end training pipeline and the reported gains rely on this (Sec. 4.2, 5.1), yet the paper does not quantify the synthetic-to-real domain gap for the fusion task.
domain assumption RGB and TIR frames are spatially aligned, making channel-wise concatenation at pixel and feature levels valid.
The fusion mechanisms concatenate at corresponding spatial positions (Sec. 4.1); misaligned pairs would weaken or break the approach.
ad hoc to paper An RGB-pretrained ResNet backbone is a suitable starting point for the TIR feature extractor.
The TIR branch is initialized from ImageNet RGB weights and fine-tuned with a higher learning rate (Sec. 5.3); whether this transfers well to thermal data is assumed rather than tested.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-Modal Fusion for End-to-End RGB-T Tracking." pith.science (2026). https://pith.science/paper/JN7C2LWJ

@misc{pith2026190811714,
  author       = {Pith},
  title        = {Pith review of: Multi-Modal Fusion for End-to-End RGB-T Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JN7C2LWJ}},
  note         = {Machine review of arXiv:1908.11714}
}

read the original abstract

We propose an end-to-end tracking framework for fusing the RGB and TIR modalities in RGB-T tracking. Our baseline tracker is DiMP (Discriminative Model Prediction), which employs a carefully designed target prediction network trained end-to-end using a discriminative loss. We analyze the effectiveness of modality fusion in each of the main components in DiMP, i.e. feature extractor, target estimation network, and classifier. We consider several fusion mechanisms acting at different levels of the framework, including pixel-level, feature-level and response-level. Our tracker is trained in an end-to-end manner, enabling the components to learn how to fuse the information from both modalities. As data to train our model, we generate a large-scale RGB-T dataset by considering an annotated RGB tracking dataset (GOT-10k) and synthesizing paired TIR images using an image-to-image translation approach. We perform extensive experiments on VOT-RGBT2019 dataset and RGBT210 dataset, evaluating each type of modality fusing on each model component. The results show that the proposed fusion mechanisms improve the performance of the single modality counterparts. We obtain our best results when fusing at the feature-level on both the IoU-Net and the model predictor, obtaining an EAO score of 0.391 on VOT-RGBT2019 dataset. With this fusion mechanism we achieve the state-of-the-art performance on RGBT210 dataset.

Figures

Figures reproduced from arXiv: 1908.11714 by the authors.

**Figure 1.** Qualitative comparison between ‘mfDiMP’ and ‘DiMP’. Two exemplar videos from RGB modality and TIR modality on the top and bottom separately, where DiMP performs on each of them with single modality input. Our mfDiMP can effectively track the object by fusing both modalities. drawn more research attention recently [31, 34, 32, 35]. As multi-modal data, i.e. from the RGB and TIR modalities, can provide complementary… view at source ↗

**Figure 2.** Overview of our multi-modal fusion framework on feature-level. We input images from RGB and TIR modalities to their feature extractor separately. Then we fuse the deep features from different blocks of the backbone. Fused features from block3 and block4 are input to IoU modulation and IoU predictor. Fused features from block4 are input to the model predictor for the final response map. is the IoU modulation for calc… view at source ↗

**Figure 3.** Precision plot and success plot by comparing our mfDiMP with the top-10 trackers on RGBT210 dataset [34]. We can see our mfDiMP outperforms DiMP with an absolute gain of 6.7% and 4.2% in terms of precision rate and success rate respectively. ECO SiamFC DaSiamRPN ATOM DiMP mfDiMP [10] [4] [61] [11] [5] EAO(↑) 0.265 0.254 0.324 0.318 0.327 0.391 A (↑) 0.580 0.594 0.604 0.575 0.586 0.615 R (↓) 0.480 0.533 0.482 0.374 0… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 57 canonical work pages

[58]

Zhang, A

L. Zhang, A. Gonzalez-Garcia, J. van de Weijer, M. Danell- jan, and F. S. Khan. Synthetic data generation for end-to-end thermal infrared tracking. IEEE Transactions on Image Pro- cessing, 28(4):1837–1850, 2019. 2, 4, 5

work page 2019
[1]

2, 5, 6, 7, 8

www.votchallenge.net/vot2019. 2, 5, 6, 7, 8

work page
[2]

https://sites.google.com/view/ ahutracking001. 6

work page
[3]

Bertinetto, J

L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, and P. H. Torr. Staple: Complementary learners for real-time tracking. In CVPR, pages 1401–1409, 2016. 8

work page 2016
[4]

Bertinetto, J

L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. Fully-convolutional siamese networks for object tracking. In ECCV workshop, 2016. 1, 2, 6, 7, 8

work page 2016
[5]

G. Bhat, M. Danelljan, L. Van Gool, and R. Timofte. Learn- ing discriminative model prediction for tracking. In ICCV,

work page
[6]

S. T. Birchﬁeld and S. Rangarajan. Spatiograms versus his- tograms for region-based tracking. In CVPR, volume 2, pages 1158–1163. IEEE, 2005. 3

work page 2005
[7]

D. S. Bolme, J. R. Beveridge, B. A. Draper, and Y . M. Lui. Visual object tracking using adaptive correlation ﬁlters. In CVPR, 2010. 1, 2

work page 2010

Show all 61 references

[8]

C. ´O. Conaire, N. E. OConnor, and A. Smeaton. Thermo- visual feature fusion for object tracking using multiple spa- tiogram trackers. Machine Vision and Applications , 19(5- 6):483–494, 2008. 3

2008
[9]

Dalal and B

N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005. 2

2005
[10]

Danelljan, G

M. Danelljan, G. Bhat, F. S. Khan, and M. Felsberg. Eco: efﬁcient convolution operators for tracking. In CVPR, 2017. 1, 2, 6, 7, 8

2017
[11]

Danelljan, G

M. Danelljan, G. Bhat, F. S. Khan, and M. Felsberg. Atom: Accurate tracking by overlap maximization. In CVPR, 2019. 1, 2, 3, 7, 8

2019
[12]

Danelljan, G

M. Danelljan, G. H ¨ager, F. Khan, and M. Felsberg. Accurate scale estimation for robust visual tracking. In BMVC, 2014. 1, 2

2014
[13]

Danelljan, G

M. Danelljan, G. Hager, F. Shahbaz Khan, and M. Fels- berg. Convolutional features for correlation ﬁlter based vi- sual tracking. In Proceedings of the IEEE International Con- ference on Computer Vision Workshops, 2015. 2

2015
[14]

Danelljan, G

M. Danelljan, G. Hager, F. Shahbaz Khan, and M. Felsberg. Learning spatially regularized correlation ﬁlters for visual tracking. In ICCV, 2015. 2, 8

2015
[15]

Danelljan, G

M. Danelljan, G. Hager, F. Shahbaz Khan, and M. Felsberg. Adaptive decontamination of the training set: A uniﬁed for- mulation for discriminative visual tracking. In CVPR, 2016. 2

2016
[16]

Danelljan, F

M. Danelljan, F. S. Khan, M. Felsberg, and J. van de Weijer. Adaptive color attributes for real-time visual tracking. In CVPR, 2014. 1, 2

2014
[17]

Danelljan, A

M. Danelljan, A. Robinson, F. S. Khan, and M. Felsberg. Beyond correlation ﬁlters: Learning continuous convolution operators for visual tracking. In ECCV, 2016. 1, 2, 8

2016
[18]

H. Fan, L. Lin, F. Yang, P. Chu, G. Deng, S. Yu, H. Bai, Y . Xu, C. Liao, and H. Ling. Lasot: A high-quality benchmark for large-scale single object tracking. CoRR, abs/1809.07845, 2018. 2, 3, 6, 8

2018 arXiv
[19]

Felsberg, A

M. Felsberg, A. Berg, G. Hager, J. Ahlberg, M. Kristan, J. Matas, A. Leonardis, L. Cehovin, G. Fernandez, T. V ojir, et al. The thermal infrared visual object tracking vot-tir2015 challenge results. In Proceedings of the IEEE International Conference on Computer Vision Worksho...

2015
[20]

H. K. Galoogahi, T. Sim, and S. Lucey. Multi-channel corre- lation ﬁlters. In ICCV, 2013. 2

2013
[21]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016. 6

2016
[22]

J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. High- speed tracking with kernelized correlation ﬁlters. TPAMI, 37(3):583–596, 2015. 1, 2, 6, 8

2015
[23]

Huang, X

L. Huang, X. Zhao, and K. Huang. Got-10k: A large high- diversity benchmark for generic object tracking in the wild. CoRR, abs/1810.11981, 2018. 2, 3, 5, 6, 8

2018 arXiv
[24]

Isola, J.-Y

P. Isola, J.-Y . Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In CVPR,
[25]

Jiang, R

B. Jiang, R. Luo, J. Mao, T. Xiao, and Y . Jiang. Acquisition of localization conﬁdence for accurate object detection. In ECCV, pages 784–799, 2018. 3, 4

2018
[26]

Kiani Galoogahi, A

H. Kiani Galoogahi, A. Fagg, and S. Lucey. Learning background-aware correlation ﬁlters for visual tracking. In ICCV, 2017. 2, 8

2017
[27]

Kristan, A

M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pﬂugfelder, L. ˇC. Zajc, T. V oj´ır, G. Bhat, A. Luke ˇziˇc, A. Eldesokey, et al. The sixth visual object tracking vot2018 challenge results. In ECCV workshop, 2018. 3, 8

2018
[28]

Kristan, J

M. Kristan, J. Matas, A. Leonardis, T. V ojir, R. Pﬂugfelder, G. Fernandez, G. Nebehay, F. Porikli, and L. ˇCehovin. A novel performance evaluation methodology for single-target trackers. TPAMI, 38(11):2137–2155, Nov 2016. 1

2016
[29]

B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, and J. Yan. Siamrpn++: Evolution of siamese visual tracking with very deep networks. CoRR, abs/1812.11703, 2018. 2

2018 arXiv
[30]

B. Li, J. Yan, W. Wu, Z. Zhu, and X. Hu. High performance visual tracking with siamese region proposal network. In CVPR, 2018. 1, 2

2018
[31]

C. Li, H. Cheng, S. Hu, X. Liu, J. Tang, and L. Lin. Learning collaborative sparse representation for grayscale- thermal tracking. IEEE Transactions on Image Processing, 25(12):5743–5756, 2016. 1, 3

2016
[32]

C. Li, X. Liang, Y . Lu, N. Zhao, and J. Tang. Rgb-t object tracking: benchmark and baseline. CoRR, abs/1805.08982,

arXiv
[33]

C. Li, X. Sun, X. Wang, L. Zhang, and J. Tang. Grayscale- thermal object tracking via multitask laplacian sparse repre- sentation. IEEE Transactions on Systems, Man, and Cyber- netics: Systems, 47(4):673–681, 2017. 1, 3

2017
[34]

C. Li, N. Zhao, Y . Lu, C. Zhu, and J. Tang. Weighted sparse representation regularized graph learning for rgb-t ob- ject tracking. In Proceedings of the 25th ACM international conference on Multimedia , pages 1856–1864. ACM, 2017. 1, 2, 3, 5, 6, 7, 8

2017
[35]

C. Li, C. Zhu, Y . Huang, J. Tang, and L. Wang. Cross-modal ranking with soft consistency and noisy labels for robust rgb- t tracking. In ECCV, pages 808–823, 2018. 1, 2, 3, 4, 8

2018
[36]

T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Doll´ar, and C. L. Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, pages 740–755. Springer,
[37]

Liu and F

H. Liu and F. Sun. Fusion tracking in color and infrared images using joint sparse representation. Science China In- formation Sciences, 55(3):590–599, 2012. 1, 3

2012
[38]

Lukezic, T

A. Lukezic, T. V oj´ır, L. C. Zajc, J. Matas, and M. Kristan. Discriminative correlation ﬁlter with channel and spatial re- liability. In CVPR, 2017. 1, 2, 8

2017
[39]

Ma, J.-B

C. Ma, J.-B. Huang, X. Yang, and M.-H. Yang. Hierarchical convolutional features for visual tracking. In ICCV, 2015. 2

2015
[40]

C. Ma, X. Yang, C. Zhang, and M.-H. Yang. Long-term correlation tracking. In CVPR, 2015. 2

2015
[41]

Mueller, N

M. Mueller, N. Smith, and B. Ghanem. Context-aware cor- relation ﬁlter tracking. In CVPR, 2017. 2

2017
[42]

Mueller, N

M. Mueller, N. Smith, and B. Ghanem. Context-aware cor- relation ﬁlter tracking. In CVPR, pages 1396–1404, 2017. 8

2017
[43]

Muller, A

M. Muller, A. Bibi, S. Giancola, S. Alsubaihi, and B. Ghanem. Trackingnet: A large-scale dataset and bench- mark for object tracking in the wild. In ECCV, 2018. 2, 3, 6, 8

2018
[44]

Nam and B

H. Nam and B. Han. Learning multi-domain convolutional neural networks for visual tracking. In CVPR, 2016. 2

2016
[45]

Park and A

E. Park and A. C. Berg. Meta-tracker: Fast and robust online adaptation for visual object trackers. In ECCV, 2018. 2

2018
[46]

Russakovsky, J

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 115(3):211–252, 2015. 2

2015
[47]

Y . Song, C. Ma, X. Wu, L. Gong, L. Bao, W. Zuo, C. Shen, R. W. Lau, and M.-H. Yang. Vital: Visual tracking via ad- versarial learning. In CVPR, 2018. 2

2018
[48]

Valmadre, L

J. Valmadre, L. Bertinetto, J. Henriques, A. Vedaldi, and P. H. Torr. End-to-end representation learning for correla- tion ﬁlter based tracking. In CVPR, 2017. 2, 8

2017
[49]

Valmadre, L

J. Valmadre, L. Bertinetto, J. F. Henriques, R. Tao, A. Vedaldi, A. W. Smeulders, P. H. Torr, and E. Gavves. Long-term tracking in the wild: A benchmark. In ECCV, pages 670–685, 2018. 1, 2

2018
[50]

Van De Weijer, C

J. Van De Weijer, C. Schmid, J. Verbeek, and D. Larlus. Learning color names for real-world applications. IEEE Transactions on Image Processing, 18(7):1512–1523, 2009. 2

2009
[51]

Q. Wang, Z. Teng, J. Xing, J. Gao, W. Hu, and S. Maybank. Learning attentions: residual attentional siamese network for high performance online visual tracking. In CVPR, 2018. 2

2018
[52]

Y . Wu, E. Blasch, G. Chen, L. Bai, and H. Ling. Multiple source data fusion via sparse representation for robust visual tracking. In 14th International Conference on Information Fusion, pages 1–8. IEEE, 2011. 1, 3

2011
[53]

Y . Wu, J. Lim, and M.-H. Yang. Object tracking benchmark. TPAMI, 37(9):1834–1848, 2015. 1, 3

2015
[54]

Yu and Q

X. Yu and Q. Yu. Online structural learning with dense sam- ples and a weighting kernel. Pattern Recognition Letters , 105:59–66, 2018. 2

2018
[55]

X. Yu, Q. Yu, Y . Shang, and H. Zhang. Dense structural learning for infrared object tracking at 200+ frames per sec- ond. Pattern Recognition Letters, 100:152–159, 2017. 2

2017
[56]

Zhang, L

K. Zhang, L. Zhang, Q. Liu, D. Zhang, and M.-H. Yang. Fast visual tracking via dense spatio-temporal context learning. In ECCV, 2014. 2

2014
[57]

Zhang, D

L. Zhang, D. Bi, Y . Zha, S. Gao, H. Wang, and T. Ku. Robust and fast visual tracking via spatial kernel phase correlation ﬁlter. Neurocomputing, 204:77–86, 2016. 2

2016
[59]

Zhang, C

T. Zhang, C. Xu, and M.-H. Yang. Learning multi-task corre- lation particle ﬁlters for visual tracking. TPAMI, 41(2):365– 378, 2019. 2

2019
[60]

G. Zhu, F. Porikli, and H. Li. Beyond local search: Track- ing objects everywhere with instance-speciﬁc proposals. In CVPR, 2016. 2

2016
[61]

Z. Zhu, Q. Wang, B. Li, W. Wu, J. Yan, and W. Hu. Distractor-aware siamese networks for visual object track- ing. In ECCV, 2018. 1, 2, 7, 8

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [58]

Zhang, A

L. Zhang, A. Gonzalez-Garcia, J. van de Weijer, M. Danell- jan, and F. S. Khan. Synthetic data generation for end-to-end thermal infrared tracking. IEEE Transactions on Image Pro- cessing, 28(4):1837–1850, 2019. 2, 4, 5

work page 2019

[2] [1]

2, 5, 6, 7, 8

www.votchallenge.net/vot2019. 2, 5, 6, 7, 8

work page

[3] [2]

https://sites.google.com/view/ ahutracking001. 6

work page

[4] [3]

Bertinetto, J

L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, and P. H. Torr. Staple: Complementary learners for real-time tracking. In CVPR, pages 1401–1409, 2016. 8

work page 2016

[5] [4]

Bertinetto, J

L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. Fully-convolutional siamese networks for object tracking. In ECCV workshop, 2016. 1, 2, 6, 7, 8

work page 2016

[6] [5]

G. Bhat, M. Danelljan, L. Van Gool, and R. Timofte. Learn- ing discriminative model prediction for tracking. In ICCV,

work page

[7] [6]

S. T. Birchﬁeld and S. Rangarajan. Spatiograms versus his- tograms for region-based tracking. In CVPR, volume 2, pages 1158–1163. IEEE, 2005. 3

work page 2005

[8] [7]

D. S. Bolme, J. R. Beveridge, B. A. Draper, and Y . M. Lui. Visual object tracking using adaptive correlation ﬁlters. In CVPR, 2010. 1, 2

work page 2010