REVIEW 3 major objections 6 minor 37 references
Attentive Deep Regression Networks for Real-Time Visual Face Tracking in Video Surveillance
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read An attentive extension of GOTURN raises surveillance face-tracking score to 0.807 while running at about 143 FPS.
desk verdict A useful ChokePoint annotation release and a plausible attention-augmented GOTURN, but the headline comparison is not controlled: the baselines were not trained on the target dataset, so the large claimed margins are not established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is channel attention: feature maps from each of the five VGG-Face convolutional layers are max-pooled to $6\times6$, flattened, and passed through a two-layer multilayer perceptron whose biased sigmoid output produces one weight per channel; each channel's activations are then multiplied by that weight. The weighted channels are concatenated along the channel dimension and fed to a $1\times1$ convolutional fusion layer followed by three fully connected layers that regress the target box coordinates and size. This machinery turns the task of choosing which features matter for a given frame into a learned per-channel weighting, and it is what the ablations credit for the score gain.
What would settle it
Retrain the GOTURN and IVT baselines on the same ChokePoint training splits used for AFTN and rerun the paper's evaluation protocol; if the overall-score gap over GOTURN shrinks to roughly the attention ablation gap (0.807 versus 0.780), then the reported margin is explained by in-domain training rather than by the proposed architecture.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that AFTN, a fully offline-trained extension of the GOTURN tracker, achieves an overall score of 0.807 on the ChokePoint G1/G2 evaluation protocol, compared with 0.506 for GOTURN and 0.348 for IVT, while running at 142.9 FPS. The improvements come from feeding all five convolutional layers of a frozen VGG-Face into per-channel attention networks, concatenating the weighted features, and regressing the bounding box with a fusion layer followed by fully connected layers. Ablations show the attention mechanism adds 0.027 overall score over using the same features unweighted, and removing the previous-frame input costs 0.017 score but raises speed to 183.4 FPS.
Load-bearing premise
The comparison is fair only if AFTN's large margin comes from the architecture: AFTN is trained on the same ChokePoint splits used for evaluation, while GOTURN and IVT are not trained on ChokePoint at all, and the GOTURN baseline is modified to use VGG-Face features.
Editorial extensions
If this is right
- Surveillance face tracking can be done with an offline-trained deep network at 142.9 FPS, far above the 25 FPS real-time requirement, so no online model updates are needed during tracking.
- Using all VGG-Face feature levels with per-channel attention raises the overall score to 0.807 on ChokePoint, versus 0.780 for the same network without attention.
- Omitting the previous-frame input yields 183.4 FPS with only a 0.017 drop in overall score, giving a concrete speed-accuracy trade-off for deployment.
- The released bounding-box annotations for ChokePoint G1 and G2 allow other surveillance face trackers to be trained and compared on identical splits.
Reading between the lines
- The paper does not train the GOTURN or IVT baselines on the ChokePoint training splits; a fair test that does so would show how much of the 0.301 margin over GOTURN is architecture and how much is in-domain training.
- The reported pattern of channel weights (low-level features suppressed more in cluttered leaving scenes) suggests attention weights could double as a per-scene difficulty signal for surveillance operators, a use the paper leaves implicit.
- The speed of the no-previous-frame variant suggests the same design can serve multi-camera or low-power settings where per-frame latency matters more than the last point of accuracy; the paper only frames the choice as speed versus score.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes the Attentive Face Tracking Network (AFTN), an extension of the GOTURN tracker for real-time visual face tracking in video surveillance. AFTN uses frozen VGG-Face convolutional features from all five layers, per-layer channel-attention MLPs that reweight feature channels, a learned 1x1 convolutional fusion layer, and a regression head that predicts the bounding box in the current frame from crops of the previous and current frames. The network is trained offline on the G1/G2 splits of the ChokePoint dataset using a cross-split protocol, and is evaluated with accuracy/robustness scores based on thresholded overlap and failure-rate curves. The manuscript reports that AFTN outperforms IVT and GOTURN by large margins in overall score (0.807 vs 0.348 and 0.506), runs at about 143 FPS, and contributes new bounding-box annotations for ChokePoint.
Significance. If the empirical claims were properly controlled, the paper would make a useful contribution: it provides the first bounding-box annotations for ChokePoint for face tracking, demonstrates that an offline-trained regression tracker can run comfortably in real time for surveillance faces, and its internal ablations are consistent with the attention mechanism and previous-frame input having modest but positive effects. The speed results and the released annotations are valuable to the community. However, the headline comparison against GOTURN and IVT is confounded by training-domain differences and baseline modifications, so the paper's central claim of outperforming state-of-the-art trackers by very large margins is not yet supported by the evidence presented.
major comments (3)
- [§II-C, §II-D, §III (Table I)] The central claim that AFTN outperforms GOTURN and IVT by very large margins rests on Table I, but the comparison is not controlled. As described in §II-C and §II-D, AFTN is trained on the ChokePoint training split (G1 when testing on G2, and vice versa), whereas the GOTURN and IVT baselines are evaluated without any ChokePoint training. The reported gap (0.807 vs 0.506 vs 0.348) therefore conflates the proposed architecture with the substantial advantage of training on the target dataset. The internal ablation in Table I shows that removing attention changes the score by only 0.027 (0.807 to 0.780), which is small relative to the 0.301 gap over GOTURN. To support the abstract claim, the authors should retrain GOTURN (and ideally a VGG-Face variant and a no-fusion variant) on the same ChokePoint training split and repeat the evaluation; without this, the headline margin cannot be attributed to the proposed attentive architecture.
- [§III (Table I, GOTURN row)] The GOTURN baseline is not the original published GOTURN model: it replaces AlexNet with VGG-Face and is forced to output square boxes. Because §II-A lists the switch to VGG-Face features as one of the paper's proposed extensions, the modified baseline prevents the reader from isolating the effect of the attention and fusion mechanisms. Please report the original GOTURN (AlexNet, unconstrained box aspect ratio) or, if the square-box constraint is needed for face tracking, justify that the constraint does not handicap the baseline; otherwise the comparison should be explicitly described as being against a modified GOTURN rather than against GOTURN itself.
- [§III (Table I, Fig. 3)] The claims of 'very large margins' and of a 'significant drop' in overall score without attention are not supported by any statistical significance test. The table reports only means and standard deviations over sequences; since the same sequences are used for all trackers, paired comparisons (e.g., Wilcoxon signed-rank tests or per-sequence overlap differences) are needed to establish that AFTN beats the baselines and that the attention mechanism helps beyond chance. The current summary statistics alone do not justify the word 'significant' in the text.
minor comments (6)
- [Abstract and §I] The phrase 'is build on top of' should be 'is built on top of'.
- [§II-E] The text 'the ratio of frames where the region overlap fails below the RT' should read 'falls below the RT'.
- [§II-C] The sentence 'we annotate the frames in the G1 and G2 sets with bounding boxes and make the ChokePoint dataset suitable' is awkward; consider 'making the ChokePoint dataset suitable' for clarity.
- [Fig. 3] The text inside Figure 3 appears garbled or corrupted in the submitted PDF, including the legend entries and axis labels. Please ensure the figure is legible and the fonts render correctly.
- [§I and Abstract] Calling GOTURN 'state-of-the-art' is dated: GOTURN is from 2016, and the paper should either cite a more recent state-of-the-art tracker or qualify the claim to 'a widely used real-time tracker'.
- [§II-A and Ref. [27]] The detailed step-by-step building procedure is deferred to the MSc thesis [27]. Since a thesis is not a peer-reviewed publication, the paper should include the key architectural details (for example, the exact way each layer's pooled feature maps are flattened and concatenated before the MLP, and the order of concatenation in the FAN) or provide them in a supplementary document.
Circularity Check
No circularity: the empirical benchmark is self-contained; self-citations are descriptive and not load-bearing.
full rationale
The paper's central claim is that the proposed AFTN outperforms GOTURN and IVT on the ChokePoint dataset. This claim is supported by direct empirical evaluation in Table I and Fig. 3, using accuracy, robustness, overall score, and speed measured on an external dataset with newly provided bounding-box annotations. No predicted quantity is defined in terms of a fitted parameter, and no equation reduces to another equation by construction. The channel attention weights are learned during offline training and then reported as behavior analysis; they are not used to define the evaluation scores. The self-references present in the paper are an adopted fusion network from earlier work by the same group (Akkaya and Halici) and a pointer to the first author's MSc thesis for implementation details. These are descriptive background choices, not load-bearing justifications for the reported performance, and they do not constitute a self-citation chain that forces the results. The potential concern that GOTURN and IVT baselines were not trained on ChokePoint while AFTN was is a fairness/validity issue for the empirical comparison, not a circularity in the derivation. Therefore no specific circular step can be exhibited, and the paper is self-contained against external benchmarks for the purpose of this analysis.
Assumptions & free parameters
free parameters (9)
- learning_rate =
1e-5
- L2_regularization_penalty =
1e-3
- batch_size =
50
- training_epochs =
10
- attention_MLP_hidden_units =
36
- sigmoid_output_bias =
0.5
- dropout_rate =
0.5
- fusion_convolution_kernels =
256
- crop_expansion_factor =
2x previous bounding box
assumptions (6)
- domain assumption Frozen VGG-Face features trained for face recognition generalize to ChokePoint surveillance faces.
- domain assumption The manually corrected face-detector bounding boxes on ChokePoint are accurate enough to serve as ground truth for training and evaluation.
- domain assumption The G1/G2 cross-evaluation protocol measures generalization to unseen surveillance conditions.
- domain assumption L1 loss on bounding box coordinates is a suitable training objective for face tracking.
- domain assumption Combining low-level and high-level convolutional features is useful for visual tracking.
- domain assumption The baseline GOTURN and IVT variants are representative state-of-the-art trackers for comparison.
Cite this review
Pith. "Pith review of Attentive Deep Regression Networks for Real-Time Visual Face Tracking in Video Surveillance." pith.science (2026). https://pith.science/paper/ABALOTML
@misc{pith2026190803812,
author = {Pith},
title = {Pith review of: Attentive Deep Regression Networks for Real-Time Visual Face Tracking in Video Surveillance},
year = {2026},
howpublished = {\url{https://pith.science/paper/ABALOTML}},
note = {Machine review of arXiv:1908.03812}
}
read the original abstract
Visual face tracking is one of the most important tasks in video surveillance systems. However, due to the variations in pose, scale, expression, and illumination it is considered to be a difficult task. Recent studies show that deep learning methods have a significant potential in object tracking tasks and adaptive feature selection methods can boost their performance. Motivated by these, we propose an end-to-end attentive deep learning based tracker, that is build on top of the state-of-the-art GOTURN tracker, for the task of real-time visual face tracking in video surveillance. Our method outperforms the state-of-the-art GOTURN and IVT trackers by very large margins and it achieves speeds that are very far beyond the requirements of real-time tracking. Additionally, to overcome the scarce data problem in visual face tracking, we also provide bounding box annotations for the G1 and G2 sets of ChokePoint dataset and make it suitable for further studies in face tracking under surveillance conditions.
Figures
Reference graph
Works this paper leans on
-
[1]
Incremental learning for robust visual tracking,
D. A. Ross, J. Lim, R.-S. Lin, and M.-H. Yang, “Incremental learning for robust visual tracking,” Int. J. Comput. Vision , vol. 77, no. 1-3, pp. 125–141, May 2008. [Online]. Available: http://dx.doi.org/10.1007/ s11263-007-0075-7
work page 2008
-
[2]
Z. Kalal, K. Mikolajczyk, and J. Matas, “Tracking-learning-detection,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 34, no. 7, pp. 1409–1422, Jul. 2012. [Online]. Available: http://dx.doi.org/10.1109/TPAMI.2011. 239
-
[3]
Online discriminative object tracking with local sparse representation,
Q. Wang, Feng Chen, Wenli Xu, and M. Yang, “Online discriminative object tracking with local sparse representation,” in2012 IEEE Workshop on the Applications of Computer Vision (WACV), Jan 2012, pp. 425–432
work page 2012
-
[4]
A comparison of adaptive appearance methods for tracking faces in video surveillance,
M. A. A. Dewan, E. Granger, F. Roli, R. Sabourin, and G. L. Marcialis, “A comparison of adaptive appearance methods for tracking faces in video surveillance,” in 5th International Conference on Imaging for Crime Detection and Prevention (ICDP 2013) , Dec 2013, pp. 1–7
work page 2013
-
[5]
A super- fast online face tracking system for video surveillance,
X. Lan, Z. Xiong, W. Zhang, S. Li, H. Chang, and W. Zeng, “A super- fast online face tracking system for video surveillance,” in 2016 IEEE International Symposium on Circuits and Systems (ISCAS) , May 2016, pp. 1998–2001
work page 2016
-
[6]
Deep learning,
Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, p. 436, 2015
2015
-
[7]
Deep learning in neural networks: An overview,
J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Networks , vol. 61, pp. 85–117, 2015, published online 2014; based on TR arXiv:1404.7828 [cs.NE]
arXiv 2015
-
[8]
Imagenet classification with deep convolutional neural networks,
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Infor- mation Processing Systems , p. 2012
work page 2012
Show all 37 references
-
[9]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” CoRR, vol. abs/1409.1556, 2014
2014 arXiv
-
[10]
Rich feature hierarchies for accurate object detection and semantic segmentation,
R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” 2014 IEEE Conference on Computer Vision and Pattern Recognition , pp. 580–587, 2014
2014
-
[11]
Decoupled deep neural network for semi-supervised semantic segmentation,
S. Hong, H. Noh, and B. Han, “Decoupled deep neural network for semi-supervised semantic segmentation,” in NIPS, 2015
2015
-
[12]
Fully convolutional networks for semantic segmentation,
E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 39, no. 4, pp. 640–651, Apr. 2017. [Online]. Available: https://doi.org/10.1109/TPAMI.2016.2572683
2017
-
[13]
L.: Deepface: Closing the gap to human-level performance in face verification,
Y . Taigman, M. Yang, and L. Wolf, “L.: Deepface: Closing the gap to human-level performance in face verification,” in In: IEEE CVPR, 2014
2014
-
[14]
The visual object tracking vot2015 challenge results,
M. Kristan, J. Matas, A. Leonardis, M. Felsberg, L. ˇCehovin Zajc, G. Fernandez, and et al., “The visual object tracking vot2015 challenge results,” in Visual Object Tracking Workshop 2015 at ICCV2015 , Dec 2015
2015
-
[15]
The visual object tracking vot2016 challenge results,
M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pflugfelder, L. ˇCehovin Zajc, T. V ojir, G. H ¨ager, A. Luke ˇziˇc, G. Fernandez, and et al., “The visual object tracking vot2016 challenge results,” Springer, Oct 2016. [Online]. Available: http://www.springer.com/gp/ book/9...
2016
-
[16]
The visual object tracking vot2017 challenge results,
M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pflugfelder, L. ˇCehovin Zajc, T. V ojir, G. H ¨ager, A. Luke ˇziˇc, A. Eldesokey, G. Fernandez, and et al., “The visual object tracking vot2017 challenge results,” 2017. [Online]. Avail- able: http://openaccess.thecvf.com/con...
2017
-
[17]
The sixth visual object tracking vot2018 challenge results,
M. Kristan, A. Leonardis, J. Matas, M. Felsberg, R. Pfugfelder, L. C. Zajc, T. V ojir, G. Bhat, A. Lukezic, A. Eldesokey, G. Fernandez, and et al., “The sixth visual object tracking vot2018 challenge results,” 2018
2018
-
[18]
A twofold siamese network for real-time object tracking,
A. He, C. Luo, X. Tian, and W. Zeng, “A twofold siamese network for real-time object tracking,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 4834–4843, 2018
2018
-
[19]
Learning attentions: Residual attentional siamese network for high performance online visual tracking,
Q. Wang, Z. Teng, J. Xing, J. Gao, W. Hu, and S. Maybank, “Learning attentions: Residual attentional siamese network for high performance online visual tracking,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018
2018
-
[20]
Learning to track at 100 fps with deep regression networks,
D. Held, S. Thrun, and S. Savarese, “Learning to track at 100 fps with deep regression networks,” in European Conference Computer Vision (ECCV), 2016
2016
-
[21]
A survey on leveraging deep neural networks for object tracking,
S. Krebs, B. Duraisamy, and F. Flohr, “A survey on leveraging deep neural networks for object tracking,” 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) , pp. 411–418, 2017
2017
-
[22]
Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition,
Y . Wong, S. Chen, S. Mau, C. Sanderson, and B. C. Lovell, “Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition,” in IEEE Biometrics Workshop, Computer Vision and Pattern Recognition (CVPR) Workshops . IEEE, June 2011...
2011
-
[23]
Hierarchical convolutional features for visual tracking,
C. Ma, J.-B. Huang, X. Yang, and M.-H. Yang, “Hierarchical convolutional features for visual tracking,” in Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) , ser. ICCV ’15. Washington, DC, USA: IEEE Computer Society, 2015, pp. 3074–
2015
-
[24]
Visual tracking with fully convolutional networks,
L. Wang, W. Ouyang, X. Wang, and H. Lu, “Visual tracking with fully convolutional networks,” 2015 IEEE International Conference on Computer Vision (ICCV) , pp. 3119–3127, 2015
2015
-
[25]
Mouse Face Tracking Using Convolutional Neural Networks,
I. B. Akkaya, “Mouse Face Tracking Using Convolutional Neural Networks,” M.Sc. Thesis, Department of Electrical and Electronics Engineering, Middle East Technical University, Ankara, Turkey, 2016
2016
-
[26]
Mouse face tracking using convolutional neural networks,
I. B. Akkaya and U. Halici, “Mouse face tracking using convolutional neural networks,” IET Computer Vision , vol. 12, no. 2, pp. 153–161, 2018
2018
-
[27]
Attentive Deep Regression Networks for Real-Time Visual Face Tracking in Video Surveillance,
S. Alver, “Attentive Deep Regression Networks for Real-Time Visual Face Tracking in Video Surveillance,” M.Sc. Thesis, Department of Elec- trical and Electronics Engineering, Middle East Technical University, Ankara, Turkey, 2019
2019
-
[28]
Deep face recognition,
O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference , 2015
2015
-
[29]
Batch normalization: Accelerating deep network training by reducing internal covariate shift,
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” CoRR, vol. abs/1502.03167, 2015. [Online]. Available: http://arxiv.org/abs/1502. 03167
2015 arXiv
-
[30]
Dropout: A simple way to prevent neural networks from overfitting,
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014. [Online]. Available: http://jmlr.org/papers/v15/srivastava14a.html
1929
-
[31]
Dlib-ml: A machine learning toolkit,
D. E. King, “Dlib-ml: A machine learning toolkit,” Journal of Machine Learning Research, vol. 10, pp. 1755–1758, 2009
2009
-
[32]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2014. [Online]. Available: http://arxiv.org/abs/1412.6980
2014 arXiv
-
[33]
Automatic differentiation in pytorch,
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” in NIPS-W, 2017
2017
-
[34]
Is my new tracker really better than yours?
L. ˇCehovin Zajc, M. Kristan, and A. Leonardis, “Is my new tracker really better than yours?” in WACV 2014: IEEE Winter Conference on Applications of Computer Vision . IEEE, Mar 2014. [Online]. Available: http://prints.vicos.si/publications/302
2014
-
[35]
Visual object tracking per- formance measures revisited,
L. Cehovin, A. Leonardis, and M. Kristan, “Visual object tracking per- formance measures revisited,” IEEE Transactions on Image Processing , vol. 25, pp. 1261–1274, 2016
2016
-
[36]
Imagenet classification with deep convolutional neural networks,
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Infor- mation Processing Systems (NIPS 2012) , 2012, p. 4
2012
-
[3082]
Available: http://dx.doi.org/10.1109/ICCV .2015.352
[Online]. Available: http://dx.doi.org/10.1109/ICCV .2015.352
2015 doi
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.