Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

Locality-constrained Spatial Transformer Network for Video Crowd Counting

T0 review · 2 major / 2 minor · reviewed 2026-05-24 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A locality-constrained spatial transformer estimates the next frame's density map from the current one to handle motion in video crowd counting.

desk verdict LSTN adds a locality-constrained spatial transformer to propagate per-frame density maps and releases a claimed largest video crowd dataset, but the abstract leaves unclear how the module handles count changes from entry/exit/occlusion. read the letter →

arxiv 1907.07911 v1 pith:6AWFLRZW submitted 2019-07-18 cs.CV

Yanyan Fang , Biyun Zhan , Wandi Cai , Shenghua Gao , Bo Hu This is my paper · ORCID
classification cs.CV
keywords videocrowdcountingspatialtransformerdensitymaplocalityconstrainttemporalrelationdataset
checked against Cost.FunctionalEquation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video crowd counting must deal with density map changes between neighboring frames caused by translation, rotation, scaling, and by people entering, exiting, or becoming occluded. The paper proposes LSTN, which first runs a CNN on each frame to produce a density map, then feeds that map into a Locality-constrained Spatial Transformer module to predict the following frame's map. This step creates a temporal link that compensates for the observed changes. The authors also release a dataset of 15K frames containing roughly 394K annotated heads from 13 scenes. Experiments on this collection and on prior datasets indicate the combined pipeline yields more accurate counts than frame-independent methods.

What carries the argument

The Locality-constrained Spatial Transformer (LST) module, which takes the current frame's density map and produces an estimate of the next frame's map under locality constraints.

What would settle it

A side-by-side test on the 15K-frame dataset that shows no reduction in counting error when the LST module is removed and each frame is processed independently by the CNN alone.

Watch

Extended reading notes

Core claim

LSTN generates per-frame density maps with a CNN and then applies a locality-constrained spatial transformer module that transforms the current density map to approximate the density map of the next frame, thereby relating neighboring maps to accommodate both geometric changes and variations in head count.

Load-bearing premise

Changes in head density maps between neighboring frames can be modeled and corrected as locality-constrained spatial transformations applied to the current map.

Editorial extensions

If this is right

  • Density maps can be propagated forward in time with spatial adjustments to maintain consistency despite crowd movement.
  • Entry, exit, and occlusion effects become addressable through the temporal relation between consecutive maps.
  • A single large video dataset with 394K annotations supplies a concrete testbed for measuring such temporal corrections.
  • The same architecture demonstrates gains on existing crowd counting collections beyond the newly collected scenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The LST module could be inserted into other video density tasks such as traffic flow or cell population tracking where local geometric shifts dominate.
  • Longer sequences might benefit from chaining multiple LST steps or combining them with recurrent connections to capture extended motion.
  • The released dataset invites direct comparisons of transformer-based temporal links against optical-flow or recurrent alternatives on the same 13 scenes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes a Locality-constrained Spatial Transformer Network (LSTN) for video crowd counting. A CNN first produces per-frame density maps; a Locality-constrained Spatial Transformer (LST) module then warps the current-frame density map to produce an estimate of the next frame's map, with the goal of handling translation/rotation/scaling as well as entry/exit and occlusion effects. The authors release a new video dataset (15 K frames, ~394 K heads, 13 scenes) and report that experiments on this and prior datasets demonstrate the effectiveness of LSTN.

Significance. If the LST module can be shown to correctly propagate density under non-rigid motion while also accounting for count changes, the approach would supply a lightweight temporal link between consecutive density maps without requiring explicit tracking or optical flow. The release of a large, multi-scene video counting dataset is a concrete contribution that future work can use for benchmarking.

major comments (2)
  1. [Abstract] Abstract: the LST module is described solely as estimating the next density map from the current one via a locality-constrained spatial transformer. Because a spatial transformer realizes a geometric warp, any change in integrated density (entry, exit, or occlusion) must be synthesized by the warp itself or by an auxiliary pathway; no equation, diagram, or loss term is supplied that would permit net mass creation or destruction, which directly undermines the claim that the module alleviates entry/exit/occlusion.
  2. [Abstract] Abstract (and implied method section): the central empirical claim rests on the assertion that the LST corrects for all listed sources of density-map change, yet the provided description contains neither the precise formulation of the locality constraint nor any ablation that isolates the contribution of the LST versus a plain CNN density estimator.
minor comments (2)
  1. [Abstract] The abstract states that the new dataset is 'the largest' but supplies no comparison table of existing video counting datasets (frame count, annotation density, scene diversity).
  2. [Abstract] No implementation details (backbone CNN, training schedule, loss weights, or inference procedure for combining the warped and observed maps) are given, making reproduction impossible from the current text.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the major comments point by point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the LST module is described solely as estimating the next density map from the current one via a locality-constrained spatial transformer. Because a spatial transformer realizes a geometric warp, any change in integrated density (entry, exit, or occlusion) must be synthesized by the warp itself or by an auxiliary pathway; no equation, diagram, or loss term is supplied that would permit net mass creation or destruction, which directly undermines the claim that the module alleviates entry/exit/occlusion.

    Authors: We agree that the abstract provides no equation, diagram, or loss term permitting net mass creation or destruction, and that a pure geometric warp cannot synthesize count changes from entry/exit/occlusion. The manuscript description does not supply an auxiliary pathway or explicit mechanism for these effects. We will revise the method section to clarify the integration of the per-frame CNN with the LST module and add discussion of how count changes are handled in practice. revision: yes

  2. Referee: [Abstract] Abstract (and implied method section): the central empirical claim rests on the assertion that the LST corrects for all listed sources of density-map change, yet the provided description contains neither the precise formulation of the locality constraint nor any ablation that isolates the contribution of the LST versus a plain CNN density estimator.

    Authors: We acknowledge that neither the abstract nor the implied method section supplies the precise mathematical formulation of the locality constraint or an ablation isolating the LST contribution. We will add the explicit formulation of the locality constraint to the method section and include a new ablation study comparing the full LSTN against a plain CNN baseline in the revised manuscript. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical NN architecture proposal with no derivation chain

full rationale

The paper proposes an LSTN architecture: a CNN estimates per-frame density maps, followed by an LST module that warps the current density map to estimate the next frame's map. No equations, uniqueness theorems, or first-principles derivations are presented that reduce to fitted parameters or self-citations by construction. Validation relies on training and testing on external video datasets (including a newly collected one), which are independent of any internal fit. This matches the default case of an empirical proposal whose central claims do not collapse to their inputs.

Assumptions & free parameters 0 free parameters · 1 assumptions · 1 invented entities

The central claim rests on the domain assumption that single-frame CNN density estimation is reliable and that local spatial transformations suffice to model temporal evolution of crowd density; the LST module itself is an invented component without independent evidence supplied in the abstract.

assumptions (1)
  • domain assumption Convolutional neural networks can reliably estimate crowd density maps from individual frames
    The paper begins by leveraging CNNs for per-frame density estimation before applying the LST module.
invented entities (1)
  • Locality-constrained Spatial Transformer (LST) module
    purpose: To estimate the next frame's density map from the current frame by constraining spatial transformations to local changes
    This is a newly proposed component introduced to address temporal inconsistencies in video density maps.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Locality-constrained Spatial Transformer Network for Video Crowd Counting." pith.science (2026). https://pith.science/paper/6AWFLRZW

@misc{pith2026190707911,
  author       = {Pith},
  title        = {Pith review of: Locality-constrained Spatial Transformer Network for Video Crowd Counting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6AWFLRZW}},
  note         = {Machine review of arXiv:1907.07911}
}
read the original abstract

Compared with single image based crowd counting, video provides the spatial-temporal information of the crowd that would help improve the robustness of crowd counting. But translation, rotation and scaling of people lead to the change of density map of heads between neighbouring frames. Meanwhile, people walking in/out or being occluded in dynamic scenes leads to the change of head counts. To alleviate these issues in video crowd counting, a Locality-constrained Spatial Transformer Network (LSTN) is proposed. Specifically, we first leverage a Convolutional Neural Networks to estimate the density map for each frame. Then to relate the density maps between neighbouring frames, a Locality-constrained Spatial Transformer (LST) module is introduced to estimate the density map of next frame with that of current frame. To facilitate the performance evaluation, a large-scale video crowd counting dataset is collected, which contains 15K frames with about 394K annotated heads captured from 13 different scenes. As far as we know, it is the largest video crowd counting dataset. Extensive experiments on our dataset and other crowd counting datasets validate the effectiveness of our LSTN for crowd counting.

Discussion (0). Continue with ORCID to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark

    cs.CV 2024-11 conditional novelty 6.0 of 10

    E-MAC, with density-guided masked modeling and optical-flow temporal fusion, achieves state-of-the-art MAE on four video counting benchmarks, including the new DroneBird bird dataset.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Locality-constrained Spatial Transformer Network for Video Crowd Counting

    INTRODUCTION Crowd counting has been widely used in computer vision be- cause of its potential applications in video surveillance, traffic control, and emergency management. However, most previ- ous works [1][2][3] focus on single image based crowd count- ing. In real applications, we have videos at hand, and usually the movement of crowd is predictable an...

    work page Pith review arXiv 1907

  2. [2]

    Crowd counting for single image

    RELA TED WORK Since our work is related to deep learning based crowd count- ing, here we only briefly discuss recent works on deep learn- ing based crowd counting. Crowd counting for single image. Recent works [3][9][10] have shown the effectiveness of CNN for density map estimation in single image crowd counting. To improve the robustness of crowd countin...

    work page

  3. [3]

    OUR APPROACH Our network architecture is shown in Fig. 1. It consists of two modules: density map regression module and Locality- constrained Spatial Transformer (LST) module. The density map regression module takes each frame as input and esti- mates its corresponding density map, and then the LST mod- ule takes the estimated density map as input to pred...

    work page 2000

  4. [4]

    EXPERIMENTS 4.1. Evaluation metric Following work [19], we adopt both the mean absolute error (MAE) and the mean squared error (MSE) as metrics to eval- uate the performance of different methods, which are defined as follows: MAE = 1 T T∑ i=1 |zi− ˆzi|,MSE = √ 1 T T∑ i=1 (zi− ˆzi)2 (9) where T is the total number of frames of all testing video sequences...

    work page

  5. [5]

    We also report the perfor- mance of our method without LST

    which achieves state-of-the-art performance for single im- age crowd counting, ConvLSTM [8] which is state-of-the-art video crowd counting method. We also report the perfor- mance of our method without LST. All results are shown in Table. 2. We can see that our method achieves the best per- formance. Further the improvement of our method compared with the...

    work page

  6. [6]

    Specifi- cally, we first leverage a density map regression module to es- timate the density map of each frame

    CONCLUSION In this paper, a Locality-constrained Spatial Transformer Net- work (LSTN) is proposed to explicitly relate the density maps of neighbouring frames for video crowd counting. Specifi- cally, we first leverage a density map regression module to es- timate the density map of each frame. Considering that people may walk in/out or are occluded, we div...

    work page

  7. [7]

    Fast crowd density estimation with convolutional neural net- works,

    M. Fu, P. Xu, X. Li, Q.Liu, M.Ye, and C.Zhu, “Fast crowd density estimation with convolutional neural net- works,” Engineering Applications of Artificial Intelli- gence, pp. 81 – 88, 2015

    work page 2015

  8. [8]

    Cross-scene crowd counting via deep convolutional neural networks,

    Cong Zhang, Hongsheng Li, Xiaogang Wang, and Xi- aokang Yang, “Cross-scene crowd counting via deep convolutional neural networks,” in CVPR, June 2015

    work page 2015

Show all 30 references
  1. [9]

    Single- image crowd counting via multi-column convolutional neural network,

    Y . Zhang, D. Zhou, S. Chen, S. Gao, and Y . Ma, “Single- image crowd counting via multi-column convolutional neural network,” in CVPR, June 2016, pp. 589–597

    2016

  2. [10]

    Context-aware trajectory prediction,

    B. Federico, L. Giuseppe, Ballan L, and A. Bimbo, “Context-aware trajectory prediction,” international conference on pattern recognition, 2017

    2017

  3. [11]

    Histograms of oriented gradi- ents for human detection,

    N. Dalal and B. Triggs, “Histograms of oriented gradi- ents for human detection,” pp. 886–893, 2005

    2005

  4. [12]

    Pedestrian detection via classification on riemannian manifolds,

    Oncel Tuzel, Fatih Porikli, and Peter Meer, “Pedestrian detection via classification on riemannian manifolds,” TPAMI, vol. 30, no. 10, pp. 1713–1727, 2008

    2008

  5. [13]

    Fcn-rlstm: Deep spatio-temporal neural networks for vehicle counting in city cameras,

    S. Zhang, G. Wu, J. P. Costeira, and J. M. F. Moura, “Fcn-rlstm: Deep spatio-temporal neural networks for vehicle counting in city cameras,” in ICCV, Oct 2017, pp. 3687–3696

    2017

  6. [14]

    Spatiotemporal model- ing for crowd counting in videos,

    X. Feng, X. Shi, and D. Yeung, “Spatiotemporal model- ing for crowd counting in videos,” inICCV. IEEE, 2017, pp. 5161–5169

    2017

  7. [15]

    Switching convolutional neural network for crowd counting,

    Deepak Babu Sam, Shiv Surya, and R. Venkatesh Babu, “Switching convolutional neural network for crowd counting,” in CVPR, July 2017

    2017

  8. [16]

    Csrnet: Dilated con- volutional neural networks for understanding the highly congested scenes,

    Y . Li, X. Zhang, and D. Chen, “Csrnet: Dilated con- volutional neural networks for understanding the highly congested scenes,” in CVPR, 2018, pp. 1091–1100

    2018

  9. [17]

    Towards perspective-free object counting with deep learning,

    Daniel D. Onoro-Rubio and R. L ´opez-Sastre, “Towards perspective-free object counting with deep learning,” in ECCV. Springer, 2016, pp. 615–629

    2016

  10. [18]

    Deci- denet: counting varying density crowds through atten- tion guided detection and density estimation,

    J. Liu, C. Gao, D. Meng, and A. Hauptmann, “Deci- denet: counting varying density crowds through atten- tion guided detection and density estimation,” in CVPR, 2018, pp. 5197–5206

    2018

  11. [19]

    Composition loss for counting, density map estimation and localization in dense crowds.,

    M. Tayyab H. Idrees, K. Athrey, D. Zhang, S. Al- maadeed, N. Rajpoot, and M. Shah, “Composition loss for counting, density map estimation and localization in dense crowds.,” arXiv: Computer Vision and Pattern Recognition, 2018

    2018

  12. [20]

    Spatial transformer networks,

    Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al., “Spatial transformer networks,” in Advances in neural information processing systems, 2015, pp. 2017– 2025

    2015

  13. [21]

    Su- pervised transformer network for efficient face detec- tion,

    Dong Chen, Gang Hua, Fang Wen, and Jian Sun, “Su- pervised transformer network for efficient face detec- tion,” in ECCV. Springer, 2016, pp. 122–138

    2016

  14. [22]

    To- ward end-to-end face recognition through alignment learning,

    Yuanyi Zhong, Jiansheng Chen, and Bo Huang, “To- ward end-to-end face recognition through alignment learning,” IEEE signal processing letters , vol. 24, no. 8, pp. 1213–1217, 2017

    2017

  15. [23]

    Recursive spatial transformer (rest) for alignment-free face recognition,

    Wanglong Wu, Meina Kan, Xin Liu, Yi Yang, Shiguang Shan, and Xilin Chen, “Recursive spatial transformer (rest) for alignment-free face recognition,” in CVPR, 2017, pp. 3772–3780

    2017

  16. [24]

    Crowd counting using deep recurrent spatial-aware network,

    Lingbo Liu, Hongjun Wang, Guanbin Li, Wanli Ouyang, and Liang Lin, “Crowd counting using deep recurrent spatial-aware network,” arXiv preprint arXiv:1807.00601, 2018

    2018 arXiv

  17. [25]

    Counting in dense crowds using deep features,

    Karunya Tota and Haroon Idrees, “Counting in dense crowds using deep features,” 2015

    2015

  18. [26]

    Privacy preserving crowd monitoring: Counting people without people models or tracking,

    A. B. Chan, Zhang-Sheng John Liang, and N. Vascon- celos, “Privacy preserving crowd monitoring: Counting people without people models or tracking,” in CVPR, June 2008, pp. 1–7

    2008

  19. [27]

    Face recognition us- ing kernel ridge regression,

    S. An, W. Liu, and S. Venkatesh, “Face recognition us- ing kernel ridge regression,” in CVPR, June 2007, pp. 1–7

    2007

  20. [28]

    Feature mining for localised crowd counting,

    Ke Chen, Chen Change Loy, Shaogang Gong, and Tao Xiang, “Feature mining for localised crowd counting,” in In BMVC

  21. [29]

    Cumulative attribute space for age and crowd density estimation,

    K. Chen, S. Gong, T. Xiang, and C. C. Loy, “Cumulative attribute space for age and crowd density estimation,” in CVPR, June 2013, pp. 2467–2474

    2013

  22. [30]

    Count forest: Co-voting uncertain number of targets using random forest for crowd density estimation,

    V . Pham, T. Kozakaya, O. Yamaguchi, and R. Okada, “Count forest: Co-voting uncertain number of targets using random forest for crowd density estimation,” in ICCV, Dec 2015, pp. 3253–3261

    2015

Pith tools

Reviewed May 24, 2026 · model on record in the stance chip above.