{"id":"57ed69da-a998-4fd1-a2be-65e4fd781bd9","arxiv_id":"2508.03749","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CCTV images alone can estimate urban rail platform crowding, with calibrated semantic segmentation outperforming detection, head counting, and classification on WMATA data.","lead":"This paper tests whether security camera footage can estimate how crowded a subway platform is, comparing six computer vision methods on 43,000 images from Washington DC stations. The best method, a segmentation model with a learned pixel-weighting step, predicted platform occupancy with the lowest errors of all approaches tested.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported accuracy is measured against ODX occupancy estimates that the authors themselves describe as noisy and unvalidated; if ODX error is non-random, the calibrated DeepLabV3 advantage and all R2/MAE values may reflect predicting ODX bias rather than true platform occupancy.","rationale":"The reader's conditional verdict is well placed, and my audit does not move it. The most load-bearing assumption is the validity of ODX as ground truth, and the paper itself flags this in Limitations. I considered other issues—the 'MILP' formulation is in fact a continuous convex quadratic program solved by ECOS/CVXPY rather than a mixed-integer program, the Crowd-ViT test accuracy is modest, and no code or data are released—but none is as decisive as the target-variable problem. The paper has genuine strengths: fixed camera views, stratified splits, event-level temporal aggregation, multiple off-the-shelf CV baselines, and a real WMATA dataset. However, independent support for the quantitative accuracy claim is limited: no manual-count validation, no error bars, and no external benchmark against true occupancy. A manual-count study on a modest subset is feasible because the video data already exist and the authors have an active WMATA collaboration; it would directly settle whether reported R2 values reflect real occupancy or ODX artifacts. Until that check is performed, the operational conclusion should remain conditional on ODX being a reasonable proxy, which the authors themselves doubt.","tokens_in":15930,"tokens_out":2858,"duration_ms":35539,"concrete_test":"Select roughly 200 train-arrival events stratified by platform and crowd class; have trained annotators count distinct alighting passengers on the platform from the three sampled frames using a written protocol. Compare these manual counts with the ODX estimates, reporting bias, variance, and correlation with occupancy level. Then recompute Table 2 metrics for the calibrated DeepLabV3 and LGBM models using the manual counts as the target. If R2 against manual counts drops materially below the ODX-based R2 (for example, from 0.64 to below 0.5), or if ODX error correlates with model predictions, the claimed accuracy does not transfer to true occupancy. If R2 against manual counts remains comparable, the ground-truth concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—'calibrated DeepLabV3 segmentation approach outperforms other models across all key metrics' (Table 2) and 'accurate, near real-time estimation'—depends entirely on the target variable. ODX-derived occupancy is used as ground truth for every per-image label y_i^a in the MILP calibration, for the Crowd-ViT class thresholds, for LGBM training targets, and for all reported R2, MAE, wMAE, and 95th AE values. The Limitations section explicitly concedes that 'ODX is not designed to provide highly accurate estimates of rail platform occupancy, and likely introduces unquantifiable noise and potential misestimation in our study, particularly at busy transfer stations.' The dataset is concentrated at Metro Center and Gallery Place–Chinatown, transfer stations where ODX uncertainty is highest. If ODX error is not mean-zero noise but is correlated with passenger counts, platform type, or train-arrival timing, then the learned weight maps and LGBM models are calibrated to a biased target; high R2 against ODX can coexist with poor accuracy against actual platform occupancy. In addition, ODX estimates correspond to a retroactively reconstructed 'magic moment' defined by tap-in/out data and AVL, not directly observed, and the paper provides no validation that the three-frame sampling window is aligned with that moment. The reported numbers therefore do not yet support the operational conclusion that CCTV alone provides accurate occupancy estimates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates whether CCTV imagery can estimate platform occupancy on WMATA rail platforms. The authors build a 43,232-image, 18-camera dataset from seven platforms, label frames with occupancy estimates from WMATA's ODX trip-assignment algorithm, and compare object detectors (YOLOv11, RT-DETRv2, APGCC with SAHI), a fine-tuned Vision Transformer for crowd-level classification, and DeepLabV3 semantic segmentation with a novel MILP-based calibration that learns camera-specific weight maps. Performance is assessed per image, per train-arrival event, and per 15-minute bin; an LGBM model fuses per-camera features. The headline result is that calibrated DeepLabV3 achieves the best mean metrics at event level (Table 2, MAE 56.42, R2 0.64) and at 15-minute level (Table 3, MAE 54.10). The authors argue that CCTV alone can provide accurate near-real-time platform crowding estimates.","tokens_in":16247,"tokens_out":6131,"duration_ms":73858,"significance":"If the accuracy numbers are taken at face value, the work is useful: it is a large real-world CCTV benchmark, compares modern CV methods under a common label source, and introduces an interpretable, efficient calibration for segmentation maps that could be operationalized by agencies with occupancy labels. The paper is honest about several limitations, including ODX label noise, data imbalance, and center-platform degradation. However, because every reported metric is computed against an ODX target that the authors themselves describe as containing 'unquantifiable noise' and 'potential misestimation,' the headline claim is conditional on an unvalidated ground truth. The stress-test concern about biased ODX error is not a circularity problem but a construct-validity problem: the models may be learning to predict ODX bias rather than true occupancy. The practical recommendation is appropriate only after independent validation.","major_comments":[{"comment":"The evaluation's target variable is ODX-derived event-level occupancy, which the authors concede 'is not designed to provide highly accurate estimates of rail platform occupancy, and likely introduces unquantifiable noise and potential misestimation in our study, particularly at busy transfer stations.' Since ODX labels are used to train the MILP calibration, to set Crowd-ViT class thresholds, to train the LGBM fusion model, and to compute all R2, MAE, wMAE, and 95th AE values, the reported numbers measure agreement with ODX, not with true platform occupancy. The dataset is concentrated at Metro Center and Gallery Place–Chinatown, both transfer stations where ODX uncertainty is acknowledged to be highest. The authors should validate a stratified random sample of frames or train-arrival events against manual counts or another independent measurement, report the error characteristics of ODX on that sample, and show how Table 2 and Table 3 change under plausible label noise. Without this, the operational claim of 'accurate, near real-time estimation' is not supported.","section":"Ground Truth Occupancy Data; Limitations & Discussion"},{"comment":"The same event-level ODX occupancy value is assigned to all three sampled frames at -5s, 0s, and +5s around the magic moment, even though the visual crowding changes across those frames, as Figure 3 itself notes for the +5s frame. This injects structured label noise into both the per-image evaluation and the MILP calibration, which is learned per image. The paper should either use a single frame per event, model the time offset, or validate frame-level labels on a subset. At minimum, the authors should report whether the per-image results and the learned weight maps change if only the center frame is used for calibration.","section":"Data Acquisition and Sampling; Figure 3"},{"comment":"The MILP formulation depends on free choices such as the pooling block size (p,q), the L2 regularization weight lambda, the detection confidence threshold (30%), and the crowd-level percentile thresholds, yet no sensitivity analysis or uncertainty quantification is reported. Table 2 reports point estimates from what appears to be a single stratified split, and Figure 17 shows that calibrated DeepLabV3 achieves R2 = 0.59 on A01-1, 0.89 on A01-2, and 0.29 on C01-C; on C01-C it is not the best model. The claim that 'the calibrated DeepLabV3 segmentation approach outperforms other models across all key metrics' is an average over platforms and should be qualified with per-platform metrics, confidence intervals, or repeated split evaluation.","section":"Segmentation Map Weighting; Tables 2 and 3"}],"minor_comments":[{"comment":"The crowd-level thresholds are platform-specific percentiles computed on 2023 weekday occupancy after excluding zeros, so the 'empty' class can include occupancy up to the 50th percentile on a busy platform; the paper should clarify this when interpreting the Crowd-ViT confusion matrices and the operational meaning of each class.","section":"Crowd Level Label Generation"},{"comment":"The wMAE weights depend on the observed y_t; this is acceptable for an offline evaluation metric, but the text should state explicitly that the metric is not computable in real time and is intended for retrospective assessment.","section":"Performance Evaluation, Eq. (7)"},{"comment":"Equation (2) uses A_c^t without a clear definition of the indexing conventions, Equation (3) capitalizes 'Where' mid-sentence, and Figure 14/16 captions refer to 'Gallery Place Chinatown Yellow/Green line C01-1' while Table 1 uses C01-C; these should be unified.","section":"Notation and captions"},{"comment":"The table lists training time as N/A for YOLOv11, RT-DETRv2, and APGCC because they are pre-trained; a table note should explicitly say this to avoid the impression that training time was not measured.","section":"Operational Integration, Table 4"}],"recommendation":"major_revision","confidential_remarks":"The key issue is ground-truth validation. The ODX limitation is acknowledged openly, so this is not a case of hidden weakness, but the manuscript's central accuracy claims cannot be accepted without an independent check of ODX against manual counts or an equivalent reference. I recommend major revision rather than rejection because the gap is addressable within the paper's scope, and the dataset and MILP contribution are potentially valuable to the transportation-vision community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid applied paper with a genuinely useful new dataset, but the headlined accuracy numbers are only as good as the ODX ground truth, which the authors themselves describe as noisy and not designed for this purpose. The methodological novelty is smaller than claimed.\n\nWhat it does well: the WMATA CCTV dataset is real and useful: 43,232 images from 18 cameras on seven platforms, with per-train-arrival event labels derived from WMATA's ODX algorithm. The evaluation is systematic: three aggregation levels, six models, transparent metrics, and an honest limitations section. The segmentation calibration pipeline is simple and efficient; it makes sense to weight image regions learned from data. The LGBM aggregation across cameras is a reasonable way to fuse multi-view estimates, and the authors are careful to avoid leakage for most platforms.\n\nWhere it is soft: the ground-truth concern is not a footnote, it is load-bearing. ODX is a retrospective trip-assignment model, not an occupancy sensor. Labeling frames at -5s, 0s, and +5s with the same event-level occupancy papers over timing and visual mismatch. If ODX's error is correlated with actual crowding, the reported R2 values may mostly reflect how well the model reproduces ODX's reconstruction. The authors concede this in Limitations, but then the conclusion says CCTV 'can provide accurate, near real-time estimation.' That jump is not supported. A validation set of manual counts, even a few hundred frames, would settle it. Without it, the numbers are promising but not operational evidence. Second, the 'MILP' label is wrong: the optimization is a continuous convex quadratic program (ridge regression on max-pooled segmentation maps), with no integer variables. It should be called a weighted ridge calibration. Minor, but accuracy matters. Third, no error bars, no significance tests, and very small samples on several platforms; the per-platform results are scattered. Code and data are not released, which limits reproducibility despite the privacy constraints.\n\nThe circularity concern raised in the notes is mild: training and testing on held-out ODX labels is standard supervised learning, not circular.\n\nBottom line: worth a serious referee. The dataset and benchmark are valuable, and the ground-truth problem is fixable with a modest manual-count validation. I would recommend acceptance after that validation and a corrected terminology.","headline":"Useful dataset and benchmark for CCTV-based platform crowding, but the accuracy claims rest on an unvalidated ODX ground truth and the 'MILP' calibration is really ridge regression.","tokens_in":16826,"tokens_out":2750,"would_cite":true,"duration_ms":32901,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CCTV footage alone can estimate urban rail platform crowding in near real time","keywords":["computer vision","CCTV","urban rail","platform crowding","occupancy estimation","semantic segmentation","DeepLabV3","object detection"],"falsifier":"Take a set of held-out train-arrival events at a busy transfer platform, count actual waiting passengers from the same CCTV frames by hand, and compare those counts to ODX and to the calibrated DeepLabV3 predictions. If manual counts disagree with ODX by more than the model's reported mean absolute error of about 56 passengers, then the reported accuracy is an artifact of the reference standard rather than a measure of real crowding.","tokens_in":15734,"feed_emoji":"🚇","tokens_out":7137,"duration_ms":76579,"temperature":0.7,"pith_summary":"This paper aims to establish that CCTV camera images alone—without fare-card data, additional sensors, or human observers—can produce accurate, near real-time estimates of how crowded an urban rail platform is. It compares object detection (YOLOv11, RT-DETRv2, APGCC), a custom crowd-level classifier (Crowd-ViT), and semantic segmentation (DeepLabV3) on a privacy-preserving dataset of more than 43,000 frames from WMATA platforms and reports that the calibrated segmentation approach is the most accurate. Across the studied platforms the calibrated model reaches $R^2 = 0.64$ with a mean absolute error of 56.42 passengers per train-arrival event, and several platforms with favorable camera views exceed $R^2 = 0.8$ after post-processing. If correct, transit agencies could turn existing surveillance cameras into a crowding sensor network independent of other data sources. The paper's reference standard is ODX, a post-hoc trip-assignment reconstruction that the authors acknowledge is noisy at busy transfer stations.","feed_headline":"CCTV alone can estimate rail platform crowding in near real time","feed_subtitle":"Calibrated segmentation hit R2=0.64 per arrival and topped 0.8 on favorable platforms using only images.","key_machinery":"The load-bearing mechanism is the MILP-calibrated segmentation weight map. DeepLabV3 produces a binary person/background map for each frame; the map is maximum-pooled into blocks, and the calibration solves a regularized ridge regression that learns a weight per block, so that the weighted sum of pooled person-pixel activations approximates the ODX occupancy label. The learned weights concentrate near train entrances and in regions far from the camera, which is how the method accounts for depth and passenger dispersion along the platform. Supporting machinery includes ODX as the labeling source, three-frame sampling around each train's 'magic moment' of peak platform crowding, and LightGBM aggregation that fuses per-camera features into a single platform-level estimate.","core_discovery":"On the paper's own terms, the central discovery is that a platform's occupancy can be read directly from its CCTV frames, and that the most reliable reading comes not from counting bodies or heads but from segmenting pixels into person/background and calibrating those activations with a learned spatial weight map. The calibrated DeepLabV3 pipeline achieves the best per-train-arrival occupancy estimates of all tested methods (MAE 56.42, $R^2=0.64$), improves weighted mean absolute error by 7.37% over the next-best model, and stays competitive at the 15-minute aggregate level. The paper also shows that raw segmentation pixel counts and pixel ratios are strong predictors on their own, that detection-based methods systematically undercount dense scenes, and that aggregating multiple frames and camera views reduces error substantially.","pith_inferences":["Editorial: If validated against manual counts, the same CCTV pipeline could be turned around to audit and correct ODX itself, converting a labeled noise source into a calibration target.","Editorial: The learned weight maps reveal where passengers queue, so the method could double as a tool for platform design decisions such as door allocation and barrier placement, not just real-time monitoring.","Editorial: Because the image-derived estimates are near real-time and univariate, they are a natural feed for short-term predictive models that fuse AFC tap data with visual crowding, though the paper does not test that combination.","Editorial: A testable extension is whether per-camera weight maps transfer to other stations with similar camera geometry, which would remove the per-platform calibration requirement."],"forward_implications":["A transit agency with CCTV and historical occupancy labels could deploy the calibrated segmentation pipeline for near real-time platform crowding estimates without installing new sensors.","Agencies without occupancy labels can still use pre-trained detection models such as RT-DETRv2 with SAHI, which balance accuracy and cost, and later fine-tune them with manual labels.","Sampling several frames around each train arrival and combining multiple camera views materially improves accuracy, so sampling strategy is as important as model choice.","The image-derived occupancy estimates can serve as input features for crowding prediction and operational decision support, filling a gap left by models built on indirect proxies such as fare-card data.","Even uncalibrated segmentation features are strong predictors, so threshold-based crowding monitoring is feasible without any calibration data."],"supporting_citations":[{"why":"The ODX algorithm generates the platform-occupancy values used to label every image and to score every model.","marker":"[27]"},{"why":"Defines DeepLabV3, the semantic segmentation model whose person-class maps feed the calibration pipeline.","marker":"[19]"},{"why":"Provides the atrous-convolution formulation of DeepLabV3 used in the experiments.","marker":"[26]"},{"why":"SAHI patch-based inference is what makes the detection baselines usable on noisy, distorted CCTV frames.","marker":"[16]"},{"why":"APGCC supplies the head-counting baseline against which segmentation calibration is compared.","marker":"[13]"},{"why":"YOLOv11 is one of the two object-detection baselines whose counts are compared with calibrated segmentation.","marker":"[24]"},{"why":"RT-DETRv2 is the other detection baseline and the strongest non-segmentation competitor.","marker":"[25]"},{"why":"The Vision Transformer architecture underlies the Crowd-ViT classification baseline.","marker":"[17]"},{"why":"LightGBM aggregation converts per-camera CV features into platform-level occupancy estimates in the evaluation.","marker":"[37]"},{"why":"The ECOS solver is used to compute the MILP-learned calibration weight map.","marker":"[35]"}],"fun_headline_variants":["Segmentation beats headcounts for CCTV-based crowding estimates","Calibrated CCTV segmentation achieves R²=0.64 for platform occupancy","CCTV pixel segmentation outperforms detection for crowd estimation","Privacy-preserving CCTV yields top crowd estimates via segmentation","Calibrated segmentation cuts platform crowd error by 7.4%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reference standard for 'true' occupancy is ODX, a trip-assignment algorithm that reconstructs passenger paths after the fact and that the authors say is noisy and potentially wrong at busy transfer stations; if those estimates are off, every reported accuracy number is off with them.","fun_headline_variants_meta":{"raw":{"variants":["Segmentation beats headcounts for CCTV-based crowding estimates","Calibrated CCTV segmentation achieves R²=0.64 for platform occupancy","CCTV pixel segmentation outperforms detection for crowd estimation","Privacy-preserving CCTV yields top crowd estimates via segmentation","Calibrated segmentation cuts platform crowd error by 7.4%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00023,"raw_usage":{"total_tokens":1505,"prompt_tokens":992,"completion_tokens":513,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":427}},"tokens_in":608,"tokens_out":513,"duration_ms":6916,"temperature":1.0,"reasoning_tokens":427,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:26:05.651315+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of held-out train-arrival events at a busy transfer platform, count actual waiting passengers from the same CCTV frames by hand, and compare those counts to ODX and to the calibrated DeepLabV3 predictions. If manual counts disagree with ODX by more than the model's reported mean absolute error of about 56 passengers, then the reported accuracy is an artifact of the reference standard rather than a measure of real crowding.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The ODX algorithm generates the platform-occupancy values used to label every image and to score every model."},{"cited_title":"Papandreou, I","cited_arxiv_id":null,"evidence_quote":"Defines DeepLabV3, the semantic segmentation model whose person-class maps feed the calibration pipeline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SAHI patch-based inference is what makes the detection baselines usable on noisy, distorted CCTV frames."},{"cited_title":"Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidance","cited_arxiv_id":"2405.10589","evidence_quote":"APGCC supplies the head-counting baseline against which segmentation calibration is compared."},{"cited_title":"Chaurasia, and J","cited_arxiv_id":null,"evidence_quote":"YOLOv11 is one of the two object-detection baselines whose counts are compared with calibrated segmentation."},{"cited_title":"Beyer, A","cited_arxiv_id":null,"evidence_quote":"The Vision Transformer architecture underlies the Crowd-ViT classification baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LightGBM aggregation converts per-camera CV features into platform-level occupancy estimates in the evaluation."},{"cited_title":"Chu, and S","cited_arxiv_id":null,"evidence_quote":"The ECOS solver is used to compute the MILP-learned calibration weight map."}],"review_version":1}