REVIEW 2 major objections 6 minor 45 references
Large-scale School Mapping using Weakly Supervised Deep Learning for Universal School Connectivity
T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Classification-level labels alone can locate schools at GPS coordinate precision, with ensembled ViT/CNN models scoring AUPRC above 0.96 across ten African countries.
desk verdict Useful applied school-mapping pipeline, but the GPS-level localization claim is unvalidated and the AUPRC numbers are inflated by test-set model selection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is two-stage weak supervision. First, an image classifier ensemble, VSC-Ensemble, averages the softmax outputs of the best vision transformer, Swin transformer, and ConvNeXt models trained on 300-by-300-meter tiles with a fixed 1:2 positive-to-negative ratio and hard 300-meter sample spacing to prevent leakage. Second, weakly supervised localization uses class activation maps: the xy-coordinate of the highest-attention pixel is converted into a lat-lon coordinate, requiring only the classification label as supervision. Attribution quality is scored with the ROAD framework, which measures average confidence drop after perturbing the top 10% of pixels in most-relevant-first order, letting the authors pick the best CAM variant per country.
What would settle it
Hold out a set of tiles whose true school-building coordinates are known, compute the CAM argmax for each, convert it to lat-lon, and measure the geodesic distance to the true coordinate; if most predictions miss by more than roughly 100 meters, the GPS-level localization claim fails even if tile-level AUPRC stays above 0.96.
Extended reading notes
Core claim
The central claim is that image-level classification labels are sufficient to approximate the precise geographical coordinates of schools: a classifier that detects whether a satellite tile contains a school, combined with a class activation map, yields a point estimate of the school's location within the tile. Across ten pilot countries, the best ensemble of ViT, SwinV2, and ConvNext models (VSC-Ensemble) achieves AUPRC above 0.96, and GradCAM-family attribution methods give the most faithful importance maps by the ROAD confidence-drop measure. Comparing model predictions with official Senegalese records at a probability threshold of 0.5 and a matching distance of 250 meters, the pipeline matched 6,983 of 9,033 government-registered schools and surfaced 5,369 unmatched predictions for field validation. The authors conclude that governments can use these validation-ready predictions to correct incomplete registries and estimate the cost of connecting every school.
Load-bearing premise
The coordinate-level claim rests on the unvalidated assumption that the single most important pixel in a class activation map points at the actual school building rather than at a surrounding field, court, or nearby structure.
Editorial extensions
If this is right
- Governments can generate or update nationwide school maps from cheap image-level labels instead of expensive bounding-box or pixel-level annotation campaigns.
- The Senegal comparison suggests the pipeline can surface thousands of unmapped candidate schools, giving field teams a shortlist ordered by confidence.
- Regional models trained on multiple countries perform about as well as country-specific models, so one model can serve countries with scarce local training data.
- The threshold-based web tool lets stakeholders trade precision against recall, adapting the map to how much field validation effort they can afford.
- Connectivity cost estimates can be built directly from the predicted school coordinates rather than from incomplete national registries.
Reading between the lines
- A natural extension the paper does not run is to validate CAM-derived coordinates against a held-out set of true school-building points; the Senegal match statistics only measure agreement with registry points, not geographic accuracy.
- Because the noise analysis shows many government coordinates sit hundreds of meters off the actual building, a practical follow-up would use model predictions to correct existing registry coordinates, not only to discover unmapped schools.
- The same classification-plus-CAM recipe could transfer to other dispersed infrastructure with distinctive overhead signatures, such as health clinics, water points, or markets, where connectivity planning faces the same data gaps.
- The cross-country generalization pattern suggests a geography-aware training schedule could outperform pooling all countries equally, since neighboring countries generalize well to each other while small-sample countries do not.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a weakly supervised pipeline for mapping schools from high-resolution Maxar satellite imagery. It builds country-level training sets from government records, OpenStreetMap, Overture Maps, and random non-school sampling; fine-tunes ViT, SwinV2, and ConvNext classifiers and their ensemble; reports AUPRC values above 0.96 on test sets for ten African countries; uses class activation maps to convert the most important pixel into a predicted lat-lon coordinate; generates nationwide school predictions with an overlapping sliding window and a duplicate-aggregation step; evaluates attribution methods with the ROAD confidence-drop measure; and presents a Senegal case study comparing model predictions with government school data at a 250 m matching threshold. The paper also introduces an interactive web-based validation tool and makes code available on GitHub.
Significance. If the classification and localization claims hold, the pipeline would be practically valuable for school connectivity planning and would be a useful example of low-cost, weakly supervised geospatial mapping. The paper has clear strengths: it combines multiple public data sources, explicitly analyzes urban/rural performance differences, examines cross-country generalization, compares regional and local models, and releases code. However, the two headline claims are not established by the evidence as presented. The reported AUPRC numbers are affected by test-set-based model selection, and the claimed GPS-coordinate-level localization accuracy is never directly validated. The ROAD evaluation measures attribution faithfulness rather than geographic accuracy, and the Senegal comparison uses a coarse 250 m matching threshold, so it does not support the coordinate-level precision claim. These issues are central to the paper's contribution and require substantial revision.
major comments (2)
- [Results and Discussion, 'Image-level school classification'; Table 2] The reported AUPRC values are not unbiased estimates because the same test set is used for model selection and final performance reporting. The text states that the authors 'select the model that achieves the highest AUPRC on the test set for each country,' and Table 2 then reports the performance of the selected models. This procedure makes the reported numbers optimistically biased because the test set has influenced the choice among architectures and ensembles. Please select models using the validation set (or nested cross-validation) and report test performance for the chosen model, or explicitly quantify the selection bias. This is load-bearing because the abstract's AUPRC-above-0.96 claim rests on these numbers.
- [Methods, 'School localization'; Results and Discussion, 'Evaluating CAM methods'; Figure 5] The paper's central claim that the method 'can approximate the precise locations of schools at the level of GPS coordinates' is not supported by any direct spatial validation. The localization procedure converts the xy coordinate of the most important CAM pixel to a lat-lon coordinate, but no experiment compares predicted coordinates to known school coordinates. Table 3 reports ROAD confidence drop, which measures whether removing high-attention pixels lowers classifier confidence; it does not measure whether the highlighted pixel is geographically located at the school. The Senegal comparison in Figure 5 uses a 250 m matching distance and compares counts rather than per-prediction distance errors, so it does not establish coordinate-level accuracy. Because the paper itself notes that schools are identified by playgrounds, fields, and grouped building structures, the argmax CAM pixel could plausibly fall tens of meters from the actual school building. Please add a direct spatial evaluation, such as distance-to-nearest-known-school distributions at several thresholds and precision/recall as a function of distance, and report how the CAM peak relates to the school building.
minor comments (6)
- [Generalizability experiments; Table 2 vs. Figure 3] The regional-model evaluation is inconsistent as written: the text says the regional model is tested on the individual test sets of each country, while the Table 2 caption says it is tested on the combined test sets, and Figure 3 compares regional and local AUPRC per country. Please clarify which evaluation was used, since this affects the regional-vs-local conclusion and the reproducibility of Figure 3.
- [Table 2] The country codes 'BW A' and 'RW A' contain stray spaces; please standardize them to 'BWA' and 'RWA'.
- [Figure 1] The caption includes 'ROAD: -2.388' without explanation; please define this value, clarify why it is negative, or remove it.
- [Evaluating CAM methods] The criterion for choosing the 'best-performing model' used to compute ROAD scores, while excluding the ensemble model, is not stated; please specify whether selection is by validation AUPRC, test AUPRC, or another metric, and use validation-based selection for consistency.
- [Data preprocessing] The manual repositioning of school GPS coordinates based on Google Satellite Hybrid imagery should be documented quantitatively, including how many points were moved and by what distance, because this affects the label accuracy used for both training and evaluation.
- [Country-wide school predictions] The aggregation buffer radius (50 m, or 150 m for Botswana) and the F2-optimal threshold are free parameters that materially affect the nationwide counts; a sensitivity analysis for the Senegal case would help interpret the reported 6,983 matched, 2,050 unmatched government, and 5,369 unmatched prediction figures.
Circularity Check
The localization-quality evaluation is self-referential: ROAD scores CAMs by the model's own confidence drop, not by true school coordinates, so the GPS-level claim is not independently supported.
-
other
[School localization (Methods) and Evaluating CAM methods for school localization (Results and Discussion)]
"From the CAMs, we can then convert the xy coordinates of the most important pixel to a lat-lon coordinate to approximate the geographic coordinates of the school. ... To assess the quality of the different pixel attribution methods, we measured the average confidence drop across all test set images after perturbing the pixels at the 90th percentile ... We employed the Remove and Debias (ROAD) evaluation framework."
The predicted school location is defined as the CAM's most important pixel, and the CAM method is selected by ROAD, i.e. by the drop in the model's own confidence when those pixels are perturbed. Thus the localization-quality metric is, by construction, a measure of the model's self-consistency rather than of geographic accuracy against known school coordinates. The paper never measures the distance from the argmax pixel to true school points; the Senegal case study uses a coarse 250 m match threshold and compares counts, not per-prediction location error. ROAD therefore cannot independently support the conclusion that the method approximates 'precise locations ... at the level of GPS coordinates.'
full rationale
The image-level classification results are genuine held-out estimates (80/10/10 country-level splits, AUPRC on test sets), so the headline classification performance is not circular, and there is no load-bearing self-citation chain. The circularity is confined to the localization stage: the school coordinate is defined as the most important CAM pixel, and the CAM method is evaluated by ROAD, which measures the model's own confidence drop. That is a self-referential validation: it shows the model's explanation is consistent with the model's decision, not that the pixel lies on the school. The Senegal comparison does provide an external anchor to government data, but it is coarse (250 m threshold), count-based, and uses the same government records that supplied training labels, so it does not establish GPS-level precision. The score reflects this partial circularity in the central localization claim while recognizing that the classification AUPRC and the external Senegal comparison give the paper some independent content.
Assumptions & free parameters
free parameters (4)
- Per-country decision threshold tau* (F2-optimal) =
0.315-0.395 (Table 4)
- Positive-to-negative sample ratio =
1:2
- Aggregation buffer radius for duplicate predictions =
50 m (150 m for Botswana)
- Matching distance threshold d =
250 m
assumptions (5)
- domain assumption Schools in Africa have identifiable overhead signatures distinguishable from other built structures in 60cm satellite imagery.
- domain assumption Random points sampled from populated areas, after removing 300m around known schools, are predominantly non-school locations.
- ad hoc to paper The most important pixel in a class activation map corresponds to the school's geographic location.
- domain assumption GHSL, Microsoft Building Footprints, and Google Open Buildings rasterizations correctly identify human settlements for filtering erroneous points.
- domain assumption ROAD (Remove and Debias) with noisy linear imputation yields a valid proxy for explanation quality.
Cite this review
Pith. "Pith review of Large-scale School Mapping using Weakly Supervised Deep Learning for Universal School Connectivity." pith.science (2026). https://pith.science/paper/YIKKSULK
@misc{pith2026241214870,
author = {Pith},
title = {Pith review of: Large-scale School Mapping using Weakly Supervised Deep Learning for Universal School Connectivity},
year = {2026},
howpublished = {\url{https://pith.science/paper/YIKKSULK}},
note = {Machine review of arXiv:2412.14870}
}
read the original abstract
Improving global school connectivity is critical for ensuring inclusive and equitable quality education. To reliably estimate the cost of connecting schools, governments and connectivity providers require complete and accurate school location data - a resource that is often scarce in many low- and middle-income countries. To address this challenge, we propose a cost-effective, scalable approach to locating schools in high-resolution satellite images using weakly supervised deep learning techniques. Our best models, which combine vision transformers and convolutional neural networks, achieve AUPRC values above 0.96 across 10 pilot African countries. Leveraging explainable AI techniques, our approach can approximate the precise geographical coordinates of the school locations using only low-cost, classification-level annotations. To demonstrate the scalability of our method, we generate nationwide maps of school location predictions in African countries and present a detailed analysis of our results, using Senegal as our case study. Finally, we demonstrate the immediate usability of our work by introducing an interactive web mapping tool to streamline human-in-the-loop model validation efforts by government partners. This work successfully showcases the real-world utility of deep learning and satellite images for planning regional infrastructure and accelerating universal school connectivity.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Bazi, Y.; Bashmal, L.; Rahhal, M. M. A.; Dayil, R. A.; and Ajlan, N. A. 2021. Vision transformers for remote sensing image classification. Remote Sensing, 13(3): 516
work page 2021
-
[4]
Beery, S.; Wu, G.; Edwards, T.; Pavetic, F.; Majewski, B.; Mukherjee, S.; Chan, S.; Morgan, J.; Rathod, V.; and Huang, J. 2022. The auto arborist dataset: a large-scale benchmark for multiview urban forest monitoring under domain shift. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 21294--21307
work page 2022
-
[5]
Canny, J. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8(6): 679--698
work page 1986
-
[6]
Chattopadhay, A.; Sarkar, A.; Howlader, P.; and Balasubramanian, V. N. 2018. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 839--847
work page 2018
-
[7]
Christie, G.; Fendley, N.; Wilson, J.; and Mukherjee, R. 2018. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6172--6180
work page 2018
-
[8]
Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, 248--255. Ieee
2009
Show all 45 references
-
[9]
L.; and Carin, L
Draelos, R. L.; and Carin, L. 2021. Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networks. arXiv:2011.08891
2021 arXiv
-
[10]
Fu, H.; Fan, X.; Yan, Z.; and Du, X. 2021. Detection of schools in remote sensing images based on attention-guided dense network. ISPRS International Journal of Geo-Information, 10(11): 736
2021
-
[11]
Fu, H.; Fan, X.; Yan, Z.; Du, X.; Jian, H.; and Xu, C. 2022. Feature Enhanced Anchor-Free Network for School Detection in High Spatial Resolution Remote Sensing Images. Applied Sciences, 12(6): 3114
2022
-
[12]
Giga. 2024. GigaMaps . https://maps.giga.global/map. [Accessed 17-07-2024]
2024
-
[13]
Gildenblat, J.; and contributors. 2021. PyTorch library for CAM methods. https://github.com/jacobgil/pytorch-grad-cam
2021
-
[14]
Handan-Nader, C.; and Ho, D. E. 2019. Deep learning to map concentrated animal feeding operations. Nature Sustainability, 2(4): 298--306
2019
-
[15]
Huang, X.; Ren, L.; Liu, C.; Wang, Y.; Yu, H.; Schmitt, M.; H \"a nsch, R.; Sun, X.; Huang, H.; and Mayer, H. 2022. Urban building classification (ubc)-a dataset for individual building detection and classification from satellite imagery. In Proceedings of the IEEE/CVF Confere...
2022
-
[16]
Jiang, P.-T.; Zhang, C.-B.; Hou, Q.; Cheng, M.-M.; and Wei, Y. 2021. LayerCAM: Exploring Hierarchical Class Activation Maps for Localization. IEEE Transactions on Image Processing, 30: 5875--5888
2021
-
[17]
Kenya Ministry of Education . 2024. Kenya Ministry of Education . https://www.education.go.ke/. [Accessed 15-07-2024]
2024
-
[18]
International Conference on Learning Representations
Kolesnikov, A.; Dosovitskiy, A.; Weissenborn, D.; Heigold, G.; Uszkoreit, J.; Beyer, L.; Minderer, M.; Dehghani, M.; Houlsby, N.; Gelly, S.; Unterthiner, T.; and Zhai, X. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In "International Confer...
2021
-
[19]
R.; Tajwar, F.; Burke, M.; Ermon, S.; Lobell, D
Lee, J.; Brooks, N. R.; Tajwar, F.; Burke, M.; Ermon, S.; Lobell, D. B.; Biswas, D.; and Luby, S. P. 2021. Scalable deep learning to identify brick kilns and aid regulatory capacity. Proceedings of the National Academy of Sciences, 118(17): e2018863118
2021
-
[20]
Liu, Z.; Hu, H.; Lin, Y.; Yao, Z.; Xie, Z.; Wei, Y.; Ning, J.; Cao, Y.; Zhang, Z.; Dong, L.; Wei, F.; and Guo, B. 2022 a . Swin Transformer V2: Scaling Up Capacity and Resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12009--12019
2022
-
[21]
Liu, Z.; Mao, H.; Wu, C.-Y.; Feichtenhofer, C.; Darrell, T.; and Xie, S. 2022 b . A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11976--11986
2022
-
[22]
Maduako, I.; Yi, Z.; Zurutuza, N.; Arora, S.; Fabian, C.; and Kim, D.-H. 2022. Automated school location mapping at scale from satellite imagery based on deep learning. Remote Sensing, 14(4): 897
2022
-
[23]
MapBox. 2024. MapBox . https://www.mapbox.com/. [Accessed 17-07-2024]
2024
-
[24]
Maxar. 2024. Maxar Global Enhanced GEOINT Delivery . https://evwhs.digitalglobe.com/. [Accessed 09-08-2024]
2024
-
[25]
Microsoft. 2023. Microsoft Global ML Building Footprints . https://github.com/microsoft/ GlobalMLBuildingFootprints. [Accessed 31-01-2024]
2023
-
[26]
D.; and Gebru, T
Mitchell, M.; Wu, S.; Zaldivar, A.; Barnes, P.; Vasserman, L.; Hutchinson, B.; Spitzer, E.; Raji, I. D.; and Gebru, T. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* '19, 220–229. New York, NY, USA: A...
2019
-
[27]
Monarch, R. M. 2021. Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster
2021
-
[28]
B.; and Yeasin, M
Muhammad, M. B.; and Yeasin, M. 2020. Eigen-CAM: Class Activation Map using Principal Components. In 2020 International Joint Conference on Neural Networks (IJCNN), 1--7
2020
-
[29]
Peng, J.; Huang, Y.; Sun, W.; Chen, N.; Ning, Y.; and Du, Q. 2022. Domain adaptation in remote sensing image classification: A survey. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15: 9842--9859
2022
-
[30]
Pesaresi, M.; and Politis, P. 2023. GHS-BUILT-C R2023A - GHS Settlement Characteristics, derived from Sentinel2 composite (2018) and other GHS R2023A data. European Commission, Joint Research Centre (JRC) PID: http://data.europa.eu/89h/3c60ddf6-0586-4190-854b-f6aa0edc2a30
2018
-
[31]
Pillai, V.; and Pirsiavash, H. 2021. Explainable models with consistent interpretations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2431--2439
2021
-
[32]
Plotly Technologies Inc. 2024. Plotly Dash. https://dash.plotly.com/
2024
-
[33]
Robinson, C.; Chugg, B.; Anderson, B.; Ferres, J. M. L.; and Ho, D. E. 2022. Mapping industrial poultry operations at scale with deep learning and aerial imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15: 7458--7471
2022
-
[34]
Rolf, E. 2023. Evaluation Challenges for Geospatial ML. arXiv:2303.18087
2023 arXiv
-
[35]
Rong, Y.; Leemann, T.; Borisov, V.; Kasneci, G.; and Kasneci, E. 2022. A Consistent and Efficient Evaluation Strategy for Attribution Methods. In Proceedings of the 39th International Conference on Machine Learning, 18770--18795. PMLR
2022
-
[36]
Schiavina, M.; Melchiorri, M.; and Pesaresi, M. 2023. GHS-SMOD R2023A - GHS settlement layers, application of the Degree of Urbanisation methodology (stage I) to GHS-POP R2023A and GHS-BUILT-S R2023A, multitemporal (1975-2030). European Commission, Joint Research Centre (JRC) ...
2023
-
[37]
R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D
Selvaraju, R. R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D. 2017. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In 2017 IEEE International Conference on Computer Vision (ICCV), 618--626
2017
-
[38]
Silva, D. 2024. Pytorch LR Finder. https://github.com/davidtvs/pytorch-lr-finder
2024
-
[39]
Sirko, W.; Kashubin, S.; Ritter, M.; Annkah, A.; Bouchareb, Y. S. E.; Dauphin, Y.; Keysers, D.; Neumann, M.; Cisse, M.; and Quinn, J. 2021. Continental-Scale Building Detection from High Resolution Satellite Imagery. arXiv:2107.12283
2021 arXiv
-
[40]
Sivasubramanian, A.; VR, P.; V, S.; and Ravi, V. 2024. Transformer based ensemble deep learning approach for remote sensing natural scene classification. International Journal of Remote Sensing, 45(10): 3289--3309
2024
-
[41]
Smith, L. N. 2017. Cyclical Learning Rates for Training Neural Networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 464--472
2017
-
[42]
Song, S.; Yu, H.; Miao, Z.; Zhang, Q.; Lin, Y.; and Wang, S. 2019. Domain adaptation for convolutional neural networks-based remote sensing scene classification. IEEE Geoscience and Remote Sensing Letters, 16(8): 1324--1328
2019
-
[43]
UNICEF. 2020. How many children and young people have internet access at home?: estimating digital connectivity during the COVID-19 pandemic. Technical report, UNICEF
2020
-
[44]
M.-C.; Heuvelink, G
Wadoux, A. M.-C.; Heuvelink, G. B.; De Bruin, S.; and Brus, D. J. 2021. Spatial cross-validation is not the right way to evaluate map accuracy. Ecological Modelling, 457: 109692
2021
-
[45]
Yi, Z.; Zurutuza, N.; Bollinger, D.; Garcia-Herranz, M.; and Kim, D. 2019. Towards equitable access to information and opportunity for all: mapping schools with high-resolution Satellite Imagery and Machine Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision...
2019
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.