{"id":"92b9c43d-d2fc-449a-9593-edca4a5c0ee6","arxiv_id":"2501.03499","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A CNN predicts PM2.5 and PM10 from mobile photos, then uses those predictions to estimate five more pollutants, with claimed test MSE of 0.0077 for the first stage on Delhi data.","lead":"This paper trains a convolutional neural network to estimate air pollutant levels from mobile phone photos of three Indian cities. It also tests whether flipping the images improves accuracy and builds a dashboard that suggests whether a location is safe for a user's health.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline MSE values are not trustworthy because the paper never documents how each photo was matched in time and space to the 2023 PM2.5/PM10 labels and reports no held-out split or comparison with prior models on the same data.","rationale":"The paper's strongest claim is comparative: its MSE values outperform existing models. For that to be true, the MSE values must be computed on a valid test set with correct image-label correspondence, and the same metric must be compared against prior work. The least secure condition is label validity. Section 4 states only that images were paired with PM2.5 and PM10 values from 2023, with no station information, no matching protocol, and no split description. The Declarations section explicitly says data availability is NA and code availability is NA, so the reader cannot verify or reproduce the result. The internal contradiction in Section 6 about which architecture is superior compounds the problem: even the object being evaluated is ambiguous. I considered whether the lack of a baseline comparison is the more load-bearing concern, but that concern is secondary: without trustworthy labels and a clear architecture, no comparison could rescue the claim. The reader's weakest assumption correctly identifies the label-matching problem, so my read agrees. The paper does have some empirical content in the augmentation analysis and the dashboard demonstration, but those contributions do not support the headline comparative MSE claim. Given the missing data, missing baselines, and internal inconsistency, the reader's REJECT verdict stands unchanged.","tokens_in":10206,"tokens_out":3858,"duration_ms":38698,"concrete_test":"Ask the authors to release, for every one of the 5,455 images, the capture timestamp and GPS location, the monitoring station ID and measurement timestamp used for each PM2.5/PM10 label, and the exact train/validation/test split used to produce Tables 5 and 6. Then retrain the stated architecture (after the Section 6 contradiction is resolved) on a split that is disjoint in both time and monitoring station, and recompute the MSE. If the low MSE of 0.0077/0.0112 does not survive a time/location-disjoint split, the reported numbers are an artifact of label leakage or an inappropriate random split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MSE 0.0077 for two pollutants and 0.0112 for five pollutants outperforms existing models. For this claim to hold, each image's label must be the pollutant concentration at the same location and time the photo was taken. Section 4 only states that 5,455 images were 'paired with corresponding PM2.5 and PM10 values from 2023'; it never specifies which monitoring stations supplied the values, how images were geolocated, whether capture time and measurement time coincide, or how the data were split into training and test sets. If labels are mismatched or the split is random rather than disjoint in time and location, the CNN can learn dataset artifacts or memorize spurious cues, making the MSE meaningless. The paper also contains an internal contradiction in Section 6: it first says the two-stage model 'outperforms' the branched model, then immediately says the branched model outperforms the two-stage model, and the low MSE values appear in Tables 5 and 6. Thus even the architecture that produced the headline number is unclear. With data and code declared unavailable, the claimed superiority cannot be independently checked.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a convolutional neural network regression model, 'HealthCamCNN', that predicts air pollutant concentrations from mobile-captured images. The authors report MSE values of 0.0077 for two pollutants (PM2.5 and PM10) and 0.0112 for five pollutants, claiming these outperform existing models. They also analyze the effect of data augmentation and implement a dashboard that maps predicted pollutant values to location-suitability recommendations based on user health conditions. The experiments use a dataset of 5,455 daytime images from Bengaluru, Delhi, and Tamil Nadu paired with 2023 PM2.5/PM10 values, but the paper provides no comparison against prior methods on the same data and does not specify the data-splitting or label-matching protocol.","tokens_in":10458,"tokens_out":4530,"duration_ms":42340,"significance":"If the headline claim were valid, the work would be practically significant: a single mobile-captured image yielding pollutant estimates competitive with or better than existing image-based models would offer a low-cost air quality monitoring tool. The paper also attempts a useful empirical check of horizontal reflection as a data augmentation technique for sky-dominated images. However, the central claim of outperforming existing models is not supported by any same-data comparison, the dataset construction is under-specified, and the architecture descriptions are internally contradictory. The paper ships no code or data, and the evaluation protocol is too incomplete to verify the reported numbers, so the significance is currently not established.","major_comments":[{"comment":"The central claim that the reported MSE values (0.0077 for two pollutants, 0.0112 for five) 'outperform existing models' is unsupported. Tables 4-6 report only absolute MSE/MAE values for the proposed models; no prior method is evaluated on the same dataset, and no comparable metrics from the cited works are given. Without a same-data baseline, the superiority claim has no empirical basis and cannot be checked.","section":"Abstract, Section 6"},{"comment":"The construction of the 5,455-image dataset is critically under-specified. The text only says images were 'paired with corresponding PM2.5 and PM10 values from 2023.' It does not state which monitoring stations supplied the pollutant values, how images were geolocated, whether capture time coincides with measurement time, or how the pairing was performed. If a photo is paired with a pollutant measurement from a different location or time, the CNN learns a spurious mapping and the reported MSE is meaningless. The declaration 'Availability of data - NA' further prevents verification.","section":"Section 4"},{"comment":"The paper contains a direct contradiction about which architecture produced the best results. It first states that the two-stage model (predicting two pollutants, then five from those two) 'outperforms the second model which branches the 2 groups of parameters after the convolution block,' then immediately states 'the latter outperforms the former, thereby establishing the branched CNN model as the superior architecture.' The headline MSE values in the abstract correspond to the Delhi entries in Tables 5 and 6, which are attributed to the branched model. The manuscript must unambiguously identify the model used for each reported result and correct the contradictory prose.","section":"Section 6"},{"comment":"No train/test split or evaluation protocol is described. The paper does not state how the 5,455 images were partitioned into training, validation, and test sets, whether the split was stratified by city or AQI class, or whether the split was disjoint in time and location. Since all models were run for 50 epochs with no mention of early stopping or regularization beyond dropout-free architecture, the reported MSE values could reflect training-set memorization rather than generalization. A clear held-out evaluation procedure must be reported for the results to be interpretable.","section":"Section 6, Tables 4-6"}],"minor_comments":[{"comment":"There is a typo in the list of contributions: 'HealhCamCNN' should be 'HealthCamCNN'.","section":"Section 1"},{"comment":"The city name 'Bengalore' appears in multiple tables; it should be 'Bengaluru.'","section":"Tables 4-6"},{"comment":"References [6] and [24] are the same paper (Kalajdjieski et al.) but are listed with different details; they should be merged and renumbered.","section":"References"},{"comment":"The augmentation results are described in prose and in Figure 2, but no quantitative comparison of final MAE/MSE values with and without augmentation is provided in the text; please add the numerical values used to support the claim of 'minimal accuracy differences.'","section":"Section 6.1"},{"comment":"The statement that 'all the output parameters range between [0,1] during training and are then unnormalized using the inverse transform' does not specify the normalization method (e.g., min-max scaling), whether scaling was fitted per city or globally, or how inverse transform was applied to the predicted values.","section":"Section 6"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an extension of the authors' prior work [1], but the novelty relative to that paper is not clearly delineated. Given that data and code availability are declared NA and no baseline comparisons are reported, the core claim of outperforming existing models is not verifiable. The editor may wish to consider whether the reproducibility standards of the journal are met."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. The headline numbers—MSE 0.0077 for two pollutants and 0.0112 for five—are not backed by any comparison against prior work on the same data. And the part worth keeping is the augmentation experiment, which gives a clean negative result: horizontally flipping sky-heavy images barely changes the final loss, contradicting the assumption in [5] that horizontal reflection should be avoided.\n\nThe augmentation study is the strongest section. The authors split images vertically, mirror the halves, and compare this with horizontal reflection. Their curves show augmentation affects the early training path but not the plateau, which is a falsifiable, useful observation. The two-stage architecture—regress PM2.5 and PM10 first, then feed those predictions into an MLP for the other five—is a reasonable design, and the dashboard is a concrete application.\n\nThe problems sit in the claim of superiority. There is no baseline on the same images. Tables 4–6 give per-city errors but no comparison to [2], [5], or [6]. The abstract cherry-picks Delhi's best numbers. Section 6 contradicts itself: first it says the two-stage model outperforms the branched model, then says the branched model outperforms the two-stage, so it is unclear which architecture produced the headline values. More fundamentally, the data description never says how images were matched to PM2.5/PM10 values from 2023—no station IDs, geolocation, timestamp alignment, or train/test split protocol. If a photo's label is a reading from a different time or place, the CNN learns dataset artifacts and the MSE is meaningless. With data and code declared unavailable, there is no way to verify any of it.\n\nMinor: the literature survey is a grab bag, and the writing repeats 'real-time' too often.\n\nWho gets value? Someone looking for a quick survey of image-based AQI ideas and a piece of evidence on augmentation. It is not ready as a research result. I would send it to review if I were an editor—the topic is real and the augmentation result deserves checking—but it would need heavy revision: document the pairing, report a proper split and same-data baselines, and fix the architecture contradiction.","headline":"A plausible application with a useful negative result on data augmentation, but the headline MSE claim is unsupported by any same-data comparison and the image-label pairing is undocumented.","tokens_in":10924,"tokens_out":3042,"would_cite":false,"duration_ms":28192,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A regression-based CNN trained on mobile-captured images predicts PM2.5 and PM10 with mean squared error as low as 0.0077, then estimates five further pollutants from those two values.","keywords":["air quality prediction","convolutional neural network","mobile camera images","data augmentation","PM2.5","PM10","regression","real-time dashboard"],"falsifier":"Take the 5,455 image-pollutant pairs, hold out all images from one or more full days or monitoring stations, retrain the same architecture, and measure test MSE; if the error jumps far above 0.0077 and 0.0112 while a random split stays low, the reported numbers are an artifact of label leakage rather than a visual signal.","tokens_in":10027,"feed_emoji":"🌫️","tokens_out":5622,"duration_ms":48912,"temperature":0.7,"pith_summary":"This paper argues that a single photograph taken with a mobile phone camera carries enough visual information about haze and sky conditions to estimate ground-level air pollution. The authors build a regression-based convolutional neural network, HealthCamCNN, that first predicts PM2.5 and PM10 from the image and then feeds those two values into a second stage that predicts five more pollutants. On their dataset of 5,455 daytime images from three Indian cities, they report mean squared errors of 0.0077 for the two-pollutant task and 0.0112 for the five-pollutant task, which they say beats existing image-based models. They also report that data augmentation changes the final error by only a negligible amount, contradicting the common assumption that augmented sky images are needed. If these results hold, mobile photos could serve as low-cost, real-time air quality sensors.","feed_headline":"Phone snapshot estimates PM2.5 at MSE 0.0077","feed_subtitle":"Two-stage CNN turns a single mobile image into pollutant levels and a health alert.","key_machinery":"The load-bearing mechanism is the two-stage branched regression CNN: three convolutional blocks (32, 64, and 64 filters of size 3x3, each followed by LeakyReLU and 2x2 max-pooling) extract features from the image; a flattened vector feeds two separate multilayer perceptrons, one that outputs PM2.5 and PM10 and one that outputs the remaining five pollutants; the predicted PM2.5 and PM10 values are then used as additional input for the second MLP. The paper also relies on a vertical split-and-mirror augmentation scheme whose key property is that it preserves the sky's position, and the finding that horizontal reflection does not change the final training loss is used to reject an earlier assumption about sky orientation.","core_discovery":"The central discovery is that a two-stage, branched regression CNN can map a 224x224 RGB mobile photo directly to pollutant concentrations, with the first stage predicting PM2.5 and PM10 and the second stage predicting five additional pollutants from those two predictions. The authors claim this architecture outperforms a model that predicts all seven outputs in parallel, and that it outperforms existing image-based air quality models, citing MSE values that are lower than the comparison baselines. Equally central is the negative result on augmentation: after training with vertically split-and-mirrored images and horizontally reflected images, the plateaued error is nearly identical to training on the original images, which the authors read as evidence that the widely used augmentation step can be dropped without harming accuracy.","pith_inferences":["If the label-matching details are not clarified, the reported MSE could reflect leakage: a random train/test split over photos taken near the same monitor at nearby times would make the task artificially easy; a temporal or geographic holdout would test the true visual signal.","The two-stage design implies that errors in the first-stage PM2.5 and PM10 predictions propagate into the five secondary pollutant predictions; the paper does not quantify this error amplification.","The augmentation finding, if general, suggests that model capacity and training data size matter less than the visual signal for this task, but it is only tested on this dataset, architecture, and augmentation recipe.","A practical extension would be to calibrate the normalized pollutant outputs back to absolute concentrations and compare them against regulatory thresholds, something the dashboard currently does only indirectly through AQI categories."],"forward_implications":["Air quality estimates become possible from ordinary smartphone photos, without deploying or maintaining physical sensors.","The two-stage scheme reduces the number of output heads and lets the second stage exploit the statistical relationship among pollutants, lowering reported error.","Data augmentation of sky images can be omitted without a meaningful accuracy penalty, saving training time and memory.","A public-facing dashboard can recommend whether a location is safe for a given user's health profile, based only on an uploaded photo.","The reported MSE values set a quantitative benchmark that later image-based air quality models would need to beat."],"supporting_citations":[{"why":"Supplies the prior modified CNN architecture and the three-city setup that HealthCamCNN builds on.","marker":"[1]"},{"why":"Provides the main image-based baseline for PM2.5, PM10, and AQI estimation that the reported MSE values are claimed to outperform.","marker":"[2]"},{"why":"Source of the common practice that horizontal reflection of sky images is harmful; the augmentation experiment targets this claim directly.","marker":"[5]"},{"why":"Supplies the smartphone-image sky classification baseline and the 59.38% accuracy figure referenced for comparison.","marker":"[7]"},{"why":"A recent image-based forecasting architecture used as a comparison point for real-time air quality prediction.","marker":"[32]"}],"fun_headline_variants":["Phone image predicts PM2.5 at MSE 0.0077","Data augmentation found unnecessary for air-quality CNN","Two-stage CNN from photo nails pollutant levels","Mobile camera shot yields air quality index instantly","CNN on phone photo matches existing models, minus augmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The labels are valid: each of the 5,455 photos was captured at the same time and place as the 2023 PM2.5 and PM10 measurement used as its target, and the train/test split does not leak information across nearby photos; the paper does not state how images were matched to pollutant data, which monitoring stations were used, or how the data were split.","fun_headline_variants_meta":{"raw":{"variants":["Phone image predicts PM2.5 at MSE 0.0077","Data augmentation found unnecessary for air-quality CNN","Two-stage CNN from photo nails pollutant levels","Mobile camera shot yields air quality index instantly","CNN on phone photo matches existing models, minus augmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000673,"raw_usage":{"total_tokens":3037,"prompt_tokens":888,"completion_tokens":2149,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":2075}},"tokens_in":504,"tokens_out":2149,"duration_ms":14381,"temperature":1.0,"reasoning_tokens":2075,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:52:45.404769+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the 5,455 image-pollutant pairs, hold out all images from one or more full days or monitoring stations, retrain the same architecture, and measure test MSE; if the error jumps far above 0.0077 and 0.0112 while a random split stays low, the reported numbers are an artifact of label leakage rather than a visual signal.","supporting_citations":[{"cited_title":"Envi- ronmental Quality Management 34 (2024) https://doi.org/10.1002/tqem.22276 13","cited_arxiv_id":null,"evidence_quote":"Supplies the prior modified CNN architecture and the three-city setup that HealthCamCNN builds on."},{"cited_title":"Journal of Environ- mental Management 307, 114560 (2022) https://doi.org/10.1016/j.jenvman.2022","cited_arxiv_id":null,"evidence_quote":"Provides the main image-based baseline for PM2.5, PM10, and AQI estimation that the reported MSE values are claimed to outperform."},{"cited_title":"Earth Science Informatics 11, 413–422 (2018)","cited_arxiv_id":null,"evidence_quote":"Supplies the smartphone-image sky classification baseline and the 59.38% accuracy figure referenced for comparison."},{"cited_title":"Chemosphere 333, 138867 (2023) https://doi.org/10.1016/j.chemosphere.2023.138867","cited_arxiv_id":null,"evidence_quote":"A recent image-based forecasting architecture used as a comparison point for real-time air quality prediction."}],"review_version":1}