{"id":"ea232159-6070-42db-a230-665ffef8c714","arxiv_id":"2509.02018","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A quantized MobileNet on a Raspberry Pi achieves 91.8% sleep/awake and 97.7% crying/normal accuracy on a public neonatal dataset, but the state-of-the-art claim is not supported by direct comparison to prior vision methods.","lead":"This paper describes a camera-based monitoring system that uses a compact neural network on a Raspberry Pi to classify whether a preterm baby is asleep or awake, and whether it is crying. The authors report high accuracy on public datasets and argue that lightweight models are more practical for low-resource hospitals than larger, more accurate ones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported accuracies are for isolated crops, not the integrated system; the paper's own 'integrated performance paradox' admits drops without quantifying them.","rationale":"The reader's verdict was CONDITIONAL, with the weakest assumption about dataset label validity and generalization. My concern is different but complementary: the central claim is about a deployed system, yet the accuracy evidence is only for an isolated classifier. The paper's own admission of an 'integrated performance paradox' strengthens the need for conditional acceptance: the authors must provide end-to-end metrics before the system-level claim can be accepted. This does not change the verdict (still CONDITIONAL) but refines the specific condition. I partially agree with the reader because they also noted internal inconsistencies and the paradox, but their primary weakest assumption was about label validity, which is not my primary concern. The integrated evaluation gap is the most load-bearing because it directly undermines the stated headline numbers, and it is a fixable omission: the authors already have the datasets and the pipeline; they simply need to report the results. The concrete test I propose would settle whether the deployed system actually achieves the claimed performance, which is the core of the paper's contribution.","tokens_in":12710,"tokens_out":4423,"duration_ms":47418,"concrete_test":"Run the complete Algorithm 1 pipeline on a set of test videos (from the same Roboflow/Mendeley datasets used for the static tests) and compute end-to-end accuracy for sleep/awake and crying/normal classification, using the same frame-level labels as ground truth. Compare those integrated accuracies to the isolated MobileNet numbers in Tables 1 and 2 (91.81% and 97.69%). If the integrated accuracy is materially lower (e.g., more than 2–3 percentage points) or if the pipeline cannot sustain real-time operation, the 'integrated performance paradox' is confirmed and the headline claim of system-level SOTA accuracy is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and conclusions claim that the deployed Raspberry Pi system achieves 91.8% sleep/awake and 97.7% crying/normal accuracy. However, these numbers come from Tables 1 and 2, which evaluate the MobileNet classifier on pre-cropped face ROIs from the test sets of two static image datasets. No evaluation is reported for the end-to-end pipeline described in Algorithm 1 (face detection via MediaPipe, ROI cropping, temporal filtering via a 5-frame sliding window, alert triggering). This matters because the paper's own Section 5 introduces an 'integrated performance paradox,' stating that 'models which excel in isolated tasks experienced significant accuracy drops when deployed within the full processing pipeline.' That is a direct admission that the standalone accuracies do not represent system-level performance, yet no integrated accuracy numbers are provided anywhere. The central claim that the system achieves state-of-the-art accuracy 'suitable for edge deployment' is therefore not supported by the evidence presented. The missing integrated evaluation is a critical omission: face detection failures, cropping errors, temporal smoothing artifacts, and CPU contention on the Raspberry Pi could all degrade accuracy, and the authors explicitly acknowledge such degradation without quantifying it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a vision-based embedded system for noncontact monitoring of preterm infant behavior (sleep/awake and crying/normal) in low-resource NICU settings. It uses a quantized MobileNetV3-Small classifier, trained on two public image datasets, deployed on a Raspberry Pi 5 with MediaPipe face detection, temporal smoothing, and Firebase-based alerting. The authors report standalone classification accuracies of 91.8% for sleep/awake and 97.7% for crying/normal, benchmark several CNN architectures for accuracy and TF-Lite inference cost, and argue that lightweight models offer the best accuracy-efficiency trade-off for edge deployment. The central claim is that the system achieves state-of-the-art accuracy while maintaining real-time, low-footprint operation.","tokens_in":13018,"tokens_out":2809,"duration_ms":32085,"significance":"If the system-level claims were supported, this would be a useful applied contribution: it targets a real gap in low-resource neonatal care, provides concrete hardware measurements (latency, memory, model size) for multiple architectures, and demonstrates that quantization and edge deployment of a modest CNN are feasible for this domain. The comparative benchmarking on Raspberry Pi is a strength, as is the explicit attention to deployment constraints. However, the accuracy claims are currently presented at the classifier level while the paper's own text acknowledges an \"integrated performance paradox,\" and the state-of-the-art comparison is not made against prior vision-based methods on the same data. These gaps are load-bearing for the headline claims.","major_comments":[{"comment":"The headline accuracies (91.8%, 97.7%) are reported for the MobileNet classifier on pre-cropped face ROIs from static test sets, but the abstract and conclusions attribute them to the deployed system. Section 5 explicitly states that \"models which excel in isolated tasks experienced significant accuracy drops when deployed within the full processing pipeline,\" yet no end-to-end accuracy is reported for Algorithm 1 (face detection, cropping, 5-frame temporal filtering, alert triggering). Face-detection failures, cropping errors, and CPU contention on the Raspberry Pi could materially reduce accuracy. Please provide integrated system-level accuracy on realistic video or state clearly that the reported numbers are classifier-only.","section":"§5 Conclusions and Tables 1–2"},{"comment":"The \"state-of-the-art\" claim is unsupported as stated. Table 3 compares against audio- and sensor-based methods on different datasets and modalities; no comparison is made to prior vision-based infant behavior classifiers on the same datasets. Moreover, the evaluation datasets appear to be the authors' own Mendeley dataset (ref [29]), so the benchmark is not independent. Please include a comparison with existing image-based methods on the same split or on a truly external dataset, and temper the SOTA wording accordingly.","section":"§4.1.2, Table 3"},{"comment":"All accuracy, precision, F1, and AUC numbers come from a single train/test split with no confidence intervals, repeated runs, or significance testing. Given the small test sets and the many tuned components (classification confidence thresholds, SE-block pruning ratio, augmentation parameters, early stopping), the reported rankings among models could be within noise. Please report mean±std over multiple seeds or bootstrap confidence intervals, and state the threshold-selection procedure.","section":"§3.4, §4.1.1, Tables 1–2"},{"comment":"Labels were assigned according to visible facial cues (eye closure, mouth openness, facial tension), but no clinical validation is provided. For preterm infants in an incubator, facial visibility may be obstructed, and the equivalence of facial-expression labels to clinically meaningful sleep/crying states is not established. The paper should either present validation against clinician scoring or physiological signals or clearly frame the results as preliminary and discuss the generalizability risk.","section":"§3.2"}],"minor_comments":[{"comment":"The abstract says a 68% size reduction; §3.6.2 says FP16 quantization reduces size by 60% and latency by 22%. Please reconcile the numbers and specify which measurement is used.","section":"Abstract vs §3.6.2"},{"comment":"The text says the datasets are from Roboflow with a Mendeley DOI [29], but the Data Availability statement says \"Not applicable.\" Clarify availability of the exact preprocessed datasets and splits.","section":"§3.2 vs Declarations"},{"comment":"Typo: \"MobileNetV3V2\" should likely be \"MobileNetV3\" or \"MobileNetV2.\"","section":"§2.2"},{"comment":"Inconsistent frame-rate statements: §3.3 says 25 FPS, while §4.2.2 says 30 FPS operation. Please use one consistent operational specification.","section":"§3.3 vs §4.2.2"},{"comment":"End-to-end latency is reported as 0.87 s/frame, while Table 4 gives per-inference MobileNet latency of 7.57 ms. Clarify which components make up the 0.87 s and reconcile with the 25/30 FPS claims.","section":"§3.7 vs Table 4"},{"comment":"Platt scaling is mentioned but its implementation and effect on calibration are not described. Please provide details or remove the unsubstantiated claim.","section":"§3.5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's core engineering direction is sensible and the hardware benchmarking is a genuine strength. However, the headline accuracy claims are not yet backed by an integrated evaluation, and the \"state-of-the-art\" comparison is not apples-to-apples. These issues are fixable with additional experiments or careful claim revision, so I recommend major revision rather than rejection. The authors should also ensure the final version avoids overclaiming clinical readiness before real-world validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful engineering report, not a research breakthrough. The headline accuracies are for isolated classifiers on cropped face ROIs, not for the end-to-end Pi system, and the SOTA claim doesn't hold up. The genuinely useful part is the deployment-side analysis: they benchmark several architectures on a Raspberry Pi 5, show MobileNet at about 7.6 ms per inference and 2.6 MB on disk, and openly discuss why system-level latency and memory dominate model choice. That's the kind of data people building edge medical devices actually want.\n\nThe paper does several things well. It doesn't hide the integrated performance paradox—it admits models that look great in isolation lose accuracy in the full pipeline. It correctly identifies O(HW) face-detection cost as the real bottleneck. And its comparative latency/memory table is informative. For a practitioner choosing a model for low-cost NICU monitoring, this is a reasonable starting point.\n\nThe soft spots are substantial.\n\nFirst, the central claim is not supported by the reported numbers. Tables 1 and 2 evaluate single models on test splits of static image datasets. No end-to-end evaluation of Algorithm 1 is reported, no face-detection failure rate, no temporal-filtering ablation. The paper's own acknowledgment of 'significant accuracy drops' in the pipeline makes the omission worse, not better. Without integrated numbers, the abstract's 'achieves state-of-the-art accuracy... suitable for edge deployment' is an overstatement.\n\nSecond, the SOTA comparison in Table 3 compares the authors' image classifiers to EEG/PSG and audio methods on different datasets and different tasks. That's not a benchmark; it's a scatter plot. A fair comparison would be against other vision-based infant behavior classifiers on the same data.\n\nThird, experimental rigor is thin. One train/test split, no confidence intervals or repeated runs, and the dataset is the authors' own Mendeley set labeled by visible facial cues. That may be acceptable as a pilot, but not for the 'conclusively shows' language. Code is promised only after acceptance, and data availability is listed as 'Not applicable,' which conflicts with the claim of using public data.\n\nThe engineering conclusion—MobileNet-class models are the right compromise for this hardware—is plausible and likely correct. The paper doesn't need a new architecture to be useful. It needs honest reframing as an embedded-systems validation, plus integrated performance numbers and a proper comparison. I'd send it to peer review with that expectation: there is enough practical value and enough honest system-level analysis to justify referee time, but the claims need heavy revision before publication. I wouldn't cite it as-is.","headline":"Useful engineering report with real deployment data, but the headline accuracy claims describe isolated classifiers, not the integrated system, and the SOTA claim does not survive scrutiny.","tokens_in":13403,"tokens_out":2567,"would_cite":false,"duration_ms":30202,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a quantized MobileNetV3 model on a Raspberry Pi can classify preterm infant sleep/awake and crying/normal states from facial images with accuracies of 91.8% and 97.7%, respectively, while remaining small and fast enou","keywords":["neonatal monitoring","embedded AI","edge deployment","MobileNetV3","model quantization","preterm infant","sleep classification","cry detection"],"falsifier":"A clinical study recording preterm infants in actual incubators with occlusions, variable lighting, and side-facing cameras, then comparing the MobileNet's classifications against expert video review or polysomnography; if the accuracy drops substantially below the reported 91.8%/97.7%—or if the face-detection stage fails on a large fraction of frames—the edge-deployability claim would be contradicted.","tokens_in":12670,"feed_emoji":"👶","tokens_out":5840,"duration_ms":56405,"temperature":0.7,"pith_summary":"Preterm babies in low-resource NICUs need continuous monitoring of sleep and crying states, but current options are invasive or rely on scarce staff. The paper argues that a lightweight, quantized convolutional network—MobileNetV3—running on a Raspberry Pi can do this job noninvasively from a camera feed, with accuracy close to much larger models. Across public neonatal image datasets, the system reaches 91.8% accuracy for sleep/awake and 97.7% for crying/normal, with per-inference latency under 8 milliseconds and a model size small enough for edge hardware. The authors benchmark five architectures and conclude that bigger networks like Inception gain only a few accuracy points at a prohibitive cost in latency and memory, making the optimized MobileNet the most viable foundation for real-time, low-cost monitoring.","feed_headline":"Raspberry Pi camera reads infant states at up to 98% accuracy","feed_subtitle":"No skin-contact sensors needed: a tiny vision model runs continuously on low-cost edge hardware and matches bigger networks.","key_machinery":"The carrying mechanism is a quantized MobileNetV3-Small—a lightweight CNN using depthwise separable convolutions and Squeeze-and-Excitation attention—fine-tuned from ImageNet weights and converted to TensorFlow Lite with FP16 quantization. It operates inside a pipeline with MediaPipe face detection, 256×256 ROI cropping, Platt-scaled confidence calibration, a five-frame temporal smoothing window, and asynchronous Firebase alerts for sustained events. The quantitative core is the accuracy-versus-cost trade-off: the MobileNet delivers its accuracies at roughly 12× smaller model size and nine times lower per-inference latency than Inception, which the paper argues is the decisive axis for low-r","core_discovery":"The central claim is that a MobileNetV3-Small backbone, pruned in its Squeeze-and-Excitation blocks and quantized to TensorFlow Lite, achieves near-state-of-the-art accuracy on preterm infant behavioral classification while being deployable on a Raspberry Pi 5. On the sleep/awake task, it scores 91.8% accuracy, 0.92 precision, and 0.98 AUC; on crying/normal, 97.7% accuracy, 0.98 precision, and 1.00 AUC. Per-inference latency on the Pi is 7.57 ms with a 2.59 MB model binary, versus InceptionV3's 94.15%/94.44% accuracy at 66.19 ms and 254 MB. The authors further report an 'integrated performance paradox': models that perform well in isolated classification can suffer accuracy drops inside the","pith_inferences":["The paper's dataset labels, derived from visible facial cues, likely cap the achievable accuracy; clinical validation with expert-scored labels could change the relative ranking of models.","The same pipeline—face detection, cropping, temporal smoothing, and edge inference—could be extended to other behavioral cues such as limb movement or posture without hardware changes.","Reported accuracy comes from public images, not real incubator feeds; occlusions, side-angle views, and lighting variation are the immediate tests the authors list as future work.","Knowledge distillation, which the authors cite as a future direction, could plausibly close the remaining accuracy gap to Inception while preserving the edge deployment budget."],"forward_implications":["Sleep/awake and crying/normal detection can run continuously on a low-cost edge device, removing the need for server-grade hardware or cloud dependence in NICU monitoring.","Model quantization reduces the deployed model size by 68% and latency by 22% with minimal accuracy loss, making continuous real-time inference feasible on Raspberry Pi–class hardware.","Vision-based monitoring can match or exceed audio-based cry classification and EEG/PSG-based sleep staging without skin contact, according to the benchmarks reported.","The integrated performance paradox implies that model selection for such systems must be validated inside the full pipeline—face detection, preprocessing, and multitasking—not in isolated classification tests.","Secure, asynchronous IoT alerting enables timely notification of sustained events such as prolonged crying, which is the clinically actionable output of the system."],"supporting_citations":[{"why":"Supplies the two public neonatal image datasets (sleep/awake and crying/normal) used to train and evaluate all models.","marker":"[29]"},{"why":"Defines the MobileNetV3 architecture with depthwise separable convolutions and Squeeze-and-Excitation blocks that the paper optimizes and deploys.","marker":"[27]"},{"why":"The EfficientNet scaling method whose model consistently underperforms on these tasks, serving as the baseline that motivates the accuracy-efficiency critique.","marker":"[14]"},{"why":"Prior work detecting infant crying from facial features, the line of research this paper extends to embedded settings.","marker":"[13]"},{"why":"Demonstrates MobileNet-family models achieving high accuracy in a medical image classification task, supporting the premise that lightweight CNNs suffice for medical visual analysis.","marker":"[26]"},{"why":"A sensor-based pediatric sleep staging benchmark that the vision approach is compared against and outperforms in the Table 3 comparison.","marker":"[20]"},{"why":"An EEG-based pediatric sleep classification benchmark providing another sensor-based accuracy reference for the comparative analysis.","marker":"[30]"},{"why":"An audio-based infant crying interpretation baseline using MFCC features, used to show that vision can match or exceed audio approaches without microphones.","marker":"[4]"}],"fun_headline_variants":["Pi camera reads preterm infant states at up to 98% accuracy","Noncontact infant monitoring: Pi vision scores 91.8% sleep, 97.7% cry","Raspberry Pi vision edge model tracks preterm behavior near-SOTA","Low-cost Pi camera monitors preterm infants without skin sensors","Quantized MobileNet on Pi: real-time preterm infant sleep & cry"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The system assumes that labels assigned from visible facial cues (eye closure, mouth openness, facial tension) are valid ground truth for sleep/awake and crying/normal states, and that a frontal, unobstructed view of the infant's face will be available in real incubator conditions.","fun_headline_variants_meta":{"raw":{"variants":["Pi camera reads preterm infant states at up to 98% accuracy","Noncontact infant monitoring: Pi vision scores 91.8% sleep, 97.7% cry","Raspberry Pi vision edge model tracks preterm behavior near-SOTA","Low-cost Pi camera monitors preterm infants without skin sensors","Quantized MobileNet on Pi: real-time preterm infant sleep & cry"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000897,"raw_usage":{"total_tokens":3761,"prompt_tokens":863,"completion_tokens":2898,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":2801}},"tokens_in":607,"tokens_out":2898,"duration_ms":20954,"temperature":1.0,"reasoning_tokens":2801,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:57:14.251103+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A clinical study recording preterm infants in actual incubators with occlusions, variable lighting, and side-facing cameras, then comparing the MobileNet's classifications against expert video review or polysomnography; if the accuracy drops substantially below the reported 91.8%/97.7%—or if the face-detection stage fails on a large fraction of frames—the edge-deployability claim would be contradicted.","supporting_citations":[{"cited_title":"Mendeley Data (2025)","cited_arxiv_id":null,"evidence_quote":"Supplies the two public neonatal image datasets (sleep/awake and crying/normal) used to train and evaluate all models."},{"cited_title":": Searching for MobileNetV3","cited_arxiv_id":null,"evidence_quote":"Defines the MobileNetV3 architecture with depthwise separable convolutions and Squeeze-and-Excitation blocks that the paper optimizes and deploys."},{"cited_title":"In: Proceedings of the 36th International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"The EfficientNet scaling method whose model consistently underperforms on these tasks, serving as the baseline that motivates the accuracy-efficiency critique."},{"cited_title":"IEEE Transactions on Biomedical Engineering 66(5), 1234– 1243 (2019) https://doi.org/10.1109/TBME.2018.2878723","cited_arxiv_id":null,"evidence_quote":"Prior work detecting infant crying from facial features, the line of research this paper extends to embedded settings."},{"cited_title":"Brain Sciences 14(12) (2024) https://doi.org/10.3390/ brainsci14121178","cited_arxiv_id":null,"evidence_quote":"Demonstrates MobileNet-family models achieving high accuracy in a medical image classification task, supporting the premise that lightweight CNNs suffice for medical visual analysis."},{"cited_title":"IEEE Transactions on Biomedical Engineering 69(12), 3612–3622 (2022) https://doi.org/10.1109/ TBME.2022.3174680","cited_arxiv_id":null,"evidence_quote":"A sensor-based pediatric sleep staging benchmark that the vision approach is compared against and outperforms in the Table 3 comparison."},{"cited_title":"Frontiers in Artificial Intelligence V olume 7 - 2024 (2024) https://doi.org/10.3389/frai.2024.1337356","cited_arxiv_id":null,"evidence_quote":"An audio-based infant crying interpretation baseline using MFCC features, used to show that vision can match or exceed audio approaches without microphones."}],"review_version":1}