Pith. sign in

REVIEW 5 major objections 5 minor 19 references

eNCApsulate: NCA for Precision Diagnosis on Capsule Endoscopes

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper demonstrates that Neural Cellular Automata can run bleeding segmentation and depth estimation on a microcontroller small enough for a capsule endoscope, opening the way to on-capsule diagnosis and visual-odometry localization.

desk verdict Solid NCA segmentation port with a useful early-stopping trick, but the depth-estimation pillar is under-evidenced and the stochastic update mismatch needs fixing. read the letter →

arxiv 2504.21562 v1 pith:3DFOWMCA submitted 2025-04-30 cs.CV cs.AI

Henry John Krumb , Anirban Mukhopadhyay This is my paper · ORCID
classification cs.CVcs.AI
keywords WirelessCapsuleEndoscopyNeuralCellularAutomataBleedingSegmentationMonocularDepthEstimationModelDistillationEdgeAIMicrocontrollerInferenceVisualOdometry
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Wireless capsule endoscopy records hours of gut video that clinicians must inspect frame by frame, and locating the capsule once swallowed remains difficult without extra sensors. This paper proposes that Neural Cellular Automata (NCAs), a lightweight architecture where one learned rule is applied repeatedly across an image grid, can perform two clinically useful tasks directly on a microcontroller small enough to fit in a capsule: segmenting bleeding regions and estimating depth from single frames. The paper reports that the segmentation NCA beats other portable segmentation models on Dice score while storing more than 100 times fewer parameters, and that the depth NCA, trained by distilling a large pretrained depth model, produces visually detailed depth maps, sometimes more detailed than the teacher. If these results hold, on-capsule diagnosis becomes practical and sensor-free localization through visual odometry becomes a realistic next step.

What carries the argument

The load-bearing object is the Neural Cellular Automaton itself: a shared MLP rule applied to every cell of an image grid, with each cell aggregating its Moore neighborhood through learned $3 \times 3$ filter banks before the MLP computes a state update that is added back with a stochastic cell update. Because the same parameters are reused over hundreds of iterative steps, the model stays tiny, about 47,000 parameters for segmentation, while still refining spatial structure. For depth, the mechanism is distillation: the outputs of a large monocular depth model, accepted only when their normalized gradient magnitude exceeds a threshold so flat pseudo-labels are discarded, become the training signal for the student NCA. On the device, the paper adds an early-stopping rule based on the summed absolute change of hidden channels after a minimum of ten steps, which is what turns a three-second inference into one under a second.

What would settle it

Measure eNCApsulateD's output against known geometry from a simulated capsule environment or a calibrated phantom; if absolute depth error is large, or if the flatness filter removes most informative frames, the depth claim fails. A patient-disjoint bleeding segmentation test would also settle whether the Dice advantage over lightweight U-Nets holds beyond the data used.

Watch

Extended reading notes

Core claim

The central claim is that distillation and iterative cellular inference make it possible to compress bleeding segmentation and monocular depth estimation into NCA models of under 70 kB that run on an ESP32-S3 microcontroller, the kind of chip that fits inside a wireless capsule endoscope. On a held-out capsule-endoscopy test set, the segmentation model reaches 0.576 Dice, outperforming U-Net backbones that use millions of parameters, while the depth model is trained on pseudo ground truth produced by Depth Anything V2 and filtered by a flatness check; qualitatively it produces depth maps that the authors judge at least as realistic as the teacher's on difficult capsule images. A from-scratch C implementation with SIMD instructions reduces inference on the ESP32-S3 from roughly 9 s to 3 s per image, and a temporal early-stopping rule that halts inference when hidden-channel activity falls below a threshold cuts total NCA steps by a factor of 5. The paper concludes that this is the first demonstration of reliable bleeding segmentation and depth estimation on a miniaturized device, paving the way for visual odometry on the capsule.

Load-bearing premise

The depth-estimation claim rests entirely on the assumption that the pseudo ground truth produced by the large teacher model is accurate enough for capsule endoscopy; the paper has no independent depth ground truth, so a wrong teacher would make the student's apparent depth quality an artifact.

Editorial extensions

If this is right

  • Bleeding segmentation can run on the capsule at frame rates around 2–3 FPS, matching typical recording rates, so only frames with findings need to be transmitted.
  • On-device depth maps supply the missing input for visual odometry, opening a route to sensor-free localization of the capsule in the GI tract.
  • Model storage drops from millions of parameters to tens of thousands, so other diagnostic tasks could be embedded on the same class of microcontroller.
  • The early-stopping scheme lets users trade a small amount of accuracy for speed, which matters on battery-limited devices.
  • If the student can match a much larger teacher on this underrepresented domain, distillation from general-purpose models becomes a viable recipe for other niche medical imaging tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The flatness-filtered pseudo-label curation is a transferable idea: any distillation pipeline facing severe domain shift could use a similar gradient-based or confidence-based filter on teacher outputs before training.
  • The paper does not implement visual odometry, but its absolute-depth claim is the key enabling step; an immediate test would be feeding the NCA depth maps into an existing monocular VO pipeline.
  • The temporal early-stop threshold and cooldown are likely domain-specific; adapting them to other video streams with mostly static content could yield similar speedups on other edge devices.
  • If the Dice advantage over much larger CNNs persists on multi-patient data, it would suggest that local texture cues dominate bleeding detection in capsule images, rather than large-scale spatial context.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes eNCApsulate, a method that trains Neural Cellular Automata (NCAs) for bleeding segmentation and monocular depth estimation in wireless capsule endoscopy, and ports the trained models to an ESP32-S3 microcontroller. Segmentation is trained on manual labels from the KID2 dataset and evaluated against lightweight U-Net baselines in Table 1, where eNCApsulateS achieves a Dice score of 0.576 with 47,152 parameters. Depth estimation is trained by distilling Depth Anything V2 pseudo-labels on 727 curated KID2 images and is evaluated qualitatively on 25 selected KvasirCapsule images. The paper also describes a temporal early-stopping scheme that reduces NCA inference steps by roughly a factor of five and reports SIMD-based runtime optimizations that reduce per-image inference time on the ESP32-S3 from 9 s to 3 s, and further to under 1 s with early stopping. The central claim is that this is the first work enabling reliable bleeding segmentation and depth estimation on a miniaturized device, as a step toward on-capsule diagnosis and visual-odometry-based localization.

Significance. If the claims hold, the contribution is practically significant: NCAs on a capsule-sized microcontroller could enable on-device bleeding screening and reduce the amount of video data that must be transmitted, and the depth-estimation work is a step toward sensor-less capsule localization. The segmentation contribution is quantitatively supported by a five-fold ensemble evaluation with standard deviations, and the parameter advantage is stark (47,152 parameters versus the smallest baseline's 14,342,596). The deployment engineering, including the ANSI C implementation, SIMD usage, and buffer reuse, is a concrete and reproducible advance. The paper also provides a public code repository, which strengthens reproducibility. However, the depth-estimation claim is not established at the same evidentiary level: it relies entirely on Depth Anything V2 pseudo-labels, has no external ground truth, and is evaluated only qualitatively. The temporal-regularization claim likewise lacks quantitative segmentation metrics at the early cutoff.

major comments (5)
  1. [Section 4, Depth Estimation with eNCApsulateD] The depth-estimation claim is supported only by qualitative comparison on 25 selected KvasirCapsule images (Figs. 5 and 6), with no quantitative metric such as AbsRel, RMSE, or delta accuracy, and with no ground-truth depth for validation or test. The text itself acknowledges 'the absence of a proper ground truth for depth' in Section 4. Because eNCApsulateD is trained and assessed inside the loop of Depth Anything V2 pseudo-labels, any systematic error in the teacher is propagated and cannot be detected by the presented evaluation. I ask the authors to either evaluate on a dataset with real or simulated depth ground truth (e.g., EndoSLAM or VR-Caps) or to remove the word 'reliable' from the depth claim and present the depth model as a qualitative pseudo-label distillation.
  2. [Sections 3.1, 3.3, and Figure 2] The stochastic update probability is inconsistent between training and device deployment. Section 3.1 states that 'only 80% of cells are stochastically updated in each timestep,' while Figure 2's caption says the update is applied 'by a chance of 50%,' and Section 3.3 says the microcontroller code uses a '50:50 condition.' If the deployed model updates only 50% of cells per step but the evaluated PyTorch model was trained with 80%, the on-chip behavior likely differs from the reported quantitative results. Please align the training and deployment probabilities and report device-side segmentation metrics under the actual deployed probability.
  3. [Section 4, Temporal Regularization] The early-stopping claim 'at the same segmentation quality' is not quantitatively supported. The paper reports only the total number of NCA steps with and without early stopping (1,222,998 vs. 6,988,560), not the Dice or IoU at the early cutoff. Please report segmentation metrics, with variance, for full inference versus early-stopped inference on a per-frame basis, along with the distribution of stopping steps, so that the accuracy-runtime trade-off is actually verified.
  4. [Section 3.2, Depth curation] The depth training data are curated with a hand-set normalized gradient magnitude threshold of 1.1, and only 727 samples remain after curation. The paper neither justifies this threshold nor analyzes its sensitivity. Please provide a histogram of the curation criterion, an ablation over thresholds, and a breakdown of the remaining samples by patient or anatomy, so that reviewers can assess whether the small curated set supports the generalization claimed from the qualitative results.
  5. [Table 1 and Abstract] The size advantage is stated inconsistently: the Abstract claims 'more than 100x fewer parameters,' while Section 4 says the model is '10x smaller than the smallest baseline' (MobileNetV3s). In addition, the column header 'Size [B]' is ambiguous: it should state whether the values are parameter counts or stored bytes, and if bytes, the numeric format and quantization should be specified. Because the miniaturization claim is central to the paper, this inconsistency and ambiguity should be resolved.
minor comments (5)
  1. [Section 3.1] The stochastic update probability is described as 80% in the text and 50% in the Figure 2 caption; the figure and text should agree (this is also noted in the major comments, but the figure itself needs a correction).
  2. [Section 4, Segmentation] The sentence 'We also used similar settings and hyperparameters for the baseline training' is vague; please report the exact optimizer, learning rate, loss, and number of epochs used for each baseline model.
  3. [Figure 3] The axis labels in Figure 3 appear corrupted in the manuscript as sequences of '/uni0000...' characters; please replace them with readable text.
  4. [Section 3.2] The model names eNCApsulateS and eNCApsulateD are introduced without an explicit naming convention; please define the suffix convention at first use.
  5. [Section 4, Depth Evaluation] The qualitative depth evaluation is based on 'a subset of the KvasirCapsule dataset with interesting benchmark images'; please specify the selection criteria to avoid the appearance of cherry-picking.

Circularity Check

1 steps flagged · score 6.0 of 10

Depth reliability is evaluated against its own pseudo-label source; segmentation is independently benchmarked.

  1. fitted input called prediction [Sec. 3.2 (NCA Model Architecture and Training) and Sec. 4 (Depth Estimation with eNCApsulateD)]
    "eNCApsulateD is trained on a subset of the KID2 [17] dataset, which is passed through Depth Anything Model V2 [18] in order to obtain pseudo ground truth depth maps. ... One central limitation of our approach is the absence of a proper ground truth for depth, forcing us to use the best-performing foundation model as a pseudo ground truth. However, qualitative results (Fig. 5) indicate that eNCApsulateD performs well, and in some cases produces more convincing depth maps than the foundation models."

    The depth model is fit by MSE, SSIM, and gradient losses to Depth Anything V2 pseudo-labels, so the student NCA is by construction a distilled function of that teacher. The evidence offered for the depth claim is a qualitative comparison on unseen KvasirCapsule images against the same family of foundation models, with no external depth ground truth and no quantitative metric such as AbsRel or RMSE. Consequently, the abstract's 'reliable depth estimation' claim reduces to the assumption that Depth Anything V2's pseudo-labels are correct, which is exactly the training input. The segmentation arm is not affected because it uses manual KID2 labels and a held-out test set, so this is a partial circularity confined to the depth pillar.

full rationale

The segmentation contribution is independent: eNCApsulateS is trained on manual KID2 labels and evaluated on a held-out subset of KID2 against ImageNet-pretrained U-Net backbones (Table 1), so its Dice improvement is an external benchmark result. The self-citations to Med-NCA [9] are used only to motivate batch duplication and NCA robustness, not to establish the segmentation numbers, so they do not create circularity. The depth contribution is different. Section 3.2 defines the training signal as Depth Anything V2 pseudo-labels via MSE, SSIM, and gradient losses, and Section 4 explicitly admits there is no proper depth ground truth. The evaluation is qualitative, comparing the student against foundation models on KvasirCapsule images and even claiming the student sometimes beats the pseudo ground truth. Because both supervision and evaluation reference the same foundation-model outputs, the 'reliable depth estimation' claim in the abstract and conclusion is not independently grounded; agreement with the teacher on unseen images is a distillation-fidelity result, not evidence of metric depth accuracy. This is a partial circularity affecting the depth pillar only.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The depth result rests on a teacher model's pseudo-labels, the segmentation result rests on standard manual annotations. Hyperparameters for depth curation and early stopping are hand-set. No new physical entities are introduced.

free parameters (4)
  • Depth map curation threshold = 1.1 (normalized gradient magnitude)
    Images whose pseudo depth maps appear flat are discarded if normalized gradient magnitude is at or below 1.1; this is a hand-set threshold with no sensitivity analysis (Sec. 3.2).
  • Temporal regularization threshold and cooldown = 0.1 threshold, cooldown 5, minimum steps 10
    Early cutoff when normalized hidden-channel change falls below 0.1, with a cooldown counter; no ablation on these values is reported (Sec. 3.4).
  • Stochastic update probability = 0.8 in training, 0.5 on device
    Training uses 80% stochastic cell updates (Sec. 3.1), but the ESP32 implementation is described as a 50:50 condition (Sec. 3.3); the discrepancy is unexplained.
  • Depth loss weights = lambda_MSE=1.0, lambda_SSIM=1.0, lambda_grad=0.1
    Hand-chosen weights for depth training; no ablation is shown (Sec. 3.2).
assumptions (4)
  • domain assumption NCA local update rule converges in about 100 time steps to a stable segmentation or depth output.
    Adopted from Mordvintsev et al. [16] and Kalkhof et al. [9]; the paper does not re-derive convergence for the capsule endoscopy setting.
  • ad hoc to paper Pseudo depth labels from Depth Anything V2 are valid enough to train and evaluate a depth estimator for capsule endoscopy.
    Stated in Sec. 3.2 as the training target; no external depth ground truth is used, so the depth claim is conditional on this assumption.
  • domain assumption Downsampling to 64x64 patches preserves sufficient spatial detail for bleeding segmentation and depth estimation.
    All training uses 64x64 patches (Sec. 3.2); no evaluation on higher resolutions is reported.
  • domain assumption The ESP32-S3 is representative of the compute and memory envelope of a wireless capsule endoscope.
    The conclusion extrapolates from the dev board to a capsule; no thermal, power, or physical integration test is presented.

how reviews work

0 comments
Cite this review

Pith. "Pith review of eNCApsulate: NCA for Precision Diagnosis on Capsule Endoscopes." pith.science (2026). https://pith.science/paper/3DFOWMCA

@misc{pith2026250421562,
  author       = {Pith},
  title        = {Pith review of: eNCApsulate: NCA for Precision Diagnosis on Capsule Endoscopes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3DFOWMCA}},
  note         = {Machine review of arXiv:2504.21562}
}
read the original abstract

Wireless Capsule Endoscopy is a non-invasive imaging method for the entire gastrointestinal tract, and is a pain-free alternative to traditional endoscopy. It generates extensive video data that requires significant review time, and localizing the capsule after ingestion is a challenge. Techniques like bleeding detection and depth estimation can help with localization of pathologies, but deep learning models are typically too large to run directly on the capsule. Neural Cellular Automata (NCA) for bleeding segmentation and depth estimation are trained on capsule endoscopic images. For monocular depth estimation, we distill a large foundation model into the lean NCA architecture, by treating the outputs of the foundation model as pseudo ground truth. We then port the trained NCA to the ESP32 microcontroller, enabling efficient image processing on hardware as small as a camera capsule. NCA are more accurate (Dice) than other portable segmentation models, while requiring more than 100x fewer parameters stored in memory than other small-scale models. The visual results of NCA depth estimation look convincing, and in some cases beat the realism and detail of the pseudo ground truth. Runtime optimizations on the ESP32-S3 accelerate the average inference speed significantly, by more than factor 3. With several algorithmic adjustments and distillation, it is possible to eNCApsulate NCA models into microcontrollers that fit into wireless capsule endoscopes. This is the first work that enables reliable bleeding segmentation and depth estimation on a miniaturized device, paving the way for precise diagnosis combined with visual odometry as a means of precise localization of the capsule -- on the capsule.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 14 canonical work pages

  1. [1]

    Chinese Journal of Digestive Diseases 4(2), 89–92 (2003) https://doi

    Ge, Z.Z., Hu, Y.B., Gao, Y.J., Xiao, S.D.: Clinical application of wireless capsule endoscopy. Chinese Journal of Digestive Diseases 4(2), 89–92 (2003) https://doi. org/10.1046/j.1443-9573.2003.t01-1-00120.x

    work page arXiv 2003

  2. [2]

    Gastrointestinal Endoscopy 68(1), 174–180 (2008) https://doi.org/10.1016/j.gie

    Li, F., Gurudu, S.R., De Petris, G., Sharma, V.K., Shiff, A.D., Heigh, R.I., Fleischer, D.E., Post, J., Erickson, P., Leighton, J.A.: Retention of the cap- sule endoscope: A single-center experience of 1000 capsule endoscopy procedures. Gastrointestinal Endoscopy 68(1), 174–180 (2008) https://doi.org/10.1016/j.gie. 2008.02.037

    work page doi:10.1016/j.gie 2008

  3. [3]

    IEEE Sen- sors Journal 24(12), 19190–19201 (2024) https://doi.org/10.1109/JSEN.2024

    Ali, M.A., Alsunaydih, F.N., Rathnayaka, A., Yuce, M.R.: Implementing an autonomous navigation system for active wireless capsule endoscopy. IEEE Sen- sors Journal 24(12), 19190–19201 (2024) https://doi.org/10.1109/JSEN.2024. 3391797

    doi:10.1109/jsen.2024 2024

  4. [4]

    Scientific Reports 11(1), 11204 (2021) https://doi.org/10.1038/s41598-021-90523-w

    Vedaei, S.S., Wahid, K.A.: A localization method for wireless capsule endoscopy using side wall cameras and IMU sensor. Scientific Reports 11(1), 11204 (2021) https://doi.org/10.1038/s41598-021-90523-w

    doi:10.1038/s41598-021-90523-w 2021

  5. [5]

    Nature Communications 15(1), 4597 (2024) https://doi.org/10.1038/ s41467-024-49019-0

    Cao, Q., Deng, R., Pan, Y., Liu, R., Chen, Y., Gong, G., Zou, J., Yang, H., Han, D.: Robotic wireless capsule endoscopy: Recent advances and upcoming tech- nologies. Nature Communications 15(1), 4597 (2024) https://doi.org/10.1038/ s41467-024-49019-0

    work page 2024

  6. [6]

    Medical image analysis 71, 102058 (2021)

    Ozyoruk, K.B., Gokceler, G.I., Bobrow, T.L., Coskun, G., Incetan, K., Almalioglu, Y., Mahmood, F., Curto, E., Perdigoto, L., Oliveira, M., et al.: Endoslam dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Medical image analysis 71, 102058 (2021)

    work page 2021

  7. [7]

    arXiv (2024)

    Han, J.J., Acar, A., Henry, C., Wu, J.Y.: Depth Anything in Medical Images: A Comparative Study. arXiv (2024)

    work page 2024

  8. [8]

    arXiv (2024)

    Li, B., Liu, B., Yue, J., Zhou, F.: Advancing Depth Anything Model for Unsu- pervised Monocular Depth Estimation in Endoscopy. arXiv (2024). https://doi. org/10.48550/arXiv.2409.07723

    arXiv doi:10.48550/arxiv.2409.07723 2024

Show all 19 references
  1. [9]

    In: International Conference on 11 Information Processing in Medical Imaging, pp

    Kalkhof, J., Gonz´ alez, C., Mukhopadhyay, A.: Med-nca: Robust and lightweight segmentation with neural cellular automata. In: International Conference on 11 Information Processing in Medical Imaging, pp. 705–716 (2023). Springer

    2023

  2. [10]

    : Kvasir- capsule, a video capsule endoscopy dataset

    Smedsrud, P.H., Thambawita, V., Hicks, S.A., Gjestang, H., Nedrejord, O.O., Næss, E., Borgli, H., Jha, D., Berstad, T.J.D., Eskeland, S.L., et al. : Kvasir- capsule, a video capsule endoscopy dataset. Scientific Data 8(1), 142 (2021)

    2021

  3. [11]

    Scientific Reports 12(1), 15708 (2022) https://doi.org/10.1038/ s41598-022-19675-7

    Vats, A., Mohammed, A., Pedersen, M.: From labels to priors in cap- sule endoscopy: A prior guided approach for improving generalization with few labels. Scientific Reports 12(1), 15708 (2022) https://doi.org/10.1038/ s41598-022-19675-7

    2022

  4. [12]

    Computers in Biology and Medicine 181, 109038 (2024) https://doi.org/10.1016/j.compbiomed.2024

    Jeong, B.H., Kim, H.K., Son, Y.D.: Depth estimation from monocular endoscopy using simulation and image transfer approach. Computers in Biology and Medicine 181, 109038 (2024) https://doi.org/10.1016/j.compbiomed.2024. 109038

    2024 doi

  5. [13]

    Scientific reports 12(1), 13723 (2022)

    Sahafi, A., Wang, Y., Rasmussen, C., Bollen, P., Baatrup, G., Blanes-Vidal, V., Herp, J., Nadimi, E.: Edge artificial intelligence wireless video capsule endoscopy. Scientific reports 12(1), 13723 (2022)

    2022

  6. [14]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Kalkhof, J., Mukhopadhyay, A.: M3d-nca: Robust 3d segmentation with built-in quality control. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 169–178 (2023). Springer

    2023

  7. [15]

    arXiv (2024)

    Kalkhof, J., Ranem, A., Mukhopadhyay, A.: Unsupervised Training of Neural Cellular Automata on Edge Devices. arXiv (2024)

    2024

  8. [16]

    Distill 5(2), 23 (2020)

    Mordvintsev, A., Randazzo, E., Niklasson, E., Levin, M.: Growing neural cellular automata. Distill 5(2), 23 (2020)

    2020

  9. [17]

    Endosc Int Open 5(6), 477–483 (2017)

    Koulaouzidis, A., Iakovidis, D.K., Yung, D.E., Rondonotti, E., Kopylov, U., Plevris, J.N., Toth, E., Eliakim, A., Wurm Johansson, G., Marlicz, W., Mavroge- nis, G., Nemeth, A., Thorlacius, H., Tontini, G.E.: KID Project: an internet-based digital video atlas of capsule endosco...

    2017

  10. [18]

    arXiv:2406.09414 (2024)

    Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., Zhao, H.: Depth anything v2. arXiv:2406.09414 (2024)

    2024 arXiv

  11. [19]

    Incetan, K., Celik, I.O., Obeid, A., Gokceler, G.I., Ozyoruk, K.B., Almalioglu, Y., Chen, R.J., Mahmood, F., Gilbert, H., Durr, N.J., Turan, M.: VR-Caps: A Virtual Environment for Capsule Endoscopy (2020) 12

    2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.