REVIEW 4 major objections 5 minor 12 references
A 102.3-million-parameter stroke lesion segmentation network can be distilled into a 0.84-million-parameter student that preserves Dice and F1 while cutting energy use by over 90% and inference time by 84%.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 11:22 UTC pith:7OANVZMN
load-bearing objection Solid deployment engineering with real, measured efficiency gains, but the performance-preservation claim is under-supported: no numeric segmentation results, no teacher-vs-student significance test, and an FP16 CPU timing inconsistency. the 4 major comments →
StrokeSeg2: Stroke Lesion Segmentation in Clinical Research Workflows
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that segmentation accuracy is largely preserved down to roughly one million parameters, and that S7 (0.84M parameters, 2.1 MB FP16) is the point where compression has not yet degraded clinically relevant metrics. Statistical equivalence between the quantised ONNX model and the PyTorch baseline was checked with TOST at δ=10⁻³ (p<0.001), and performance curves show that lesion-wise detection (F1) stays flat while Dice and boundary accuracy degrade only below S7. The runtime experiments show that the architectural compression of S7 already delivers 15.2× speed-up on CPU, 5.5× on iGPU and 4.7× on dGPU at FP32, and that FP16 does not materially change speed. The paper therefo
What carries the argument
The load-bearing mechanism is knowledge distillation: the student is trained with L = (1−α)L_hard + α T² D_KL(σ(z_t/T) || σ(z_s/T)) with α = 0.5 and T = 4.0, so the student learns both the ground-truth mask and the teacher's softened output distribution. This lets a small architecture imitate the teacher's boundary decisions rather than only the binary labels. ONNX Runtime FP16 quantisation then shrinks the checkpoint by roughly 70% while the TOST test shows numerical equivalence. The third piece is the modular C++/Qt shell: it separates preprocessing, inference, and postprocessing, bundles all dependencies into an installer, and auto-selects among dGPU, iGPU, and CPU.
Load-bearing premise
The load-bearing premise is that the chosen evaluation metrics on 300 publicly available T1-weighted chronic stroke subjects, at a fixed 0.5 threshold, capture what makes segmentation clinically useful; if small-lesion accuracy — where even the teacher scores visibly lower — is the clinically critical quantity, then S7's 'preserved performance' claim would not follow.
What would settle it
Run S7 and the teacher side-by-side on an independent cohort of predominantly small chronic stroke lesions and test lesion-wise Dice and F1 with a paired non-inferiority or equivalence test at a clinically chosen margin; a significant deficit for S7 would falsify the preserved-performance claim.
If this is right
- A standard clinical workstation without a dedicated GPU can run near-teacher-quality stroke segmentation in seconds rather than tens of seconds per volume.
- Energy per inference drops by more than 90%, making large retrospective stroke cohorts computationally affordable and lowering the carbon cost of segmentation.
- A 2.1 MB model weight enables a fully self-contained installer, so reproducibility no longer depends on reconstructing a Python environment.
- Because distillation, not quantisation, drives the gains, future compression efforts should focus on architecture size while treating FP16 as a storage optimisation.
- The same auto-device-selection design lets one installer serve CPU-only and GPU-equipped sites identically.
Where Pith is reading between the lines
- The 'preserved performance' conclusion is tied to the evaluation protocol: 300 T1w chronic stroke scans, a fixed 0.5 threshold, and global/lesion-wise Dice/F1. On small lesions, confidence intervals are wide even for the teacher, so the student could fall short in settings where small-lesion accuracy drives clinical decisions; a lesion-size-stratified paired test would settle this.
- The runtime/energy figures are for inference on the model only. End-to-end clinical processing includes brain extraction, resampling, patch merging, and postprocessing, so a workflow-level measurement could report smaller overall savings.
- The same distillation-plus-FP16 recipe should transfer to other 3D lesion segmentation tasks (tumour, multiple sclerosis) wherever teacher logits are available; the paper only demonstrates stroke.
- A practical extension the authors leave implicit is silent per-site validation: the self-contained installer could collect local Dice or ASD statistics on a small labelled subset, letting each hospital confirm the compressed model before routine use.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. StrokeSeg2 presents a lightweight, cross-platform C++/Qt framework for deploying compressed stroke lesion segmentation models in clinical research settings. The authors distill an nnU-Net teacher (102.3M parameters) into a 0.84M-parameter student (S7) via knowledge distillation, then convert to ONNX Runtime with FP16 quantization. Evaluated on 300 ATLAS v2.1 subjects across CPU, iGPU, and dGPU, the paper reports >90% energy savings, an average 84% inference time reduction, and a 2.1 MB model footprint while claiming preserved segmentation performance. The main contribution is the software engineering pipeline and the efficiency evaluation, with the performance-preservation claim resting primarily on Fig. 2 and a PyTorch-vs-ONNX equivalence test.
Significance. If the claims hold, StrokeSeg2 would be a practically useful contribution: it demonstrates that a heavily compressed stroke segmentation model can run efficiently on standard clinical workstations and is packaged as self-contained installers. The efficiency measurements are direct and reproducible on public data, and the paper ships code links and documents the compute budget. The central 'preserved segmentation performance' claim, however, is not yet quantitatively supported, and the model-selection procedure risks biasing the reported performance. The software engineering aspects are solid, but the evaluation of the core compression trade-off needs strengthening before the results can be fully accepted.
major comments (4)
- [§3.2, Fig. 2] The central claim that S7 'preserves robust lesion localisation and competitive segmentation performance' is not supported numerically. The text only qualitatively describes the curves. Provide a table with point estimates and 95% confidence intervals for global Dice, lesion-wise Dice, lesion-wise F1, and ASD for the teacher and all students, especially S7, along with a paired statistical comparison (e.g., a paired test or equivalence test with a clinically motivated margin) between teacher and S7. Without this, 'preserved' is ambiguous. Additionally, Fig. 2 shows small lesions have visibly lower scores and wider intervals; the paper does not quantify the teacher-vs-S7 difference in the small-lesion stratum, which is a gap if small-lesion accuracy matters clinically.
- [§3.1, second paragraph] The TOST is used to establish statistical equivalence between PyTorch and ONNX FP16 (δ=10⁻³), but this does not address teacher-vs-student equivalence. The reader is left without a significance test for the actual compression step. Also, the manuscript states 'all comparisons yielding p<0.001' but does not report TOST results for ASD or F1, only Dice. Please clarify whether equivalence holds for all metrics and state the assumptions of the TOST (e.g., normality or use of bootstrap).
- [§3.3, Table 2] Time and energy values are single point estimates with no variance, number of repetitions, or measurement protocol (e.g., power measurement device, idle-power correction). The 'over 90% reduction' claim needs at least a brief description of the measurement procedure and, ideally, confidence intervals or standard deviations. Furthermore, the table shows S7 FP16 CPU inference at 2.7s versus S7 FP32 at 1.5s—an 80% slowdown—which contradicts the statement that FP16 'produced only minor changes in inference time' compared with FP32. This must be explained or corrected.
- [§2.2 / §3.2] The nine student architectures were evaluated on the same 300-subject ATLAS v2.1 test set used to select S7 as the 'most favourable trade-off'. Because S7 is chosen post hoc from the performance curves on this test set, the reported performance is subject to selection bias. Please either use a separate validation set for model selection (and hold out the test set for final evaluation), or explicitly discuss the potential magnitude of this bias and its effect on the 'preserved' claim.
minor comments (5)
- [Abstract and §3.3] The abstract states 'average 84% reduction in inference time', but Table 2 does not show how this average is computed across the three hardware configurations. Specify the averaging method (e.g., mean across CPU/iGPU/dGPU for FP32 or FP16).
- [Fig. 1] The abbreviations 'Se', 'Cv', and 'Cr' in Fig. 1 are not explained in the caption or main text. Please define them in the caption for clarity.
- [§2.4] The code repositories are mentioned only in the Data and software availability section. Consider referencing them in the main text where the software architecture is described, to improve reproducibility visibility.
- [Table 1] The column 'Red. Ratio' is used without a definition. It appears to be the ratio of ONNX FP16 size to PyTorch FP32 size, but the caption should state this explicitly.
- [Conclusion] The conclusion restates that the student 'maintains segmentation performance' without the caveat of 'moderate reductions' noted in §3.2. Align the wording to be consistent with the quantitative results.
Circularity Check
No significant circularity: the core segmentation and efficiency claims are direct measurements on public ATLAS data; self-citations are contextual inputs, not load-bearing reductions.
full rationale
The paper's central claims are experimental rather than derivational: inference time, energy consumption, model size, and segmentation metrics (Dice, ASD, F1) are directly measured on 300 ATLAS v2.1 subjects across three hardware configurations. No prediction is obtained by inverting a fitted parameter, and no equation reduces to its own input. The two self-references — the teacher model of Di Matteo et al. [2] and the earlier StrokeSeg framework [8] — are used as an external pretrained starting point and as a packaging baseline, respectively; they are not invoked as proof of the new results. The TOST equivalence claims concern PyTorch versus ONNX FP16 implementations of the same architecture and are again direct paired measurements, not circular derivations. Selecting S7 by inspecting the same test set is a methodological selection-bias concern, but it is not a definitional equivalence or a fitted-input-called-prediction circularity. Thus the paper is self-contained against public external benchmarks, with only minor self-citation that is not load-bearing.
Axiom & Free-Parameter Ledger
free parameters (4)
- distillation mixture weight α =
0.5
- distillation temperature T =
4.0
- student architecture widths (S7 selected) =
[4, 8, 16, 32, 48, 48] features per stage
- TOST equivalence margin δ =
1e-3
axioms (5)
- domain assumption The pretrained teacher from [2] — same research group — is a valid high-performance reference for stroke lesion segmentation.
- domain assumption ATLAS v2.1's 300 subjects are independent of the v2.0 training set and representative of chronic stroke T1w scans.
- domain assumption Dice, lesion-wise F1 with 10% overlap, and ASD at a fixed 0.5 threshold capture clinically relevant segmentation behaviour.
- domain assumption Energy measurements corrected for baseline idle power isolate true inference energy consumption.
- standard math Standard softmax KL-divergence distillation behaves as expected for this model family.
read the original abstract
Deep learning frameworks like nnU-Net achieve state-of-theart brain lesion segmentation performance but remain difficult to deploy in clinical research environments due to, among other reasons, software dependencies and computational requirements. We introduce StrokeSeg2, a lightweight, modular, cross-platform C++/Qt framework designed to adapt resource-intensive 3D stroke segmentation pipelines into portable and reproducible applications. To improve compatibility with standard clinical workstations, we investigate the combined effect of architectural compression through knowledge distillation and inference optimisation using ONNX Runtime with Float16 quantisation. Across heterogeneous hardware configurations (CPU, integrated GPU, and dedicated GPU) architectural distillation emerged as the primary contributor to efficiency gains, contributing to over 90% reduction in energy consumption and an average 84% reduction in inference time. Specifically, we identify a 0.84M-parameter student model as the most favourable trade-off, reducing the original 102.3M-parameter teacher architecture to a 2.1 MB disk footprint while preserving robust lesion localisation and competitive segmentation performance. This small footprint supports the development of a self-contained installer for clinical workstation targets. Finally, StrokeSeg2 packages these optimisations into standalone installers for Windows, macOS, and Linux. By providing both graphical and commandline interfaces without Docker or external environment dependencies, StrokeSeg2 facilitates deployment of high-performance segmentation workflows for routine clinical research pipelines.
Figures
Reference graph
Works this paper leans on
-
[1]
International Journal of Stroke12(5), 444–450 (2017).https://doi.org/10.1177/1747493017711816
Bernhardt, J., Borschmann, K., Boyd, L., et al.: Agreed definitions and a shared vision for new standards in stroke recovery research: the stroke recovery and re- habilitation roundtable taskforce. International Journal of Stroke12(5), 444–450 (2017).https://doi.org/10.1177/1747493017711816
-
[2]
Pattern Recognition Letters199, 225–231 (2026)
Di Matteo, A., Mahé, Y., Leplaideur, S., Bonan, I., Bannier, E., Galassi, F.: Deep learning and multi-modal mri for the segmentation of sub-acute and chronic stroke lesions. Pattern Recognition Letters199, 225–231 (2026). https://doi.org/https://doi.org/10.1016/j.patrec.2025.11.017,https:// www.sciencedirect.com/science/article/pii/S0167865525003745
-
[3]
https://anima.irisa.fr/, accessed: 2025-10-19
Empenn Team, I..I.: Anima and anima-scripts pipelines for medical image analysis. https://anima.irisa.fr/, accessed: 2025-10-19
2025
-
[4]
The Lancet Neurology20(10), 795–820 (2021).https://doi.org/10.1016/S1474-4422(21)00252-0
Feigin,V.L.,Stark,B.A.,Johnson,C.O.,Collaborators,G..S.:Global,regional,and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the global burden of disease study 2019. The Lancet Neurology20(10), 795–820 (2021).https://doi.org/10.1016/S1474-4422(21)00252-0
-
[5]
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network (2015),https://arxiv.org/abs/1503.02531
Pith/arXiv arXiv 2015
-
[6]
Nature Methods18(2), 203–211 (2021).https://doi.org/10.1038/s41592-020-01008-z
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-adapting framework for u-net-based medical image segmentation. Nature Methods18(2), 203–211 (2021).https://doi.org/10.1038/s41592-020-01008-z
-
[7]
Jungo, A., Scheidegger, O., Reyes, M., Balsiger, F.: pymia: A python pack- age for data handling and evaluation in deep learning-based medical image analysis. Computer Methods and Programs in Biomedicine198, 105796 (2021).https://doi.org/https://doi.org/10.1016/j.cmpb.2020.105796, https://www.sciencedirect.com/science/article/pii/S0169260720316291
arXiv 2021
-
[8]
Kerverdo, Y., Leray, F., Mahé, Y., Leplaideur, S., Galassi, F.: Stroke lesion seg- mentation in clinical workflows: A modular, lightweight, and deployment-ready tool (2025),https://arxiv.org/abs/2510.24378
arXiv 2025
-
[9]
Scientific Data5, 180011 (2018).https: //doi.org/10.1038/sdata.2018.11
Liew,S.L.,Anglin,J.M.,Banks,M.A.,etal.:Theatlasdatabaseofstrokestructural images and manual lesion segmentations. Scientific Data5, 180011 (2018).https: //doi.org/10.1038/sdata.2018.11
-
[10]
Project MONAI: Monai deploy: Medical ai deployment framework.https:// monai.io/deploy.html(2023), accessed: 2025-09-22
2023
-
[11]
Pérez-García, F., Sparks, R., Ourselin, S.: Torchio: A python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Computer Methods and Programs in Biomedicine208, 106236 (2021).https://doi.org/https://doi.org/10.1016/ j.cmpb.2021.106236,https://www.sciencedirect.com/science/article/pii/ S...
arXiv 2021
-
[12]
Canadian Journal of Cardiology40(10), 1828–1840 (2024)
Theriault-Lauzier, P., Cobin, D., Tastet, O., et al.: A responsible framework for applying artificial intelligence on medical images and signals at the point of care: The pacs-ai platform. Canadian Journal of Cardiology40(10), 1828–1840 (2024). https://doi.org/10.1016/j.cjca.2024.05.025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.