Pith. sign in

REVIEW 3 major objections 4 minor 26 references

TotalRegistrator: Towards a Lightweight Foundation Model for CT Image Registration

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A whole-body CT registration model trained on 591 pairs competes with specialist models on unseen tasks.

desk verdict A credible, useful whole-body CT registration model with a genuinely practical training trick; the main open question is whether the in-house train/eval split is patient-disjoint. read the letter →

arxiv 2508.04450 v1 pith:UAIUHO5H submitted 2025-08-06 eess.IV cs.CV

Xuan Loc Pham , Gwendolyn Vuurberg , Marjan Doppen , Joey Roosen , Tip Stille , Thi Quynh Ha , Thuy Duong Quach , Quoc Vu Dang
show 7 more authors
Manh Ha Luu Ewoud J. Smit Hong Son Mai Mattias Heinrich Bram van Ginneken Mathias Prokop Alessa Hering
This is my paper · ORCID
classification eess.IVcs.CV
keywords imageregistrationwhole-bodyCTmulti-organfielddecompositionunsupervisedlearningUNetmedicalfoundationmodellongitudinal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TotalRegistrator claims that a general-purpose whole-body CT registration model can be trained from scratch on one carefully curated longitudinal dataset of 591 whole-body scan pairs, without aggregating many public datasets or using heavy compute. The paper's key move is to replace one complex global deformation field with several independently trained region-specific fields, composed sequentially and finally smoothed by a whole-body block. On its held-out in-house test set, the model generally outperforms a classical iterative tool and a recent foundation model on multi-organ abdominal alignment, while lagging slightly on lungs and bone. On three external datasets it remains competitive with single-organ specialist models despite never being fine-tuned. If this holds, whole-body registration becomes feasible for hospitals with ordinary GPUs and modest training data.

What carries the argument

The central mechanism is field decomposition: the whole-body deformation field is split into region-specific displacement fields for bone, thorax, and abdomen, each produced by an independent standard UNet after a shared affine alignment. The fields are composed sequentially, and a whole-body block, trained on the frozen outputs of the earlier blocks, integrates and regularizes them. Training memory is kept to about 11GB by performing forward and backward passes per organ instead of maintaining one cumulative gradient graph for all organs.

What would settle it

Manually correct the training-time organ masks for a subset of cases, retrain under identical settings, and compare held-out Dice; if scores change materially, the automatic labels were load-bearing. Alternatively, test on whole-body CT scans from scanners and patient populations not represented in the training cohort; a large drop in abdominal Dice would falsify the generalizability claim.

Watch

Extended reading notes

Core claim

The central claim is that a single well-curated longitudinal dataset of whole-body CT pairs is enough to train a general multi-organ registrator from scratch, provided the deformation problem is decomposed by anatomical region. The model uses standard UNet blocks, an unsupervised training loss combining mutual-information similarity, overlap of automatically generated organ masks, and deformation regularization. The authors report that, on their held-out in-house pairs, TotalRegistrator achieves the highest average Dice for abdominal organs among the compared methods in both intra-patient and inter-patient settings, with a lower folding rate than a recent foundation model in intra-patient ca

Load-bearing premise

The model's training depends on organ segmentation masks that were generated automatically and never manually corrected; if those masks carry systematic errors, the learned deformation fields and the reported Dice improvements are biased.

Editorial extensions

If this is right

  • If the claim is correct, a whole-body multi-organ registration model can be built from a single well-curated longitudinal dataset rather than a large mixture of public datasets.
  • A standard UNet backbone plus the decomposition strategy is enough to compete with specialized single-organ models on unseen tasks, reducing the need for task-specific training.
  • Training on 11GB of GPU memory makes the approach practical for hospitals and smaller research groups.
  • The model's robustness to contrast variation, shown on multiphase liver data, suggests it can be applied to non-contrast and multiphase abdominal CT without retraining.
  • Bone and lung alignment remain weaker points, indicating that decomposition alone does not automatically make every anatomical region a specialist.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The design implies the decomposition, not the UNet itself, is the active ingredient; a testable extension is to add more region blocks (for example, head or neck) and check whether the whole-body integration block absorbs them without retraining earlier blocks.
  • Because only test labels were manually refined, the reported Dice may depend on the quality of automatically generated training masks; retraining with expert-corrected masks would reveal how much the automatic labels matter.
  • The per-organ backpropagation trick decouples memory use from the number of organs, so it could be transplanted to higher-capacity architectures such as transformer-based registration networks.
  • The reliance on segmentation masks in training means organs without reliable automatic masks are currently uncovered; the stated path to a full foundation model is to extend to all structures an automatic whole-body segmenter can label.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. TotalRegistrator is a learning-based whole-body CT registration method trained on a single in-house longitudinal dataset. The architecture uses standard UNet blocks for affine and deformable stages, plus a field-decomposition strategy that divides the deformation field into thoracic, abdominal, bone, and whole-body components; components are trained independently and then composed. The model is evaluated on 104 in-house whole-body pairs (intra- and inter-patient) and on three external datasets (AbdomenCTCT, NLST, H108M). The central claim is that TotalRegistrator generally outperforms Elastix and uniGradICON on multi-organ abdominal Dice while remaining competitive on external data, and that it can be trained on 11GB GPU memory. The authors acknowledge limitations in bone registration, folding in small organs, and the segmentation-mask-driven loss.

Significance. If the reported results hold, the paper makes a useful practical contribution: it demonstrates that a lightweight, single-dataset-trained model with region-specific field decomposition can approach, and on abdominal structures exceed, a larger foundation model, lowering hardware barriers for multi-organ CT registration. The manuscript has concrete strengths: a carefully curated 695-pair whole-body dataset with demographic and scanner diversity, external evaluation, ablations of the individual registration blocks, a transparent limitation section, and a plan for public code. The main caveats are the ambiguous train/test patient split, unvalidated automatic training labels, and lack of statistical significance testing, which currently leave the strength of the central comparative claim uncertain.

major comments (3)
  1. [Section 3.1, Table 1] The paper must state explicitly that the 591 training and 104 evaluation pairs are patient-disjoint. The current text says 695 pairs 'from patients' and describes patient demographics, which suggests one pair per patient, but it never says 'one pair per patient' or that the split was stratified by patient. If the same patient contributed to both training and test sets, the segmentation-overlay loss could memorize patient-specific anatomy and inflate the reported DSC gain over Elastix and uniGradICON, which were not trained on RUMC data. This is load-bearing for the primary in-house claim. Please clarify, and if the split was not patient-disjoint, re-run the evaluation on a patient-disjoint split.
  2. [Section 3.2(a), Fig. 6] Training masks are produced by TotalSegmentator without expert refinement, while only test masks are manually refined. Because the segmentation-overlay loss is a core training signal, systematic TotalSegmentator errors on small structures such as the pancreas and gallbladder could bias the learned deformation fields and downstream organ-specific comparisons. The authors should quantify label noise (e.g., Dice between TotalSegmentator and refined masks on the test set) or otherwise justify that the automatic training labels are accurate enough to drive the reported results.
  3. [Section 4, Tables 2-5] The paper uses phrases such as 'significantly better' and 'significant difference' (e.g., Section 4.1) without reporting statistical tests. Given the large variability for small abdominal organs shown in Fig. 7, the claimed superiority over Elastix and uniGradICON should be supported by paired tests or confidence intervals across the 104 in-house pairs. Without this, the central comparative claim is not fully supported.
minor comments (4)
  1. [Section 3.1(b), Abstract] External evaluations use the Learn2Reg validation subsets, not the official test sets. The abstract and Section 5.2 should consistently say 'validation subset' rather than phrasing that implies full official evaluation.
  2. [Section 2, Fig. 4] The heading contains a typo: 'Apporach' should be 'Approach'. Also, the term 'novel field decomposition' should be positioned against prior cascaded registration works (Zhao et al., 2019; Pham et al., 2024) to clarify the incremental contribution.
  3. [Section 3.2(a)] The list of annotated structures totals 12 if kidneys and lungs are counted as single labels, not the stated 13. Please clarify whether left/right organs are counted separately or whether the whole-body mask is included in the count.
  4. [Fig. 7] The boxplots are dense; method names and organ labels are difficult to read at print size. Consider splitting by region or increasing font size.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: central results are held-out and externally benchmarked; self-citations are not load-bearing.

full rationale

This is an empirical machine-learning paper, not a formal derivation, so the classic circularity failure modes largely do not apply. The central claims — that TotalRegistrator surpasses Elastix and uniGradICON on multi-organ abdominal registration and generalizes to external datasets — are supported by evaluation on a held-out RUMC test subset (104 pairs) and on three external datasets (AbdomenCTCT, NLST, H108M) that were not used for training. The training labels were generated automatically with TotalSegmentator, while the test labels were manually refined by medical experts (Figure 6: 'We first used TotalSegmentator to obtain the multi-organ coarse labels for all scans in the dataset. Subsequently, medical experts were invited to manually refine the coarse labels of the test set.'), so evaluation is not performed on the same labels that drove training. The main baseline, uniGradICON, is an independent method. Self-citations appear — the SPIE preliminary version (Pham et al., 2025) and the H108M dataset from Pham et al., 2024 — but they are not load-bearing: the design choice based on the preliminary work is corroborated by in-paper ablations, and H108M is used only as an external test set. No equation or definition reduces a claimed prediction to an input. The potential patient-level leakage concern is a data-integrity risk, not a circularity, and no quoted text exhibits same-patient overlap between training and test. The paper's own stated limitation in Section 5.3 that the DSC-based loss may bias toward global shape alignment is a metric limitation, not circularity. Overall, the derivation chain is self-contained against external benchmarks, so no significant circularity is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

TotalRegistrator introduces an architectural strategy, not physical entities. The hand-chosen hyperparameters are input resolution, region grouping, pipeline order and loss weights. The main domain assumptions concern label quality, dataset diversity, metric validity and downsampling adequacy.

free parameters (4)
  • Fixed input resolution (128x96x160) = hand-set
    All training volumes are resampled to this resolution; the paper states this in Section 5.1 and notes the model generalizes to high-resolution inputs. This choice discards image detail and affects registration accuracy.
  • Region grouping and pipeline order = hand-set
    The division into bone, thorax, abdomen and whole-body blocks, and the order affine -> bone -> thorax -> abdomen -> whole-body, is a design choice validated by ablation rather than derived from first principles.
  • Loss weights for similarity, segmentation overlay and regularization = not fully reported
    The paper mentions a combination of similarity loss, segmentation overlay loss and deformation regularization loss, but the supplied text does not give the exact weights used in training.
  • Deformation field regularization strength = not fully reported
    A global regularization weight is used to control folding; the paper suggests per-organ regularization as future work, confirming the global weight is a hand-chosen hyperparameter.
assumptions (4)
  • domain assumption TotalSegmentator coarse labels on the training set are accurate enough to drive the segmentation-overlay loss
    Section 3.1 and Figure 6 describe using TotalSegmentator for coarse labels with manual refinement reserved for the test set. If training labels are noisy, the learned deformations and reported Dice are biased.
  • domain assumption A single-center longitudinal whole-body CT dataset of 591 pairs captures sufficient anatomical and scanner diversity for a general-purpose model
    The paper argues this in Sections 1 and 5.1, citing scanner variety and pathological diversity. The external test results are the main evidence, but the assumption is load-bearing for the foundation model claim.
  • domain assumption Organ-level Dice on 13 structures is an adequate proxy for registration quality
    The paper uses DSC as the main metric and acknowledges in Section 5.3 that it is biased toward global shape alignment and unaware of local details inside organs.
  • domain assumption Downsampling all training volumes to 128x96x160 retains registration-relevant information across all regions
    The model is trained at this resolution and evaluated on scan volumes with many more slices. The paper argues it works, but fine local deformations, especially in small organs, may be lost.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TotalRegistrator: Towards a Lightweight Foundation Model for CT Image Registration." pith.science (2026). https://pith.science/paper/UAIUHO5H

@misc{pith2026250804450,
  author       = {Pith},
  title        = {Pith review of: TotalRegistrator: Towards a Lightweight Foundation Model for CT Image Registration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UAIUHO5H}},
  note         = {Machine review of arXiv:2508.04450}
}
read the original abstract

Image registration is a fundamental technique in the analysis of longitudinal and multi-phase CT images within clinical practice. However, most existing methods are tailored for single-organ applications, limiting their generalizability to other anatomical regions. This work presents TotalRegistrator, an image registration framework capable of aligning multiple anatomical regions simultaneously using a standard UNet architecture and a novel field decomposition strategy. The model is lightweight, requiring only 11GB of GPU memory for training. To train and evaluate our method, we constructed a large-scale longitudinal dataset comprising 695 whole-body (thorax-abdomen-pelvic) paired CT scans from individual patients acquired at different time points. We benchmarked TotalRegistrator against a generic classical iterative algorithm and a recent foundation model for image registration. To further assess robustness and generalizability, we evaluated our model on three external datasets: the public thoracic and abdominal datasets from the Learn2Reg challenge, and a private multiphase abdominal dataset from a collaborating hospital. Experimental results on the in-house dataset show that the proposed approach generally surpasses baseline methods in multi-organ abdominal registration, with a slight drop in lung alignment performance. On out-of-distribution datasets, it achieved competitive results compared to leading single-organ models, despite not being fine-tuned for those tasks, demonstrating strong generalizability. The source code will be publicly available at: https://github.com/DIAGNijmegen/oncology_image_registration.git.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 20 canonical work pages

  1. [5]

    Informatik aktuell, pp

    Springer Vieweg, Wiesbaden. Informatik aktuell, pp. 285–290. doi:10.1007/978-3-658-25326-4_67 . Chen, J., Frey, E.C., He, Y ., Segars, W.P., Li, Y ., Du, Y .,

    work page doi:10.1007/978-3-658-25326-4_67

  2. [8]

    Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V .,

    doi:https://doi.org/10.3389/fninf.2014.00044. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V .,

    work page arXiv 2014

  3. [11]

    Medical Image Analysis 78, 102379

    Dual-stream pyramid registration network. Medical Image Analysis 78, 102379. doi: https: //doi.org/10.1016/j.media.2022.102379. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.W.,

    work page arXiv 2022

  4. [13]

    Computer Methods and Programs in Biomedicine doi:https://doi.org/10.1016/j.cmpb.2023.107453

    Quantification of liver-lung shunt fraction on 3d spect/ct images for selective internal radiation therapy of liver cancer using cnn-based segmentations and nonrigid registration. Computer Methods and Programs in Biomedicine doi:https://doi.org/10.1016/j.cmpb.2023.107453. Mok, T.C.W., Chung, A.C.S.,

    work page arXiv 2023

  5. [16]

    (Eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pp

    U-net: Convolutional networks for biomedical image segmentation, in: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (Eds.), Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pp. 234–241. Siebert, H., Großbr ¨ohmer, C., Hansen, L., Heinrich, M.P.,

    work page 2015

  6. [17]

    IEEE Transactions on Medical Imaging 44, 738–748

    Convexadam: Self-configuring dual-optimisation-based 3d multitask medical image registration. IEEE Transactions on Medical Imaging 44, 738–748. doi:10.1109/TMI.2024.3462248. 20 Given-name Surname et al. / Medical Image Analysis (2025) Sokooti, H., de V os, B., Berendsen, F., Ghafoorian, M., Yousefi, S., Lelieveldt, B.P.F., Isgum, I., Staring, M.,

    work page arXiv 2024

  7. [18]

    doi: 10.48550/ARXIV.1908.10235

    3d convolutional neural networks image registration based on efficient supervised learning from artificial deformations. doi: 10.48550/ARXIV.1908.10235. Sokooti, H., de V os, B., Berendsen, F., Lelieveldt, B., Iˇsgum, I., Staring, M.,

    arXiv doi:10.48550/arxiv.1908.10235 1908

  8. [21]

    (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, Cham

    unigradicon: A foundation model for medical image registration, in: Linguraru, M.G., Dou, Q., Feragen, A., Giannarou, S., Glocker, B., Lekadir, K., Schnabel, J.A. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, Cham. pp. 749–760. Tian, L., Greer, H., Vialard, F.X., Kwitt, R., Est´epar, R.S.J., Rushmore, R.J., Makris, N., ...

    work page 2024

Show all 26 references
  1. [22]

    18084–18094

    Gradicon: Approximate diffeomorphisms via gradient inverse consistency, in: 2023 IEEE /CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18084–18094. doi: 10.1109/ CVPR52729.2023.01734. Wang, H., Ni, D., Wang, Y .,

    2023

  2. [23]

    IEEE Transactions on Medical Imaging 43, 2229–2240

    Recursive deformable pyramid network for unsupervised medical image registration. IEEE Transactions on Medical Imaging 43, 2229–2240. doi: 10.1109/TMI.2024.3362968. Wasserthal, J., Breit, H.C., Meyer, M.T., Pradella, M., Hinck, D., Sauter, A.W., Heye, T., Boll, D., Cyriac, J.,...

    2024

  3. [25]

    10599–10609

    Recursive cascaded networks for unsupervised medical image registration, in: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10599–10609. doi:10.1109/ICCV.2019.01070. Zou, W., Luo, Y ., Cao, W.,

    2019

  4. [26]

    Neural Computing and Applications 33, 16471–16487

    A cascaded registration network rcinet with segmentation mask. Neural Computing and Applications 33, 16471–16487. doi:https://doi.org/10.1007/s00521-021-06243-9

    doi

  5. [2000]

    IEEE Transactions on Image Processing 9, 2083–2099

    Optimization of mutual information for multiresolution image registration. IEEE Transactions on Image Processing 9, 2083–2099. doi:10.1109/83.887976. Tian, L., Greer, H., Kwitt, R., Vialard, F.X., San Jos´e Est´epar, R., Bouix, S., Rushmore, R., Niethammer, M.,

    doi

  6. [2008]

    Medical image analysis 12(1), 26–41

    Symmetric di ffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis 12(1), 26–41. doi: https://doi.org/10.1016/j.media.2007.06.004. Avants, B.B., Tustison, N.J., Stauffer, M., Song, G....

    2007 doi

  7. [2010]

    IEEE Transactions on Medical Imaging 29(1), 196–205

    elastix: A toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging 29(1), 196–205. doi: 10.1109/TMI.2009.2035616. Luu, M.H., Mai, H.S., Pham, X.L., Le, Q.A., Le, Q.K., van Walsum, T., Le, N.H., Franklin, D., Le, V .H., Moelker, A., Chu, D.T...

    2009

  8. [2011]

    New England Journal of Medicine 365, 395–409

    Reduced lung-cancer mortality with low-dose computed tomographic screening. New England Journal of Medicine 365, 395–409. doi: https://doi.org/10.1056/nejmoa1102873. Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.,

    doi

  9. [2015]

    URL: https://proceedings.neurips.cc/paper_files/paper/2015/file/33ceb07bf4eeb3da587e268d663aba1a-Paper.pdf

    Spatial transformer networks, in: Advances in Neural Information Processing Systems. URL: https://proceedings.neurips.cc/paper_files/paper/2015/file/33ceb07bf4eeb3da587e268d663aba1a-Paper.pdf. Kang, M., Hu, X., Huang, W., Scott, M.R., Reyes, M.,

    2015

  10. [2016]

    IEEE Transactions on Biomedical Engineering 63, 1563–1572

    Evaluation of six registration methods for the human abdomen on clinically acquired ct. IEEE Transactions on Biomedical Engineering 63, 1563–1572. doi: 10.1109/TBME.2016.2574816. Zhao, S., Dong, Y ., Chang, E., Xu, Y .,

    2016

  11. [2017]

    (Eds.), Medical Image Computing and Computer Assisted Intervention, MICCAI 2017, pp

    Nonrigid image registration using multi-scale 3d convolutional neural networks, in: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D., Duchesne, S. (Eds.), Medical Image Computing and Computer Assisted Intervention, MICCAI 2017, pp. 232–239. Thevenaz, P., Unser, M.,

    2017

  12. [2019]

    IEEE Transactions on Medical Imaging 38(8), 1788–1800

    V oxelmorph: A learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging 38(8), 1788–1800. doi:10.1109/TMI.2019.2897538. Budelmann, D., K ¨onig, L., Papenberg, N., Lellmann, J.,

    2019

  13. [2020]

    (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, pp

    Large deformation di ffeomorphic image registration with laplacian pyramid networks, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI...

    2020

  14. [2021]

    Medical Image Analysis 72, 102139

    Cnn-based lung ct registration with multiple anatomical constraints. Medical Image Analysis 72, 102139. doi: https://doi.org/10.1016/j.media.2021.102139. Hering, A., Westphal, M., Gerken, A., Almansour, H., Maurer, M., Geisler, B., Kohlbrandt, T., Eigentler, T., Amaral, T., Le...

    2021

  15. [2022]

    Medical Image Analysis 82, 102615

    Transmorph: Transformer for unsupervised medical image registration. Medical Image Analysis 82, 102615. doi: https://doi.org/10.1016/j.media.2022.102615. Gunay, G., Luu, M.H., Moelker, A., van Walsum, T., Klein, S.,

    2022

  16. [2023]

    IEEE Transactions on Medical Imaging 42, 697–712

    Learn2reg: Comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning. IEEE Transactions on Medical Imaging 42, 697–712. doi:10.1109/TMI.2022.3213983. Hering, A., H¨ager, S., Moltz, J., Lessmann, N., Heldmann, S., van Ginn...

    2022

  17. [2024]

    International Journal of Computer Assisted Radiology and Surgery 19(9), 1689–1697

    Improving assessment of lesions in longitudinal ct scans: a bi-institutional reader study on an ai- assisted registration and volumetric segmentation workflow. International Journal of Computer Assisted Radiology and Surgery 19(9), 1689–1697. doi: https: //doi.org/10.1007/s115...

    doi

  18. [2025]

    Ronneberger, O., Fischer, P., Brox, T.,

    Divide to conquer: A field decomposition approach for multi-organ whole-body ct image registration, in: SPIE Medical Imaging: Image Processing 2025 (SPIE MI25). Ronneberger, O., Fischer, P., Brox, T.,

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.