Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Synthetic LGE images generated from bSSFP scans, with zero labelled LGE data, yield accurate cardiac segmentation.

desk verdict Solid integrative paper showing unsupervised LGE cardiac segmentation by translating labeled bSSFP to synthetic LGE; the central claim is plausible and externally benchmarked, but the anatomy-preservation assumption needs quantitative support. read the letter →

arxiv 1908.07344 v3 pith:NQD6BOIP submitted 2019-08-20 eess.IV cs.CVcs.LG

Chen Chen , Cheng Ouyang , Giacomo Tarroni , Jo Schlemper , Huaqi Qiu , Wenjia Bai , Daniel Rueckert This is my paper · ORCID
classification eess.IVcs.CVcs.LG
keywords cardiacMRIsegmentationlategadoliniumenhancementunsuperviseddomainadaptationmulti-modalimagetranslationstyletransfercascadedU-netsynthetictrainingdataunpaired
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a cardiac MRI segmentation network can learn to segment late-gadolinium enhanced (LGE) images without ever seeing a labelled LGE image during training. The idea is to translate easily annotated balanced steady-state free precession (bSSFP) scans into realistic, diverse synthetic LGE images with an unpaired style-transfer network, then fine-tune a segmentation network pre-trained on labelled bSSFP data using those synthetic images with their original labels. On the held-out test set the resulting pipeline reports an average Dice score of 0.92 for the left ventricle, 0.83 for the myocardium, and 0.88 for the right ventricle. The significance, if the claim holds, is that annotation effort can be shifted from a modality with blurry, ambiguous borders to one where borders are clear, and the same recipe may transfer to other modalities.

What carries the argument

The load-bearing mechanism is MUNIT's disentangled representation: a content encoder extracts anatomy, a style encoder captures modality-specific appearance, and the decoder re-renders the content with a sampled LGE style. Because the style space is trained to match a standard Gaussian, one bSSFP image yields many synthetic LGE images, providing data diversity. On the segmentation side, the cascaded U-net and its composite loss — weighted cross-entropy plus a Sobel-filter edge loss — carry the argument that combining shape priors with appearance handles the blurry infarct boundaries typical of LGE.

What would settle it

Take a batch of synthetic LGE images generated from bSSFP inputs, have an expert segment the three cardiac structures on them, and compute Dice against the original bSSFP ground-truth labels; if myocardium Dice on this synthetic-to-source check falls well below the 0.83 reported on real LGE, the translation is not preserving anatomy and the fine-tuning labels are systematically corrupted.

Watch

Extended reading notes

Core claim

The central claim is that an unsupervised, unpaired image-translation step can close the appearance gap between bSSFP and LGE enough for labels to transfer. Using MUNIT, each bSSFP image is decomposed into a domain-invariant content code and a domain-specific style code; sampling style codes from a Gaussian prior produces multiple LGE-stylized versions of the same anatomy. These synthetic LGE images, paired with the original bSSFP labels, are used to fine-tune a two-stage cascaded U-net that first produces coarse probability maps and then refines them by combining image appearance with the predicted shape prior. The paper reports that this synthetic fine-tuning improves myocardium Dice by roughly 15 percentage points over training on bSSFP alone and outperforms both a registration-based transfer baseline and a single fine-tuned U-net.

Load-bearing premise

The load-bearing premise is that translating a bSSFP image into LGE style preserves the anatomy pixel for pixel; if the translator changes shape or introduces artifacts in the enhanced myocardium, the synthetic labels mislead the segmentation network.

Editorial extensions

If this is right

  • If the reported Dice scores hold, LGE cardiac segmentation can be automated in settings where no LGE annotations exist, as long as bSSFP labels are available.
  • The synthetic LGE images act as a data-augmentation engine: sampling different style codes from the same anatomy increases training diversity and can improve generalization.
  • The cascaded shape-plus-appearance design adds robustness on apical and basal slices, where single-pass networks and registration methods tend to fail.
  • The same two-stage recipe — unpaired translation followed by fine-tuning a pretrained segmenter — is proposed as a general route for other cross-modality tasks with one labelled modality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit is measuring anatomy-preservation fidelity directly: compare expert outlines of synthetic LGE images against the original bSSFP labels to isolate how much of the reported Dice comes from translation quality rather than the cascaded segmenter.
  • The method's ceiling is probably set by the content code's invariance; if the scarred, enhanced myocardium carries appearance information that the encoder routes into style, the transferred labels will systematically misrepresent infarct regions, so per-region Dice on scar versus remote myocardium would be a sharper evaluation than whole-structure Dice.
  • The same pipeline could be pointed at other unpaired modality pairs, such as CT to MRI or different MRI sequences, whenever one modality is easy to annotate and the other is not; the paper itself names brain MRI as future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a fully automatic, two-stage framework for segmenting cardiac structures in late-gadolinium enhanced (LGE) MRI without using labelled LGE images during training. First, an MUNIT-based multi-modal translation network, trained on unpaired bSSFP and LGE images, converts annotated bSSFP images into multiple synthetic LGE-style images while keeping the underlying anatomy fixed. Second, a cascaded two-U-net segmentation network is pre-trained on labelled bSSFP images and then fine-tuned on the synthetic LGE images plus the original bSSFP labels. At inference, dense CRF and morphological post-processing are applied. The authors evaluate on the MS-CMRSeg 2019 challenge data: on a 5-volume validation set the cascaded fine-tuned network improves Dice over a bSSFP-only U-net and a registration baseline, and on the 40-subject test set they report average Dice of 0.92 for the left ventricle, 0.83 for the myocardium, and 0.88 for the right ventricle.

Significance. The result, if supported, is practically significant: it removes the need for costly LGE annotations by transferring anatomical knowledge from bSSFP, which is easier to label, and it is evaluated on an external challenge test set rather than an in-house dataset. The paper also ships a clear experimental pipeline, uses the official MUNIT implementation, and includes a quantitative comparison to both a registration-based method and a single U-net on the validation set. However, the main claim rests on the untested assumption that the synthetic LGE images preserve the bSSFP anatomy exactly, and the statistical evidence for the central performance gain is thin (five validation images, no error bars, no significance tests, and no test-set comparators).

major comments (4)
  1. [§2.1, §3.1, Fig. 5] The central claim that fine-tuning on synthetic LGE images yields accurate real-LGE segmentation depends on the assertion in §2.1 that MUNIT preserves underlying anatomical structure when swapping style codes. This is currently supported only by a citation to [8] and by qualitative examples in Fig. 5. Because the synthetic training labels are taken unchanged from the source bSSFP labels, any leakage of bSSFP-specific appearance into the content code or any decoder-introduced artifact will produce systematically wrong labels. Please add a quantitative anatomy-preservation check, for example by segmenting the synthetic LGE images with a network trained on the same bSSFP labels and measuring Dice with the original masks, or by measuring contour/landmark agreement between input bSSFP and synthetic LGE across all 150 generated images.
  2. [§3.3, Table 1] The validation experiments reported in Table 1 are computed on only five LGE volumes, and the paper gives no per-subject numbers, error bars, or significance tests. The statement in §3.3 that fine-tuning with synthetic LGE 'significantly improves' the MYO Dice by about 15% is therefore not statistically supported; with n=5, a single favorable volume could drive the apparent improvement. Please report the per-volume Dice/ASD distributions for every method and a paired statistical test (e.g., Wilcoxon signed-rank or paired t-test) on the validation set.
  3. [§3.3, test-set results] The final test-set results on 40 subjects are reported as absolute Dice and ASD values (LV 0.92, MYO 0.83, RV 0.88) but are not compared with any alternative method on the same test set. Without a test-set comparison, the reader cannot judge whether the proposed method is actually competitive for LGE segmentation, and no per-subject distribution is given, so the variability of the test performance is unknown. Please provide test-set results for at least one baseline (e.g., U-net (FT) or MIRTK), or report the challenge leaderboard/ranking, together with per-subject statistics.
  4. [§1, §3.3] The introduction identifies the heterogeneous, enhanced infarcted myocardium as the main challenge in LGE segmentation, but the bSSFP source domain contains no late-gadolinium enhancement information. Consequently, the synthetic LGE images cannot reproduce the bright, heterogeneous scarred myocardium that the method is ultimately asked to segment, and the lower MYO Dice (0.83 test, 0.816 validation) compared with LV and RV is consistent with the network relying on bSSFP-like boundary cues rather than true LGE appearance. This is not fatal for the proposed pipeline, but the claim of accurate LGE segmentation should be qualified as applying primarily to myocardium with bSSFP-like contrast; the paper should discuss this limitation explicitly and, if feasible, stratify the evaluation by the presence or extent of myocardial scar.
minor comments (4)
  1. [§2.2] The sentence 'the posterior modelled by the network p(yb|xbl) matches p(yl|xl) when image space and label space are shared' is imprecise: the image spaces are not literally shared after translation, and the equality is the goal of adaptation rather than a condition. Please rephrase.
  2. [§3.2] Several hyperparameters (edge loss weight λ=0.5, class weights 0.2:0.25:0.3:0.25, five synthetic samples per bSSFP image) are introduced without sensitivity analysis; a brief justification or a reference for each choice would make the pipeline more reproducible.
  3. [Supplemental Material] Figure 5 is only mentioned as 'supplemental material' in §3.1 and is not referenced by number in the main text; please cite it explicitly.
  4. [Table 1] The N/A entries for the U-net ASD values should be clarified: if the network produced no prediction in some volumes, the corresponding Dice scores should be treated consistently (e.g., as zero) or those volumes should be excluded with the criterion stated.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: synthetic-data pipeline and external challenge evaluation are independent of the target LGE labels.

full rationale

The derivation chain is not circular. The synthetic LGE training set is generated from unpaired bSSFP and LGE images via MUNIT, with labels inherited from bSSFP; these synthetic images do not contain or depend on the test LGE labels. The segmentation network is pre-trained on labelled bSSFP and fine-tuned on synthetic LGE, and evaluated on a held-out challenge test set (40 subjects), with reported Dice 0.92/0.83/0.88. No fitted parameter is renamed as a prediction; hyperparameters are set on a 5-image validation subset and the final numbers are from the external test set. The main load-bearing premise—that MUNIT's content code preserves anatomy—is assumed via prior work (Huang et al. [7], Qin et al. [8]) rather than proven in-paper, but this is an empirical modelling assumption, not a definitional equivalence: anatomy preservation is not defined in terms of the final segmentation, and the synthetic labels are not constructed from the target quantities being predicted. The overlap of Rueckert in [8] is a self-citation, but it supports a general property of disentangled representation learning and is not the mechanism by which the reported test Dice is forced. No equation reduces to itself and no prediction is equivalent to an input by construction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central result depends on the assumption that unpaired style transfer preserves anatomy, on hyperparameters chosen by hand, and on a small validation set for model selection; no new physical entities are postulated.

free parameters (4)
  • edge loss weight lambda = 0.5
    Set empirically to balance weighted cross-entropy and edge loss in Section 2.2.
  • class weights in weighted cross-entropy = 0.2:0.25:0.3:0.25 (BG:LV:MYO:RV)
    Empirically chosen to address class imbalance, with myocardium weighted highest, as stated in Section 3.2.
  • number of synthetic LGE samples per bSSFP image = 5
    Randomly sampled style codes from N(0,I) per bSSFP image during fine-tuning; this controls synthetic training set size and diversity (Section 3.1).
  • ensemble size = 4 models
    Four independently trained segmentation models were averaged at test time; the reported test scores depend on this choice (Section 3.3).
assumptions (3)
  • domain assumption MUNIT disentanglement: the content code is domain-invariant and preserves anatomy, while the style code carries modality-specific appearance.
    Central to synthetic LGE generation; if false, labels from bSSFP do not transfer to synthetic LGE. Stated in Section 2.1.
  • domain assumption Shared image and label spaces: p(yb|xbl) approximates p(yl|xl) after translation.
    Explicit in Section 2.2: 'Ideally, the posterior modelled by the network p(yb|xbl) matches p(yl|xl) when image space and label space are shared.'
  • domain assumption The five annotated LGE validation images are representative of the test distribution.
    Hyperparameters and model selection rely on this small validation subset, described in Section 3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation." pith.science (2026). https://pith.science/paper/NQD6BOIP

@misc{pith2026190807344,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Multi-modal Style Transfer for Cardiac MR Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NQD6BOIP}},
  note         = {Machine review of arXiv:1908.07344}
}
read the original abstract

In this work, we present a fully automatic method to segment cardiac structures from late-gadolinium enhanced (LGE) images without using labelled LGE data for training, but instead by transferring the anatomical knowledge and features learned on annotated balanced steady-state free precession (bSSFP) images, which are easier to acquire. Our framework mainly consists of two neural networks: a multi-modal image translation network for style transfer and a cascaded segmentation network for image segmentation. The multi-modal image translation network generates realistic and diverse synthetic LGE images conditioned on a single annotated bSSFP image, forming a synthetic LGE training set. This set is then utilized to fine-tune the segmentation network pre-trained on labelled bSSFP images, achieving the goal of unsupervised LGE image segmentation. In particular, the proposed cascaded segmentation network is able to produce accurate segmentation by taking both shape prior and image appearance into account, achieving an average Dice score of 0.92 for the left ventricle, 0.83 for the myocardium, and 0.88 for the right ventricle on the test set.

Figures

Figures reproduced from arXiv: 1908.07344 by the authors.

Figure 1
Figure 1. The differences of image appearance (a) and intensity distributions (b) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the multi-modal image translation network. The network employs the structure of MUNIT [7], which consists of two encoder￾decoder pairs for the two domains: bSSFP and LGE, respectively. carries the information for rendering the structure which is determined by the imaging modality. The image-to-image translation from one domain to the other is achieved by swapping latent codes in two domains. For example,… view at source ↗
Figure 3
Figure 3. Overview of the two-stage cascaded segmentation network. The architecture of each U-net is the same as the one of the vanilla U-net, except for two main differences: (1) batch normalization is applied after each convolutional layer; (2) a dropout layer (dropout rate=0.1) is applied after each concatenation operation in the network’s expanding path to encourage model generalizability. Of note, in this diagram, we sim… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Segmentation results for the proposed Cascaded U-net and the baseline approaches. Our proposed method (the right-most column) produces more anatomically plausible segmentation results on the images, greatly outper￾forming the baseline methods, especially in the challen…
Figure 5
Figure 5. Figure 5: Exemplar synthetic LGE images generated from bSSFP images using the multi-modal image translation network. Given one bSSFP image (column 1), the translation network translates the image into multi￾modal LGE-like images (column 2 to 4). These translated images differ in…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation

    eess.IV 2019-08 conditional novelty 3.0 of 10

    A structured review of deep learning segmentation techniques for scarce and weak annotations, with cost-gain recommendations.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [8]

    Unsupervised deformable registration for multi-modal images via disentan- gled representations

    Chen Qin, Bibo Shi, Rui Liao, Tommaso Mansi, Daniel Rueckert, and Ali Ka- men. Unsupervised deformable registration for multi-modal images via disentan- gled representations. In Information Processing in Medical Imaging, pages 249–261. Springer, Cham, June 2019

    work page 2019

  2. [1]

    Multivariate mixture model for cardiac segmentation from multi- sequence MRI

    Xiahai Zhuang. Multivariate mixture model for cardiac segmentation from multi- sequence MRI. In MICCAI, pages 581–588, 2016

    work page 2016

  3. [2]

    Automatic myocardium segmentation of LGE MRI by deformable models with prior shape data

    YingLi Lu et al. Automatic myocardium segmentation of LGE MRI by deformable models with prior shape data. JCMR, 15(1):P14, 2013

    work page 2013

  4. [3]

    Automated left ventricle segmentation in late gadolinium-enhanced MRI for objective myocardial scar assessment

    Qian Tao et al. Automated left ventricle segmentation in late gadolinium-enhanced MRI for objective myocardial scar assessment. JMRI, 42(2):390–399, 2015

    work page 2015

  5. [4]

    Multivariate mixture model for myocardium segmentation com- bining multi-source images

    Xiahai Zhuang. Multivariate mixture model for myocardium segmentation com- bining multi-source images. PAMI, 2018

    work page 2018

  6. [5]

    Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors

    Qian Yue et al. Cardiac Segmentation from LGE MRI Using Deep Neural Network Incorporating Shape and Spatial Priors. MICCAI, 2019

    work page 2019

  7. [6]

    U-net: Convolutional networks for biomedical image segmen- tation

    Ronneberger et al. U-net: Convolutional networks for biomedical image segmen- tation. In MICCAI, 2015

    work page 2015

  8. [7]

    Multimodal unsupervised image-to-image translation

    Xun Huang et al. Multimodal unsupervised image-to-image translation. In ECCV, 2018

    work page 2018

Show all 14 references
  1. [9]

    Cur- riculum learning

    Yoshua Bengio, J´ erˆ ome Louradour, Ronan Collobert, and Jason Weston. Cur- riculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, pages 41–48, New York, NY, USA, 2009. ACM

    2009

  2. [10]

    Auto-context and its application to high-level vision tasks and 3D brain image segmentation

    Zhuowen Tu and Xiang Bai. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. PAMI, 2010

    2010

  3. [11]

    A 3x3 isotropic gradient operator for image processing

    Irwin Sobel and G Feldman. A 3x3 isotropic gradient operator for image processing. Pattern Classification and Scene Analysis , pages 271–272, 01 1973

    1973

  4. [12]

    Efficient inference in fully connected CRFs with gaussian edge potentials

    Philipp Kr¨ ahenb¨ uhl et al. Efficient inference in fully connected CRFs with gaussian edge potentials. In NeuralIPS, 2011

    2011

  5. [13]

    Arbitrary style transfer in real-time with adaptive instance normalization

    Xun Huang et al. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, 2017

    2017

  6. [14]

    Multi-task learning for left atrial segmentation on GE-MRI

    Chen Chen, Wenjia Bai, and Daniel Rueckert. Multi-task learning for left atrial segmentation on GE-MRI. In Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges , 2019. Unsupervised Multi-modal Style Transfer for Cardiac...

    2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.