REVIEW 4 major objections 4 minor 12 references
Detection and classification of radio sources with deep learning
T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Self-supervised learning on unlabelled radio sky images makes source classification work with 2% of the labels.
desk verdict The frozen-feature SSL result is real, but the low-label fine-tuning claim conflates self-supervision with generic transfer learning. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is SimCLR, a contrastive self-supervised framework that trains a ResNet18 encoder by maximizing agreement between differently augmented views of the same image while separating views of different images, using a two-layer projection head. The learned encoder is then fixed (or fine-tuned) as the backbone of a downstream classifier. The key design choice is the pre-training data: two unlabelled ASKAP-EMU image sets (random patches and resolved-source-centred patches) and a two-step training that combines both, which the paper finds gives the best transfer. For the other two applications, the mechanisms are Mask R-CNN instance segmentation for source finding and a LightGBM classifier fed radio-infrared colour indices for compact source classification.
What would settle it
Train the same SimCLR pre-training on unlabelled VLA FIRST images and fine-tune on the RGZ task with 2% labels; if the FIRST-pretrained model does not match or beat the EMU-pretrained one, the claimed benefit is survey-specific rather than general. Alternatively, run the EMU-pretrained model on labelled EMU sources once such labels exist and check whether F1≈0.65 at 2% labels reproduces; a large drop would indicate the domain shift is the real bottleneck.
Extended reading notes
Core claim
The paper's central claim is that self-supervised pre-training on unlabelled radio images produces representations that make downstream radio source classification far more label-efficient. Using SimCLR with a ResNet18 backbone, the authors pre-train on ~170,000 random 256×256 patches from the ASKAP-EMU survey, on ~17,000 images centred on resolved sources, and on both in a two-step sequence. When the frozen backbone is used to classify Radio Galaxy Zoo sources from VLA FIRST images into six morphological classes, all three self-supervised models beat the ImageNet-pretrained baseline, with the two-step model about 10% higher for every class. Fine-tuning experiments show the self-supervised model reaches F1≈0.65 with 2% of labels, where a from-scratch model is nearly untrainable below 10% of labels; above that threshold the supervised model wins by a few points but costs far more training time.
Load-bearing premise
The results assume that representations learned on ASKAP-EMU images transfer to VLA FIRST/Radio Galaxy Zoo classification despite differences in resolution, noise, and source density, and the paper does not analyse this domain gap.
Editorial extensions
If this is right
- A self-supervised pre-trained backbone can be fine-tuned on a few hundred labels to produce a viable radio source classifier, cutting the annotation burden for EMU-scale catalogues.
- Instance segmentation with Mask R-CNN can detect compact, extended, and multi-island radio sources along with artefacts, though spurious-source performance remains suboptimal.
- Radio-infrared colour indices, boosted by additional mid- and far-infrared bands and spectral index, separate Galactic from extragalactic compact sources with high accuracy.
- The two-step self-supervised model provides a ~10% F1 gain over ImageNet pre-training across all Radio Galaxy Zoo classes, suggesting survey-specific pre-training is worth the compute.
- Supervised training is only competitive above ~10% of labels, so in the low-label regime that dominates radio astronomy, self-supervised initialization is the recommended starting point.
Reading between the lines
- Because the reported transfer is cross-survey (ASKAP-EMU pre-training, VLA FIRST evaluation), the paper leaves open whether the same gains appear when pre-training and fine-tuning happen inside the same survey; the random-patch pre-training may be sensitive to domain shift in PSF and noise.
- A natural testable extension is to pre-train SimCLR on unlabelled VLA FIRST images and compare with the EMU-pretrained model on the RGZ task; equal or better performance would locate the gain in the SSL objective rather than in the survey choice.
- The self-supervised representations could also seed the Mask R-CNN backbone in caesar-mrcnn, potentially improving the spurious/flagged classes that currently score lowest, which the paper states it is exploring.
- Combining latent-diffusion-generated synthetic images with self-supervised pre-training is a concrete path to address the class imbalance that currently keeps spurious-source F1 around 36–44%.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes three deep-learning tools being developed for the ASKAP-EMU and MeerKAT radio surveys: (1) a Mask R-CNN instance-segmentation source finder (caesar-mrcnn) trained on labelled radio images, (2) a LightGBM and CNN pipeline for classifying compact sources into seven astronomical classes using radio plus infrared data, and (3) a SimCLR self-supervised model pretrained on unlabelled ASKAP-EMU images and evaluated on downstream Radio Galaxy Zoo morphological classification. The most concrete claim is that the self-supervised representation outperforms an ImageNet-pretrained baseline in frozen-head evaluation and, in a low-label fine-tuning experiment, reaches F1 ~0.65 with only 2% of training labels whereas a fully supervised model fails below 0.45 with fewer than 10% of labels.
Significance. If the self-supervised learning result holds, it would be a practically important contribution for radio surveys that lack large labelled datasets. The paper is honest about the sub-optimal detection of spurious and flagged sources and reports failures without overstating them. The use of real ASKAP-EMU and VLA FIRST data is a strength, as is the comparison against an ImageNet baseline in the frozen-head experiment. However, the quantitative claims are not supported by error bars or significance tests, and the low-label comparison lacks an ImageNet-pretrained fine-tuning control, so the central promise that self-supervision on radio data specifically compensates for label scarcity is not yet established. No code or data are released, which limits reproducibility.
major comments (4)
- [Section 4] The low-label fine-tuning experiment compares only a model trained from scratch against a model initialized with SimCLR backbone weights. The text states 'we compared two models: one trained from scratch, and the other with backbone weights initialized to the best SimCLR model backbone weights.' An ImageNet-pretrained fine-tuning baseline is missing, despite the availability of such a model from the frozen-head experiment. Without this control, the reported advantage at low label fractions could be due to generic transfer learning from any pretrained initialization, not to self-supervised learning on radio data. Please add an ImageNet-initialized fine-tuning baseline at the same label fractions, or explicitly justify why it is not needed.
- [Table 1 / Section 2] The text claims the new TensorFlow 2 implementation 'achieves slightly better source detection performances compared to the original version,' but Table 1 shows mixed results: tf2-resnet101 improves COMPACT, EXTENDED, and EXTENDED-MULTI, yet degrades SPURIOUS from 44.0 to 36.5 and FLAGGED from 81.9 to 78.1, while tf2-resnet18 degrades FLAGGED from 81.9 to 54.9. The class-averaged or global claim is not supported without a proper aggregate metric, error bars, or significance testing. Please report per-class uncertainties and either qualify the claim per class or use a statistically justified aggregate.
- [Section 4] The key quantitative results — the ~10% F1 improvement over the ImageNet baseline and the ~0.65 F1 at 2% of labels — are reported without error bars, confidence intervals, or repeated runs. The test set has 600 images per class (3,600 total), so run-to-run variance from weight initialization and data shuffling is likely non-negligible. Please provide error bars from multiple training runs or a statistical significance test to support the claim that self-supervised pre-training outperforms the baselines.
- [Section 4] The self-supervised models are pretrained on ASKAP-EMU images (10" resolution, ~15 µJy/beam rms, many random patches) but evaluated on VLA FIRST images of resolved Radio Galaxy Zoo sources (5.4" resolution). The manuscript does not discuss the domain shift in point spread function, noise level, or source density, nor does it justify why representations learned on mostly empty or diffuse ASKAP patches transfer to compact FIRST morphologies. While the ImageNet comparison is valid on the same test set, the practical claim that the SSL features are valuable for EMU data specifically is not demonstrated. Please add an analysis of domain differences or an evaluation on an EMU-based test set for the downstream classification task.
minor comments (4)
- [Section 4] The phrase 'the best SimCLR model backbone weights' is ambiguous; earlier the text describes three SimCLR models (random patches, resolved sources, and a two-step combination) and notes that 'the third model' achieved ~10% higher scores. Please specify which model was used in the low-label fine-tuning experiment.
- [Section 4] The low-label paragraph refers to 'scores<0.45' and 'scores ~0.65' without defining the metric; the preceding paragraph uses F1-scores. Please state explicitly whether these are macro-F1, micro-F1, or some other measure.
- [Section 3] The 'significant boost in performance (~10%)' for PNe, H ii regions, and pulsars and the statement that LightGBM 'slightly outperforms' a CNN are given without a table, figure, or specific numbers. Please include the quantitative results or a reference to a dedicated publication with the details.
- [Section 2] The use of a comma as thousands separator (e.g., '~12.000', '~38.000', '~20.000') is non-standard in English; consider using '12,000' or '12 000' for clarity.
Circularity Check
No significant circularity: the self-supervised learning result is trained on unlabeled radio images and evaluated with external Radio Galaxy Zoo labels; self-citations are background, not load-bearing.
full rationale
The paper's central empirical claim is that a SimCLR model trained on unlabeled ASKAP-EMU images outperforms an ImageNet-pretrained baseline on frozen-feature classification of Radio Galaxy Zoo sources, and that fine-tuning with very few labels yields F1 scores near 0.65. This result is not circular: the self-supervised features are learned without labels from survey images, the downstream labels come from the external Radio Galaxy Zoo catalogue, and no fitted parameter of the model is renamed as a prediction. The low-label comparison is made against a randomly initialized model rather than an ImageNet-pretrained model, which is a legitimate experimental-design gap, but it does not reduce the reported SSL advantage to the paper's own inputs. The self-citations to Riggi et al. 2023 and Sortino et al. 2023a,b describe the earlier caesar-mrcnn tool, synthetic image generation, and a framework survey; none is invoked as an external uniqueness theorem or as the justification for the Section 4 result. These citations are disclosed and are not load-bearing in the derivation chain. I find no step where a definition, fitted parameter, or self-citation forces the claimed outcome by construction, so the appropriate finding is no significant circularity, with only a minor self-citation presence.
Assumptions & free parameters
assumptions (3)
- domain assumption The reference catalogs used to label training and test sources are sufficiently accurate and representative.
- domain assumption Self-supervised representations learned on ASKAP-EMU images transfer to VLA FIRST images and Radio Galaxy Zoo labels.
- domain assumption The train/test split avoids contamination between overlapping surveys and duplicate sources.
Cite this review
Pith. "Pith review of Detection and classification of radio sources with deep learning." pith.science (2026). https://pith.science/paper/FQRNTNDE
@misc{pith2026241108519,
author = {Pith},
title = {Pith review of: Detection and classification of radio sources with deep learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/FQRNTNDE}},
note = {Machine review of arXiv:2411.08519}
}
read the original abstract
In this paper we present three different applications, based on deep learning methodologies, that we are developing to support the scientific analysis conducted within the ASKAP-EMU and MeerKAT radio surveys. One employs instance segmentation frameworks to detect compact and extended radio sources and imaging artefacts from radio continuum images. Another application uses gradient boosting decision trees and convolutional neural networks to classify compact sources into different astronomical classes using combined radio and infrared multi-band images. Finally, we discuss how self-supervised learning can be used to obtain valuable radio data representations for source detection, and classification studies.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archive author booktitle chapter collaboration edition editor eid eprint howpublished institution journal key month note number numpages organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.con...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....
- [3]
-
[4]
Chen, T., et al. 2020, in Proc. of the 37th International Conference on Machine Learning (PMLR), vol. 119, 1597
work page 2020
-
[5]
2009, in 2009 IEEE conference on computer vision and pattern recognition, 248
Deng, J., et al. 2009, in 2009 IEEE conference on computer vision and pattern recognition, 248
work page 2009
-
[6]
2017, in 2017 IEEE International Conference on Computer Vision (ICCV), 2980
He, K., et al. 2017, in 2017 IEEE International Conference on Computer Vision (ICCV), 2980
work page 2017
- [7]
-
[8]
2023, IEEE Transactions on Knowledge & Data Engineering, 35, 857
Liu, X., et al. 2023, IEEE Transactions on Knowledge & Data Engineering, 35, 857
work page 2023
Show all 12 references
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.