REVIEW 4 major objections 6 minor 65 references
EMUSE: Evolutionary Map of the Universe Search Engine
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Fine-tuning a general-purpose vision-language model lets astronomers search the radio sky by image or text.
desk verdict A useful and honest tool paper: first CLIP-based radio morphology retrieval over EMU, but retrieval effectiveness is supported by four examples and the compressed similarity scores need a benchmark before I'd trust the search. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the shared embedding space produced by contrastive fine-tuning of OpenCLIP. OpenCLIP, a CLIP-style model pre-trained on 2.32 billion LAION image-text pairs, is fine-tuned with adapters on the contrastive branch only, using 2,900 radio galaxy image-text pairs from RadioGalaxyNET with expanded labels such as 'An image of an FR-II or Fanaroff-Riley type II radio galaxy with edge-brightened lobes bent at an angle'. The fine-tuned image encoder converts each EMU source cutout into a compact vector, and retrieval is the $\arg\max$ of cosine similarity between a query embedding and the stored image embeddings, as given in Eqs. (1) and (2). The text encoder does the same for natural-language queries, which is what makes zero-shot retrieval possible without retraining on specific classes. A UMAP projection of the learned embeddings shows distinct clusters for the main categories, indicating that the embedding geometry carries the classification signal.
What would settle it
Run a labelled retrieval benchmark: take held-out FR-I, FR-II, FR-x, R-type and Peculiar sources from RadioGalaxyNET, hide their labels, query EMUSE with each source image, and measure what fraction of the top-10 neighbours carry the same morphological label; if that fraction is no better than random or no better than the untuned OpenCLIP baseline, the cosine-similarity assumption fails. A sharper version is to query with a verified ORC and check whether the top-50 returns a statistically significant excess of confirmed ORCs and ORC candidates over chance, since the paper's own example already shows several non-ORC sources among the retrieved results.
Extended reading notes
Core claim
The paper's central claim is that a foundation model pre-trained on ordinary internet image-text pairs can be cheaply adapted to radio astronomy and then serve as the core of a similarity search engine over survey-scale data. Fine-tuning only the contrastive branch of OpenCLIP on 2,900 RadioGalaxyNET sources, with radio and AllWISE infrared channels stacked into three-channel cutouts and paired with paraphrased morphological descriptions, yields a model that classifies held-out test sources into FR-I, FR-II, FR-x, R-type and Peculiar at 84±3% accuracy averaged over ten random splits. Deployed over first-year EMU data, EMUSE retrieves the top-$k$ sources by cosine similarity between the query embedding and precomputed image embeddings. The authors report that a text query for bent-tailed galaxies returns 50 visually consistent sources, and an image query using the known ORC J2103-6200 retrieves a known ORC candidate, half-ring structures, and potential GLAREs, even though only two ORCs were in the fine-tuning set. They also state that the same retrieval is imperfect: text queries are sensitive to phrasing, and rare or unseen classes such as supernova remnants and cluster relics are poorly handled.
Load-bearing premise
The whole retrieval story rests on the assumption that cosine similarity in the fine-tuned embedding space tracks astrophysical morphological similarity across the full survey; the paper validates this with only four qualitative query examples, with no ranking metrics and no comparison against the unfine-tuned model.
Editorial extensions
If this is right
- Astronomers can query EMU survey data for a morphological class using plain language, such as 'a bent-tailed radio galaxy', and obtain ranked candidate lists with sky positions, integrated fluxes, and likely host galaxies.
- Image-based queries let users find more examples of a known rare object: querying with ORC J2103-6200 surfaces an ORC candidate, half-ring structures, and potential GLAREs, suggesting a route to enlarge the ORC catalogue.
- Replacing roughly 150 GB of survey cutouts with a few hundred megabytes of embeddings makes interactive similarity search feasible across the first-year EMU footprint of about 3 million detected sources.
- The same fine-tuning recipe transfers to future EMU data releases and other multiwavelength surveys, since only new image embeddings and catalogue metadata need to be computed.
- Classification of the five standard radio-galaxy families reaches 84±3% accuracy from 100 epochs of adapter training on one H100 GPU, so foundation-model adaptation is a practical path for other Big Data astronomy projects.
Reading between the lines
- If cosine similarity in the fine-tuned space is taken as the definition of morphological similarity, EMUSE's ranking should be validated with labelled retrieval metrics such as precision@k and mean reciprocal rank and compared against the untuned OpenCLIP baseline; the paper currently shows only four qualitative query examples.
- The same approach should be testable on other survey-scale imaging problems: fine-tune a CLIP-style model on labelled cutouts, embed the full survey, and search, with labelled exemplars rather than compute or storage as the bottleneck.
- Retrieval of ORC-like objects despite only two training examples hints that the model generalizes structural features such as closed rings and edge-brightened geometry rather than memorizing instances; a controlled study with deliberately sparse rare classes could test this directly.
- Because the catalogue inherits Gal-DINO and Selavy detection limits, very faint sources are invisible to EMUSE; a catalogue-agnostic version that embeds all pixels or tiles would be a natural extension of the same pipeline.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents EMUSE, a search engine for the Evolutionary Map of the Universe (EMU) radio survey. The authors fine-tune the OpenCLIP vision-language model on 2,900 radio galaxy images from the RadioGalaxyNET dataset, using adapter-based fine-tuning and expanded text descriptions, and report 84±3% classification accuracy on held-out test sets for the FR-I, FR-II, FR-x, R, and Peculiar categories. The fine-tuned model is used to embed approximately 170,000 extended radio sources from the EMU first-year data, together with infrared (AllWISE) cutouts and RG-CAT catalogue metadata. EMUSE supports zero-shot text and image queries by ranking database entries by cosine similarity to the query embedding. The paper demonstrates the tool with two text and two image queries, including an image query of an Odd Radio Circle that retrieves a known ORC candidate, and makes the search engine and code publicly available.
Significance. If the retrieval performance is validated quantitatively, EMUSE would be a valuable community tool for mining large radio surveys, with a sensible design: adapting a general foundation model to the radio domain via lightweight fine-tuning, and compressing survey images into a searchable embedding database. The authors provide open-source code, a public demo, and transparent reporting of the classification accuracy across multiple random splits. The paper also honestly reports limitations (rare-source retrieval, text-query sensitivity). What is missing is a rigorous evaluation of the retrieval claim, which is the central function of the tool.
major comments (4)
- [Section 4, Eq. (1)-(2), Tables A1-A4] The central retrieval claim is supported only by four qualitative query examples, with no precision@k, recall, mean reciprocal rank, or baseline comparison (e.g., unfine-tuned OpenCLIP). The similarity scores in Tables A1-A4 are almost all in the range 0.89–1.00, which suggests the embedding space is compressed; without ranking metrics, it is unclear whether the retrieved order reflects morphological similarity or other image properties such as brightness, orientation, or background. Please add a quantitative retrieval evaluation with labelled queries and report standard ranking metrics and a baseline.
- [Section 3.2, Fig. 2] The 84±3% accuracy is measured on random splits of the 2,900 RadioGalaxyNET sources from EMU-PS1, and the final model is fine-tuned on the full dataset. However, the retrieval database comprises ~170,000 extended sources from the EMU main survey that are not independently labelled. The reported accuracy therefore does not demonstrate that the embedding space is morphologically structured at the scale of the actual retrieval database. Please evaluate on a labelled subset of the retrieval database, or otherwise provide evidence that the classifier/embedding generalises across the survey.
- [Section 4.2, ORC image query] The validation of this key example relies on a known ORC candidate that is cross-identified using the authors' own catalogue (Gupta et al. 2025), and the training labels and descriptions derive from the same group's prior work. This limits the independence of the demonstration. The claim that EMUSE retrieves 'potential GLAREs' and 'half-ring-like structures' is based on the authors' visual classification rather than external validation. Please either provide independent confirmation (e.g., existing external catalogues, follow-up observations, or expert labels from outside the group) or temper the claims accordingly.
- [Abstract and Conclusions] The paper states in Section 4.1 that text queries currently underperform and are sensitive to phrasing, and that rare classes such as 'odd radio circle' return no results above a 0.9 threshold without a more descriptive prompt. Yet the abstract concludes the model is 'effective in retrieving and classifying radio sources' without these caveats. Please make the abstract consistent with the reported limitations, particularly regarding rare-source and text-query performance.
minor comments (6)
- [Introduction] The phrase 'zero-shota learning' contains a stray footnote marker within the word; please fix it.
- [Section 3.2] The description 'Data clipping is applied between the 50th percentile level and the maximum values of the 99th and 99.9th percentiles for the first and second channels, respectively' is ambiguous; please clarify the clipping procedure for each channel.
- [Tables A1-A4] Several entries in the CatWISE Potential Host column contain 'nan'; please explain in the text or table caption that not all sources have a plausible infrared host.
- [Data Availability] The Data Availability section says that fine-tuned models, embeddings, and metadata are accessible 'within main.py', which is not a direct access mechanism; please provide explicit links or download instructions.
- [Conclusions] The Conclusion contains two consecutive sentences beginning 'Future work should focus on...'; consider merging them for readability.
- [References] The reference list includes 'Norris et al. submitted' and 'Gupta et al. in preparation'; please update them or indicate their availability.
Circularity Check
No significant circularity: the fine-tuning evaluation is a standard held-out supervised benchmark, and the EMUSE retrieval is an embedding similarity search rather than a fitted prediction.
full rationale
The classification claim (84±3%, Fig. 2) is obtained by fine-tuning OpenCLIP on RadioGalaxyNET with repeated 80/20 random splits and testing on held-out images; this is a conventional supervised evaluation, not a derivation from the training labels by construction. The retrieval pipeline (Eqs. 1–2) computes cosine similarity between a query embedding and precomputed embeddings of approximately 170,000 EMU sources; no parameter is fitted to the retrieval database, and the four query examples are demonstrations rather than fitted predictions. The paper relies on same-author resources (RadioGalaxyNET, RG-CAT, 'Norris et al. submitted', and Gupta et al. 2025 for the ORC candidate), but this reliance is not load-bearing: the ORC candidate identification is used as cross-reference for one validation example, and removing it would not change the structure of the derivation. The paper explicitly discloses its limitations—Section 4.1 notes that text queries are phrasing-sensitive and underperform image queries; Section 4.2 notes that different ORC screenshots yield different result sets; Section 5 notes that sources absent from the fine-tuning set, such as cluster relics and supernova remnants, may not be retrieved accurately—which is consistent with an honest, non-circular study. The absence of precision@k or an unfine-tuned baseline weakens external validation of the retrieval claim, but that is a benchmarking gap, not a circularity. No equation in the paper reduces a predicted quantity to a fitted input, and no central claim is forced by a self-citation chain.
Assumptions & free parameters
free parameters (2)
- Number of fine-tuning epochs =
100
- Image percentile clipping levels =
50th, 99th, 99.9th percentiles
assumptions (4)
- domain assumption RadioGalaxyNET class labels (FR-I, FR-II, FR-x, R, Peculiar) are correct and representative of the radio source population.
- domain assumption Cosine similarity in the fine-tuned embedding space corresponds to astrophysical morphological similarity.
- domain assumption The Gal-DINO/RG-CAT catalogue accurately identifies extended radio sources and their infrared hosts in EMU first-year data.
- domain assumption The text descriptions used in fine-tuning are sufficient to align natural-language queries with visual features.
Cite this review
Pith. "Pith review of EMUSE: Evolutionary Map of the Universe Search Engine." pith.science (2026). https://pith.science/paper/NPAW62WN
@misc{pith2026250615090,
author = {Pith},
title = {Pith review of: EMUSE: Evolutionary Map of the Universe Search Engine},
year = {2026},
howpublished = {\url{https://pith.science/paper/NPAW62WN}},
note = {Machine review of arXiv:2506.15090}
}
read the original abstract
We present EMUSE (Evolutionary Map of the Universe Search Engine), a tool designed for searching specific radio sources within the extensive datasets of the EMU (Evolutionary Map of the Universe) survey, with potential applications to other Big Data challenges in astronomy. Built on a multimodal approach to radio source classification and retrieval, EMUSE fine-tunes the OpenCLIP model on curated radio galaxy datasets. Leveraging the power of foundation models, our work integrates visual and textual embeddings to enable efficient and flexible searches within large radio astronomical datasets. We fine-tune OpenCLIP using a dataset of 2,900 radio galaxies, encompassing various morphological classes, including FR-I, FR-II, FR-x, R-type, and other rare and peculiar sources. The model is optimized using adapter-based fine-tuning, ensuring computational efficiency while capturing the unique characteristics of radio sources. The fine-tuned model is then deployed in EMUSE, allowing for seamless image- and text-based queries over the EMU survey dataset. Our results demonstrate the model's effectiveness in retrieving and classifying radio sources, particularly in recognizing distinct morphological features. However, challenges remain in identifying rare or previously unseen radio sources, highlighting the need for expanded datasets and continuous refinement. This study showcases the potential of multimodal machine learning in radio astronomy, paving the way for more scalable and accurate search tools in the field. The search engine is accessible at https://askap-emuse.streamlit.app/ and can be used locally by cloning the repository at https://github.com/Nikhel1/EMUSE.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter doi edition editor eprint howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all :=...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION format.doi doi empty "" "doi:" doi * "" * if FUNCTION format.eprint eprint empty "" archivePrefix empty "" archivePrefix ":" * if eprint field.or.null * if FUNCTION format.pid eprint empty format.doi format.eprint if FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "-...
-
[3]
2022, Advances in neural information processing systems, 35, 23716
Alayrac, J.-B., Donahue, J., Luc, P., et al. 2022, Advances in neural information processing systems, 35, 23716
work page 2022
- [4]
-
[5]
P., Tollerud , E
Astropy Collaboration , Robitaille , T. P., Tollerud , E. J., et al. 2013, , 558, A33
2013
-
[6]
M., Sip o cz , B
Astropy Collaboration , Price-Whelan , A. M., Sip o cz , B. M., et al. 2018, , 156, 123
2018
-
[7]
M., Lim , P
Astropy Collaboration , Price-Whelan , A. M., Lim , P. L., et al. 2022, , 935, 167
2022
-
[8]
2022, in European Conference on Computer Vision, Springer, 348--367
Bachmann, R., Mizrahi, D., Atanov, A., & Zamir, A. 2022, in European Conference on Computer Vision, Springer, 348--367
work page 2022
Show all 65 references
-
[9]
A., Adeli, E., et al
Bommasani, R., Hudson, D. A., Adeli, E., et al. 2021, arXiv preprint arXiv:2108.07258
2021 arXiv
-
[10]
2020, Advances in neural information processing systems, 33, 1877
Brown, T., Mann, B., Ryder, N., et al. 2020, Advances in neural information processing systems, 33, 1877
2020
-
[11]
2024, in European Conference on Computer Vision, Springer, 370--387
Chen, L., Li, J., Dong, X., et al. 2024, in European Conference on Computer Vision, Springer, 370--387
2024
-
[12]
2023, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2818--2829
Cherti, M., Beaumont, R., Wightman, R., et al. 2023, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2818--2829
2023
-
[13]
M., Wright , E
Cutri , R. M., Wright , E. L., Conrow , T., et al. 2021, VizieR Online Data Catalog, II/328
2021
-
[14]
R., Gough , R
DeBoer , D. R., Gough , R. G., Bunton , J. D., et al. 2009, IEEE Proceedings, 97, 1507
2009
-
[15]
2023, Advances in Neural Information Processing Systems, 36, 35544
Fan, L., Krishnan, D., Isola, P., Katabi, D., & Tian, Y. 2023, Advances in Neural Information Processing Systems, 36, 35544
2023
-
[16]
L., & Riley , J
Fanaroff , B. L., & Riley , J. M. 1974, MNRAS, 167, 31P
1974
-
[17]
C., Sek, Y
Fayou, S., Ngo, H. C., Sek, Y. W., & Meng, Z. 2024, Scientific Reports, 14, 11879
2024
-
[18]
P., Huynh , M., & Petersson , L
Gupta , N., Hayder , Z., Norris , R. P., Huynh , M., & Petersson , L. 2024 a , , 41, e001
2024
-
[19]
P., Hyunh , M., & Petersson , L
Gupta , N., Hayder , Z., Norris , R. P., Hyunh , M., & Petersson , L. 2023, NeurIPS ML4PS 2023, arXiv:2312.06728
2023 arXiv
-
[20]
P., et al
Gupta , N., Huynh , M., Norris , R. P., et al. 2022, , 39, e051
2022
-
[21]
P., et al
Gupta, N., Hayder, Z., Norris, R. P., et al. 2023, Publications of the Astronomical Society of Australia, 40, e044
2023
-
[22]
P., Hayder , Z., et al
Gupta , N., Norris , R. P., Hayder , Z., et al. 2024 b , , 41, e027
2024
- [23]
-
[24]
in preparation
Gupta et al. in preparation
-
[25]
M., Kapinska, A., Marvil, J., et al
Hopkins, A. M., Kapinska, A., Marvil, J., et al. 2025, Publications of the Astronomical Society of Australia, 1–32
2025
-
[26]
W., Bunton , J
Hotan , A. W., Bunton , J. D., Chippendale , A. P., et al. 2021, PASA, 38, e009
2021
-
[27]
2021, in International conference on machine learning, PMLR, 4904--4916
Jia, C., Yang, Y., Xia, Y., et al. 2021, in International conference on machine learning, PMLR, 4904--4916
2021
-
[28]
2007, , 24, 174
Johnston , S., Bailes , M., Bartel , N., et al. 2007, , 24, 174
2007
-
[29]
2024, arXiv e-prints, arXiv:2402.15120
Kim , H., Yoon , S., Bui , T., et al. 2024, arXiv e-prints, arXiv:2402.15120
2024 arXiv
-
[30]
2025, arXiv e-prints, arXiv:2501.09883
Lao , B., Andernach , H., Yang , X., et al. 2025, arXiv e-prints, arXiv:2501.09883
2025 arXiv
-
[31]
2024, , 690, A310
Lastufka , E., Bait , O., Taran , O., et al. 2024, , 690, A310
2024
-
[32]
2025, , 169, 121
Lochner , M., & Rudnick , L. 2025, , 169, 121
2025
-
[33]
Lochner , M., Rudnick , L., Heywood , I., Knowles , K., & Shabala , S. S. 2023, , 520, 1439
2023
-
[34]
A., Albarri, S., Xian, Z., et al
Manzoor, M. A., Albarri, S., Xian, Z., et al. 2023, ACM Transactions on Multimedia Computing, Communications and Applications, 20, 1
2023
-
[35]
Marocco , F., Eisenhardt , P. R. M., Fowler , J. W., et al. 2021, , 253, 8
2021
-
[36]
2018, arXiv e-prints, arXiv:1802.03426
McInnes , L., Healy , J., & Melville , J. 2018, arXiv e-prints, arXiv:1802.03426
2018 arXiv
-
[37]
2010, Proceedings of the 9th Python in Science Conference, 445, 51
McKinney, W. 2010, Proceedings of the 9th Python in Science Conference, 445, 51
2010
-
[38]
2024, , 530, 1274
Mohale , K., & Lochner , M. 2024, , 530, 1274
2024
-
[39]
Mostert , R. I. J., Duncan , K. J., R \"o ttgering , H. J. A., et al. 2021, , 645, A89
2021
-
[40]
Mostert , R. I. J., Oei , M. S. S. L., Barkus , B., et al. 2024, , 691, A185
2024
-
[41]
Y., Ilharco, G., Oh, S., & Schmidt, L
Nguyen, T., Gadre, S. Y., Ilharco, G., Oh, S., & Schmidt, L. 2023, Advances in Neural Information Processing Systems, 36, 22047
2023
-
[42]
P., Koribalski , B
Norris , R. P., Koribalski , B. S., Hale , C. L., et al. 2025, , 537, L42
2025
-
[43]
P., Marvil , J., Collier , J
Norris , R. P., Marvil , J., Collier , J. D., et al. 2021 a , , 38, e046
2021
-
[44]
P., Intema , H
Norris , R. P., Intema , H. T., Kapi \'n ska , A. D., et al. 2021 b , , 38, e003
2021
-
[45]
submitted
Norris et al. submitted
-
[46]
2024, , 531, 4990
Parker , L., Lanusse , F., Golkar , S., et al. 2024, , 531, 4990
2024
-
[47]
2017, in NIPS-W
Paszke, A., Gross, S., Chintala, S., et al. 2017, in NIPS-W
2017
-
[48]
2011, Journal of Machine Learning Research, 12, 2825
Pedregosa, F., Varoquaux, G., Gramfort, A., et al. 2011, Journal of Machine Learning Research, 12, 2825
2011
-
[49]
W., Hallacy, C., et al
Radford, A., Kim, J. W., Hallacy, C., et al. 2021, in International conference on machine learning, PMLR, 8748--8763
2021
-
[50]
2022, arXiv preprint arXiv:2204.06125, 1, 3
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. 2022, arXiv preprint arXiv:2204.06125, 1, 3
2022 arXiv
-
[51]
2025, arXiv e-prints, arXiv:2503.23859
Riggi , S., Cecconello , T., Pilzer , A., et al. 2025, arXiv e-prints, arXiv:2503.23859
2025 arXiv
-
[52]
2024, , 41, e085
Riggi , S., Cecconello , T., Palazzo , S., et al. 2024, , 41, e085
2024
-
[53]
2022, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684--10695
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. 2022, in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684--10695
2022
-
[54]
2022, Advances in neural information processing systems, 35, 25278
Schuhmann, C., Beaumont, R., Vencu, R., et al. 2022, Advances in neural information processing systems, 35, 25278
2022
-
[55]
2023, , 521, 1429
Segal , G., Parkinson , D., Norris , R., et al. 2023, , 521, 1429
2023
-
[56]
V., Scaife , A
Slijepcevic , I. V., Scaife , A. M. M., Walmsley , M., et al. 2024, RAS Techniques and Instruments, 3, 19
2024
-
[57]
2024, Research Notes of the American Astronomical Society, 8, 265
Tanoglidis , D., & Jain , B. 2024, Research Notes of the American Astronomical Society, 8, 265
2024
-
[58]
2023, arXiv preprint arXiv:2312.11805
Team, G., Anil, R., Borgeaud, S., et al. 2023, arXiv preprint arXiv:2312.11805
2023 arXiv
-
[59]
2023, arXiv preprint arXiv:2307.09288
Touvron, H., Martin, L., Stone, K., et al. 2023, arXiv preprint arXiv:2307.09288
2023 arXiv
-
[60]
Walmsley , M., Scaife , A. M. M., Lintott , C., et al. 2022, , 513, 1581
2022
- [61]
-
[62]
2012, , 29, 371
Whiting , M., & Humphreys , B. 2012, , 29, 371
2012
-
[63]
L., Eisenhardt , P
Wright , E. L., Eisenhardt , P. R. M., Mainzer , A. K., et al. 2010, , 140, 1868
2010
-
[64]
2022, arXiv preprint arXiv:2205.01917
Yu, J., Wang, Z., Vasudevan, V., et al. 2022, arXiv preprint arXiv:2205.01917
2022 arXiv
-
[65]
2024, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14022--14032
Yu, Q., Sun, Q., Zhang, X., et al. 2024, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14022--14032
2024
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.