{"id":"89581dc8-3d49-4bb2-a5a6-437adc1c5ad9","arxiv_id":"2411.10515","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Unsupervised domain adaptation lets a neural network trained on one cosmological simulation recover the matter density from unlabeled HI maps of a different simulation with R² ≥ 0.9.","lead":"This paper shows that two machine-learning domain adaptation techniques, adversarial training and optimal transport, can align the features of neutral hydrogen (HI) maps from two different cosmological simulations so that a model trained on one simulation can infer the matter density from the other without ever seeing its labels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported Ωm R²≥0.9 may be inflated by a map-level rather than simulation-level train/test split; Section II only says 'unseen instances' are used for testing, not that test maps come from simulations absent from training.","rationale":"The paper is a clean proof-of-concept: it uses two standard unsupervised domain adaptation methods, tests both transfer directions, reports honest failures for σ8, and provides a public implementation and architecture tables. The reader's weakest-assumption pick, untested covariate shift in Section III A, is real and worth flagging, but the empirical R² on a properly split test set would largely test that assumption for Ωm in these two simulation suites. The concern I find more load-bearing is at the evaluation level: Section II's 'unseen instances' does not guarantee unseen simulations. Because each CAMELS simulation is a unique parameter point with a fixed initial condition, maps drawn from the same simulation share both the underlying large-scale structure and the label. If the 1,500 test maps come from simulations already represented among the 12,000 training maps in either domain, the adapted encoder can exploit structure-level familiarity rather than learning a generalizable map-to-cosmology relation. The adaptation procedure aligns feature marginals, not pairs, but initiatilizing the target encoder from the source encoder and feeding it maps that are projections of the same cosmic volumes can preserve per-simulation correspondence. This would inflate R² and undermine the abstract's claim about inferring cosmology from new OOD data. A simulation-level split is a decisive, inexpensive check; the verdict should remain conditional on that test. The paper deserves credit for its honest discussion of σ8 and the small-sample experiment, but the headline Ωm result needs this strengthening before the method is framed as critical for upcoming surveys.","tokens_in":14899,"tokens_out":11652,"duration_ms":128273,"concrete_test":"Re-run both SIMBA→TNG and TNG→SIMBA experiments with a simulation-level split: assign complete CAMELS simulations (not individual maps) to source pre-training and target adaptation versus testing, e.g., use 800 simulations for pre-training and adaptation and 200 held-out simulations for testing, ensuring no test map shares an initial condition or parameter point with any training map. Report Ωm and σ8 R² for both ADDA and OT. If Ωm R² drops materially below the reported 0.90-0.95 (or below the source supervised performance), the headline result is inflated by map-level leakage; if it remains ≥0.9, the transfer claim survives this objection.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that a target encoder, without target labels, retrieves Ωm from OOD HI maps with R²≥0.9 (Abstract, Table I). The most load-bearing issue is not only the covariate-shift condition p_S(y|x)=p_T(y|x) asserted in Section III A; it is whether the evaluation actually measures generalization to genuinely new cosmological structures. Section II states that '1,500 of unseen instances from both domains are used for testing,' but it does not state that these instances are drawn from simulations excluded from the 12,000 training maps. In CAMELS, each simulation has a unique (Ωm, σ8, ASN1, ASN2, AAGN1, AAGN2) point and a fixed initial condition, and the 2D maps are projections of that same volume. If the test maps are from the same simulations as the training maps, the target encoder and the frozen source regressor can exploit a per-simulation 'fingerprint' — the shared large-scale structure and known label of that simulation — rather than learning a general map-to-Ωm mapping. The adaptation losses (Eq. 2-3 and Eq. 9) align only feature marginals, so they do not prevent this. Under that leakage, the headline R²≥0.9 would not support the advertised ability to infer cosmology from new, unseen OOD data such as a real survey sky. The covariate-shift assumption is related but secondary: a properly independent test set would itself probe that assumption empirically for Ωm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an unsupervised domain adaptation (UDA) framework for field-level cosmological inference from 21-cm HI maps. Using the CAMELS IllustrisTNG and SIMBA simulation suites as source and target domains, the authors pre-train a source encoder and regressor on labeled in-distribution maps, then adapt a target encoder on unlabeled target maps using either adversarial discriminative domain adaptation (ADDA) or optimal transport (Wasserstein distance). The adapted target encoder is evaluated with the frozen source regressor on target test maps, achieving reported R² values around 0.9 for Ωm in the full-data experiments and degrading performance in the 100-instance adaptation experiments. The authors conclude that UDA can transfer cosmological inference across simulation suites without target labels, with implications for upcoming HI surveys.","tokens_in":15234,"tokens_out":5636,"duration_ms":54972,"significance":"If the central claim holds, this is a useful proof of concept for applying simulation-trained networks to unlabeled survey data, an important practical problem for upcoming HI intensity mapping experiments. The method is clearly described, the code is publicly linked, and the adaptation losses do not use target labels, so there is no obvious circularity in the label usage. However, the significance depends critically on the evaluation protocol: the reported generalization must be measured on simulations whose labels and initial conditions were not seen in training. The paper also includes a reproducible public-data setup, which is a strength, but the current evidence is weakened by the ambiguous train/test split description and by the absence of repeated-run statistics.","major_comments":[{"comment":"The train/test split is not defined at the simulation level. Section II states that 12,000 pairs of {map, label} from each domain are used for training and 1,500 unseen instances for testing, but it does not say whether the test maps come from simulations that also contributed training maps. In CAMELS, each simulation has a unique (Ωm, σ8, ASN1, ASN2, AAGN1, AAGN2) combination and a fixed initial condition, with 2D maps being projections of the same volume. If training and test maps share simulations, the encoder and frozen regressor could exploit a per-simulation fingerprint (shared large-scale structure and label), inflating the reported R² and undermining the out-of-distribution generalization claim. Please clarify the split, and if simulations overlap, re-run the experiments with a simulation-exclusive split (train on some simulations, test on unseen simulations).","section":"Section II and Section IV"},{"comment":"The abstract's claim that 'in all scenarios considered ... the target encoder ... is able to retrieve the underlying Ωm ... to a great accuracy of R² ≥ 0.9' is contradicted by Table II: for TNG→SIMBA with 100 instances, Ωm R² is 0.698 (ADDA) and 0.692 (OT). In addition, σ8 values in Table I are far below 0.9 in several rows (e.g., 0.394 for SIMBA→TNG/OT, 0.205 and -0.215 for TNG→SIMBA). The claim should be restricted to full-data Ωm results, or the experiments and conclusions should be revised accordingly.","section":"Abstract, Table II, and Section IV.C"},{"comment":"All reported R² values come from single runs without error bars or multiple random seeds. Since the comparison between ADDA and OT (e.g., SIMBA→TNG σ8: 0.735 vs 0.394) and between full-data and 100-instance settings (e.g., TNG→SIMBA Ωm: 0.903/0.924 vs 0.698/0.692) is central to the paper's conclusions, the stability of these numbers is important. Please provide repeated runs (e.g., 5–10 seeds) with mean ± standard deviation for at least the headline results in Tables I and II.","section":"Section IV and Tables I-II"},{"comment":"The paper assumes p_S(y|x) = p_T(y|x) on the grounds that both suites share the same parameter priors and that topology is 'equally sensitive' to cosmology, but this concept-shift assumption is not directly tested. If the different baryonic physics changes the map-to-cosmology mapping, aligning feature marginals (Eqs. 2–3 and Eq. 9) can distort or discard information needed by the frozen source regressor. A concrete diagnostic would be to compare the adapted target encoder with a supervised oracle trained on target labels (used only for assessment) to measure how much label-relevant information is preserved after alignment; at minimum, the assumption should be discussed as an additional limitation.","section":"Section III.A"}],"minor_comments":[{"comment":"The phrase 'join probability' should be 'joint probability'.","section":"Section III.A"},{"comment":"The vertical bars in Figures 4 and 6 are labeled 'error bars' but they represent the absolute difference between the ground truth and the predicted parameter, not an uncertainty estimate. Please clarify the caption.","section":"Section IV and Figures 4, 6"},{"comment":"The sentence '1,500 of unseen instances from both domains are used for testing' is ambiguous; state whether this is 1,500 instances per domain or 1,500 total.","section":"Section II"},{"comment":"Define TNADDA and TNOT in the caption or in the text near the table, since these abbreviations are not self-explanatory outside the main text.","section":"Table II"},{"comment":"The phrase 'about 100 times smaller than the original target domain dataset' is imprecise; 100 instances versus 12,000 is 120 times smaller, which is two orders of magnitude as stated in the text.","section":"Section IV.C"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a timely problem and uses a public dataset, but the central generalization claim hinges on the train/test split being at the simulation level. The current text does not establish this, and if the split is map-level, the headline R²≥0.9 could be an artifact of memorizing simulation fingerprints rather than learning a transferable map-to-cosmology relation. The abstract's 'all scenarios' overclaim, together with the lack of repeated runs, further weaken the presentation. I recommend a major revision: clarify or re-run the split, correct the claims, and add error bars. The paper is otherwise within the scope of astro-ph.IM and would be a useful contribution after these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper shows that two off-the-shelf domain adaptation methods (ADDA, optimal transport) can align HI map representations across IllustrisTNG and SIMBA enough that a frozen source regressor recovers Ωm on the target domain with R²≥0.9 in the full-data runs. That is a real, useful proof-of-concept for the 'unlabeled OOD HI data' problem, and the authors are upfront that σ8 mostly fails. The small-sample experiment (100 target maps) is a nice addition, even if the TNG→SIMBA case drops to R²~0.69.\n\nThe main thing to check before believing the strong claim: the paper says 'unseen instances' for the test set, but never says the test maps come from simulations that were excluded from training. In CAMELS, each simulation has a unique parameter point and a fixed initial condition, and the maps are projections of that same volume. If train and test maps share simulations, the discriminator or the frozen regressor could key on a per-simulation fingerprint—the specific large-scale structure and its known label—rather than a general map-to-Ωm relation. The adaptation losses align only feature marginals, so they don't prevent that. If that is the case, the R²≥0.9 does not support the paper's advertised generalization to new, unseen surveys. The covariate-shift assumption (Section III A) is related but secondary: a proper simulation-level test split would itself probe that assumption for Ωm.\n\nOther soft spots are minor by comparison: the abstract's 'all scenarios' conflicts with the 100-instance TNG→SIMBA result; there are no error bars or multiple seeds, so we don't know how stable the R² values are; and the OT σ8 numbers wobble between the full and small-sample experiments without explanation. None of these change the core finding if the split issue is resolved.\n\nWho is this for? Anyone working on field-level cosmological inference, domain adaptation in astrophysics, or preparing analysis pipelines for upcoming HI surveys. It deserves a serious referee: the question is real, the setup is clean, and the methods are standard enough to reproduce. I'd send it out, but the referee report should ask for the splitting scheme explicitly and a simulation-level split as a robustness test. If the authors can show the R² holds with held-out simulations, it becomes a much stronger paper.","headline":"Solid proof-of-concept for unsupervised domain adaptation in cosmology, but the headline R² may depend on an unstated map-level train/test split.","tokens_in":15733,"tokens_out":4423,"would_cite":false,"duration_ms":39160,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A neural network trained on simulated 21-centimetre maps can be adapted to unlabeled maps from a different simulation and still recover the matter density $\\Omega_{\\rm m}$ with $R^2 \\ge 0.9$, without ever seeing target labels.","keywords":["unsupervised domain adaptation","HI intensity mapping","cosmological parameter inference","out-of-distribution generalization","adversarial domain adaptation","optimal transport","CAMELS simulations","field-level inference"],"falsifier":"A direct test: bin both simulation suites by ground-truth $\\Omega_{\\rm m}$ and train a classifier to tell SIMBA from IllustrisTNG maps within each bin. If the classifier succeeds well above chance within bins, then $p_S(y|x) \\ne p_T(y|x)$ and the covariate-shift premise is false; adaptation could then be discarding rather than preserving cosmological information.","tokens_in":14696,"feed_emoji":"📡","tokens_out":6642,"duration_ms":58552,"temperature":0.7,"pith_summary":"The paper asks whether a neural network trained to infer cosmology from simulated 21-centimetre (HI) maps can be transplanted to maps from a different simulation, or eventually from a real survey, without any labels from the new data. It argues yes: unsupervised domain adaptation, either by adversarial feature alignment or by optimal transport, aligns the latent representations of the two datasets closely enough that a frozen regressor pre-trained on the source still recovers the matter density $\\Omega_{\\rm m}$ on out-of-distribution maps, with $R^2 \\ge 0.9$ in the full-data experiments. This matters because forthcoming HI intensity-mapping surveys will deliver unlabeled data whose statistical properties differ from any training simulation. If the claim holds, field-level cosmological inference can be made robust to simulation-versus-survey mismatch without needing target labels.","feed_headline":"Domain adaptation lets unlabeled HI maps reveal cosmology","feed_subtitle":"Aligning features across two simulation suites recovers matter density with R² ≥ 0.9, rivaling supervised training.","key_machinery":"The machinery is two-step feature alignment in the encoder's latent space. A source encoder plus a regressor are pre-trained supervised on in-distribution maps; a target encoder with identical architecture is initialized with the source weights; then, with the source encoder and regressor frozen, either (a) an adversarial discriminator (ADDA) tries to tell source codes from target codes while the target encoder tries to fool it, or (b) the Wasserstein distance between the two code distributions is minimized by optimal transport. The regressor trained on the source is then applied to the aligned target codes. The argument relies on the two domains sharing the same label space and the same conditional map-to-cosmology mapping, so that aligning features does not destroy the cosmological information.","core_discovery":"The central claim is that a target encoder adapted without access to the target labels can match the source encoder's supervised performance on the matter density, provided only covariate shift separates the domains. In the SIMBA-to-IllustrisTNG direction, the adapted target network achieves $R^2 = 0.945$ on $\\Omega_{\\rm m}$ and $0.735$ on $\\sigma_8$, nearly identical to the source network's in-distribution $R^2 = 0.948$, while the un-adapted source network collapses to $R^2 = -0.474$. The paper further shows that the same recovery works with only 100 target instances, reaching $R^2 \\approx 0.89$ for $\\Omega_{\\rm m}$ in that direction, and documents that the reverse direction (IllustrisTNG to SIMBA) recovers $\\Omega_{\\rm m}$ but not $\\sigma_8$, which it attributes to the loss of small-scale information in the blurrier SIMBA maps.","pith_inferences":["If the covariate-shift assumption carries over to real surveys, the same pipeline could be run with a simulation as the source and survey maps as the target; a natural first test would be on semi-realistic mock observations with injected foregrounds and noise, where the true cosmology is still known.","The observed asymmetry (blurry-to-detailed succeeds, detailed-to-blurry fails for $\\sigma_8$) suggests that domain adaptation cannot manufacture information that is absent in the target maps; for real surveys this would mean resolution and foreground cleaning determine which cosmological parameters are recoverable after adaptation.","Feature alignment in latent space is only a proxy for label alignment. A diagnostic that directly tests whether $p_S(y|x) = p_T(y|x)$ holds, for example checking whether source and target maps with the same $\\Omega_{\\rm m}$ remain separable, would tell whether the aligned features preserve cosmological information or simply discard it."],"forward_implications":["A model trained on simulated HI maps can be adapted to unlabeled maps from a different simulation and still infer $\\Omega_{\\rm m}$ with $R^2 \\ge 0.9$, close to the supervised source performance.","Only 100 out-of-distribution maps, about 100 times fewer than the full training set, are enough to adapt the target encoder and recover $\\Omega_{\\rm m}$ with $R^2 \\approx 0.89$ in the SIMBA-to-IllustrisTNG setup.","The success is asymmetric: recovering $\\sigma_8$ from the blurrier SIMBA maps fails even after adaptation, suggesting that the target maps' small-scale content limits which parameters can be transferred.","The same two-step adaptation recipe can in principle be applied to other large-scale observables whose simulated training data differ from the data to which the model is deployed."],"supporting_citations":[{"why":"Introduces Adversarial Discriminative Domain Adaptation, the adversarial alignment method used for the target encoder.","marker":"[49]"},{"why":"Supplies the optimal-transport formulation and transport-map cost that underpin the OT alignment.","marker":"[15]"},{"why":"Provides the optimal-transport library used to compute the Wasserstein loss during adaptation.","marker":"[21]"},{"why":"Defines the regression loss and pre-training prescription used for the source encoder and regressor.","marker":"[2]"},{"why":"Provides the multifield dataset of HI map-label pairs used in the experiments.","marker":"[52]"},{"why":"Presents the CAMELS project and its simulation design, the origin of the maps.","marker":"[50]"},{"why":"Describes the SIMBA simulation suite, one of the two domains.","marker":"[16]"},{"why":"Describes the IllustrisTNG simulation suite, the other domain.","marker":"[39]"},{"why":"Supplies the gradient-reversal layer used to implement the adversarial update.","marker":"[22]"}],"fun_headline_variants":["Unlabeled HI maps infer cosmology via domain adaptation","Domain adaptation recovers matter density from unlabeled maps","Adapting across simulations reveals cosmology without labels","Covariate shift handled: unlabeled HI maps yield matter density","Few unlabeled maps suffice for matter density inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the two datasets differ only in the appearance of the maps, not in how a map is related to the cosmology that produced it; if that map-to-cosmology relation differs between simulations, aligning the features could erase the information the frozen source regressor needs.","fun_headline_variants_meta":{"raw":{"variants":["Unlabeled HI maps infer cosmology via domain adaptation","Domain adaptation recovers matter density from unlabeled maps","Adapting across simulations reveals cosmology without labels","Covariate shift handled: unlabeled HI maps yield matter density","Few unlabeled maps suffice for matter density inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000315,"raw_usage":{"total_tokens":1830,"prompt_tokens":1032,"completion_tokens":798,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":723}},"tokens_in":648,"tokens_out":798,"duration_ms":7896,"temperature":1.0,"reasoning_tokens":723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:39:07.039411+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: bin both simulation suites by ground-truth $\\Omega_{\\rm m}$ and train a classifier to tell SIMBA from IllustrisTNG maps within each bin. If the classifier succeeds well above chance within bins, then $p_S(y|x) \\ne p_T(y|x)$ and the covariate-shift premise is false; adaptation could then be discarding rather than preserving cosmological information.","supporting_citations":[{"cited_title":"pp 443–450","cited_arxiv_id":null,"evidence_quote":"Introduces Adversarial Discriminative Domain Adaptation, the adversarial alignment method used for the target encoder."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the optimal-transport library used to compute the Wasserstein loss during adaptation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the regression loss and pre-training prescription used for the source encoder and regressor."},{"cited_title":"pp 7167–7176","cited_arxiv_id":null,"evidence_quote":"Presents the CAMELS project and its simulation design, the origin of the maps."},{"cited_title":"H., Appleby S., 2019, Monthly Notices of the Royal 11 Astronomical Society, 486, 2827","cited_arxiv_id":null,"evidence_quote":"Describes the SIMBA simulation suite, one of the two domains."},{"cited_title":"pp 1180–1189","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-reversal layer used to implement the adversarial update."}],"review_version":1}