{"id":"8101c18b-1153-47e1-beac-7519521e74fa","arxiv_id":"2412.05842","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A framework learns domain-invariant features from probability outputs to reveal attributes of black-box models whose training data is unknown.","lead":"DREAM is a method to reveal hidden attributes of a black-box machine learning model, such as the number of layers or the optimizer used, without knowing the model's training data. It learns features that stay stable across data domains and improves attribute inference accuracy over prior work that assumed the training dataset was known.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Closed attribute grid is load-bearing: DREAM outputs fixed softmaxes over Table I/VIII values, so an out-of-grid target architecture is silently forced into an in-grid class, limiting the 'arbitrary domain' claim.","rationale":"I read the paper in good faith. Within its tested scope, the central empirical claim is supported by the reported numbers: DREAM beats KENNEN, SVM, and the four OOD baselines on held-out PACS and MEDU domains, on a CIFAR cross-test, on a larger attribute space, and on a ViT attribute grid. I found no internal inconsistency in the MDGAN objective or training loop that would invalidate those results. The most load-bearing assumption is not the domain shift itself but the finite attribute vocabulary. The output layer is a fixed categorical distribution over Table I values (or Table VIII/ViT variants), with no out-of-vocabulary mechanism or abstention option. Consequently, a genuinely arbitrary deployed model can only be mapped onto the attacker's predefined architecture language; the security motivation is exactly where this closed-world restriction is weakest. The paper's mitigation experiments enlarge the grid but do not test generalization to attribute values absent from training, which is the precise condition the concern requires. Since the reader's verdict already conditions on this limitation, my read does not change the CONDITIONAL verdict.","tokens_in":24482,"tokens_out":10764,"duration_ms":114073,"concrete_test":"Train DREAM exactly as in Section IV but restrict source models to kernel sizes {3,5} from Table I; construct a target model with kernel size 7 and all other attributes in-grid. Measure #ks and average attribute accuracy, and also check whether any confidence or abstention signal can detect the out-of-grid value. Repeat with a residual-connection attribute value present only in the target. If #ks accuracy falls to chance and no out-of-grid signal is produced, the closed grid is load-bearing; if accuracy transfers to the unseen value, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-A constructs modelsets by 'enumerat[ing] all possible attribute values' over Table I; the meta-model in Eq. 2-3 is K independent softmax classifiers, one per attribute, over exactly those finite value sets. Thus the method is a closed-set classifier over a hand-designed architecture language. This is separate from the unknown-training-data relaxation: even if the target's domain is arbitrary, any target architecture whose attribute value is outside the enumerated space (e.g., kernel size 7 in the main experiments, residual connections, GroupNorm, or a non-enumerated optimizer) has no correct answer in the output space. The classifier is forced to choose the nearest in-grid value and cannot abstain or report 'unknown'. The experiments do not exercise this gap: Tables II-VII and the larger space in Table VIII train and test within the same value sets; Section IV-G3 only holds out combinations of values, not the values themselves. So the abstract's 'arbitrary domain' is supported only for an arbitrary input distribution over a fixed architecture grid, not for arbitrary deployed models, which is the security scenario motivating the work.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies black-box model attribute reverse engineering when the target model's training data is unknown. The authors cast the problem as out-of-distribution (OOD) generalization: they train white-box models on multiple source domains with overlapping label spaces, obtain model outputs via queries, learn domain-invariant features with a multi-discriminator GAN (MDGAN), and train per-attribute classifiers on these features. At inference, queries are sent to the target black-box model and the extracted features are classified into attribute values. The paper reports experiments on PACS and MEDU modelsets, on a larger attribute space, on Vision Transformers, on a cross-dataset CIFAR setting, and on a model-extraction application, with DREAM consistently ahead of the compared baselines.","tokens_in":24615,"tokens_out":6822,"duration_ms":66643,"significance":"If the claims hold, the paper makes a useful advance over KENNEN by removing the requirement that the attacker know the target model's training dataset, which is a practically important relaxation. The paper also introduces a plausible OOD formulation for probability-output data rather than images, which is a novel angle, and it provides extensive experiments across multiple benchmarks, a larger attribute space, a ViT extension, and a downstream model-extraction validation. The consistent improvements over the baselines, while often modest, are a meaningful positive result. However, the central 'arbitrary domain' claim in the abstract is broader than what the closed attribute grid can support, and several experimental details are missing or inconsistent, so the manuscript needs substantial revision before the contribution can be fully assessed.","major_comments":[{"comment":"DREAM is a closed-set classifier over the attribute values enumerated in Table I (and the larger spaces in Tables VIII and IX). Each per-attribute head outputs a softmax over exactly the N^k values in the grid, so a target model whose attribute value is not in the grid (e.g., kernel size 7 in the main experiments, or a residual connection, GroupNorm, or a non-enumerated optimizer) has no correct output and is silently assigned an in-grid value. The abstract's claim that DREAM 'can gracefully apply to an arbitrary domain' is therefore not supported for out-of-grid architectures; Section IV-G3 only holds out combinations of values within the same grid, not values outside it. Please either restrict the claim to the enumerated attribute space or add an open-set/unknown-value mechanism with experiments on out-of-grid attributes.","section":"Section IV-A and Eq. (2)"},{"comment":"The manuscript refers to 'Appendix.B' for the statistics of the PACS and MEDU modelsets, but no appendix is present in the submitted text. In addition, Section IV-A describes per-domain splits of 5,000/1,000/1,000 (train/validation/test), which would imply 15,000/3,000/3,000 models across the three PACS domains, whereas Section V states that the PACS modelset comprises 13,000 models (10,000 training, 2,000 validation, 1,000 testing). These numbers need to be reconciled, and the full construction procedure (training epochs, hyperparameters, query sampling, random seeds, and per-domain statistics) must be provided to make the experiments reproducible and to verify the stated 45 GPU-day cost.","section":"Section IV-A and Section V"},{"comment":"The CIFAR experiment changes the number of classes between source and target (7 source classes versus 5 selected target classes), but the generator's input dimension is fixed to N*C as stated in Section IV-B (e.g., 700 for PACS with C=7). The paper does not explain how a target output vector of dimension N*5 is fed into this generator, nor how the target model's output probabilities over its full label set are reduced to the five overlapping classes. The text also says the source domains are Sketch and Cartoon, while the Table VI caption says Photo and Sketch. As written, the claimed result that DREAM handles different numbers of classes is not reproducible and the description is internally inconsistent.","section":"Section IV-G2 and Table VI"},{"comment":"The paper reports averages over 10 trials but does not report standard deviations or significance tests. Since several of the reported improvements over the best baseline are around 2-5 percentage points in average accuracy, it is not possible to assess whether these differences are reliable or within trial-to-trial noise. Reporting variance and, where feasible, paired significance tests would materially strengthen the central claim that DREAM outperforms the baselines.","section":"Section IV-D, Tables II and III"}],"minor_comments":[{"comment":"The claim that MDGAN 'does not suffer from mode collapse' because G is an encoder rather than a generator is not substantiated; an encoder network in an adversarial setup can still collapse if different inputs map to nearly identical features. Please either provide empirical evidence or soften this claim.","section":"Section III-D"},{"comment":"The citation mapping in the reverse-engineering related works appears to be off: the text says '[37] theoretically proves the weight and bias can be reversed in linear network with ReLU activation' (which corresponds to Rolnick and Kording, reference [36]) and '[38] infers hyperparameters and loss functions of generative models' (which corresponds to Asnani et al., reference [37]). Please correct the citation numbers.","section":"Section II, references [36]-[38]"},{"comment":"There is a typo: 'the number of classes if C = 10' should read 'the number of classes is C = 10'.","section":"Section IV-B"},{"comment":"The caption of Figure 2 is grammatically incomplete: 'The performance of KENNEN [22] on black-box model trained on Cartoon, Sketch and Photo dataset [24].' should be a full sentence describing the setup and the plotted quantity.","section":"Section I, Figure 2"},{"comment":"The stopping criterion in Algorithm 1 is 'while difference in training loss Lc of meta-model between two consecutive epochs ≥ epsilon', but no value or selection procedure for epsilon is given; the convergence criterion should be specified.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a meaningful problem and the experimental effort is substantial, but the main selling point of 'arbitrary domain' is broader than the closed attribute grid actually supports. The missing appendix and the inconsistent class-number handling in the CIFAR experiment are concrete obstacles to verifying the results. If the authors can resolve these issues, the paper is likely publishable; if the CIFAR class-mismatch cannot be resolved, the corresponding claim should be removed or substantially weakened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper does something real—it relaxes KENNEN's known-training-data assumption and shows consistent, modest gains across multiple modelsets and transfer settings. But the 'arbitrary domain' claim in the abstract is too strong. The method is a closed-set classifier over a fixed attribute grid. If the target model uses an attribute value outside that grid—kernel size 7, residual connections, GroupNorm, a non-enumerated optimizer—there is no correct class in the output space, and the meta-model silently picks the nearest in-grid value. The stress-test note is right about this, and it bears directly on the security scenario motivating the paper.\n\nWhat is genuinely new: the problem relaxation itself and the OOD framing. Relative to KENNEN, removing the requirement that the attacker knows the target's training data is a meaningful step. MDGAN is basically a standard multi-domain adversarial alignment (DANN-style) applied to probability outputs, so the methodological novelty is modest, but the experiments are extensive: leave-one-domain-out on PACS and MEDU, class shift, CIFAR transfer, larger attribute space, and a ViT grid. The average attribute accuracy gains over the best baseline are typically 2–5 points, not huge but consistent across nearly all target domains and attribute types. The extraction demo, despite a garbled Table IV, shows the inferred architecture beats a random one and approaches same-as-victim performance. No circularity: the meta-model trains on source white-box outputs and is tested on held-out target domains.\n\nSoft spots, in order of importance:\n\n1. Closed attribute grid. Load-bearing for the 'arbitrary domain' claim. The experiments hold out combinations of attribute values but never the values themselves. The paper should either widen the tested grid or explicitly scope the claim.\n\n2. Reproducibility. The appendix is referenced but missing; no code or data; no confidence intervals. For a security paper, that hurts. Table IV is garbled.\n\n3. Cost and scaling. 45 GPU-days per modelset is disclosed, but Fig. 6 suggests more white-box models do not reliably help, which raises a question about the approach's scalability.\n\nThe math is simple and checks out. The citation pattern is fine. This is an incremental but genuine contribution. It deserves serious peer review; the main fixes are to rewrite the abstract to match the actual closed-grid scope and to provide the missing materials.","headline":"New problem setting (unknown training data) with solid but modest gains; overclaims 'arbitrary domain' because the attribute grid is fixed and closed.","tokens_in":25252,"tokens_out":3046,"would_cite":true,"duration_ms":27675,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Black-box model attributes can be inferred from probability outputs alone, without any access to the target's training data, by learning domain-invariant output features with a multi-discriminator GAN.","keywords":["black-box reverse engineering","model attribute inference","out-of-distribution generalization","domain-invariant features","multi-discriminator GAN","model extraction attack","probability outputs","MLaaS security"],"falsifier":"Train a target model with an architecture value outside the enumerated grid—for example, kernel size 7 or a residual connection—while keeping every other attribute inside the grid, and run DREAM on it. If the average accuracy over the remaining attributes stays near the reported level, the grid limitation is weak; if accuracy collapses toward random choice, the enumerated-grid assumption is load-bearing for the paper's broad domain-agnostic claim.","tokens_in":24172,"feed_emoji":"🕵️","tokens_out":7505,"duration_ms":71753,"temperature":0.7,"pith_summary":"Model-reverse-engineering attacks previously worked only when the attacker already knew the target model's training dataset, because the meta-model was trained on white-box models trained on that same data. This paper claims that the training data can be dropped entirely: if the attacker knows only the target's label space and can query it, white-box models trained on other datasets with overlapping labels can serve as source domains, and an out-of-distribution generalization step can bridge the domain gap in the output distributions. DREAM learns domain-invariant features from probability outputs with a multi-discriminator GAN, then classifies the nine architecture and training attributes from those features. The experiments report consistent gains over the prior method and four OOD baselines on held-out domains of PACS and MEDU, and show that a surrogate model built with DREAM-inferred attributes extracts the victim's function nearly as well as a surrogate using the exact architecture.","feed_headline":"Black-box model attributes leak without its training data","feed_subtitle":"Queries alone, plus a label-space overlap, let a meta-model infer architecture choices and sharpen model-extraction attacks.","key_machinery":"MDGAN (multi-discriminator generative adversarial network): a generator network that maps each model's concatenated probability outputs into a 128-dimensional latent vector, together with one discriminator per source domain. Each discriminator is trained to call its own domain's features 'real' and every other domain's features 'fake'; the generator is trained to fool all of them while also minimizing cross-entropy attribute-classification loss with a trade-off weight $\\lambda$, so the latent representation becomes domain-invariant across source domains and transfers to the unseen target domain. The domain-agnostic reverse meta-model is a set of nine small MLPs, one per attribute, that classify the invariant features into the enumerated attribute values.","core_discovery":"The paper's claim is that attributes of a black-box classifier—activation function, dropout, pooling, kernel size, number of convolutional and fully connected layers, optimizer, batch size, and batchnorm—remain inferable from probability outputs alone when the target's training set is unknown. The key move is to treat the unknown training-set distribution as an out-of-distribution target domain: the attacker collects datasets whose label space overlaps the target's, trains a large model zoo of 5,184 attribute combinations per domain, and uses those outputs as source domains. A multi-discriminator GAN encodes the output vectors into an invariant latent space that no source-domain discriminator can tell apart, and $K=9$ classifiers predict each attribute from that space. On leave-one-domain-out tests, DREAM reaches 52.38% average attribute accuracy on the Photo target of PACS versus 48.04% for the best baseline, and 56.86% versus 52.33% on the EMNIST target of MEDU; the model-extraction experiment raises stolen-model accuracy from 45.88% for a random surrogate to 62.81% with a DREAM-inferred surrogate, close to 68.46% for an exact-architecture surrogate.","pith_inferences":["Because the attribute language is a fixed grid, the practical reach of the method is grid-bound; an open-set or ordinal version that predicts unseen kernel sizes or depths would be the natural next test.","The same domain-adversarial alignment should apply to other probability-output settings, for example exposing training hyperparameters of language or speech models, but the label-space overlap assumption is harder to satisfy when outputs are open-ended.","The extraction experiment suggests that approximate architecture knowledge captures most of the value of exact architecture knowledge, which would strengthen the threat model for MLaaS even when reverse engineering is imperfect.","An ablation varying the number of discriminators could isolate whether the gain comes from multi-domain alignment rather than from the adversarial objective alone; the paper does not report this comparison."],"forward_implications":["Attribute leakage no longer requires possession of the target's training set; overlapping label space plus a query budget is enough.","An adversary can use inferred attributes to pick a surrogate architecture, recovering most of the extraction gain of knowing the exact victim architecture.","Out-of-distribution generalization machinery can operate directly on probability output vectors, not only on images or raw features.","The framework remains competitive under partial class overlap, when source and target have different numbers of classes, and when training and test attribute combinations are disjoint.","The approach extends beyond the main CNN grid to a larger architecture space and to Vision Transformer attributes."],"supporting_citations":[{"why":"Defines the prior white-box meta-model reverse-engineering task and the strong assumption (known training dataset) that DREAM relaxes; also supplies the basic attribute-grid idea.","marker":"[22]"},{"why":"Provides the PACS multi-domain image dataset used to build source-domain white-box models with overlapping label spaces.","marker":"[24]"},{"why":"Supplies the adversarial GAN min-max formulation that MDGAN extends to multiple discriminators.","marker":"[71]"},{"why":"The model-extraction attack used to demonstrate that DREAM-inferred architecture attributes improve surrogate selection.","marker":"[41]"},{"why":"MMD is a representative domain-invariant learning baseline that DREAM must outperform on probability outputs.","marker":"[28]"},{"why":"SelfReg is an OOD generalization baseline used for comparison in learning invariant features from probability outputs.","marker":"[29]"},{"why":"MixStyle is a style-based OOD generalization baseline compared on model-output features.","marker":"[30]"},{"why":"Spectral Decoupling is an OOD generalization baseline that benchmarks DREAM's invariant-feature learning.","marker":"[77]"}],"fun_headline_variants":["Black-box attributes leak even without training data","No training data? Queries still reveal model attributes","Domain-agnostic model attribute attack without data","Reverse engineering black-box models without their data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the target model's attributes must fit the fixed enumerated grid of attribute values that the white-box models are trained on; kernels, depths, optimizers, or structural choices outside that grid have no representable answer in the meta-model.","fun_headline_variants_meta":{"raw":{"variants":["Black-box attributes leak even without training data","No training data? Queries still reveal model attributes","Domain-agnostic model attribute attack without data","Reverse engineering black-box models without their data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000765,"raw_usage":{"total_tokens":3424,"prompt_tokens":1006,"completion_tokens":2418,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":2372}},"tokens_in":622,"tokens_out":2418,"duration_ms":19443,"temperature":1.0,"reasoning_tokens":2372,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:17:28.743557+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a target model with an architecture value outside the enumerated grid—for example, kernel size 7 or a residual connection—while keeping every other attribute inside the grid, and run DREAM on it. If the average accuracy over the remaining attributes stays near the reported level, the grid limitation is weak; if accuracy collapses toward random choice, the enumerated-grid assumption is load-bearing for the paper's broad domain-agnostic claim.","supporting_citations":[{"cited_title":"Towards reverse- engineering black-box neural networks,","cited_arxiv_id":null,"evidence_quote":"Defines the prior white-box meta-model reverse-engineering task and the strong assumption (known training dataset) that DREAM relaxes; also supplies the basic attribute-grid idea."},{"cited_title":"Maze: Data-free model stealing attack using zeroth-order gradient estimation,","cited_arxiv_id":null,"evidence_quote":"The model-extraction attack used to demonstrate that DREAM-inferred architecture attributes improve surrogate selection."},{"cited_title":"Selfreg: Self-supervised contrastive regularization for domain generalization,","cited_arxiv_id":null,"evidence_quote":"SelfReg is an OOD generalization baseline used for comparison in learning invariant features from probability outputs."},{"cited_title":"Gradient starvation: A learning proclivity in neural networks,","cited_arxiv_id":null,"evidence_quote":"Spectral Decoupling is an OOD generalization baseline that benchmarks DREAM's invariant-feature learning."}],"review_version":1}