{"id":"f78ee6c7-e242-4cdd-b376-214d628eb5b8","arxiv_id":"2608.12403","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A two-step estimator transfers a black-box predictor's knowledge across mismatched feature spaces, with risk bounds that beat non-transfer learning when the residual component is small or smooth.","lead":"This paper introduces a two-step neural network estimator that transfers knowledge from a black-box predictor defined on one set of features to a target regression on different features, using abundant unlabeled feature pairs plus limited labels. It proves prediction risk bounds showing transfer beats learning from the labeled data alone when the non-transferable residual is small or smooth.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Assumption B on the induced transferable component h=E[bf(Z)|X] is load-bearing; near-deterministic coupling makes h inherit black-box oscillations, so the claimed first-stage rate can fail.","rationale":"The mathematical core of the paper is internally coherent: the two-step estimator, the oracle inequality in Lemma 1, the validation argument in Theorem 1, and the minimax lower bound in Theorem 3 all follow from standard empirical-process and network-approximation techniques, and the rate comparisons are consistent. The reader's identification of Assumption B as the weakest structural point is correct and is also the most load-bearing concern for the central claim. The first-stage rate φ_h^{n_A} is the mechanism by which the abundant unlabeled pairs make the transferable component negligible; if h is not in the postulated compositional Hölder class, that mechanism has no basis. More specifically, the near-deterministic coupling regime Z≈m(X), which is a natural reading of the motivating sensor-upgrade application, gives h≈bf∘m, so irregularity of the black box transfers directly to h. The paper's simulations choose Z=0.9X^T B+0.1V, where convolution with the uniform noise smooths h; this is favorable and does not test the degenerate-coupling failure mode. The real-data analysis reports gains but does not estimate or diagnose the effective smoothness of h. Since the theorems are conditional, the concern does not invalidate the mathematics; it narrows the domain in which the central practical claim is supported. The reader's CONDITIONAL verdict remains appropriate, and I would not change it: the paper should be accepted conditionally on strengthening the empirical evidence or explicitly delimiting the applicability of Assumption B (and, as the reader notes, on releasing code or a full preprocessing specification).","tokens_in":48424,"tokens_out":20243,"duration_ms":221251,"concrete_test":"Simulation with a deterministic bridge: set Z=m(X) with a fixed nonlinear m (e.g., m(x)=x^2 coordinatewise) and black box bf(z)=cos(c z) with c large or growing with n_A; draw D_A with tiny coupling noise and D_T with n_T=400, δ=0. Compute the empirical L2 error of the first-stage bh as n_A grows and compare its exponent to the n_A^{-ρ_h} rate predicted by any fixed compositional Hölder class; also compare BB-transfer versus non-transfer NN. If the first-stage error fails to improve or the transfer advantage vanishes or reverses, Assumption B is indeed load-bearing for the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Theorems 1 and 3 deliver strict improvement over non-transfer learning only if the first-stage error C_h φ_h^{n_A} log^3 n_A is o(n_T^{-ρ_h}). That rate is supplied entirely by Assumption B, which postulates h(x)=E[bf(Z)|X=x] ∈ G(q_h,d_h,t_h,β_h,K_h). This is not a primitive assumption about the black box; it is a regularity condition on the conditional expectation induced by the coupling between Z and X. In the near-deterministic regime that motivates the paper (new sensors measuring the same scene), Z≈m(X) and h≈bf∘m. A highly nonlinear black box then makes h as oscillatory as bf, with Hölder norm growing with the oscillation frequency, so no fixed compositional class with finite K_h can capture it. When this happens, the n_A-pairs first stage need not converge faster than n_T-pairs, and the strict polynomial improvement claimed after Theorem 3 disappears. The issue is not an algebraic error: the rate algebra is correct conditional on Assumption B. But the central practical claim that a black box can be transferred across feature sets rests on an unverified regularity premise about the bridge distribution, and the paper offers no evidence (e.g., estimation of h's effective smoothness in the real data) that it holds.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for transferring predictive knowledge from a pre-trained black-box function bf: Z -> R to a target regression g(X) = E[Y|X] on a different feature space X. Using the algebraic decomposition g = h + delta, with h(x) = E[bf(Z)|X=x] and delta(x) = E[Y - bf(Z)|X=x], it estimates h from abundant unlabeled paired observations (X,Z) by regressing bf(Z) on X, and delta from limited labeled observations (X,Y), followed by a validation-based choice of whether to include the delta estimate. The main theoretical results are a risk bound for the two-step estimator (Theorem 1), an upper bound for a non-transfer baseline (Theorem 2), a minimax lower bound showing that the transfer bound can be of strictly smaller polynomial order than any estimator based only on the labeled data (Theorem 3), and an ensemble extension to multiple black boxes (Theorem 4). Simulations and a real-data chlorophyll application support the qualitative predictions.","tokens_in":48760,"tokens_out":19958,"duration_ms":190301,"significance":"The problem is timely and the black-box, heterogeneous-feature setting is well motivated. The paper's main strengths are the full supplementary proofs, which appear complete; the minimax lower bound in Theorem 3, which formalizes when transfer learning from unlabeled pairs is fundamentally beneficial; the clean oracle-style ensemble inequality in Theorem 4; and the treatment of imputation as a competing baseline. If the theorems hold, the paper is a substantive contribution to nonparametric transfer learning. The principal caveat is that the rate gains are conditional on a smoothness assumption on the induced transferable component h = E[bf(Z)|X], which is not a property of the black box itself and may fail in the near-deterministic coupling regimes that motivate the applications; the paper would be strengthened by addressing this gap.","major_comments":[{"comment":"The rate improvement that is the paper's central claim rests on Assumption B, which postulates that h(x) = E[bf(Z)|X=x] lies in a compositional Holder class G(q_h, d_h, t_h, beta_h, K_h). This is not a primitive assumption about the black box; it is a regularity condition on the bridge distribution. In the near-deterministic regime Z approximately m(X) that motivates the new-sensor applications, h is approximately bf composed with m, so a highly oscillatory black box makes h as rough as bf and outside any fixed class with finite K_h. When this occurs, the first-stage estimate from n_A pairs need not converge faster than the direct estimate from n_T pairs, and the strict polynomial improvement claimed after Theorem 3 disappears. The manuscript offers no diagnostic or empirical check, such as an estimate of h's effective smoothness in the real-data application, that Assumption B holds. I recommend adding an explicit discussion of when this assumption is plausible, a robustness analysis for rough h, or a clear scope limitation in the abstract and conclusions.","section":"§4.1, Assumption B; Theorem 1; Theorem 3"},{"comment":"The claim that the estimator 'adaptively attains the oracle guarantee' is only adaptivity over the binary inclusion decision lambda in {0,1}. The class F_delta in Assumption C is constructed using the true compositional parameters (q_delta, t_delta, beta_delta, K_delta), so the rate C_delta phi_delta^{n_T} log^3 n_T is not achieved without prior knowledge of delta's smoothness and intrinsic dimension. Similarly, F_h uses (q_h, t_h, beta_h, K_h). The paper states that architectures are tuned by validation in practice, but the theory does not analyze this. Please either qualify the adaptivity claim in Section 4.2 and the abstract, or extend the analysis to model selection over a sieve of network classes.","section":"§4.2, Theorem 1 and Assumption C"}],"minor_comments":[{"comment":"In the sentence 'we report the average relative MSE ... with it's standard error', 'it's' should be 'its'.","section":"§6.1"},{"comment":"The phrase 'The correspondingly labeled panels (a)-(c) of Figure 3' reads awkwardly; consider 'The corresponding panels (a)-(c) of Figure 3'.","section":"§6.2"},{"comment":"The paper should clarify that Lemma 1's oracle estimator, which fits e_delta on all of D_T, is a non-implementable benchmark, whereas Algorithm 1 splits D_T into training and validation; this distinction is easy to miss on first reading.","section":"Algorithm 1 and Lemma 1"},{"comment":"In the acknowledgments, 'for the use of data' should be 'for the use of the data'.","section":"§7"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically sound and the supplementary proofs are complete. The main concern is the gap between the strong smoothness assumption on the induced transferable component h and the practical claims in the abstract and introduction. The adaptivity claim also needs qualification because the theory assumes known compositional parameters. I recommend major revision rather than rejection, since these issues can be addressed by adding discussion, empirical diagnostics, or a more careful statement of the scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper deserves a serious referee. It gives the first nonparametric risk bounds I know for transferring a fixed black-box predictor when source and target features are different spaces. The decomposition g = E[bf(Z)|X] + delta is elementary but right, and the two-step estimator - estimate the transferable component h from abundant unlabeled pairs, then estimate delta from scarce labels with a validation step that decides whether to keep it - is natural and clean. The proof structure is standard (sparse ReLU approximation, empirical process bounds, Fano for the minimax lower bound), and the supplementary material looks coherent. The minimax lower bound is a genuinely careful piece: constructing a coupling so that any candidate h is a conditional expectation of bf(Z) given X is neat. The ensemble extension is a modest but useful bonus.\n\nThe soft spot is exactly what the stress-test note says, and it is the main story: Assumption B postulates that h lies in a compositional Holder class, but h is not a primitive property of the black box. It is an induced object that depends on the coupling between Z and X. In the motivating near-deterministic case (new sensors measuring the same scene), h is approximately bf composed with m, so a highly oscillatory black box makes h rough and the first-stage rate from n_A pairs can be no better than the target-only rate. This does not break the theorems - the rates are correctly stated conditional on Assumption B - but it means the practical claim that 'transfer helps when delta is small or smooth' is incomplete: transfer also requires h to be learnable, and that requires smoothness of the induced h. The abstract and introduction should say this clearly. The real-data analysis does not estimate or diagnose the smoothness of h, so the field application provides no evidence that the central assumption holds. The absence of released code is a smaller but real practical gap.\n\nOverall, the paper is a solid theoretical contribution with an honest limitation that the authors should be pushed to expose. I would send it to a statistics journal or strong ML venue. If the authors clarify the assumption's role, add a diagnostic or discussion of when it may fail, and release code or a full preprocessing specification, it would be a useful reference. I would be happy to cite it once those caveats are on the record.","headline":"A genuinely new nonparametric framework for black-box transfer across feature spaces, with rate guarantees that hold conditional on a smoothness assumption on the induced transferable component; the real weakness is that the abstract oversells the regime of improvement and the motivating near-deterministic coupling case can violate that assumption.","tokens_in":665,"tokens_out":853,"would_cite":true,"duration_ms":44527,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","62G20","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-step estimator transfers a black-box predictor to a different feature space and proves gains over labels-only learning when the leftover term is small or smooth.","keywords":["heterogeneous transfer learning","black-box prediction","unlabeled paired data","nonparametric regression","deep neural networks","prediction risk bounds","ensemble learning"],"falsifier":"Construct a model satisfying Assumptions A and C in which $\\delta=0$ but $h(x)=E[b_f(Z)\\mid X=x]$ is a sum of many high-frequency sine terms with frequency growing with $n_A$, so that $h$ is not in any compositional H\\\"older class with fixed smoothness; the two-step estimator's risk should fail to decay at the claimed $\\varphi_h^{n_A}$ rate, showing Assumption B is doing the work.","tokens_in":48154,"feed_emoji":"🧠","tokens_out":7935,"duration_ms":69093,"temperature":0.7,"pith_summary":"The paper claims that a pre-trained black-box predictor can still be useful when the target data live in a different feature space, without any access to its training data or internals. The route is an identity: the target regression $g(x)=E[Y\\mid X=x]$ splits into $h(x)=E[b_f(Z)\\mid X=x]$, the part the black box can explain, and $\\delta(x)=E[Y-b_f(Z)\\mid X=x]$, the new information. The method estimates $h$ from abundant unlabeled pairs $(Z,X)$ and $\\delta$ from limited labeled pairs $(X,Y)$, with a validation step that discards $\\delta$ when it is not worth estimating. The main theoretical claim is a prediction-risk bound that is small when $\\delta$ is small or smooth, and that under an abundant auxiliary sample beats the minimax rate of any estimator using only the labeled data. A reader should care because mismatched feature spaces are common when sensors or record systems are upgraded, and this gives a rigorous way to reuse expensive trained predictors.","feed_headline":"Two-step transfer provably beats labels-only learning in broad regimes","feed_subtitle":"Splitting the target function into transferable and new parts turns abundant unlabeled pairs into statistical leverage.","key_machinery":"The load-bearing identity is $g(x)=h(x)+\\delta(x)$, with $h(x)=E[b_f(Z)\\mid X=x]$ estimated first from abundant unlabeled pairs and $\\delta(x)=E[Y-b_f(Z)\\mid X=x]$ estimated second from the limited labels. Two deep ReLU network classes approximate the components, and the theory uses compositional H\\\"older classes, where each coordinate function depends on a small number of variables, to define rates $\\varphi_h^n$ and $\\varphi_\\delta^n$. The validation split of the labeled data, choosing $\\lambda\\in\\{0,1\\}$, is what converts the oracle-style minimum into an adaptive procedure at the price of a $n_T^{-1/2}$ term.","core_discovery":"On its own terms, the paper establishes that the two-step estimator $\\widehat{g}_{\\widehat{\\lambda}}=\\widehat{h}+\\widehat{\\lambda}\\widehat{\\delta}$, with $\\widehat{\\lambda}\\in\\{0,1\\}$ chosen by validation, has risk bounded by $C_h \\varphi_h^{n_A}\\log^3 n_A + \\min\\{2\\|\\delta\\|_2^2, C_\\delta \\varphi_\\delta^{n_T}\\log^3 n_T\\} + C_{\\mathrm{val}} n_T^{-1/2}$. Here $\\varphi_h^{n_A}$ is the sparse-ReLU estimation rate for a compositional H\\\"older $h$ from $n_A$ pairs, and $\\varphi_\\delta^{n_T}$ the analogous rate for $\\delta$. The minimum encodes the two favorable regimes, small $\\delta$ or smooth $\\delta$, and the validation choice makes the estimator adaptive to whichever holds. When the auxiliary sample is large enough ($n_A\\gg n_T^{\\rho/\\rho_h}$) and $\\delta$ is strictly easier than $h$ ($\\rho_\\delta>\\rho_h$, $\\rho\\le 1/2$), the paper proves the worst-case risk is of strictly smaller polynomial order than the minimax risk of any estimator using only the labeled data. For multiple black boxes, an ensemble of the per-box estimators is shown to be no worse than the best single box up to a $n_T^{-1/2}$ validation cost and strictly better when the boxes are comparably accurate but distinct.","pith_inferences":["The decomposition suggests a pilot diagnostic the paper does not develop: estimating $\\|\\delta\\|_2$ and the effective smoothness of $\\delta$ from a small labeled sample could decide whether to include $\\widehat{\\delta}$ before the final fit, making the adaptive step even more direct.","The Neyman-orthogonal residual estimator in the supplement could be turned into confidence bands for $\\delta$ itself, which would elevate the transferable/non-transferable split from a prediction device to an inferential one.","For the multi-box ensemble, the theory points to $D=\\|\\widehat{g}_Z-\\widehat{g}_W\\|_2^2$ as the operative measure of diversity; a practitioner could measure this quantity and use it to decide whether acquiring a second black box is worthwhile.","The paper stops at prediction; a natural extension is to ask when transfer actively hurts, for instance when $\\delta$ is both large and rough, which would provide a formal 'do not transfer' rule."],"forward_implications":["When unlabeled paired features are abundant, the transferable component $h$ can be estimated almost for free; the scarce target labels only need to resolve the residual $\\delta$.","The validation step makes the method work in either favorable regime without knowing in advance whether the black box is accurate or explains the complex part of $g$.","Under the polynomial conditions $n_A\\gg n_T^{\\rho/\\rho_h}$, $\\rho_\\delta>\\rho_h$, $\\rho\\le 1/2$, no estimator using only labeled data can match the transfer estimator's worst-case risk order.","With several black boxes on different feature spaces, ensembling the per-box transfer estimators adapts to the best one and can strictly improve on it when the boxes are comparably accurate but distinct.","Imputation-based transfer, which plugs imputed source features into the black box, can fail when the black box is highly nonlinear; the paper's method estimates the black-box output conditional on $X$ directly and avoids the Jensen gap."],"supporting_citations":[{"why":"Defines the compositional H\\\"older class and gives the sparse-ReLU approximation and entropy rates that produce $\\varphi_h^{n_A}$ and $\\varphi_\\delta^{n_T}$.","marker":"Schmidt-Hieber (2020)"},{"why":"Supplies the fully connected deep ReLU regression-rate results used for both estimation steps.","marker":"Kohler and Langer (2021)"},{"why":"Provides the sub-Gaussian concentration inequalities and empirical-process bounds behind the risk upper bounds.","marker":"Wainwright (2019)"},{"why":"Supplies Fano's inequality and packing constructions used in the minimax lower bound.","marker":"Tsybakov (2009)"},{"why":"Gives the classical stacking rationale for why combining comparably accurate but distinct predictors reduces error.","marker":"Breiman (1996)"},{"why":"Supplies the oracle-style super-learner risk bound that motivates the ensemble selection over black boxes.","marker":"van der Laan et al. (2007)"}],"fun_headline_variants":["Black-box knowledge leaps feature barriers via two-step net","Labels no longer required: transfer across feature sets","Provable transfer: beat labels-only with unlabeled pairs","Bridging feature gaps: transfer knowledge without labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes the transferable and non-transferable components $h$ and $\\delta$ are compositional H\\\"older functions with limited effective dimension, so that deep ReLU networks learn them at the stated polynomial rates; if the black box's conditional mean given $X$ is highly oscillatory, the first-stage estimate from $n_A$ unlabeled pairs may not converge fast enough for the claimed gains.","fun_headline_variants_meta":{"raw":{"variants":["Black-box knowledge leaps feature barriers via two-step net","Labels no longer required: transfer across feature sets","Provable transfer: beat labels-only with unlabeled pairs","Bridging feature gaps: transfer knowledge without labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1595,"prompt_tokens":1062,"completion_tokens":533,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":470}},"tokens_in":678,"tokens_out":533,"duration_ms":4977,"temperature":1.0,"reasoning_tokens":470,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:11:09.003611+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a model satisfying Assumptions A and C in which $\\delta=0$ but $h(x)=E[b_f(Z)\\mid X=x]$ is a sum of many high-frequency sine terms with frequency growing with $n_A$, so that $h$ is not in any compositional H\\\"older class with fixed smoothness; the two-step estimator's risk should fail to decay at the claimed $\\varphi_h^{n_A}$ rate, showing Assumption B is doing the work.","supporting_citations":[{"cited_title":"and Langer, S","cited_arxiv_id":null,"evidence_quote":"Supplies the fully connected deep ReLU regression-rate results used for both estimation steps."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the sub-Gaussian concentration inequalities and empirical-process bounds behind the risk upper bounds."},{"cited_title":"(1996), Stacked regressions, Machine learning, 24, 49--64","cited_arxiv_id":null,"evidence_quote":"Gives the classical stacking rationale for why combining comparably accurate but distinct predictors reduces error."}],"review_version":1}