{"id":"cde2e0cf-5b6b-416d-bf9e-c6374b0c9cae","arxiv_id":"2607.20309","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Sparse-penalized deep networks for Huber and quantile regression under covariate shift are shown to attain minimax rates (up to logs) for dependent data satisfying a generalized Bernstein inequality.","lead":"New theory for deep neural regression when training and test data differ in covariate distribution and observations are dependent: the paper derives non-asymptotic error bounds for quantile and Huber regression under covariate shift, claiming near-optimal rates for mixing processes. A useful result for anyone building theory for distribution-shift learning on time series.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.4/4.2 only give rate φ^{-(δ/(1+δ))·2s/(2s+d)}; for fixed δ this is suboptimal, and letting δ grow is unjustified because constants depend on U_μ. The claim of minimax optimality for ratios with all moments is not established.","rationale":"The central claim is that the estimators are adaptively minimax optimal, including the unbounded-density-ratio case. The reader's weakest assumption was the imported (A1) concentration condition; that is a valid concern, but the more internally decisive gap is the unbounded-ratio theorem itself: the rate in Theorem 3.4 is suboptimal for every finite δ, and the limiting argument δ→∞ ignores the hidden U_μ constant, which can grow with δ (and hence with n) for perfectly natural distributions with all moments. This is a mathematical gap in the upper bound, not merely a missing lower bound or an external dependency. The concrete test above isolates the bias-variance balance with explicit constants; if it confirms the explosion, the abstract's statement that the truncated estimator 'remains optimal' for ratios with moments of any order is not supported by the theorems. The paper still contains valid contributions for bounded ratios and finite-δ rates, so a conditional acceptance remains appropriate; the condition should be to either control the growth of U_μ or weaken the optimality claim.","tokens_in":35807,"tokens_out":21116,"duration_ms":194296,"concrete_test":"Re-derive Theorem 3.4 keeping all constants explicit, especially the U_μ factor from (7.34). Take d=1, s=1, X=[0,1], P=U[0,1], and Q chosen so that r has a log-normal distribution (log r∼N(0,1)), giving U_μ=e^{μ²/2}. Set δ_n=log φ(n) (φ(n)=n) and η=n^{2/((1+δ_n)·3)} as in the theorem. Compute the two error terms in (7.35)–(7.36): the bias term U_{1+δ_n}/η^{δ_n} and the variance term η (log n)^ν n^{-2/3}. If the resulting bound is not O(n^{-2/3}) — in this example the bias term behaves like exp((log n)²/2) and dominates — then the 'moments of any order ⇒ minimax optimal' claim fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section 3.2 claim that with a truncated density ratio the estimator 'remains optimal' whenever r has moments of any order. But Theorem 3.4 (with μ=1+δ) bounds the L2(Q_X0) error by (log φ(n))^ν φ(n)^{-(δ/(1+δ))·2s/(2s+d)}. For every fixed δ this exponent is strictly smaller in magnitude than 2s/(2s+d); e.g. d=1, s=1, δ=1 gives n^{-1/3} instead of n^{-2/3}. The text says 'when δ→+∞ ... the bound coincides', but δ is not a parameter of the data-generating process; the proof's bias term is U_μ/η^δ (see (7.34)), and the final big-O suppresses a factor U_μ. For a ratio with all moments, U_μ can grow arbitrarily fast (e.g. log-normal ratio has U_μ=e^{μ²/2}), so choosing δ=δ_n→∞ to close the gap injects an n-dependent constant that can destroy the rate. Thus the theorem does not establish the advertised minimax optimality under (A3) with moments of any order; it only gives a family of suboptimal bounds indexed by δ. This is not a matter of an unproven lower bound: the upper bound itself fails to reach the claimed rate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops sparse-penalized deep neural network (SPDNN) estimators for nonparametric Huber and quantile regression under covariate shift when observations are dependent. The authors assume a generalized Bernstein inequality (A1) with an effective sample size φ(n), covering i.i.d., φ-mixing, exponential α-mixing, and geometrically C-mixing processes. They provide non-asymptotic L2(Q_X0) error bounds for unweighted, reweighted, and two-step pre-training reweighted estimators, with the density ratio known, estimated, or truncated. The stated central claim is that these estimators attain, up to logarithmic factors, the minimax optimal rate φ(n)^{-2s/(2s+d)} on Hölder classes, including the case where the density ratio is unbounded but has moments of any order.","tokens_in":36199,"tokens_out":10360,"duration_ms":95138,"significance":"If the claims are fully established, the paper would be a useful unification: it extends the i.i.d. covariate-shift results of [11] to dependent data, adds Huber regression, and replaces a sparsity constraint with a sparse penalty, which is computationally more tractable. The explicit dependence on φ(n) and the summary table for several dependence structures are helpful. The main proofs contain detailed calculations for the Huber loss and for the reweighted excess risk decomposition. However, the advertised minimax optimality under unbounded density ratios, and the optimality claim for dependent processes, are not actually established by the results as written; these issues affect the abstract, the introduction, Theorem 3.4, Theorem 4.2, and Table 1.","major_comments":[{"comment":"The claim that the truncated-ratio estimator 'remains optimal' whenever r has moments of any order is not established. For fixed δ>0 the bound in (3.20) has exponent (δ/(1+δ))·2s/(2s+d), which is strictly smaller than 2s/(2s+d). The text says that as δ→∞ the bound coincides with the optimal rate, but δ is not a data parameter: U_μ in (A3) depends on μ=1+δ, and the bias term in (7.34) is U_μ/η^δ with η=φ(n)^{2s/((1+δ)(2s+d))}. The final O(·) suppresses U_μ, which can grow arbitrarily fast with μ (e.g., for a log-normal ratio, U_μ=e^{μ^2/2}). Choosing δ=δ_n→∞ injects an n-dependent constant that can destroy the rate. Thus Theorem 3.4 provides only a family of suboptimal bounds for fixed δ, not the advertised minimax rate under (A3) with all moments. This also affects Proposition 4.2, Theorem 4.2, and the corresponding rows of Table 1. The claim should be weakened, or an explicit uniform co","section":"§3.2, Theorem 3.4 and §7.8, Eq. (7.34)"},{"comment":"The 'minimax optimal' statements for dependent data are upper-bound-only. The rates in Theorems 3.1–4.2 are of the form φ(n)^{-2s/(2s+d)} (up to logs), but the only lower bound invoked is Stone's i.i.d. result [42]. Assumption (A1) is a concentration condition that is satisfied with a very small φ(n) by processes with strong dependence; over the class of all processes satisfying (A1), no estimator can have a rate faster than the worst φ(n), so φ(n)^{-2s/(2s+d)} is not a lower bound for that class. For the specific mixing classes listed in Section 5, a matching minimax lower bound is not proved or quoted. The abstract and Section 3.2 should either provide/quote such lower bounds for these dependent classes or state more precisely that the paper establishes upper bounds matching the i.i.d. minimax rate with φ(n) playing the role of n.","section":"§2.3 (A1), §3.2, §5"},{"comment":"Several load-bearing steps are imported from the authors' own unpublished or very recent preprints [21,22], and the present paper does not state or verify those results. In particular, (7.16) is asserted to follow from Corollary 3.4 of [22] verbatim; Theorem 3.2's proof says 'the rest ... is obtained by following the same steps as in the proof of Theorem 4.1 in [21]'; Proposition 4.1 and Proposition 4.2 similarly rely on 'the same steps as' [21]. Since the optimal-rate claims depend on these imported results, the manuscript should include the precise statements and either prove them in the appendix or cite published versions with a clear verification that their assumptions are satisfied in the present setting.","section":"§7.3, Eq. (7.16); §7.5; §7.9"}],"minor_comments":[{"comment":"The statement says 'Consider the estimator brS defined in (4.1)' but the theorem concerns the truncated estimator brη,S defined in (4.3).","section":"Theorem 4.2"},{"comment":"The exponent '2α/(2α+d)' appears to be a typo; it should be 2s1/(2s1+d). Similarly, (7.48) uses 's' where s2 is meant.","section":"§7.12, Eq. (7.47)"},{"comment":"The sentence 'we get from (7.42)' should refer to (7.41).","section":"§7.5, after Eq. (7.41)"},{"comment":"'the convergence rate in (4.1)' should be 'the convergence rate in (4.7)'.","section":"§4.2, Remark 4.1"},{"comment":"Typographical issues: 'Geoemtrically' should be 'Geometrically'; 'satisfaying' in Section 2.3 should be 'satisfying'.","section":"§5, Table 1"},{"comment":"The theorem chooses η and ν4 depending on the smoothness parameter s, which is in tension with the paper's 'adaptive' claim. Please clarify whether an s-free choice, such as ν4=1 as in Corollary 3.1, is sufficient for the stated rate.","section":"§3.2, Theorem 3.4"}],"recommendation":"major_revision","confidential_remarks":"The core upper bounds for fixed δ and bounded density ratios appear plausible, but the 'moments of any order' optimality and the dependent-data minimax claims need substantial revision. The heavy dependence on the authors' own preprints [21,22] should be addressed, perhaps by including the relevant statements and proofs in the appendix. The paper may be publishable after these issues are resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The bounded-ratio part is the real contribution: sparse-penalized DNNs for Huber and quantile regression under covariate shift, with rates expressed through φ(n) for i.i.d., φ-mixing, α-mixing, and C-mixing processes. The rates are plausible, and the Huber/quantile decomposition lemmas are actually proven in the appendix. The unbounded-ratio claim is not supported. Theorem 3.4 gives an L2(Q) bound of order φ(n)^{-(δ/(1+δ))·2s/(2s+d)} for fixed δ. Sending δ→∞ is unjustified because the moment constant U_μ enters the bias term FK_{\\ell_\\tau} U_μ/η^δ, and for ratios with all moments U_μ can grow arbitrarily fast (a log-normal ratio gives U_μ=e^{μ^2/2}). So the advertised “remains minimax optimal” under (A3) with moments of any order is not established; the same problem propagates to Theorem 4.2 and to the table. \n\nWhat the paper does well: it takes the Huber and quantile losses seriously, gives a clean two-step estimation procedure for the density ratio, and is transparent about where the main rates come from. The i.i.d./mixing recovery via φ(n) is useful and the citation of the authors' own framework [22] and [21] is explicit rather than hidden.\n\nSoft spots, in proportion. First, the dependence on unpublished [22] and on [21] is heavy: the key unweighted excess-risk bound is imported verbatim from Corollary 3.4 of [22], and Theorem 3.2 is explicitly proved by following Theorem 4.1 of [21] with φ(n) in place of n^α. That is acceptable if those results are solid, but an independent check is needed; the current text does not supply one. Second, the word “adaptive” overstates the theorems: the architecture conditions in (3.9) include N_n ≳ φ(n)^{d/(2s+d)}, so s appears in the network width, and the claim that the architecture can be chosen independently of s is not automatic. Third, there are several notation slips (for example the B_m exponent in (4.5) and the ν/ν_0 notation in Proposition 4.1); these are minor. None of these issues undermines the bounded-ratio results.\n\nWho should read this: researchers working on deep nonparametric theory under dependence or distribution shift. It deserves a serious referee. The referee should ask for a fixed-δ statement of Theorems 3.4 and 4.2 with explicit dependence on U_μ, a cleaner adaptivity argument, and a verification of the imported bounds from [22]. With that revision the paper would be in good shape; as it stands the abstract overclaims.","headline":"A genuinely new dependent-data extension for deep Huber and quantile regression under covariate shift, with sound-looking bounded-ratio rates, but the claimed optimality for unbounded density ratios rests on an unjustified δ→∞ argument and should be revised.","tokens_in":36698,"tokens_out":2761,"would_cite":false,"duration_ms":30289,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","62M10","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse-penalized deep networks achieve minimax-optimal regression rates under covariate shift for a broad class of dependent processes.","keywords":["deep neural networks","covariate shift","nonparametric regression","Huber regression","quantile regression","dependent data","generalized Bernstein inequality","minimax optimality"],"falsifier":"Take a stationary ergodic process with long memory (e.g., fractional Gaussian noise with Hurst parameter > 1/2) for which the generalized Bernstein inequality in (A1) is believed to fail, fit the reweighted SPDNN estimator to simulated data under a known covariate shift, and compare the empirical L2(Q_X0) error with the predicted φ(n)^{-2s/(2s+d)} decay. A decay strictly slower than the theory prescribes, or a failure of the tail bound for a simple bounded test function, would show the assumptions are tighter than stated.","tokens_in":35686,"feed_emoji":"📈","tokens_out":6856,"duration_ms":54642,"temperature":0.7,"pith_summary":"This paper establishes that sparse-penalized deep neural networks can achieve, up to logarithmic factors, the minimax-optimal L2 error for nonparametric regression under covariate shift, even when the training observations are dependent rather than i.i.d. The setting covers both Huber and quantile regression with Hölder-smooth regression functions, and the target error is measured under the test distribution Q, whose covariate density may differ from the training distribution P. The key device is a generalized Bernstein inequality that measures dependence through an 'effective number of observations' φ(n), unifying i.i.d., φ-mixing, strong-mixing, and C-mixing processes. When the density ratio q/p is uniformly bounded, the reweighted estimator attains the rate φ(n)^{-2s/(2s+d)}; a truncated density ratio preserves optimality when the ratio is unbounded but has moments of any order. The sparse penalty makes the estimators adaptive, so architecture and tuning parameters do not need to know the smoothness s.","feed_headline":"Deep nets match minimax rates under covariate shift and dependence","feed_subtitle":"Sparse-penalized networks adapt to unknown smoothness for mixing time series and heavy-tailed errors.","key_machinery":"The load-bearing object is Assumption (A1), a generalized Bernstein inequality holding for every bounded mean-zero function of the process; it packages all dependence structures through an effective number of observations φ(n) ≤ n. The sparse penalty J_n(h) = Σ_j π_{λ_n,τ_n}(|θ_j(h)|), with π nondecreasing and π(x)=λ_n for x>τ_n (e.g., clipped L1), provides adaptation to the unknown smoothness s. Reweighting by r(x) or its truncation T_η r turns the source risk into the target risk; when r is unknown, a least-squares SPDNN estimates it from unlabeled samples of both distributions. The proof decomposes the L2(Q) error into a statistical term controlled by (A1) and an approximation term for Hö","core_discovery":"The paper proposes a sparse-penalized deep network (SPDNN) estimator that reweights each sample by the density ratio r(x)=q_X0(x)/p_X0(x), and proves its L2(Q_X0) error is O((log φ(n))^ν φ(n)^{-2s/(2s+d)}) for any ν>3, up to log factors the minimax rate, whenever the process satisfies a generalized Bernstein inequality with effective sample size φ(n). This covers i.i.d. (φ(n)=n), φ-mixing (n), exponential α-mixing (n/(log n)^2), and geometrically C-mixing (n/(log n)^{2/ϱ}) processes. With r unknown, a two-step pre-training estimates r from unlabeled source and target samples and achieves the same rate for large enough m. For unbounded r with finite moments of any order, truncating the ratio","pith_inferences":["The φ(n) abstraction is a plug-in principle: any dependence model that admits a Bernstein inequality of the stated form automatically inherits these rates, so the theory likely extends to non-standard settings once such inequalities are available.","The precise threshold prescription for the truncated ratio could inform practice in other importance-weighting problems (e.g., off-policy evaluation), where clipping is a common but ad hoc heuristic.","A natural experiment would simulate an AR(1) or TAR process with known α-mixing coefficients, induce a covariate shift, and check whether empirical L2(Q) errors track φ(n)^{-2s/(2s+d)} with the predicted log-powers; this would calibrate how tight the logarithmic factors really are.","If the smoothness class were replaced by composition-structured or piecewise-smooth functions, the architecture choices would change but the φ(n)-based error decomposition should remain the template."],"forward_implications":["For any process satisfying (A1), the reweighted SPDNN estimator adaptively attains the minimax rate φ(n)^{-2s/(2s+d)} up to log factors, without knowing the smoothness s.","The unweighted estimator remains minimax optimal when the density ratio is bounded; under only a second moment it degrades to φ(n)^{-s/(2s+d)}, showing reweighting is essential.","Truncating the density ratio at a data-dependent threshold extends optimality to unbounded ratios with moments of any order.","The two-step pre-training estimator splits its error into a density-ratio estimation term (m unlabeled samples) plus a regression term (n labeled samples); for large enough m the regression rate dominates.","The same rates hold, up to log factors, for i.i.d., φ-mixing, exponential α-mixing, and geometrically C-mixing processes, so independence and weak dependence are unified."],"fun_headline_variants":["Deep nets hit minimax rate under covariate shift and dependence","Sparse deep nets hit minimax rate with covariate shift and mixing data","Adaptive deep regression for covariate shift and dependent data","Reweighted deep nets attain minimax rate in covariate shift and mixing","Deep nets with density-ratio reweighting optimal for dependent covariate shift"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper's dependent-data results hold only if the process satisfies Assumption (A1), a uniform Bernstein inequality with an effective number of observations φ(n) ≤ n; if dependence is too strong for such an inequality to hold, the claimed φ(n)-rates do not follow.","fun_headline_variants_meta":{"raw":{"variants":["Deep nets hit minimax rate under covariate shift and dependence","Sparse deep nets hit minimax rate with covariate shift and mixing data","Adaptive deep regression for covariate shift and dependent data","Reweighted deep nets attain minimax rate in covariate shift and mixing","Deep nets with density-ratio reweighting optimal for dependent covariate shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000637,"raw_usage":{"total_tokens":2818,"prompt_tokens":834,"completion_tokens":1984,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":1900}},"tokens_in":578,"tokens_out":1984,"duration_ms":13077,"temperature":1.0,"reasoning_tokens":1900,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T10:10:35.455073+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a stationary ergodic process with long memory (e.g., fractional Gaussian noise with Hurst parameter > 1/2) for which the generalized Bernstein inequality in (A1) is believed to fail, fit the reweighted SPDNN estimator to simulated data under a known covariate shift, and compare the empirical L2(Q_X0) error with the predicted φ(n)^{-2s/(2s+d)} decay. A decay strictly slower than the theory prescribes, or a failure of the tail bound for a simple bounded test function, would show the assumptions are tighter than stated.","supporting_citations":[],"review_version":1}