{"id":"0fb2964f-cb3a-4991-b520-bc1380348f2f","arxiv_id":"2506.17880","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"For indirect elicitation with parametric assumptions, the target estimate changes monotonically with each sub-loss weight when the sub-property trajectory is one-sided and the link function is monotone along it; in 2-D, derivative conditions determine when the best weight is 0, infinity, or…","lead":"This paper studies how the weights assigned to sub-scores in a weighted loss affect the parametric estimate of a target property such as variance or skewness. It gives conditions under which the estimate moves monotonically with each weight, and when the best weight is zero or infinite.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 6.3 overclaims 'for all p': its proof assumes R^{-1}(rhat2)<rhat1 as WLOG, but this is a real domain restriction; when it fails the derivative comparison can predict c*_1=0 while the true c*_1 is finite.","rationale":"The reader's CONDITIONAL verdict is appropriate. I agree with the reader that global monotonicity and sign assumptions are load-bearing, but the failure is sharper: the proof of Theorem 6.3 contains a 'without loss of generality' ordering assumption that is actually a substantive restriction on p. The concrete p above satisfies the derivative comparison on the model domain and is a valid distribution, yet the predicted 0/infinity optimality fails. This does not invalidate the framework or the empirical monotonicity observed near the model curve, but it means the stated theorem and the 'perfectly explains' claim need qualification. The proposed check is simple and decisive. Credit is due to the useful decomposition in Theorem 5.2, the careful simulation study, and the paper's transparency about incomplete higher-dimensional theory; the issue is in the 2D universal quantifier, not the overall program.","tokens_in":27851,"tokens_out":27291,"duration_ms":276783,"concrete_test":"Run the one-dimensional minimization for the counterexample with c2 fixed at 1: minimize L_{c1}(theta)=c1(theta+1)^2+(theta^2+theta-1.5)^2 over theta>0 for c1=0,1,10,infinity. Verify theta*(0) approx 0.823, theta*(1)=0.5, theta*(infinity) tends to 0, and hence gamma=theta moves 0.823 -> 0.5 -> 0, giving c*_1 approx 1, while Theorem 6.3(b) with R'=1+2r1 > 2r1 = T' predicts c*_1=0. This isolates the missing R^{-1}(rhat2)<rhat1 condition.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest load-bearing point is the hidden quadrant/domain assumption in Theorem 6.3. In Appendix B the proof says \"Without loss of generality, assume that R^{-1}(rhat2)<rhat1\"; this is not WLOG. The conclusion uses r_A=(R^{-1}(rhat2),rhat2) and r_B=(rhat1,R(rhat1)) as the c1=0 and c1=infinity endpoints, so both axis intersections must exist and be ordered. For a valid p where rhat1 lies outside the r1-domain of R, r_B does not exist and the endpoint is a boundary limit; the ordering argument in cases (a)-(c) collapses. Concretely, take t(r)=r2-r1^2, R(r1)=r1+r1^2 on r1>0, and p=N(-1,0.5), so rhat=(-1,1.5), Gamma=0.5, and rhat is not in R_Theta. On the model domain R'=1+2r1 > 2r1 = T', so Theorem 6.3(b) would give c*_1=0. But minimizing L_{c1}(theta)=c1(theta+1)^2+(theta^2+theta-1.5)^2 gives theta*_0=(-1+sqrt(7))/2 approx 0.823, theta*_1=0.5, and theta*_infinity to 0; hence gamma(theta*) goes 0.823 -> 0.5 -> 0 as c1 increases, and c*_1 is finite (about 1), not 0. The theorem needs an explicit condition such as rhat1 in the domain of R and R^{-1}(rhat2)<rhat1, and the global monotone-contour assumption (ii) must be stated on the relevant domain.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the problem of choosing weights in a fully separable weighted sum of proper loss functions for indirect elicitation of a target property under a parametric model. For a target Gamma(p)=t(rhat(p)) with directly-elicited sub-properties rhat and a parametric sub-property curve r(theta), the minimizer theta*_c(p) depends on the weights c, and the paper studies how gamma(theta*_c(p)) changes with each c_i and which weight is best. The authors first report simulation evidence that weight trajectories are usually monotone and that optimal weights are often 0 or infinity. They then give an elementary decomposition (Theorem 5.2) into monotonicity of the sub-property trajectory and monotonicity of the link function, and provide 2-D sufficient conditions comparing the slope of the model curve R(r1) with the slope of the target contours T(r1;t0) (Theorem 6.3). For higher dimensions they prove a slice-wise monotonicity condition for the trajectory (Theorem 7.1) and treat linear cases (Theorem 7.2). The variance and skewness simulation studies are used to support the claimed empirical pattern.","tokens_in":28253,"tokens_out":7307,"duration_ms":75297,"significance":"If the main results were correct as stated, the paper would make a novel contribution to the sparse literature on choosing among proper scoring rules: it identifies a concrete geometric condition under which weight choice in indirect elicitation is determined by a slope comparison between the model curve and the target contour. The decomposition in Section 5 is elegant and potentially reusable, and the observation that boundary weights are often optimal is practically relevant. The paper is also commendably explicit about several limitations, including the incompleteness of the higher-dimensional theory and the assumed linearity of high-dimensional trajectories. However, the central 2-D theorem is stated for all p but its proof requires domain restrictions that are not part of the assumptions; this is a load-bearing gap rather than a presentational issue.","major_comments":[{"comment":"The proof uses 'Without loss of generality, assume that R^{-1}(rhat_2)<rhat_1' and then defines the endpoints r_A=(R^{-1}(rhat_2), rhat_2) and r_B=(rhat_1, R(rhat_1)) as the points reached as c1 approaches 0 and infinity. This is not WLOG: the assumptions of Theorem 6.3 do not imply that rhat_1 lies in the domain of R or that rhat_2 lies in the range of R, and they do not imply the required ordering. Since Problem 1 explicitly allows rhat outside R_Theta, the theorem's claim 'for all p' is unsupported. A concrete counterexample is t(r)=r2-r1^2, R(r1)=r1+r1^2 on r1>0, and p=N(-1,0.5), giving rhat=(-1,1.5). Here R'(r1)=1+2r1 > 2r1 = T'(r1;t0), so case (b) would predict c*_1=0, but minimizing c1(theta+1)^2 + (theta^2+theta-1.5)^2 over theta>0 gives gamma(theta*_0)≈0.823, gamma(theta*_1)≈0.5, and gamma(theta*_infinity)≈0, so the best weight is finite and close to 1. The theorem needs an explicit condition such as rhat_1 in the domain of R and R^{-1}(rhat_2)<rhat_1, or the statement must be restricted to p satisfying that condition.","section":"Appendix B, proof of Theorem 6.3"},{"comment":"The global monotone-contour assumption is not satisfied by the paper's own main example t(r)=r2-r1^2 on its full domain. Lemma B.1(2) requires the signs of partial derivatives to be unchanged over the whole space, but for this link function dt/dr1 = -2r1 changes sign, and the contour r2=r1^2+t0 is not a globally monotone function of r1. The paper applies the variance-link theory in Section 6.4 only on restricted positive-orthant regions, but Theorem 6.2 and Theorem 6.3 are stated with 'for all r1 and t0' and 'over the whole space.' This mismatch means the 'for all p' formulation of the 2-D result is not justified by the assumptions. The authors should either state the domain restriction explicitly in the theorems or reformulate the monotone-contour condition on the relevant domain of the model curve.","section":"Section 6.2, Lemma B.1 and Theorem 6.2"},{"comment":"The induction proof of Theorem 7.1 rests on the claim that the intercepts epsilon'_k(r'_j) of each slice keep the same sign as epsilon_k for all r'_j below the axis intersection. This is the crux of the induction step, but the proof only says 'We can verify' and does not provide the verification. Given that the slice is merely strictly monotone, the sign preservation is not immediate and may require additional assumptions about how the slices vary with r'_j. The higher-dimensional condition (A) is therefore not fully proven as written. The later linear-case theorem (Theorem 7.2) also assumes linearity of the trajectories T_ci(p), which the paper explicitly states is not known to follow from linearity of r(theta); this should be clearly labeled as a conditional result rather than a theorem about the linear model alone.","section":"Section 7.1 and Appendix D.1, Theorem 7.1"}],"minor_comments":[{"comment":"The statement 'the choice of sub-losses does not affect our observation and conclusions' is too strong: the theorems require sub-losses to be accuracy-rewarding, and the simulation evidence is limited to quadratic losses. A more guarded statement would be more accurate.","section":"Section 3, Problem 1"},{"comment":"The sentence 'In fact, the setting of c_{-i} does not matter for our empirical observations and theoretical results' is contradicted for M>2 by Remark 3, which notes that the normalization argument only applies in the 2-D case. Please qualify this claim.","section":"Section 4"},{"comment":"There is a duplicated word in the introduction: 'there has been been more and more publications' should read 'there have been more and more publications.'","section":"Section 2.1"},{"comment":"The definition of 'one-sided from \\tilde r_i' introduces a new symbol \\tilde r, but the subsequent theorems use \\hat r(p). Please make the notation consistent and specify which r is meant in each result.","section":"Definition 5.2"},{"comment":"The reported skewness values for the log-normal examples are negative in the first block of Table 3, which is unexpected for log-normal models and mixtures of log-normals. Please check whether these are typos or whether a different sign convention is being used.","section":"Appendix F, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The 2-D theory is potentially salvageable by adding the domain conditions that the counterexample shows are necessary. I therefore do not recommend rejection, but the manuscript as submitted overclaims the scope of Theorem 6.3. The authors should also decide whether the finite-sample simulation claims are meant to illustrate the theory (which is about a fixed p with known moments) or to make claims about empirical risk minimization; at present the relationship between the empirical phat and the theoretical p is not discussed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has a genuinely new problem and a theorem that is currently overclaimed. The two-step decomposition (Lemma 5.1 + Theorem 5.2) is clean and likely correct. But Theorem 6.3, the main 2-D payoff, has a hidden domain assumption that the proof calls \"WLOG\" and that is not WLOG at all. On reading the appendix, the proof assumes R^{-1}(rhat2) < rhat1 and uses points r_A and r_B as the c1=0 and c1=infinity endpoints. If rhat1 is outside the domain of R, r_B does not exist and the ordering arguments collapse. A concrete example: t(r)=r2-r1^2, R(r1)=r1+r1^2 on r1>0, and p=N(-1,0.5). Theorem 6.3(b) predicts c*_1=0, but direct minimization gives gamma(theta*) moving 0.823 -> 0.5 -> 0 as c1 increases, so the true c*_1 is finite. The theorem needs an explicit condition like rhat1 in the domain of R and R^{-1}(rhat2)<rhat1. The global monotone-contour assumption is also overly strong for the paper's own variance example, which only has monotone contours in a restricted region.\n\nWhat is good: the problem statement is new, the simulations motivate the theory, and the decomposition into 'sub-property movement' and 'link monotonicity' is a useful way to think about weight selection. The paper is transparent that higher-dimensional theory is incomplete and that the renormalization trick is practical. The linear case (Theorem 7.2) is simple but fine.\n\nThe soft spots are proportional: the WLOG issue is load-bearing and makes Theorem 6.3 false as stated; the proof of Theorem 6.1 is a sketch with a domination argument; the simulations have no error bars and the renormalization is data-dependent, though disclosed. None of these sink the core idea, but they require a revision before the 2-D claims are trustworthy.\n\nWho is this for? Researchers in property elicitation and scoring rules. It would get a serious referee, not a desk reject, because the problem is real and the counterexample points to a fix rather than a dead end. I would send it to review, but I would not cite the 2-D theorem in its current form.","headline":"New problem and a sound decomposition, but Theorem 6.3 overclaims: its proof hides a domain restriction that makes the 2-D optimality results false in natural cases.","tokens_in":28820,"tokens_out":4091,"would_cite":false,"duration_ms":35765,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that in two-dimensional indirect elicitation with a parametric model, the optimal scoring-rule weight is decided by a single global comparison of the model curve's slope with the slope of the target property's contour…","keywords":["proper scoring rules","indirect elicitation","parametric model estimation","weight selection","variance elicitation","accuracy-rewarding losses","monotone trajectories","elicitation complexity"],"falsifier":"Build a two-subproperty example that satisfies every condition of Theorem 6.3 - accuracy-rewarding losses, a smooth strictly increasing model curve $R$, differentiable monotone contours $T$ with $0<R'(r_1)<T'(r_1;t_0)$ throughout - choose a true point $\\hat r$ off the curve, numerically minimise $c_1L_1+c_2L_2$ over the model for a sweep of $c_1$, and check whether $\\gamma(\\theta^*_c)$ monotonically approaches $\\Gamma(p)$ with $c_1^*=+\\infty$; any interior optimum or non-monotone approach would show the theorem's conclusion fails under its own hypotheses.","tokens_in":27582,"feed_emoji":"🎯","tokens_out":13737,"duration_ms":128776,"temperature":0.7,"pith_summary":"People often want to forecast an indirect statistical quantity such as variance, which is not directly elicitable but can be obtained from directly elicitable subproperties like the first two moments. When the forecast is restricted to a parametric family, changing the weights in a weighted sum of proper scoring losses changes the fitted subproperties and hence the estimated target. This paper argues that in the two-subproperty case the whole effect is governed by one geometric comparison: the slope of the parametric model curve against the slope of the target property's contour lines. If that comparison keeps a fixed sign, the estimated target moves monotonically as the weight grows and the best weight is an extreme value (zero or infinity in the limit); if the slopes oppose, an interior best weight exists. The claim matters because it turns a seemingly arbitrary choice of scoring-rule weights into a checkable structural condition on the model and the target.","feed_headline":"A slope comparison decides the best scoring-rule weights","feed_subtitle":"When slopes stay ordered, the best weight is 0 or infinity; opposing slopes give a finite optimum.","key_machinery":"The load-bearing object is the derivative comparison between the model curve $R(r_1)$ and the target-contour function $T(r_1;t_0)$, expressed by the sign of $R'(r_1)-T'(r_1;t_0)$ at their intersections. The proof route is a two-step decomposition: Lemma 5.1 shows that increasing a weight $c_1$ can only lower that sub-loss at the minimizer, and with accuracy-rewarding losses this makes the fitted subproperty move monotonically toward the true value along the model curve. Theorem 6.2 shows that the target link $t$ is monotone along the model curve if and only if the slope difference keeps a constant sign, provided the target contours are differentiable and monotone. Theorem 6.3 combines these into the exhaustive zero/infinity/interior prescription for $c_1^*$.","core_discovery":"The central discovery, stated as Theorem 6.3, is that under its assumptions - a differentiable strictly monotone parametric model curve, differentiable monotone target contours, and accuracy-rewarding sub-losses - the two-subproperty weight-selection problem collapses into one global inequality. Write the parametric subproperty model as a curve $r_2=R(r_1)$ and write the level sets of the target link $t$ as $r_2=T(r_1;t_0)$. If $R'(r_1)$ and $T'(r_1;t_0)$ have the same sign everywhere and $R'(r_1)<T'(r_1;t_0)$, then increasing $c_1$ always moves the estimated target $\\gamma(\\theta^*_c(p))$ closer to the true $\\Gamma(p)$, so the best weight is $c_1^*=+\\infty$; the reverse inequality gives $c_1^*=0$. If the two derivatives have opposite signs everywhere, the estimate first moves closer and then farther away, giving $c_1^*\\in(0,+\\infty)$. The mechanism is a decomposition: increasing $c_1$ improves the estimate of $r_1$ monotonically, and the target changes monotonically along the model curve exactly when $R'(r_1)-T'(r_1;t_0)$ keeps one sign. Thus, under the theorem's conditions, the choice among weights is not a matter of taste but a structural property of the model curve relative to the target's contours.","pith_inferences":["A testable extension the paper does not run: construct a model where the slope ordering holds only in a local region and force the fitted trajectory to cross the sign-change boundary; the target-response curve should acquire a kink or reversal, which would delimit how far the global theorem extends.","The paper does not draw this conclusion, but the extreme-weight result suggests that a fully separable weighted loss is not a neutral tool for target estimation in the same-sign regimes: it wins by ignoring one of the subproperties entirely, so coupling the sublosses or choosing a different functional form might do better than any weight setting.","An implicit consequence for multi-objective loss design: the slope comparison can serve as a diagnostic for which objective to emphasize - put more weight on the coordinate along which the model curve cuts most steeply across the target's contours - which generalises beyond elicitation to any weighted composite loss."],"forward_implications":["In any two-subproperty model meeting the theorem's conditions, the optimal weight can be read off from a global slope inequality, so the choice requires no numerical search over weights.","Equal or balanced weights are generally not optimal in the same-sign regimes: the best configuration discards one subproperty or concentrates all weight on it, so the common default of equal weights can be systematically inferior.","When the slope signs are opposite, a finite interior best weight exists, but the true target value may still be unreachable at that optimum, as the paper notes.","In higher dimensions the same monotonicity pattern holds for linear models, linear links, and linear trajectories, and nonlinear settings can be studied locally by linear approximation; this helps explain why simulations show monotonicity across different distribution families."],"supporting_citations":[{"why":"Supplies the definition of strictly proper scoring rules and the convex characterisation that the weighted-sum loss framework builds on.","marker":"Gneiting and Raftery, 2007"},{"why":"Defines accuracy-rewarding scoring rules, the property that drives Lemma 5.1 and the one-sided trajectory conclusion.","marker":"Lambert et al., 2008"},{"why":"Provides the classical characterisation of proper scoring rules for probabilities and expectations, the basis for taking means and moments as directly elicitable subproperties.","marker":"Savage, 1971"},{"why":"Gives the quadratic loss for linear properties used as the sub-losses in the simulation studies.","marker":"Abernethy and Frongillo, 2012"},{"why":"Establishes that variance is not directly elicitable, which motivates the indirect route through eliciting the first two moments.","marker":"Osband, 1985"},{"why":"Frames the complexity of indirect elicitation and shows why a target property can be elicited through simpler subproperties, placing the paper's framework in that line.","marker":"Frongillo and Kash, 2021"},{"why":"Gives conditions under which proper scoring rules are accuracy-rewarding, supporting the blanket accuracy-rewarding assumption in Theorem 6.3.","marker":"Steinwart et al., 2014"}],"fun_headline_variants":["Slope sign picks optimal scoring weight","Monotone slopes force weight to 0 or infinity","Opposing slopes give finite best weight","Slope comparison picks extreme weight"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the assumption that, across the entire region the fitted trajectory can visit, the model curve keeps a consistently ordered slope relative to the target property's contour lines, and that those contours are differentiable and monotone; if that slope ordering reverses anywhere, the monotonicity and zero-or-infinity conclusions are not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["Slope sign picks optimal scoring weight","Monotone slopes force weight to 0 or infinity","Opposing slopes give finite best weight","Slope comparison picks extreme weight"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000557,"raw_usage":{"total_tokens":2761,"prompt_tokens":1166,"completion_tokens":1595,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":782,"completion_tokens_details":{"reasoning_tokens":1553}},"tokens_in":782,"tokens_out":1595,"duration_ms":11915,"temperature":1.0,"reasoning_tokens":1553,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:59:45.636162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a two-subproperty example that satisfies every condition of Theorem 6.3 - accuracy-rewarding losses, a smooth strictly increasing model curve $R$, differentiable monotone contours $T$ with $0<R'(r_1)<T'(r_1;t_0)$ throughout - choose a true point $\\hat r$ off the curve, numerically minimise $c_1L_1+c_2L_2$ over the model for a sweep of $c_1$, and check whether $\\gamma(\\theta^*_c)$ monotonically approaches $\\Gamma(p)$ with $c_1^*=+\\infty$; any interior optimum or non-monotone approach would show the theorem's conclusion fails under its own hypotheses.","supporting_citations":[{"cited_title":"A characterization of scoring rules for linear properties","cited_arxiv_id":null,"evidence_quote":"Gives the quadratic loss for linear properties used as the sub-losses in the simulation studies."},{"cited_title":"Providing Incentives for Better Cost Forecasting (Prediction, Uncertainty Elicitation)","cited_arxiv_id":null,"evidence_quote":"Establishes that variance is not directly elicitable, which motivates the indirect route through eliciting the first two moments."},{"cited_title":"Elicitation complexity of statistical properties","cited_arxiv_id":null,"evidence_quote":"Frames the complexity of indirect elicitation and shows why a target property can be elicited through simpler subproperties, placing the paper's framework in that line."},{"cited_title":"Elicitation and identification of properties","cited_arxiv_id":null,"evidence_quote":"Gives conditions under which proper scoring rules are accuracy-rewarding, supporting the blanket accuracy-rewarding assumption in Theorem 6.3."}],"review_version":2}