{"id":"37499d9c-98c8-43d3-90ed-8e9585904645","arxiv_id":"2412.14916","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A benchmark of ten gradient boosting algorithms on five insurance datasets shows probabilistic versions can improve fit without losing predictive accuracy, with LightGBM and XGBoostLSS fastest.","lead":"This paper surveys ten gradient boosting algorithms for insurance claim frequency and severity, then runs them on five public datasets. It finds that probabilistic boosting can improve model adequacy without sacrificing predictive accuracy, and that LightGBM and XGBoostLSS are fastest.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 'no trade-off' claim is contradicted by the paper's own NB2 results (Table 9/16) and by the lognormal BelgianMTPL results (Table 7), where the best-coverage probabilistic models have negative pseudo-R2.","rationale":"Read in good faith, this is a genuinely useful benchmark: the unified notation in Algorithms 1–6 is a real contribution, the exposure-as-offset explanation for frequency boosting is clear, and the body is honest enough to report the NB2 adequacy results that point against the headline (Section 5.3) and to note why gamma models show no adequacy gain. The problem is that the abstract's central claim is not scoped. Separating the four settings, the evidence base for 'no trade-off' collapses to essentially the lognormal severity block. Poisson trivially equates point and probabilistic predictions because they share the same mean parameter, so it cannot test the claim; gamma shows no adequacy improvement from multi-parametric boosting, so the trade-off question is moot there; NB2 shows probabilistic methods worsening adequacy, directly contrary to the claim; and in the lognormal block itself, the models that most improve coverage (XGBoostLSS, cyc-GBM on BelgianMTPL) have negative pseudo-R2, a visible accuracy loss. Only the existential reading—NGBoost achieves both on some lognormal datasets—is defensible, and that is weaker than the flat 'we find there is no trade-off.' The reader's weakest assumption (single split, no error bars) is real and compounds this: many rank-deciding pseudo-R2 gaps are under one percentage point, so the 'no hindering' leg is currently unverifiable. I partially agree with the reader because the sharper defect is the internal contradiction between the abstract's scope and the paper's own tables, not merely under-sampling; nevertheless both diagnoses point to the same remedy: scope the claim per distribution and add repeated-split uncertainty. That remedy is consistent with the reader's CONDITIONAL verdict, so the verdict need not change.","tokens_in":32314,"tokens_out":7291,"duration_ms":64940,"concrete_test":"Run 20 stratified re-splits of BelgianMTPL, freMTPL, and pg15training following Algorithm 7's protocol (identical tuning grids and packages), fitting LightGBM, XGBoostLSS, and NGBoost for the lognormal severity and NB2 frequency settings, and report CI coverage, uniform CRPS, and McFadden pseudo-R2 with bootstrap 95% confidence intervals. If the NB2 probabilistic CRPS ranks remain above the point-algorithm ranks, and if XGBoostLSS's negative BelgianMTPL lognormal pseudo-R2 persists outside the confidence band while its coverage gains are confirmed, then the 'no trade-off' statement must be restricted to the lognormal setting (or to specific algorithms) in the revised claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion—'there is no trade-off between model adequacy and predictive accuracy: both are achievable simultaneously'—is stated in the abstract without scope, but the paper's own tables contradict it in two of the four distributional settings. In the NB2 frequency setting, probabilistic algorithms are on average less adequate than point algorithms using a global dispersion parameter: average CRPS ranks in Table 9 are 7.4–9.2 for XGBoostLSS, NGBoost, cyc-GBM, and XGBoostLSSd versus 2.6–3.6 for LightGBM, XGBoost, and CatBoost, and the Table 16 values agree on all five datasets. The body acknowledges this ('allowing the shape parameter φi to vary with xi hinders model adequacy, at least on all the studied datasets,' Section 5.3), so the unqualified abstract claim contradicts the body's own finding in the frequency setting. In the lognormal severity setting, the probabilistic models with the best CI coverage on BelgianMTPL (Figure 3, Tables 19–23) are XGBoostLSS and cyc-GBM, yet their McFadden pseudo-R2 on the same test split is −0.33 and −0.50 (Table 7), worse than the constant-mean benchmark—a direct instance of the trade-off the claim denies. What survives is a weaker existential statement: NGBoost achieves near-nominal coverage with competitive R2 on lognormal datasets. Compounding this, Algorithm 7 uses one 85/15 split per dataset with no repeated runs, so the 'without hindering predictive performance' leg rests on average-rank gaps that are often under one percentage point of pseudo-R2 (e.g., Table 5 freMTPL: 24.92 vs 24.87; Table 7 WorkComp: 0.44 vs 0.51) and cannot currently be distinguished from sampling noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper surveys point and probabilistic gradient boosting for decision trees (GBM, XGBoost, DART, LightGBM, CatBoost, EGBM, PGBM, XGBoostLSS, cyc-GBM, NGBoost) in a unified notation, and compares them on five public claim-frequency and five claim-severity datasets. The numerical study evaluates computational efficiency, McFadden pseudo-R2, CRPS, confidence-interval coverage, portfolio balance, auto-calibration, and tariff segmentation. The central claim, stated in the abstract and again in Section 6, is that there is no trade-off between model adequacy and predictive accuracy: both are achievable simultaneously.","tokens_in":32605,"tokens_out":3121,"duration_ms":26494,"significance":"If the central claim were correct and appropriately scoped, the paper would give practitioners a clear reason to prefer probabilistic boosting when calibrated predictive distributions matter. The unified survey of ten GBDT variants is a useful reference, and the benchmark design has real strengths: separate validation and test splits, identical data partitions across algorithms, five datasets of varied size and cardinality, and multiple evaluation metrics including proper scoring rules and CI coverage. The paper also makes a useful practical point about treating exposure offsets in frequency boosting. However, the headline no-trade-off claim is not supported by the paper's own tables, and the single-split design leaves the key comparative statements without uncertainty quantification. The manuscript is valuable as a comparative study, but its central claim needs to be substantially qualified and re-tested before it can be accepted as stated.","major_comments":[{"comment":"The unqualified claim that 'there is no trade-off between model adequacy and predictive accuracy' is contradicted by the NB2 frequency results. Table 9 gives average CRPS ranks of 8.2, 8.4, 9.2, and 7.4 for XGBoostLSS, NGBoost, cyc-GBM, and XGBoostLSSd, versus 2.6, 2.8, and 3.6 for LightGBM, XGBoost, and CatBoost; Table 16 shows the same ordering on all five datasets. The body itself acknowledges in Section 5.3 that 'allowing the shape parameter phi_i to vary with x_i hinders model adequacy, at least on all the studied datasets.' The abstract and Section 6 must be rewritten to state a scoped version of the claim, e.g., one that specifies the distributional settings and algorithms for which adequacy and accuracy are simultaneously achieved, and that explicitly records the NB2 exception.","section":"Abstract and Section 5.3 (Tables 9 and 16)"},{"comment":"The lognormal BelgianMTPL results are a second direct counterexample to the no-trade-off claim. Table 7 reports McFadden pseudo-R2 values of -0.33 for XGBoostLSS and -0.50 for cyc-GBM, worse than the constant-mean benchmark (GLM 0.27, XGBoost 0.51), while Table 19 shows that these same probabilistic models have the best CI coverage among the GBDT models (e.g., XGBoostLSS coverage 27.90%, 65.62%, 95.33% at the 50%, 75%, 95% levels). This is precisely a trade-off between adequacy and predictive accuracy. The paper needs to either drop the global no-trade-off claim or demonstrate, with additional evidence, why these negative pseudo-R2 values should not count as hindered predictive performance.","section":"Section 5.2 and 5.3 (Tables 7 and 19)"},{"comment":"The comparative conclusions rest on a single random 85/15 split per dataset, with no repeated runs, no standard errors, and no paired significance tests. Algorithm 7 samples one Dtrain,k and one Dtest,k per dataset, and the average ranks in Tables 5-9 are point estimates. Many of the reported differences are small; for example, Table 5 shows Poisson freMTPL pseudo-R2 of 24.92 for LightGBM and 24.84 for XGBoostLSS, and Table 8 shows WorkComp gamma pseudo-R2 of 33.75 for NGBoost and 32.14 for XGBoost. Without uncertainty information, the statement that probabilistic algorithms do not 'hinder' predictive performance is not established. I recommend adding repeated splits or bootstrap/paired tests, or at minimum reporting standard errors over several seeds, before the comparative claims can be taken as quantitative.","section":"Appendix A (Algorithm 7) and Section 5.2 (Tables 5-9)"},{"comment":"The discussion paragraph states that 'model adequacy may be enhanced by using probabilistic GBDT without hurting much predictive performance.' This weaker statement is closer to the data, but it still needs a scope qualifier. The paper's own results show that in the NB2 setting the probabilistic algorithms hurt adequacy, and in the lognormal BelgianMTPL setting the best-coverage models have negative pseudo-R2. The discussion should be revised to summarize which algorithms and distributions support the 'no trade-off' reading, rather than presenting it as a general finding.","section":"Section 6 (Discussion)"}],"minor_comments":[{"comment":"The column header 'Avg. ranl' appears to be a typo for 'Avg. rank'; please fix.","section":"Table 17"},{"comment":"The text says 'cyc-GBM looses' where 'loses' is intended; please correct the typo.","section":"Section 5.2"},{"comment":"The entry for XGBoostLSSd on freMPL reads '2 .27' with an extra space; please correct the formatting.","section":"Table 10"},{"comment":"The paper does not state whether code and trained-model configurations are publicly available. Since Algorithm 7 defines the exact split procedure, a code repository would substantially help reproducibility; please add a data/code availability statement.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The survey component is solid and the benchmark is carefully designed in many respects, but the headline claim currently overreaches relative to the paper's own tables. The manuscript would be suitable for publication after the central claim is scoped to specific settings, the NB2 and lognormal BelgianMTPL counterexamples are explicitly addressed, and the single-split uncertainty issue is either resolved or clearly acknowledged as a limitation. I would not recommend rejection: the core comparative material is useful and the issues are fixable within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this as a useful benchmark, not as evidence for the no-trade-off claim. The paper does a real service: it puts ten GBDT algorithms (point and probabilistic) in unified notation, explains exposure offsets for frequency models, and evaluates them on five public datasets across efficiency, predictive performance, adequacy, calibration, balance, and tariff structure. The practical takeaways are mostly credible: LightGBM is fastest, CatBoost helps with high-cardinality categoricals, EGBM is competitive while interpretable, XGBoostLSS is the efficient probabilistic option. The use of CRPS, DPIT residuals, Murphy diagrams, and auto-calibration tests is a cut above typical benchmark papers.\n\nThe soft spot is the abstract's headline: 'there is no trade-off between model adequacy and predictive accuracy.' The paper's own results contradict that in two settings. In the NB2 frequency models, the probabilistic algorithms with covariate-varying dispersion are less adequate (Table 9/16: CRPS ranks 7.4–9.2 vs 2.6–3.6 for point algorithms with a global dispersion), and the body admits it. On BelgianMTPL lognormal severity, the best-coverage probabilistic models (XGBoostLSS, cyc-GBM) have negative pseudo-R2 of –0.33 and –0.50, worse than the constant-mean benchmark. What survives is a weaker, still interesting claim: NGBoost can give near-nominal coverage on lognormal data without sacrificing predictive accuracy. The abstract should say that.\n\nSecond soft spot: one random 85/15 split per dataset, no repeated runs, no error bars. Some average-rank gaps are under a percentage point of pseudo-R2 (e.g., 24.92 vs 24.87 in Table 5); with a single split those are sampling noise. The paper should supply seeds, code, and ideally repeated splits or confidence intervals. The Gini index analysis and calibration tests are a nice addition, and the comparison with GLM/GAMLSS anchors the results.\n\nThis is a paper for actuaries choosing a GBDT implementation and for researchers wanting a common ground to compare probabilistic boosting methods. It deserves a serious referee: the benchmark is reproducible in design and the unified notation is a contribution. But the authors should be asked to soften the abstract and to either add repeated evaluations or at least be explicit that rankings are point estimates from one split.","headline":"Useful benchmark of ten GBDT algorithms for insurance data, but the abstract's 'no trade-off' claim outruns the evidence—the body's own NB2 and lognormal results show real trade-offs.","tokens_in":33220,"tokens_out":3724,"would_cite":true,"duration_ms":27635,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62P05","62-08"],"pacs":[],"model":"deepseek-v4-flash","headline":"Probabilistic gradient boosting can match point-prediction accuracy while improving model adequacy, so insurers do not have to trade off the two.","keywords":["gradient boosting","decision trees","probabilistic prediction","model adequacy","CRPS","claim frequency","claim severity","insurance pricing"],"falsifier":"Re-run every algorithm on 50 random 85/15 splits of the same five datasets and record the pseudo-$R^2$ and CRPS average ranks; if the point-versus-probabilistic ordering flips across splits or the adequacy gaps shrink to within the noise of the splits, the no-trade-off claim would not hold.","tokens_in":32040,"feed_emoji":"📈","tokens_out":8655,"duration_ms":62922,"temperature":0.7,"pith_summary":"The paper asks whether actuarial claim frequency and severity models can get the benefits of probabilistic prediction -- a full predictive distribution per policy, not just a point forecast -- without sacrificing the predictive accuracy that makes gradient boosting attractive. It reviews the point and probabilistic GBDT algorithms in a single notation and benchmarks them on five public insurance datasets. Its central finding is that model adequacy and predictive accuracy do not trade off: probabilistic algorithms match point algorithms on deviance-based pseudo-$R^2$, while scoring better on CRPS and confidence-interval coverage, most clearly for lognormal severity. It also reports that LightGBM is the most efficient point model, XGBoostLSS the most efficient probabilistic model, CatBoost the strongest with high-cardinality categoricals, and EGBM the interpretable option at competitive accuracy.","feed_headline":"No trade-off: probabilistic boosting matches point-forecast accuracy","feed_subtitle":"On five claim frequency and severity datasets, probabilistic boosting improves distributional fit without lowering point accuracy.","key_machinery":"The central object is the multi-parameter probabilistic GBDT: instead of a single tree sequence for the conditional mean, the algorithm fits $\\kappa$ sequences of decision trees, one for each parameter of an assumed distribution (e.g., location and scale for lognormal, mean and dispersion for negative binomial), each with its own link function $\\varphi$ to keep predictions in the valid domain, and minimises the negative log-likelihood. XGBoostLSS cycles Newton steps over parameters, NGBoost uses the natural gradient to make updates invariant to reparametrisation, and cyc-GBM uses cyclic coordinate descent; the paper's unified notation is what lets the algorithms be compared fairly. Varying exposure-to-risk is incorporated in frequency models through an offset $\\ln e_i$ in the initial prediction, and model adequacy is operationalised with CRPS, DPIT residuals, and confidence-interval coverage.","core_discovery":"The central claim is that there is no trade-off between model adequacy and predictive accuracy in gradient boosting for claims. On the paper's own terms, a practitioner can switch from point-prediction GBDTs to probabilistic GBDTs and obtain predictive distributions whose CRPS and confidence-interval coverage are better or equal, while deviance-based McFadden pseudo-$R^2$ stays essentially unchanged. The evidence is the average ranks across five datasets for Poisson and negative-binomial frequency models and lognormal and gamma severity models, with probabilistic methods winning the adequacy rankings in the lognormal case and point methods holding their ground in predictive performance. The paper presents this as a reconciliation of the 'data modelling' and 'algorithmic modelling' cultures in actuarial science.","pith_inferences":["Beyond the paper: the single-split design means the no-trade-off claim would be stronger if it survives repeated resampling; the reported average ranks have no stated uncertainty.","Beyond the paper: the pattern suggests probabilistic methods will help most where the point loss is location-only, as in the lognormal case, because a gamma deviance already modifies the shape of the fitted distribution.","Beyond the paper: combining EGBM's interpretability with probabilistic multi-parameter prediction appears to be an untested but natural next step for actuarial use."],"forward_implications":["Insurers can adopt probabilistic GBDTs such as XGBoostLSS without expecting a meaningful loss in point predictive performance.","Better-calibrated predictive distributions become available for risk management, since CRPS and confidence-interval coverage improve without degrading pseudo-$R^2$.","CatBoost's edge on datasets with high-cardinality categorical variables makes it a sensible default when categorical features dominate.","EGBM's competitive accuracy makes it a viable interpretable alternative for regulated or stakeholder-facing tariffs.","Exposure-to-risk can be handled in tree boosting by the same offset device used in GLMs, so frequency modelling with GBDTs need not abandon that actuarial convention."],"supporting_citations":[{"why":"Supplies the gradient boosting machine procedure from which all compared point and probabilistic algorithms descend.","marker":"[9]"},{"why":"Provides the XGBoost algorithm, the computational base for XGBoostLSS and PGBM and a benchmark point method.","marker":"[10]"},{"why":"Provides LightGBM, the point algorithm reported as most computationally efficient.","marker":"[11]"},{"why":"Provides CatBoost, whose categorical-feature handling drives the predictive gains on high-cardinality datasets.","marker":"[12]"},{"why":"Introduces XGBoostLSS, the probabilistic algorithm reported as most computationally efficient and adequacy-improving.","marker":"[21]"},{"why":"Introduces NGBoost and the natural gradient, a key probabilistic competitor in the adequacy comparisons.","marker":"[22]"},{"why":"Introduces cyc-GBM, the cyclic probabilistic boosting framework used as a benchmark.","marker":"[17]"},{"why":"Defines EGBM, the interpretable GAM-based boosting variant claimed to be competitively accurate.","marker":"[13, 14]"},{"why":"Defines GAMLSS, the traditional distributional-regression benchmark against which probabilistic GBDTs are compared.","marker":"[28]"},{"why":"Supplies the auto-calibration test used to evaluate portfolio balance and calibration.","marker":"[61]"}],"fun_headline_variants":["Probabilistic boosting matches point accuracy, no trade-off","No trade-off: probabilistic GBDT improves fit without losing accuracy","Claim boosting: probabilistic models fix distributions, keep accuracy","From point to probabilistic: no accuracy trade-off in claim boosting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion that there is no trade-off rests on the assumption that a single random 85/15 split per dataset is enough to rank the algorithms, and that the five chosen datasets represent actuarial claim frequency and severity problems.","fun_headline_variants_meta":{"raw":{"variants":["Probabilistic boosting matches point accuracy, no trade-off","No trade-off: probabilistic GBDT improves fit without losing accuracy","Claim boosting: probabilistic models fix distributions, keep accuracy","From point to probabilistic: no accuracy trade-off in claim boosting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1248,"prompt_tokens":915,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":264}},"tokens_in":531,"tokens_out":333,"duration_ms":3124,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:48:02.760898+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run every algorithm on 50 random 85/15 splits of the same five datasets and record the pseudo-$R^2$ and CRPS average ranks; if the point-versus-probabilistic ordering flips across splits or the adequacy gaps shrink to within the noise of the splits, the no-trade-off claim would not hold.","supporting_citations":[{"cited_title":"Annals of statistics 29, 1189–1232 (2001)","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient boosting machine procedure from which all compared point and probabilistic algorithms descend."},{"cited_title":"Advances in neural information processing systems 30, 3146– 3154 (2017)","cited_arxiv_id":null,"evidence_quote":"Provides LightGBM, the point algorithm reported as most computationally efficient."},{"cited_title":"Advances in neural information processing systems 31, 6639–6649 (2018)","cited_arxiv_id":null,"evidence_quote":"Provides CatBoost, whose categorical-feature handling drives the predictive gains on high-cardinality datasets."},{"cited_title":"XGBoostLSS -- An extension of XGBoost to probabilistic forecasting","cited_arxiv_id":"1907.03178","evidence_quote":"Introduces XGBoostLSS, the probabilistic algorithm reported as most computationally efficient and adequacy-improving."},{"cited_title":"Proceedings of the 37th International Conference on Machine Learning 119, 2690–2700 (2020)","cited_arxiv_id":null,"evidence_quote":"Introduces NGBoost and the natural gradient, a key probabilistic competitor in the adequacy comparisons."},{"cited_title":"Available at SSRN 4352505 (2023) https://doi.org/10.2139/ssrn.4352505","cited_arxiv_id":null,"evidence_quote":"Introduces cyc-GBM, the cyclic probabilistic boosting framework used as a benchmark."},{"cited_title":"Insurance: Mathematics and Economics 117, 130–139 (2024) https://doi.org/10","cited_arxiv_id":null,"evidence_quote":"Supplies the auto-calibration test used to evaluate portfolio balance and calibration."}],"review_version":1}