Pith. sign in

REVIEW 5 cited by

Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2109.10254 v1 pith:YKRPSTNI submitted 2021-09-21 cs.LG stat.ML

Youngseog Chung , Ian Char , Han Guo , Jeff Schneider , Willie Neiswanger This is my paper · ORCID
classification cs.LGstat.ML
keywords uncertaintytoolboxlearningmachinequantificationlibraryopen-sourceworks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

With increasing deployment of machine learning systems in various real-world tasks, there is a greater need for accurate quantification of predictive uncertainty. While the common goal in uncertainty quantification (UQ) in machine learning is to approximate the true distribution of the target data, many works in UQ tend to be disjoint in the evaluation metrics utilized, and disparate implementations for each metric lead to numerical results that are not directly comparable across different works. To address this, we introduce Uncertainty Toolbox, an open-source python library that helps to assess, visualize, and improve UQ. Uncertainty Toolbox additionally provides pedagogical resources, such as a glossary of key terms and an organized collection of key paper references. We hope that this toolbox is useful for accelerating and uniting research efforts in uncertainty in machine learning.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors

    stat.ML 2026-08 conditional novelty 7.0 of 10

    A model's forward-then-backward round-trip error predicts, without ensembles or ground truth, how large its actual rollout error will be.

  2. Trustworthy Protein-Ligand Binding Affinity Prediction via Reliability-Aware Multi-Engine Fusion

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A reliability-aware evidential fusion model combines four docking engines and produces well-calibrated confidence scores that enable selective prediction with up to 25.7% error reduction.

  3. DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods

    cs.LG 2024-11 conditional novelty 5.0 of 10

    On simple toy data, both DE and DER scale their predicted noise with the injected level, but miscalibration appears in half of DE experiments and nearly all DER experiments.

  4. Providing Machine Learning Potentials with High Quality Uncertainty Estimates

    physics.comp-ph 2025-01 conditional novelty 4.0 of 10

    A Bayesian neural network version of the ANI-1x potential provides uncertainty estimates that, in the tested cases, cover the model's errors at least as well as a nine-model ensemble.

  5. Conformal Prediction on Quantifying Uncertainty of Dynamic Systems

    cs.LG 2024-12 conditional novelty 4.0 of 10

    Conformal prediction gives well-calibrated uncertainty intervals for neural-operator predictions of Navier-Stokes dynamics, beating Monte Carlo dropout and ensembles on calibration metrics in the reported experiments.

Pith tools