{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T00:14:33Z","timestamp":1787271273608,"version":"build-2736575974"},"reference-count":76,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","funder":[{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SFB\/TRR 165"],"award-info":[{"award-number":["SFB\/TRR 165"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007316","name":"Klaus Tschira Stiftung","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100007316","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Rev."],"published-print":{"date-parts":[[2024,2]]},"abstract":"<jats:p>How can we quantify uncertainty if our favorite computational tool---be it a numerical, statistical, or machine learning approach, or just any computer model---provides single-valued output only? In this article, we introduce the Easy Uncertainty Quantification (EasyUQ) technique, which transforms real-valued model output into calibrated statistical distributions, based solely on training data of model output--outcome pairs, without any need to access model input. In its basic form, EasyUQ is a special case of the recently introduced isotonic distributional regression (IDR) technique that leverages the pool-adjacent-violators algorithm for nonparametric isotonic regression. EasyUQ yields discrete predictive distributions that are calibrated and optimal in finite samples, subject to stochastic monotonicity. The workflow is fully automated, without any need for tuning. The Smooth EasyUQ approach supplements IDR with kernel smoothing, to yield continuous predictive distributions that preserve key properties of the basic form, including both stochastic monotonicity with respect to the original model output and asymptotic consistency. For the selection of kernel parameters, we introduce multiple one-fit grid search, a computationally much less demanding approximation to leave-one-out cross-validation. We use simulation examples and forecast data from weather prediction to illustrate the techniques. In a study of benchmark problems from machine learning, we show how EasyUQ and Smooth EasyUQ can be integrated into the workflow of neural network learning and hyperparameter tuning, and we find EasyUQ to be competitive with conformal prediction as well as more elaborate input-based approaches.<\/jats:p>","DOI":"10.1137\/22m1541915","type":"journal-article","created":{"date-parts":[[2024,2,8]],"date-time":"2024-02-08T04:47:48Z","timestamp":1707367668000},"page":"91-122","source":"Crossref","is-referenced-by-count":10,"title":["Easy Uncertainty Quantification (EasyUQ): Generating Predictive Distributions from Single-Valued Model Output"],"prefix":"10.1137","volume":"66","author":[{"given":"Eva-Maria","family":"Walz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Henzi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johanna","family":"Ziegel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9397-3271","authenticated-orcid":true,"given":"Tilmann","family":"Gneiting","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,2,8]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.05.008"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1214\/21-STS822"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1038\/nature14956"},{"key":"atypb4","volume-title":"The Rise of Data-Driven Weather Forecasting: A First Statistical Assessment of Machine Learning-Based Weather Forecasts in an Operational-Like Context, preprint, https:\/\/arxiv.org\/abs\/2307.10128","author":"Bouall\u00e8gue Z. Ben","year":"2023"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-statistics-030718-105232"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06185-3"},{"key":"atypb7","first-page":"3","volume-title":"Tenth Symposium on Conformal and Probabilistic Prediction and Applications, Proc. Mach. Learn. Res. 102","author":"Bostr\u00f6m H.","year":"2021"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/85.4.799"},{"key":"atypb9","volume-title":"Algorithms for Minimization without Derivatives","author":"Brent R. P.","year":"1973"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.1111\/j.1600-0870.2008.00333.x"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1615\/Int.J.UncertaintyQuantification.2021034623"},{"key":"atypb12","first-page":"14987","volume":"23","author":"\u0106evid D.","year":"2022","journal-title":"J. Mach. Learn. Res."},{"key":"atypb13","volume-title":"FengWu: Pushing the Skillful Global Medium-Range Weather Forecast beyond 10 Days Lead, preprint, https:\/\/arxiv.org\/abs\/2304.02948","author":"Chen K.","year":"2023"},{"key":"atypb14","first-page":"1","volume-title":"35th Conference on Neural Information Processing Systems (NeurIPS), Neural Information Processing Systems Foundation","author":"Chung Y.","year":"2021"},{"key":"atypb15","first-page":"1","volume-title":"35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation","author":"Daxberger E.","year":"2021"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1051\/0004-6361\/201731326"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1016\/j.solener.2020.05.051"},{"key":"atypb18","first-page":"2690","volume-title":"37th International Conference on Machine Learning, Proc. Mach. Learn. Res. 119","author":"Duan T.","year":"2020"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-023-02084-9"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1198\/016214504000000764"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/86.1.153"},{"key":"atypb22","first-page":"1050","volume-title":"33rd International Conference on Machine Learning, Proc. Mach. Learn. Res. 48","author":"Gal Y.","year":"2016"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1198\/016214504000000872"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-12385-1"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2007.00587.x"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-statistics-062713-085831"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1126\/science.1115255"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1198\/016214506000001437"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1175\/MWR2904.1"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-021-06115-2"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1214\/18-STS665"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1080\/07350015.2022.2116026"},{"key":"atypb33","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2021.1938582"},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1007\/s11009-022-09937-2"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1111\/rssb.12450"},{"key":"atypb36","first-page":"1861","volume-title":"32nd International Conference on Machine Learning, Proc. Mach. Learn. Res. 37","author":"Hern\u00e1ndez-Lobato J. M.","year":"2015"},{"key":"atypb37","first-page":"4563","volume-title":"38th International Conference on Machine Learning, 2021. Proc. Mach. Learn. Res. 139","author":"Immer A.","year":"2021"},{"key":"atypb38","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v090.i12"},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1111\/insr.12039"},{"key":"atypb40","first-page":"95","volume-title":"Machine Learning Challenges: Evaluating Predictive Uncertainty, Visual Object Classification, and Recognizing Textual Entailment, J. Quin\u0342onero-Candela, I. Dagan, B. Magnini, and F. d'Alch\u00e9-Buc","author":"Kohonen J.","year":"2006"},{"key":"atypb41","first-page":"2796","volume-title":"35th International Conference on Machine Learning, Proc. Mach. Learn. Res. 80","author":"Kuleshov V.","year":"2018"},{"key":"atypb42","volume-title":"31st Conference on Neural Information Processing Systems (NIPS), Neural Information Processing Systems Foundation","author":"Lakshminarayanan B.","year":"2017"},{"key":"atypb43","volume-title":"GraphCast: Learning Skillful Medium-Range Global Weather Forecasting, preprint, https:\/\/arxiv.org\/abs\/2212.12794","author":"Lam R.","year":"2022"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2007.02.014"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1080\/07350015.2012.738955"},{"key":"atypb46","first-page":"15180","volume-title":"39th International Conference on Machine Learning, 2022. Proc. Mach. Learn. Res. 62","author":"Marx C.","year":"2022"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.22.10.1087"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1175\/MWR-D-13-00355.1"},{"key":"atypb49","doi-asserted-by":"publisher","DOI":"10.1002\/qj.49712252905"},{"key":"atypb50","doi-asserted-by":"publisher","DOI":"10.1214\/19-EJS1659"},{"key":"atypb51","doi-asserted-by":"publisher","DOI":"10.1007\/s00180-014-0523-0"},{"key":"atypb52","doi-asserted-by":"publisher","DOI":"10.1088\/0034-4885\/63\/2\/201"},{"key":"atypb53","volume-title":"Python Software Foundation","author":"Reference Python Language","year":"2023"},{"key":"atypb54","first-page":"1","volume-title":"Machine Learning Challenges: Evaluating Predictive Uncertainty, Visual Object Classification, and Recognizing Textual Entailment, J. Quin\u0342onero-Candela, I. Dagan, B. Magnini, and F. d'Alch\u00e9-Buc","author":"Quin\u0342onero-Candela J.","year":"2006"},{"key":"atypb55","doi-asserted-by":"publisher","DOI":"10.1175\/MWR2906.1"},{"key":"atypb56","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"atypb57","doi-asserted-by":"publisher","DOI":"10.1029\/2020MS002203"},{"key":"atypb58","doi-asserted-by":"publisher","DOI":"10.1175\/MWR-D-18-0187.1"},{"key":"atypb59","first-page":"1","volume-title":"International Conference on Learning Representations, ICLR","author":"Ritter H.","year":"2018"},{"key":"atypb60","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2011.03.016"},{"key":"atypb61","doi-asserted-by":"publisher","DOI":"10.1214\/13-STS443"},{"key":"atypb62","doi-asserted-by":"publisher","DOI":"10.1002\/qj.2183"},{"key":"atypb63","doi-asserted-by":"publisher","DOI":"10.1214\/19-AOAS1247"},{"key":"atypb64","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2020.0097"},{"key":"atypb65","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-34675-5"},{"key":"atypb66","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-3324-9"},{"key":"atypb67","doi-asserted-by":"publisher","DOI":"10.1175\/MWR3441.1"},{"key":"atypb68","volume-title":"Uncertainty Quantification: Theory, Implementation, and Applications","author":"Smith R. C.","year":"2014"},{"key":"atypb69","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v023.i07"},{"key":"atypb70","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-23395-6"},{"key":"atypb71","first-page":"1","volume":"45","author":"Trefethen N.","year":"2012","journal-title":"SIAM News"},{"key":"atypb72","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-06649-8"},{"key":"atypb73","first-page":"37","volume-title":"Conformal and Probabilistic Prediction and Applications, Proc. Mach. Learn. Res. 91","author":"Vovk V.","year":"2018"},{"key":"atypb74","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.10.110"},{"key":"atypb75","first-page":"84","volume-title":"Conformal and Probabilistic Prediction and Applications, Proc. Mach. Learn. Res. 128","author":"Vovk V.","year":"2020"},{"key":"atypb76","doi-asserted-by":"crossref","unstructured":"E.M. Walz,\n                      Replication material for \u201cEasy Uncertainty Quantification (EasyUQ): Generating predictive distributions from single-valued model output,''\n                      2023; available athttps:\/\/github.com\/evwalz\/easyuq.","DOI":"10.1137\/22M1541915"}],"container-title":["SIAM Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/22M1541915","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T12:18:42Z","timestamp":1787228322000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/22M1541915"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2]]},"references-count":76,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2]]}},"alternative-id":["10.1137\/22M1541915"],"URL":"https:\/\/doi.org\/10.1137\/22m1541915","relation":{},"ISSN":["0036-1445","1095-7200"],"issn-type":[{"value":"0036-1445","type":"print"},{"value":"1095-7200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2]]}}}