{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T08:45:28Z","timestamp":1770885928040,"version":"3.50.1"},"reference-count":68,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2024,10,22]],"date-time":"2024-10-22T00:00:00Z","timestamp":1729555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2024,10,22]],"date-time":"2024-10-22T00:00:00Z","timestamp":1729555200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"crossref","award":["101001890-FIAMMA"],"award-info":[{"award-number":["101001890-FIAMMA"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100009150","name":"National Center of Competence in Research Materials\u2019 Revolution: Computational Design and Discovery of Novel Materials","doi-asserted-by":"crossref","award":["182892"],"award-info":[{"award-number":["182892"]}],"id":[{"id":"10.13039\/501100009150","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","award":["200020_214879"],"award-info":[{"award-number":["200020_214879"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2024,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Quantifying the uncertainty of regression models is essential to ensure their reliability, particularly since their application often extends beyond their training domain. Based on the solution of a constrained optimization problem, this work proposes \u2018prediction rigidities\u2019 as a formalism to obtain uncertainties of arbitrary pre-trained regressors. A clear connection between the suggested framework and Bayesian inference is established, and a last-layer approximation is developed and rigorously justified to enable the application of the method to neural networks. This extension affords cheap uncertainties without any modification to the neural network itself or its training procedure. The effectiveness of this approach is shown for a wide range of regression tasks, ranging from simple toy models to applications in chemistry and meteorology.<\/jats:p>","DOI":"10.1088\/2632-2153\/ad805f","type":"journal-article","created":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T23:02:34Z","timestamp":1727391754000},"page":"045018","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["A prediction rigidity formalism for low-cost uncertainties in trained neural networks"],"prefix":"10.1088","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9338-7317","authenticated-orcid":true,"given":"Filippo","family":"Bigi","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6948-1602","authenticated-orcid":true,"given":"Sanggyu","family":"Chong","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2571-2832","authenticated-orcid":true,"given":"Michele","family":"Ceriotti","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4284-0094","authenticated-orcid":true,"given":"Federico","family":"Grasselli","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2024,10,22]]},"reference":[{"key":"mlstad805fbib1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10462-023-10562-9","article-title":"A survey of uncertainty in deep neural networks","volume":"56","author":"Gawlikowski","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"mlstad805fbib2","first-page":"237","article-title":"Active learning literature survey","author":"Settles","year":"2009","journal-title":"Science"},{"key":"mlstad805fbib3","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"mlstad805fbib4","volume":"vol 1","author":"Bengio","year":"2017"},{"key":"mlstad805fbib5","first-page":"p 30","article-title":"Simple and scalable predictive uncertainty estimation using deep ensembles","author":"Lakshminarayanan","year":"2017"},{"key":"mlstad805fbib6","author":"Kunapuli","year":"2023"},{"key":"mlstad805fbib7","article-title":"Pitfalls of in-domain uncertainty estimation and ensembling in deep learning","author":"Ashukha","year":"2020"},{"key":"mlstad805fbib8","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ad594a","article-title":"Uncertainty quantification by direct propagation of shallow ensembles","volume":"5","author":"Kellner","year":"2024","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstad805fbib9","first-page":"pp 370","article-title":"Deep kernel learning","author":"Gordon Wilson","year":"2016"},{"key":"mlstad805fbib10","article-title":"Benchmarking the neural linear model for regression","author":"Ober","year":"2019"},{"key":"mlstad805fbib11","first-page":"pp 1206","article-title":"The promises and pitfalls of deep kernel learning","author":"Ober","year":"2021"},{"key":"mlstad805fbib12","first-page":"pp 20089","article-title":"Laplace redux-effortless Bayesian deep learning","volume":"vol 34","author":"Daxberger","year":"2021"},{"key":"mlstad805fbib13","article-title":"Variational Bayesian last layers","author":"Harrison","year":"2024"},{"key":"mlstad805fbib14","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1016\/j.inffus.2021.05.008","article-title":"A review of uncertainty quantification in deep learning: techniques, applications and challenges","volume":"76","author":"Abdar","year":"2021","journal-title":"Inf. Fusion"},{"key":"mlstad805fbib15","volume":"vol 405","author":"Bernardo","year":"2009"},{"key":"mlstad805fbib16","volume":"vol 118","author":"Neal","year":"2012"},{"key":"mlstad805fbib17","article-title":"A scalable laplace approximation for neural networks","volume":"vol 6","author":"Ritter","year":"2018"},{"key":"mlstad805fbib18","article-title":"Fusion framework and multimodality for the Laplacian approximation of Bayesian neural networks","author":"Malmstr\u00f6m","year":"2023"},{"key":"mlstad805fbib19","first-page":"p 24","article-title":"Practical variational inference for neural networks","author":"Graves","year":"2011"},{"key":"mlstad805fbib20","first-page":"pp 1613","article-title":"Weight uncertainty in neural network","author":"Blundell","year":"2015"},{"key":"mlstad805fbib21","first-page":"pp 1861","article-title":"Probabilistic backpropagation for scalable learning of Bayesian neural networks","author":"Miguel Hern\u00e1ndez-Lobato","year":"2015"},{"key":"mlstad805fbib22","first-page":"p 31","article-title":"Predictive uncertainty estimation via prior networks","author":"Malinin","year":"2018"},{"key":"mlstad805fbib23","first-page":"pp 7498","article-title":"Simple and principled uncertainty estimation with deterministic deep learning via distance awareness","volume":"vol 33","author":"Liu","year":"2020"},{"key":"mlstad805fbib24","first-page":"pp 1050","article-title":"Dropout as a Bayesian approximation: representing model uncertainty in deep learning","author":"Gal","year":"2016"},{"key":"mlstad805fbib25","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"mlstad805fbib26","first-page":"pp 29","article-title":"Priors for infinite networks","author":"Neal","year":"1996"},{"key":"mlstad805fbib27","first-page":"p 9","article-title":"Computing with infinite networks","author":"Williams","year":"1996"},{"key":"mlstad805fbib28","article-title":"Deep neural networks as Gaussian processes","author":"Lee","year":"2017"},{"key":"mlstad805fbib29","first-page":"pp 1010","article-title":"Bayesian deep ensembles via the neural tangent kernel","volume":"vol 33","author":"Bobby","year":"2020"},{"key":"mlstad805fbib30","article-title":"Deep ensembles from a Bayesian perspective","author":"Hoffmann","year":"2021"},{"key":"mlstad805fbib31","first-page":"pp 20063","article-title":"Uncertainty quantification and deep ensembles","volume":"vol 34","author":"Rahaman","year":"2021"},{"key":"mlstad805fbib32","first-page":"p 32","article-title":"Can you trust your model\u2019s uncertainty? Evaluating predictive uncertainty under dataset shift","author":"Ovadia","year":"2019"},{"key":"mlstad805fbib33","article-title":"Snapshot ensembles: train 1, get m for free","author":"Huang","year":"2017"},{"key":"mlstad805fbib34","first-page":"p 31","article-title":"Loss surfaces, mode connectivity and fast ensembling of dnns","author":"Garipov","year":"2018"},{"key":"mlstad805fbib35","article-title":"Cyclical stochastic gradient MCMC for Bayesian deep learning","author":"Zhang","year":"2019"},{"key":"mlstad805fbib36","volume":"vol 6","author":"Laplace","year":"1774"},{"key":"mlstad805fbib37","first-page":"pp 1775","article-title":"Kernel interpolation for scalable structured Gaussian processes (KISS-GP)","author":"Wilson","year":"2015"},{"key":"mlstad805fbib38","first-page":"p 31","article-title":"Neural tangent kernel: convergence and generalization in neural networks","author":"Jacot","year":"2018"},{"key":"mlstad805fbib39","article-title":"Approximate inference turns deep networks into Gaussian processes","volume":"vol 32","author":"Khan","year":"2019"},{"key":"mlstad805fbib40","doi-asserted-by":"publisher","first-page":"8020","DOI":"10.1021\/acs.jctc.3c00704","article-title":"Robustness of local predictions in atomistic machine learning models","volume":"19","author":"Chong","year":"2023","journal-title":"J. Chem. Theory Comput."},{"key":"mlstad805fbib41","first-page":"1","article-title":"A framework and benchmark for deep batch active learning for regression","volume":"24","author":"Holzm\u00fcller","year":"2023","journal-title":"J. Mach. Learn. Res."},{"key":"mlstad805fbib42","doi-asserted-by":"publisher","first-page":"1723","DOI":"10.1162\/08997660260028683","article-title":"Fast curvature matrix-vector products for second-order gradient descent","volume":"14","author":"Schraudolph","year":"2002","journal-title":"Neural Comput."},{"key":"mlstad805fbib43","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1090\/qam\/10666","article-title":"A method for the solution of certain non-linear problems in least squares","volume":"2","author":"Levenberg","year":"1944","journal-title":"Q. Appl. Math."},{"key":"mlstad805fbib44","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1137\/0111030","article-title":"An algorithm for least-squares estimation of nonlinear parameters","volume":"11","author":"Marquardt","year":"1963","journal-title":"J. Soc. Ind. Appl. Math."},{"key":"mlstad805fbib45","author":"Press","year":"2007"},{"key":"mlstad805fbib46","volume":"vol 2","author":"Hastie","year":"2009"},{"key":"mlstad805fbib47","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1162\/neco.1992.4.3.415","article-title":"Bayesian interpolation","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"mlstad805fbib48","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1162\/neco.1992.4.4.590","article-title":"Information-based objective functions for active data selection","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"mlstad805fbib49","doi-asserted-by":"publisher","first-page":"1071","DOI":"10.1016\/0893-6080(95)00137-9","article-title":"Neural network exploration using optimal experiment design","volume":"9","author":"Cohn","year":"1996","journal-title":"Neural Netw."},{"key":"mlstad805fbib50","doi-asserted-by":"publisher","DOI":"10.1063\/5.0036522","article-title":"Uncertainty estimation for molecular dynamics and sampling","volume":"154","author":"Imbalzano","year":"2021","journal-title":"J. Chem. Phys."},{"key":"mlstad805fbib51","first-page":"p 32","article-title":"Wide neural networks of any depth evolve as linear models under gradient descent","author":"Lee","year":"2019"},{"key":"mlstad805fbib52","author":"Woodbury","year":"1950"},{"key":"mlstad805fbib53","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2014.22","article-title":"Quantum chemistry structures and properties of 134 kilo molecules","volume":"1","author":"Ramakrishnan","year":"2014","journal-title":"Sci. Data"},{"key":"mlstad805fbib54","doi-asserted-by":"publisher","first-page":"B864","DOI":"10.1103\/PhysRev.136.B864","article-title":"Inhomogeneous electron gas","volume":"136","author":"Hohenberg","year":"1964","journal-title":"Phys. Rev."},{"key":"mlstad805fbib55","doi-asserted-by":"publisher","first-page":"A1133","DOI":"10.1103\/PhysRev.140.A1133","article-title":"Self-consistent equations including exchange and correlation effects","volume":"140","author":"Kohn","year":"1965","journal-title":"Phys. Rev."},{"key":"mlstad805fbib56","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.98.146401","article-title":"Generalized neural-network representation of high-dimensional potential-energy surfaces","volume":"98","author":"Behler","year":"2007","journal-title":"Phys. Rev. Lett."},{"key":"mlstad805fbib57","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.87.184115","article-title":"On representing chemical environments","volume":"87","author":"Bart\u00f3k","year":"2013","journal-title":"Phys. Rev. B"},{"key":"mlstad805fbib58","article-title":"Australian weather dataset","author":"Young","year":"2019"},{"key":"mlstad805fbib59","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1007783811760","article-title":"Spatial statistics and real estate","volume":"17","author":"Kelley Pace","year":"1998","journal-title":"J. Real Estate Finance Econ."},{"key":"mlstad805fbib60","author":"Rasmussen","year":"2006"},{"key":"mlstad805fbib61","first-page":"p 29","article-title":"Toward deeper understanding of neural networks: the power of initialization and a dual view on expressivity","author":"Daniely","year":"2016"},{"key":"mlstad805fbib62","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2208779120","article-title":"Wide and deep neural networks achieve consistency for classification","volume":"120","author":"Radhakrishnan","year":"2023","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstad805fbib63","first-page":"pp 5436","article-title":"Being Bayesian, even just a bit, fixes overconfidence in relu networks","author":"Kristiadi","year":"2020"},{"key":"mlstad805fbib64","author":"Wolfram Research Inc.","year":"2024"},{"key":"mlstad805fbib65","article-title":"On the infinite width limit of neural networks with a standard parameterization","author":"Sohl-Dickstein","year":"2020"},{"key":"mlstad805fbib66","first-page":"p 32","article-title":"Pytorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019"},{"key":"mlstad805fbib67","article-title":"Jax: composable transformations of python+ numpy programs","author":"Bradbury","year":"2018"},{"key":"mlstad805fbib68","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with numpy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,22]],"date-time":"2024-10-22T10:15:44Z","timestamp":1729592144000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad805f"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,22]]},"references-count":68,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,10,22]]},"published-print":{"date-parts":[[2024,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ad805f","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,22]]},"assertion":[{"value":"A prediction rigidity formalism for low-cost uncertainties in trained neural networks","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2024-03-04","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-09-26","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-10-22","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}