{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T16:48:13Z","timestamp":1792601293155,"version":"4.5.0"},"reference-count":46,"publisher":"Wiley","issue":"23-24","license":[{"start":{"date-parts":[[2026,10,18]],"date-time":"2026-10-18T00:00:00Z","timestamp":1792281600000},"content-version":"vor","delay-in-days":17,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T00:00:00Z","timestamp":1790812800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Statistics in Medicine"],"published-print":{"date-parts":[[2026,10]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Statistical prediction models for binary outcomes are becoming increasingly popular. One significant challenge is calibrating these models to suit the characteristics of a target population that is structurally different from the original population. Calibration is especially challenging when there is no training data available from the target population. To address this problem, we propose a novel calibration method, SimCal, which uses synthetic data generated from the model development data in conjunction with marginal statistics from the calibration cohort. We show that expert judgment modeling (EJM) may be used for calibration if cross\u2010sectional data from the target population are available comprising expert judgments about the potential outcome and the covariates. We describe three alternative calibration approaches when calibration data are lacking: similarity\u2010binning averaging (SBA), adaptive calibration of predictions (ACP), and Elkan calibration. In a simulation study, we compare SBA, ACP, Elkan calibration, and SimCal. R code for applying these methods is provided from the re\u2010analysis of data on coronary artery disease. We illustrate all 5 calibration approaches with a real data set for predicting functional outcome after stroke and all approaches but EJM in the re\u2010analysis of the Cleveland Clinic data. None of the approaches performed convincingly well in all situations. SimCal performed well when model parameters were correctly specified. EJM failed on the stroke data. Further research is urgently required for calibration in the absence of calibration data.<\/jats:p>","DOI":"10.1002\/sim.70751","type":"journal-article","created":{"date-parts":[[2026,10,19]],"date-time":"2026-10-19T04:17:36Z","timestamp":1792383456000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Calibrating Machine Learning Approaches for Probability Estimation Without Calibration Data"],"prefix":"10.1002","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5618-1141","authenticated-orcid":false,"given":"Eleonora Di","family":"Carluccio","sequence":"first","affiliation":[{"name":"Cardio\u2010CARE Medizincampus Davos  Davos Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0667-6178","authenticated-orcid":false,"given":"Georgios","family":"Koliopanos","sequence":"additional","affiliation":[{"name":"Cardio\u2010CARE Medizincampus Davos  Davos Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francisco M.","family":"Ojeda","sequence":"additional","affiliation":[{"name":"Department of Cardiology University Heart and Vascular Center Hamburg, University Medical Center Hamburg\u2010Eppendorf  Hamburg Germany"},{"name":"Center for Population Health Innovation (POINT) University Heart and Vascular Center Hamburg, University Medical Center Hamburg\u2010Eppendorf  Hamburg Germany"},{"name":"Partner Site Hamburg\/Kiel\/L\u00fcbeck German Center for Cardiovascular Research (DZHK)  Hamburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Weimar","sequence":"additional","affiliation":[{"name":"BDH\u2010Klinik Elzach  Elzach Germany"},{"name":"Institute for Medical Informatics, Biometry and Epidemiology University Hospital, University Duisburg\u2010Essen  Essen Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8386-5397","authenticated-orcid":false,"given":"Andreas","family":"Ziegler","sequence":"additional","affiliation":[{"name":"Cardio\u2010CARE Medizincampus Davos  Davos Switzerland"},{"name":"Department of Cardiology University Heart and Vascular Center Hamburg, University Medical Center Hamburg\u2010Eppendorf  Hamburg Germany"},{"name":"Center for Population Health Innovation (POINT) University Heart and Vascular Center Hamburg, University Medical Center Hamburg\u2010Eppendorf  Hamburg Germany"},{"name":"Discipline of Statistics, School of Agriculture and Science University of KwaZulu\u2010Natal  Pietermaritzburg South Africa"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,10,18]]},"reference":[{"key":"e_1_2_12_2_1","doi-asserted-by":"publisher","DOI":"10.3414\/ME00-01-0052"},{"key":"e_1_2_12_3_1","doi-asserted-by":"publisher","DOI":"10.1515\/9781400882618-014"},{"key":"e_1_2_12_4_1","doi-asserted-by":"publisher","DOI":"10.1093\/eurheartj\/ehu207"},{"key":"e_1_2_12_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz228"},{"key":"e_1_2_12_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.9921"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.10051"},{"key":"e_1_2_12_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04394-9_42"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/1113.003.0008"},{"key":"e_1_2_12_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-012-0388-2"},{"key":"e_1_2_12_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2020.101641"},{"key":"e_1_2_12_12_1","doi-asserted-by":"publisher","DOI":"10.1214\/17-EJS1338SI"},{"key":"e_1_2_12_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2015.12.005"},{"key":"e_1_2_12_14_1","first-page":"73","volume-title":"VALDOR\u00a0\u2013\u00a0Values in Decisions on Risk. Proceedings","author":"Rosqvist T.","year":"1999"},{"key":"e_1_2_12_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-306-47630-3_9"},{"key":"e_1_2_12_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/risa.13718"},{"key":"e_1_2_12_17_1","doi-asserted-by":"publisher","DOI":"10.1287\/deca.2019.0391"},{"key":"e_1_2_12_18_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000291"},{"key":"e_1_2_12_19_1","first-page":"973","volume-title":"Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence","author":"Elkan C.","year":"2001"},{"key":"e_1_2_12_20_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"R Core Team","year":"2025"},{"key":"e_1_2_12_21_1","doi-asserted-by":"publisher","DOI":"10.1161\/01.STR.0000106761.94985.8B"},{"key":"e_1_2_12_22_1","volume-title":"Simulated Data for \u201cCalibrating Machine Learning Approaches for Probability Estimation Without Calibration Data\u201d","author":"Di Carluccio E.","year":"2026"},{"key":"e_1_2_12_23_1","volume-title":"R Package Version 1.0.1","author":"Ziegler A.","year":"2025"},{"key":"e_1_2_12_24_1","doi-asserted-by":"publisher","DOI":"10.1214\/07-AOAS107"},{"key":"e_1_2_12_25_1","doi-asserted-by":"publisher","DOI":"10.1055\/a-2048-7692"},{"key":"e_1_2_12_26_1","volume-title":"R Package Version 7.3\u201365","author":"Ripley B. D.","year":"2025"},{"key":"e_1_2_12_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10519-009-9281-0"},{"key":"e_1_2_12_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/wics.1514"},{"key":"e_1_2_12_29_1","doi-asserted-by":"publisher","DOI":"10.1515\/1557-4679.1395"},{"key":"e_1_2_12_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-017-1133-2"},{"key":"e_1_2_12_31_1","first-page":"717","article-title":"Vorhersage des Funktionsstatus 100 Tage nach einem isch\u00e4mischen Schlaganfall: Design einer prospektiven Studie zur externen Validierung eines prognostischen Modells","volume":"97","author":"K\u00f6nig I. R.","year":"2003","journal-title":"Zeitschrift f\u00fcr \u00c4rztliche Fortbildung und Qualit\u00e4tssicherung"},{"key":"e_1_2_12_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00415-002-0755-8"},{"key":"e_1_2_12_33_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.g7594"},{"key":"e_1_2_12_34_1","first-page":"56","article-title":"Functional Evaluation: The Barthel Index","volume":"14","author":"Mahoney F. I.","year":"1965","journal-title":"Maryland State Medical Journal"},{"key":"e_1_2_12_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/0002-9149(89)90524-9"},{"key":"e_1_2_12_36_1","doi-asserted-by":"publisher","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2"},{"key":"e_1_2_12_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-23534-9_20"},{"key":"e_1_2_12_38_1","doi-asserted-by":"publisher","DOI":"10.1198\/016214506000001437"},{"key":"e_1_2_12_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-16399-0"},{"key":"e_1_2_12_40_1","first-page":"1","article-title":"Statistical Comparisons of Classifiers Over Multiple Data Sets","volume":"7","author":"Dem\u0161ar J.","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_12_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_12_42_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v077.i01"},{"key":"e_1_2_12_43_1","article-title":"An Infrastructure for Performance Estimation and Experimental Comparison of Predictive Models in R","author":"Torgo L.","year":"2015","journal-title":"arXiv"},{"key":"e_1_2_12_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(24)00191-2"},{"key":"e_1_2_12_45_1","first-page":"250","article-title":"SimCorrMix: Simulation of Correlated Data With Multiple Variable Types Including Continuous and Count Mixture Distributions","volume":"11","author":"Fialkowski A.","year":"2019","journal-title":"R Journal"},{"key":"e_1_2_12_46_1","doi-asserted-by":"publisher","DOI":"10.1002\/bimj.201300068"},{"key":"e_1_2_12_47_1","doi-asserted-by":"publisher","DOI":"10.1002\/bimj.201300077"}],"container-title":["Statistics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.70751","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/sim.70751","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.70751","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T16:35:15Z","timestamp":1792600515000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sim.70751"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10]]},"references-count":46,"journal-issue":{"issue":"23-24","published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1002\/sim.70751"],"URL":"https:\/\/doi.org\/10.1002\/sim.70751","archive":["Portico"],"relation":{},"ISSN":["0277-6715","1097-0258"],"issn-type":[{"value":"0277-6715","type":"print"},{"value":"1097-0258","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10]]},"assertion":[{"value":"2025-11-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-16","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70751"}}