{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T11:17:28Z","timestamp":1790853448125,"version":"4.1.0"},"reference-count":80,"publisher":"Wiley","issue":"14","license":[{"start":{"date-parts":[[2014,1,14]],"date-time":"2014-01-14T00:00:00Z","timestamp":1389657600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2015,9,1]],"date-time":"2015-09-01T00:00:00Z","timestamp":1441065600000},"content-version":"tdm","delay-in-days":595,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100001826","name":"Netherlands Organisation for Health Research and Development","doi-asserted-by":"publisher","award":["91208004"],"award-info":[{"award-number":["91208004"]}],"id":[{"id":"10.13039\/501100001826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001826","name":"Netherlands Organisation for Health Research and Development","doi-asserted-by":"publisher","award":["91810615"],"award-info":[{"award-number":["91810615"]}],"id":[{"id":"10.13039\/501100001826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001826","name":"Netherlands Organisation for Health Research and Development","doi-asserted-by":"publisher","award":["91611126"],"award-info":[{"award-number":["91611126"]}],"id":[{"id":"10.13039\/501100001826","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["wiley.com"],"crossmark-restriction":true},"short-container-title":["Statistics in Medicine"],"published-print":{"date-parts":[[2014,6,30]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Published clinical prediction models are often ignored during the development of novel prediction models despite similarities in populations and intended usage. The plethora of prediction models that arise from this practice may still perform poorly when applied in other populations. Incorporating prior evidence might improve the accuracy of prediction models and make them potentially better generalizable. Unfortunately, aggregation of prediction models is not straightforward, and methods to combine differently specified models are currently lacking. We propose two approaches for aggregating previously published prediction models when a validation dataset is available: model averaging and stacked regressions. These approaches yield user\u2010friendly stand\u2010alone models that are adjusted for the new validation data. Both approaches rely on weighting to account for model performance and between\u2010study heterogeneity but adopt a different rationale (averaging versus combination) to combine the models. We illustrate their implementation in a clinical example and compare them with established methods for prediction modeling in a series of simulation studies. Results from the clinical datasets and simulation studies demonstrate that aggregation yields prediction models with better discrimination and calibration in a vast majority of scenarios, and results in equivalent performance (compared to developing a novel model from scratch) when validation datasets are relatively large. In conclusion, model aggregation is a promising strategy when several prediction models are available from the literature and a validation dataset is at hand. The aggregation methods do not require existing models to have similar predictors and can be applied when relatively few data are at hand. Copyright \u00a9 2014 John Wiley &amp; Sons, Ltd.<\/jats:p>","DOI":"10.1002\/sim.6080","type":"journal-article","created":{"date-parts":[[2014,1,14]],"date-time":"2014-01-14T06:12:57Z","timestamp":1389679977000},"page":"2341-2362","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":62,"title":["Meta\u2010analysis and aggregation of multiple published prediction models"],"prefix":"10.1002","volume":"33","author":[{"given":"Thomas P.A.","family":"Debray","sequence":"first","affiliation":[{"name":"Julius Center for Health Sciences and Primary Care University Medical Center Utrecht  Utrecht The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hendrik","family":"Koffijberg","sequence":"additional","affiliation":[{"name":"Julius Center for Health Sciences and Primary Care University Medical Center Utrecht  Utrecht The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daan","family":"Nieboer","sequence":"additional","affiliation":[{"name":"Center for Medical Decision Sciences, Department of Public Health Erasmus Medical Center  Rotterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yvonne","family":"Vergouwe","sequence":"additional","affiliation":[{"name":"Center for Medical Decision Sciences, Department of Public Health Erasmus Medical Center  Rotterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ewout W.","family":"Steyerberg","sequence":"additional","affiliation":[{"name":"Center for Medical Decision Sciences, Department of Public Health Erasmus Medical Center  Rotterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karel G.M.","family":"Moons","sequence":"additional","affiliation":[{"name":"Julius Center for Health Sciences and Primary Care University Medical Center Utrecht  Utrecht The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2014,1,14]]},"reference":[{"key":"e_1_2_12_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/07357900802572110"},{"key":"e_1_2_12_3_1","first-page":"1","article-title":"A systematic review of breast cancer incidence risk prediction models with meta\u2010analysis of their performance","volume":"132","author":"Meads C","year":"2011","journal-title":"Breast Cancer Research and Treatment"},{"key":"e_1_2_12_4_1","doi-asserted-by":"publisher","DOI":"10.1136\/heartjnl-2011-301246"},{"key":"e_1_2_12_5_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b375"},{"key":"e_1_2_12_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-77244-8"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1161\/01.CIR.97.18.1837"},{"key":"e_1_2_12_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0195-668X(03)00114-3"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.1136\/hrt.2006.108167"},{"key":"e_1_2_12_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1010-7940(99)00134-7"},{"key":"e_1_2_12_11_1","doi-asserted-by":"publisher","DOI":"10.1161\/hc0302.102575"},{"key":"e_1_2_12_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(97)08140-3"},{"key":"e_1_2_12_13_1","first-page":"424a","article-title":"A simple clinical model to categorize pretest probability in patients with suspected pulmonary embolism","volume":"90","author":"Wells PS","year":"1999","journal-title":"Blood"},{"key":"e_1_2_12_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00134-011-2314-1"},{"key":"e_1_2_12_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(99)00103-1"},{"key":"e_1_2_12_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X0102100106"},{"key":"e_1_2_12_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.5732"},{"key":"e_1_2_12_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(03)00207-5"},{"key":"e_1_2_12_19_1","doi-asserted-by":"publisher","DOI":"10.1136\/heartjnl-2011-301247"},{"key":"e_1_2_12_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2008.04.008"},{"key":"e_1_2_12_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2007.04.018"},{"key":"e_1_2_12_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.1844"},{"key":"e_1_2_12_23_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-6-38"},{"key":"e_1_2_12_24_1","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.109.926808"},{"key":"e_1_2_12_25_1","doi-asserted-by":"publisher","DOI":"10.1136\/heartjnl-2011-300734"},{"key":"e_1_2_12_26_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b605"},{"issue":"24","key":"e_1_2_12_27_1","doi-asserted-by":"crossref","first-page":"3401","DOI":"10.1002\/1097-0258(20001230)19:24<3401::AID-SIM554>3.0.CO;2-2","article-title":"Validation, calibration, revision and combination of prognostic survival models","volume":"19","author":"Houwelingen HC","year":"2000","journal-title":"Statistics in Medicine"},{"key":"e_1_2_12_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2004.06.017"},{"key":"e_1_2_12_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/0197-2456(86)90046-2"},{"key":"e_1_2_12_30_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.3.4.486"},{"key":"e_1_2_12_31_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b4184"},{"key":"e_1_2_12_32_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-9-29"},{"key":"e_1_2_12_33_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-12-121"},{"key":"e_1_2_12_34_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.5412"},{"issue":"1","key":"e_1_2_12_35_1","first-page":"200","article-title":"Ruling out deep venous thrombosis in primary care. A simple diagnostic algorithm including D\u2010dimer testing","volume":"94","author":"Oudega R","year":"2005","journal-title":"Thrombosis and Haemostasis"},{"key":"e_1_2_12_36_1","doi-asserted-by":"publisher","DOI":"10.1093\/fampra\/cmm075"},{"key":"e_1_2_12_37_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.e5900"},{"key":"e_1_2_12_38_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.e3186"},{"key":"e_1_2_12_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2012.06.020"},{"key":"e_1_2_12_40_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0040249"},{"key":"e_1_2_12_41_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0032844"},{"key":"e_1_2_12_42_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.38446.498542.8F"},{"key":"e_1_2_12_43_1","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2001.0080391"},{"key":"e_1_2_12_44_1","doi-asserted-by":"publisher","DOI":"10.1186\/1741\u20107015\u20102\u201023"},{"key":"e_1_2_12_45_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1001221"},{"key":"e_1_2_12_46_1","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-144-6-200603210-00010"},{"key":"e_1_2_12_47_1","unstructured":"FawcettT.ROC graphs: Notes and practical considerations for researchers.Tech. Rep. HPL\u20102003\u20104 HP Labs Tech Report 2004."},{"key":"e_1_2_12_48_1","doi-asserted-by":"publisher","DOI":"10.1148\/radiology.143.1.7063747"},{"key":"e_1_2_12_49_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X9101100204"},{"key":"e_1_2_12_50_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X9301300107"},{"key":"e_1_2_12_51_1","doi-asserted-by":"publisher","DOI":"10.1097\/EDE.0b013e3181c30fb2"},{"key":"e_1_2_12_52_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/45.3-4.562"},{"issue":"2","key":"e_1_2_12_53_1","first-page":"3491","article-title":"Multivariate classification rules: calibration and discrimination","volume":"5","author":"Schmid CH","year":"2005","journal-title":"Encyclopedia of Biostatistics"},{"key":"e_1_2_12_54_1","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-143-2-200507190-00008"},{"key":"e_1_2_12_55_1","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1009212519"},{"key":"e_1_2_12_56_1","doi-asserted-by":"publisher","DOI":"10.1093\/pan\/mpq001"},{"key":"e_1_2_12_57_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4"},{"key":"e_1_2_12_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-3462-1"},{"key":"e_1_2_12_59_1","doi-asserted-by":"publisher","DOI":"10.1001\/archinte.162.8.907"},{"key":"e_1_2_12_60_1","doi-asserted-by":"crossref","unstructured":"MonteithK CarrollJL SEppiK MartinezT.Turning Bayesian model averaging into Bayesian model combination Proceedings of the 2011 International Joint Conference on Neural Networks San Jose California 2011;2657\u20132663.","DOI":"10.1109\/IJCNN.2011.6033566"},{"key":"e_1_2_12_61_1","unstructured":"MinkaT.Bayesian model averaging is not model combination.Technical report MIT 2000."},{"key":"e_1_2_12_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00117832"},{"issue":"1","key":"e_1_2_12_63_1","first-page":"article 25","article-title":"Super learner","volume":"6","author":"Laan MJ","year":"2007","journal-title":"Statistical Applications in Genetics and Molecular Biology"},{"key":"e_1_2_12_64_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80023-1"},{"key":"e_1_2_12_65_1","doi-asserted-by":"publisher","DOI":"10.1137\/0916069"},{"key":"e_1_2_12_66_1","first-page":"140","volume-title":"An Example of Dependencies among Variables in a Conditional Logistic Regression","author":"Davis C","year":"1986"},{"key":"e_1_2_12_67_1","doi-asserted-by":"publisher","DOI":"10.2307\/1403683"},{"issue":"1","key":"e_1_2_12_68_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/1536867X0100100101","article-title":"Flexible parametric alternatives to the Cox model, and more","volume":"1","author":"Royston P","year":"2001","journal-title":"The Stata Journal"},{"key":"e_1_2_12_69_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.3618"},{"key":"e_1_2_12_70_1","doi-asserted-by":"publisher","DOI":"10.1002\/9780470510445"},{"key":"e_1_2_12_71_1","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316696"},{"key":"e_1_2_12_72_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2004.01.020"},{"key":"e_1_2_12_73_1","article-title":"Exclusion of deep vein thrombosis using the wells\u2010rule in various clinically important subgroups; an individual patient data meta\u2010analysis","author":"Geersing GJ","year":"2013","journal-title":"British Medical Journal"},{"key":"e_1_2_12_74_1","doi-asserted-by":"publisher","DOI":"10.1089\/neu.2006.0036"},{"key":"e_1_2_12_75_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5"},{"key":"e_1_2_12_76_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b606"},{"key":"e_1_2_12_77_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2007.02.004"},{"key":"e_1_2_12_78_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(20000130)19:2<141::AID-SIM334>3.0.CO;2-O"},{"key":"e_1_2_12_79_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-0258(20000430)19:8<1059::AID-SIM412>3.0.CO;2-0"},{"key":"e_1_2_12_80_1","first-page":"223","volume-title":"In Proceedings of the 17th International Conference on Machine Learning","author":"Domingos P","year":"2000"},{"key":"e_1_2_12_81_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2331"}],"container-title":["Statistics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fsim.6080","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.6080","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,4]],"date-time":"2023-10-04T23:26:56Z","timestamp":1696462016000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sim.6080"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,1,14]]},"references-count":80,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2014,6,30]]}},"alternative-id":["10.1002\/sim.6080"],"URL":"https:\/\/doi.org\/10.1002\/sim.6080","archive":["Portico"],"relation":{},"ISSN":["0277-6715","1097-0258"],"issn-type":[{"value":"0277-6715","type":"print"},{"value":"1097-0258","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,1,14]]},"assertion":[{"value":"2014-01-14","name":"published_online","label":"Published Online","group":{"name":"publication_history","label":"Publication History"}}]}}