{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T14:55:06Z","timestamp":1786978506534,"version":"build-2736575974"},"reference-count":48,"publisher":"Wiley","issue":"23","license":[{"start":{"date-parts":[[2020,6,24]],"date-time":"2020-06-24T00:00:00Z","timestamp":1592956800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100000030","name":"Centers for Disease Control and Prevention","doi-asserted-by":"publisher","award":["SSuN, CDC\u2010RFA\u2010PS13\u20101306"],"award-info":[{"award-number":["SSuN, CDC\u2010RFA\u2010PS13\u20101306"]}],"id":[{"id":"10.13039\/100000030","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007301","name":"Harvard University Center for AIDS Research","doi-asserted-by":"publisher","award":["P30 AI060354"],"award-info":[{"award-number":["P30 AI060354"]}],"id":[{"id":"10.13039\/100007301","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["K23 MH098795"],"award-info":[{"award-number":["K23 MH098795"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Statistics in Medicine"],"published-print":{"date-parts":[[2020,10,15]]},"abstract":"<jats:p>\n                    Human immunodeficiency virus (HIV) pre\u2010exposure prophylaxis (PrEP) protects high risk patients from becoming infected with HIV. Clinicians need help to identify candidates for PrEP based on information routinely collected in electronic health records (EHRs). The greatest statistical challenge in developing a risk prediction model is that acquisition is extremely rare.\n                    <jats:bold>Methods<\/jats:bold>\n                    : Data consisted of 180 covariates (demographic, diagnoses, treatments, prescriptions) extracted from records on 399\u2009385 patient (150 cases) seen at Atrius Health (2007\u20102015), a clinical network in Massachusetts. Super learner is an ensemble machine learning algorithm that uses\n                    <jats:italic>k<\/jats:italic>\n                    \u2010fold cross validation to evaluate and combine predictions from a collection of algorithms. We trained 42 variants of sophisticated algorithms, using different sampling schemes that more evenly balanced the ratio of cases to controls. We compared super learner's cross validated area under the receiver operating curve (cv\u2010AUC) with that of each individual algorithm.\n                    <jats:bold>Results<\/jats:bold>\n                    : The least absolute shrinkage and selection operator (lasso) using a 1:20 class ratio outperformed the super learner (cv\u2010AUC = 0.86 vs 0.84). A traditional logistic regression model restricted to 23 clinician\u2010selected main terms was slightly inferior (cv\u2010AUC = 0.81).\n                    <jats:bold>Conclusion<\/jats:bold>\n                    : Machine learning was successful at developing a model to predict 1\u2010year risk of acquiring HIV based on a physician\u2010curated set of predictors extracted from \nEHRs.\n                  <\/jats:p>","DOI":"10.1002\/sim.8591","type":"journal-article","created":{"date-parts":[[2020,6,24]],"date-time":"2020-06-24T05:39:21Z","timestamp":1592977161000},"page":"3059-3073","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Using electronic health records to identify candidates for human immunodeficiency virus pre\u2010exposure prophylaxis: An application of super learning to risk prediction when the outcome is rare"],"prefix":"10.1002","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0127-0099","authenticated-orcid":false,"given":"Susan","family":"Gruber","sequence":"first","affiliation":[{"name":"Putnam Data Sciences, LLC  Cambridge Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Douglas","family":"Krakower","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases Beth Israel Deaconess Medical Center  Boston Massachusetts USA"},{"name":"The Fenway Institute Fenway Health  Boston Massachusetts USA"},{"name":"Harvard Medical School  Boston Massachusetts USA"},{"name":"Department of Population Medicine Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John T.","family":"Menchaca","sequence":"additional","affiliation":[{"name":"Department of Population Medicine Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katherine","family":"Hsu","sequence":"additional","affiliation":[{"name":"Massachusetts Department of Public Health  Boston Massachusetts USA"},{"name":"Department of Pediatrics Boston Medical Center  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca","family":"Hawrusik","sequence":"additional","affiliation":[{"name":"Massachusetts Department of Public Health  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Judith C.","family":"Maro","sequence":"additional","affiliation":[{"name":"Department of Population Medicine Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noelle M.","family":"Cocoros","sequence":"additional","affiliation":[{"name":"Department of Population Medicine Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin A.","family":"Kruskal","sequence":"additional","affiliation":[{"name":"Atrius Health  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ira B.","family":"Wilson","sequence":"additional","affiliation":[{"name":"Department of Health Services, Policy and Practice Brown University  Providence Rhode Island USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kenneth H.","family":"Mayer","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases Beth Israel Deaconess Medical Center  Boston Massachusetts USA"},{"name":"The Fenway Institute Fenway Health  Boston Massachusetts USA"},{"name":"Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Klompas","sequence":"additional","affiliation":[{"name":"Department of Population Medicine Harvard Medical School  Boston Massachusetts USA"},{"name":"Division of Infectious Diseases Brigham and Women's Hospital  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2020,6,24]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa1108524"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1097\/QAI.0000000000001129"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2352-3018(19)30139-0"},{"key":"e_1_2_8_5_1","unstructured":"Centers for Disease Control and PreventionPreexposure prophylaxis for the prevention of HIV infection in the United States \u2010 2017 Update: clinical providers' supplement;2018.https:\/\/www.cdc.gov\/hiv\/pdf\/risk\/prep\u2010cdc\u2010hiv\u2010prep\u2010provider\u2010supplement\u20102017.pdf. Accessed November 12 2019."},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwr389"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1097\/OLQ.0b013e3181a9cc41"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1097\/QAI.0b013e318256b2f6"},{"issue":"25","key":"e_1_2_8_9_1","first-page":"1","article-title":"Super learner","volume":"6","author":"Laan M","year":"2007","journal-title":"Stat Appl Genet Mol Biol"},{"key":"e_1_2_8_10_1","unstructured":"PolleyEC van derLaanMJ. Super learner in prediction. Tech. Rep. 200 U.C. Berkeley Division of Biostatistics Working Paper Series;2010."},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1097\/TA.0b013e3182914553"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kws241"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2213-2600(14)70239-5"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.7296"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/ciz1096"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1524\/stnd.2006.24.3.351"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.594"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80023-1"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00117832"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.2776"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1214\/09-SS054"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.6322"},{"key":"e_1_2_8_23_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"R Core Team","year":"2016"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v033.i01"},{"key":"e_1_2_8_25_1","volume-title":"The Elements of Statistical Learning","author":"Hastie T","year":"2002"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-21706-2"},{"key":"e_1_2_8_27_1","volume-title":"Classification and Regression Trees","author":"Breiman L","year":"1984"},{"issue":"3","key":"e_1_2_8_28_1","first-page":"18","article-title":"Classification and regression by random forest","volume":"2","author":"Liaw A","year":"2002","journal-title":"R News"},{"key":"e_1_2_8_29_1","unstructured":"DimitriadouE HornikK LeischF MeyerD WeingesselA. e1071: Misc Functions of the Department of Statistics (e1071) TU Wien. R Package Version 1.6\u20107;2010."},{"key":"e_1_2_8_30_1","unstructured":"BerwickR.An idiot's guide to support vector machines (SVMs);2003.http:\/\/web.mit.edu\/6.034\/wwwbob\/svm.pdf. Accessed January 11 2017."},{"key":"e_1_2_8_31_1","first-page":"1","volume-title":"Regularization, Optimization, Kernels, and Support Vector Machines","author":"Jaggi M","year":"2014"},{"key":"e_1_2_8_32_1","first-page":"13","volume-title":"Proceedings of the 1st International Conference on Advanced Data and Information Engineering (DaEng\u20102013)","author":"Yap B","year":"2014"},{"key":"e_1_2_8_33_1","first-page":"1","volume-title":"Using Random Forest to Learn Imbalanced Data","author":"Chen C","year":"2004"},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1093\/oxfordjournals.pan.a004868"},{"key":"e_1_2_8_35_1","volume-title":"Statistics for Epidemiology","author":"Jewell N","year":"2004"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2015.10.005"},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.1136\/sextrans-2016-052865"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1097\/QAD.0000000000002111"},{"key":"e_1_2_8_39_1","unstructured":"Centers for Disease Control and PreventionDiagnoses of HIV Infection in the United States and dependent areas 2016. HIV Surveillance Report 2016http:\/\/www.cdc.gov\/hiv\/library\/reports\/hiv\u2010surveillance.html. Accessed November 12 2019."},{"key":"e_1_2_8_40_1","unstructured":"PolleyE.SuperLearner: super learner prediction. 2016. R Package Version 2.0\u201019 http:\/\/CRAN.R\u2010project.org\/package=SuperLearner."},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1214\/15-EJS1035"},{"key":"e_1_2_8_42_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"e_1_2_8_43_1","doi-asserted-by":"crossref","unstructured":"CaruanaR Niculescu\u2010MizilA. An empirical comparison of supervised learning algorithms. Paper presented at: Proceedings of the ICML '06. Association for Computing Machinery;2006:161\u2010168; New York NY.","DOI":"10.1145\/1143844.1143865"},{"key":"e_1_2_8_44_1","unstructured":"ChenT HeT BenestyM et al. xgboost: extreme gradient boosting. R Package Version 0.81.0.1;2019.https:\/\/CRAN.R\u2010project.org\/package=xgboost."},{"key":"e_1_2_8_45_1","doi-asserted-by":"crossref","unstructured":"FritschS GuentherF WrightMN. Neuralnet: training of neural networks. R Package Version 1.44.2;2019.https:\/\/CRAN.R\u2010project.org\/package=neuralnet.","DOI":"10.32614\/RJ-2010-006"},{"key":"e_1_2_8_46_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"issue":"363","key":"e_1_2_8_47_1","first-page":"1","article-title":"Joint use of over\u2010 and under\u2010sampling techniques and cross\u2010validation for the development and assessment of prediction models","volume":"16","author":"Blagus R","year":"2015","journal-title":"BMC Bioinform"},{"key":"e_1_2_8_48_1","doi-asserted-by":"publisher","DOI":"10.7763\/IJMLC.2013.V3.307"},{"key":"e_1_2_8_49_1","doi-asserted-by":"publisher","DOI":"10.1148\/radiology.143.1.7063747"}],"container-title":["Statistics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fsim.8591","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.8591","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/sim.8591","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.8591","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,4]],"date-time":"2023-09-04T10:45:40Z","timestamp":1693824340000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sim.8591"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,24]]},"references-count":48,"journal-issue":{"issue":"23","published-print":{"date-parts":[[2020,10,15]]}},"alternative-id":["10.1002\/sim.8591"],"URL":"https:\/\/doi.org\/10.1002\/sim.8591","archive":["Portico"],"relation":{},"ISSN":["0277-6715","1097-0258"],"issn-type":[{"value":"0277-6715","type":"print"},{"value":"1097-0258","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,24]]},"assertion":[{"value":"2019-05-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-05-07","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-06-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}