{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T16:50:40Z","timestamp":1790355040852,"version":"4.1.0"},"reference-count":51,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2023,2,6]],"date-time":"2023-02-06T00:00:00Z","timestamp":1675641600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01\u2010DA046620"],"award-info":[{"award-number":["R01\u2010DA046620"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Addiction"],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background and aims<\/jats:title><jats:p>Low outcome prevalence, often observed with opioid\u2010related outcomes, poses an underappreciated challenge to accurate predictive modeling. Outcome class imbalance, where non\u2010events (i.e. negative class observations) outnumber events (i.e. positive class observations) by a moderate to extreme degree, can distort measures of predictive accuracy in misleading ways, and make the overall predictive accuracy and the discriminatory ability of a predictive model appear spuriously high. We conducted a simulation study to measure the impact of outcome class imbalance on predictive performance of a simple SuperLearner ensemble model and suggest strategies for reducing that impact.<\/jats:p><\/jats:sec><jats:sec><jats:title>Design, Setting, Participants<\/jats:title><jats:p>Using a Monte Carlo design with 250 repetitions, we trained and evaluated these models on four simulated data sets with 100\u2009000 observations each: one with perfect balance between events and non\u2010events, and three where non\u2010events outnumbered events by an approximate factor of 10:1, 100:1, and 1000:1, respectively.<\/jats:p><\/jats:sec><jats:sec><jats:title>Measurements<\/jats:title><jats:p>We evaluated the performance of these models using a comprehensive suite of measures, including measures that are more appropriate for imbalanced data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Findings<\/jats:title><jats:p>Increasing imbalance tended to spuriously improve overall accuracy (using a high threshold to classify events vs non\u2010events, overall accuracy improved from 0.45 with perfect balance to 0.99 with the most severe outcome class imbalance), but diminished predictive performance was evident using other metrics (corresponding positive predictive value decreased from 0.99 to 0.14).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Increasing reliance on algorithmic risk scores in consequential decision\u2010making processes raises critical fairness and ethical concerns. This paper provides broad guidance for analytic strategies that clinical investigators can use to remedy the impacts of outcome class imbalance on risk prediction tools.<\/jats:p><\/jats:sec>","DOI":"10.1111\/add.16133","type":"journal-article","created":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T04:11:28Z","timestamp":1674447088000},"page":"1167-1176","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["Outcome class imbalance and rare events: An underappreciated complication for overdose risk prediction modeling"],"prefix":"10.1111","volume":"118","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5794-0146","authenticated-orcid":false,"given":"Abigail R.","family":"Cartus","sequence":"first","affiliation":[{"name":"Department of Epidemiology Brown University School of Public Health  Providence Rhode Island USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elizabeth A.","family":"Samuels","sequence":"additional","affiliation":[{"name":"Department of Epidemiology Brown University School of Public Health  Providence Rhode Island USA"},{"name":"Department of Emergency Medicine Alpert Medical School of Brown University  Providence Rhode Island USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8441-7046","authenticated-orcid":false,"given":"Magdalena","family":"Cerd\u00e1","sequence":"additional","affiliation":[{"name":"Division of Epidemiology, Department of Population Health, Center for Opioid Epidemiology and Policy, School of Medicine New York University  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0134-7052","authenticated-orcid":false,"given":"Brandon D. L.","family":"Marshall","sequence":"additional","affiliation":[{"name":"Department of Epidemiology Brown University School of Public Health  Providence Rhode Island USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2023,2,6]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.25538"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2021.0074"},{"key":"e_1_2_8_4_1","doi-asserted-by":"crossref","unstructured":"HedegaardH Mini\u00f1oA SpencerM WarnerM.Drug overdose deaths in the United States 1999\u20132020. NCHS Data Brief No. 428. National Center for Health Statistics.2021.","DOI":"10.15620\/cdc:112340"},{"key":"e_1_2_8_5_1","volume-title":"Drug overdose deaths in the US top 100,000 annually","author":"Control CfD, Prevention","year":"2021"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amepre.2019.07.026"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1080\/10826084.2020.1868520"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.drugalcdep.2019.04.016"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0000000000001389"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamapsychiatry.2020.1689"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-017-4288-3"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13040-019-0193-0"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103725"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1093\/pm\/pnx009"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0241083"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.drugalcdep.2014.02.701"},{"key":"e_1_2_8_17_1","first-page":"389","article-title":"Machine learning based opioid overdose prediction using electronic health records","volume":"2019","author":"Dong X","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1093\/pm\/pnaa026"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2019.0968"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0235981"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.37765\/ajmc.2021.88617"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1111\/pme.12777"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1093\/jnci\/djz200"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.9778\/cmajo.20200076"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jtcvs.2020.06.052"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-021-01302-4"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1177\/0962280220980484"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0000000000000384"},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2008.239"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.106.672402"},{"key":"e_1_2_8_31_1","unstructured":"SzalavitzMThe pain was unbeearable. So why did doctors turn her away?WIRED.2021."},{"key":"e_1_2_8_32_1","unstructured":"HuziengaJEBB PatelVR SpeightsDB.NARxCHECK score as a predictor of unintentional overdose death Appriss Health.2016."},{"key":"e_1_2_8_33_1","doi-asserted-by":"crossref","unstructured":"KilbyAE Algorithmic fairness in predicting opioid use disorder using machine learning. Proceedings of the 2021 ACM Conference on Fairness Accountability and Transparency;2021.","DOI":"10.1145\/3442188.3445891"},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1097\/j.pain.0000000000001078"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.trsl.2021.03.012"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v028.i05"},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2009.11.009"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjstel-2016-000124"},{"key":"e_1_2_8_39_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0118432"},{"key":"e_1_2_8_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6849-3"},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2017.05.008"},{"key":"e_1_2_8_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2016.2606104"},{"key":"e_1_2_8_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2014.2372060"},{"key":"e_1_2_8_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-011-0465-6"},{"key":"e_1_2_8_45_1","doi-asserted-by":"publisher","DOI":"10.7763\/IJMLC.2013.V3.307"},{"key":"e_1_2_8_46_1","doi-asserted-by":"crossref","unstructured":"SantosoB WijayantoH NotodiputroK SartonoB editors.Synthetic over sampling methods for handling class imbalanced problems: A review. IOP conference series: earth and environmental science; 2017: IOP Publishing.","DOI":"10.1088\/1755-1315\/58\/1\/012031"},{"key":"e_1_2_8_47_1","doi-asserted-by":"publisher","DOI":"10.3390\/app10041276"},{"key":"e_1_2_8_48_1","first-page":"231","article-title":"Cost\u2010sensitive learning and the class imbalance problem","volume":"2011","author":"Ling CX","year":"2008","journal-title":"Encyclopedia of Mach Learn"},{"key":"e_1_2_8_49_1","volume-title":"Automating inequality: how high\u2010tech tools profile, police, and punish the poor: St","author":"Eubanks V","year":"2018"},{"key":"e_1_2_8_50_1","doi-asserted-by":"publisher","DOI":"10.18574\/nyu\/9781479833641.001.0001"},{"key":"e_1_2_8_51_1","doi-asserted-by":"crossref","unstructured":"OlivaJD.Dosing Discrimination: Regulating PDMP Risk Scores.2021 https:\/\/doi.org\/10.2139\/ssrn.3768774","DOI":"10.2139\/ssrn.3768774"},{"key":"e_1_2_8_52_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2021.11013"}],"container-title":["Addiction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/add.16133","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/add.16133","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/add.16133","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,21]],"date-time":"2023-09-21T09:43:26Z","timestamp":1695289406000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/add.16133"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,6]]},"references-count":51,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["10.1111\/add.16133"],"URL":"https:\/\/doi.org\/10.1111\/add.16133","archive":["Portico"],"relation":{},"ISSN":["0965-2140","1360-0443"],"issn-type":[{"value":"0965-2140","type":"print"},{"value":"1360-0443","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,6]]},"assertion":[{"value":"2022-04-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-22","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}