{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T19:02:46Z","timestamp":1776193366647,"version":"3.50.1"},"reference-count":69,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T00:00:00Z","timestamp":1711152000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["1R01LM014239"],"award-info":[{"award-number":["1R01LM014239"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,4,19]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objectives<\/jats:title><jats:p>Leveraging artificial intelligence (AI) in conjunction with electronic health records (EHRs) holds transformative potential to improve healthcare. However, addressing bias in AI, which risks worsening healthcare disparities, cannot be overlooked. This study reviews methods to handle various biases in AI models developed using EHR data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines, analyzing articles from PubMed, Web of Science, and IEEE published between January 01, 2010 and December 17, 2023. The review identified key biases, outlined strategies for detecting and mitigating bias throughout the AI model development, and analyzed metrics for bias assessment.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Of the 450 articles retrieved, 20 met our criteria, revealing 6 major bias types: algorithmic, confounding, implicit, measurement, selection, and temporal. The AI models were primarily developed for predictive tasks, yet none have been deployed in real-world healthcare settings. Five studies concentrated on the detection of implicit and algorithmic biases employing fairness metrics like statistical parity, equal opportunity, and predictive equity. Fifteen studies proposed strategies for mitigating biases, especially targeting implicit and selection biases. These strategies, evaluated through both performance and fairness metrics, predominantly involved data collection and preprocessing techniques like resampling and reweighting.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>This review highlights evolving strategies to mitigate bias in EHR-based AI models, emphasizing the urgent need for both standardized and detailed reporting of the methodologies and systematic real-world testing and evaluation. Such measures are essential for gauging models\u2019 practical impact and fostering ethical AI that ensures fairness and equity in healthcare.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocae060","type":"journal-article","created":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T19:21:35Z","timestamp":1711221695000},"page":"1172-1183","source":"Crossref","is-referenced-by-count":126,"title":["Unmasking bias in artificial intelligence: a systematic review of bias detection and mitigation strategies in electronic health record-based models"],"prefix":"10.1093","volume":"31","author":[{"given":"Feng","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Harvard Medical School , Boston, MA 02115, United States"},{"name":"Department of Biomedical Informatics and Health Education, University of Washington , Seattle, WA 98105, United States"}]},{"given":"Liqin","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Harvard Medical School , Boston, MA 02115, United States"},{"name":"Division of General Internal Medicine and Primary Care, Brigham and Women\u2019s Hospital , Boston, MA 02115, United States"}]},{"given":"Julie","family":"Hong","sequence":"additional","affiliation":[{"name":"Wellesley High School , Wellesley, MA 02481, United States"}]},{"given":"Jiaqi","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Harvard Medical School , Boston, MA 02115, United States"}]},{"given":"Li","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Harvard Medical School , Boston, MA 02115, United States"},{"name":"Division of General Internal Medicine and Primary Care, Brigham and Women\u2019s Hospital , Boston, MA 02115, United States"}]}],"member":"286","published-online":{"date-parts":[[2024,3,23]]},"reference":[{"issue":"8","key":"2024041923351959300_ocae060-B1","doi-asserted-by":"crossref","first-page":"1416","DOI":"10.1377\/hlthaff.2016.1651","article-title":"HITECH act drove large gains in hospital electronic health record adoption","volume":"36","author":"Adler-Milstein","year":"2017","journal-title":"Health Aff (Millwood)"},{"issue":"1","key":"2024041923351959300_ocae060-B2","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat Med"},{"key":"2024041923351959300_ocae060-B3","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.ebiom.2019.07.019","article-title":"Artificial intelligence to support clinical decision-making processes","volume":"46","author":"Garcia-Vidal","year":"2019","journal-title":"EBioMedicine"},{"issue":"12","key":"2024041923351959300_ocae060-B4","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1167\/tvst.12.12.2","article-title":"Reducing ophthalmic health disparities through transfer learning: a novel application to overcome data inequality","volume":"12","author":"Lee","year":"2023","journal-title":"Transl Vis Sci Technol"},{"key":"2024041923351959300_ocae060-B5","first-page":"2612","author":"Hee","year":"2017"},{"issue":"1","key":"2024041923351959300_ocae060-B6","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/s43856-021-00028-w","article-title":"Mitigating bias in machine learning for medicine","volume":"1","author":"Vokinger","year":"2021","journal-title":"Commun Med (Lond)"},{"issue":"10","key":"2024041923351959300_ocae060-B7","doi-asserted-by":"crossref","first-page":"100347","DOI":"10.1016\/j.patter.2021.100347","article-title":"Addressing bias in big data and AI for health care: a call for open science","volume":"2","author":"Norori","year":"2021","journal-title":"Patterns"},{"key":"2024041923351959300_ocae060-B8","author":"Miko\u0142ajczyk-Bare\u0142a"},{"issue":"1","key":"2024041923351959300_ocae060-B9","doi-asserted-by":"crossref","first-page":"4581","DOI":"10.1038\/s41467-022-32186-3","article-title":"Addressing fairness in artificial intelligence for medical imaging","volume":"13","author":"Ferrante RLMER E","year":"2022","journal-title":"Nat Commun"},{"issue":"1","key":"2024041923351959300_ocae060-B10","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.cell.2019.02.039","article-title":"Personalized medicine and the power of electronic health records","volume":"177","author":"Abul-Husn","year":"2019","journal-title":"Cell"},{"issue":"1","key":"2024041923351959300_ocae060-B11","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1186\/s13104-022-05911-w","article-title":"A multi-step approach to managing missing data in time and patient variant electronic health records","volume":"15","author":"Cesare","year":"2022","journal-title":"BMC Res Notes"},{"issue":"1","key":"2024041923351959300_ocae060-B12","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1038\/s41591-021-01614-0","article-title":"AI in health and medicine","volume":"28","author":"Rajpurkar","year":"2022","journal-title":"Nat Med"},{"issue":"9","key":"2024041923351959300_ocae060-B13","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1038\/s41591-019-0548-6","article-title":"Do no harm: a roadmap for responsible machine learning for health care","volume":"25","author":"Wiens","year":"2019","journal-title":"Nat Med"},{"issue":"5","key":"2024041923351959300_ocae060-B14","doi-asserted-by":"crossref","first-page":"e36388","DOI":"10.2196\/36388","article-title":"Evaluation and mitigation of racial bias in clinical machine learning models: scoping review","volume":"10","author":"Huang","year":"2022","journal-title":"JMIR Med Inform"},{"issue":"3","key":"2024041923351959300_ocae060-B15","doi-asserted-by":"crossref","first-page":"e0000022","DOI":"10.1371\/journal.pdig.0000022","article-title":"Sources of bias in artificial intelligence that perpetuate healthcare disparities\u2014a global review","volume":"1","author":"Celi","year":"2022","journal-title":"PLOS Digital Health"},{"key":"2024041923351959300_ocae060-B16","doi-asserted-by":"crossref","first-page":"i4919","DOI":"10.1136\/bmj.i4919","article-title":"ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions","volume":"355","author":"Sterne","year":"2016","journal-title":"BMJ"},{"issue":"1","key":"2024041923351959300_ocae060-B17","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1186\/s13643-018-0915-2","article-title":"The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures","volume":"7","author":"Bero","year":"2018","journal-title":"Syst Rev"},{"issue":"1","key":"2024041923351959300_ocae060-B18","doi-asserted-by":"crossref","first-page":"51","DOI":"10.7326\/M18-1376","article-title":"PROBAST: a tool to assess the risk of bias and applicability of prediction model studies","volume":"170","author":"Wolff","year":"2019","journal-title":"Ann Int Med"},{"issue":"6","key":"2024041923351959300_ocae060-B19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3457607","article-title":"A survey on bias and fairness in machine learning","volume":"54","author":"Mehrabi","year":"2021","journal-title":"ACM Comput Surv"},{"key":"2024041923351959300_ocae060-B20","article-title":"Fairness in machine learning: a survey","author":"Caton","year":"2020","journal-title":"ACM Comput Surv"},{"key":"2024041923351959300_ocae060-B21","author":"Aghaei","year":"2019"},{"issue":"1","key":"2024041923351959300_ocae060-B22","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s12910-017-0179-8","article-title":"Implicit bias in healthcare professionals: a systematic review","volume":"18","author":"FitzGerald","year":"2017","journal-title":"BMC Med Ethics"},{"issue":"4","key":"2024041923351959300_ocae060-B23","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1023\/A:1009589812697","article-title":"Sampling bias and other methodological threats to the validity of health survey research","volume":"7","author":"Johnson","year":"2000","journal-title":"Int J Stress Manag"},{"issue":"4","key":"2024041923351959300_ocae060-B24","doi-asserted-by":"crossref","first-page":"e23","DOI":"10.1097\/MLR.0000000000000011","article-title":"Distinguishing selection bias and confounding bias in comparative effectiveness research","volume":"54","author":"Haneuse","year":"2016","journal-title":"Med Care"},{"issue":"11","key":"2024041923351959300_ocae060-B25","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.jclinepi.2009.03.013","article-title":"Formal definitions of measurement bias and explanation bias clarify measurement and conceptual perspectives on response shift","volume":"62","author":"Oort","year":"2009","journal-title":"J Clin Epidemiol"},{"issue":"4","key":"2024041923351959300_ocae060-B26","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1007\/s43681-022-00138-8","article-title":"AI bias: exploring discriminatory algorithmic decision-making models and the application of possible machine-centric solutions adapted from the pharmaceutical industry","volume":"2","author":"Belenguer","year":"2022","journal-title":"AI Ethics"},{"issue":"7","key":"2024041923351959300_ocae060-B27","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.1093\/jamia\/ocac052","article-title":"Assessing socioeconomic bias in machine learning algorithms in health care: a case study of the HOUSES index","volume":"29","author":"Juhn","year":"2022","journal-title":"J Am Med Inform Assoc"},{"issue":"15","key":"2024041923351959300_ocae060-B28","doi-asserted-by":"crossref","first-page":"4342","DOI":"10.3390\/jcm11154342","article-title":"Prediction of influenza complications: development and validation of a machine learning prediction model to improve and expand the identification of vaccine-hesitant patients at risk of severe influenza complications","volume":"11","author":"Wolk","year":"2022","journal-title":"J Clin Med"},{"key":"2024041923351959300_ocae060-B29","first-page":"64","author":"Khoshnevisan","year":"2020"},{"key":"2024041923351959300_ocae060-B30","doi-asserted-by":"crossref","first-page":"104294","DOI":"10.1016\/j.jbi.2023.104294","article-title":"Evaluating and mitigating bias in machine learning models for cardiovascular disease prediction","volume":"138","author":"Li","year":"2023","journal-title":"J Biomed Inform"},{"issue":"1","key":"2024041923351959300_ocae060-B31","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1038\/s41597-021-01110-7","article-title":"Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model","volume":"9","author":"R\u00f6\u00f6sli","year":"2022","journal-title":"Sci Data"},{"key":"2024041923351959300_ocae060-B32","author":"Karlsson","year":"2014"},{"key":"2024041923351959300_ocae060-B33","first-page":"4571","author":"Zhu","year":"2017"},{"key":"2024041923351959300_ocae060-B34","doi-asserted-by":"crossref","first-page":"970281","DOI":"10.3389\/fdgth.2022.970281","article-title":"Fairness in the prediction of acute postoperative pain using machine learning models","volume":"4","author":"Davoudi","year":"2022","journal-title":"Front Digit Health"},{"key":"2024041923351959300_ocae060-B35","author":"Raza","year":"2023"},{"issue":"4","key":"2024041923351959300_ocae060-B36","doi-asserted-by":"crossref","first-page":"e22400","DOI":"10.2196\/22400","article-title":"A racially unbiased, machine learning approach to prediction of mortality: algorithm development study","volume":"6","author":"Allen","year":"2020","journal-title":"JMIR Public Health Surveill"},{"key":"2024041923351959300_ocae060-B37","first-page":"291","article-title":"Timeline registration for electronic health records","volume":"2023","author":"Jiang","year":"2023","journal-title":"AMIA Summits on Transl Sci Proc"},{"issue":"1","key":"2024041923351959300_ocae060-B38","doi-asserted-by":"crossref","first-page":"7166","DOI":"10.1038\/s41598-022-11012-2","article-title":"Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset","volume":"12","author":"Meng","year":"2022","journal-title":"Sci Rep"},{"key":"2024041923351959300_ocae060-B39","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1186\/s12911-022-01871-0","article-title":"Comparison between machine learning methods for mortality prediction for sepsis patients with different social determinants","volume":"22(Suppl 2)","author":"Wang","year":"2022","journal-title":"BMC Med Inform Decis Mak"},{"key":"2024041923351959300_ocae060-B40","doi-asserted-by":"crossref","first-page":"104545","DOI":"10.1016\/j.jbi.2023.104545","article-title":"A transformer-based deep learning approach for fairly predicting post-liver transplant risk factors","volume":"149","author":"Li","year":"2024","journal-title":"J Biomed Inform"},{"issue":"11","key":"2024041923351959300_ocae060-B41","first-page":"13235","article-title":"Bipartite ranking fairness through a model agnostic ordering adjustment","volume":"45","author":"Cui","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2024041923351959300_ocae060-B42","first-page":"1","author":"Huda","year":"2019"},{"issue":"2","key":"2024041923351959300_ocae060-B43","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1097\/EDE.0000000000001578","article-title":"Performance of multiple imputation using modern machine learning methods in electronic health records data","volume":"34","author":"Getz","year":"2022","journal-title":"Epidemiology"},{"key":"2024041923351959300_ocae060-B44","first-page":"1","article-title":"PATNet: propensity-adjusted temporal network for joint imputation and prediction using binary EHRs with observation bias","author":"Yin","year":"2023","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2024041923351959300_ocae060-B45","first-page":"214","author":"Dwork","year":"2012"},{"issue":"1","key":"2024041923351959300_ocae060-B46","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"NPJ Digit Med"},{"issue":"4","key":"2024041923351959300_ocae060-B47","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1001\/jama.2022.23867","article-title":"Prevention of bias and discrimination in clinical practice algorithms","volume":"329","author":"Shachar","year":"2023","journal-title":"JAMA"},{"key":"2024041923351959300_ocae060-B48","article-title":"Bias in data-driven artificial intelligence systems\u2014an introductory survey","author":"Ntoutsi","year":"2020","journal-title":"Wiley Interdiscip Rev: Data Min Knowl Discov"},{"issue":"11","key":"2024041923351959300_ocae060-B49","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1001\/jamainternmed.2018.3763","article-title":"Potential biases in machine learning algorithms using electronic health record data","volume":"178","author":"Gianfrancesco","year":"2018","journal-title":"JAMA Intern Med"},{"issue":"3","key":"2024041923351959300_ocae060-B50","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1097\/EDE.0000000000001338","article-title":"Deep learning-based propensity scores for confounding control in comparative effectiveness research: a large-scale, real-world data study","volume":"32","author":"Weberpals","year":"2021","journal-title":"Epidemiology"},{"key":"2024041923351959300_ocae060-B51","first-page":"1086","author":"Mi","year":"2020"},{"key":"2024041923351959300_ocae060-B52","doi-asserted-by":"crossref","first-page":"20552076231178577","DOI":"10.1177\/20552076231178577","article-title":"Benzodiazepine-related dementia risks and protopathic biases revealed by multiple-kernel learning with electronic medical records","volume":"9","author":"Hayakawa","year":"2023","journal-title":"Digit Health"},{"issue":"1","key":"2024041923351959300_ocae060-B53","doi-asserted-by":"crossref","first-page":"11654","DOI":"10.1038\/s41598-022-15245-z","article-title":"Temporal quality degradation in AI models","volume":"12","author":"Vela","year":"2022","journal-title":"Sci Rep"},{"issue":"1","key":"2024041923351959300_ocae060-B54","doi-asserted-by":"crossref","first-page":"1107","DOI":"10.1038\/s41467-021-21390-2","article-title":"Temporal bias in case-control design: preventing reliable predictions of the future","volume":"12","author":"Yuan","year":"2021","journal-title":"Nat Commun"},{"key":"2024041923351959300_ocae060-B55","doi-asserted-by":"crossref","first-page":"561802","DOI":"10.3389\/frai.2020.561802","article-title":"Addressing fairness, bias, and appropriate use of artificial intelligence and machine learning in global health","volume":"3","author":"Fletcher","year":"2020","journal-title":"Front Artif Intell"},{"key":"2024041923351959300_ocae060-B56","author":"Jun","year":"2023"},{"key":"2024041923351959300_ocae060-B57","doi-asserted-by":"crossref","first-page":"100496","DOI":"10.1016\/j.ajpc.2023.100496","article-title":"Natural language processing to identify reasons for sex disparity in statin prescriptions","volume":"14","author":"Witting","year":"2023","journal-title":"Am J Prev Cardiol"},{"key":"2024041923351959300_ocae060-B58","author":"Berk"},{"key":"2024041923351959300_ocae060-B59","first-page":"259","author":"Feldman","year":"2015"},{"key":"2024041923351959300_ocae060-B60","author":"Beutel","year":"2017"},{"key":"2024041923351959300_ocae060-B61","author":"Celis"},{"key":"2024041923351959300_ocae060-B62","author":"Edwards"},{"key":"2024041923351959300_ocae060-B63","author":"H\u00e9bert-Johnson","year":"2018:"},{"key":"2024041923351959300_ocae060-B64","first-page":"4051","author":"Liu","year":"2019"},{"key":"2024041923351959300_ocae060-B65","author":"Liu"},{"key":"2024041923351959300_ocae060-B66","first-page":"3323","author":"Hardt","year":"2016"},{"key":"2024041923351959300_ocae060-B67","first-page":"1375","author":"Iosifidis","year":"2019"},{"key":"2024041923351959300_ocae060-B68","author":"Valera"},{"issue":"1","key":"2024041923351959300_ocae060-B69","doi-asserted-by":"crossref","first-page":"4209","DOI":"10.1038\/s41598-022-07939-1","article-title":"A clarification of the nuances in the fairness metrics landscape","volume":"12","author":"Castelnovo","year":"2022","journal-title":"Sci Rep"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/5\/1172\/57286276\/ocae060.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/5\/1172\/57286276\/ocae060.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T21:59:56Z","timestamp":1731535196000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/5\/1172\/7634193"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,23]]},"references-count":69,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,3,23]]},"published-print":{"date-parts":[[2024,4,19]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae060","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,5,1]]},"published":{"date-parts":[[2024,3,23]]}}}