{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T04:35:16Z","timestamp":1775277316432,"version":"3.50.1"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2024,6,29]],"date-time":"2024-06-29T00:00:00Z","timestamp":1719619200000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100009192","name":"Alberta Innovates","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100009192","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"name":"River Fund at Calgary Foundation, Canada Research Chairs Program"},{"name":"University of Calgary Department of Pediatrics"},{"DOI":"10.13039\/501100003206","name":"Alberta Children's Hospital Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003206","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Artificial intelligence (AI) models trained using medical images for clinical tasks often exhibit bias in the form of subgroup performance disparities. However, since not all sources of bias in real-world medical imaging data are easily identifiable, it is challenging to comprehensively assess their impacts. In this article, we introduce an analysis framework for systematically and objectively investigating the impact of biases in medical images on AI models.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Our framework utilizes synthetic neuroimages with known disease effects and sources of bias. We evaluated the impact of bias effects and the efficacy of 3 bias mitigation strategies in counterfactual data scenarios on a convolutional neural network (CNN) classifier.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The analysis revealed that training a CNN model on the datasets containing bias effects resulted in expected subgroup performance disparities. Moreover, reweighing was the most successful bias mitigation strategy for this setup. Finally, we demonstrated that explainable AI methods can aid in investigating the manifestation of bias in the model using this framework.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>The value of this framework is showcased in our findings on the impact of bias scenarios and efficacy of bias mitigation in a deep learning model pipeline. This systematic analysis can be easily expanded to conduct further controlled in silico trials in other investigations of bias in medical imaging AI.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Our novel methodology for objectively studying bias in medical imaging AI can help support the development of clinical decision-support tools that are robust and responsible.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae165","type":"journal-article","created":{"date-parts":[[2024,6,29]],"date-time":"2024-06-29T02:39:14Z","timestamp":1719628754000},"page":"2613-2621","source":"Crossref","is-referenced-by-count":20,"title":["Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7802-6820","authenticated-orcid":false,"given":"Emma A M","family":"Stanley","sequence":"first","affiliation":[{"name":"Biomedical Engineering Graduate Program, University of Calgary , Calgary, Alberta, T2N 1N4,","place":["Canada"]},{"name":"Department of Radiology, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Hotchkiss Brain Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Alberta Children\u2019s Hospital Research Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7455-3383","authenticated-orcid":false,"given":"Raissa","family":"Souza","sequence":"additional","affiliation":[{"name":"Biomedical Engineering Graduate Program, University of Calgary , Calgary, Alberta, T2N 1N4,","place":["Canada"]},{"name":"Department of Radiology, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Hotchkiss Brain Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Alberta Children\u2019s Hospital Research Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]}]},{"given":"Anthony J","family":"Winder","sequence":"additional","affiliation":[{"name":"Department of Radiology, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Hotchkiss Brain Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]}]},{"given":"Vedant","family":"Gulve","sequence":"additional","affiliation":[{"name":"Department of Radiology, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]}]},{"given":"Kimberly","family":"Amador","sequence":"additional","affiliation":[{"name":"Biomedical Engineering Graduate Program, University of Calgary , Calgary, Alberta, T2N 1N4,","place":["Canada"]},{"name":"Department of Radiology, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Hotchkiss Brain Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Alberta Children\u2019s Hospital Research Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]}]},{"given":"Matthias","family":"Wilms","sequence":"additional","affiliation":[{"name":"Hotchkiss Brain Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Alberta Children\u2019s Hospital Research Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Department of Pediatrics, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Department of Community Health Sciences, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2556-3224","authenticated-orcid":false,"given":"Nils D","family":"Forkert","sequence":"additional","affiliation":[{"name":"Department of Radiology, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Hotchkiss Brain Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Alberta Children\u2019s Hospital Research Institute, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Department of Community Health Sciences, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Department of Clinical Neuroscience, University of Calgary , Calgary, Alberta, T2N 4N1,","place":["Canada"]},{"name":"Department of Electrical and Software Engineering, University of Calgary , Calgary, Alberta, T2N 1N4,","place":["Canada"]}]}],"member":"286","published-online":{"date-parts":[[2024,6,28]]},"reference":[{"issue":"12","key":"2024102107531706400_ocae165-B1","doi-asserted-by":"publisher","first-page":"2176","DOI":"10.1038\/s41591-021-01595-0","article-title":"Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations","volume":"27","author":"Seyyed-Kalantari","year":"2021","journal-title":"Nat Med"},{"key":"2024102107531706400_ocae165-B2","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1007\/978-3-030-87199-4_39","volume-title":"Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2021","author":"Puyol-Ant\u00f3n","year":"2021"},{"issue":"23","key":"2024102107531706400_ocae165-B3","doi-asserted-by":"publisher","first-page":"12592","DOI":"10.1073\/pnas.1919012117","article-title":"Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis","volume":"117","author":"Larrazabal","year":"2020","journal-title":"Proc Natl Acad Sci USA"},{"issue":"32","key":"2024102107531706400_ocae165-B4","doi-asserted-by":"publisher","first-page":"eabq6147","DOI":"10.1126\/sciadv.abq6147","article-title":"Disparities in dermatology AI performance on a diverse, curated clinical image set","volume":"8","author":"Daneshjou","year":"2022","journal-title":"Sci Adv"},{"key":"2024102107531706400_ocae165-B5","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1007\/978-3-031-23223-7_2","volume-title":"Ethical and Philosophical Issues in Medical Imaging, Multimodal Learning and Fusion Across Scales for Clinical Decision Support, and Topological Data Analysis for Biomedical Imaging","author":"Stanley","year":"2022"},{"key":"2024102107531706400_ocae165-B6","doi-asserted-by":"publisher","first-page":"102607","DOI":"10.1016\/j.artmed.2023.102607","article-title":"FDA-approved machine learning algorithms in neuroradiology: a systematic review of the current evidence for approval","volume":"143","author":"Yearley","year":"2023","journal-title":"Artif Intell Med"},{"key":"2024102107531706400_ocae165-B7","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1007\/978-3-031-45249-9_19","volume-title":"Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging","author":"Pi\u00e7arra","year":"2023"},{"issue":"12","key":"2024102107531706400_ocae165-B8","doi-asserted-by":"publisher","first-page":"1925","DOI":"10.1093\/jamia\/ocad171","article-title":"Image-encoded biological and non-biological variables may be used as shortcuts in deep learning models trained on multisite neuroimaging data","volume":"30","author":"Souza","year":"2023","journal-title":"J Am Med Inf Assoc"},{"issue":"6","key":"2024102107531706400_ocae165-B9","doi-asserted-by":"publisher","first-page":"e406","DOI":"10.1016\/S2589-7500(22)00063-2","article-title":"AI recognition of patient race in medical imaging: a modelling study","volume":"4","author":"Gichoya","year":"2022","journal-title":"Lancet Digit Health"},{"issue":"6","key":"2024102107531706400_ocae165-B10","doi-asserted-by":"publisher","first-page":"061106","DOI":"10.1117\/1.JMI.10.6.061106","article-title":"Ability of artificial intelligence to identify self-reported race in chest X-ray using pixel intensity counts","volume":"10","author":"Burns","year":"2023","journal-title":"J Med Imaging (Bellingham)"},{"issue":"9","key":"2024102107531706400_ocae165-B11","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1016\/j.jacr.2023.06.025","article-title":"\u2018Shortcuts\u2019 causing bias in radiology artificial intelligence: causes, evaluation, and mitigation","volume":"20","author":"Banerjee","year":"2023","journal-title":"J Am Coll Radiol"},{"key":"2024102107531706400_ocae165-B12","doi-asserted-by":"publisher","first-page":"2512","DOI":"10.1109\/WACV48630.2021.00256","author":"Adeli","year":"2021"},{"key":"2024102107531706400_ocae165-B13","first-page":"504","author":"Marcinkevics","year":"2022"},{"key":"2024102107531706400_ocae165-B14","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1007\/978-3-031-16431-6_70","volume-title":"Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2022","author":"Wu","year":"2022"},{"key":"2024102107531706400_ocae165-B15","doi-asserted-by":"publisher","first-page":"117689","DOI":"10.1016\/j.neuroimage.2020.117689","article-title":"Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal","volume":"228","author":"Dinsdale","year":"2021","journal-title":"NeuroImage"},{"key":"2024102107531706400_ocae165-B16","doi-asserted-by":"publisher","first-page":"104467","DOI":"10.1016\/j.ebiom.2023.104467","article-title":"Algorithmic encoding of protected characteristics in chest X-ray disease detection models","volume":"89","author":"Glocker","year":"2023","journal-title":"eBioMedicine"},{"issue":"1","key":"2024102107531706400_ocae165-B17","doi-asserted-by":"publisher","first-page":"2023","DOI":"10.1038\/s41467-023-39902-7","article-title":"Detecting shortcut learning for fair medical AI using shortcut testing","volume":"14","author":"Brown","year":"2023","journal-title":"Nat Commun"},{"key":"2024102107531706400_ocae165-B18","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/978-3-031-43898-1_18","volume-title":"Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2023","author":"Jones","year":"2023"},{"key":"2024102107531706400_ocae165-B19","first-page":"204","author":"Zhang","year":"2022"},{"key":"2024102107531706400_ocae165-B20","author":"Zhong","year":"2023"},{"issue":"11","key":"2024102107531706400_ocae165-B21","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc IEEE"},{"key":"2024102107531706400_ocae165-B22","author":"Bahng"},{"issue":"178","key":"2024102107531706400_ocae165-B23","first-page":"1","article-title":"Morpho-MNIST: quantitative assessment and diagnostics for representation learning","volume":"20","author":"Castro","year":"2019","journal-title":"J Mach Learn Res"},{"key":"2024102107531706400_ocae165-B24","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1007\/978-3-031-43895-0_46","volume-title":"Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2023","author":"Stanley","year":"2023"},{"issue":"5","key":"2024102107531706400_ocae165-B25","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1002\/hbm.20906","article-title":"The SRI24 multichannel atlas of normal adult human brain structure","volume":"31","author":"Rohlfing","year":"2010","journal-title":"Hum Brain Mapp"},{"issue":"3","key":"2024102107531706400_ocae165-B26","doi-asserted-by":"publisher","first-page":"1064","DOI":"10.1016\/j.neuroimage.2007.09.031","article-title":"Construction of a 3D probabilistic atlas of human cortical structures","volume":"39","author":"Shattuck","year":"2008","journal-title":"NeuroImage"},{"key":"2024102107531706400_ocae165-B27","doi-asserted-by":"publisher","first-page":"924","DOI":"10.1007\/11866565_113","volume-title":"Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2006","author":"Arsigny","year":"2006"},{"key":"2024102107531706400_ocae165-B28","author":"Biomedical Image Analysis Group"},{"key":"2024102107531706400_ocae165-B29","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1109\/ICDMW.2009.83","author":"Calders","year":"2009"},{"key":"2024102107531706400_ocae165-B30","author":"Smilkov","year":"2017"},{"issue":"6","key":"2024102107531706400_ocae165-B31","doi-asserted-by":"publisher","first-page":"061102","DOI":"10.1117\/1.JMI.9.6.061102","article-title":"Fairness-related performance and explainability effects in deep learning models for brain image analysis","volume":"9","author":"Stanley","year":"2022","journal-title":"JMI"},{"key":"2024102107531706400_ocae165-B32","doi-asserted-by":"publisher","first-page":"105002","DOI":"10.1016\/j.ebiom.2024.105002","article-title":"(Predictable) performance bias in unsupervised anomaly detection","volume":"101","author":"Meissen","year":"2024","journal-title":"eBioMedicine"},{"key":"2024102107531706400_ocae165-B33","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1007\/978-3-031-17899-3_2","volume-title":"Machine Learning in Clinical Neuroimaging","author":"Ioannou","year":"2022"},{"key":"2024102107531706400_ocae165-B34","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1007\/978-3-031-45249-9_14","volume-title":"Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging","author":"Weng","year":"2023"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/11\/2613\/59813548\/ocae165.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/11\/2613\/59813548\/ocae165.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T07:53:32Z","timestamp":1729497212000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/11\/2613\/7701447"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,28]]},"references-count":34,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,6,28]]},"published-print":{"date-parts":[[2024,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae165","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,11]]},"published":{"date-parts":[[2024,6,28]]}}}