{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T14:47:26Z","timestamp":1779893246467,"version":"3.53.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2022,7,5]],"date-time":"2022-07-05T00:00:00Z","timestamp":1656979200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,5]],"date-time":"2022-07-05T00:00:00Z","timestamp":1656979200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["NIBIB R01 EB017205"],"award-info":[{"award-number":["NIBIB R01 EB017205"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology, Taiwan","doi-asserted-by":"publisher","award":["MOST109-2222-E-007-004-MY3"],"award-info":[{"award-number":["MOST109-2222-E-007-004-MY3"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006919","name":"Massachusetts Institute of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100006919","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The unprecedented global crisis brought about by the COVID-19 pandemic has sparked numerous efforts to create predictive models for the detection and prognostication of SARS-CoV-2 infections with the goal of helping health systems allocate resources. Machine learning models, in particular, hold promise for their ability to leverage patient clinical information and medical images for prediction. However, most of the published COVID-19 prediction models thus far have little clinical utility due to methodological flaws and lack of appropriate validation. In this paper, we describe our methodology to develop and validate multi-modal models for COVID-19 mortality prediction using multi-center patient data. The models for COVID-19 mortality prediction were developed using retrospective data from Madrid, Spain (<jats:italic>N<\/jats:italic>\u2009=\u20092547) and were externally validated in patient cohorts from a community hospital in New Jersey, USA (<jats:italic>N<\/jats:italic>\u2009=\u2009242) and an academic center in Seoul, Republic of Korea (<jats:italic>N<\/jats:italic>\u2009=\u2009336). The models we developed performed differently across various clinical settings, underscoring the need for a guided strategy when employing machine learning for clinical decision-making. We demonstrated that using features from both the structured electronic health records and chest X-ray imaging data resulted in better 30-day mortality prediction performance across all three datasets (areas under the receiver operating characteristic curves: 0.85 (95% confidence interval: 0.83\u20130.87), 0.76 (0.70\u20130.82), and 0.95 (0.92\u20130.98)). We discuss the rationale for the decisions made at every step in developing the models and have made our code available to the research community. We employed the best machine learning practices for clinical model development. Our goal is to create a toolkit that would assist investigators and organizations in building multi-modal models for prediction, classification, and\/or optimization.\n<\/jats:p>","DOI":"10.1007\/s10278-022-00674-z","type":"journal-article","created":{"date-parts":[[2022,7,5]],"date-time":"2022-07-05T15:44:20Z","timestamp":1657035860000},"page":"1514-1529","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Developing and Validating Multi-Modal Models for Mortality Prediction in COVID-19 Patients: a Multi-center Retrospective Study"],"prefix":"10.1007","volume":"35","author":[{"given":"Joy Tzung-yu","family":"Wu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miguel \u00c1ngel Armengol","family":"de la Hoz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4020-3147","authenticated-orcid":false,"given":"Po-Chih","family":"Kuo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joseph Alexander","family":"Paguio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jasper Seth","family":"Yao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward Christopher","family":"Dee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wesley","family":"Yeung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jerry","family":"Jurado","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Achintya","family":"Moulick","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carmelo","family":"Milazzo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paloma","family":"Peinado","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paula","family":"Villares","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antonio","family":"Cubillo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jos\u00e9 Felipe","family":"Varona","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyung-Chul","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alberto","family":"Estirado","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jos\u00e9 Maria","family":"Castellano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leo Anthony","family":"Celi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,7,5]]},"reference":[{"key":"674_CR1","doi-asserted-by":"publisher","unstructured":"M. Xu et al., \u201cAccurately Differentiating COVID-19, Other Viral Infection, and Healthy Individuals Using Multimodal Features via Late Fusion Learning,\u201d medRxiv, p. 2020.08.18.20176776, Aug. 2020,   https:\/\/doi.org\/10.1101\/2020.08.18.20176776.","DOI":"10.1101\/2020.08.18.20176776"},{"key":"674_CR2","doi-asserted-by":"publisher","unstructured":"G. Chassagnon and N. Paragios, \u201cHolistic AI-Driven Quantification, Staging and Prognosis of COVID-19 Pneumonia,\u201d medRxiv, p. 2020.04.17.20069187, Jul. 2020,   https:\/\/doi.org\/10.1101\/2020.04.17.20069187.","DOI":"10.1101\/2020.04.17.20069187"},{"key":"674_CR3","doi-asserted-by":"publisher","unstructured":"X. Wang et al., \u201cMulticenter Study of Temporal Changes and Prognostic Value of a CT Visual Severity Score in Hospitalized Patients With Coronavirus Disease (COVID-19),\u201d Am. J. Roentgenol., pp. 1\u201310, Sep. 2020,   https:\/\/doi.org\/10.2214\/AJR.20.24044.","DOI":"10.2214\/AJR.20.24044"},{"key":"674_CR4","doi-asserted-by":"publisher","unstructured":"T. Ramtohul et al., \u201cQuantitative CT Extent of Lung Damage in COVID-19 Pneumonia Is an Independent Risk Factor for Inpatient Mortality in a Population of Cancer Patients: A Prospective Study,\u201d Front. Oncol., vol. 10, Sep. 2020,   https:\/\/doi.org\/10.3389\/fonc.2020.01560.","DOI":"10.3389\/fonc.2020.01560"},{"key":"674_CR5","doi-asserted-by":"publisher","unstructured":"N. Lassau et al., \u201cIntegration of clinical characteristics, lab tests and a deep learning CT scan analysis to predict severity of hospitalized COVID-19 patients,\u201d medRxiv, p. 2020.05.14.20101972, Oct. 2020,   https:\/\/doi.org\/10.1101\/2020.05.14.20101972.","DOI":"10.1101\/2020.05.14.20101972"},{"issue":"16","key":"674_CR6","doi-asserted-by":"publisher","first-page":"7231","DOI":"10.7150\/thno.46428","volume":"10","author":"Q Wu","year":"2020","unstructured":"Q. Wu et al., \u201cRadiomics Analysis of Computed Tomography helps predict poor prognostic outcome in COVID-19,\u201d Theranostics, vol. 10, no. 16, pp. 7231\u20137244, 2020,   https:\/\/doi.org\/10.7150\/thno.46428.","journal-title":"Theranostics"},{"issue":"8","key":"674_CR7","doi-asserted-by":"publisher","first-page":"1007","DOI":"10.3348\/kjr.2020.0485","volume":"21","author":"Y Zheng","year":"2020","unstructured":"Y. Zheng et al., \u201cDevelopment and Validation of a Prognostic Nomogram Based on Clinical and CT Features for Adverse Outcome Prediction in Patients with COVID-19,\u201d Korean J. Radiol., vol. 21, no. 8, pp. 1007\u20131017, Aug. 2020,   https:\/\/doi.org\/10.3348\/kjr.2020.0485.","journal-title":"Korean J. Radiol."},{"key":"674_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101844","volume":"67","author":"H Chao","year":"2021","unstructured":"H. Chao et al., \u201cIntegrative analysis for COVID-19 patient outcome prediction,\u201d Med. Image Anal., vol. 67, p. 101844, Jan. 2021,   https:\/\/doi.org\/10.1016\/j.media.2020.101844.","journal-title":"Med. Image Anal."},{"key":"674_CR9","doi-asserted-by":"publisher","unstructured":"M. Roberts et al., \u201cCommon pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans,\u201d Nat. Mach. Intell., vol. 3, no. 3, Art. no. 3, Mar. 2021,   https:\/\/doi.org\/10.1038\/s42256-021-00307-0.","DOI":"10.1038\/s42256-021-00307-0"},{"issue":"9","key":"674_CR10","doi-asserted-by":"publisher","first-page":"e489","DOI":"10.1016\/S2589-7500(20)30186-2","volume":"2","author":"J Futoma","year":"2020","unstructured":"J. Futoma, M. Simons, T. Panch, F. Doshi-Velez, and L. A. Celi, \u201cThe myth of generalisability in clinical research and machine learning in health care,\u201d Lancet Digit. Health, vol. 2, no. 9, pp. e489\u2013e492, Sep. 2020,   https:\/\/doi.org\/10.1016\/S2589-7500(20)30186-2.","journal-title":"Lancet Digit. Health"},{"key":"674_CR11","unstructured":"S. on Facebook, S. on Twitter, and S. on LinkedIn, \u201cMajor flaws found in machine learning for COVID-19 diagnosis,\u201d VentureBeat, Mar. 23, 2021. https:\/\/venturebeat.com\/2021\/03\/23\/major-flaws-found-in-machine-learning-for-covid-19-diagnosis\/ (accessed Jun. 04, 2021)."},{"key":"674_CR12","doi-asserted-by":"publisher","unstructured":"A. J. DeGrave, J. D. Janizek, and S.-I. Lee, \u201cAI for radiographic COVID-19 detection selects shortcuts over signal,\u201d Nat. Mach. Intell., pp. 1\u201310, May 2021,   https:\/\/doi.org\/10.1038\/s42256-021-00338-7.","DOI":"10.1038\/s42256-021-00338-7"},{"issue":"3","key":"674_CR13","doi-asserted-by":"publisher","first-page":"e195","DOI":"10.1016\/S2589-7500(20)30292-2","volume":"3","author":"UJ Muehlematter","year":"2021","unstructured":"U. J. Muehlematter, P. Daniore, and K. N. Vokinger, \u201cApproval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015\u201320): a comparative analysis,\u201d Lancet Digit. Health, vol. 3, no. 3, pp. e195\u2013e203, Mar. 2021,   https:\/\/doi.org\/10.1016\/S2589-7500(20)30292-2.","journal-title":"Lancet Digit. Health"},{"key":"674_CR14","doi-asserted-by":"publisher","unstructured":"S. Benjamens, P. Dhunnoo, and B. Mesk\u00f3, \u201cThe state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database,\u201d Npj Digit. Med., vol. 3, no. 1, Art. no. 1, Sep. 2020,   https:\/\/doi.org\/10.1038\/s41746-020-00324-0.","DOI":"10.1038\/s41746-020-00324-0"},{"key":"674_CR15","doi-asserted-by":"publisher","unstructured":"E. Wu, K. Wu, R. Daneshjou, D. Ouyang, D. E. Ho, and J. Zou, \u201cHow medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals,\u201d Nat. Med., vol. 27, no. 4, Art. no. 4, Apr. 2021,   https:\/\/doi.org\/10.1038\/s41591-021-01312-x.","DOI":"10.1038\/s41591-021-01312-x"},{"key":"674_CR16","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m1328","volume":"369","author":"L Wynants","year":"2020","unstructured":"L. Wynants et al., \u201cPrediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal,\u201d BMJ, vol. 369, p. m1328, Apr. 2020,   https:\/\/doi.org\/10.1136\/bmj.m1328.","journal-title":"BMJ"},{"issue":"2","key":"674_CR17","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2020200029","volume":"2","author":"J Mongan","year":"2020","unstructured":"J. Mongan, L. Moy, and C. E. Kahn, \u201cChecklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers,\u201d Radiol. Artif. Intell., vol. 2, no. 2, p. e200029, Mar. 2020,   https:\/\/doi.org\/10.1148\/ryai.2020200029.","journal-title":"Radiol. Artif. Intell."},{"key":"674_CR18","unstructured":"\u201cCovid Data Save Lives English Version.\u201d https:\/\/www.hmhospitales.com\/coronavirus\/covid-data-save-lives\/english-version (accessed Jun. 24, 2021)."},{"issue":"1","key":"674_CR19","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1016\/j.chest.2020.06.082","volume":"159","author":"JS Yao","year":"2021","unstructured":"J. S. Yao et al., \u201cThe Minimal Effect of Zinc on the Survival of Hospitalized Patients With COVID-19: An Observational Study,\u201d Chest, vol. 159, no. 1, pp. 108\u2013111, Jan. 2021,   https:\/\/doi.org\/10.1016\/j.chest.2020.06.082.","journal-title":"Chest"},{"key":"674_CR20","doi-asserted-by":"crossref","unstructured":"G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, \u201cDensely connected convolutional networks,\u201d in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700\u20134708.","DOI":"10.1109\/CVPR.2017.243"},{"key":"674_CR21","unstructured":"S. Lundberg and S.-I. Lee, \u201cA Unified Approach to Interpreting Model Predictions,\u201d ArXiv170507874 Cs Stat, Nov. 2017, Accessed: Jun. 27, 2021. [Online]. Available: http:\/\/arxiv.org\/abs\/1705.07874"},{"issue":"2","key":"674_CR22","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","volume":"128","author":"RR Selvaraju","year":"2020","unstructured":"R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, \u201cGrad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization,\u201d Int. J. Comput. Vis., vol. 128, no. 2, pp. 336\u2013359, Feb. 2020,   https:\/\/doi.org\/10.1007\/s11263-019-01228-7.","journal-title":"Int. J. Comput. Vis."},{"issue":"4","key":"674_CR23","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1001\/jamainternmed.2020.7968","volume":"181","author":"OA Panagiotou","year":"2021","unstructured":"O. A. Panagiotou et al., \u201cRisk Factors Associated With All-Cause 30-Day Mortality in Nursing Home Residents With COVID-19,\u201d JAMA Intern. Med., vol. 181, no. 4, p. 439, Apr. 2021,   https:\/\/doi.org\/10.1001\/jamainternmed.2020.7968.","journal-title":"JAMA Intern. Med."},{"key":"674_CR24","doi-asserted-by":"publisher","unstructured":"Y. X. Gue, M. Tennyson, J. Gao, S. Ren, R. Kanji, and D. A. Gorog, \u201cDevelopment of a novel risk score to predict mortality in patients admitted to hospital with COVID-19,\u201d Sci. Rep., vol. 10, no. 1, Art. no. 1, Dec. 2020,   https:\/\/doi.org\/10.1038\/s41598-020-78505-w.","DOI":"10.1038\/s41598-020-78505-w"},{"key":"674_CR25","doi-asserted-by":"publisher","DOI":"10.1136\/thoraxjnl-2020-216001","author":"J Berenguer","year":"2021","unstructured":"J. Berenguer et al., \u201cDevelopment and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score,\u201d Thorax, Feb. 2021,   https:\/\/doi.org\/10.1136\/thoraxjnl-2020-216001.","journal-title":"Thorax"},{"issue":"4","key":"674_CR26","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1001\/jamainternmed.2020.8193","volume":"181","author":"DA Asch","year":"2021","unstructured":"D. A. Asch et al., \u201cVariation in US Hospital Mortality Rates for Patients Admitted With COVID-19 During the First 6 Months of the Pandemic,\u201d JAMA Intern. Med., vol. 181, no. 4, pp. 471\u2013478, Apr. 2021,   https:\/\/doi.org\/10.1001\/jamainternmed.2020.8193.","journal-title":"JAMA Intern. Med."},{"key":"674_CR27","doi-asserted-by":"publisher","unstructured":"J. Wu et al., \u201cAutomatic Bounding Box Annotation of Chest X-Ray Data for Localization of Abnormalities,\u201d in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA, Apr. 2020, pp. 799\u2013803.   https:\/\/doi.org\/10.1109\/ISBI45749.2020.9098482.","DOI":"10.1109\/ISBI45749.2020.9098482"},{"key":"674_CR28","doi-asserted-by":"publisher","unstructured":"J. Irvin et al., \u201cCheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison,\u201d Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, Art. no. 01, Jul. 2019,   https:\/\/doi.org\/10.1609\/aaai.v33i01.3301590.","DOI":"10.1609\/aaai.v33i01.3301590"},{"issue":"1","key":"674_CR29","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1038\/s41597-019-0322-0","volume":"6","author":"AEW Johnson","year":"2019","unstructured":"A. E. W. Johnson et al., \u201cMIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports,\u201d Sci. Data, vol. 6, no. 1, p. 317, Dec. 2019,   https:\/\/doi.org\/10.1038\/s41597-019-0322-0.","journal-title":"Sci. Data"},{"key":"674_CR30","first-page":"625","volume":"2017","author":"SE Davis","year":"2017","unstructured":"S. E. Davis, T. A. Lasko, G. Chen, and M. E. Matheny, \u201cCalibration Drift Among Regression and Machine Learning Models for Hospital Mortality,\u201d AMIA Annu. Symp. Proc. AMIA Symp., vol. 2017, pp. 625\u2013634, 2017.","journal-title":"AMIA Annu. Symp. Proc. AMIA Symp."},{"issue":"6","key":"674_CR31","doi-asserted-by":"publisher","first-page":"1052","DOI":"10.1093\/jamia\/ocx030","volume":"24","author":"SE Davis","year":"2017","unstructured":"S. E. Davis, T. A. Lasko, G. Chen, E. D. Siew, and M. E. Matheny, \u201cCalibration drift in regression and machine learning models for acute kidney injury,\u201d J. Am. Med. Inform. Assoc. JAMIA, vol. 24, no. 6, pp. 1052\u20131061, Nov. 2017,   https:\/\/doi.org\/10.1093\/jamia\/ocx030.","journal-title":"J. Am. Med. Inform. Assoc. JAMIA"},{"issue":"6","key":"674_CR32","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1177\/0310057X1204000609","volume":"40","author":"E Paul","year":"2012","unstructured":"E. Paul, M. Bailey, A. Van Lint, and V. Pilcher, \u201cPerformance of APACHE III over time in Australia and New Zealand: a retrospective cohort study,\u201d Anaesth. Intensive Care, vol. 40, no. 6, pp. 980\u2013994, Nov. 2012,   https:\/\/doi.org\/10.1177\/0310057X1204000609.","journal-title":"Anaesth. Intensive Care"},{"key":"674_CR33","doi-asserted-by":"publisher","unstructured":"J. Wawira Gichoya, L. G. McCoy, L. A. Celi, and M. Ghassemi, \u201cEquity in essence: a call for operationalising fairness in machine learning for healthcare,\u201d BMJ Health Amp Care Inform., vol. 28, no. 1, p. e100289, Apr. 2021,   https:\/\/doi.org\/10.1136\/bmjhci-2020-100289.","DOI":"10.1136\/bmjhci-2020-100289"},{"issue":"12","key":"674_CR34","doi-asserted-by":"publisher","first-page":"1448","DOI":"10.1093\/jamia\/ocz127","volume":"26","author":"SE Davis","year":"2019","unstructured":"[34]S. E. Davis, R. A. Greevy Jr, C. Fonnesbeck, T. A. Lasko, C. G. Walsh, and M. E. Matheny, \u201cA nonparametric updating method to correct clinical prediction model drift,\u201d J. Am. Med. Inform. Assoc., vol. 26, no. 12, pp. 1448\u20131457, 2019.","journal-title":"J. Am. Med. Inform. Assoc."},{"issue":"1","key":"674_CR35","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/s41746-021-00393-9","volume":"4","author":"P-C Kuo","year":"2021","unstructured":"P.-C. Kuo et al., \u201cRecalibration of deep learning models for abnormality detection in smartphone-captured chest radiograph,\u201d NPJ Digit. Med., vol. 4, no. 1, p. 25, Feb. 2021,   https:\/\/doi.org\/10.1038\/s41746-021-00393-9.","journal-title":"NPJ Digit. Med."},{"issue":"12","key":"674_CR36","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0243262","volume":"15","author":"D Bertsimas","year":"2020","unstructured":"D. Bertsimas et al., \u201cCOVID-19 mortality risk assessment: An international multi-center study,\u201d PLOS ONE, vol. 15, no. 12, p. e0243262, Dec. 2020,   https:\/\/doi.org\/10.1371\/journal.pone.0243262.","journal-title":"PLOS ONE"},{"issue":"6464","key":"674_CR37","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1126\/science.aaz3873","volume":"366","author":"R Benjamin","year":"2019","unstructured":"R. Benjamin, \u201cAssessing risk, automating racism,\u201d Science, vol. 366, no. 6464, p. 421, Oct. 2019,   https:\/\/doi.org\/10.1126\/science.aaz3873.","journal-title":"Science"}],"container-title":["Journal of Digital Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-022-00674-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-022-00674-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-022-00674-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T19:20:31Z","timestamp":1669836031000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-022-00674-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,5]]},"references-count":37,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["674"],"URL":"https:\/\/doi.org\/10.1007\/s10278-022-00674-z","relation":{},"ISSN":["0897-1889","1618-727X"],"issn-type":[{"value":"0897-1889","type":"print"},{"value":"1618-727X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,5]]},"assertion":[{"value":"4 November 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 June 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 July 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}