{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T13:34:51Z","timestamp":1793712891086,"version":"4.6.1"},"reference-count":88,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,5,3]],"date-time":"2022-05-03T00:00:00Z","timestamp":1651536000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,3]],"date-time":"2022-05-03T00:00:00Z","timestamp":1651536000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The recent release of large-scale healthcare datasets has greatly propelled the research of data-driven deep learning models for healthcare applications. However, due to the nature of such deep black-boxed models, concerns about interpretability, fairness, and biases in healthcare scenarios where human lives are at stake call for a careful and thorough examination of both datasets and models. In this work, we focus on MIMIC-IV (Medical Information Mart for Intensive Care, version IV), the largest publicly available healthcare dataset, and conduct comprehensive analyses of interpretability as well as dataset representation bias and prediction fairness of deep learning models for in-hospital mortality prediction. First, we analyze the interpretability of deep learning mortality prediction models and observe that (1) the best-performing interpretability method successfully identifies critical features for mortality prediction on various prediction models as well as recognizing new important features that domain knowledge does not consider; (2) prediction models rely on demographic features, raising concerns in fairness. Therefore, we then evaluate the fairness of models and do observe the unfairness: (1) there exists disparate treatment in prescribing mechanical ventilation among patient groups across ethnicity, gender and age; (2) models often rely on racial attributes unequally across subgroups to generate their predictions. We further draw concrete connections between interpretability methods and fairness metrics by showing how feature importance from interpretability methods can be beneficial in quantifying potential disparities in mortality predictors. Our analysis demonstrates that the prediction performance is not the only factor to consider when evaluating models for healthcare applications, since high prediction performance might be the result of unfair utilization of demographic features. Our findings suggest that future research in AI models for healthcare applications can benefit from utilizing the analysis workflow of interpretability and fairness as well as verifying if models achieve superior performance at the cost of introducing bias.<\/jats:p>","DOI":"10.1038\/s41598-022-11012-2","type":"journal-article","created":{"date-parts":[[2022,5,3]],"date-time":"2022-05-03T12:03:54Z","timestamp":1651579434000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":168,"title":["Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset"],"prefix":"10.1038","volume":"12","author":[{"given":"Chuizheng","family":"Meng","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Loc","family":"Trinh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nan","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Enouen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yan","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,5,3]]},"reference":[{"key":"11012_CR1","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.jbi.2018.04.007","volume":"83","author":"S Purushotham","year":"2018","unstructured":"Purushotham, S., Meng, C., Che, Z. & Liu, Y. Benchmarking deep learning models on large healthcare datasets. J. Biomed. Inf. 83, 112\u2013134. https:\/\/doi.org\/10.1016\/j.jbi.2018.04.007 (2018).","journal-title":"J. Biomed. Inf."},{"key":"11012_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41597-019-0103-9","volume":"6","author":"H Harutyunyan","year":"2019","unstructured":"Harutyunyan, H., Khachatrian, H., Kale, D. C., Ver Steeg, G. & Galstyan, A. Multitask learning and benchmarking with clinical time series data. Sci. Data 6, 1\u201318 (2019).","journal-title":"Sci. Data"},{"key":"11012_CR3","doi-asserted-by":"crossref","unstructured":"Wang, S. et\u00a0al. Mimic-extract: A data extraction, preprocessing, and representation pipeline for mimic-iii. In Proceedings of the ACM Conference on Health, Inference, and Learning, 222\u2013235 (2020).","DOI":"10.1145\/3368555.3384469"},{"key":"11012_CR4","unstructured":"Chen, I., Johansson, F.\u00a0D. & Sontag, D. Why is my classifier discriminatory? In Adv. Neural Inf. Process. Syst., 3539\u20133550 (2018)."},{"key":"11012_CR5","unstructured":"Johnson, A. et\u00a0al. Mimic-iv (version 0.4). PhysioNet (2020)."},{"issue":"23","key":"11012_CR6","first-page":"e215","volume":"101","author":"A Goldberger","year":"2000","unstructured":"Goldberger, A. et al. Physiobank, physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals. Circulation [Online] 101(23), e215\u2013e220 (2000).","journal-title":"Circulation [Online]"},{"key":"11012_CR7","unstructured":"Hooker, S., Erhan, D., Kindermans, P.-J. & Kim, B. A benchmark for interpretability methods in deep neural networks. Adv. Neural Inf. Process. Syst., 9737\u20139748 (2019)."},{"key":"11012_CR8","first-page":"6147","volume":"33","author":"M Tsang","year":"2020","unstructured":"Tsang, M., Rambhatla, S. & Liu, Y. How does this interaction affect me? Interpretable attribution for feature interactions. Adv. Neural Inf. Process. Syst. 33, 6147\u20136159 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"11012_CR9","unstructured":"Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)."},{"key":"11012_CR10","unstructured":"Sundararajan, M., Taly, A. & Yan, Q. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, 3319\u20133328 (2017)."},{"key":"11012_CR11","unstructured":"Shrikumar, A., Greenside, P. & Kundaje, A. Learning important features through propagating activation differences. In International Conference on Machine Learning, 3145\u20133153 (2017)."},{"key":"11012_CR12","doi-asserted-by":"crossref","unstructured":"Ancona, M., Ceolini, E., \u00d6ztireli, C. & Gross, M. Towards better understanding of gradient-based attribution methods for deep neural networks. In International Conference on Learning Representations (2018).","DOI":"10.1007\/978-3-030-28954-6_9"},{"key":"11012_CR13","unstructured":"Lundberg, S.\u00a0M. & Lee, S.-I. A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst., 4765\u20134774 (2017)."},{"key":"11012_CR14","unstructured":"Smilkov, D., Thorat, N., Kim, B., Vi\u00e9gas, F. & Wattenberg, M. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017)."},{"key":"11012_CR15","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.1016\/j.cor.2008.04.004","volume":"36","author":"J Castro","year":"2009","unstructured":"Castro, J., G\u00f3mez, D. & Tejada, J. Polynomial calculation of the shapley value based on sampling. Comput. Oper. Res. 36, 1726\u20131730 (2009).","journal-title":"Comput. Oper. Res."},{"key":"11012_CR16","first-page":"1","volume":"11","author":"E Strumbelj","year":"2010","unstructured":"Strumbelj, E. & Kononenko, I. An efficient explanation of individual classifications using game theory. J. Mach. Learn. Res. 11, 1\u201318 (2010).","journal-title":"J. Mach. Learn. Res."},{"key":"11012_CR17","volume-title":"Interpretable Machine Learning","author":"C Molnar","year":"2020","unstructured":"Molnar, C. Interpretable Machine Learning (Lulu. com, 2020)."},{"key":"11012_CR18","unstructured":"Suresh, H. et\u00a0al. Clinical intervention prediction and understanding using deep networks. arXiv preprint arXiv:1705.08498 (2017)."},{"key":"11012_CR19","doi-asserted-by":"crossref","unstructured":"Zeiler, M.\u00a0D. & Fergus, R. Visualizing and Understanding Convolutional Networks. In European conference on computer vision, 818\u2013833 (Springer, 2014).","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"11012_CR20","unstructured":"Sundararajan, M., Dhamdhere, K. & Agarwal, A. The shapley taylor interaction index. In International Conference on Machine Learning, 9259\u20139268 (PMLR, 2020)."},{"key":"11012_CR21","unstructured":"Janizek, J.\u00a0D., Sturmfels, P. & Lee, S.-I. Explaining explanations: Axiomatic feature interactions for deep networks. arXiv preprint arXiv:2002.04138 (2020)."},{"key":"11012_CR22","doi-asserted-by":"crossref","unstructured":"Sorokina, D., Caruana, R., Riedewald, M. & Fink, D. Detecting statistical interactions with additive groves of trees. In Proceedings of the 25th international conference on Machine learning, 1000\u20131007 (2008).","DOI":"10.1145\/1390156.1390282"},{"key":"11012_CR23","unstructured":"Tsang, M., Cheng, D. & Liu, Y. Detecting statistical interactions from neural network weights. In International Conference on Learning Representations (2018)."},{"key":"11012_CR24","unstructured":"Tsang, M., Liu, H., Purushotham, S., Murali, P. & Liu, Y. Neural interaction transparency (nit): Disentangling learned interactions for improved interpretability. Adv. Neural Inf. Process. Syst., 5804\u20135813 (2018)."},{"key":"11012_CR25","unstructured":"Dhamdhere, K., Sundararajan, M. & Yan, Q. How important is a neuron? arXiv preprint arXiv:1805.12233 (2018)."},{"key":"11012_CR26","unstructured":"Shrikumar, A., Su, J. & Kundaje, A. Computationally efficient measures of internal neuron importance. arXiv preprint arXiv:1807.09946 (2018)."},{"key":"11012_CR27","doi-asserted-by":"crossref","unstructured":"Leino, K., Sen, S., Datta, A., Fredrikson, M. & Li, L. Influence-directed explanations for deep convolutional networks. In 2018 IEEE International Test Conference (ITC), 1\u20138 (IEEE, 2018).","DOI":"10.1109\/TEST.2018.8624792"},{"key":"11012_CR28","unstructured":"Springenberg, J., Dosovitskiy, A., Brox, T. & Riedmiller, M. Striving for simplicity: The all convolutional net. In ICLR (workshop track) (2015)."},{"key":"11012_CR29","unstructured":"Kim, B. et\u00a0al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference on Machine Learning, 2668\u20132677 (PMLR, 2018)."},{"key":"11012_CR30","unstructured":"Ghorbani, A., Wexler, J., Zou, J.\u00a0Y. & Kim, B. Towards automatic concept-based explanations. Adv. Neural Inf. Process. Syst., 9277\u20139286 (2019)."},{"key":"11012_CR31","doi-asserted-by":"crossref","unstructured":"Zhou, B., Sun, Y., Bau, D. & Torralba, A. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), 119\u2013134 (2018).","DOI":"10.1007\/978-3-030-01237-3_8"},{"key":"11012_CR32","first-page":"6441","volume":"33","author":"AA Ismail","year":"2020","unstructured":"Ismail, A. A., Gunady, M., Corrada Bravo, H. & Feizi, S. Benchmarking deep learning interpretability in time series predictions. Adv. Neural Inf. Process. Syst. 33, 6441\u20136452 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"11012_CR33","doi-asserted-by":"crossref","unstructured":"Hardt, M. et\u00a0al. Explaining an increase in predicted risk for clinical alerts. In Proceedings of the ACM Conference on Health, Inference, and Learning, 80\u201389 (2020).","DOI":"10.1145\/3368555.3384460"},{"key":"11012_CR34","first-page":"5898","volume":"33","author":"B Sanchez-Lengeling","year":"2020","unstructured":"Sanchez-Lengeling, B. et al. Evaluating attribution for graph neural networks. Adv. Neural Inf. Process. Syst. 33, 5898\u20135910 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"11012_CR35","doi-asserted-by":"publisher","first-page":"2660","DOI":"10.1109\/TNNLS.2016.2599820","volume":"28","author":"W Samek","year":"2016","unstructured":"Samek, W., Binder, A., Montavon, G., Lapuschkin, S. & M\u00fcller, K.-R. Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Netw. Learn. Syst. 28, 2660\u20132673 (2016).","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"11012_CR36","doi-asserted-by":"publisher","first-page":"2966","DOI":"10.1287\/mnsc.2018.3093","volume":"65","author":"A Lambrecht","year":"2019","unstructured":"Lambrecht, A. & Tucker, C. Algorithmic bias? an empirical study of apparent gender-based discrimination in the display of stem career ads. Manage. Sci. 65, 2966\u20132981 (2019).","journal-title":"Manage. Sci."},{"key":"11012_CR37","doi-asserted-by":"crossref","unstructured":"Raji, I.\u00a0D. & Buolamwini, J. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. In Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, 429\u2013435 (2019).","DOI":"10.1145\/3306618.3314244"},{"key":"11012_CR38","unstructured":"Schnabel, T., Swaminathan, A., Singh, A., Chandak, N. & Joachims, T. Recommendations as treatments: Debiasing learning and evaluation. arXiv preprint arXiv:1602.05352 (2016)."},{"key":"11012_CR39","doi-asserted-by":"publisher","first-page":"eaao5580","DOI":"10.1126\/sciadv.aao5580","volume":"4","author":"J Dressel","year":"2018","unstructured":"Dressel, J. & Farid, H. The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4, eaao5580 (2018).","journal-title":"Sci. Adv."},{"key":"11012_CR40","doi-asserted-by":"crossref","unstructured":"Fu, R., Huang, Y. & Singh, P.\u00a0V. Artificial intelligence and algorithmic bias: Source, detection, mitigation, and implications. In Pushing the Boundaries: Frontiers in Impactful OR\/OM Research, 39\u201363 (INFORMS, 2020).","DOI":"10.1287\/educ.2020.0215"},{"key":"11012_CR41","unstructured":"Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 (2019)."},{"key":"11012_CR42","unstructured":"Hardt, M., Price, E. & Srebro, N. Equality of opportunity in supervised learning. Adv. Neural Inf. Process. Syst., 3315\u20133323 (2016)."},{"key":"11012_CR43","unstructured":"Bellamy, R.\u00a0K. et\u00a0al. Ai fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arXiv:1810.01943 (2018)."},{"key":"11012_CR44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10115-011-0463-8","volume":"33","author":"F Kamiran","year":"2012","unstructured":"Kamiran, F. & Calders, T. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33, 1\u201333 (2012).","journal-title":"Knowl. Inf. Syst."},{"key":"11012_CR45","first-page":"9084","volume":"31","author":"D Moyer","year":"2018","unstructured":"Moyer, D., Gao, S., Brekelmans, R., Galstyan, A. & Ver Steeg, G. Invariant representations without adversarial training. Adv. Neural Inf. Process. Syst. 31, 9084\u20139093 (2018).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"11012_CR46","unstructured":"Singh, H., Singh, R., Mhasawade, V. & Chunara, R. Fair predictors under distribution shift. arXiv preprint arXiv:1911.00677 (2019)."},{"issue":"3","key":"11012_CR47","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1093\/jamia\/ocaa283","volume":"28","author":"N Barda","year":"2020","unstructured":"Barda, N. et al. Addressing bias in prediction models by improving subpopulation calibration. J. Am. Med. Inf. Assoc. 28(3), 549\u2013558 (2020).","journal-title":"J. Am. Med. Inf. Assoc."},{"key":"11012_CR48","unstructured":"Martinez, N., Bertran, M. & Sapiro, G. Minimax pareto fairness: A multi objective perspective. In International Conference on Machine Learning, 6755\u20136764 (PMLR, 2020)."},{"key":"11012_CR49","doi-asserted-by":"crossref","unstructured":"Zhang, H., Lu, A.\u00a0X., Abdalla, M., McDermott, M. & Ghassemi, M. Hurtful words: quantifying biases in clinical contextual word embeddings. In Proceedings of the ACM Conference on Health, Inference, and Learning, 110\u2013120 (2020).","DOI":"10.1145\/3368555.3384448"},{"key":"11012_CR50","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1001\/amajethics.2019.167","volume":"21","author":"IY Chen","year":"2019","unstructured":"Chen, I. Y., Szolovits, P. & Ghassemi, M. Can AI help reduce disparities in general medical and mental health care?. AMA J. Ethics 21, 167\u2013179 (2019).","journal-title":"AMA J. Ethics"},{"key":"11012_CR51","unstructured":"Cui, S., Pan, W., Zhang, C. & Wang, F. xorder: A model agnostic post-processing framework for achieving ranking fairness while maintaining algorithm utility. arXiv preprint arXiv:2006.08267 (2020)."},{"key":"11012_CR52","doi-asserted-by":"crossref","unstructured":"Chen, J., Berlot-Atwell, I., Hossain, S., Wang, X. & Rudzicz, F. Exploring text specific and blackbox fairness algorithms in multimodal clinical nlp. arXiv preprint arXiv:2011.09625 (2020).","DOI":"10.18653\/v1\/2020.clinicalnlp-1.33"},{"key":"11012_CR53","doi-asserted-by":"crossref","unstructured":"Sharma, S., Henderson, J. & Ghosh, J. Certifai: Counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models. arXiv preprint arXiv:1905.07857 (2019).","DOI":"10.1145\/3375627.3375812"},{"key":"11012_CR54","doi-asserted-by":"crossref","unstructured":"Chu, E., Gillani, N. & Priscilla\u00a0Makini, S. Games for fairness and interpretability. In Companion Proceedings of the Web Conference 2020, 520\u2013524 (2020).","DOI":"10.1145\/3366424.3384374"},{"key":"11012_CR55","unstructured":"Doshi-Velez, F. & Kim, B. A roadmap for a rigorous science of interpretability. arXiv preprint arXiv:1702.086082 (2017)."},{"key":"11012_CR56","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","volume":"16","author":"ZC Lipton","year":"2018","unstructured":"Lipton, Z. C. The mythos of model interpretability. Queue 16, 31\u201357 (2018).","journal-title":"Queue"},{"issue":"4","key":"11012_CR57","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1109\/MIS.2020.3000681","volume":"36","author":"M Du","year":"2020","unstructured":"Du, M., Yang, F., Zou, N. & Hu, X. Fairness in deep learning: A computational perspective. IEEE Intell. Syst. 36(4), 25\u201334 (2020).","journal-title":"IEEE Intell. Syst."},{"key":"11012_CR58","unstructured":"Adebayo, J. & Kagal, L. Iterative orthogonal feature projection for diagnosing bias in black-box models. arXiv preprint arXiv:1611.04967 (2016)."},{"key":"11012_CR59","unstructured":"Wadsworth, C., Vera, F. & Piech, C. Achieving fairness through adversarial learning: an application to recidivism prediction. arXiv preprint arXiv:1807.00199 (2018)."},{"key":"11012_CR60","doi-asserted-by":"crossref","unstructured":"Cesaro, J. & Cozman, F.\u00a0G. Measuring unfairness through game-theoretic interpretability. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 253\u2013264 (Springer, 2019).","DOI":"10.1007\/978-3-030-43823-4_22"},{"key":"11012_CR61","doi-asserted-by":"crossref","unstructured":"Kleinberg, J. & Mullainathan, S. Simplicity creates inequity: implications for fairness, stereotypes, and interpretability. In Proceedings of the 2019 ACM Conference on Economics and Computation, 807\u2013808 (2019).","DOI":"10.1145\/3328526.3329621"},{"key":"11012_CR62","unstructured":"Jabbari, S., Ou, H.-C., Lakkaraju, H. & Tambe, M. An empirical study of the trade-offs between interpretability and fairness. ICML 2020 Workshop on Human Interpretability in Machine Learning (2020)."},{"key":"11012_CR63","unstructured":"Wang, C., Han, B., Patel, B., Mohideen, F. & Rudin, C. In pursuit of interpretable, fair and accurate machine learning for criminal recidivism prediction. arXiv preprint arXiv:2005.04176 (2020)."},{"key":"11012_CR64","unstructured":"Sjoding, M. et\u00a0al. Democratizing ehr analyses a comprehensive pipeline for learning from clinical data. Machine Learning For Healthcare (Clinical Abstracts Track) (2019)."},{"key":"11012_CR65","doi-asserted-by":"crossref","unstructured":"Song, W. et\u00a0al. Autoint: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 1161\u20131170 (2019).","DOI":"10.1145\/3357384.3357925"},{"key":"11012_CR66","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735\u20131780 (1997).","journal-title":"Neural Comput."},{"key":"11012_CR67","unstructured":"Bai, S., Kolter, J.\u00a0Z. & Koltun, V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)."},{"key":"11012_CR68","first-page":"5998","volume":"30","author":"A Vaswani","year":"2017","unstructured":"Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998\u20136008 (2017).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"11012_CR69","unstructured":"Guo, T., Lin, T. & Antulov-Fantulin, N. Exploring interpretable lstm neural networks over multi-variable data. In International Conference on Machine Learning, 2494\u20132504 (2019)."},{"key":"11012_CR70","unstructured":"Jain, S. & Wallace, B.\u00a0C. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 3543\u20133556 (2019)."},{"key":"11012_CR71","unstructured":"Grimsley, C., Mayfield, E. & R.S.\u00a0Bursten, J. Why attention is not explanation: Surgical intervention and causal reasoning about neural models. In Proceedings of the 12th Language Resources and Evaluation Conference, 1780\u20131790 (European Language Resources Association, Marseille, France, 2020)."},{"key":"11012_CR72","doi-asserted-by":"publisher","first-page":"1619","DOI":"10.1378\/chest.100.6.1619","volume":"100","author":"WA Knaus","year":"1991","unstructured":"Knaus, W. A. et al. The apache iii prognostic system: Risk prediction of hospital mortality for critically iii hospitalized adults. Chest 100, 1619\u20131636 (1991).","journal-title":"Chest"},{"key":"11012_CR73","doi-asserted-by":"publisher","first-page":"802","DOI":"10.1001\/jama.1996.03540100046027","volume":"276","author":"J-R Le Gall","year":"1996","unstructured":"Le Gall, J.-R. et al. The logistic organ dysfunction system: A new way to assess organ dysfunction in the intensive care unit. Jama 276, 802\u2013810 (1996).","journal-title":"Jama"},{"key":"11012_CR74","doi-asserted-by":"publisher","first-page":"1711","DOI":"10.1097\/CCM.0b013e31828a24fe","volume":"41","author":"AE Johnson","year":"2013","unstructured":"Johnson, A. E., Kramer, A. A. & Clifford, G. D. A new severity of illness scale using a subset of acute physiology and chronic health evaluation data elements shows comparable predictive accuracy. Crit. Care Med. 41, 1711\u20131718 (2013).","journal-title":"Crit. Care Med."},{"key":"11012_CR75","doi-asserted-by":"publisher","first-page":"2957","DOI":"10.1001\/jama.1993.03510240069035","volume":"270","author":"J-R Le Gall","year":"1993","unstructured":"Le Gall, J.-R., Lemeshow, S. & Saulnier, F. A new simplified acute physiology score (saps ii) based on a European\/north American multicenter study. Jama 270, 2957\u20132963 (1993).","journal-title":"Jama"},{"key":"11012_CR76","doi-asserted-by":"publisher","first-page":"1644","DOI":"10.1378\/chest.101.6.1644","volume":"101","author":"RC Bone","year":"1992","unstructured":"Bone, R. C. et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest 101, 1644\u20131655 (1992).","journal-title":"Chest"},{"key":"11012_CR77","doi-asserted-by":"crossref","unstructured":"Vincent, J.-L. et\u00a0al. The sofa (sepsis-related organ failure assessment) score to describe organ dysfunction\/failure (1996).","DOI":"10.1007\/BF01709751"},{"key":"11012_CR78","unstructured":"Wong, A., Wang, X.\u00a0Y. & Hryniowski, A. How much can we really trust you? towards simple, interpretable trust quantification metrics for deep neural networks. arXiv preprint arXiv:2009.05835 (2020)."},{"key":"11012_CR79","doi-asserted-by":"publisher","first-page":"54","DOI":"10.3389\/frai.2020.00054","volume":"3","author":"M Cheng","year":"2020","unstructured":"Cheng, M., Nazarian, S. & Bogdan, P. There is hope after all: Quantifying opinion and trustworthiness in neural networks. Front. Artif. Intell. 3, 54 (2020).","journal-title":"Front. Artif. Intell."},{"key":"11012_CR80","doi-asserted-by":"crossref","unstructured":"Chen, J., Kallus, N., Mao, X., Svacha, G. & Udell, M. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the conference on fairness, accountability, and transparency, 339\u2013348 (2019).","DOI":"10.1145\/3287560.3287594"},{"key":"11012_CR81","doi-asserted-by":"publisher","first-page":"1479","DOI":"10.1001\/jama.2017.14418","volume":"318","author":"CJ Yarnell","year":"2017","unstructured":"Yarnell, C. J. et al. Association between immigrant status and end-of-life care in Otario, Canada. JAMA 318, 1479\u20131488 (2017).","journal-title":"JAMA"},{"key":"11012_CR82","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.jpainsymman.2015.08.008","volume":"51","author":"JJ Lee","year":"2016","unstructured":"Lee, J. J., Long, A. C., Curtis, J. R. & Engelberg, R. A. The influence of race\/ethnicity and education on family ratings of the quality of dying in the ICU. J. Pain Symp. Manage. 51, 9\u201316. https:\/\/doi.org\/10.1016\/j.jpainsymman.2015.08.008 (2016).","journal-title":"J. Pain Symp. Manage."},{"key":"11012_CR83","first-page":"666","volume":"94","author":"A Nelson","year":"2002","unstructured":"Nelson, A. Unequal treatment: Confronting racial and ethnic disparities in health care. J. Natl. Med. Assoc. 94, 666 (2002).","journal-title":"J. Natl. Med. Assoc."},{"key":"11012_CR84","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.jcrc.2013.08.023","volume":"29","author":"MA Rubin","year":"2014","unstructured":"Rubin, M. A., Dhar, R. & Diringer, M. N. Racial differences in withdrawal of mechanical ventilation do not alter mortality in neurologically injured patients. J. Crit. Care 29, 49\u201353 (2014).","journal-title":"J. Crit. Care"},{"key":"11012_CR85","unstructured":"Kleinberg, J., Mullainathan, S. & Raghavan, M. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016)."},{"key":"11012_CR86","first-page":"728","volume":"33","author":"P Lahoti","year":"2021","unstructured":"Lahoti, P. et al. Fairness without demographics through adversarially reweighted learning. Adv. Neural Inf. Process. Syst. 33, 728\u2013740 (2021).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"11012_CR87","unstructured":"Corbett-Davies, S. & Goel, S. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018)."},{"key":"11012_CR88","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/MCSE.2007.55","volume":"9","author":"JD Hunter","year":"2007","unstructured":"Hunter, J. D. Matplotlib: A 2d graphics environment. Computi. Sci. Eng. 9, 90\u201395. https:\/\/doi.org\/10.1109\/MCSE.2007.55 (2007).","journal-title":"Computi. Sci. Eng."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-11012-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-11012-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-11012-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T02:20:36Z","timestamp":1669342836000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-11012-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,3]]},"references-count":88,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["11012"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-11012-2","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,3]]},"assertion":[{"value":"7 April 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 May 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"7166"}}