{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:31:52Z","timestamp":1772253112250,"version":"3.50.1"},"reference-count":103,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,8]],"date-time":"2021-01-08T00:00:00Z","timestamp":1610064000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Artificial intelligence (AI)-based systems have achieved significant success in healthcare since 2016, and AI models have accomplished medical tasks, at or above the performance levels of humans. Despite these achievements, various challenges exist in the application of AI in healthcare. One of the main challenges is safety, which is related to unsafe and incorrect actions and recommendations by AI algorithms. In response to the need to address the safety challenges, this research aimed to develop a safety controlling system (SCS) framework to reduce the risk of potential healthcare-related incidents. The framework was developed by adopting the multi-attribute value model approach (MAVT), which comprises four symmetrical parts: extracting attributes, generating weights for the attributes, developing a rating scale, and finalizing the system. The framework represents a set of attributes in different layers and can be used as a checklist in healthcare institutions with implemented AI models. Having these attributes in healthcare systems will lead to high scores in the SCS, which indicates safe application of AI models. The proposed framework provides a basis for implementing and monitoring safety legislation, identifying the risks in AI models\u2019 activities, improving human-AI interactions, preventing incidents from occurring, and having an emergency plan for remaining risks.<\/jats:p>","DOI":"10.3390\/sym13010102","type":"journal-article","created":{"date-parts":[[2021,1,10]],"date-time":"2021-01-10T23:03:42Z","timestamp":1610319822000},"page":"102","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["Controlling Safety of Artificial Intelligence-Based Systems in Healthcare"],"prefix":"10.3390","volume":"13","author":[{"given":"Mohammad Reza","family":"Davahli","sequence":"first","affiliation":[{"name":"Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9134-3441","authenticated-orcid":false,"given":"Waldemar","family":"Karwowski","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5711-1498","authenticated-orcid":false,"given":"Krzysztof","family":"Fiok","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL 32816, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5235-4586","authenticated-orcid":false,"given":"Thomas","family":"Wan","sequence":"additional","affiliation":[{"name":"Health Management and Informatics, University of Central Florida, Orlando, FL 32816, USA"}]},{"given":"Hamid R.","family":"Parsaei","sequence":"additional","affiliation":[{"name":"Department of Industrial &amp; Systems Engineering, Texas A&amp;M University, College Station, TX 77843, USA"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1038\/s41591-018-0307-0","article-title":"The Practical Implementation of Artificial Intelligence Technologies in Medicine","volume":"25","author":"He","year":"2019","journal-title":"Nat. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-Performance Medicine: The Convergence of Human and Artificial Intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"S36","DOI":"10.1016\/j.metabol.2017.01.011","article-title":"Artificial Intelligence in Medicine","volume":"69","author":"Hamet","year":"2017","journal-title":"Metabolism"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1037\/h0048495","article-title":"Elements of a Theory of Human Problem Solving","volume":"65","author":"Newell","year":"1958","journal-title":"Psychol. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1147\/rd.33.0210","article-title":"Some Studies in Machine Learning Using the Game of Checkers","volume":"3","author":"Samuel","year":"1959","journal-title":"IBM J. Res. Dev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1001\/jama.1961.03040290005002","article-title":"A Mathematical Approach to Medical Diagnosis: Application to Congenital Heart Disease","volume":"177","author":"Warner","year":"1961","journal-title":"JAMA"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/365153.365168","article-title":"ELIZA\u2014A Computer Program for the Study of Natural Language Communication between Man and Machine","volume":"9","author":"Weizenbaum","year":"1966","journal-title":"Commun. ACM"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1136\/bmj.2.5804.9","article-title":"Computer-Aided Diagnosis of Acute Abdominal Pain","volume":"2","author":"Leaper","year":"1972","journal-title":"Br. Med. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"80","DOI":"10.7326\/0003-4819-108-1-80","article-title":"Artificial Intelligence in Medical Diagnosis","volume":"108","author":"Szolovits","year":"1988","journal-title":"Ann. Intern. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1038\/538020a","article-title":"Can We Open the Black Box of AI?","volume":"538","author":"Castelvecchi","year":"2016","journal-title":"Nat. News"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1287","DOI":"10.1126\/science.aaw4399","article-title":"Adversarial Attacks on Medical Machine Learning","volume":"363","author":"Finlayson","year":"2019","journal-title":"Science"},{"key":"ref_12","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet Classification with Deep Convolutional Neural Networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_13","unstructured":"Marcus, G. (2018). Deep Learning: A Critical Appraisal. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1920","DOI":"10.1161\/CIRCULATIONAHA.115.001593","article-title":"Machine Learning in Medicine","volume":"132","year":"2015","journal-title":"Circulation"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1016\/j.cels.2017.10.014","article-title":"Association of Omics Features with Histopathology Patterns in Lung Adenocarcinoma","volume":"5","author":"Yu","year":"2017","journal-title":"Cell Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R.M. (2017, January 21\u201326). Chestx-Ray8: Hospital-Scale Chest x-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.369"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"015001","DOI":"10.1088\/1361-6579\/aaf34d","article-title":"Detecting and Interpreting Myocardial Infarction Using Fully Convolutional Neural Networks","volume":"40","author":"Strodthoff","year":"2019","journal-title":"Physiol. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2199","DOI":"10.1001\/jama.2017.14585","article-title":"Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women with Breast Cancer","volume":"318","author":"Bejnordi","year":"2017","journal-title":"JAMA"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1038\/s41551-018-0305-z","article-title":"Artificial Intelligence in Healthcare","volume":"2","author":"Yu","year":"2018","journal-title":"Nat. Biomed. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gerke, S., Minssen, T., and Cohen, I.G. (2020). Ethical and Legal Challenges of Artificial Intelligence-Driven Health Care. SSRN Electron. J.","DOI":"10.2139\/ssrn.3570129"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M.-E., Ruggieri, S., Turini, F., Papadopoulos, S., and Krasanakis, E. (2019). Bias in Data-Driven Artificial Intelligence Systems-An Introductory Survey. Wiley Interdiscip. Rev.-Data Min. Knowl. Discov., e1356.","DOI":"10.1002\/widm.1356"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Vandewiele, G., De Backere, F., Lannoye, K., Vanden Berghe, M., Janssens, O., Van Hoecke, S., Keereman, V., Paemeleire, K., Ongenae, F., and De Turck, F. (2018). A Decision Support System to Follow up and Diagnose Primary Headache Patients Using Semantically Enriched Data. BMC Med. Inform. Decis. Mak., 18.","DOI":"10.1186\/s12911-018-0679-6"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TVCG.2018.2865027","article-title":"RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records","volume":"25","author":"Kwon","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","article-title":"Clinically Applicable Deep Learning for Diagnosis and Referral in Retinal Disease","volume":"24","author":"Ledsam","year":"2018","journal-title":"Nat. Med."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1136\/bjophthalmol-2018-313173","article-title":"Artificial Intelligence and Deep Learning in Ophthalmology","volume":"103","author":"Ting","year":"2019","journal-title":"Br. J. Ophthalmol."},{"key":"ref_28","unstructured":"Bleicher, A. (2020, March 04). Demystifying the Black Box That Is AI. Available online: https:\/\/www.scientificamerican.com\/article\/demystifying-the-black-box-that-is-ai\/."},{"key":"ref_29","unstructured":"Heaven, W.D. (2020, March 04). Why Asking an AI to Explain Itself Can Make Things Worse. Available online: https:\/\/www.technologyreview.com\/s\/615110\/why-asking-an-ai-to-explain-itself-can-make-things-worse\/."},{"key":"ref_30","unstructured":"Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2016). Understanding Deep Learning Requires Rethinking Generalization. arXiv."},{"key":"ref_31","unstructured":"Schemelzer, R. (2020, April 10). Understanding Explainable AI. Available online: https:\/\/www.forbes.com\/sites\/cognitiveworld\/2019\/07\/23\/understanding-explainable-ai\/#406c97957c9e."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1002\/hast.973","article-title":"Artificial Intelligence and Black-Box Medical Decisions: Accuracy versus Explainability","volume":"49","author":"London","year":"2019","journal-title":"Hastings Cent. Rep."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"59","DOI":"10.7326\/M19-2548","article-title":"Should Health Care Demand Interpretable Artificial Intelligence or Accept \u201cBlack Box\u201d Medicine?","volume":"172","author":"Wang","year":"2019","journal-title":"Ann. Intern. Med."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Buehler, M., Iagnemma, K., and Singh, S. (2009). The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, Springer.","DOI":"10.1007\/978-3-642-03991-1"},{"key":"ref_35","first-page":"20","article-title":"Explaining Supervised Learning Models: A Preliminary Study on Binary Classifiers","volume":"28","author":"Wang","year":"2020","journal-title":"Ergon. Des."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1584","DOI":"10.1016\/j.buildenv.2005.06.005","article-title":"Developing a Model to Measure the Effectiveness of Safety Management Systems of Construction Sites","volume":"41","author":"Teo","year":"2006","journal-title":"Build. Environ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.jlp.2006.10.002","article-title":"Safety Management System: Development and Validation of a Multidimensional Scale","volume":"20","year":"2007","journal-title":"J. Loss Prev. Process Ind."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., G\u00f8tzsche, P.C., Ioannidis, J.P.A., Clarke, M., Devereaux, P.J., Kleijnen, J., and Moher, D. (2009). The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med., 6.","DOI":"10.2427\/5768"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Davahli, M.R., Karwowski, W., Gutierrez, E., Fiok, K., Wr\u00f3bel, G., Taiar, R., and Ahram, T. (2020). Identification and Prediction of Human Behavior through Mining of Unstructured Textual Data. Symmetry, 12.","DOI":"10.3390\/sym12111902"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1108\/11766091111162070","article-title":"The qualitative research interview","volume":"8","author":"Qu","year":"2011","journal-title":"Qual. Res. Account. Manag."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1111\/j.1365-2929.2006.02418.x","article-title":"The Qualitative Research Interview","volume":"40","author":"Crabtree","year":"2006","journal-title":"Med. Educ."},{"key":"ref_42","unstructured":"(2020, December 27). HAWKIRB Studies That Are Not Human Subjects Research. Available online: https:\/\/hso.research.uiowa.edu\/studies-are-not-human-subjects-research."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Davahli, M.R., Karwowski, W., Fiok, K., Wan, T.T., and Parsaei, H.R. (2020). A Safety Controlling System Framework for Implementing Artificial Intelligence in Healthcare. Preprints, 2020120313.","DOI":"10.20944\/preprints202012.0313.v1"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1198\/108571105X46642","article-title":"Species Associations: The Kendall Coefficient of Concordance Revisited","volume":"10","author":"Legendre","year":"2005","journal-title":"J. Agric. Biol. Environ. Stat."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"102602","DOI":"10.1016\/j.futures.2020.102602","article-title":"The Adaptiveness of the Healthcare System to the Fourth Industrial Revolution: A Preliminary Analysis","volume":"122","author":"Klich","year":"2020","journal-title":"Futures"},{"key":"ref_46","unstructured":"Hale, A.R., and Baram, M.S. (1998). Safety Management: The Challenge of Change, Pergamon Oxford."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1001\/jama.2019.21579","article-title":"Artificial Intelligence in Health Care: A Report From the National Academy of Medicine","volume":"323","author":"Matheny","year":"2020","journal-title":"JAMA"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1182","DOI":"10.3389\/fgene.2019.01182","article-title":"Graph Embedding Deep Learning Guides Microbial Biomarkers\u2019 Identification","volume":"10","author":"Zhu","year":"2019","journal-title":"Front. Genet."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1136\/bmjqs-2018-008370","article-title":"Artificial Intelligence, Bias and Clinical Safety","volume":"28","author":"Challen","year":"2019","journal-title":"BMJ Qual. Saf."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Kelly, C.J., Karthikesalingam, A., Suleyman, M., Corrado, G., and King, D. (2019). Key Challenges for Delivering Clinical Impact with Artificial Intelligence. BMC Med., 17.","DOI":"10.1186\/s12916-019-1426-2"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"e181404","DOI":"10.1001\/jamanetworkopen.2018.1404","article-title":"Machine Learning for Prediction in Electronic Health Data","volume":"1","author":"Rose","year":"2018","journal-title":"JAMA Netw. Open"},{"key":"ref_52","unstructured":"U.S. Food and Drug Administration (2019). Proposed Regulatory Framework for Modifications to Artificial Intelligence\/Machine Learning (AI\/ML)-Based Software as a Medical Device (SaMD)-Discussion Paper, Discussion Paper and Request for Feedback."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-019-0148-3","article-title":"Artificial Intelligence and Machine Learning in Clinical Development: A Translational Perspective","volume":"2","author":"Shah","year":"2019","journal-title":"NPJ Digit. Med."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"S103","DOI":"10.1038\/d41586-019-02872-2","article-title":"A Fairer Way Forward for AI in Health Care","volume":"573","author":"Nordling","year":"2019","journal-title":"Nature"},{"key":"ref_55","unstructured":"Stewart, E. (2020, March 07). Self-Driving Cars Have to Be Safer than Regular Cars. The Question Is How Much. Available online: https:\/\/www.vox.com\/recode\/2019\/5\/17\/18564501\/self-driving-car-morals-safety-tesla-waymo."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"2184","DOI":"10.1001\/jama.2017.14580","article-title":"Deep Learning Algorithms for Detection of Lymph Node Metastases from Breast Cancer: Helping Artificial Intelligence Be Seen","volume":"318","author":"Golden","year":"2017","journal-title":"JAMA"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P.N., and Inkpen, K. (2019, January 4\u20139). Guidelines for Human-AI Interaction. Proceedings of the 2019 Chi Conference on Human Factors in Computing Systems, Scotland, UK.","DOI":"10.1145\/3290605.3300233"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1001\/jamainternmed.2019.5170","article-title":"Leading the Call for Reform of Medical Device Safety Surveillance","volume":"180","author":"Salazar","year":"2020","journal-title":"JAMA Intern. Med."},{"key":"ref_59","first-page":"348","article-title":"Challenges in Evaluating and Standardizing Medical Devices in Health Care Facilities","volume":"33","author":"Ventola","year":"2008","journal-title":"Pharm. Ther."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.jmsy.2016.05.002","article-title":"Combined Strength of Holons, Agents and Function Blocks in Cyber-Physical Systems","volume":"40","author":"Wang","year":"2016","journal-title":"J. Manuf. Syst."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-019-0168-z","article-title":"Medical Device Surveillance with Electronic Health Records","volume":"2","author":"Callahan","year":"2019","journal-title":"NPJ Digit. Med."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1093\/jlb\/lsz013","article-title":"Integrating Artificial Intelligence into Health Care through Data Access: Can the GDPR Act as a Beacon for Policymakers?","volume":"6","author":"Forcier","year":"2019","journal-title":"J. Law Biosci."},{"key":"ref_63","unstructured":"Westerheide, F. (2020, March 09). The Artificial Intelligence Industry and Global Challenges. Available online: https:\/\/www.forbes.com\/sites\/cognitiveworld\/2019\/11\/27\/the-artificial-intelligence-industry-and-global-challenges\/."},{"key":"ref_64","unstructured":"Nicola, S., Behrmann, E., and Mawad, M. (2018). It\u2019s a Good Thing Europe\u2019s Autonomous Car Testing Is Slow, Bloomberg."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1038\/s41591-018-0272-7","article-title":"Privacy in the Age of Medical Big Data","volume":"25","author":"Price","year":"2019","journal-title":"Nat. Med."},{"key":"ref_66","unstructured":"Wenyan, W. (2020, February 29). China Is Waking up to Data Protection and Privacy. Here\u2019s Why That Matters. Available online: https:\/\/www.weforum.org\/agenda\/2019\/11\/china-data-privacy-laws-guideline\/."},{"key":"ref_67","unstructured":"Lindsey, N. (2020, February 29). China\u2019s Privacy Challenges with AI and Mobile Apps. Available online: https:\/\/www.cpomagazine.com\/data-privacy\/chinas-privacy-challenges-with-ai-and-mobile-apps\/."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1038\/d41586-019-02360-7","article-title":"Will China Lead the World in AI by 2030?","volume":"572","year":"2019","journal-title":"Nature"},{"key":"ref_69","unstructured":"US Department of Health and Human Services (2017). Software as a Medical Device (SAMD): Clinical Evaluation, Guidance for Industry and Food and Drug Administration Staff, 2017."},{"key":"ref_70","unstructured":"(2020, November 07). Digital Health Innovation Action Plan, Available online: https:\/\/www.fda.gov\/downloads\/MedicalDevices\/DigitalHealth\/UCM568735.pdf."},{"key":"ref_71","first-page":"187","article-title":"Sparse Algorithms Are Not Stable: A No-Free-Lunch Theorem","volume":"34","author":"Xu","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_72","unstructured":"Sun, W. (2020, March 09). Stability of Machine Learning Algorithms. Open Access Diss. Available online: https:\/\/docs.lib.purdue.edu\/dissertations\/AAI3720039\/."},{"key":"ref_73","unstructured":"Cheatham, B., Javanmardian, K., and Samandari, H. (2019). Confronting the Risks of Artificial Intelligence. McKinsey Q., 1\u20139. Available online: https:\/\/assets.noviams.com\/novi-file-uploads\/MISBO\/Shared_Resources\/AI_Resources\/Confronting-the-risks-of-artificial-intelligence-vF.pdf."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1148\/radiol.2019190613","article-title":"A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH\/RSNA\/ACR\/The Academy Workshop","volume":"291","author":"Langlotz","year":"2019","journal-title":"Radiology"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"231","DOI":"10.5566\/ias.1155","article-title":"Feedback on a Publicly Distributed Image Database: The Messidor Database","volume":"33","author":"Zhang","year":"2014","journal-title":"Image Anal. Stereol."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"12474","DOI":"10.1038\/ncomms12474","article-title":"Predicting Non-Small Cell Lung Cancer Prognosis by Fully Automated Microscopic Pathology Image Features","volume":"7","author":"Yu","year":"2016","journal-title":"Nat. Commun."},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Bhagwat, N., Viviano, J.D., Voineskos, A.N., Chakravarty, M.M., and Initiative, A.D.N. (2018). Modeling and Prediction of Clinical Symptom Trajectories in Alzheimer\u2019s Disease Using Longitudinal Data. PLoS Comput. Biol., 14.","DOI":"10.1371\/journal.pcbi.1006376"},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1001\/jamainternmed.2018.3763","article-title":"Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data","volume":"178","author":"Gianfrancesco","year":"2018","journal-title":"JAMA Intern. Med."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s10479-018-2891-2","article-title":"Massive Datasets and Machine Learning for Computational Biomedicine: Trends and Challenges","volume":"276","author":"Kocheturov","year":"2019","journal-title":"Ann. Oper. Res."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Li, Z., Wang, C., Han, M., Xue, Y., Wei, W., Li, L.-J., and Fei-Fei, L. (2018, January 18\u201322). Thoracic Disease Identification and Localization with Limited Supervision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00865"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1148\/radiol.2018180237","article-title":"Development and Validation of Deep Learning\u2013Based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs","volume":"290","author":"Nam","year":"2019","journal-title":"Radiology"},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"1636","DOI":"10.1097\/PAS.0000000000001151","article-title":"Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer","volume":"42","author":"Steiner","year":"2018","journal-title":"Am. J. Surg. Pathol."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"2388","DOI":"10.1016\/S0140-6736(18)31645-3","article-title":"Deep Learning Algorithms for Detection of Critical Findings in Head CT Scans: A Retrospective Study","volume":"392","author":"Chilamkurthy","year":"2018","journal-title":"Lancet"},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","article-title":"Man against Machine: Diagnostic Performance of a Deep Learning Convolutional Neural Network for Dermoscopic Melanoma Recognition in Comparison to 58 Dermatologists","volume":"29","author":"Haenssle","year":"2018","journal-title":"Ann. Oncol."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1148\/radiol.2017170706","article-title":"Deep Learning with Convolutional Neural Network for Differentiation of Liver Masses at Dynamic Contrast-Enhanced CT: A Preliminary Study","volume":"286","author":"Yasaka","year":"2018","journal-title":"Radiology"},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1016\/j.cell.2018.02.010","article-title":"Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning","volume":"172","author":"Kermany","year":"2018","journal-title":"Cell"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.carj.2019.03.001","article-title":"Canadian Association of Radiologists White Paper on Ethical and Legal Issues Related to Artificial Intelligence in Radiology","volume":"70","author":"Jaremko","year":"2019","journal-title":"Can. Assoc. Radiol. J."},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1634\/theoncologist.2017-0170","article-title":"Enhancing Next-Generation Sequencing-Guided Cancer Care through Cognitive Computing","volume":"23","author":"Patel","year":"2018","journal-title":"Oncologist"},{"key":"ref_89","unstructured":"(2020, March 07). CBINSIGHTS Google Healthcare with AI l CB Insights. Available online: https:\/\/www.cbinsights.com\/research\/report\/google-strategy-healthcare\/."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/srep26094","article-title":"Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records","volume":"6","author":"Miotto","year":"2016","journal-title":"Sci. Rep."},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1016\/j.cell.2018.05.015","article-title":"Next-Generation Machine Learning for Biological Networks","volume":"173","author":"Camacho","year":"2018","journal-title":"Cell"},{"key":"ref_92","unstructured":"Hill, J. (2020, March 08). Simulation: The Bedrock of AI. Available online: https:\/\/medium.com\/simudyne\/simulation-the-bedrock-of-ai-12153eaf7971."},{"key":"ref_93","unstructured":"Chawla, V. (2019). How Training AI Models In Simulated Environments Is Helping Researchers. Anal. India Mag., Available online: https:\/\/analyticsindiamag.com\/how-training-ai-models-in-simulated-environments-is-helping-researchers\/."},{"key":"ref_94","unstructured":"O\u2019Kane, S. (2020, March 02). Tesla and Waymo Are Taking Wildly Different Paths to Creating Self-Driving Cars. Available online: https:\/\/www.theverge.com\/transportation\/2018\/4\/19\/17204044\/tesla-waymo-self-driving-car-data-simulation."},{"key":"ref_95","unstructured":"Upton, R. (2019). Artificial Intelligence\u2019s Need for Health Data\u2014Finding An Ethical Balance. Hit Consult."},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"016","DOI":"10.1055\/s-0039-1677908","article-title":"AI in Health: State of the Art, Challenges, and Future Directions","volume":"28","author":"Wang","year":"2019","journal-title":"Yearb. Med. Inform."},{"key":"ref_97","unstructured":"Hsu, J. (2020, February 19). Spectrum AI Could Make Detecting Autism Easier. Available online: https:\/\/www.spectrumnews.org\/features\/deep-dive\/can-computer-diagnose-autism\/."},{"key":"ref_98","unstructured":"Christian, J., Dasgupta, N., Jordan, M., Juneja, M., Nilsen, W., and Reites, J. (2018). Digital Health and Patient Registries: Today, Tomorrow, and the Future, 21st Century Patient Registries: Registries for Evaluating Patient Outcomes: A User\u2019s Guide: 3rd Edition, Addendum [Internet]."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-020-0218-6","article-title":"SMART Markers: Collecting Patient-Generated Health Data as a Standardized Property of Health Information Technology","volume":"3","author":"Sayeed","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_100","unstructured":"(2020, April 11). U.S. Food and Drug Administration Medical Device Data Systems, Medical Image Storage Devices, and Medical Image Communications Devices, Available online: https:\/\/www.fda.gov\/regulatory-information\/search-fda-guidance-documents\/medical-device-data-systems-medical-image-storage-devices-and-medical-image-communications-devices."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/j.eng.2019.08.015","article-title":"Artificial Intelligence in Healthcare: Review and Prediction Case Studies","volume":"6","author":"Rong","year":"2020","journal-title":"Engineering"},{"key":"ref_102","unstructured":"Machanick, P. (2002). Approaches to Addressing the Memory Wall. Sch. IT Electr. Eng. Univ. QLD, Available online: https:\/\/www.researchgate.net\/profile\/Philip_Machanick\/publication\/228813498_Approaches_to_addressing_the_memory_wall\/links\/00b7d51c988e408fb3000000.pdf."},{"key":"ref_103","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1136\/bjophthalmol-2019-315016","article-title":"Glaucoma Management in the Era of Artificial Intelligence","volume":"104","author":"Devalla","year":"2020","journal-title":"Br. J. Ophthalmol."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/1\/102\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:08:34Z","timestamp":1760159314000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/1\/102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,8]]},"references-count":103,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["sym13010102"],"URL":"https:\/\/doi.org\/10.3390\/sym13010102","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202012.0313.v1","asserted-by":"object"},{"id-type":"doi","id":"10.20944\/preprints202012.0313.v2","asserted-by":"object"}]},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,8]]}}}