{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T02:48:02Z","timestamp":1790304482725,"version":"4.1.0"},"reference-count":53,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T00:00:00Z","timestamp":1646870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"publisher","award":["16\/RC\/3948"],"award-info":[{"award-number":["16\/RC\/3948"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Irish Health Research Board","award":["ICE\/2012\/6 JPND\/2013\/1"],"award-info":[{"award-number":["ICE\/2012\/6 JPND\/2013\/1"]}]},{"name":"American ALS Association","award":["17 CM-324"],"award-info":[{"award-number":["17 CM-324"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JPM"],"abstract":"<jats:p>Amyotrophic Lateral Sclerosis (ALS), also known as Motor Neuron Disease (MND), is a rare and fatal neurodegenerative disease. As ALS is currently incurable, the aim of the treatment is mainly to alleviate symptoms and improve quality of life (QoL). We designed a prototype Clinical Decision Support System (CDSS) to alert clinicians when a person with ALS is experiencing low QoL in order to inform and personalise the support they receive. Explainability is important for the success of a CDSS and its acceptance by healthcare professionals. The aim of this work isto announce our prototype (C-ALS), supported by a first short evaluation of its explainability. Given the lack of similar studies and systems, this work is a valid proof-of-concept that will lead to future work. We developed a CDSS that was evaluated by members of the team of healthcare professionals that provide care to people with ALS in the ALS\/MND Multidisciplinary Clinic in Dublin, Ireland. We conducted a user study where participants were asked to review the CDSS and complete a short survey with a focus on explainability. Healthcare professionals demonstrated some uncertainty in understanding the system\u2019s output. Based on their feedback, we altered the explanation provided in the updated version of our CDSS. C-ALS provides local explanations of its predictions in a post-hoc manner, using SHAP (SHapley Additive exPlanations). The CDSS predicts the risk of low QoL in the form of a probability, a bar plot shows the feature importance for the specific prediction, along with some verbal guidelines on how to interpret the results. Additionally, we provide the option of a global explanation of the system\u2019s function in the form of a bar plot showing the average importance of each feature. C-ALS is available online for academic use.<\/jats:p>","DOI":"10.3390\/jpm12030435","type":"journal-article","created":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T11:46:47Z","timestamp":1646912807000},"page":"435","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["A Clinical Decision Support System for the Prediction of Quality of Life in ALS"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8285-1997","authenticated-orcid":false,"given":"Anna Markella","family":"Antoniadi","sequence":"first","affiliation":[{"name":"UCD School of Computer Science, University College Dublin, Dublin 4, Ireland"},{"name":"FutureNeuro SFI Research Centre, RCSI University of Medicine and Health Sciences, Dublin 2, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8232-8020","authenticated-orcid":false,"given":"Miriam","family":"Galvin","sequence":"additional","affiliation":[{"name":"Academic Unit of Neurology, Trinity Biomedical Sciences Institute, Trinity College Dublin, Dublin 2, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7326-4432","authenticated-orcid":false,"given":"Mark","family":"Heverin","sequence":"additional","affiliation":[{"name":"Academic Unit of Neurology, Trinity Biomedical Sciences Institute, Trinity College Dublin, Dublin 2, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7236-3965","authenticated-orcid":false,"given":"Lan","family":"Wei","sequence":"additional","affiliation":[{"name":"UCD School of Computer Science, University College Dublin, Dublin 4, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2610-1291","authenticated-orcid":false,"given":"Orla","family":"Hardiman","sequence":"additional","affiliation":[{"name":"FutureNeuro SFI Research Centre, RCSI University of Medicine and Health Sciences, Dublin 2, Ireland"},{"name":"Academic Unit of Neurology, Trinity Biomedical Sciences Institute, Trinity College Dublin, Dublin 2, Ireland"},{"name":"Department of Neurology, National Neuroscience Centre, Beaumont Hospital, Dublin 9, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7696-1364","authenticated-orcid":false,"given":"Catherine","family":"Mooney","sequence":"additional","affiliation":[{"name":"UCD School of Computer Science, University College Dublin, Dublin 4, Ireland"},{"name":"FutureNeuro SFI Research Centre, RCSI University of Medicine and Health Sciences, Dublin 2, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1001\/jama.1987.03400010065029","article-title":"Computer programs to support clinical decision making","volume":"258","author":"Shortliffe","year":"1987","journal-title":"JAMA"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1038\/s41746-020-0221-y","article-title":"An overview of clinical decision support systems: Benefits, risks, and strategies for success","volume":"3","author":"Sutton","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s10877-016-9849-1","article-title":"Precision diagnosis: A view of the clinical decision support systems (CDSS) landscape through the lens of critical care","volume":"31","author":"Belard","year":"2017","journal-title":"J. Clin. Monit. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1136\/bmj.38398.500764.8F","article-title":"Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success","volume":"330","author":"Kawamoto","year":"2005","journal-title":"BMJ"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"79","DOI":"10.14236\/jhi.v16i2.679","article-title":"The use and effectiveness of electronic clinical decision support tools in the ambulatory\/primary care setting: A systematic review of the literature","volume":"16","author":"Bryan","year":"2008","journal-title":"J. Innov. Health Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1177\/2150131914546325","article-title":"Evaluation of the effect of decision support on the efficiency of primary care providers in the outpatient practice","volume":"6","author":"Wagholikar","year":"2015","journal-title":"J. Prim. Care Community Health"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3359206","article-title":"\u201cHello AI\u201d: Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making","volume":"Volume 3","author":"Cai","year":"2019","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Antoniadi, A.M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B.A., and Mooney, C. (2021). Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review. Appl. Sci., 11.","DOI":"10.3390\/app11115088"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: The convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bussone, A., Stumpf, S., and O\u2019Sullivan, D. (2015, January 21\u201323). The role of explanations on trust and reliance in clinical decision support systems. Proceedings of the 2015 International Conference on Healthcare Informatics, Dallas, TX, USA.","DOI":"10.1109\/ICHI.2015.26"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Matthews, M., Doherty, G., Coyle, D., and Sharry, J. (2008). Designing mobile applications to support mental health interventions. Handbook of Research on User Interface Design and Evaluation for Mobile Technology, IGI Global.","DOI":"10.4018\/978-1-59904-871-0.ch038"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., and Kagal, L. (2018, January 1\u20133). Explaining explanations: An overview of interpretability of machine learning. Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy.","DOI":"10.1109\/DSAA.2018.00018"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e1312","DOI":"10.1002\/widm.1312","article-title":"Causability and explainability of artificial intelligence in medicine","volume":"9","author":"Holzinger","year":"2019","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_15","unstructured":"Tonekaboni, S., Joshi, S., McCradden, M.D., and Goldenberg, A. (2019, January 8\u201310). What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use. Proceedings of the Machine Learning for Healthcare Conference, Ann Arbor, MI, USA."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat. Biomed. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kulesza, T., Stumpf, S., Burnett, M., and Kwan, I. (2012, January 5\u201310). Tell me more? The effects of mental model soundness on personalizing an intelligent agent. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA.","DOI":"10.1145\/2207676.2207678"},{"key":"ref_18","unstructured":"World Health Organization (2021). Ethics and Governance of Artificial Intelligence for Health: WHO Guidance."},{"key":"ref_19","unstructured":"General Data Protection Regulation (GDPR) (2019, July 10). General Data Protection Regulation (GDPR)\u2014Final Text Neatly Arranged. Available online: https:\/\/gdpr-info.eu\/."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.ejca.2021.06.049","article-title":"Skin cancer classification via convolutional neural networks: Systematic review of studies involving human experts","volume":"156","author":"Maron","year":"2021","journal-title":"Eur. J. Cancer"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e271","DOI":"10.1016\/S2589-7500(19)30123-2","article-title":"A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis","volume":"1","author":"Liu","year":"2019","journal-title":"Lancet Digit. Health"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e19918","DOI":"10.2196\/19918","article-title":"Is artificial intelligence better than human clinicians in predicting patient outcomes?","volume":"22","author":"Lee","year":"2020","journal-title":"J. Med. Internet Res."},{"key":"ref_25","first-page":"1","article-title":"A decision support system for quality of life in head and neck oncology patients","volume":"4","author":"Rocha","year":"2012","journal-title":"Head Neck Oncol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.jns.2007.03.001","article-title":"A review of psychosocial aspects of motor neurone disease","volume":"258","author":"McLeod","year":"2007","journal-title":"J. Neurol. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/1750-1172-4-3","article-title":"Amyotrophic lateral sclerosis","volume":"4","author":"Wijesekera","year":"2009","journal-title":"Orphanet J. Rare Dis."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12955-020-01551-5","article-title":"Individual quality of life in spousal ALS patient-caregiver dyads","volume":"18","author":"Galvin","year":"2020","journal-title":"Health Qual. Life Outcomes"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1177\/026921639500900306","article-title":"The McGill Quality of Life Questionnaire: A measure of quality of life appropriate for people with advanced disease. A preliminary study of validity and acceptability","volume":"9","author":"Cohen","year":"1995","journal-title":"Palliat. Med."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Antoniadi, A.M., Galvin, M., Heverin, M., Hardiman, O., and Mooney, C. (2021, January 22\u201326). Development of an Explainable Clinical Decision Support System for the Prediction of Patient Quality of Life in Amyotrophic Lateral Sclerosis. Proceedings of the 36th ACM\/SIGAPP Symposium on Applied Computing (SAC \u201921), Virtual Event, Korea.","DOI":"10.1145\/3412841.3441940"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/3477127.3477128","article-title":"Prediction of quality of life in people with ALS: On the road towards explainable clinical decision support","volume":"21","author":"Antoniadi","year":"2021","journal-title":"ACM SIGAPP Appl. Comput. Rev."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Antoniadi, A.M., Galvin, M., Heverin, M., Hardiman, O., and Mooney, C. (December, January 30). Identifying Features That Are Predictive of Quality of Life in People With Amyotrophic Lateral Sclerosis. Proceedings of the 2020 IEEE International Conference on Healthcare Informatics (ICHI), Oldenburg, Germany.","DOI":"10.1109\/ICHI48887.2020.9374298"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3","DOI":"10.3892\/br.2017.922","article-title":"Personalized medicine could transform healthcare","volume":"7","author":"Mathur","year":"2017","journal-title":"Biomed. Rep."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1093\/pcmedi\/pbz001","article-title":"Maximizing quality of life remains an ultimate goal in the era of precision medicine: Exemplified by lung cancer","volume":"2","author":"Yang","year":"2019","journal-title":"Precis. Clin. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13040-017-0142-8","article-title":"EFS: An ensemble feature selection tool implemented as R-package and web-application","volume":"10","author":"Neumann","year":"2017","journal-title":"BioData Min."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201916), San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy Function Approximation: A Gradient Boosting Machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_38","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30, Curran Associates, Inc."},{"key":"ref_39","first-page":"326","article-title":"Survey Fatigue During the COVID-19 Pandemic: An Analysis of Neurosurgery Survey Response Rates","volume":"8","author":"Egiz","year":"2021","journal-title":"Front. Surg."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I.D., and Gebru, T. (2019). Model cards for model reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency, ACM.","DOI":"10.1145\/3287560.3287596"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1001\/archinternmed.2012.2364","article-title":"Factors important to patients\u2019 quality of life at the end of life","volume":"172","author":"Zhang","year":"2012","journal-title":"Arch. Intern. Med."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1191\/026921698670234618","article-title":"Measuring quality of life for patients with terminal illness: The Missoula\u2013VITAS\u00ae quality of life index","volume":"12","author":"Byock","year":"1998","journal-title":"Palliat. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1080\/17482960802163721","article-title":"Quality of life in amyotrophic lateral sclerosis\/motor neuron disease: A structured review","volume":"10","author":"Epton","year":"2009","journal-title":"Amyotroph. Lateral Scler."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-018-0061-1","article-title":"Machine learning and medical education","volume":"1","author":"Kolachalama","year":"2018","journal-title":"NPJ Digit. Med."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"102684","DOI":"10.1016\/j.ijhcs.2021.102684","article-title":"Human-centered XAI: Developing design patterns for explanations of clinical decision support systems","volume":"154","author":"Schoonderwoerd","year":"2021","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cag.2020.06.004","article-title":"A visual approach to explainable computerized clinical decision support","volume":"91","author":"Stoehr","year":"2020","journal-title":"Comput. Graph."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.artmed.2019.01.001","article-title":"Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach","volume":"94","author":"Lamy","year":"2019","journal-title":"Artif. Intell. Med."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1001\/jama.293.10.1223","article-title":"Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review","volume":"293","author":"Garg","year":"2005","journal-title":"JAMA"},{"key":"ref_49","first-page":"60","article-title":"Designing computerized decision support that works for clinicians and families","volume":"41","author":"Fiks","year":"2011","journal-title":"Curr. Probl. Pediatr. Adolesc. Health Care"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13012-017-0644-2","article-title":"What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation","volume":"12","author":"Liberati","year":"2017","journal-title":"Implement. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1170","DOI":"10.1038\/s41598-022-05112-2","article-title":"An explainable machine learning-based clinical decision support system for prediction of gestational diabetes mellitus","volume":"12","author":"Du","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"12237","DOI":"10.1038\/s41598-021-91632-2","article-title":"Prediction of caregiver quality of life in amyotrophic lateral sclerosis using explainable machine learning","volume":"11","author":"Antoniadi","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"e033109","DOI":"10.1136\/bmjopen-2019-033109","article-title":"Prediction of caregiver burden in amyotrophic lateral sclerosis: A machine learning approach using random forests applied to a cohort study","volume":"10","author":"Antoniadi","year":"2020","journal-title":"BMJ Open"}],"container-title":["Journal of Personalized Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4426\/12\/3\/435\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:34:15Z","timestamp":1760135655000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4426\/12\/3\/435"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,10]]},"references-count":53,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["jpm12030435"],"URL":"https:\/\/doi.org\/10.3390\/jpm12030435","relation":{},"ISSN":["2075-4426"],"issn-type":[{"value":"2075-4426","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,10]]}}}