{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T12:20:01Z","timestamp":1776082801652,"version":"3.50.1"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2020,10,14]],"date-time":"2020-10-14T00:00:00Z","timestamp":1602633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"FCT","award":["SFRH\/BD\/113694\/2015, UIDB\/50009\/2020"],"award-info":[{"award-number":["SFRH\/BD\/113694\/2015, UIDB\/50009\/2020"]}]},{"name":"ERDF through LISBOA 2020 and FCT","award":["LISBOA-01-0247-FEDER-045948"],"award-info":[{"award-number":["LISBOA-01-0247-FEDER-045948"]}]},{"name":"National Science Foundation","award":["CNS-1518865"],"award-info":[{"award-number":["CNS-1518865"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2020,10,14]]},"abstract":"<jats:p>Clinical decision support systems have the potential to improve work flows of experts in practice (e.g. therapist's evidence-based rehabilitation assessment). However, the adoption of these systems is challenging, and the gains of these systems have not fully demonstrated yet. In this paper, we identified the needs of therapists to assess patient's functional abilities (e.g. alternative perspectives with quantitative information on patient's exercise motions). As a result, we co-designed and developed an intelligent decision support system that automatically identifies salient features of assessment using reinforcement learning to assess the quality of motion and generate patient-specific analysis. We evaluated this system with seven therapists using the dataset from 15 patients performing three exercises. The results show that therapists have higher usage intent on our system than a traditional system without patient-specific analysis ($p &lt; 0.05$). While presenting richer information ($p &lt; 0.10$), our system significantly reduces therapists' effort on assessment ($p &lt; 0.10$) and improves their agreement on assessment from 0.66 to 0.71 F1-scores ($p &lt; 0.01$). This work discusses the importance of human centered design and development of a machine learning-based decision support system that presents contextually relevant information and salient explanations on its prediction for better adoption in practice.<\/jats:p>","DOI":"10.1145\/3415227","type":"journal-article","created":{"date-parts":[[2020,10,15]],"date-time":"2020-10-15T22:27:49Z","timestamp":1602800869000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":43,"title":["Co-Design and Evaluation of an Intelligent Decision Support System for Stroke Rehabilitation Assessment"],"prefix":"10.1145","volume":"4","author":[{"given":"Min Hun","family":"Lee","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}]},{"given":"Daniel P.","family":"Siewiorek","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}]},{"given":"Asim","family":"Smailagic","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}]},{"given":"Alexandre","family":"Bernardino","sequence":"additional","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Lisboa, Portugal"}]},{"given":"Sergi","family":"Berm\u00fadez i Badia","sequence":"additional","affiliation":[{"name":"University of Madeira, Madeira-ITI, NOVA-LINCS, Funchal, Portugal"}]}],"member":"320","published-online":{"date-parts":[[2020,10,15]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300233"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2011.09.002"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-017-0760-1"},{"key":"e_1_2_1_4_1","volume-title":"Comparative accuracy of diagnosis by collective intelligence of multiple physicians vs individual physicians. JAMA network open","author":"Barnett Michael L","year":"2019"},{"key":"e_1_2_1_5_1","volume-title":"Marilyn McKay-Lyons, Edward Harrison, Robert Teasell, Margaret Harrison, et almbox.","author":"Bayley Mark T","year":"2012"},{"key":"e_1_2_1_6_1","volume-title":"IJCAI-17 workshop on explainable AI (XAI)","volume":"8","author":"Biran Or","year":"2017"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.7797"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300234"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.4108\/icst.pervasivehealth.2012.248714"},{"key":"e_1_2_1_11_1","volume-title":"Movement quantification in neurological diseases: methods and applications","author":"do Carmo Vilas-Boas Maria","year":"2016"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604056"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5898\/jhri.2.2.fasola"},{"key":"e_1_2_1_14_1","volume-title":"Global burden of stroke. Circulation research","author":"Feigin Valery L","year":"2017"},{"key":"e_1_2_1_15_1","volume-title":"Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC medical research methodology","author":"Gale Nicola K","year":"2013"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.293.10.1223"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0161-6420(87)33527-4"},{"key":"e_1_2_1_18_1","volume-title":"Explainable artificial intelligence (xai)","author":"Gunning David","year":"2017"},{"key":"e_1_2_1_19_1","volume-title":"Gene selection for cancer classification using support vector machines. Machine learning","author":"Guyon Isabelle","year":"2002"},{"key":"e_1_2_1_20_1","volume-title":"Reviewing intuitive decision-making and uncertainty: the implications for medical education. Medical education","author":"Hall Katherine H","year":"2002"},{"key":"e_1_2_1_21_1","volume-title":"Advances in psychology.","author":"Hart Sandra G"},{"key":"e_1_2_1_22_1","volume-title":"Motor recovery after stroke: a systematic review of the literature. Archives of physical medicine and rehabilitation","author":"Hendricks Henk T","year":"2002"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/0018720814547570"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013959"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1136\/jnnp.2007.131045"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.38398.500764.8F"},{"key":"e_1_2_1_27_1","volume-title":"Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR medical informatics","author":"Khairat Saif","year":"2018"},{"key":"e_1_2_1_28_1","unstructured":"Been Kim Julie A Shah and Finale Doshi-Velez. 2015. Mind the gap: A generative approach to interpretable feature selection and extraction. In Advances in Neural Information Processing Systems. 2260--2268.  Been Kim Julie A Shah and Finale Doshi-Velez. 2015. Mind the gap: A generative approach to interpretable feature selection and extraction. In Advances in Neural Information Processing Systems. 2260--2268."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858402"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1161\/01.STR.30.5.934"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2018.01.034"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701399"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"M. H. Lee. 2019 a. Intelligent Agent for Assessing and Guiding Rehabilitation Exercises. In IJCAI. 6444--6445.  M. H. Lee. 2019 a. Intelligent Agent for Assessing and Guiding Rehabilitation Exercises. In IJCAI. 6444--6445.","DOI":"10.24963\/ijcai.2019\/911"},{"key":"e_1_2_1_34_1","volume-title":"An Intelligent Decision Support System for Stroke Rehabilitation Assessment. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. 694--696","author":"Lee Min Hun","year":"2019"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302273"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3340631.3394872"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368555.3384452"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN47096.2020.9223462"},{"key":"e_1_2_1_39_1","volume-title":"Exoskeleton robots for upper-limb rehabilitation: State of the art and future prospects. Medical engineering & physics","author":"Lo Ho Shing","year":"2012"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7489(03)00015-4"},{"key":"e_1_2_1_41_1","volume-title":"Advances in upper limb stroke rehabilitation: a technology push. Medical & biological engineering & computing","author":"Loureiro Rui CV","year":"2011"},{"key":"e_1_2_1_42_1","volume-title":"Interrater reliability: the kappa statistic. Biochemia medica: Biochemia medica","author":"McHugh Mary L","year":"2012"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1186\/1743-0003-11-156"},{"key":"e_1_2_1_44_1","volume-title":"Biomedical informatics","author":"Musen Mark A"},{"key":"e_1_2_1_45_1","unstructured":"Susan B O'Sullivan Thomas J Schmitz and George Fulk. 2019. Physical rehabilitation .FA Davis.  Susan B O'Sullivan Thomas J Schmitz and George Fulk. 2019. Physical rehabilitation .FA Davis."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2019.2899927"},{"key":"e_1_2_1_47_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).  Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3214277"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2648.2008.04925.x"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.22-18-08297.2002"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2007.04.021"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1080\/09638280701625237"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1176\/appi.ajp.2012.12010083"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2014.04.020"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1161\/STROKEAHA.110.592766"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0966-6362(02)00004-8"},{"key":"e_1_2_1_58_1","volume-title":"Feature selection for classification: A review. Data classification: Algorithms and applications","author":"Tang Jiliang","year":"2014"},{"key":"e_1_2_1_59_1","volume-title":"Mary Bowman, Camille Bryson, Staci Bishop, Sonya Pearson, and Sharon E Shaw.","author":"Taub Edward","year":"2011"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.5555\/3016100.3016191"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.clinph.2008.07.220"},{"key":"e_1_2_1_62_1","volume-title":"Iain B McInnes, Michael R Barnes, and Luciano Floridi.","author":"Watson David S","year":"2019"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1186\/1743-0003-11-108"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0003-9993(00)90228-4"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300468"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2007.09.001"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415227","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3415227","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:03:11Z","timestamp":1750197791000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415227"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,14]]},"references-count":66,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2020,10,14]]}},"alternative-id":["10.1145\/3415227"],"URL":"https:\/\/doi.org\/10.1145\/3415227","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,14]]},"assertion":[{"value":"2020-10-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}