{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T03:09:07Z","timestamp":1774667347213,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":61,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,3,22]],"date-time":"2022-03-22T00:00:00Z","timestamp":1647907200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"name":"LISBOA 2020 and the FCT","award":["LISBOA-01-0247-FEDER-045948"],"award-info":[{"award-number":["LISBOA-01-0247-FEDER-045948"]}]},{"name":"Singapore Ministry of Education (MOE)","award":["21-SIS-SMU-034"],"award-info":[{"award-number":["21-SIS-SMU-034"]}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-1518865"],"award-info":[{"award-number":["CNS-1518865"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"FCT","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,3,22]]},"DOI":"10.1145\/3490099.3511112","type":"proceedings-article","created":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T23:24:13Z","timestamp":1647905053000},"page":"4-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Towards Efficient Annotations for a Human-AI Collaborative, Clinical Decision Support System: A Case Study on Physical Stroke Rehabilitation Assessment"],"prefix":"10.1145","author":[{"given":"Min Hun","family":"Lee","sequence":"first","affiliation":[{"name":"Singapore Management University, Singapore"}]},{"given":"Daniel P.","family":"Siewiorek","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, United States"}]},{"given":"Asim","family":"Smailagic","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, United States"}]},{"given":"Alexandre","family":"Bernardino","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Instituto Superior Tecnico, University of Lisbon, Portugal"}]},{"given":"Sergi","family":"Berm\u00fadez i Badia","sequence":"additional","affiliation":[{"name":"Faculdade de Ci\u00eancias Exatas e da Engenharia, Universidade da Madeira, Portugal"}]}],"member":"320","published-online":{"date-parts":[[2022,3,22]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300233"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2011.09.002"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2798607"},{"key":"e_1_3_2_1_5_1","volume-title":"Marilyn McKay-Lyons, Edward Harrison","author":"Bayley T","year":"2012","unstructured":"Mark\u00a0 T Bayley , Amanda Hurdowar , Carol\u00a0 L Richards , Nicol Korner-Bitensky , Sharon Wood-Dauphinee , Janice\u00a0J Eng , Marilyn McKay-Lyons, Edward Harrison , Robert Teasell, Margaret Harrison , 2012 . Barriers to implementation of stroke rehabilitation evidence: findings from a multi-site pilot project. Disability and rehabilitation 34, 19 (2012), 1633\u20131638. Mark\u00a0T Bayley, Amanda Hurdowar, Carol\u00a0L Richards, Nicol Korner-Bitensky, Sharon Wood-Dauphinee, Janice\u00a0J Eng, Marilyn McKay-Lyons, Edward Harrison, Robert Teasell, Margaret Harrison, 2012. Barriers to implementation of stroke rehabilitation evidence: findings from a multi-site pilot project. Disability and rehabilitation 34, 19 (2012), 1633\u20131638."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376718"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-018-0004-1"},{"key":"e_1_3_2_1_8_1","volume-title":"IJCAI-17 workshop on explainable AI (XAI), Vol.\u00a08. 1.","author":"Biran Or","year":"2017","unstructured":"Or Biran and Courtenay Cotton . 2017 . Explanation and justification in machine learning: A survey . In IJCAI-17 workshop on explainable AI (XAI), Vol.\u00a08. 1. Or Biran and Courtenay Cotton. 2017. Explanation and justification in machine learning: A survey. In IJCAI-17 workshop on explainable AI (XAI), Vol.\u00a08. 1."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2011.2119372"},{"key":"e_1_3_2_1_10_1","volume-title":"Advances in computers. Vol.\u00a022","author":"Buchanan G","unstructured":"Bruce\u00a0 G Buchanan and Richard\u00a0 O Duda . 1983. Principles of rule-based expert systems . In Advances in computers. Vol.\u00a022 . Elsevier , 163\u2013216. Bruce\u00a0G Buchanan and Richard\u00a0O Duda. 1983. Principles of rule-based expert systems. In Advances in computers. Vol.\u00a022. Elsevier, 163\u2013216."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.7797"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300234"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2788613"},{"key":"e_1_3_2_1_15_1","volume-title":"How to develop machine learning models for healthcare. Nature materials 18, 5","author":"Chen Hsuan\u00a0Cameron","year":"2019","unstructured":"Po- Hsuan\u00a0Cameron Chen , Yun Liu , and Lily Peng . 2019. How to develop machine learning models for healthcare. Nature materials 18, 5 ( 2019 ), 410. Po-Hsuan\u00a0Cameron Chen, Yun Liu, and Lily Peng. 2019. How to develop machine learning models for healthcare. Nature materials 18, 5 (2019), 410."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/IEMBS.2011.6091674"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376638"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287572"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSCI.2016.7850050"},{"key":"e_1_3_2_1_20_1","volume-title":"Dermatologist-level classification of skin cancer with deep neural networks. nature 542, 7639","author":"Esteva Andre","year":"2017","unstructured":"Andre Esteva , Brett Kuprel , Roberto\u00a0 A Novoa , Justin Ko , Susan\u00a0 M Swetter , Helen\u00a0 M Blau , and Sebastian Thrun . 2017. Dermatologist-level classification of skin cancer with deep neural networks. nature 542, 7639 ( 2017 ), 115\u2013118. Andre Esteva, Brett Kuprel, Roberto\u00a0A Novoa, Justin Ko, Susan\u00a0M Swetter, Helen\u00a0M Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks. nature 542, 7639 (2017), 115\u2013118."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359152"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2013-001815"},{"key":"e_1_3_2_1_23_1","volume-title":"International Conference on Machine Learning. PMLR, 1321\u20131330","author":"Guo Chuan","year":"2017","unstructured":"Chuan Guo , Geoff Pleiss , Yu Sun , and Kilian\u00a0 Q Weinberger . 2017 . On calibration of modern neural networks . In International Conference on Machine Learning. PMLR, 1321\u20131330 . Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian\u00a0Q Weinberger. 2017. On calibration of modern neural networks. In International Conference on Machine Learning. PMLR, 1321\u20131330."},{"key":"e_1_3_2_1_24_1","volume-title":"Intelligent decision-making support systems: foundations, applications and challenges","author":"Gupta ND","unstructured":"Jatinder\u00a0 ND Gupta , Guisseppi\u00a0 A Forgionne , and Manuel Mora . 2007. Intelligent decision-making support systems: foundations, applications and challenges . Springer Science & Business Media . Jatinder\u00a0ND Gupta, Guisseppi\u00a0A Forgionne, and Manuel Mora. 2007. Intelligent decision-making support systems: foundations, applications and challenges. Springer Science & Business Media."},{"key":"e_1_3_2_1_25_1","volume-title":"Visual analytics in deep learning: An interrogative survey for the next frontiers","author":"Hohman Fred","year":"2018","unstructured":"Fred Hohman , Minsuk Kahng , Robert Pienta , and Duen\u00a0Horng Chau . 2018. Visual analytics in deep learning: An interrogative survey for the next frontiers . IEEE transactions on visualization and computer graphics 25, 8( 2018 ), 2674\u20132693. Fred Hohman, Minsuk Kahng, Robert Pienta, and Duen\u00a0Horng Chau. 2018. Visual analytics in deep learning: An interrogative survey for the next frontiers. IEEE transactions on visualization and computer graphics 25, 8(2018), 2674\u20132693."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376177"},{"key":"e_1_3_2_1_27_1","first-page":"11","article-title":"Challenges in applying best evidence to physiotherapy","volume":"4","author":"Jones Mark","year":"2006","unstructured":"Mark Jones , Karen Grimmer , Ian Edwards , Joy Higgs , and Franziska Trede . 2006 . Challenges in applying best evidence to physiotherapy . Internet Journal of Allied Health Sciences and Practice 4 , 3(2006), 11 . Mark Jones, Karen Grimmer, Ian Edwards, Joy Higgs, and Franziska Trede. 2006. Challenges in applying best evidence to physiotherapy. Internet Journal of Allied Health Sciences and Practice 4, 3(2006), 11.","journal-title":"Internet Journal of Allied Health Sciences and Practice"},{"key":"e_1_3_2_1_28_1","volume-title":"JAABA: interactive machine learning for automatic annotation of animal behavior. Nature methods 10, 1","author":"Kabra Mayank","year":"2013","unstructured":"Mayank Kabra , Alice\u00a0 A Robie , Marta Rivera-Alba , Steven Branson , and Kristin Branson . 2013. JAABA: interactive machine learning for automatic annotation of animal behavior. Nature methods 10, 1 ( 2013 ), 64\u201367. Mayank Kabra, Alice\u00a0A Robie, Marta Rivera-Alba, Steven Branson, and Kristin Branson. 2013. JAABA: interactive machine learning for automatic annotation of animal behavior. Nature methods 10, 1 (2013), 64\u201367."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753529"},{"key":"e_1_3_2_1_30_1","unstructured":"Danielle\u00a0Leah Kehl and Samuel\u00a0Ari Kessler. 2017. Algorithms in the criminal justice system: Assessing the use of risk assessments in sentencing. (2017).  Danielle\u00a0Leah Kehl and Samuel\u00a0Ari Kessler. 2017. Algorithms in the criminal justice system: Assessing the use of risk assessments in sentencing. (2017)."},{"key":"e_1_3_2_1_31_1","volume-title":"Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR medical informatics 6, 2","author":"Khairat Saif","year":"2018","unstructured":"Saif Khairat , David Marc , William Crosby , and Ali Al\u00a0Sanousi . 2018. Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR medical informatics 6, 2 ( 2018 ), e24. Saif Khairat, David Marc, William Crosby, and Ali Al\u00a0Sanousi. 2018. Reasons for physicians not adopting clinical decision support systems: critical analysis. JMIR medical informatics 6, 2 (2018), e24."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3214366"},{"key":"e_1_3_2_1_33_1","unstructured":"Been Kim Julie\u00a0A Shah and Finale Doshi-Velez. 2015. Mind the gap: A generative approach to interpretable feature selection and extraction. In Advances in Neural Information Processing Systems. 2260\u20132268.  Been Kim Julie\u00a0A Shah and Finale Doshi-Velez. 2015. Mind the gap: A generative approach to interpretable feature selection and extraction. In Advances in Neural Information Processing Systems. 2260\u20132268."},{"key":"e_1_3_2_1_34_1","volume-title":"Human decisions and machine predictions. The quarterly journal of economics 133, 1","author":"Kleinberg Jon","year":"2018","unstructured":"Jon Kleinberg , Himabindu Lakkaraju , Jure Leskovec , Jens Ludwig , and Sendhil Mullainathan . 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 ( 2018 ), 237\u2013293. Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 (2018), 237\u2013293."},{"key":"e_1_3_2_1_35_1","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations. 5\u20139.","author":"Klie Jan-Christoph","year":"2018","unstructured":"Jan-Christoph Klie , Michael Bugert , Beto Boullosa , Richard\u00a0Eckart de Castilho , and Iryna Gurevych . 2018 . The inception platform: Machine-assisted and knowledge-oriented interactive annotation . In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations. 5\u20139. Jan-Christoph Klie, Michael Bugert, Beto Boullosa, Richard\u00a0Eckart de Castilho, and Iryna Gurevych. 2018. The inception platform: Machine-assisted and knowledge-oriented interactive annotation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations. 5\u20139."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.3301622"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701399"},{"key":"e_1_3_2_1_38_1","volume-title":"Stroke rehabilitation. The Lancet 377, 9778","author":"Langhorne Peter","year":"2011","unstructured":"Peter Langhorne , Julie Bernhardt , and Gert Kwakkel . 2011. Stroke rehabilitation. The Lancet 377, 9778 ( 2011 ), 1693\u20131702. Peter Langhorne, Julie Bernhardt, and Gert Kwakkel. 2011. Stroke rehabilitation. The Lancet 377, 9778 (2011), 1693\u20131702."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302273"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415227"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3340631.3394872"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445472"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1214\/15-AOAS848"},{"key":"e_1_3_2_1_44_1","volume-title":"Confidence-based active learning","author":"Li Mingkun","year":"2006","unstructured":"Mingkun Li and Ishwar\u00a0 K Sethi . 2006. Confidence-based active learning . IEEE transactions on pattern analysis and machine intelligence 28, 8( 2006 ), 1251\u20131261. Mingkun Li and Ishwar\u00a0K Sethi. 2006. Confidence-based active learning. IEEE transactions on pattern analysis and machine intelligence 28, 8(2006), 1251\u20131261."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7489(03)00015-4"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102430"},{"key":"e_1_3_2_1_47_1","unstructured":"Susan\u00a0B O\u2019Sullivan Thomas\u00a0J Schmitz and George Fulk. 2019. Physical rehabilitation. FA Davis.  Susan\u00a0B O\u2019Sullivan Thomas\u00a0J Schmitz and George Fulk. 2019. Physical rehabilitation. FA Davis."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2019.2899927"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmi.2019.09.009"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.22-18-08297.2002"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0048-x"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-017-9552-8"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372827"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"crossref","unstructured":"Emily Seto Kevin\u00a0J Leonard Joseph\u00a0A Cafazzo Jan Barnsley Caterina Masino and Heather\u00a0J Ross. 2012. Developing healthcare rule-based expert systems: case study of a heart failure telemonitoring system. International journal of medical informatics 81 8(2012) 556\u2013565.  Emily Seto Kevin\u00a0J Leonard Joseph\u00a0A Cafazzo Jan Barnsley Caterina Masino and Heather\u00a0J Ross. 2012. Developing healthcare rule-based expert systems: case study of a heart failure telemonitoring system. International journal of medical informatics 81 8(2012) 556\u2013565.","DOI":"10.1016\/j.ijmedinf.2012.03.001"},{"key":"e_1_3_2_1_56_1","volume-title":"Human-Computer Interaction","author":"Smith J","unstructured":"Philip\u00a0 J Smith , Norman\u00a0 D Geddes , and Roger Beatty . 2009. Human-centered design of decision-support systems . In Human-Computer Interaction . CRC Press , 263\u2013292. Philip\u00a0J Smith, Norman\u00a0D Geddes, and Roger Beatty. 2009. Human-centered design of decision-support systems. In Human-Computer Interaction. CRC Press, 263\u2013292."},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1161\/STROKEAHA.110.592766"},{"key":"e_1_3_2_1_58_1","volume-title":"Wolf motor function test (WMFT) manual. Birmingham: University of Alabama","author":"Taub Edward","year":"2011","unstructured":"Edward Taub , David\u00a0 M Morris , Jean Crago , Danna\u00a0Kay King , Mary Bowman , Camille Bryson , Staci Bishop , Sonya Pearson , and Sharon\u00a0 E Shaw . 2011. Wolf motor function test (WMFT) manual. Birmingham: University of Alabama , CI Therapy Research Group ( 2011 ). Edward Taub, David\u00a0M Morris, Jean Crago, Danna\u00a0Kay King, Mary Bowman, Camille Bryson, Staci Bishop, Sonya Pearson, and Sharon\u00a0E Shaw. 2011. Wolf motor function test (WMFT) manual. Birmingham: University of Alabama, CI Therapy Research Group (2011)."},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1186\/1743-0003-11-108"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300468"},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775151"}],"event":{"name":"IUI '22: 27th International Conference on Intelligent User Interfaces","location":"Helsinki Finland","acronym":"IUI '22","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["27th International Conference on Intelligent User Interfaces"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3490099.3511112","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3490099.3511112","content-type":"text\/html","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3490099.3511112","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3490099.3511112","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:23Z","timestamp":1750188683000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3490099.3511112"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,22]]},"references-count":61,"alternative-id":["10.1145\/3490099.3511112","10.1145\/3490099"],"URL":"https:\/\/doi.org\/10.1145\/3490099.3511112","relation":{},"subject":[],"published":{"date-parts":[[2022,3,22]]},"assertion":[{"value":"2022-03-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}