{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T18:07:14Z","timestamp":1784138834604,"version":"3.55.0"},"publisher-location":"Cham","reference-count":24,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783032312846","type":"print"},{"value":"9783032312853","type":"electronic"}],"license":[{"start":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T00:00:00Z","timestamp":1784160000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T00:00:00Z","timestamp":1784160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2027]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Intelligent Personal Assistants (IPAs) are currently limited to mostly voice input by users, which often does not work for Deaf and Hard of Hearing (DHH) users\u2019 accessibility. Touch interfaces are an accessible alternative in principle, and these recently have been combined with large language models (LLMs) for usability enhancements. However, these are not hands-free, and it is not always feasible to walk up to a device and interact with its touchscreen, such as in the kitchen with dirty hands. This paper situates an LLM-powered touch interface against hands-free options. We present a study with 23 DHH participants who tested three potential input methods for interacting with IPAs: American Sign Language (ASL) in a Wizard-of-Oz setting, LLM-assisted touch through a touchscreen, and LLM-assisted touch through hands-free head pointing. ASL and LLM-assisted touch had comparable usability scores, while headpointing scored much worse. Despite comparable usability between ASL and touch, participants were much more enthusiastic about ASL input. This suggests ASL recognition should be the ultimate goal, but because such technology is not yet commercially viable, further research is needed for identifying practical hands-free alternatives to voice interaction with IPAs.<\/jats:p>","DOI":"10.1007\/978-3-032-31285-3_4","type":"book-chapter","created":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T17:23:38Z","timestamp":1784136218000},"page":"28-38","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Beyond the\u00a0Touchscreen: Hands-Free Sign Language and\u00a0Head-Pointing Interfaces for\u00a0Deaf Interaction with Intelligent Assistants"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5011-5188","authenticated-orcid":false,"given":"Nora","family":"Dunphy","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5566-7951","authenticated-orcid":false,"given":"Gidey","family":"Gezae","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9815-7121","authenticated-orcid":false,"given":"Paige","family":"DeVries","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0177-5178","authenticated-orcid":false,"given":"Ming","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3926-007X","authenticated-orcid":false,"given":"Pranav","family":"Pidathala","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1763-4352","authenticated-orcid":false,"given":"Abraham","family":"Glasser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0493-413X","authenticated-orcid":false,"given":"Raja","family":"Kushalnagar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2590-6880","authenticated-orcid":false,"given":"Christian","family":"Vogler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,7,16]]},"reference":[{"key":"4_CR1","unstructured":"Ballati, F., Corno, F., De\u00a0Russis, L.: Hey siri, do you understand me?\u201d: Virtual assistants and dysarthria. In: Intelligent Environments 2018, pp. 557\u2013566. IOS Press (2018)"},{"issue":"2","key":"4_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3314205","volume":"12","author":"L Berke","year":"2019","unstructured":"Berke, L., Huenerfauth, M., Patel, K.: Design and psychometric evaluation of American sign language translations of usability questionnaires. ACM Trans. Accessible Comput. (TACCESS) 12(2), 1\u201343 (2019)","journal-title":"ACM Trans. Accessible Comput. (TACCESS)"},{"key":"4_CR3","doi-asserted-by":"crossref","unstructured":"Bragg, D., et al.: Sign language recognition, generation, and translation: An interdisciplinary perspective. In: Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, pp. 16\u201331 (2019)","DOI":"10.1145\/3308561.3353774"},{"key":"4_CR4","doi-asserted-by":"crossref","unstructured":"Desai, A., De Meulder, M., Hochgesang, J.A., Kocab, A., Lu, A.X.: Systemic biases in sign language ai research: a deaf-led call to reevaluate research agendas. In: Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages: Evaluation of Sign Language Resources, pp. 54\u201365 (2024)","DOI":"10.63317\/4qt6t64rxwd2"},{"key":"4_CR5","doi-asserted-by":"publisher","unstructured":"DeVries, P.S., et al.: Deaf and hard of hearing access to intelligent personal assistants: comparison of voice-based options with an llm-powered touch interface. In: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. CHI 2026. Association for Computing Machinery, New York (2026). https:\/\/doi.org\/10.1145\/3772318.3791869","DOI":"10.1145\/3772318.3791869"},{"key":"4_CR6","doi-asserted-by":"crossref","unstructured":"Devries, P.S., et al.: Sign language-based versus touch-based input for deaf users with interactive personal assistants in simulated kitchen environments. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pp. 1\u20139 (2024)","DOI":"10.1145\/3613905.3651075"},{"key":"4_CR7","unstructured":"Doc\u00edo-Fern\u00e1ndez, L., et al.: Lse_uvigo: A multi-source database for spanish sign language recognition. In: Proceedings of the LREC2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives, pp. 45\u201352 (2020)"},{"key":"4_CR8","doi-asserted-by":"crossref","unstructured":"Glasser, A.: Automatic speech recognition services: Deaf and hard-of-hearing usability. In: Extended abstracts of the 2019 CHI conference on human factors in computing systems, pp. 1\u20136 (2019)","DOI":"10.1145\/3290607.3308461"},{"key":"4_CR9","doi-asserted-by":"crossref","unstructured":"Glasser, A., Kushalnagar, K., Kushalnagar, R.: Deaf, hard of hearing, and hearing perspectives on using automatic speech recognition in conversation. In: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 427\u2013432 (2017)","DOI":"10.1145\/3132525.3134781"},{"key":"4_CR10","doi-asserted-by":"crossref","unstructured":"Glasser, A., Watkins, M., Hart, K., Lee, S., Huenerfauth, M.: Analyzing deaf and hard-of-hearing users\u2019 behavior, usage, and interaction with a personal assistant device that understands sign-language input. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1\u201312 (2022)","DOI":"10.1145\/3491102.3501987"},{"key":"4_CR11","doi-asserted-by":"crossref","unstructured":"Hansen, J.P., T\u00f8rning, K., Johansen, A.S., Itoh, K., Aoki, H.: Gaze typing compared with input by head and hand. In: Proceedings of the 2004 symposium on Eye tracking research & applications, pp. 131\u2013138 (2004)","DOI":"10.1145\/968363.968389"},{"key":"4_CR12","unstructured":"Mahmood, A., Wang, J., Yao, B., Wang, D., Huang, C.: Llm-powered conversational voice assistants: Interaction patterns, opportunities, challenges, and design guidelines (arxiv preprint arxiv: 2309.13879). arXiv preprint arXiv:2309.13879 (2023)"},{"key":"4_CR13","doi-asserted-by":"crossref","unstructured":"Mande, V., Glasser, A., Dingman, B., Huenerfauth, M.: Deaf users\u2019 preferences among wake-up approaches during sign-language interaction with personal assistant devices. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1\u20136 (2021)","DOI":"10.1145\/3411763.3451592"},{"key":"4_CR14","doi-asserted-by":"crossref","unstructured":"Pradhan, A., Mehta, K., Findlater, L.: \u201cAccessibility came by accident\u201d use of voice-controlled intelligent personal assistants by people with disabilities. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1\u201313 (2018)","DOI":"10.1145\/3173574.3174033"},{"key":"4_CR15","doi-asserted-by":"crossref","unstructured":"Punde, P.A., Jadhav, M.E., Manza, R.R.: A study of eye tracking technology and its applications. In: 2017 1st International Conference on Intelligent Systems and Information Management (ICISIM), pp. 86\u201390. IEEE (2017)","DOI":"10.1109\/ICISIM.2017.8122153"},{"key":"4_CR16","unstructured":"Sauro, J.: Predicting net promoter scores from system usability scale scores. MeasuringU. Retrieved January 25, 2024 16, \u00a02022 (2012)"},{"key":"4_CR17","doi-asserted-by":"crossref","unstructured":"Schwarz, J., Marais, C.C., Leyvand, T., Hudson, S.E., Mankoff, J.: Combining body pose, gaze, and gesture to determine intention to interact in vision-based interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp. 3443\u20133452 (2014)","DOI":"10.1145\/2556288.2556989"},{"key":"4_CR18","doi-asserted-by":"crossref","unstructured":"Tran, N., DeVries, P.S., Seita, M., Kushalnagar, R., Glasser, A., Vogler, C.: Assessment of sign language-based versus touch-based input for deaf users interacting with intelligent personal assistants. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1\u201315 (2024)","DOI":"10.1145\/3613904.3642094"},{"key":"4_CR19","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.patrec.2013.07.003","volume":"36","author":"M Turk","year":"2014","unstructured":"Turk, M.: Multimodal interaction: a review. Pattern Recogn. Lett. 36, 189\u2013195 (2014)","journal-title":"Pattern Recogn. Lett."},{"key":"4_CR20","unstructured":"Vu, M.D., Wang, H., Li, Z., Chen, J., Zhao, S., Xing, Z., Chen, C.: Gptvoicetasker: Llm-powered virtual assistant for smartphone. arXiv preprint arXiv:2401.14268 (2024)"},{"key":"4_CR21","doi-asserted-by":"crossref","unstructured":"Wasti, S.M., Pu, K.Q., Neshati, A.: Large language user interfaces: Voice interactive user interfaces powered by llms. In: Intelligent Systems Conference, pp. 639\u2013655. Springer (2024)","DOI":"10.1007\/978-3-031-66329-1_41"},{"key":"4_CR22","unstructured":"Wojtanowski, G., Gilmore, C., Seravalli, B., Fargas, K., Vogler, C., Kushalnagar, R.: \u201calexa, can you see me?\u201d making individual personal assistants for the home accessible to deaf consumers. The Journal on Technology and Persons with Disabilities, p.\u00a0130 (2020)"},{"key":"4_CR23","unstructured":"Yoshitake, M.: Amazon announces llm-powered tools for developers at the devices and services fall launch event: New developer tools to build llm-powered experiences with alexa. https:\/\/developer.amazon.com\/en-US\/blogs\/alexa\/alexa-skills-kit\/2023\/09\/alexa-llm-fall-devices-services-sep-2023 (2023), retrieved May 27, 2024"},{"key":"4_CR24","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Wu, J., Wang, X., Tang, W., Wang, D., de\u00a0Rijke, M.: Let me do it for you: Towards llm empowered recommendation via tool learning. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1796\u20131806 (2024)","DOI":"10.1145\/3626772.3657828"}],"container-title":["Lecture Notes in Computer Science","Computers Helping People with Special Needs"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-032-31285-3_4","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T17:23:42Z","timestamp":1784136222000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-032-31285-3_4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,16]]},"ISBN":["9783032312846","9783032312853"],"references-count":24,"URL":"https:\/\/doi.org\/10.1007\/978-3-032-31285-3_4","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,16]]},"assertion":[{"value":"16 July 2026","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":1,"name":"Ethics","label":"Disclosure of Interests","group":{"name":"EthicsHeading","label":"Ethics"}},{"value":"ICCHP","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Computers Helping People with Special Needs","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Brno","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Czech Republic","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2026","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"15 July 2026","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"17 July 2026","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"20","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"icchp2026","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/www.icchp.org","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}