{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T11:34:43Z","timestamp":1781091283812,"version":"3.54.1"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006374","name":"NSF","doi-asserted-by":"publisher","award":["2150429"],"award-info":[{"award-number":["2150429"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,5,2]]},"abstract":"<jats:p>This study investigates innovative interaction designs for communication and collaborative learning between learners of mixed hearing and signing abilities, leveraging advancements in mixed reality technologies like Apple Vision Pro and generative AI for animated avatars. Adopting a participatory design approach, we engaged 15 d\/Deaf and hard of hearing (DHH) students to brainstorm ideas for an AI avatar with interpreting ability (sign language to English and English to sign language) that would facilitate their face-to-face communication with hearing peers. Participants envisioned the AI avatars to address some issues with human interpreters, such as lack of availability, and provide affordable options to expensive personalized interpreting services. Our findings indicate a range of preferences for integrating the AI avatars with actual human figures of both DHH and hearing communication partners. The participants highlighted the importance of having control over customizing the AI avatar, such as AI-generated signs, voices, facial expressions, and their synchronization for enhanced emotional display in communication. Based on our findings, we propose a suite of design recommendations that balance respecting sign language norms with adherence to hearing social norms. Our study offers insights into improving the authenticity of generative AI in scenarios involving specific and sometimes unfamiliar social norms.<\/jats:p>","DOI":"10.1145\/3710953","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T11:36:19Z","timestamp":1747740979000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Customizing Generated Signs and Voices of AI Avatars: Deaf-Centric Mixed-Reality Design for Deaf-Hearing Communication"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0640-6883","authenticated-orcid":false,"given":"Si","family":"Chen","sequence":"first","affiliation":[{"name":"Information Sciences, University of Illinois Urbana Champaign, Champaign, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1535-1714","authenticated-orcid":false,"given":"Haocong","family":"Cheng","sequence":"additional","affiliation":[{"name":"Information Sciences, University of Illinois Urbana-Champaign, Champaign, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1344-8095","authenticated-orcid":false,"given":"Suzy","family":"Su","sequence":"additional","affiliation":[{"name":"Information Sciences, University of Illinois Urbana-Champaign, Urbana-Champaign, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3387-2454","authenticated-orcid":false,"given":"Stephanie","family":"Patterson","sequence":"additional","affiliation":[{"name":"Computer Science, University of Illinois Urbana-Champaign, Champaign, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0493-413X","authenticated-orcid":false,"given":"Raja","family":"Kushalnagar","sequence":"additional","affiliation":[{"name":"Gallaudet University, Washington, DC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0399-8032","authenticated-orcid":false,"given":"Yun","family":"Huang","sequence":"additional","affiliation":[{"name":"Information Sciences, University of Illinois Urbana-Champaign, Champaign, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5294-2352","authenticated-orcid":false,"given":"Qi","family":"Wang","sequence":"additional","affiliation":[{"name":"Gallaudet University, Washington, DC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1--12","author":"Sedeeq","year":"2021","unstructured":"Sedeeq Al-khazraji, Becca Dingman, Sooyeon Lee, and Matt Huenerfauth. 2021. At a Different Pace: Evaluating Whether Users Prefer Timing Parameters in American Sign Language Animations to Differ from Human Signers' Timing. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1--12."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376563"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517428.3563373"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581274"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3312921"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3373625.3417026"},{"key":"e_1_2_1_7_1","volume-title":"Design fiction: A short essay on design, science, fact, and fiction. Machine Learning and the City: Applications in Architecture and Urban Design","author":"Bleecker Julian","year":"2022","unstructured":"Julian Bleecker. 2022. Design fiction: A short essay on design, science, fact, and fiction. Machine Learning and the City: Applications in Architecture and Urban Design (2022), 561--578."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557098"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308561.3353774"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581336"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579509"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3563657.3595980"},{"key":"e_1_2_1_13_1","volume-title":"Thematic analysis: a practical guide. Thematic Analysis","author":"Clarke Victoria","year":"2021","unstructured":"Victoria Clarke and Virginia Braun. 2021. Thematic analysis: a practical guide. Thematic Analysis (2021), 1--100."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10209-023-00992--1nolinkurl10.1007\/s10209-023-00992--1"},{"key":"e_1_2_1_15_1","volume-title":"accessibility barriers for immersive technologies. Universal Access in the Information Society","author":"Creed Chris","year":"2023","unstructured":"Chris Creed, Maadh Al-Kalbani, Arthur Theil, Sayan Sarcar, and Ian Williams. 2023. Inclusive AR\/VR: accessibility barriers for immersive technologies. Universal Access in the Information Society (2023), 1--15."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359626"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1075\/tis.18008.demnolinkurl10.1075\/tis.18008.dem"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICACCP.2019.8883009"},{"key":"e_1_2_1_19_1","volume-title":"Content strategy in technical communication","author":"Duin Ann Hill","unstructured":"Ann Hill Duin, Dawn M Armfield, and Isabel Pedersen. 2019. Human-centered content design in augmented reality. In Content strategy in technical communication. Routledge, 89--116."},{"key":"e_1_2_1_20_1","volume-title":"2015 ASEE Annual Conference & Exposition. ASEE Conferences","author":"Stanislow Joseph S.","year":"2015","unstructured":"Joseph S. Stanislow Gary W Behm, Raja S Kushalnagar and Aaron Weir Kelstone. 2015. Enhancing Accessibility of Engineering Lectures for Deaf & Hard of Hearing (DHH): Real-time Tracking Text Displays (RTTD) in Classrooms. In 2015 ASEE Annual Conference & Exposition. ASEE Conferences, Seattle, Washington. https:\/\/peer.asee.org\/23995."},{"key":"e_1_2_1_21_1","volume-title":"The presentation of self in everyday life. Double Day","author":"Goffman Erving","year":"1959","unstructured":"Erving Goffman. 1959. The presentation of self in everyday life. Double Day. Garden City, NY, Vol. 259 (1959)."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.3389\/frvir.2021.619811"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3373625.3418031"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517593"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702393"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3461778.3462106"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.06.063"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICE.2012.6297673nolinkurl10.1109\/ICE.2012.6297673"},{"key":"e_1_2_1_29_1","unstructured":"Ben S Kosa Ai Minakawa Patrick Boudreault Christian Vogler Poorna Kushalnagar and Raja Kushalnagar. 2023. ASL Consent in the Digital Informed Consent Process. (2023)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR-Adjunct57072.2022.00189nolinkurl10.1109\/ISMAR-Adjunct57072.2022.00189"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3441852.3471200"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512922nolinkurl10.1145\/3512922"},{"key":"e_1_2_1_33_1","volume-title":"CVPR Workshop on Computer Vision for Augmented and Virtual Reality","author":"Lu Duo","year":"2020","unstructured":"Duo Lu and Linzhen Luo. 2020. Fmkit: An in-air-handwriting analysis library and data repository. In CVPR Workshop on Computer Vision for Augmented and Virtual Reality, 2020."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0305000915000835"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519799"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","unstructured":"Le Luo Dongdong Weng Guo Songrui Jie Hao and Ziqi Tu. 2022b. Avatar Interpreter: Improving Classroom Experiences for Deaf and Hard-of-Hearing People Based on Augmented Reality. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans LA USA) (CHI EA '22). Association for Computing Machinery New York NY USA Article 318 5 pages.https:\/\/doi.org\/10.1145\/3491101.3519799doi:nolinkurl10.1145\/3491101.3519799","DOI":"10.1145\/3491101.3519799nolinkurl10.1145\/3491101.3519799"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415196"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3441852.3471199"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581481"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501882"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517428.3551352"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517428.3550417"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580809"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1002\/aqc.2820"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1353\/sls.2006.0019nolinkurl10.1353\/sls.2006.0019"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1093\/deafed\/enh017"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1353\/sls.2005.0004"},{"key":"e_1_2_1_48_1","volume-title":"Interpreting","volume":"17","author":"Nicodemus Brenda","year":"2015","unstructured":"Brenda Nicodemus and Karen Emmorey. 2015. Directionality in ASL-English interpreting: Accuracy and articulation quality in L1 and L2. Interpreting, Volume 17, Issue 2, Jan 2015 (2015), 145--166."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.12840\/issn.2255-4165.2018.06.01.015"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR-AMH.2009.5336735"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300617"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.4236\/psych.2019.1015135nolinkurl10.4236\/psych.2019.1015135"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3374920.3374956"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.3390\/app10020636"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1--16","author":"Xia Ang Jazz Rui","year":"2022","unstructured":"Jazz Rui Xia Ang, Ping Liu, Emma McDonnell, and Sarah Coppola. 2022. \u201dIn this online environment, we're limited\u201d: Exploring Inclusive Video Conferencing Design for Signers. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1--16."},{"key":"e_1_2_1_56_1","volume-title":"Necati Cihan Camgoz, and Richard Bowden","author":"Saunders Ben","year":"2022","unstructured":"Ben Saunders, Necati Cihan Camgoz, and Richard Bowden. 2022. Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Production. (2022). arxiv: 2203.15354 [cs.CV]"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dhjo.2020.100932nolinkurl10.1016\/j.dhjo.2020.100932"},{"key":"e_1_2_1_58_1","volume-title":"Participatory design: Principles and practices","author":"Schuler Douglas","unstructured":"Douglas Schuler and Aki Namioka. 1993. Participatory design: Principles and practices. CRC Press."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3375039"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196766"},{"key":"e_1_2_1_61_1","volume-title":"The methodology of participatory design. Technical communication","author":"Spinuzzi Clay","year":"2005","unstructured":"Clay Spinuzzi. 2005. The methodology of participatory design. Technical communication, Vol. 52, 2 (2005), 163--174."},{"key":"e_1_2_1_62_1","unstructured":"Faryal Fatima Syeda. 2023. Development of a VR\/3D Hand Prosthesis Software Application: Avatar Customization and Hand Motion Control. (2023)."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449104"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","unstructured":"Tsubasa Uchida Naoki Nakatani Taro Miyazaki Hiroyuki Kaneko and Masanori Sano. 2023. Motion Editing Tool for Reproducing Grammatical Elements of Japanese Sign Language Avatar Animation. In 2023 IEEE International Conference on Acoustics Speech and Signal Processing Workshops (ICASSPW). 1--5. https:\/\/doi.org\/10.1109\/ICASSPW59220.2023.10193198doi:nolinkurl10.1109\/ICASSPW59220.2023.10193198","DOI":"10.1109\/ICASSPW59220.2023.10193198nolinkurl10.1109\/ICASSPW59220.2023.10193198"},{"key":"e_1_2_1_65_1","volume-title":"Linguistics of American sign language: An introduction","author":"Valli Clayton","unstructured":"Clayton Valli and Ceil Lucas. 2000. Linguistics of American sign language: An introduction. Gallaudet University Press."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/VRW50115.2020.00148"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2023.3247072nolinkurl10.1109\/TVCG.2023.3247072"},{"key":"e_1_2_1_68_1","volume-title":"Healthcare providers' and deaf patients' interpreting preferences for critical care and non-critical care: Video remote interpreting. Disability and health journal","author":"Yabe Manako","year":"2020","unstructured":"Manako Yabe. 2020. Healthcare providers' and deaf patients' interpreting preferences for critical care and non-critical care: Video remote interpreting. Disability and health journal, Vol. 13, 2 (2020), 100870."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1002\/cav.2082"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517428.3544829"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449235"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501855"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710953","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3710953","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T09:31:40Z","timestamp":1755768700000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710953"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":72,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,5,2]]}},"alternative-id":["10.1145\/3710953"],"URL":"https:\/\/doi.org\/10.1145\/3710953","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"assertion":[{"value":"2025-05-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}