{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T03:35:59Z","timestamp":1768016159903,"version":"3.49.0"},"reference-count":80,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T00:00:00Z","timestamp":1668124800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Access. Comput."],"published-print":{"date-parts":[[2022,12,31]]},"abstract":"<jats:p>Deaf persons, whether or not they are sign language users, make up one of various existing marginalized populations that historically have been socially and politically underrepresented. Unfortunately, this also happens in technology design. Conducting user studies in which marginalized populations are represented is a step towards guaranteeing their right to participate in choices and decisions that are made for, with, and by them. This article presents and discusses results from a Systematic Literature Review (SLR) of user studies in the design of systems for Automatic Sign Language Processing (ASLP). Following our SLR protocol, from 2,486 papers initially found, we applied inclusion and exclusion criteria to finally select 37 papers in our review. We excluded publications that were not full papers, were not related to our main topic of interest, or that reported results that had been updated by more recent papers. All the selected papers focus on user studies as a basis for the design of three major aspects of ASLP: generation (ASLG), recognition (ASLR), and translation (ASLT). With regard to our specific area of interest, we analyzed four areas related to our research questions: goals and research methods, types of user involvement in the interaction design life cycle, cultural and collaborative aspects, and other lessons learned from the primary studies under review. Salient findings from our analysis show that numerical scale questionnaires are the most frequently used research instruments, co-designing ASLP systems with sign language users is not a common practice (as potential users are included mostly in the evaluation phase), and only seldom are Deaf persons who are sign language users included as members of research teams. These findings point to the need of conducting more inclusive and qualitative research for, with and by Deaf persons who are sign language users.<\/jats:p>","DOI":"10.1145\/3563395","type":"journal-article","created":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T13:16:15Z","timestamp":1665148575000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["A Systematic Review of User Studies as a Basis for the Design of Systems for Automatic Sign Language Processing"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4552-3142","authenticated-orcid":false,"given":"Soraia","family":"Prietch","sequence":"first","affiliation":[{"name":"Universidade Federal de Rondon\u00f3polis, Mato Grosso, Brazil and Postdoctoral scholar at the Doctoral Program in Educational Systems and Environments (DSAE), Facultad de Ciencias de la Electr\u00f3nica, Benem\u00e9rita Universidad Aut\u00f3noma de Puebla (BUAP), Puebla, Mexico"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7728-7087","authenticated-orcid":false,"given":"J. Alfredo","family":"S\u00e1nchez","sequence":"additional","affiliation":[{"name":"Laboratorio Nacional de Inform\u00e1tica Avanzada (LANIA), Xalapa, Veracruz, Mexico"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3393-610X","authenticated-orcid":false,"given":"Josefina","family":"Guerrero","sequence":"additional","affiliation":[{"name":"Doctoral Program in Educational Systems and Environments (DSAE), Facultad de Ciencias de la Electr\u00f3nica, Benem\u00e9rita Universidad Aut\u00f3noma de Puebla (BUAP), Puebla, M\u00e9xico and Facultad de Ciencias de la Computaci\u00f3n, Benem\u00e9rita Universidad Aut\u00f3noma de Puebla (BUAP), Puebla, Mexico"}]}],"member":"320","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"e_1_3_2_2_3_1_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-20681-329"},{"key":"e_1_3_2_2_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2764918"},{"issue":"2","key":"e_1_3_2_2_3_3_2","first-page":"144","article-title":"CineLibras: Automatic generation and distribution of libras tracks for digital cinema platforms","volume":"6","author":"Domingues Leonardo A.","year":"2015","unstructured":"Leonardo A. Domingues, Tiago M. U. de Ara\u00fajo, Danilo A. N. S. Silva, Felipe L. S. Ferreira, Guido L. de S. Filho, and Fernando T. Paulino. 2015. CineLibras: Automatic generation and distribution of libras tracks for digital cinema platforms. J. Inf. Data Manag. 6, 2 (June 2015), 144\u2013159. Retrieved from https:\/\/periodicos.ufmg.br\/index.php\/jidm\/article\/view\/317.","journal-title":"J. Inf. Data Manag"},{"key":"e_1_3_2_2_3_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10209-015-0408-1"},{"key":"e_1_3_2_2_3_5_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-5105"},{"key":"e_1_3_2_2_3_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-20681-3 36"},{"issue":"3","key":"e_1_3_2_2_3_7_2","first-page":"75","article-title":"HospiSign: An interactive sign language platform for hearing impaired","volume":"11","author":"S\u00fczg\u00fcn Muhammed M.","year":"2015","unstructured":"Muhammed M. S\u00fczg\u00fcn, Hilal \u00d6zdemir, Necati Cihan Camg\u00f6z, Ahmet Alp Kindiro\u011flu, Do\u011fa\u00e7 Ba\u015faran, Cengiz Togay, and Lale Akarun. 2015. HospiSign: An interactive sign language platform for hearing impaired. J. Naval Sci. Eng. 11, 3 (2015), 75\u201392.","journal-title":"J. Naval Sci. Eng"},{"key":"e_1_3_2_2_3_8_2","doi-asserted-by":"publisher","DOI":"10.1080\/10400435.2014.952845"},{"issue":"1","key":"e_1_3_2_2_3_9_2","first-page":"129","article-title":"Using educational games for sign language learning\u2014A SignWriting learning game: Case study","volume":"19","author":"Bouzid Yosra","year":"2016","unstructured":"Yosra Bouzid, Mohamed Ali Khenissi, Fathi Essalmi, and Mohamed Jemni. 2016. Using educational games for sign language learning\u2014A SignWriting learning game: Case study. Educ. Technol. Societ. 19, 1 (2016), 129\u2013141.","journal-title":"Educ. Technol. Societ"},{"key":"e_1_3_2_2_3_10_2","first-page":"2084","article-title":"Continuous profile models in ASL syntactic facial expression synthesis","author":"Kacorri H.","year":"2016","unstructured":"H. Kacorri and M. Huenerfauth. 2016. Continuous profile models in ASL syntactic facial expression synthesis. In ACL. 2084\u20132093.","journal-title":"ACL"},{"key":"e_1_3_2_2_3_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10209-015-0407-2"},{"key":"e_1_3_2_2_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACPR.2017.130"},{"key":"e_1_3_2_2_3_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-58703-5_1"},{"key":"e_1_3_2_2_3_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998203"},{"key":"e_1_3_2_2_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3046787"},{"key":"e_1_3_2_2_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3175536.3175584"},{"key":"e_1_3_2_2_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3136907.3136945"},{"key":"e_1_3_2_2_3_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49625-2_7"},{"key":"e_1_3_2_2_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3234695.3236356"},{"key":"e_1_3_2_2_3_20_2","doi-asserted-by":"publisher","DOI":"10.18280\/mmc_c.790402"},{"key":"e_1_3_2_2_3_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-92049-8_11"},{"key":"e_1_3_2_2_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240530"},{"key":"e_1_3_2_2_3_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2018.00083"},{"key":"e_1_3_2_2_3_24_2","doi-asserted-by":"publisher","DOI":"10.1080\/10400435.2016.1268218"},{"key":"e_1_3_2_2_3_25_2","doi-asserted-by":"publisher","DOI":"10.22441\/sinergi.2018.2.004"},{"key":"e_1_3_2_2_3_26_2","article-title":"Sign language gesture recognition through computer vision","author":"Nyaga Casam Njagi","year":"2018","unstructured":"Casam Njagi Nyaga and Ruth Diko Wario. 2018. Sign language gesture recognition through computer vision. In IST-Africa.","journal-title":"IST-Africa"},{"key":"e_1_3_2_2_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308561.3353791"},{"key":"e_1_3_2_2_3_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-019-00609-9"},{"key":"e_1_3_2_2_3_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-27192-3_4"},{"key":"e_1_3_2_2_3_30_2","doi-asserted-by":"publisher","DOI":"10.1108\/JET-12-2018-0058"},{"key":"e_1_3_2_2_3_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-23560-4_37"},{"key":"e_1_3_2_2_3_32_2","doi-asserted-by":"publisher","DOI":"10.3169\/mta.7.126"},{"key":"e_1_3_2_2_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377552"},{"key":"e_1_3_2_2_3_34_2","first-page":"46","article-title":"A speech translation system for medical dialogue in sign language \u2014 Questionnaire on user perspective of videos and the use of avatar technology","author":"Bouillon Pierrette","year":"2020","unstructured":"Pierrette Bouillon, Bastien David, Irene Strasly, Herv\u00e9 Spechbach. 2020. A speech translation system for medical dialogue in sign language \u2014 Questionnaire on user perspective of videos and the use of avatar technology. In BfC. 46\u201354","journal-title":"BfC"},{"issue":"1","key":"e_1_3_2_2_3_35_2","first-page":"263","article-title":"Determining the effectiveness of the developed prototype that translates Pakistan sign language into English text and speech","volume":"7","author":"Naseem Maria","year":"2020","unstructured":"Maria Naseem, Summaira Sarfraz, Ali Abbas, and Husnain Raza. 2020. Determining the effectiveness of the developed prototype that translates Pakistan sign language into English text and speech. Int. J. Lit., Lang. Ling. 7, 1 (2020), 263\u2013275.","journal-title":"Int. J. Lit., Lang. Ling."},{"key":"e_1_3_2_2_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383652.3423871"},{"key":"e_1_3_2_2_3_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-3434-8_1"},{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1201\/9781420064995"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.3233\/TAD-2008-20206"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1353\/sls.0.0011"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308561.3353774"},{"key":"e_1_3_3_6_2","unstructured":"CRPD. 2006. Convention on the Rights of Persons with Disabilities. Retrieved from t.ly\/w5ok."},{"key":"e_1_3_3_7_2","volume-title":"Interactions","author":"Walker Ashley","year":"2019","unstructured":"Ashley Walker, Yaxing Yao, Christine Geeng, Roberto Hoyle, and Pamela Wisniewski. 2019. Moving beyond \u201cone size fits all\u201d: Research considerations for working with vulnerable populations. Interactions (IX), XXVI. (Nov-Dec. Nov-Dec. 2019)."},{"key":"e_1_3_3_8_2","article-title":"La Torre de Babel 3.0: Los Derechos de las minor\u00edas ling\u00fc\u00edsticas desde las RRII","author":"Sep\u00falveda Mu\u00f1oz Sandra","year":"2018","unstructured":"Sandra Sep\u00falveda Mu\u00f1oz. 2018. La Torre de Babel 3.0: Los Derechos de las minor\u00edas ling\u00fc\u00edsticas desde las RRII. In Interculturalizaciones: Transiciones, Mediaciones y Conflictos en Lenguas, Comunidades y Educaci\u00f3n Escolar. Universidad Aut\u00f3noma Metropolitana, Unidad Iztapalapa, Consejo Editorial de Ciencias Sociales y Humanidades.","journal-title":"Interculturalizaciones: Transiciones, Mediaciones y Conflictos en Lenguas, Comunidades y Educaci\u00f3n Escolar"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2011.01.004"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8091047"},{"key":"e_1_3_3_11_2","volume-title":"Human-Computer Interaction (Communications in Computer and Information Science)","author":"Prietch Soraia Silva","year":"2019","unstructured":"Soraia Silva Prietch, Polianna dos Santos Paim, Ivan Pineda Olmos, Josefina Garc\u00eda Guerrero, and Juan Manuel Calleros Gonzalez. 2019b. The human and the context components in the design of automatic sign language recognition systems. In Human-Computer Interaction (Communications in Computer and Information Science). Springer, 1114."},{"key":"e_1_3_3_12_2","volume-title":"Cook & Hussey's Assistive Technologies: Principles and Practice","author":"Cook Albert M.","year":"2008","unstructured":"Albert M. Cook and Jan M. Polgar. 2008. Cook & Hussey's Assistive Technologies: Principles and Practice. Mosby Elsevier, St. Louis, Mo."},{"key":"e_1_3_3_13_2","unstructured":"Hernisa Kacorri. 2015. TR-2015001: A survey and critique of facial expression synthesis in sign language animation. Computer Science Technical Reports. City University of New York (CUNY) Academic Works. Retrieved from https:\/\/academicworks.cuny.edu\/gc_cs_tr\/403."},{"key":"e_1_3_3_14_2","unstructured":"Oscar Koller. 2020. Quantitative Survey of the State of the Art in Sign Language Recognition. Retrieved from https:\/\/www.microsoft.com\/en-us\/research\/people\/oskoller\/."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.3390\/s18072208"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-23563-5_30"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-37830-1_3"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.113794"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1000097"},{"issue":"10","key":"e_1_3_3_20_2","article-title":"Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews","volume":"7","author":"Shea B. J.","year":"2007","unstructured":"B. J. Shea, J. M. Grimshaw, G. A. Wells, M. Boers, N. Andersson, C. Hamel, et al. 2007. Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med. Res. Methodol. 7, 10 (2007).","journal-title":"BMC Med. Res. Methodol"},{"key":"e_1_3_3_21_2","article-title":"A guide to conducting a standalone systematic literature review","volume":"37","author":"Okoli Chitu","year":"2015","unstructured":"Chitu Okoli. 2015. A guide to conducting a standalone systematic literature review. Commun. Assoc. Inf. Syst. 37 (2015). Retrieved from http:\/\/aisel.aisnet.org\/cais\/vol37\/iss1\/43.","journal-title":"Commun. Assoc. Inf. Syst"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.10.006"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2907069"},{"key":"e_1_3_3_24_2","volume-title":"Handbook for Synthesizing Qualitative Research","author":"Sandelowski Margarete","year":"2007","unstructured":"Margarete Sandelowski and Julie Barroso. 2007. Handbook for Synthesizing Qualitative Research Springer. Retrieved from https:\/\/bit.ly\/2VcCq5C."},{"key":"e_1_3_3_25_2","volume-title":"Research Methods in Human-computer Interaction","author":"Lazar Jonathan","year":"2017","unstructured":"Jonathan Lazar, Jinjuan Feng, and Harry Hochheiser. 2017. Research Methods in Human-computer Interaction. 2nd ed. Elsevier.","edition":"2"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2347504.2347523"},{"key":"e_1_3_3_27_2","volume-title":"Interaction Design: Beyond Human-computer Interaction","author":"Preece Jennifer","year":"2002","unstructured":"Jennifer Preece, Yvonne Rogers, and Helen Sharp. 2002. Interaction Design: Beyond Human-computer Interaction. Wiley."},{"key":"e_1_3_3_28_2","volume-title":"Codesign de Redes Digitais: Tecnologia e Educa\u00e7\u00e3o a Servi\u00e7o da Inclus\u00e3o Social","author":"Baranauskas Maria Cecilia Calini","year":"2013","unstructured":"Maria Cecilia Calini Baranauskas, Maria Cecilia Martins, and Jose Armando Valente. 2013. Codesign de Redes Digitais: Tecnologia e Educa\u00e7\u00e3o a Servi\u00e7o da Inclus\u00e3o Social. Penso Editora, Porto Alegre, RS."},{"key":"e_1_3_3_29_2","volume-title":"Intera\u00e7\u00e3o Humano-computador","author":"Barbosa Simone D. J.","year":"2017","unstructured":"Simone D. J. Barbosa and Bruno S. da Silva. 2017. Intera\u00e7\u00e3o Humano-computador. Elsevier, Rio de Janeiro."},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1080\/01449290301782"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1080\/014492996120049"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1080\/0954412989333"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3166.3170"},{"key":"e_1_3_3_34_2","first-page":"1051","article-title":"Participatory design: The third space in HCI","author":"Muller Michael J.","year":"2002","unstructured":"Michael J. Muller and Allison Druin. 2002. Participatory design: The third space in HCI. The Human-computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. L. Erlbaum Associates Inc., 1051\u20131068.","journal-title":"The Human-computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/2723869"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781483345321"},{"key":"e_1_3_3_37_2","volume-title":"eLEOT","author":"Adamo-Villani Nicoletta","year":"2015","unstructured":"Nicoletta Adamo-Villani, Jason Lestina, and Saikiran Anasingaraju. 2015. Does character's visual style affect viewer's perception of signing avatars? In eLEOT. Retrieved from https:\/\/www.springer.com\/gp\/book\/9783319288826."},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-20681-331"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3358961.3359000"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1590\/S1413-65382013000200002"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357155.3358437"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386296.338630"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3436996"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2019.103777"}],"container-title":["ACM Transactions on Accessible Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3563395","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3563395","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:35Z","timestamp":1750182575000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3563395"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,11]]},"references-count":80,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,12,31]]}},"alternative-id":["10.1145\/3563395"],"URL":"https:\/\/doi.org\/10.1145\/3563395","relation":{},"ISSN":["1936-7228","1936-7236"],"issn-type":[{"value":"1936-7228","type":"print"},{"value":"1936-7236","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,11]]},"assertion":[{"value":"2020-08-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-29","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}