{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T13:03:28Z","timestamp":1780578208871,"version":"3.54.1"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2015,11,14]],"date-time":"2015-11-14T00:00:00Z","timestamp":1447459200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["IIS-1353312"],"award-info":[{"award-number":["IIS-1353312"]}]},{"name":"Google Research Award"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Access. Comput."],"published-print":{"date-parts":[[2015,11,14]]},"abstract":"<jats:p>\n            While sighted users may learn to perform touchscreen gestures through observation (e.g., of other users or video tutorials), such mechanisms are inaccessible for users with visual impairments. As a result, learning to perform gestures without visual feedback can be challenging. We propose and evaluate two techniques to teach touchscreen gestures to users with visual impairments: (1)\n            <jats:italic>gesture sonification<\/jats:italic>\n            to generate sound based on finger touches, creating an audio representation of a gesture; and (2)\n            <jats:italic>corrective verbal feedback<\/jats:italic>\n            that combined automatic analysis of the user's drawn gesture with speech feedback. To refine and evaluate the techniques, we conducted three controlled laboratory studies. The first study, with 12 sighted participants, compared parameters for sonifying gestures in an eyes-free scenario. We identified pitch+stereo panning as the best combination. In the second study, ten blind and low-vision participants completed gesture replication tasks for single-stroke, multistroke, and multitouch gestures using the gesture sonification feedback. We found that multistroke gestures were more difficult to understand in sonification, but that playing each finger sound serially may improve understanding. In the third study, six blind and low-vision participants completed gesture replication tasks with both the sonification and corrective verbal feedback techniques. Subjective data and preliminary performance findings indicated that the techniques offer complementary advantages: although verbal feedback was preferred overall primarily due to the precision of its instructions, almost all participants appreciated the sonification for certain situations (e.g., to convey speed). This article extends our previous publication on gesture sonification by extending these techniques to multistroke and multitouch gestures. These findings provide a foundation for nonvisual training systems for touchscreen gestures.\n          <\/jats:p>","DOI":"10.1145\/2764917","type":"journal-article","created":{"date-parts":[[2015,11,18]],"date-time":"2015-11-18T13:42:28Z","timestamp":1447854148000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Audio-Based Feedback Techniques for Teaching Touchscreen Gestures"],"prefix":"10.1145","volume":"7","author":[{"given":"Uran","family":"Oh","sequence":"first","affiliation":[{"name":"University of Maryland, College Park"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stacy","family":"Branham","sequence":"additional","affiliation":[{"name":"University of Maryland, Baltimore County"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leah","family":"Findlater","sequence":"additional","affiliation":[{"name":"University of Maryland, College Park"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaun K.","family":"Kane","sequence":"additional","affiliation":[{"name":"University of Colorado, Boulder"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2015,11,14]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2485760.2485775"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of Graphics Interface","author":"Azenkot S.","year":"2012","unstructured":"S. Azenkot , J. O. Wobbrock , S. Prasain , and R. E. Ladner . 2012. Input finger detection for nonvisual touch screen text entry in Perkinput . In Proceedings of Graphics Interface 2012 , Canadian Information Processing Society, 121--129. S. Azenkot, J. O. Wobbrock, S. Prasain, and R. E. Ladner. 2012. Input finger detection for nonvisual touch screen text entry in Perkinput. In Proceedings of Graphics Interface 2012, Canadian Information Processing Society, 121--129."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449724"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-12654-3_24"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the International Conference on Auditory Display.","author":"Brown L. M.","unstructured":"L. M. Brown and S. A. Brewster . 2003. Drawing by ear: Interpreting sonified line graphs . In Proceedings of the International Conference on Auditory Display. L. M. Brown and S. A. Brewster. 2003. Drawing by ear: Interpreting sonified line graphs. In Proceedings of the International Conference on Auditory Display."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1006\/nimg.2001.0812"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1408760.1408766"},{"key":"e_1_2_1_8_1","volume-title":"Brailletouch: Mobile texting for the visually impaired. In Proceedings of the 6th International Conference on Universal Access in Human-Computer Interaction: Context Diversity","author":"Frey B.","year":"2011","unstructured":"B. Frey , C. Southern , and M. Romero . 2011 . Brailletouch: Mobile texting for the visually impaired. In Proceedings of the 6th International Conference on Universal Access in Human-Computer Interaction: Context Diversity . Springer-Verlag , 19--25. B. Frey, C. Southern, and M. Romero. 2011. Brailletouch: Mobile texting for the visually impaired. In Proceedings of the 6th International Conference on Universal Access in Human-Computer Interaction: Context Diversity. Springer-Verlag, 19--25."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2008.88"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979354"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"E. Hoggan and S. Brewster. 2012. Non-speech auditory and crossmodal output. In The Human-Computer Interaction Handbook Julie A. Jacko and Andrew Sears (Eds.). L. Erlbaum Associates Inc. Hillsdale NJ 220--239.  E. Hoggan and S. Brewster. 2012. Non-speech auditory and crossmodal output. In The Human-Computer Interaction Handbook Julie A. Jacko and Andrew Sears (Eds.). L. Erlbaum Associates Inc. Hillsdale NJ 220--239.","DOI":"10.1201\/b11963-ch-10"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 7th International Conference on Auditory Display.","author":"Kamel H. M.","unstructured":"H. M. Kamel , P. Roth , and R. R. Sinha . 2001. Graphics and user's exploration via simple sonics (GUESS): Providing interrelational representation of objects in a non-visual environment . In Proceedings of the 7th International Conference on Auditory Display. H. M. Kamel, P. Roth, and R. R. Sinha. 2001. Graphics and user's exploration via simple sonics (GUESS): Providing interrelational representation of objects in a non-visual environment. In Proceedings of the 7th International Conference on Auditory Display."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1414471.1414487"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047232"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979001"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2021164.2021181"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2414536.2414591"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2007.04.001"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 6th International Conference on EuroHaptics.","author":"Noble N.","unstructured":"N. Noble and B. Martin . 2006. Shape discovering using tactile guidance . In Proceedings of the 6th International Conference on EuroHaptics. N. Noble and B. Martin. 2006. Shape discovering using tactile guidance. In Proceedings of the 6th International Conference on EuroHaptics."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1744161.1744163"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2513383.2513455"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1152215.1152260"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1993060.1993067"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1851600.1851606"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the International Conference on Auditory Display. 260--263","author":"Walker B. N.","unstructured":"B. N. Walker , J. Lindsay , and S. Lab . 2005. Navigation performance in a virtual environment with bonephones . In Proceedings of the International Conference on Auditory Display. 260--263 . B. N. Walker, J. Lindsay, and S. Lab. 2005. Navigation performance in a virtual environment with bonephones. In Proceedings of the International Conference on Auditory Display. 260--263."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1714458.1714459"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207734"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1622176.1622198"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1352782.1352786"}],"container-title":["ACM Transactions on Accessible Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2764917","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2764917","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:12:23Z","timestamp":1750227143000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2764917"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,11,14]]},"references-count":29,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2015,11,14]]}},"alternative-id":["10.1145\/2764917"],"URL":"https:\/\/doi.org\/10.1145\/2764917","relation":{},"ISSN":["1936-7228","1936-7236"],"issn-type":[{"value":"1936-7228","type":"print"},{"value":"1936-7236","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,11,14]]},"assertion":[{"value":"2014-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-11-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}