{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T21:29:16Z","timestamp":1781990956242,"version":"3.54.5"},"reference-count":58,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,11,13]],"date-time":"2019-11-13T00:00:00Z","timestamp":1573603200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004895","name":"European Social Fund","doi-asserted-by":"publisher","award":["5007016"],"award-info":[{"award-number":["5007016"]}],"id":[{"id":"10.13039\/501100004895","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express their content and mechanics. Existing ARAG implementations have focused on exploring the surroundings and navigating to virtual sound sources as the main mode of interaction. This paper suggests that gestural activity with a handheld device can realize complex modes of sonic interaction in the augmented environment, resulting in an enhanced immersive game experience. The ARAG \u201cAudio Legends\u201d was designed and tested to evaluate the usability and immersion of a system featuring an exploration phase based on auditory navigation, as well as an action phase, in which players aim at virtual sonic targets and wave the device to hit them or hold the device to block them. The results of the experiment provide evidence that players are easily accustomed to auditory navigation and that gestural sonic interaction is perceived as difficult, yet this does not affect negatively the system\u2019s usability and players\u2019 immersion. Findings also include indications that elements, such as sound design, the synchronization of sound and gesture, the fidelity of audio augmentation, and environmental conditions, also affect significantly the game experience, whereas background factors, such as age, sex, and game or music experience, do not have any critical impact.<\/jats:p>","DOI":"10.3390\/mti3040073","type":"journal-article","created":{"date-parts":[[2019,11,14]],"date-time":"2019-11-14T10:56:34Z","timestamp":1573728994000},"page":"73","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Audio Legends: Investigating Sonic Interaction in an Augmented Reality Audio Game"],"prefix":"10.3390","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7404-4063","authenticated-orcid":false,"given":"Emmanouel","family":"Rovithis","sequence":"first","affiliation":[{"name":"Department of Audiovisual Arts, Ionian University, GR49100 Corfu, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1470-7352","authenticated-orcid":false,"given":"Nikolaos","family":"Moustakas","sequence":"additional","affiliation":[{"name":"Department of Audiovisual Arts, Ionian University, GR49100 Corfu, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8477-9490","authenticated-orcid":false,"given":"Andreas","family":"Floros","sequence":"additional","affiliation":[{"name":"Department of Audiovisual Arts, Ionian University, GR49100 Corfu, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kostas","family":"Vogklis","sequence":"additional","affiliation":[{"name":"Department of Audiovisual Arts, Ionian University, GR49100 Corfu, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,13]]},"reference":[{"key":"ref_1","unstructured":"Moustakas, N., Floros, A., and Kanellopoulos, N. (2009, January 9\u201312). Eidola: An interactive augmented reality audio-game prototype. Proceedings of the Audio Engineering Society Convention 127, New York, NY, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Rovithis, E., Floros, A., Mniestris, A., and Grigoriou, N. (2014, January 22\u201324). Audio games as educational tools: Design principles and examples. Proceedings of the 2014 IEEE Games Media Entertainment, Toronto, ON, Canada.","DOI":"10.1109\/GEM.2014.7048083"},{"key":"ref_3","unstructured":"Moustakas, N., Floros, A., Rovithis, E., and Vogklis, K. (2019, January 20\u201323). Augmented Audio-Only Games: A New Generation of Immersive Acoustic Environments through Advanced Mixing. Proceedings of the Audio Engineering Society Convention 146, Dublin, Ireland."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"144","DOI":"10.34190\/JEL.17.2.07","article-title":"Bridging Audio and Augmented Reality towards a new Generation of Serious Audio-only Games","volume":"17","author":"Rovithis","year":"2019","journal-title":"Electron. J. e-Learn."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Winberg, F., and Hellstr\u00f6m, S.O. (2000, January 1\u20136). Investigating auditory direct manipulation: Sonifying the Towers of Hanoi. Proceedings of the CHI\u201900 Extended Abstracts on Human Factors in Computing Systems, The Hague, The Netherlands.","DOI":"10.1145\/633292.633456"},{"key":"ref_6","unstructured":"Collins, K., and Kapralos, B. (2012, January 7\u20138). Beyond the Screen. Proceedings of the Computer Games Multimedia and Allied Technology, CGAT 2012, Bali Dynasty Resort, Bali, Indonesia."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"S\u00e1nchez, J., S\u00e1enz, M., Pascual-Leone, A., and Merabet, L. (2010, January 10\u201315). Enhancing navigation skills through audio gaming. Proceedings of the CHI\u201910 Extended Abstracts on Human Factors in Computing Systems, Atlanta, GA, USA.","DOI":"10.1145\/1753846.1754091"},{"key":"ref_8","first-page":"5","article-title":"Designing with sound to enhance learning: Four recommendations from the film industry","volume":"2","author":"Bishop","year":"2012","journal-title":"J. Appl. Instr. Des."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Franinovi\u0107, K., and Serafin, S. (2013). Sonic Interaction Design, MIT Press.","DOI":"10.7551\/mitpress\/8555.001.0001"},{"key":"ref_10","unstructured":"Targett, S., and Fernstrom, M. (2003). Audio Games: Fun for All? All for Fun!, Georgia Institute of Technology."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2008\/178923","article-title":"Audio interaction in computer mediated games","volume":"2008","author":"Parker","year":"2008","journal-title":"Int. J. Comput. Games Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liljedahl, M., and Papworth, N. (2008, January 22\u201323). Beowulf field test paper. Proceedings of the Audio Mostly Conference, Pitea, Sweden.","DOI":"10.1145\/1255047.1255088"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Beksa, J., Fizek, S., and Carter, P. (2015). Audio games: Investigation of the potential through prototype development. A Multimodal End-2-End Approach to Accessible Computing, Springer.","DOI":"10.1007\/978-1-4471-6708-2_11"},{"key":"ref_14","unstructured":"R\u00f6ber, N. (2005, January 16\u201320). Playing Audio-only Games: A compendium of interacting with virtual, auditory Worlds. Proceedings of the 2005 DiGRA International Conference: Changing Views: Worlds in Play, Vancouver, Canada."},{"key":"ref_15","unstructured":"Baer, R.H., and Morrison, H. (2019, November 12). Simon, Electronic Game. Available online: https:\/\/en.wikipedia.org\/wiki\/Simon_(game)."},{"key":"ref_16","unstructured":"(2019, August 30). Bop It. Available online: https:\/\/en.wikipedia.org\/wiki\/Bop_It."},{"key":"ref_17","unstructured":"(2019, August 30). Dance Dance Revolution. Available online: https:\/\/en.wikipedia.org\/wiki\/Dance_Dance_Revolution_(1998_video_game)."},{"key":"ref_18","unstructured":"Harmonix, N. (2005). Budcat Creations, Vicarious Visions & FreeStyleGames. (2005\u2013present)\n\t\t\t. Guitar Hero [Interactive Music Game], Red Octane\/Activision."},{"key":"ref_19","first-page":"1","article-title":"Investigating the impact of music activities incorporating Soundbeam Technology on children with multiple disabilities","volume":"10","author":"Lee","year":"2015","journal-title":"J. Eur. Teach. Educ. Netw."},{"key":"ref_20","unstructured":"(2019, August 30). Bit Generations: Soundvoyager. Available online: https:\/\/nintendo.fandom.com\/wiki\/Bit_Generations:_Soundvoyager."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Urbanek, M., and G\u00fcldenpfennig, F. (2017, January 20\u201323). Tangible Audio Game Development Kit: Prototyping Audio Games with a Tangible Editor. Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction, Yokohama, Japan.","DOI":"10.1145\/3024969.3025077"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1111\/bjet.12628","article-title":"Accessible electronic games for blind children and young people","volume":"49","author":"Giannakopoulos","year":"2018","journal-title":"Br. J. Educ. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hugill, A., and Amelides, P. (2016). Audio only computer games\u2014Papa Sangre. Expanding the Horizon of Electroacoustic Music Analysis, Cambridge University Press.","DOI":"10.1017\/CBO9781316339633.018"},{"key":"ref_24","unstructured":"(2019, August 30). Audio Game Hub. Available online: http:\/\/www.audiogamehub.com\/audio-game-hub\/."},{"key":"ref_25","unstructured":"(2019, August 30). Bloom (Version 3.1). Available online: https:\/\/apps.apple.com\/us\/app\/bloom\/id292792586."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1162\/pres.1997.6.4.355","article-title":"A survey of augmented reality","volume":"6","author":"Azuma","year":"1997","journal-title":"Presence Teleoper. Virtual Environ."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hug, D., Kemper, M., Panitz, K., and Franinovi\u0107, K. (2016, January 4\u20136). Sonic Playgrounds: Exploring Principles and Tools for Outdoor Sonic Interaction. Proceedings of the Audio Mostly 2016, Norrkoping, Sweden.","DOI":"10.1145\/2986416.2986448"},{"key":"ref_28","unstructured":"(2019, August 30). RjDj. Available online: https:\/\/en.wikipedia.org\/wiki\/RjDj."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"987","DOI":"10.1007\/s00779-011-0459-0","article-title":"Auditory display design for exploration in mobile audio-augmented reality","volume":"16","author":"Oakley","year":"2012","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_30","unstructured":"Mariette, N. (2007, January 15\u201318). From backpack to handheld: The recent trajectory of personal location aware spatial audio. Proceedings of the PerthDAC 2007: 7th Digital Arts and Culture Conference, Perth, Australia."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cohen, M., and Villegas, J. (2011, January 19\u201320). From whereware to whence-and whitherware: Augmented audio reality for position-aware services. Proceedings of the 2011 IEEE International Symposium on VR Innovation, SUNTEC, Singapore International Convention and Exhibition Center, Singapore.","DOI":"10.1109\/ISVRI.2011.5759650"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mcgookin, D., Brewster, S., and Priego, P. (2009, January 10\u201311). Audio bubbles: Employing non-speech audio to support tourist wayfinding. Proceedings of the International Conference on Haptic and Audio Interaction Design, Dresden, Germany.","DOI":"10.1007\/978-3-642-04076-4_5"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Stahl, C. (2007, January 9\u201312). The roaring navigator: A group guide for the zoo with shared auditory landmark display. Proceedings of the 9th International Conference on Human Computer Interaction with Mobile Devices and Services, Singapore.","DOI":"10.1145\/1377999.1378042"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1007\/s007790200025","article-title":"AudioGPS: Spatial audio navigation with a minimal attention interface","volume":"6","author":"Holland","year":"2002","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4517150","DOI":"10.1155\/2018\/4517150","article-title":"Towards the Design of Effective Haptic and Audio Displays for Augmented Reality and Mixed Reality Applications","volume":"2018","author":"Ren","year":"2018","journal-title":"Adv. Multimedia"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chatzidimitris, T., Gavalas, D., and Michael, D. (2016, January 18\u201320). SoundPacman: Audio augmented reality in location-based games. Proceedings of the 2016 18th IEEE Mediterranean Electrotechnical Conference (MELECON), Limassol, Cyprus.","DOI":"10.1109\/MELCON.2016.7495414"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Reid, J., Geelhoed, E., Hull, R., Cater, K., and Clayton, B. (2005, January 2\u20137). Parallel worlds: Immersion in location-based experiences. Proceedings of the CHI\u201905 extended abstracts on Human factors in computing systems, Portland, OR, USA.","DOI":"10.1145\/1056808.1057009"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Paterson, N., Naliuka, K., Jensen, S.K., Carrigy, T., Haahr, M., and Conway, F. (2010, January 15\u201317). Design, implementation and evaluation of audio for a location aware augmented reality game. Proceedings of the 3rd International Conference on Fun and Games, Leuven, Belgium.","DOI":"10.1145\/1823818.1823835"},{"key":"ref_39","first-page":"618","article-title":"Augmented reality audio for mobile and wearable appliances","volume":"52","author":"Jakka","year":"2004","journal-title":"J. Audio Eng. Soc."},{"key":"ref_40","unstructured":"Ekman, I. (2007, January 27\u201328). Sound-based gaming for sighted audiences\u2014Experiences from a mobile multiplayer location aware game. Proceedings of the 2nd Audio Mostly Conference, Ilmenau, Germany."},{"key":"ref_41","unstructured":"Shepard, M. (2006, January 2\u20133). Tactical sound garden [tsg] toolkit. Proceedings of the 3rd International Workshop on Mobile Music Technology, Brighton, UK."},{"key":"ref_42","unstructured":"Tikander, M., Karjalainen, M., and Riikonen, V. (2008, January 1\u20134). An augmented reality audio headset. Proceedings of the 11th International Conference on Digital Audio Effects (DAFx-08), Espoo, Finland."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Blum, J.R., Bouchard, M., and Cooperstock, J.R. (2011, January 6\u20139). What\u2019s around me? Spatialized audio augmented reality for blind users with a smartphone. Proceedings of the International Conference on Mobile and Ubiquitous Systems: Computing, Networking, and Services, Copenhagen, Denmark.","DOI":"10.1007\/978-3-642-30973-1_5"},{"key":"ref_44","first-page":"21","article-title":"Soundpark: Exploring ubiquitous computing through a mixed reality multi-player game experiment","volume":"8","author":"Pellerin","year":"2009","journal-title":"Stud. Inform. Universalis"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Strachan, S., Eslambolchilar, P., Murray-Smith, R., Hughes, S., and O\u2019Modhrain, S. (2005, January 19\u201322). GpsTunes: Controlling navigation via audio feedback. Proceedings of the 7th International Conference on Human Computer Interaction With Mobile Devices & Services, Salzburg, Austria.","DOI":"10.1145\/1085777.1085831"},{"key":"ref_46","unstructured":"Etter, R., and Specht, M. (2005, January 8\u201313). Melodious walkabout: Implicit navigation with contextualized personal audio contents. Proceedings of the Third International Conference on Pervasive Computing, Munich, Germany."},{"key":"ref_47","unstructured":"Eckel, G. (2001, January 25\u201327). Immersive audio-augmented environments: The LISTEN project. Proceedings of the Fifth IEEE International Conference on Information Visualisation, London, UK."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Hatala, M., Kalantari, L., Wakkary, R., and Newby, K. (2004, January 14\u201317). Ontology and rule based retrieval of sound objects in augmented audio reality system for museum visitors. Proceedings of the 2004 ACM Symposium on Applied Computing, Nicosia, Cyprus.","DOI":"10.1145\/967900.968114"},{"key":"ref_49","unstructured":"Rozier, J., Karahalios, K., and Donath, J. Hear&There: An Augmented Reality System of Linked Audio. In Proceedings of the International Conference on Auditory Display, Available online: https:\/\/www.icad.org\/websiteV2.0\/Conferences\/ICAD2000\/ICAD2000.html."},{"key":"ref_50","unstructured":"Magnusson, C., Breidegard, B., and Rassmus-Gr\u00f6hn, K. (2009, January 10\u201311). Soundcrumbs\u2013Hansel and Gretel in the 21st century. Proceedings of the 4th International Workshop on Haptic and Audio Interaction Design (HAID \u201909), Dresden, Germany."},{"key":"ref_51","unstructured":"Alderman, N. (2019, November 12). Zombies, run!, Electronic Game. Available online: https:\/\/zombiesrungame.com."},{"key":"ref_52","unstructured":"Lyons, K., Gandy, M., and Starner, T. (2000). Guided by Voices: An Audio Augmented Reality System, Georgia Institute of Technology."},{"key":"ref_53","unstructured":"Moustakas, N., Floros, A., and Grigoriou, N. (2011, January 13\u201316). Interactive audio realities: An augmented\/mixed reality audio game prototype. Proceedings of the Audio Engineering Society Convention 130, London, UK."},{"key":"ref_54","unstructured":"St\u00f6\u00dfel, C., and Blessing, L. (2009). Is gesture-based interaction a way to make interfaces more intuitive and accessible. HCI 2009 Electronic Proceedings: WS4\u2013Prior Experience, British Computer Society."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Buchmann, V., Violich, S., Billinghurst, M., and Cockburn, A. (2004, January 15\u201318). FingARtips: Gesture based direct manipulation in Augmented Reality. Proceedings of the 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia, Singapore.","DOI":"10.1145\/988834.988871"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Yusof, C.S., Bai, H., Billinghurst, M., and Sunar, M.S. (2016). A review of 3D gesture interaction for handheld augmented reality. J. Teknol., 78.","DOI":"10.11113\/jt.v78.6923"},{"key":"ref_57","first-page":"95","article-title":"Including visually impaired players in a graphical adventure game: A study of immersion","volume":"10","author":"Brusk","year":"2015","journal-title":"IADIS Int. J. Comput. Sci. Inf. Syst."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Sodnik, J., Tomazic, S., Grasset, R., Duenser, A., and Billinghurst, M. (2006, January 20\u201324). Spatial sound localization in an augmented reality environment. Proceedings of the 18th Australia Conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments, Sydney, Australia.","DOI":"10.1145\/1228175.1228197"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/3\/4\/73\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:34:11Z","timestamp":1760189651000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/3\/4\/73"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,13]]},"references-count":58,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["mti3040073"],"URL":"https:\/\/doi.org\/10.3390\/mti3040073","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,13]]}}}