{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:09:37Z","timestamp":1760234977337,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,6,27]],"date-time":"2021-06-27T00:00:00Z","timestamp":1624752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["PD\/BD\/128230\/2016"],"award-info":[{"award-number":["PD\/BD\/128230\/2016"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Mobile handheld devices, such as smartphones and tablets, have become some of the most prominent ubiquitous terminals within the information and communication technology landscape. Their transformative power within the digital music domain changed the music ecosystem from production to distribution and consumption. Of interest here is the ever-expanding number of mobile music applications. Despite their growing popularity, their design in terms of interaction perception and control is highly arbitrary. It remains poorly addressed in related literature and lacks a clear, systematized approach. In this context, our paper aims to provide the first steps towards defining guidelines for optimal sonic interaction design practices in mobile music applications. Our design approach is informed by user data in appropriating mobile handheld devices. We conducted an experiment to learn links between control gestures and musical parameters, such as pitch, duration, and amplitude. A twofold action\u2014reflection protocol and tool-set for evaluating the aforementioned links\u2014are also proposed. The results collected from the experiment show statistically significant trends in pitch and duration control gesture mappings. On the other hand, amplitude appears to elicit a more diverse mapping approach, showing no definitive trend in this experiment.<\/jats:p>","DOI":"10.3390\/mti5070032","type":"journal-article","created":{"date-parts":[[2021,6,27]],"date-time":"2021-06-27T23:57:22Z","timestamp":1624838242000},"page":"32","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Musical Control Gestures in Mobile Handheld Devices: Design Guidelines Informed by Daily User Experience"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7560-9943","authenticated-orcid":false,"given":"Alexandre","family":"Cl\u00e9ment","sequence":"first","affiliation":[{"name":"Faculdade de Engenharia (FEUP), Universidade do Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3162-1021","authenticated-orcid":false,"given":"Luciano","family":"Moreira","sequence":"additional","affiliation":[{"name":"Faculdade de Engenharia (FEUP), Universidade do Porto, 4200-465 Porto, Portugal"},{"name":"Centro de Investiga\u00e7\u00e3o em Qu\u00edmica da Universidade do Porto (CIQUP), 4169-007 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6806-0569","authenticated-orcid":false,"given":"Miriam","family":"Rosa","sequence":"additional","affiliation":[{"name":"Instituto Universit\u00e1rio de Lisboa (Iscte-IUL), CIS-IUL, 1649-026 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3884-2687","authenticated-orcid":false,"given":"Gilberto","family":"Bernardes","sequence":"additional","affiliation":[{"name":"Faculdade de Engenharia (FEUP), Universidade do Porto, 4200-465 Porto, Portugal"},{"name":"Instituto de Engenharia de Sistemas e Computadores, Tecnologia e Ci\u00eancia (Inesc-Tec), 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,27]]},"reference":[{"key":"ref_1","unstructured":"Levin, G. (2021, February 02). DIALTONES (A TELESYMPHONY). Available online: http:\/\/www.flong.com\/storage\/experience\/telesymphony\/index.html."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Han, Q., and Cho, D. (2016, January 17\u201319). Characterizing the technological evolution of smartphones: Insights from performance benchmarks. Proceedings of the ACM International Conference Proceeding Series, Suwon, Korea.","DOI":"10.1145\/2971603.2971635"},{"key":"ref_3","unstructured":"Holst, A. (2021, February 02). Number of Smartphone Users Worldwide from 2016 to 2021 (in Billions). Available online: https:\/\/www.statista.com\/statistics\/330695\/number-of-smartphone-users-worldwide."},{"key":"ref_4","unstructured":"Brinkmann, P., Mccormick, C., Kirn, P., Roth, M., and Lawler, R. (2011, January 8\u201312). Embedding Pure Data with libpd. Proceedings of the Fourth International Pure Data Convention, Weimar, Germany."},{"key":"ref_5","unstructured":"Cl\u00e9ment, A.R., Ribeiro, F., Rodrigues, R., and Penha, R. (July, January 29). Bridging the gap between performers and the audience using networked smartphones: The a. bel system. Proceedings of the of ICLI 16, Brighton, United Kingdom."},{"key":"ref_6","unstructured":"Cl\u00e9ment, A., Rodrigues, R., and Penha, R. (2015, January 11\u201312). Tools and Template Development for Live Networked Musical Performance System. Proceedings of the 1st Doctoral Congress in Engineering, Porto, Portugal."},{"key":"ref_7","unstructured":"Tanaka, A., Parkinson, A., Settel, Z., and Tahiroglu, K. (2012, January 21\u201323). A Survey and Thematic Analysis Approach as Input to the Design of Mobile Music GUIs. Proceedings of the International Conference on New Interfaces for Musical Expression, Ann Arbor, Michigan, USA."},{"key":"ref_8","unstructured":"Essl, G., and Lee, S.W. (2017, January 25\u201328). Mobile Devices as Musical Instruments -State of the Art and Future Prospects. Proceedings of the 13th International Symposium on CMMR, Matosinhos, Portugal."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"8944","DOI":"10.1109\/ACCESS.2018.2876891","article-title":"Smart Musical Instruments: Vision, Design Principles, and Future Directions","volume":"7","author":"Turchet","year":"2019","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Magnusson, T. (2006, January 14\u201318). Affordances and constraints in screen-based musical instruments. Proceedings of the 4th Nordic Conference on Human-Computer Interaction, Oslo, Norway.","DOI":"10.1145\/1182475.1182532"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Papetti, S., Fr\u00f6hlich, M., and Schiesser, S. (2019, January 9\u201312). The TouchBox: An open-source audio-haptic device for finger-based interaction. Proceedings of the IEEE World Haptics Conference, Tokyo, Japan.","DOI":"10.1109\/WHC.2019.8816172"},{"key":"ref_12","unstructured":"Tanaka, A. (2010, January 15\u201318). Mapping Out Instruments, Affordances, and Mobiles. Proceedings of the International Conference on New Interfaces for Musical Expression, Sydney, Australia."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/s12193-009-0027-3","article-title":"From expressive gesture to sound: The development of an embodied mapping trajectory inside a musical interface","volume":"3","author":"Maes","year":"2010","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Magnusson, T. (2019). Sonic Writing: Technologies of Material, Symbolic, and Signal Inscriptions, Bloomsbury Academic.","DOI":"10.5040\/9781501313899"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1162\/COMJ_a_00026","article-title":"Designing Constraints: Composing and Performing with Digital Musical Systems","volume":"34","author":"Magnusson","year":"2010","journal-title":"Comput. Music. J."},{"key":"ref_16","unstructured":"Hunt, A., Wanderley, M.M., and Paradis, M. (2002, January 24\u201326). The Importance of Parameter Mapping in Electronic Instrument Design. Proceedings of the 2002 Conference on New Interfaces for Musical Expression, Dublin, Ireland."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1017\/S1355771802002030","article-title":"Mapping performer parameters to synthesis engines","volume":"7","author":"Hunt","year":"2003","journal-title":"Organised Sound"},{"key":"ref_18","unstructured":"Gillian, N.E. (2021, February 02). Gesture Recognition for Musician Computer Interaction Doctoral dissertation, Faculty of Arts, Humanities and Social Sciences, Queen\u2019s University, Belfast, Ireland. Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.725.1727&rep=rep1&type=pdf."},{"key":"ref_19","unstructured":"Cont, A., Coduys, T., and Henry, C. (2004, January 3\u20135). Real-time gesture mapping in PD environment using neural networks. Proceedings of the 2004 Conference on New Interfaces for Musical Expression, Hamamatsu, Japan."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Michon, R., Smith, J.O., Wright, M., Chafe, C., Granzow, J., and Wang, G. (2017). Mobile music, sensors, physical modeling, and digital fabrication: Articulating the augmented mobile instrument. Appl. Sci., 7.","DOI":"10.3390\/app7121311"},{"key":"ref_21","unstructured":"(2021, February 02). INFOGRAPHIC: How Computing Power Has Changed over Time. Available online: https:\/\/www.businessinsider.com\/infographic-how-computing-power-has-changed-over-time-2017-11."},{"key":"ref_22","unstructured":"(2021, February 19). Mobile CPUs Are Now as Fast as Most Desktop PCs. Available online: https:\/\/www.howtogeek.com\/393139\/mobile-cpus-are-now-as-fast-as-your-desktop-pc\/."},{"key":"ref_23","unstructured":"Jord\u00e0, S. (2004, January 3\u20135). Digital Instruments and Players: Part I\u2014Efficiency and Apprenticeship. Proceedings of the 2004 Conference on New Interfaces for Musical Expression, Hamamatsu, Japan."},{"key":"ref_24","unstructured":"(2021, February 19). This App Blows! (Ocarina 2 Launches Today)\u2014Smule. Available online: https:\/\/blog.smule.com\/this-app-blows-ocarina-2-launches-today."},{"key":"ref_25","unstructured":"(2021, February 19). The Motion Synth: Turn Movement into Music by AUUG. Available online: https:\/\/www.kickstarter.com\/projects\/1892750571\/the-motion-synth-turn-movement-into-music."},{"key":"ref_26","unstructured":"(2021, February 19). 9 Artists Pioneering the Art of the Musical Mobile App. Available online: https:\/\/flypaper.soundfly.com\/discovery\/artists-pioneering-musical-mobile-apps\/."},{"key":"ref_27","unstructured":"(2021, February 02). Software by Miller Puckette. Available online: http:\/\/msp.ucsd.edu\/software.html."},{"key":"ref_28","unstructured":"Cifter, a.S., and Dong, H. (2009, January 8\u201310). User characteristics: Professional vs. lay users. Proceedings of the fifth international conference on inclusive design, London, United Kingdom."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1177\/0956797616645672","article-title":"Marginally Significant Effects as Evidence for Hypotheses: Changing Attitudes Over Four Decades","volume":"27","author":"Pritschet","year":"2016","journal-title":"Psychol. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1177\/0956797619830326","article-title":"The Prevalence of Marginally Significant Results in Psychology over Time","volume":"30","author":"Hartgerink","year":"2019","journal-title":"Psychol. Sci."},{"key":"ref_31","unstructured":"(2021, February 02). Summary of MIDI 1.0 Messages. Available online: https:\/\/www.midi.org\/specifications-old\/item\/table-1-summary-of-midi-message."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hwang, S., Bianchi, A., and Wohn, K.Y. (2013, January 27\u201330). VibPress: Estimating pressure input using vibration absorption on mobile devices. Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, Munich, Germany.","DOI":"10.1145\/2493190.2493193"},{"key":"ref_33","unstructured":"Winter, A.E., Cox, B.R., Ginn, L.K.E., Whitt, D.O., Fitz-Coy, A.A., Picciotto, C.E., Yun, G.G., and Nelson, J.J. (2021, February 02). Input Device Haptics and Pressure Sensing, United States of America, Patent No.: US 9448631 B2, Available online: https:\/\/patentcenter.uspto.gov\/#!\/applications\/14698318."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/MPRV.2016.70","article-title":"ForcePhone: Software Lets Smartphones Sense Touch Force","volume":"15","author":"Tung","year":"2016","journal-title":"IEEE Pervasive Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"22","DOI":"10.3758\/s13428-020-01404-5","article-title":"Smartphone sensor accuracy varies from device to device in mobile research: The case of spatial orientation","volume":"53","author":"Kuhlmann","year":"2020","journal-title":"Behav. Res. Methods"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/7\/32\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:25:11Z","timestamp":1760163911000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/7\/32"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,27]]},"references-count":35,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["mti5070032"],"URL":"https:\/\/doi.org\/10.3390\/mti5070032","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2021,6,27]]}}}