{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T04:12:57Z","timestamp":1773288777260,"version":"3.50.1"},"reference-count":63,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T00:00:00Z","timestamp":1607299200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["2063473"],"award-info":[{"award-number":["2063473"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Interactive music uses wearable sensors (i.e., gestural interfaces\u2014GIs) and biometric datasets to reinvent traditional human\u2013computer interaction and enhance music composition. In recent years, machine learning (ML) has been important for the artform. This is because ML helps process complex biometric datasets from GIs when predicting musical actions (termed performance gestures). ML allows musicians to create novel interactions with digital media. Wekinator is a popular ML software amongst artists, allowing users to train models through demonstration. It is built on the Waikato Environment for Knowledge Analysis (WEKA) framework, which is used to build supervised predictive models. Previous research has used biometric data from GIs to train specific ML models. However, previous research does not inform optimum ML model choice, within music, or compare model performance. Wekinator offers several ML models. Thus, we used Wekinator and the Myo armband GI and study three performance gestures for piano practice to solve this problem. Using these, we trained all models in Wekinator and investigated their accuracy, how gesture representation affects model accuracy and if optimisation can arise. Results show that neural networks are the strongest continuous classifiers, mapping behaviour differs amongst continuous models, optimisation can occur and gesture representation disparately affects model mapping behaviour; impacting music practice.<\/jats:p>","DOI":"10.3390\/e22121384","type":"journal-article","created":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T21:37:42Z","timestamp":1607377062000},"page":"1384","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["New Interfaces and Approaches to Machine Learning When Classifying Gestures within Music"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1899-8222","authenticated-orcid":false,"given":"Chris","family":"Rhodes","sequence":"first","affiliation":[{"name":"NOVARS Research Centre, University of Manchester, Manchester M13 9PL, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1236-3143","authenticated-orcid":false,"given":"Richard","family":"Allmendinger","sequence":"additional","affiliation":[{"name":"Alliance Manchester Business School, University of Manchester, Manchester M15 6PB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0484-8055","authenticated-orcid":false,"given":"Ricardo","family":"Climent","sequence":"additional","affiliation":[{"name":"NOVARS Research Centre, University of Manchester, Manchester M13 9PL, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dean, R.T. (2011). Sensor-based musical instruments and interactive music. The Oxford Handbook of Computer Music, Oxford University Press.","DOI":"10.1093\/oxfordhb\/9780199792030.001.0001"},{"key":"ref_2","unstructured":"Miranda, E.R., and Wanderley, M.M. (2006). New Digital Musical Instruments: Control and Interaction beyond the Keyboard (Computer Music and Digital Audio Series), AR Editions, Inc."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1177\/135485659700300405","article-title":"Danger! Retro-Affectivity! The cultural career of the theremin","volume":"3","author":"Hayward","year":"1997","journal-title":"Convergence"},{"key":"ref_4","unstructured":"Finamore, E. (2020, September 08). A Tribute to the Synth: How Synthesisers Revolutionised Modern Music. Available online: https:\/\/www.bbc.co.uk\/programmes\/articles\/3ryZCdlXtpkNG3yRl3Y7pnh\/a-tribute-to-the-synth-how-synthesisers-revolutionised-modern-music."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1162\/COMJ_a_00356","article-title":"The Hands: The making of a digital musical instrument","volume":"40","author":"Torre","year":"2016","journal-title":"Comput. Music J."},{"key":"ref_6","unstructured":"Rose, J. (2020, September 08). k-bow: The Palimpolin Project. Available online: http:\/\/www.jonroseweb.com\/e_vworld_k-bow.php."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"13","DOI":"10.2307\/3681041","article-title":"An interview with Sensorband","volume":"22","author":"Bongers","year":"1998","journal-title":"Comput. Music J."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wu, D., Li, C., Yin, Y., Zhou, C., and Yao, D. (2010). Music composition from the brain signal: Representing the mental state by music. Comput. Intell. Neurosci., 2010.","DOI":"10.1155\/2010\/267671"},{"key":"ref_9","unstructured":"Jackson, B. (2020, September 08). Thalmic Labs Myo armband hits consumer release, for sale on Amazon. Available online: https:\/\/www.itbusiness.ca\/news\/thalmic-labs-myo-armband-hits-consumer-release-for-sale-on-amazon\/54056."},{"key":"ref_10","unstructured":"Nymoen, K., Haugen, M.R., and Jensenius, A.R. (2015). Mumyo\u2013evaluating and exploring the myo armband for musical interaction. International Conference on New Interfaces For Musical Expression, NIME."},{"key":"ref_11","unstructured":"Fiebrink, R., Trueman, D., and Cook, P.R. (2009). A Meta-Instrument for Interactive, On-the-Fly Machine Learning. International Conference on New Interfaces For Musical Expression, NIME."},{"key":"ref_12","unstructured":"Schedel, M., and Fiebrink, R. (2011). A demonstration of bow articulation recognition with Wekinator and K-Bow. International Computer Music Conference, ICMC."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Santini, G. (2019). Synesthesizer: Physical modelling and machine learning for a color-based synthesizer in virtual reality. International Conference on Mathematics and Computation in Music, Springer.","DOI":"10.1007\/978-3-030-21392-3_18"},{"key":"ref_14","unstructured":"Hantrakul, L., and Kondak, Z. (2018). GestureRNN: A neural gesture system for the Roli Lightpad Block. New Interfaces for Musical Expression, NIME."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Fiebrink, R., Cook, P.R., and Trueman, D. (2011). Human model evaluation in interactive supervised learning. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Association for Computing Machinery. CHI \u201911.","DOI":"10.1145\/1978942.1978965"},{"key":"ref_16","unstructured":"Caramiaux, B., and Tanaka, A. (2013). Machine Learning of Musical Gestures: Principles and Review. International Conference on New Interfaces for Musical Expression, NIME."},{"key":"ref_17","unstructured":"Tanaka, A., Di Donato, B., Zbyszynski, M., and Roks, G. (2019). Designing gestures for continuous sonic interaction. International Conference on New Interfaces for Musical Expression, NIME."},{"key":"ref_18","unstructured":"North (2020, September 08). Myo Connect, SDK and Firmware Downloads. Available online: https:\/\/support.getmyo.com\/hc\/en-us\/articles\/360018409792-Myo-Connect-SDK-and-firmware-downloads."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Marin, G., Dominio, F., and Zanuttigh, P. (2014). Hand gesture recognition with leap motion and kinect devices. IEEE International Conference on Image Processing, IEEE.","DOI":"10.1109\/ICIP.2014.7025313"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kosmopoulos, D.I., Doulamis, A., and Doulamis, N. (2005). Gesture-based video summarization. IEEE International Conference on Image Processing, IEEE.","DOI":"10.1109\/ICIP.2005.1530618"},{"key":"ref_21","unstructured":"Trigueiros, P., Ribeiro, F., and Reis, L.P. (2012). A comparison of machine learning algorithms applied to hand gesture recognition. 7th Iberian Conference on Information Systems and Technologies, IEEE."},{"key":"ref_22","unstructured":"Han, S., Ho Yu, T., Zhang, P., Isaac, E., Liu, B., Cabezas, R., Vajda, P., and Wang, R. (2020, September 08). Using deep neural networks for accurate hand-tracking on Oculus Quest. Available online: https:\/\/ai.facebook.com\/blog\/hand-tracking-deep-neural-networks\/."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Benalc\u00e1zar, M.E., Motoche, C., Zea, J.A., Jaramillo, A.G., Anchundia, C.E., Zambrano, P., Segura, M., Palacios, F.B., and P\u00e9rez, M. (2017, January 16\u201320). Real-time hand gesture recognition using the Myo armband and muscle activity detection. Proceedings of the 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM), Salinas, Ecuador.","DOI":"10.1109\/ETCM.2017.8247458"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dalmazzo, D., and Ramirez, R. (2017). Air violin: A machine learning approach to fingering gesture recognition. Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, ACM Press.","DOI":"10.1145\/3139513.3139526"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ruan, X., Wang, Q., Liu, R., Fan, J., and Liu, J. (2018). Air-Ukulele: Finger Movement Detection Based on sEMG for Ukulele playing. Proceedings of the Sixth International Symposium of Chinese CHI, ACM.","DOI":"10.1145\/3202667.3202689"},{"key":"ref_26","unstructured":"Di Donato, B., Dooley, J., Hockman, J., Hall, S., and Bullock, J. (2017). MyoSpat: A hand-gesture controlled system for sound and light projections manipulation. International Computer Music Conference, ICMC."},{"key":"ref_27","unstructured":"Erdem, C., Martin, C., Lan, Q., T\u00f8rresen, J., Fuhrer, J., and Jensenius Refsum, A. (2020). Towards Playing in the \u2018Air\u2019: Modeling Motion\u2013Sound Energy Relationships in Electric Guitar Performance Using Deep Neural Networks. 17th Sound and Music Computing Conference, SMC."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"344","DOI":"10.3389\/fpsyg.2019.00344","article-title":"Bowing Gestures Classification in Violin Performance: A Machine Learning Approach","volume":"10","author":"Dalmazzo","year":"2019","journal-title":"Front. Psychol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Arief, Z., Sulistijono, I.A., and Ardiansyah, R.A. (2015). Comparison of five time series EMG features extractions using Myo Armband. International Electronics Symposium, IEEE.","DOI":"10.1109\/ELECSYM.2015.7380805"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1350016","DOI":"10.1142\/S0219477513500168","article-title":"EMG Amplitude Estimators Based on Probability Distribution for Muscle-Computer Interface","volume":"12","author":"Phinyomark","year":"2013","journal-title":"Fluct. Noise Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Phinyomark, A., Khushaba, R.N., and Scheme, E. (2018). Feature Extraction and Selection for Myoelectric Control Based on Wearable EMG Sensors. Sensors, 18.","DOI":"10.3390\/s18051615"},{"key":"ref_32","unstructured":"Fran\u00e7oise, J., Schnell, N., Borghesi, R., and Bevilacqua, F. (2014). Probabilistic models for designing motion and sound relationships. International Conference on New Interfaces For Musical Expression, NIME."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1109\/MMUL.2017.18","article-title":"A Method and Toolkit for Digital Musical Instruments: Generating Ideas and Prototypes","volume":"24","author":"Calegario","year":"2017","journal-title":"IEEE MultiMed."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"God\u00f8y, R.I., and Leman, M. (2010). Musical gestures: Concepts and methods in research. Musical Gestures: Sound, Movement, and Meaning, Routledge.","DOI":"10.4324\/9780203863411"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1145\/319382.319388","article-title":"Machine learning and data mining","volume":"42","author":"Mitchell","year":"1999","journal-title":"Commun. ACM"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2358","DOI":"10.1016\/j.arth.2018.02.067","article-title":"Artificial intelligence, machine learning, deep learning, and cognitive computing: What do these terms mean and how will they impact health care?","volume":"33","author":"Bini","year":"2018","journal-title":"J. Arthroplast."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zheng, J., and Dagnino, A. (2014). An initial study of predictive machine learning analytics on large volumes of historical data for power system applications. IEEE International Conference on Big Data, IEEE.","DOI":"10.1109\/BigData.2014.7004327"},{"key":"ref_38","unstructured":"North (2020, September 08). What is the Wireless Range of Myo?. Available online: https:\/\/support.getmyo.com\/hc\/en-us\/articles\/202668603-What-is-the-wireless-range-of-Myo-."},{"key":"ref_39","unstructured":"North (2020, September 08). Creating a Custom Calibration Profile for Your Myo Armband. Available online: https:\/\/support.getmyo.com\/hc\/en-us\/articles\/203829315-Creating-a-custom-calibration-profile-for-your-Myo-armband."},{"key":"ref_40","unstructured":"Tanaka, A. (2020, January 10). Sarah Nicolls performs Suspensions, by Atau Tanaka. Available online: https:\/\/www.youtube.com\/watch?v=HumBMPJUroQ."},{"key":"ref_41","unstructured":"EarMaster (2020, April 10). What are Intervals in Music?. Available online: https:\/\/www.earmaster.com\/wiki\/music-memos\/what-are-intervals-in-music.html."},{"key":"ref_42","unstructured":"Pianists for Alternatively Sized Keyboards (2020, June 10). Gender Differences. Available online: http:\/\/paskpiano.org\/hand-span-data\/."},{"key":"ref_43","unstructured":"RagtimeDorianHenry (2020, February 10). 10th Interval Rag by Harry Ruby (1923, Ragtime piano). Available online: https:\/\/www.youtube.com\/watch?v=L5hK-OF9Kx8."},{"key":"ref_44","unstructured":"Zadeh, A.S., Calitz, A., and Greyling, J. Evaluating a biosensor-based interface to recognize hand-finger gestures using a Myo armband. Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists."},{"key":"ref_45","unstructured":"Hanon-online.com (2020, February 10). Piano Technique Exercise N\u22181 in C. Available online: https:\/\/www.hanon-online.com\/the-virtuoso-pianist-part-i\/hanon-exercise-n-1\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"4058","DOI":"10.1016\/j.eswa.2010.09.068","article-title":"Feature extraction of forearm EMG signals for prosthetics","volume":"38","author":"Rafiee","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref_47","unstructured":"Berndt, D.J., and Clifford, J. (1994). Using Dynamic Time Warping to Find Patterns in Time Series. Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, AAAI Press. AAAIWS\u201994."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Sammut, C., and Webb, G.I. (2010). Decision Stump. Encyclopedia of Machine Learning, Springer US.","DOI":"10.1007\/978-0-387-30164-8"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Rhodes, C., Allmendinger, R., and Climent, R. (2019). New Interfaces for Classifying Performance Gestures in Music. International Conference on Intelligent Data Engineering and Automated Learning, IDEAL.","DOI":"10.1007\/978-3-030-33617-2_4"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1017\/S1355771802002030","article-title":"Mapping performer parameters to synthesis engines","volume":"7","author":"Hunt","year":"2002","journal-title":"Organised Sound"},{"key":"ref_51","unstructured":"Winkler, T. (2001). Composing Interactive Music: Techniques and Ideas Using Max, MIT Press."},{"key":"ref_52","unstructured":"Dobrian, C. (2020, September 08). ADSR Amplitude Envelope. Available online: https:\/\/music.arts.uci.edu\/dobrian\/maxcookbook\/adsr-amplitude-envelope."},{"key":"ref_53","unstructured":"Hermann, T. (2008). Taxonomy and definitions for sonification and auditory display. Proceedings of the 14th International Conference on Auditory Display, ICAD."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"61994","DOI":"10.1109\/ACCESS.2018.2872625","article-title":"Internet of musical things: Vision and challenges","volume":"6","author":"Turchet","year":"2018","journal-title":"IEEE Access"},{"key":"ref_55","unstructured":"Witten, I.H., Frank, E., and Hall, M.A. (2017). Data Mining: Practical Machine Learning Tools and Techniques, Elsevier. [4th ed.]."},{"key":"ref_56","unstructured":"Scurto, H., and Fiebrink, R. (2016). Grab-and-play mapping: Creative machine learning approaches for musical inclusion and exploration. International Computer Music Conference, ICMC."},{"key":"ref_57","first-page":"21","article-title":"Comparative analysis of classification algorithms on different datasets using WEKA","volume":"54","author":"Arora","year":"2012","journal-title":"Int. J. Comput. Appl."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/TSMC.1985.6313426","article-title":"A fuzzy k-nearest neighbor algorithm","volume":"SMC-15","author":"Keller","year":"1985","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1007\/s11294-007-9090-2","article-title":"Multiclass corporate failure prediction by Adaboost. M1","volume":"13","author":"Cortes","year":"2007","journal-title":"Int. Adv. Econ. Res."},{"key":"ref_60","unstructured":"Quinlan, J. (1992). C4.5: Programs for Machine Learning, Morgan Kaufmann."},{"key":"ref_61","first-page":"1114","article-title":"Decision tree analysis on j48 algorithm for data mining","volume":"3","author":"Bhargava","year":"2013","journal-title":"Int. J. Adv. Res. Comput. Sci. Softw. Eng."},{"key":"ref_62","first-page":"41","article-title":"An empirical study of the naive Bayes classifier","volume":"3","author":"Rish","year":"2001","journal-title":"IJCAI Workshop Empir. Methods Artif. Intell."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"1955","DOI":"10.1016\/j.asr.2007.07.020","article-title":"Comparison of decision tree methods for finding active objects","volume":"41","author":"Zhao","year":"2008","journal-title":"Adv. Space Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/12\/1384\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:42:07Z","timestamp":1760179327000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/12\/1384"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,7]]},"references-count":63,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["e22121384"],"URL":"https:\/\/doi.org\/10.3390\/e22121384","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,7]]}}}