{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T05:19:53Z","timestamp":1773465593772,"version":"3.50.1"},"reference-count":41,"publisher":"MIT Press","issue":"1","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We present research for automatic assessment of pianist hand posture that is intended to help beginning piano students improve their piano-playing technique during practice sessions. To automatically assess a student's hand posture, we propose a system that is able to recognize three categories of postures from a single depth map containing a pianist's hands during performance. This is achieved through a computer vision pipeline that uses machine learning on the depth maps for both hand segmentation and detection of hand posture. First, we segment the left and right hands from the scene captured in the depth map using per-pixel classification. To train the hand-segmentation models, we experiment with two feature descriptors, depth image features and depth context features, that describe the context of individual pixels' neighborhoods. After the hands have been segmented from the depth map, a posture-detection model classifies each hand as one of three possible posture categories: correct posture, low wrists, or flat hands. Two methods are tested for extracting descriptors from the segmented hands, histograms of oriented gradients and histograms of normal vectors. To account for variation in hand size and practice space, detection models are individually built for each student using support vector machines with the extracted descriptors. We validate this approach using a data set that was collected by recording four beginning piano students while performing standard practice exercises. The results presented in this article show the effectiveness of this approach, with depth context features and histograms of normal vectors performing the best.<\/jats:p>","DOI":"10.1162\/comj_a_00500","type":"journal-article","created":{"date-parts":[[2020,1,7]],"date-time":"2020-01-07T20:47:43Z","timestamp":1578430063000},"page":"59-78","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":30,"title":["Detecting Hand Posture in Piano Playing Using Depth Data"],"prefix":"10.1162","volume":"43","author":[{"given":"David","family":"Johnson","sequence":"first","affiliation":[{"name":"Department of Computer Science University of Victoria 3800 Finnerty Road Engineering and Computer Science Building, Room 504 Victoria, BC V8P 5C2 Canada davidjo@uvic.ca"}]},{"given":"Daniela","family":"Damian","sequence":"additional","affiliation":[{"name":"Department of Computer Science University of Victoria 3800 Finnerty Road Engineering and Computer Science Building, Room 504 Victoria, BC V8P 5C2 Canada danielad@uvic.ca"}]},{"given":"George","family":"Tzanetakis","sequence":"additional","affiliation":[{"name":"Department of Computer Science University of Victoria 3800 Finnerty Road Engineering and Computer Science Building, Room 504 Victoria, BC V8P 5C2 Canada gtzan@ieee.org"}]}],"member":"281","published-online":{"date-parts":[[2019,1,1]]},"reference":[{"key":"p_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"p_3","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"p_5","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2017.2703829"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"p_7","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1145\/3172944.3172950","author":"Chen N.","year":"2018","journal-title":"Proceedings of the International Conference on Intelligent User Interfaces"},{"key":"p_8","first-page":"886","author":"Dalal N.","year":"2005","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"p_10","doi-asserted-by":"publisher","DOI":"10.1080\/09298219008570563"},{"key":"p_11","first-page":"143","author":"Dannenberg R. B.","year":"1993","journal-title":"Proceedings of the Biennial Arts and Technology Symposium"},{"key":"p_12","first-page":"248","author":"Deng J.","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"p_13","first-page":"384","author":"Ferguson S.","year":"2006","journal-title":"Proceedings of the International Conference on New Interfaces for Musical Expression"},{"key":"p_14","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2019.00334"},{"key":"p_15","first-page":"303","author":"Hadjakos A.","year":"2012","journal-title":"Proceedings of the Sound and Music Computing Conference"},{"key":"p_16","first-page":"321","author":"Hadjakos A.","year":"2009","journal-title":"Proceedings of the Sound and Music Computing Conference"},{"key":"p_17","first-page":"878","author":"Han H.","year":"2005","journal-title":"Springer"},{"key":"p_18","first-page":"1322","author":"He H.","year":"2008","journal-title":"Proceedings of the IEEE International Joint Conference on Neural Networks"},{"key":"p_19","doi-asserted-by":"publisher","DOI":"10.1007\/s40708-016-0042-6"},{"key":"p_20","first-page":"151","author":"Johnson D.","year":"2017","journal-title":"Proceedings of the Conference on New Interfaces for Musical Expression"},{"key":"p_21","first-page":"168","author":"Johnson D.","year":"2016","journal-title":"Proceedings of the International Computer Music Conference"},{"key":"p_22","first-page":"259","author":"Kang B.","year":"2016","journal-title":"Proceedings of the IEEE Global Conference on Signal and Information Processing"},{"key":"p_23","first-page":"119","author":"Keskin C.","year":"2013","journal-title":"Springer"},{"key":"p_24","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980382"},{"key":"p_25","first-page":"762","author":"Li M.","year":"2014","journal-title":"Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society"},{"key":"p_26","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2014.2306177"},{"key":"p_27","doi-asserted-by":"publisher","DOI":"10.1145\/2856400.2856411"},{"key":"p_28","doi-asserted-by":"publisher","DOI":"10.1145\/1459359.1459552"},{"key":"p_30","doi-asserted-by":"publisher","DOI":"10.1109\/HAVE.2006.283791"},{"key":"p_31","first-page":"225","author":"Ng K.","year":"2008","journal-title":"Proceedings of the International Conference on New Interfaces for Musical Expression"},{"key":"p_32","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1145\/1322192.1322252","author":"Ng K. C.","year":"2007","journal-title":"Proceedings of the International Conference on Multimodal Interfaces"},{"key":"p_33","doi-asserted-by":"publisher","DOI":"10.1504\/IJKESDP.2011.039875"},{"key":"p_34","first-page":"1","author":"Oka A.","year":"2013","journal-title":"Proceedings of the Korea-Japan Joint Workshop on Frontiers of Computer Vision"},{"key":"p_35","first-page":"2825","volume":"12","author":"Pedregosa F.","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"p_36","doi-asserted-by":"publisher","DOI":"10.1145\/1290144.1290156"},{"key":"p_37","first-page":"383","author":"Raptis S.","year":"2005","journal-title":"Proceedings of the International Computer Music Conference"},{"issue":"2","key":"p_38","doi-asserted-by":"crossref","first-page":"82","DOI":"10.21091\/mppa.2005.2016","volume":"20","author":"Riley K.","year":"2005","journal-title":"Medical Problems of Performing Artists"},{"key":"p_39","first-page":"159","author":"Salgian A.","year":"2016","journal-title":"Proceedings of the International Computer Music Conference"},{"key":"p_40","first-page":"97","author":"Schoonderwaldt E.","year":"2005","journal-title":"Proceedings of the International Computer Music Conference"},{"key":"p_42","first-page":"1297","author":"Shotton J.","year":"2011","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"p_43","first-page":"3838","author":"Spinello L.","year":"2011","journal-title":"Proceedings of the International Conference on Intelligent Robots and Systems"},{"key":"p_44","first-page":"525","author":"Tang S.","year":"2013","journal-title":"Springer"},{"key":"p_45","first-page":"102","author":"Tits M.","year":"2015","journal-title":"Proceedings of the International Computer Music Conference"},{"key":"p_46","doi-asserted-by":"publisher","DOI":"10.1145\/2629500"}],"container-title":["Computer Music Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/comj\/article-pdf\/43\/1\/59\/2005111\/comj_a_00500.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/comj\/article-pdf\/43\/1\/59\/2005111\/comj_a_00500.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,10]],"date-time":"2022-10-10T06:33:42Z","timestamp":1665383622000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/comj\/article\/43\/1\/59\/94631\/Detecting-Hand-Posture-in-Piano-Playing-Using"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,1]]},"references-count":41,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,1,1]]},"published-print":{"date-parts":[[2019,1,1]]}},"URL":"https:\/\/doi.org\/10.1162\/comj_a_00500","relation":{},"ISSN":["0148-9267","1531-5169"],"issn-type":[{"value":"0148-9267","type":"print"},{"value":"1531-5169","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019]]},"published":{"date-parts":[[2019,1,1]]}}}