{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T19:27:10Z","timestamp":1762457230020,"version":"3.41.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"1s","license":[{"start":{"date-parts":[[2014,10,1]],"date-time":"2014-10-01T00:00:00Z","timestamp":1412121600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2014,10]]},"abstract":"<jats:p>We present a hand-and-foot-based multimodal interaction approach for handheld devices. Our method combines input modalities (i.e., hand and foot) and provides a coordinated output to both modalities along with audio and video. Human foot gesture is detected and tracked using contour-based template detection (CTD) and Tracking-Learning-Detection (TLD) algorithm. 3D foot pose is estimated from passive homography matrix of the camera. 3D stereoscopic and vibrotactile are used to enhance the immersive feeling. We developed a multimodal football game based on the multimodal approach as a proof-of-concept. We confirm our systems user satisfaction through a user study.<\/jats:p>","DOI":"10.1145\/2645860","type":"journal-article","created":{"date-parts":[[2014,10,1]],"date-time":"2014-10-01T13:34:59Z","timestamp":1412170499000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":85,"title":["Multimodal Hand and Foot Gesture Interaction for Handheld Devices"],"prefix":"10.1145","volume":"11","author":[{"given":"Zhihan","family":"Lv","sequence":"first","affiliation":[{"name":"Chinese Academy of Sciences and Ume\u00e5 University, China"}]},{"given":"Alaa","family":"Halawani","sequence":"additional","affiliation":[{"name":"Ume\u00e5 University and Palestine Polytechnic University, Sweden"}]},{"given":"Shengzhong","family":"Feng","sequence":"additional","affiliation":[{"name":"Chinese Academy of Sciences, China"}]},{"given":"Haibo","family":"Li","sequence":"additional","affiliation":[{"name":"Royal Institute of Technology, Stockholm, Sweden"}]},{"given":"Shafiq Ur","family":"R\u00e9hman","sequence":"additional","affiliation":[{"name":"Ume\u00e5 University, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2014,10]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501643.2501649"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208575"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2009.44"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1785455.1785465"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866064"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208576"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449746"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1477862.1477911"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/2146303.2146366"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.135"},{"volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.","author":"Grauman Kristen","key":"e_1_2_1_11_1","unstructured":"Kristen Grauman , Margrit Betke , James Gips , and Gary R. Bradski . 2001. Communication via eye blinks - detection and duration analysis in real time . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Kristen Grauman, Margrit Betke, James Gips, and Gary R. Bradski. 2001. Communication via eye blinks - detection and duration analysis in real time. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2009.5336470"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348816.2348826"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.241"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2541016.2541059"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11370-011-0098-3"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2037373.2037379"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/21.44063"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493988.2494328"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1622176.1622199"},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"R. I. Hartley and A. Zisserman. 2004. Multiple View Geometry in Computer Vision (2nd Ed.). Cambridge University Press ISBN: 0521540518.   R. I. Hartley and A. Zisserman. 2004. Multiple View Geometry in Computer Vision (2nd Ed.). Cambridge University Press ISBN: 0521540518.","DOI":"10.1017\/CBO9780511811685"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2006.10.019"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0924-2716(00)00009-5"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.239"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 7th International Workshop on Networking Issues in Multimedia Entertainment (NIME'11)","author":"Kondapalli Ravi","year":"2011","unstructured":"Ravi Kondapalli and Ben-Zhen Sung . 2011 . Daft datum\u2014An interface for producing music through foot-based interaction . In Proceedings of the 7th International Workshop on Networking Issues in Multimedia Entertainment (NIME'11) . Ravi Kondapalli and Ben-Zhen Sung. 2011. Daft datum\u2014An interface for producing music through foot-based interaction. In Proceedings of the 7th International Workshop on Networking Issues in Multimedia Entertainment (NIME'11)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502163"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICVRV.2013.59"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2542302.2542336"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 7th International Joint Conference on Artificial Intelligence -","volume":"2","author":"Bruce","unstructured":"Bruce D. Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision . In Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2 (IJCAI'81). Morgan Kaufmann Publishers Inc., San Francisco, CA, 674--679. http:\/\/dl.acm.org\/citation.cfm&quest;id=1623264.1623280 Bruce D. Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2 (IJCAI'81). Morgan Kaufmann Publishers Inc., San Francisco, CA, 674--679. http:\/\/dl.acm.org\/citation.cfm&quest;id=1623264.1623280"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.64"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2559206.2580096"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2541831.2541833"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2543651.2543677"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1667146.1667160"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1520340.1520626"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1067343.1067390"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2006.06.006"},{"volume-title":"Proceedings of the ACM International Workshop MobiVis Workshop at MobileHCI.","author":"R\u00e9hman S.","key":"e_1_2_1_39_1","unstructured":"S. R\u00e9hman , A. Khan , and H. Li . 2012. Interactive feet for mobile immersive interaction . In Proceedings of the ACM International Workshop MobiVis Workshop at MobileHCI. S. R\u00e9hman, A. Khan, and H. Li. 2012. Interactive feet for mobile immersive interaction. In Proceedings of the ACM International Workshop MobiVis Workshop at MobileHCI."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2350046.2350053"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866063"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2012.6343750"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/1545968309341647"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321261.1321266"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CRV.2013.51"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/SITIS.2007.102"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2008.2001352"},{"volume-title":"Proceedings of the International Conference on Virtual Reality.","author":"Valkov D.","key":"e_1_2_1_48_1","unstructured":"D. Valkov , F. Steinicke , G. Bruder , and K. Hinrichs . 2010. Traveling in 3D virtual environments with foot gestures and a multi-touch enabled WIM . In Proceedings of the International Conference on Virtual Reality. D. Valkov, F. Steinicke, G. Bruder, and K. Hinrichs. 2010. Traveling in 3D virtual environments with foot gestures and a multi-touch enabled WIM. In Proceedings of the International Conference on Virtual Reality."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2008.4637338"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.99"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1166253.1166270"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.2307\/3001968"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.5555\/2044575.2044654"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2013.02.004"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2645860","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2645860","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T07:19:11Z","timestamp":1750231151000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2645860"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,10]]},"references-count":54,"journal-issue":{"issue":"1s","published-print":{"date-parts":[[2014,10]]}},"alternative-id":["10.1145\/2645860"],"URL":"https:\/\/doi.org\/10.1145\/2645860","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2014,10]]},"assertion":[{"value":"2013-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-10-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}