{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:41:58Z","timestamp":1740123718379,"version":"3.37.3"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,8,24]],"date-time":"2020-08-24T00:00:00Z","timestamp":1598227200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,8,24]],"date-time":"2020-08-24T00:00:00Z","timestamp":1598227200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Institutional Strategy of the University of T\u00fcbingen","award":["Deutsche Forschungsgemeinschaft, ZUK 63"],"award-info":[{"award-number":["Deutsche Forschungsgemeinschaft, ZUK 63"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["User Model User-Adap Inter"],"published-print":{"date-parts":[[2021,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Pervasive computing environments deliver a multitude of possibilities for human\u2013computer interactions. Modern technologies, such as gesture control or speech recognition, allow different devices to be controlled without additional hardware. A drawback of these concepts is that gestures and commands need to be learned. We propose a system that is able to learn actions by observation of the user. To accomplish this, we use a camera and deep learning algorithms in a self-supervised fashion. The user can either train the system directly by showing gestures examples and perform an action, or let the system learn by itself. To evaluate the system, five experiments are carried out. In the first experiment, initial detectors are trained and used to evaluate our training procedure. The following three experiments are used to evaluate the adaption of our system and the applicability to new environments. In the last experiment, the online adaption is evaluated as well as adaption times and intervals are shown.<\/jats:p>","DOI":"10.1007\/s11257-020-09275-3","type":"journal-article","created":{"date-parts":[[2020,8,24]],"date-time":"2020-08-24T14:03:01Z","timestamp":1598277781000},"page":"105-120","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["From perception to action using observed actions to learn gestures"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7128-298X","authenticated-orcid":false,"given":"Wolfgang","family":"Fuhl","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,8,24]]},"reference":[{"key":"9275_CR1","doi-asserted-by":"crossref","unstructured":"Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the Twenty-First International Conference on Machine Learning. ACM, p. 1 (2004)","DOI":"10.1145\/1015330.1015430"},{"issue":"5","key":"9275_CR2","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1109\/LED.2005.846589","volume":"26","author":"AT Alastalo","year":"2005","unstructured":"Alastalo, A.T., Kaajakari, V.: Intermodulation in capacitively coupled microelectromechanical filters. IEEE Electron Device Lett. 26(5), 289\u2013291 (2005)","journal-title":"IEEE Electron Device Lett."},{"key":"9275_CR3","doi-asserted-by":"crossref","unstructured":"Arce, F., Valdez, J.M.G.: Accelerometer-based hand gesture recognition using artificial neural networks. In: Soft Computing for Intelligent Control and Mobile Robotics. Springer, pp. 67\u201377 (2010)","DOI":"10.1007\/978-3-642-15534-5_5"},{"key":"9275_CR4","volume-title":"Educational Psychology: A Cognitive View","author":"DP Ausubel","year":"1968","unstructured":"Ausubel, D.P., Novak, J.D., Hanesian, H., et al.: Educational Psychology: A Cognitive View, vol. 6. Holt, Rinehart and Winston, New York (1968)"},{"key":"9275_CR5","doi-asserted-by":"crossref","unstructured":"Basanta, H., Huang, Y.P., Lee, T.T.: Using voice and gesture to control living space for the elderly people. In: 2017 International Conference on System Science and Engineering (ICSSE). IEEE, pp. 20\u201323 (2017)","DOI":"10.1109\/ICSSE.2017.8030829"},{"key":"9275_CR6","volume-title":"Human Characteristics and School Learning","author":"BS Bloom","year":"1976","unstructured":"Bloom, B.S.: Human Characteristics and School Learning. McGraw-Hill, New York (1976)"},{"key":"9275_CR7","doi-asserted-by":"crossref","unstructured":"Chen, Q., Georganas, N.D., Petriu, E.M., et al.: Real-time vision-based hand gesture recognition using haar-like features. In: Instrumentation and Measurement Technology Conference Proceedings. Citeseer, pp. 1\u20136 (2007)","DOI":"10.1109\/IMTC.2007.379068"},{"key":"9275_CR8","unstructured":"Corera, S., Krishnarajah, N.: Capturing hand gesture movement: a survey on tools, techniques and logical considerations. Proceedings of chi sparks (2011)"},{"issue":"11","key":"9275_CR9","doi-asserted-by":"publisher","first-page":"3592","DOI":"10.1109\/TIM.2011.2161140","volume":"60","author":"NH Dardas","year":"2011","unstructured":"Dardas, N.H., Georganas, N.D.: Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques. IEEE Trans. Instrum. Meas. 60(11), 3592\u20133607 (2011)","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"9275_CR10","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Jia, L., Li, K., Fei-fei, L.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2009a)","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"9275_CR11","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Jia, L., Li, K., Fei-fei, L.: Imagenet: a large-scale hierarchical image database. In: In CVPR (2009b)","DOI":"10.1109\/CVPR.2009.5206848"},{"issue":"12","key":"9275_CR12","first-page":"1","volume":"2","author":"DSK Dixit","year":"2012","unstructured":"Dixit, D.S.K., Shingi, M.N.S.: Implementation of flex sensor and electronic compass for hand gesture based wireless automation of material handling robot. Int. J. Sc. Res. Publ. 2(12), 1 (2012)","journal-title":"Int. J. Sc. Res. Publ."},{"key":"9275_CR13","doi-asserted-by":"crossref","unstructured":"Dragan, A.D., Srinivasa, S.S.: Online customization of teleoperation interfaces. In: RO-MAN, 2012 IEEE, IEEE, pp. 919\u2013924 (2012)","DOI":"10.1109\/ROMAN.2012.6343868"},{"key":"9275_CR14","doi-asserted-by":"crossref","unstructured":"Francke, H., Ruiz-del Solar, J., Verschae, R.: Real-time hand gesture detection and recognition using boosted classifiers and active learning. In: Pacific-Rim Symposium on Image and Video Technology. Springer, pp. 533\u2013547 (2007)","DOI":"10.1007\/978-3-540-77129-6_47"},{"key":"9275_CR15","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1007\/s00138-016-0776-4","volume":"27","author":"W Fuhl","year":"2016","unstructured":"Fuhl, W., Tonsen, M., Bulling, A., Kasneci, E.: Pupil detection in the wild: an evaluation of the state of the art in mobile head-mounted eye tracking. Mach. Vis. Appl. 27, 1275\u20131288 (2016)","journal-title":"Mach. Vis. Appl."},{"key":"9275_CR16","doi-asserted-by":"crossref","unstructured":"Fuhl, W., Santini, T., Kasneci, E.: Fast and robust eyelid outline and aperture detection in real-world scenarios. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp. 1089\u20131097 (2017a)","DOI":"10.1109\/WACV.2017.126"},{"key":"9275_CR17","unstructured":"Fuhl, W., Santini, T., Kasneci, E.: Fast camera focus estimation for gaze-based focus control. (2017b). arXiv preprint arXiv:171103306"},{"key":"9275_CR18","doi-asserted-by":"crossref","unstructured":"Fuhl, W., Castner, N., Zhuang, L., Holzer, M., Rosenstiel, W., Kasneci, E.: Mam: Transfer learning for fully automatic video annotation and specialized detector creation. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018a)","DOI":"10.1007\/978-3-030-11021-5_23"},{"key":"9275_CR19","doi-asserted-by":"crossref","unstructured":"Fuhl, W., Eivazi, S., Hosp, B., Eivazi, A., Rosenstiel, W., Kasneci, E.: Bore: boosted-oriented edge optimization for robust, real time remote pupil center detection. In: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, pp. 1\u20135 (2018b)","DOI":"10.1145\/3204493.3204558"},{"key":"9275_CR20","doi-asserted-by":"crossref","unstructured":"Gidaris, S., Komodakis, N.: Dynamic few-shot visual learning without forgetting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4367\u20134375 (2018)","DOI":"10.1109\/CVPR.2018.00459"},{"key":"9275_CR21","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770\u2013778 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"issue":"5","key":"9275_CR22","doi-asserted-by":"publisher","first-page":"1285","DOI":"10.1109\/TMI.2016.2528162","volume":"35","author":"S Hoo-Chang","year":"2016","unstructured":"Hoo-Chang, S., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M.: Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285 (2016)","journal-title":"IEEE Trans. Med. Imaging"},{"issue":"2","key":"9275_CR23","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1145\/3054912","volume":"50","author":"A Hussein","year":"2017","unstructured":"Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. (CSUR) 50(2), 21 (2017)","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"9275_CR24","unstructured":"Ijspeert, AJ., Nakanishi, J., Schaal, S.: Movement imitation with nonlinear dynamical systems in humanoid robots. In: Proceedings of IEEE International Conference on Robotics and Automation, 2002. ICRA\u201902, vol 2. IEEE, pp. 1398\u20131403 (2002)"},{"key":"9275_CR25","unstructured":"Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097\u20131105 (2012)"},{"key":"9275_CR26","doi-asserted-by":"crossref","unstructured":"Lamberti, L., Camastra, F.: Real-time hand gesture recognition using a color glove. In: International Conference on Image Analysis and Processing. Springer, pp. 365\u2013373 (2011)","DOI":"10.1007\/978-3-642-24085-0_38"},{"issue":"11","key":"9275_CR27","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278\u20132324 (1998)","journal-title":"Proc. IEEE"},{"issue":"7553","key":"9275_CR28","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)","journal-title":"Nature"},{"key":"9275_CR29","unstructured":"Li, Z., Jarvis, R.: Real time hand gesture recognition using a range camera. In: Australasian Conference on Robotics and Automation, pp. 21\u201327 (2009)"},{"key":"9275_CR30","doi-asserted-by":"crossref","unstructured":"Li, Y., Cao, Z., Wang, J.: Gazture: design and implementation of a gaze based gesture control system on tablets. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 3, p. 74 (2017)","DOI":"10.1145\/3130939"},{"key":"9275_CR31","doi-asserted-by":"crossref","unstructured":"Liu, Y., Gupta, A., Abbeel, P., Levine, S.: Imitation from observation: learning to imitate behaviors from raw video via context translation. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp. 1118\u20131125 (2018)","DOI":"10.1109\/ICRA.2018.8462901"},{"key":"9275_CR32","doi-asserted-by":"crossref","unstructured":"Nielsen, M., St\u00f6rring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: International Gesture Workshop. Springer, pp. 409\u2013420 (2003)","DOI":"10.1007\/978-3-540-24598-8_38"},{"key":"9275_CR33","doi-asserted-by":"crossref","unstructured":"Pandit, A., Dand, D., Mehta, S., Sabesan, S., Daftery, A.: A simple wearable hand gesture recognition device using imems. In: International Conference of Soft Computing and Pattern Recognition, 2009. SOCPAR\u201909. IEEE, pp. 592\u2013597 (2009)","DOI":"10.1109\/SoCPaR.2009.117"},{"issue":"5","key":"9275_CR34","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/s11257-010-9082-4","volume":"20","author":"A Paramythis","year":"2010","unstructured":"Paramythis, A., Weibelzahl, S., Masthoff, J.: Layered evaluation of interactive adaptive systems: framework and formative methods. User Model. User Adap. Int. 20(5), 383\u2013453 (2010)","journal-title":"User Model. User Adap. Int."},{"issue":"1","key":"9275_CR35","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1162\/neco.1991.3.1.88","volume":"3","author":"DA Pomerleau","year":"1991","unstructured":"Pomerleau, D.A.: Efficient training of artificial neural networks for autonomous navigation. Neural Comput. 3(1), 88\u201397 (1991)","journal-title":"Neural Comput."},{"issue":"6","key":"9275_CR36","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1177\/004005990603800604","volume":"38","author":"SM Rao","year":"2006","unstructured":"Rao, S.M., Gagie, B.: Learning through seeing and doing: visual supports for children with autism. Teach. Except. Child. 38(6), 26\u201333 (2006)","journal-title":"Teach. Except. Child."},{"key":"9275_CR37","unstructured":"Ross, S., Gordon, G., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 627\u2013635 (2011)"},{"issue":"6","key":"9275_CR38","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1016\/S1364-6613(99)01327-3","volume":"3","author":"S Schaal","year":"1999","unstructured":"Schaal, S.: Is imitation learning the route to humanoid robots? Trends Cogn. Sci. 3(6), 233\u2013242 (1999)","journal-title":"Trends Cogn. Sci."},{"issue":"2","key":"9275_CR39","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1109\/TNSRE.2002.1031981","volume":"10","author":"RC Simpson","year":"2002","unstructured":"Simpson, R.C., Levine, S.P.: Voice control of a powered wheelchair. IEEE Trans. Neural Syst. Rehabil. Eng. 10(2), 122\u2013125 (2002)","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"9275_CR40","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818\u20132826 (2016)","DOI":"10.1109\/CVPR.2016.308"},{"issue":"3","key":"9275_CR41","first-page":"63","volume":"28","author":"RY Wang","year":"2009","unstructured":"Wang, R.Y., Popovi\u0107, J.: Real-time hand-tracking with a color glove. ACM TOG 28(3), 63 (2009)","journal-title":"ACM TOG"},{"issue":"1","key":"9275_CR42","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1109\/3468.553220","volume":"27","author":"J Yang","year":"1997","unstructured":"Yang, J., Xu, Y., Chen, C.S.: Human action learning via hidden Markov model. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 27(1), 34\u201344 (1997)","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"9275_CR43","unstructured":"Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320\u20133328 (2014)"}],"container-title":["User Modeling and User-Adapted Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11257-020-09275-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11257-020-09275-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11257-020-09275-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,23]],"date-time":"2021-08-23T23:24:37Z","timestamp":1629761077000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11257-020-09275-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,24]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,3]]}},"alternative-id":["9275"],"URL":"https:\/\/doi.org\/10.1007\/s11257-020-09275-3","relation":{},"ISSN":["0924-1868","1573-1391"],"issn-type":[{"type":"print","value":"0924-1868"},{"type":"electronic","value":"1573-1391"}],"subject":[],"published":{"date-parts":[[2020,8,24]]},"assertion":[{"value":"17 October 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 August 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 August 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}