{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T09:16:22Z","timestamp":1769159782535,"version":"3.49.0"},"reference-count":51,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,4,24]],"date-time":"2020-04-24T00:00:00Z","timestamp":1587686400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["RTI2018-093337-B-I00"],"award-info":[{"award-number":["RTI2018-093337-B-I00"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010198","name":"Ministerio de Econom\u00eda, Industria y Competitividad, Gobierno de Espa\u00f1a","doi-asserted-by":"publisher","award":["IT900-16"],"award-info":[{"award-number":["IT900-16"]}],"id":[{"id":"10.13039\/501100010198","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007065","name":"Nvidia","doi-asserted-by":"publisher","award":["GPU Grant program"],"award-info":[{"award-number":["GPU Grant program"]}],"id":[{"id":"10.13039\/100007065","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Action recognition in robotics is a research field that has gained momentum in recent years. In this work, a video activity recognition method is presented, which has the ultimate goal of endowing a robot with action recognition capabilities for a more natural social interaction. The application of Common Spatial Patterns (CSP), a signal processing approach widely used in electroencephalography (EEG), is presented in a novel manner to be used in activity recognition in videos taken by a humanoid robot. A sequence of skeleton data is considered as a multidimensional signal and filtered according to the CSP algorithm. Then, characteristics extracted from these filtered data are used as features for a classifier. A database with 46 individuals performing six different actions has been created to test the proposed method. The CSP-based method along with a Linear Discriminant Analysis (LDA) classifier has been compared to a Long Short-Term Memory (LSTM) neural network, showing that the former obtains similar or better results than the latter, while being simpler.<\/jats:p>","DOI":"10.3390\/s20082436","type":"journal-article","created":{"date-parts":[[2020,4,24]],"date-time":"2020-04-24T11:42:14Z","timestamp":1587728534000},"page":"2436","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Shedding Light on People Action Recognition in Social Robotics by Means of Common Spatial Patterns"],"prefix":"10.3390","volume":"20","author":[{"given":"Itsaso","family":"Rodr\u00edguez-Moreno","sequence":"first","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, University of the Basque Country, Manuel Lardizabal 1, 20018 Donostia-San Sebasti\u00e1n, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5015-1315","authenticated-orcid":false,"given":"Jos\u00e9 Mar\u00eda","family":"Mart\u00ednez-Otzeta","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, University of the Basque Country, Manuel Lardizabal 1, 20018 Donostia-San Sebasti\u00e1n, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Izaro","family":"Goienetxea","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, University of the Basque Country, Manuel Lardizabal 1, 20018 Donostia-San Sebasti\u00e1n, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1432-102X","authenticated-orcid":false,"given":"Igor","family":"Rodriguez-Rodriguez","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, University of the Basque Country, Manuel Lardizabal 1, 20018 Donostia-San Sebasti\u00e1n, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Basilio","family":"Sierra","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Artificial Intelligence, University of the Basque Country, Manuel Lardizabal 1, 20018 Donostia-San Sebasti\u00e1n, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Breazeal, C. (2004). Designing Sociable Robots, Intelligent Robotics and Autonomous Agents, MIT Press.","DOI":"10.7551\/mitpress\/2376.001.0001"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"88","DOI":"10.3390\/computers2020088","article-title":"A review on video-based human activity recognition","volume":"2","author":"Ke","year":"2013","journal-title":"Computers"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1007\/s00371-012-0752-6","article-title":"A survey on activity recognition and behavior understanding in video surveillance","volume":"29","author":"Vishwakarma","year":"2013","journal-title":"Vis. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"976","DOI":"10.1016\/j.imavis.2009.11.014","article-title":"A survey on vision-based human action recognition","volume":"28","author":"Poppe","year":"2010","journal-title":"Image Vis. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.imavis.2017.01.010","article-title":"Going deeper into action recognition: A survey","volume":"60","author":"Herath","year":"2017","journal-title":"Image Vis. Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, C.C., and Aggarwal, J. (2009, January 8\u20139). Recognizing human action from a far field of view. Proceedings of the 2009 Workshop on Motion and Video Computing (WMVC), Snowbird, UT, USA.","DOI":"10.1109\/WMVC.2009.5399231"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chaudhry, R., Ravichandran, A., Hager, G., and Vidal, R. (2009, January 20\u201325). Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206821"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Oreifej, O., and Liu, Z. (2013, January 23\u201328). Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.98"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1932","DOI":"10.1109\/TMM.2017.2786868","article-title":"Robust 3D action recognition through sampling local appearances and global distributions","volume":"20","author":"Liu","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_10","unstructured":"Simonyan, K., and Zisserman, A. (2014). Two-stream convolutional networks for action recognition in videos. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1435321","DOI":"10.1155\/2016\/1435321","article-title":"User adapted motor-imaginary brain-computer interface by means of EEG channel selection based on estimation of distributed algorithms","volume":"2016","author":"Astigarraga","year":"2016","journal-title":"Math. Probl. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., and Sheikh, Y. (2018). OpenPose: Realtime multi-person 2D pose estimation using Part Affinity Fields. arXiv.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1037\/h0071325","article-title":"Analysis of a complex of statistical variables into principal components","volume":"24","author":"Hotelling","year":"1933","journal-title":"J. Educ. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_15","unstructured":"Chollet, F. (2020, April 24). Keras. Available online: https:\/\/keras.io."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/34.910878","article-title":"The recognition of human movement using temporal templates","volume":"23","author":"Bobick","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bobick, A., and Davis, J. (1996, January 25\u201319). An appearance-based representation of action. Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria.","DOI":"10.1109\/ICPR.1996.546039"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Schuldt, C., Laptev, I., and Caputo, B. (2004, January 26). Recognizing human actions: A local SVM approach. Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1334462"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Niebles, J.C., and Fei-Fei, L. (2007, January 17\u201322). A hierarchical model of shape and appearance for human action classification. Proceedings of the Computer Vision and Pattern Recognition, CVPR\u201907, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383132"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Laptev, I., Marszalek, M., Schmid, C., and Rozenfeld, B. (2008, January 23\u201328). Learning realistic human actions from movies. Proceedings of the Computer Vision and Pattern Recognition, Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587756"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bosch, A., Zisserman, A., and Munoz, X. (2007, January 9\u201311). Representing shape with a spatial pyramid kernel. Proceedings of the 6th ACM International Conference on Image And Video Retrieval, Amsterdam, The Netherlands.","DOI":"10.1145\/1282280.1282340"},{"key":"ref_22","unstructured":"Lazebnik, S., Schmid, C., and Ponce, J. (2006, January 17\u201322). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Marsza\u0142ek, M., Schmid, C., Harzallah, H., and Van De Weijer, J. (2007, January 14\u201320). Learning object representations for visual object class recognition. Proceedings of the Visual Recognition Challange Workshop, in Conjunction with ICCV, Rio de Janeiro, Brazil.","DOI":"10.1109\/CVPR.2007.383272"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/s11263-006-9794-4","article-title":"Local features and kernels for classification of texture and object categories: A comprehensive study","volume":"73","author":"Zhang","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Efros, A.A., Berg, A.C., Mori, G., and Malik, J. (2003, January 13\u201316). Recognizing action at a distance. Proceedings of the Ninth International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238420"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tran, D., and Sorokin, A. (2008, January 12\u201318). Human activity recognition with metric learning. Proceedings of the European Conference on Computer Vision, Marseille, France.","DOI":"10.1007\/978-3-540-88682-2_42"},{"key":"ref_27","unstructured":"Ercis, F. (2012). Comparison of Histogram of Oriented Optical Flow Based Action Recognition Methods. [Ph.D. Thesis, Middle East Technical University]."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lertniphonphan, K., Aramvith, S., and Chalidabhongse, T.H. (2011, January 12\u201314). Human action recognition using direction histograms of optical flow. Proceedings of the Communications and Information Technologies (ISCIT), 2011 11th International Symposium on Communications & Information Technologies (ISCIT 2011), Hangzhou, China.","DOI":"10.1109\/ISCIT.2011.6089701"},{"key":"ref_29","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An Iterative Image Registration Technique With an Application To Stereo Vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_30","unstructured":"Akpinar, S., and Alpaslan, F.N. (2014, January 21\u201324). Video action recognition using an optical flow based representation. Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV), Las Vegas, NV, USA."},{"key":"ref_31","first-page":"1","article-title":"Action recognition using multi-directional projected depth motion maps","volume":"9","author":"Satyamurthi","year":"2018","journal-title":"J. Ambient. Intell. Humaniz. Comput."},{"key":"ref_32","unstructured":"Yang, X., Zhang, C., and Tian, Y. (November, January 29). Recognizing actions using depth motion maps-based histograms of oriented gradients. Proceedings of the 20th ACM International Conference on Multimedia, Nara, Japan."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Choutas, V., Weinzaepfel, P., Revaud, J., and Schmid, C. (2018, January 18\u201322). PoTion: Pose MoTion Representation for Action Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00734"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ren, J., Reyes, N.H., Barczak, A., Scogings, C., and Liu, M. (2018, January 27\u201329). An Investigation of Skeleton-Based Optical Flow-Guided Features for 3D Action Recognition Using a Multi-Stream CNN Model. Proceedings of the 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), Chongqing, China.","DOI":"10.1109\/ICIVC.2018.8492894"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei, L. (2014, January 23\u201328). Large-scale video classification with convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.223"},{"key":"ref_36","unstructured":"Wang, L., Xiong, Y., Wang, Z., and Qiao, Y. (2015). Towards good practices for very deep two-stream ConvNets. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wang, L., Qiao, Y., and Tang, X. (2015, January 7\u201312). Action recognition with trajectory-pooled deep-convolutional descriptors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299059"},{"key":"ref_38","unstructured":"Feichtenhofer, C., Pinz, A., and Zisserman, A. (July, January 26). Convolutional two-stream network fusion for video action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/ACCESS.2017.2778011","article-title":"Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features","volume":"6","author":"Ullah","year":"2018","journal-title":"IEEE Access"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1109\/T-C.1970.222918","article-title":"Application of the Karhunen-Lo\u00e8ve Expansion to Feature Selection and Ordering","volume":"100","author":"Fukunaga","year":"1970","journal-title":"IEEE Trans. Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1109\/86.895946","article-title":"Optimal spatial filtering of single trial EEG during imagined hand movement","volume":"8","author":"Ramoser","year":"2000","journal-title":"IEEE Trans. Rehabil. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wang, Y., Gao, S., and Gao, X. (2006, January 17\u201318). Common spatial pattern method for channel selection in motor imagery based brain-computer interface. Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China.","DOI":"10.1109\/IEMBS.2005.1615701"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Novi, Q., Guan, C., Dat, T.H., and Xue, P. (2007, January 2\u20135). Sub-band common spatial pattern (SBCSP) for brain-computer interface. Proceedings of the 2007 3rd International IEEE\/EMBS Conference on Neural Engineering, Kohala Coast, HI, USA.","DOI":"10.1109\/CNE.2007.369647"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1111\/j.1469-1809.1936.tb02137.x","article-title":"The use of multiple measurements in taxonomic problems","volume":"7","author":"Fisher","year":"1936","journal-title":"Ann. Eugen."},{"key":"ref_45","unstructured":"Ho, T.K. (1995, January 14\u201316). Random decision forests. Proceedings of the 3rd international conference on document analysis and recognition, Montreal, QC, Canada."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft COCO: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_47","unstructured":"Kremer, S.C., and Kolen, J.F. (2001). Gradient flow in recurrent nets: The difficulty of learning long-term dependencies. A Field Guide to Dynamical Recurrent Neural Networks, IEEE Press."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_49","unstructured":"Talathi, S.S., and Vartak, A. (2015). Improving performance of recurrent neural network with relu nonlinearity. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1016\/j.knosys.2015.05.015","article-title":"Dynamic selection of the best base classifier in one versus one","volume":"85","author":"Mendialdua","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_51","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2436\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:51:13Z","timestamp":1760363473000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2436"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,24]]},"references-count":51,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20082436"],"URL":"https:\/\/doi.org\/10.3390\/s20082436","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,24]]}}}