{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T07:38:11Z","timestamp":1765438691987,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Research on the human activity recognition could be utilized for the monitoring of elderly people living alone to reduce the cost of home care. Video sensors can be easily deployed in the different zones of houses to achieve monitoring. The goal of this study is to employ a linear-map convolutional neural network (CNN) to perform action recognition with RGB videos. To reduce the amount of the training data, the posture information is represented by skeleton data extracted from the 300 frames of one film. The two-stream method was applied to increase the accuracy of recognition by using the spatial and motion features of skeleton sequences. The relations of adjacent skeletal joints were employed to build the direct acyclic graph (DAG) matrices, source matrix, and target matrix. Two features were transferred by DAG matrices and expanded as color texture images. The linear-map CNN had a two-dimensional linear map at the beginning of each layer to adjust the number of channels. A two-dimensional CNN was used to recognize the actions. We applied the RGB videos from the action recognition datasets of the NTU RGB+D database, which was established by the Rapid-Rich Object Search Lab, to execute model training and performance evaluation. The experimental results show that the obtained precision, recall, specificity, F1-score, and accuracy were 86.9%, 86.1%, 99.9%, 86.3%, and 99.5%, respectively, in the cross-subject source, and 94.8%, 94.7%, 99.9%, 94.7%, and 99.9%, respectively, in the cross-view source. An important contribution of this work is that by using the skeleton sequences to produce the spatial and motion features and the DAG matrix to enhance the relation of adjacent skeletal joints, the computation speed was faster than the traditional schemes that utilize single frame image convolution. Therefore, this work exhibits the practical potential of real-life action recognition.<\/jats:p>","DOI":"10.3390\/s21093112","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T10:30:41Z","timestamp":1619692241000},"page":"3112","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Using Direct Acyclic Graphs to Enhance Skeleton-Based Action Recognition with a Linear-Map Convolution Neural Network"],"prefix":"10.3390","volume":"21","author":[{"given":"Tan-Hsu","family":"Tan","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin-Hao","family":"Hus","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3923-4387","authenticated-orcid":false,"given":"Shing-Hong","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, Chaoyang University of Technology, Taichung 413310, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3337-3551","authenticated-orcid":false,"given":"Yung-Fa","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Engineering, Chaoyang University of Technology, Taichung 413310, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6613-7435","authenticated-orcid":false,"given":"Munkhjargal","family":"Gochoo","sequence":"additional","affiliation":[{"name":"Department of Computer Science &amp; Software Engineering, College of Information Technology, United Arab Emirates University, Al Ain P.O. Box 15551, Abu Dhabi, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,29]]},"reference":[{"key":"ref_1","unstructured":"United Nations, Department of Economic Social Affairs (2019). World Population Ageing 2019, United Nations."},{"key":"ref_2","unstructured":"(2021, April 28). Available online: https:\/\/www.ndc.gov.tw\/Content_List.aspx?n=695E69E28C6AC7F3."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"43","DOI":"10.3109\/08039488.2012.675588","article-title":"Low levels of physical activity in patients with severe mental illness","volume":"67","author":"Nyboe","year":"2013","journal-title":"Nord. J. Psychiatry"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1097\/HCR.0b013e31818c3b83","article-title":"Effects of a home-based cardiac rehabilitation program on the physical activity levels of patients with coronary artery disease","volume":"28","author":"Oliveira","year":"2008","journal-title":"J. Cardiopulm. Rehabil. Prev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"12301","DOI":"10.3390\/s120912301","article-title":"Fall detection with the support vector machine during scripted and continuous unscripted activities","volume":"12","author":"Liu","year":"2012","journal-title":"Sensors"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1089\/tmj.2009.0032","article-title":"Using accelerometers for physical actions recognition by a neural fuzzy network","volume":"15","author":"Liu","year":"2009","journal-title":"Telemed. e-Health"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3101","DOI":"10.1109\/JSEN.2019.2956901","article-title":"Human action recognition using deep learning methods on limited sensory data","volume":"20","author":"Tufek","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1080\/03093640600983949","article-title":"Clinical applications of sensors for human posture and movement analysis: A review","volume":"31","author":"Wong","year":"2007","journal-title":"Prosthet. Orthot. Int."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Feichtenhofer, C., Pinz, A., and Zisserman, A. (2016, January 27\u201330). Convolutional two-stream network fusion for video action recognition. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.213"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Fernando, B., Gavves, E., Oramas, M.J., Ghodrati, A., and Tuytelaars, T. (2015, January 7\u201312). Modeling video evolution for action recognition. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299176"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei, L. (2014, January 23\u201328). Large-scale video classification with convolutional neural networks. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.223"},{"key":"ref_12","unstructured":"Simonyan, K., and Zisserman, A. (2014). Two-stream convolutional networks for action recognition in videos. arXiv Prepr."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., and Gool, L.V. (2016, January 11\u201314). Temporal segment networks: Towards good practices for deep action recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46484-8_2"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3137","DOI":"10.1109\/TMM.2018.2823900","article-title":"Modeling multimodal clues in a hybrid deep learning framework for video classification","volume":"20","author":"Jiang","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3D convolutional neural networks for human action recognition","volume":"35","author":"Ji","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chaudhry, R., Ofli, F., Kurillo, G., Bajcsy, R., and Vidal, R. (2013, January 23\u201328). Bioinspired dynamic 3D discriminative skeletal features for human action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPRW.2013.153"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.cviu.2016.04.005","article-title":"R3DG features: Relative 3D geometry-based skeletal representations for human action recognition","volume":"152","author":"Vemulapalli","year":"2016","journal-title":"Comput. Vis. Image Understand."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1109\/THMS.2015.2504550","article-title":"Action recognition from depth maps using deep convolutional neural networks","volume":"46","author":"Wang","year":"2016","journal-title":"IEEE Trans. Hum.\u2013Mach. Syst."},{"key":"ref_19","unstructured":"Du, Y., Wang, W., and Wang, L. (2015, January 7\u201312). Hierarchical recurrent neural network for skeleton based action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Veeriah, V., Zhuang, N., and Qi, G.-J. (2015, January 7\u201312). Differential recurrent neural networks for action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/ICCV.2015.460"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1109\/TCSVT.2016.2628339","article-title":"Skeleton optical spectra-based action recognition using convolutional neural networks","volume":"28","author":"Hou","year":"2018","journal-title":"IEEE Trans. Circ. Syst. Video Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1097\/EDE.0b013e318127181b","article-title":"Four types of effect modification: A classification based on directed acyclic graphs","volume":"18","author":"VanderWeele","year":"2007","journal-title":"Epidemiology"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Fang, H., Xie, S., Tai, Y., and Lu, C. (2017, January 22\u201329). RMPE: Regional multi-person pose estimation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.256"},{"key":"ref_24","unstructured":"Rapid-Rich Object Search (ROSE) Lab (2021, April 28). Available online: http:\/\/rose1.ntu.edu.sg\/datasets\/actionrecognition.asp."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Vemulapalli, R., Arrate, F., and Chellappa, R. (2014, January 23\u201328). Human action recognition by representing 3d skeletons as points in a lie group. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.82"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Shahroudy, A., Liu, J., Ng, T., and Wang, G. (2016, January 27\u201330). NTU RGB+D: A large scale dataset for 3d human activity analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.115"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, J., Shahroudy, A., Xu, D., and Wang, G. (2016, January 11\u201314). Spatial-temporal LSTM with trust gates for 3D human action recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_50"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, H., and Wang, L. (2017, January 21\u201326). Modeling temporal dynamics and spatial configurations of actions using two-stream recurrent neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.387"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ke, Q., Bennamoun, M., An, S., Sohel, F., and Boussaid, F. (2017, January 21\u201326). A new representation of skeleton sequences for 3D action recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.486"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yan, S., Xiong, Y., and Lin, D. (2018, January 2\u20137). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12328"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Si, C., Jing, Y., Wang, W., Wang, L., and Tan, T. (2018, January 8\u201314). Skeleton-based action recognition with spatial reasoning and temporal stack learning. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_7"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/3112\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:55:33Z","timestamp":1760162133000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/3112"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,29]]},"references-count":31,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["s21093112"],"URL":"https:\/\/doi.org\/10.3390\/s21093112","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,4,29]]}}}