{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,28]],"date-time":"2025-11-28T12:37:15Z","timestamp":1764333435590,"version":"build-2065373602"},"reference-count":46,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,7,7]],"date-time":"2024-07-07T00:00:00Z","timestamp":1720310400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In the era of inclusive education, students with attention deficits are integrated into the general classroom. To ensure a seamless transition of students\u2019 focus towards the teacher\u2019s instruction throughout the course and to align with the teaching pace, this paper proposes a continuous recognition algorithm for capturing teachers\u2019 dynamic gesture signals. This algorithm aims to offer instructional attention cues for students with attention deficits. According to the body landmarks of the teacher\u2019s skeleton by using vision and machine learning-based MediaPipe BlazePose, the proposed method uses simple rules to detect the teacher\u2019s hand signals dynamically and provides three kinds of attention cues (Pointing to left, Pointing to right, and Non-pointing) during the class. Experimental results show the average accuracy, sensitivity, specificity, precision, and F1 score achieved 88.31%, 91.03%, 93.99%, 86.32%, and 88.03%, respectively. By analyzing non-verbal behavior, our method of competent performance can replace verbal reminders from the teacher and be helpful for students with attention deficits in inclusive education.<\/jats:p>","DOI":"10.3390\/a17070300","type":"journal-article","created":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T07:57:45Z","timestamp":1720425465000},"page":"300","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Continuous Recognition of Teachers\u2019 Hand Signals for Students with Attention Deficits"],"prefix":"10.3390","volume":"17","author":[{"given":"Ivane Delos Santos","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, National Dong Hwa University, Hualien 97401, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6614-4635","authenticated-orcid":false,"given":"Chieh-Ming","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Dong Hwa University, Hualien 97401, Taiwan"}]},{"given":"Shang-Shu","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Education and Human Potentials Development, National Dong Hwa University, Hualien 97401, Taiwan"}]},{"given":"Chih-Kang","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Special Education, National Dong Hwa University, Hualien 97401, Taiwan"}]},{"given":"Mei-Juan","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Dong Hwa University, Hualien 97401, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2837-6662","authenticated-orcid":false,"given":"Chia-Hung","family":"Yeh","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taiwan Normal University, Taipei 10610, Taiwan"},{"name":"Department of Electrical Engineering, National Sun Yat-sen University, Kaohsiung 80424, Taiwan"}]},{"given":"Yuan-Hong","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Dong Hwa University, Hualien 97401, Taiwan"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1177\/1362361315593536","article-title":"Joint attention revisited: Finding strengths among children with autism","volume":"20","author":"Hurwitz","year":"2016","journal-title":"Autism"},{"key":"ref_2","unstructured":"Lai, Y.H., Chang, Y.C., Ma, Y.W., Huang, S.Y., and Chao, H.C. (2019, January 3\u20136). Improvement of ADHD Behaviors with AI Perception Technology. Proceedings of the International Cognitive Cities Conference, Kyoto, Japan."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"66","DOI":"10.5430\/wje.v8n2p66","article-title":"Universal design for learning to support access to the general education curriculum for students with intellectual disabilities","volume":"8","author":"Ahmad","year":"2018","journal-title":"World J. Educ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.pediatrneurol.2021.06.010","article-title":"Moving toward understanding autism: Visual-motor integration, imitation, and social skill development","volume":"122","author":"Lidstone","year":"2021","journal-title":"Pediatr. Neurol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1875549","DOI":"10.1080\/23311908.2021.1875549","article-title":"Visual communication analysis (VCA): Implementing self-determination theory and research-based practices in special education classrooms","volume":"8","author":"Shkedy","year":"2021","journal-title":"Cogent Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1080\/08856257.2019.1703548","article-title":"Augmented reality in special education: A meta-analysis of single-subject design studies","volume":"35","author":"Baragash","year":"2020","journal-title":"Eur. J. Spec. Needs Educ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/s10055-019-00379-9","article-title":"Systematic review and meta-analysis of augmented reality in educational settings","volume":"23","author":"Baldiris","year":"2019","journal-title":"Virtual Real."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1829","DOI":"10.1002\/gps.5603","article-title":"Effects of virtual reality cognitive training in individuals with mild cognitive impairment: A systematic review and meta-analysis","volume":"36","author":"Zhong","year":"2021","journal-title":"Int. J. Geriatr. Psychiatry"},{"key":"ref_9","first-page":"603","article-title":"Meta-analysis of artificial intelligence works in ubiquitous learning environments and technologies","volume":"11","author":"Sam","year":"2020","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21\u201326). Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime multi-person 2d pose estimation using part affinity fields","volume":"43","author":"Cao","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liu, T., Chen, Z., and Wang, X. (2019, January 24\u201327). Automatic Instructional Pointing Gesture Recognition by Machine Learning in the Intelligent Learning Environment. Proceedings of the 2019 4th International Conference on Distance Education and Learning, Shanghai, China.","DOI":"10.1145\/3338147.3338163"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103464","DOI":"10.1016\/j.infrared.2020.103464","article-title":"Human hand gesture recognition with convolutional neural networks for K-12 double-teachers instruction mode classroom","volume":"111","author":"Wang","year":"2020","journal-title":"Infrared Phys. Tech."},{"key":"ref_14","unstructured":"Hern\u00e1ndez Correa, J., Farsani, D., and Araya, R. (December, January 30). An Application of Machine Learning and Image Processing to Automatically Detect Teachers\u2019 Gestures. Proceedings of the International Conference on Computational Collective Intelligence, Da Nang, Vietnam."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gu, Y., Hu, J., Zhou, Y., and Lu, L. (2020, January 10\u201313). Online Teaching Gestures Recognition Model Based on Deep Learning. Proceedings of the 2020 International Conference on Networking and Network Applications, Haikou, China.","DOI":"10.1109\/NaNA51271.2020.00076"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Araya, R., and Sossa-Rivera, J. (2021). Automatic detection of gaze and body orientation in elementary school classrooms. Front. Robot. AI, 8.","DOI":"10.3389\/frobt.2021.729832"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1006","DOI":"10.1111\/jcal.12933","article-title":"A non-verbal teaching behaviour analysis for improving pointing out gestures: The case of asynchronous video lecture analysis using deep learning","volume":"40","author":"Yoon","year":"2024","journal-title":"J. Comput. Assist. Learn."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e2218","DOI":"10.1002\/cav.2218","article-title":"GestureTeach: A gesture guided online teaching interactive model","volume":"35","author":"Liu","year":"2024","journal-title":"Comput. Animat. Virtual Worlds"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chen, Z., Feng, X., Liu, T., Wang, C., and Zhang, C. (2017, January 6\u20138). A Computer-Assisted Teaching System with Gesture Recognition Technology and Its Applications. Proceedings of the International Conference on Digital Technology in Education, Taipei, Taiwan.","DOI":"10.1145\/3134847.3134848"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4835","DOI":"10.1007\/s11042-018-5922-8","article-title":"A virtual tutor movement learning system in eLearning","volume":"78","author":"Chiang","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Goto, T., Sakurai, D., and Ooi, S. (2020, January 19\u201322). Proposal of Feedback System Based on Skeletal Analysis in Physical Education Classes. Proceedings of the 4th International Conference on Education and Multimedia Technology, Kyoto, Japan.","DOI":"10.1145\/3416797.3416814"},{"key":"ref_22","unstructured":"Amrutha, K., Prabu, P., and Paulose, J. (2021, January 27\u201329). Human Body Pose Estimation and Applications. Proceedings of the 2021 Innovations in Power and Advanced Computing Technologies, Kuala Lumpur, Malaysia."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1080\/10749039.2022.2060260","article-title":"Gestures, systemic functional linguistics and mathematics education","volume":"29","author":"Farsani","year":"2022","journal-title":"Mind Cult. Act."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1111\/jcal.12745","article-title":"Digital skills critical for education: Video analysis of students' technology use in Norwegian secondary English classrooms","volume":"39","author":"Kure","year":"2023","journal-title":"J. Comput. Assist. Learn."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5308","DOI":"10.1109\/ACCESS.2018.2791861","article-title":"Towards emotionally aware AI smart classroom: Current issues and directions for engineering and education","volume":"6","author":"Kim","year":"2018","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Swain, D., Satapathy, S., Acharya, B., Shukla, M., Gerogiannis, V.C., Kanavos, A., and Giakovis, D. (2022). Deep learning models for yoga pose monitoring. Algorithms, 15.","DOI":"10.3390\/a15110403"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Connie, T., Aderinola, T.B., Ong, T.S., Goh, M.K.O., Erfianto, B., and Purnama, B. (2022). Pose-based gait analysis for diagnosis of Parkinson\u2019s disease. Algorithms, 15.","DOI":"10.3390\/a15120474"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Gesnouin, J., Pechberti, S., Bresson, G., Stanciulescu, B., and Moutarde, F. (2020). Predicting intentions of pedestrians from 2D skeletal pose sequences with a representation-focused multi-branch deep learning network. Algorithms, 13.","DOI":"10.3390\/a13120331"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Martinez, J., Hossain, R., Romero, J., and Little, J.J. (2017, January 22\u201329). A Simple Yet Effective Baseline for 3D Human Pose Estimation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.288"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Luvizon, D.C., Picard, D., and Tabia, H. (2018, January 18\u201322). 2D\/3D Pose Estimation and Action Recognition using Multitask Deep Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00539"},{"key":"ref_31","first-page":"1146","article-title":"LCR-Net++: Multi-person 2D and 3D pose detection in natural images","volume":"42","author":"Rogez","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1282","DOI":"10.1109\/TMM.2022.3141231","article-title":"Exploiting temporal contexts with strided transformer for 3D human pose estimation","volume":"25","author":"Li","year":"2023","journal-title":"IEEE Trans. Multimed."},{"key":"ref_33","first-page":"11","article-title":"Deep Learning-based human pose estimation: A survey","volume":"56","author":"Zheng","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_34","unstructured":"Bazarevsky, V., Grishchenko, I., Raveendran, K., Zhu, T., Zhang, F., and Grundmann, M. (2020, January 15). BlazePose: On-Device Real-Time Body Pose Tracking. Proceedings of the CVPR Workshop on Computer Vision for Augmented and Virtual Reality, Seattle, WA, USA."},{"key":"ref_35","unstructured":"(2024, April 02). MediaPipe. Available online: https:\/\/developers.google.com\/mediapipe\/solutions\/vision\/pose_landmarker\/."},{"key":"ref_36","unstructured":"Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., and Grundmann, M. (2020, January 15). MediaPipe Hands: On-device Real-Time Hand Tracking. Proceedings of the CVPR Workshop on Computer Vision for Augmented and Virtual Reality, Seattle, WA, USA."},{"key":"ref_37","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M., and Lee, J. (2019, January 17). MediaPipe: A Framework for Perceiving and Processing Reality. Proceedings of the CVPR Workshop on Computer Vision for Augmented and Virtual Reality, Long Beach, CA, USA."},{"key":"ref_38","unstructured":"Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for Multi-Class Classification: An Overview. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yu, M., Kim, N., Jung, Y., and Lee, S. (2020). A frame detection method for real-time hand gesture recognition systems using CW-radar. Sensors, 20.","DOI":"10.3390\/s20082321"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"33610","DOI":"10.1109\/ACCESS.2019.2903586","article-title":"Short-range radar based real-time hand gesture recognition using LSTM encoder","volume":"7","author":"Choi","year":"2019","journal-title":"IEEE Access"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ryumin, D., Ivanko, D., and Ryumina, E. (2023). Audio-visual speech and gesture recognition by sensors of mobile devices. Sensors, 23.","DOI":"10.3390\/s23042284"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sidiya, K., Alzanbagi, N., and Bensenouci, A. Google Glass and Apple Watch Will They Become Our Learning Tools? In Proceedings of the 12th Learning and Technology Conference, Jeddah, Saudi Arabia, 12\u201313 April 2015.","DOI":"10.1109\/LT.2015.7587222"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1007\/s10882-018-9623-z","article-title":"Applying a vibration reminder to ameliorate the hyperactive behavior of students with attention deficit hyperactivity disorder in class","volume":"30","author":"Lai","year":"2018","journal-title":"J. Dev. Phys. Disabil."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"16837","DOI":"10.1109\/ACCESS.2019.2893971","article-title":"Using a google glass-based classroom feedback system to improve students to teacher communication","volume":"7","author":"Zarraonandia","year":"2019","journal-title":"IEEE Access"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Ayearst, L.E., Brancaccio, R., and Weiss, M.D. (2023). An open-label study of a wearable device targeting ADHD, executive function, and academic performance. Brain Sci., 13.","DOI":"10.3390\/brainsci13121728"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Whitmore, N., Chan, S., Zhang, J., Chwalek, P., Chin, S., and Maes, P. (2024, January 11\u201316). Improving Attention Using Wearables via Haptic and Multimodal Rhythmic Stimuli. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3613904.3642256"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/7\/300\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:11:19Z","timestamp":1760109079000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/7\/300"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,7]]},"references-count":46,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["a17070300"],"URL":"https:\/\/doi.org\/10.3390\/a17070300","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2024,7,7]]}}}