{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T12:20:39Z","timestamp":1770985239219,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2021,11,27]],"date-time":"2021-11-27T00:00:00Z","timestamp":1637971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001321","name":"National Research Foundation","doi-asserted-by":"publisher","award":["NRF-2019H1D3A1A01071115"],"award-info":[{"award-number":["NRF-2019H1D3A1A01071115"]}],"id":[{"id":"10.13039\/501100001321","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Institute of Information &amp; communications Technology Planning &amp; Evaluation (IITP)","award":["No.2020-0-00056, To create AI systems that act appropriately and effectively in novel situations that occur in open worlds"],"award-info":[{"award-number":["No.2020-0-00056, To create AI systems that act appropriately and effectively in novel situations that occur in open worlds"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>An authorized traffic controller (ATC) has the highest priority for direct road traffic. In some irregular situations, the ATC supersedes other traffic control. Human drivers indigenously understand such situations and tend to follow the ATC; however, an autonomous vehicle (AV) can become confused in such circumstances. Therefore, autonomous driving (AD) crucially requires a human-level understanding of situation-aware traffic gesture recognition. In AVs, vision-based recognition is particularly desirable because of its suitability; however, such recognition systems have various bottlenecks, such as failing to recognize other humans on the road, identifying a variety of ATCs, and gloves in the hands of ATCs. We propose a situation-aware traffic control hand-gesture recognition system, which includes ATC detection and gesture recognition. Three-dimensional (3D) hand model-based gesture recognition is used to mitigate the problem associated with gloves. Our database contains separate training and test videos of approximately 60 min length, captured at a frame rate of 24 frames per second. It has 35,291 different frames that belong to traffic control hand gestures. Our approach correctly recognized traffic control hand gestures; therefore, the proposed system can be considered as an extension of the operational domain of the AV.<\/jats:p>","DOI":"10.3390\/s21237914","type":"journal-article","created":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T01:45:02Z","timestamp":1638323102000},"page":"7914","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Authorized Traffic Controller Hand Gesture Recognition for Situation-Aware Autonomous Driving"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8579-5583","authenticated-orcid":false,"given":"Ashutosh","family":"Mishra","sequence":"first","affiliation":[{"name":"Yonsei Institute of Convergence Technology, Yonsei University, Incheon 21983, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinhyuk","family":"Kim","sequence":"additional","affiliation":[{"name":"Yonsei Institute of Convergence Technology, Yonsei University, Incheon 21983, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8029-4146","authenticated-orcid":false,"given":"Jaekwang","family":"Cha","sequence":"additional","affiliation":[{"name":"Yonsei Institute of Convergence Technology, Yonsei University, Incheon 21983, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dohyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Yonsei Institute of Convergence Technology, Yonsei University, Incheon 21983, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9935-1721","authenticated-orcid":false,"given":"Shiho","family":"Kim","sequence":"additional","affiliation":[{"name":"Yonsei Institute of Convergence Technology, Yonsei University, Incheon 21983, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.neucom.2019.07.103","article-title":"Visual recognition of traffic police gestures with convolutional pose machine and handcrafted features","volume":"390","author":"He","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wiederer, J., Bouazizi, A., Kressel, U., and Belagiannis, V. (January, January 24). Traffic Control Gesture Recognition for Autonomous Vehicles. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341214"},{"key":"ref_3","unstructured":"Litman, T. (2020). Autonomous Vehicle Implementation Predictions: Implications for Transport Planning, Available online: https:\/\/www.vtpi.org\/avip.pdf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MSPEC.2020.9173901","article-title":"The new driver\u2019s ED: Game developers teach Cruise\u2019s autonomous vehicles to understand gestures made by people on the street","volume":"57","author":"Weaver","year":"2020","journal-title":"IEEE Spectr."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mishra, A., Cha, J., and Kim, S. (2020, January 24\u201326). HCI Based In-Cabin Monitoring System for Irregular Situations with Occupants Facial Anonymization. Proceedings of the International Conference on Intelligent Human Computer Interaction, Daegu, Korea.","DOI":"10.1007\/978-3-030-68452-5_40"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mishra, A., Kim, J., Kim, D., Cha, J., and Kim, S. (2020, January 21\u201324). An intelligent in-cabin monitoring system in fully autonomous vehicles. Proceedings of the International SoC Design Conference (ISOCC), Yeosu, Korea.","DOI":"10.1109\/ISOCC50952.2020.9333062"},{"key":"ref_7","unstructured":"National Highway Traffic Safety Administration (2021, August 02). Automated Driving Systems 2.0: A Vision for Safety, Available online: https:\/\/www.nhtsa.gov\/sites\/nhtsa.gov\/files\/documents\/13069a-ads2.0_090617_v9a_tag.pdf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sun, C., Deng, Z., Chu, W., Li, S., and Cao, D. (2021). Acclimatizing the Operational Design Domain for Autonomous Driving Systems. IEEE Intell. Transp. Syst. Mag., 2\u201316.","DOI":"10.1109\/MITS.2021.3070651"},{"key":"ref_9","unstructured":"BERMAN, B. (2021, August 02). The Key to Autonomous Vehicle Safety Is ODD. Available online: https:\/\/www.sae.org\/news\/2019\/11\/odds-for-av-testing."},{"key":"ref_10","unstructured":"Koopman, P., and Fratrik, F. (2019, January 27). How many operational design domains, objects, and events?. Proceedings of the AAAI Workshop on Artificial Intelligence Safety Co-Located with the Thirty-Third AAAI Conference on Artificial Intelligence 2019 (AAAI 2019), Honolulu, HI, USA."},{"key":"ref_11","unstructured":"On-Road Automated Driving (ORAD) Committee (2021, August 02). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Available online: https:\/\/saemobilus.sae.org\/content\/j3016_202104."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mujahid, A., Awan, M.J., Yasin, A., Mohammed, M.A., Dama\u0161evi\u010dius, R., Maskeli\u016bnas, R., and Abdulkareem, K.H. (2021). Real-Time Hand Gesture Recognition Based on Deep Learning YOLOv3 Model. Appl. Sci., 11.","DOI":"10.3390\/app11094164"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hang, K. (2021, January 28\u201330). A 3D Gesture Modeling Method and Its Application in 3D Gesture Prediction. Proceedings of the 2nd International Conference on Computing and Data Science, Stanford, CA, USA.","DOI":"10.1145\/3448734.3450937"},{"key":"ref_14","first-page":"25","article-title":"Hand gesture recognition systems: A survey","volume":"71","author":"Sarkar","year":"2013","journal-title":"Int. J. Comput. Appl."},{"key":"ref_15","first-page":"12","article-title":"Hand gesture modeling and recognition using geometric features: A review","volume":"3","author":"Hasan","year":"2012","journal-title":"Can. J. Image Process. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-012-9356-9","article-title":"Vision based hand gesture recognition for human computer interaction: A survey","volume":"43","author":"Rautaray","year":"2015","journal-title":"Artif. Intell. Rev."},{"key":"ref_17","unstructured":"Chaudhary, A., Raheja, J.L., Das, K., and Raheja, S. (2013). Intelligent approaches to interact with machines using hand gesture recognition in natural way: A survey. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1109\/TSMCC.2007.893280","article-title":"Gesture recognition: A survey","volume":"37","author":"Mitra","year":"2007","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_19","first-page":"972","article-title":"Vision based hand gesture recognition","volume":"49","author":"Garg","year":"2009","journal-title":"World Acad. Sci. Eng. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Fang, Y., Wang, K., Cheng, J., and Lu, H. (2007, January 2\u20135). A Real-Time Hand Gesture Recognition Method. Proceedings of the Multimedia and Expo, IEEE International Conference on Multimedia and Expo, Beijing, China.","DOI":"10.1109\/ICME.2007.4284820"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Oudah, M., Al-Naji, A., and Chahl, J. (2020). Hand Gesture Recognition Based on Computer Vision: A Review of Techniques. J. Imaging, 6.","DOI":"10.3390\/jimaging6080073"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"192527","DOI":"10.1109\/ACCESS.2020.3032140","article-title":"Deep learningbased approach for sign language gesture recognition with efficient hand gesture representation","volume":"8","author":"Muhammad","year":"2020","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Vaitkevi\u010dius, A., Taroza, M., Bla\u017eauskas, T., Dama\u0161evi\u010dius, R., Maskeliunas, R., and Wo\u2019zniak, M. (2019). Recognition of american sign language gestures in a virtual reality using leap motion. Appl. Sci., 9.","DOI":"10.3390\/app9030445"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"10449","DOI":"10.1007\/s00521-021-05802-4","article-title":"Development and validation of a brazilian sign language database for human gesture recognition","volume":"33","author":"Rezende","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1007\/s12652-019-01209-1","article-title":"Recognition of basketball referee signals from real-time videos","volume":"11","author":"Raudonis","year":"2020","journal-title":"J. Ambient. Intell. Humaniz. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"104090","DOI":"10.1016\/j.imavis.2020.104090","article-title":"A framework of human action recognition using length control features fusion and weighted entropy-variances based feature selection","volume":"106","author":"Afza","year":"2021","journal-title":"Image Vis. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1783","DOI":"10.1016\/S0031-3203(99)00176-4","article-title":"Facial feature extraction and pose determination","volume":"33","author":"Nikolaidis","year":"2000","journal-title":"Pattern Recognit."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e442","DOI":"10.7717\/peerj-cs.442","article-title":"Detection of sitting posture using hierarchical image composition and deep learning","volume":"7","author":"Kulikajevas","year":"2021","journal-title":"Peer J. Comput. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1186\/s13673-020-00256-4","article-title":"Multiple kinect based system to monitor and analyze key performance indicators of physical training","volume":"10","author":"Ryselis","year":"2020","journal-title":"Hum. Cent. Comput. Inf. Sci."},{"key":"ref_30","first-page":"1967","article-title":"An ANN-based gesture recognition algorithm for smart-home applications","volume":"14","author":"Huu","year":"2020","journal-title":"KSII Trans. Internet Inf. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1109\/TCSVT.2015.2469551","article-title":"Survey on 3D hand gesture recognition","volume":"26","author":"Cheng","year":"2015","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chatzis, T., Stergioulas, A., Konstantinidis, D., Dimitropoulos, K., and Daras, P. (2020). A comprehensive study on deep learning-based 3d hand pose estimation methods. Appl. Sci., 10.","DOI":"10.3390\/app10196850"},{"key":"ref_33","unstructured":"Oikonomidis, I., Kyriazis, N., and Argyros, A.A. (September, January 29). Efficient model-based 3D tracking of hand articulations using Kinect. Proceedings of the British Machine Vision Conference, Dundee, UK."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sridhar, S., Oulasvirta, A., and Theobalt, C. (2013, January 1\u20138). Interactive markerless articulated hand motion tracking using RGB and depth data. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.305"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2980179.2980226","article-title":"Sphere-meshes for real-time hand modeling and tracking","volume":"35","author":"Tkach","year":"2016","journal-title":"ACM Trans. Graph (ToG)"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1007\/s11263-016-0895-4","article-title":"Capturing hands in action using discriminative salient points and physics simulation","volume":"118","author":"Tzionas","year":"2016","journal-title":"Int. J. Comput. Vis."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Gao, D., Zhang, B., Wang, Q., Zhang, X., Pan, P., and Xu, Y. (2021, January 11\u201317). SCAT: Stride Consistency with Auto-regressive regressor and Transformer for hand pose estimation. Proceedings of the IEEE\/CVF International Conference on Computer Vision\u2013ICCV 2021, Montreal, Canada.","DOI":"10.1109\/ICCVW54120.2021.00256"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Armagan, A., Garcia-Hernando, G., Baek, S., Hampali, S., Rad, M., Zhang, Z., Xie, S., Chen, M., Zhang, B., and Xiong, F. (2020, January 23\u201328). Measuring generalisation to unseen viewpoints, articulations, shapes and objects for 3D hand pose estimation under hand-object interaction. Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK.","DOI":"10.1007\/978-3-030-58592-1_6"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3130800.3130883","article-title":"Embodied hands: Modeling and capturing hands and bodies together","volume":"36","author":"Romero","year":"2017","journal-title":"ACM Trans. Graph (ToG)"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2816795.2818013","article-title":"SMPL: A skinned multi-person linear model","volume":"34","author":"Loper","year":"2015","journal-title":"ACM Trans. Graph (TOG)"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A.A., Tzionas, D., and Black, M.J. (2019, January 15\u201320). Expressive body capture: 3d hands, face, and body from a single image. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01123"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Osman, A.A., Bolkart, T., and Black, M.J. (2020, January 23\u201328). Star: Sparse trained articulated human body regressor. Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK. Part VI 16.","DOI":"10.1007\/978-3-030-58539-6_36"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Rong, Y., Shiratori, T., and Joo, H. (2020). FrankMocap: Fast monocular 3D hand and body motion capture by regression and integration. arXiv.","DOI":"10.1109\/ICCVW54120.2021.00201"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1701","DOI":"10.1016\/j.patrec.2004.06.016","article-title":"Recognition-based gesture spotting in video games","volume":"25","author":"Kang","year":"2004","journal-title":"Pattern Recog. Lett."},{"key":"ref_45","first-page":"270","article-title":"Accelerometer-based Chinese traffic police gesture recognition system","volume":"19","author":"Yuan","year":"2010","journal-title":"Chin. J. Electron."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Guo, F., Cai, Z., and Tang, J. (2011, January 16\u201318). Chinese traffic police gesture recognition in complex scene. Proceedings of the 10th International Conference on Trust, Security and Privacy in Computing and Communications, Changsha, China.","DOI":"10.1109\/TrustCom.2011.208"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1007\/s10044-014-0383-9","article-title":"Max-covering scheme for gesture recognition of Chinese traffic police","volume":"18","author":"Cai","year":"2015","journal-title":"Pattern Anal. Appl."},{"key":"ref_48","first-page":"1","article-title":"Road traffic control gesture recognition using depth images","volume":"1","author":"Le","year":"2012","journal-title":"IEIE Trans. Smart Process. Comput."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1700","DOI":"10.1016\/j.procs.2015.02.113","article-title":"Framework for traffic personnel gesture recognition","volume":"46","author":"Sathya","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"8915","DOI":"10.1007\/s11042-016-3497-9","article-title":"Gesture recognition of traffic police based on static and dynamic descriptor fusion","volume":"76","author":"Guo","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Ma, C., Zhang, Y., Wang, A., Wang, Y., and Chen, G. (2018). Traffic command gesture recognition for virtual urban scenes based on a spatiotemporal convolution neural network. ISPRS Int. J. Geo-Inf., 7.","DOI":"10.3390\/ijgi7010037"},{"key":"ref_52","unstructured":"Shaotran, E., Cruz, J.J., and Reddi, V.J. (2019). GLADAS: Gesture Learning for Advanced Driver Assistance Systems. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Chen, Y., Tu, Z., Ge, L., Zhang, D., Chen, R., and Yuan, J. (2019, January 15\u201320). So-handnet: Self-organizing network for 3d hand pose estimation with semi-supervised learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00706"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"79491","DOI":"10.1109\/ACCESS.2020.2990434","article-title":"Hand gesture recognition for sign language using 3DCNN","volume":"8","author":"Muhammad","year":"2020","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7914\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:36:41Z","timestamp":1760168201000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7914"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,27]]},"references-count":54,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21237914"],"URL":"https:\/\/doi.org\/10.3390\/s21237914","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,27]]}}}