{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T16:08:05Z","timestamp":1759939685668},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,3,1]],"date-time":"2023-03-01T00:00:00Z","timestamp":1677628800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,20]],"date-time":"2023-03-20T00:00:00Z","timestamp":1679270400000},"content-version":"vor","delay-in-days":19,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Intell Robot Syst"],"published-print":{"date-parts":[[2023,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\nWe describe the design and validation of a vision-based system that allows the dynamic identification of ramp signals performed by airport ground staff. This ramp signals\u2019 recognizer increases the autonomy of unmanned vehicles and prevents errors caused by visual misinterpretations or lack of attention from the pilot of manned vehicles. This system is based on supervised machine learning techniques, developed with our own training dataset and two models. The first model is based on a pre-trained Convolutional Pose Machine followed by a classifier, for which we have evaluated two possibilities: A Random Forest and a Multi-Layer Perceptron based classifier. The second model is based on a single Convolutional Neural Network that classifies the gestures directly imported from real images. When experimentally tested, the first model proved to be more accurate and scalable than the second one. Its strength relies on a better capacity to extract information from the images and transform the domain of pixels into spatial vectors, which increases the robustness of the classification layer. The second model instead is more adequate for gestures\u2019 identification in low visibility environments, such as during night operations, conditions in which the first model appeared to be more limited, segmenting the shape of the operator. Our results support the use of supervised learning and computer vision techniques for the correct identification and classification of ramp hand signals performed by airport marshallers.\n<\/jats:p>","DOI":"10.1007\/s10846-023-01832-3","type":"journal-article","created":{"date-parts":[[2023,3,20]],"date-time":"2023-03-20T02:02:43Z","timestamp":1679277763000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Real-Time Visual Recognition of Ramp Hand Signals for UAS Ground Operations"],"prefix":"10.1007","volume":"107","author":[{"given":"Miguel \u00c1ngel","family":"de Frutos Carro","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fernando Carlos","family":"L\u00f3pezHern\u00e1ndez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jos\u00e9 Javier Rainer","family":"Granados","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,3,20]]},"reference":[{"key":"1832_CR1","unstructured":"ICAO, Annex 2 - Rules of the Air - Tenth Edition, no. November. (2005)"},{"issue":"3","key":"1832_CR2","doi-asserted-by":"publisher","first-page":"467","DOI":"10.5604\/01.3001.0012.4369","volume":"25","author":"J Tomaszewska","year":"2018","unstructured":"Tomaszewska, J., Zieja, M., Woch, M., Krzysiak, P.: Statistical analysis of ground-related incidents at airports. J. KONES 25(3), 467\u2013472 (2018). https:\/\/doi.org\/10.5604\/01.3001.0012.4369","journal-title":"J. KONES"},{"key":"1832_CR3","unstructured":"Dempsey, M. E., Rasmussen, S.: \u201cEyes of the army--US Army roadmap for unmanned aircraft systems, 2010--2035,\u201d (2010)"},{"key":"1832_CR4","doi-asserted-by":"publisher","unstructured":"Song, Y., Demirdjian, D., Davis, R.: \u201cTracking body and hands for gesture recognition: NATOPS aircraft handling signals database,\u201d 2011 IEEE Int. Conf. Autom. Face Gesture Recognit. Work. FG 2011, pp. 500\u2013506 (2011).https:\/\/doi.org\/10.1109\/FG.2011.5771448","DOI":"10.1109\/FG.2011.5771448"},{"key":"1832_CR5","doi-asserted-by":"publisher","unstructured":"Civil Aviation Authority (CAA), \u201cVisual aids handbook,\u201d Aids.10(6), 690\u2013691, (1996). https:\/\/doi.org\/10.1097\/00002030-199606000-00024","DOI":"10.1097\/00002030-199606000-00024"},{"key":"1832_CR6","doi-asserted-by":"publisher","unstructured":"Castillo, J.C., Alonso-Mart\u00edn, F., C\u00e1ceres-Dom\u00edngue, D., Malfaz, M., Salichs M. Malfaz, A., Salichs, M.A.: \u201cThe Influence of Speed and Position in Dynamic Gesture Recognition for Human-Robot Interaction,\u201d J. Sensors., (2019). https:\/\/doi.org\/10.1155\/2019\/7060491","DOI":"10.1155\/2019\/7060491"},{"key":"1832_CR7","doi-asserted-by":"publisher","unstructured":"Shannon, C.E.: \u201cThe Mathematical Theory of Communication,\u201d M.D. Comput., (1997). https:\/\/doi.org\/10.2307\/410457","DOI":"10.2307\/410457"},{"key":"1832_CR8","doi-asserted-by":"publisher","unstructured":"Demarco, K.J., West, M.E., Howard, A.M.: \u201cUnderwater human-robot communication: A case study with human divers,\u201d Conf. Proc. - IEEE Int. Conf. Syst. Man Cybern., vol. 2014-Janua, no. January, pp. 3738\u20133743, (2014). https:\/\/doi.org\/10.1109\/smc.2014.6974512","DOI":"10.1109\/smc.2014.6974512"},{"issue":"2","key":"1832_CR9","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1093\/jcde\/qwab080","volume":"9","author":"T Baek","year":"2022","unstructured":"Baek, T., Lee, Y.G.: Traffic control hand signal recognition using convolution and recurrent neural networks. J. Comput. Des. Eng. 9(2), 296\u2013309 (2022). https:\/\/doi.org\/10.1093\/jcde\/qwab080","journal-title":"J. Comput. Des. Eng."},{"key":"1832_CR10","doi-asserted-by":"publisher","unstructured":"Molchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., Kautz, J.: \u201cOnline Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks,\u201d Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Decem, 4207\u20134215 (2016). https:\/\/doi.org\/10.1109\/CVPR.2016.456","DOI":"10.1109\/CVPR.2016.456"},{"key":"1832_CR11","doi-asserted-by":"publisher","unstructured":"Kapuscinski, T., Oszust, Wysocki, M.,D. Warchol.: \u201cRecognition of hand gestures observed by depth cameras,\u201d Int. J. Adv. Robot. Syst., vol. 12, (2015). https:\/\/doi.org\/10.5772\/60091","DOI":"10.5772\/60091"},{"key":"1832_CR12","doi-asserted-by":"publisher","unstructured":"Choi, C., Ahn, J.H., Byun, H.: \u201cVisual recognition of aircraft marshalling signals using gesture phase analysis,\u201d IEEE Intell. Veh. Symp. Proc., pp. 853\u2013858 (2008). https:\/\/doi.org\/10.1109\/IVS.2008.4621186","DOI":"10.1109\/IVS.2008.4621186"},{"issue":"2","key":"1832_CR13","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1023\/A:1008918401478","volume":"9","author":"S Waldherr","year":"2000","unstructured":"Waldherr, S., Romero, R., Thrun, S.: Gesture based interface for human-robot interaction. Auton. Robots 9(2), 151\u2013173 (2000). https:\/\/doi.org\/10.1023\/A:1008918401478","journal-title":"Auton. Robots"},{"issue":"6","key":"1832_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5815\/ijisa.2016.06.01","volume":"8","author":"A Rib\u00f3","year":"2016","unstructured":"Rib\u00f3, A., Warchol, D., M. prz edu pl Oszust: An approach to gesture recognition with skeletal data using dynamic time warping and nearest neighbour classifier\u201d. Int. J. Intell. Syst. Appl. 8(6), 1\u20138 (2016). https:\/\/doi.org\/10.5815\/ijisa.2016.06.01","journal-title":"Int. J. Intell. Syst. Appl."},{"key":"1832_CR15","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijleo.2015.02.043","author":"JL Raheja","year":"2015","unstructured":"Raheja, J.L., Minhas, M., Prashanth, D., Shah, T., Chaudhary, A.: Robust gesture recognition using Kinect: A comparison between DTW and HMM. Optik (Stuttg) (2015). https:\/\/doi.org\/10.1016\/j.ijleo.2015.02.043","journal-title":"Optik (Stuttg)"},{"issue":"4","key":"1832_CR16","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1109\/TPAMI.2016.2599174","volume":"39","author":"J Donahue","year":"2017","unstructured":"Donahue, J., et al.: Long-Term Recurrent Convolutional Networks for Visual Recognition and Description. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 677\u2013691 (2017). https:\/\/doi.org\/10.1109\/TPAMI.2016.2599174","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"1832_CR17","doi-asserted-by":"publisher","unstructured":"Zhou, B., Andonian, A., Oliva, A., Torralba, A.: \u201cTemporal Relational Reasoning in Videos,\u201d Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 11205 LNCS,831\u2013846 (2018). https:\/\/doi.org\/10.1007\/978-3-030-01246-5_49","DOI":"10.1007\/978-3-030-01246-5_49"},{"key":"1832_CR18","doi-asserted-by":"publisher","unstructured":"Hara, K., Kataoka, H., Satoh, Y.: \u201cCan Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?,\u201d Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 6546\u20136555, (2018). https:\/\/doi.org\/10.1109\/CVPR.2018.00685","DOI":"10.1109\/CVPR.2018.00685"},{"key":"1832_CR19","doi-asserted-by":"publisher","unstructured":"L. Abraham, A. Urru, N. Normani, M. P. Wilk, M. Walsh, and B. O\u2019flynn, \u201cHand tracking and gesture recognition using lensless smart sensors,\u201d Sensors (Switzerland), vol. 18, no. 9, (2018). https:\/\/doi.org\/10.3390\/s18092834","DOI":"10.3390\/s18092834"},{"key":"1832_CR20","doi-asserted-by":"publisher","unstructured":"Viola, P., Jones, M.: \u201cRapid object detection using a boosted cascade of simple features,\u201d in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2001).https:\/\/doi.org\/10.1109\/cvpr.2001.990517","DOI":"10.1109\/cvpr.2001.990517"},{"key":"1832_CR21","doi-asserted-by":"publisher","unstructured":"Dalal, N., Triggs, B.: \u201cHistograms of oriented gradients for human detection,\u201d in Proceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR (2005). https:\/\/doi.org\/10.1109\/CVPR.2005.177","DOI":"10.1109\/CVPR.2005.177"},{"key":"1832_CR22","unstructured":"Krizhevsky, A., Sutskever, I., Hinton, G.E.: \u201cImageNet classification with deep convolutional neural networks,\u201d in Advances in Neural Information Processing Systems (2012)"},{"key":"1832_CR23","doi-asserted-by":"publisher","unstructured":"Wei, S.E., Ramakrishna, V., Kanade, T., Sheikh, Y.: \u201cConvolutional pose machines,\u201d Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, pp. 4724\u20134732 (2016). https:\/\/doi.org\/10.1109\/CVPR.2016.511","DOI":"10.1109\/CVPR.2016.511"},{"key":"1832_CR24","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1016\/j.neucom.2019.07.103","volume":"390","author":"J He","year":"2020","unstructured":"He, J., Zhang, C., He, X., Dong, R.: Visual Recognition of traffic police gestures with convolutional pose machine and handcrafted features. Neurocomputing 390, 248\u2013259 (2020). https:\/\/doi.org\/10.1016\/j.neucom.2019.07.103","journal-title":"Neurocomputing"},{"key":"1832_CR25","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.neucom.2022.05.107","volume":"501","author":"S Wang","year":"2022","unstructured":"Wang, S., et al.: Skeleton-based traffic command recognition at road intersections for intelligent vehicles. Neurocomputing 501, 123\u2013134 (2022). https:\/\/doi.org\/10.1016\/j.neucom.2022.05.107","journal-title":"Neurocomputing"},{"key":"1832_CR26","doi-asserted-by":"publisher","unstructured":"Schneider, P., Memmesheimer, R., Kramer, I., Paulus, D.: \u201cGesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping,\u201d Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11531 LNAI, pp. 281\u2013293, (2019). https:\/\/doi.org\/10.1007\/978-3-030-35699-6_22","DOI":"10.1007\/978-3-030-35699-6_22"},{"key":"1832_CR27","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.256","author":"HS Fang","year":"2017","unstructured":"Fang, H.S., Xie, S., Tai, Y.W., Lu, C.: RMPE: Regional Multi-person Pose Estimation. Proc. IEEE Conf. Comput. Vis. (2017). https:\/\/doi.org\/10.1109\/ICCV.2017.256","journal-title":"Proc. IEEE Conf. Comput. Vis."},{"key":"1832_CR28","doi-asserted-by":"publisher","unstructured":"Cao, Z., Hidalgo Martinez, G., Simon, T., Wei, S.-E., Sheikh, Y.A.: \u201cOpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields,\u201d IEEE Trans. Pattern Anal. Mach. Intell., (2019). https:\/\/doi.org\/10.1109\/tpami.2019.2929257.","DOI":"10.1109\/tpami.2019.2929257"},{"key":"1832_CR29","doi-asserted-by":"publisher","unstructured":"Cao, Z., Simon, T., Wei, S.E., Sheikh, Y : \u201cRealtime multi-person 2D pose estimation using part affinity fields,\u201d Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 1302\u20131310 (2017). https:\/\/doi.org\/10.1109\/CVPR.2017.143","DOI":"10.1109\/CVPR.2017.143"},{"key":"1832_CR30","doi-asserted-by":"publisher","unstructured":"Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: \u201cEnd-to-End Recovery of Human Shape and Pose,\u201d in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2018). https:\/\/doi.org\/10.1109\/CVPR.2018.00744","DOI":"10.1109\/CVPR.2018.00744"},{"key":"1832_CR31","unstructured":"Liu, J., Akhtar, N., Mian, A.: \u201cSkepxels: Spatio-temporal Image Representation of Human Skeleton Joints for Action Recognition,\u201d pp. 10\u201319, (2017), [Online]. Available: http:\/\/arxiv.org\/abs\/1711.05941"},{"key":"1832_CR32","doi-asserted-by":"publisher","unstructured":"Lin, T.Y., et al : \u201cMicrosoft COCO: Common objects in context,\u201d Lect. Notes Comput. Sci.(including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 8693(5)740\u2013755 (2014). https:\/\/doi.org\/10.1007\/978-3-319-10602-1_48","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"1832_CR33","doi-asserted-by":"publisher","unstructured":"Singh, M., Mandal, M., Basu, A.: \u201cVisual gesture recognition for ground air traffic control using the radon transform,\u201d 2005 IEEE\/RSJ Int. Conf. Intell. Robot. Syst. IROS, pp. 2850\u20132855, (2005). https:\/\/doi.org\/10.1109\/IROS.2005.1545408","DOI":"10.1109\/IROS.2005.1545408"},{"issue":"August","key":"1832_CR34","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1007\/978-3-030-85540-6_50","volume":"319","author":"C Blackett","year":"2022","unstructured":"Blackett, C., Fernandes, A., Teigen, E., Thoresen, T.: Effects of Signal Latency on Human Performance in Teleoperations. Lect. Notes Networks Syst. 319(August), 386\u2013393 (2022). https:\/\/doi.org\/10.1007\/978-3-030-85540-6_50","journal-title":"Lect. Notes Networks Syst."},{"key":"1832_CR35","doi-asserted-by":"publisher","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: \u201cDeep residual learning for image recognition,\u201d Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, pp. 770\u2013778, (2016). https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"1832_CR36","doi-asserted-by":"publisher","unstructured":"Breiman, L.: \u201cRandom forests,\u201d Random For., pp. 1\u2013122, (2001), doi: https:\/\/doi.org\/10.1201\/9780367816377-11","DOI":"10.1201\/9780367816377-11"}],"container-title":["Journal of Intelligent &amp; Robotic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-023-01832-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10846-023-01832-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-023-01832-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,18]],"date-time":"2023-05-18T09:11:01Z","timestamp":1684401061000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10846-023-01832-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3]]},"references-count":36,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["1832"],"URL":"https:\/\/doi.org\/10.1007\/s10846-023-01832-3","relation":{},"ISSN":["0921-0296","1573-0409"],"issn-type":[{"value":"0921-0296","type":"print"},{"value":"1573-0409","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3]]},"assertion":[{"value":"16 February 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 May 2023","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Missing Open Access funding information has been added in the Funding Note.","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that this work is original and does not include experiments with animals.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"All individuals participating in the study provided an informed consent. The captured information has been nonetheless adequately anonymized.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"The participants in the experiments provided informed consent for publication of the related images. Nevertheless, their faces or any other biometric data can be recognised in the relevant images.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}],"article-number":"44"}}