{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T10:35:08Z","timestamp":1769164508894,"version":"3.49.0"},"reference-count":42,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2018,3,17]],"date-time":"2018-03-17T00:00:00Z","timestamp":1521244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Spanish Ministry of Economy and Competitiveness through the project VA4UAV","award":["DPI2014-60139-R"],"award-info":[{"award-number":["DPI2014-60139-R"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Aerial robots with cameras on board can be used in surface inspection to observe areas that are difficult to reach by other means. In this type of problem, it is desirable for aerial robots to have a high degree of autonomy. A way to provide more autonomy would be to use computer vision techniques to automatically detect anomalies on the surface. However, the performance of automated visual recognition methods is limited in uncontrolled environments, so that in practice it is not possible to perform a fully automatic inspection. This paper presents a solution for visual inspection that increases the degree of autonomy of aerial robots following a semi-automatic approach. The solution is based on human-robot collaboration in which the operator delegates tasks to the drone for exploration and visual recognition and the drone requests assistance in the presence of uncertainty. We validate this proposal with the development of an experimental robotic system using the software framework Aerostack. The paper describes technical challenges that we had to solve to develop such a system and the impact on this solution on the degree of autonomy to detect anomalies on the surface.<\/jats:p>","DOI":"10.3390\/s18030893","type":"journal-article","created":{"date-parts":[[2018,3,20]],"date-time":"2018-03-20T06:57:11Z","timestamp":1521529031000},"page":"893","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["A Collaborative Approach for Surface Inspection Using Aerial Robots and Computer Vision"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7145-1974","authenticated-orcid":false,"given":"Martin","family":"Molina","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence, Technical University of Madrid, Boadilla del Monte, Madrid 28660, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pedro","family":"Frau","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, Technical University of Madrid, Boadilla del Monte, Madrid 28660, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dario","family":"Maravall","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, Technical University of Madrid, Boadilla del Monte, Madrid 28660, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,3,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1006\/cviu.1995.1017","article-title":"A survey of automated visual inspection","volume":"61","author":"Newman","year":"1995","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1186\/1687-5281-2014-50","article-title":"Review of vision-based steel surface inspection systems","volume":"2014","author":"Neogi","year":"2014","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Xie, X. (2008). A review of recent advances in surface defect detection using texture analysis techniques. ELCVIA Electron. Lett. Comput. Vis. Image Anal., 7.","DOI":"10.5565\/rev\/elcvia.268"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"817473","DOI":"10.1155\/2010\/817473","article-title":"On the evaluation of texture and colour features for nondestructive corrosion detection","volume":"2010","author":"Medeiros","year":"2010","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Medina, R., Llamas, J., and Zalama, E. (2017). Crack detection in concrete tunnels using a Gabor Filter invariant to rotation. Sensors, 17.","DOI":"10.3390\/s17071670"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1007\/s11760-017-1181-9","article-title":"A vision-based system for robotic inspection of marine vessels","volume":"12","author":"Maglietta","year":"2017","journal-title":"Signal Image Video Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"07003","DOI":"10.1051\/matecconf\/201710307003","article-title":"Visual Inspection of Historical Buildings Using Micro UAV","volume":"Volume 103","author":"Kaamin","year":"2017","journal-title":"MATEC Web of Conferences"},{"key":"ref_8","unstructured":"Hallermann, N., Morgenthal, G., and Rodehorst, V. (2015, January 15\u201317). Unmanned Aerial Systems (UAS)\u2013Case Studies of Vision Based Monitoring of Ageing Structures. Proceedings of the International Symposium Non-Destructive Testing in Civil Engineering, Berlin, Germany."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"687","DOI":"10.12720\/jcm.9.9.687-692","article-title":"Unmanned aerial vehicles for power line inspection: A cooperative way in platforms and communications","volume":"9","author":"Deng","year":"2014","journal-title":"J. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s10846-013-9852-4","article-title":"Vessel inspection: A micro-aerial vehicle-based approach","volume":"76","author":"Ortiz","year":"2014","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_11","unstructured":"Merino, L., Caballero, F., and Ollero, A. (2005, January 18\u201322). Cooperative fire detection using unmanned aerial vehicles. Proceedings of the ICRA 2005, Barcelona, Spain."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"\u00d6zaslan, T., Shen, S., Mulgaonkar, Y., Michael, N., and Kumar, V. (2015). Inspection of penstocks and featureless tunnel-like environments using micro UAVs. Field and Service Robotics, Springer.","DOI":"10.1007\/978-3-319-07488-7_9"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1561\/1100000005","article-title":"Human-Robot Interaction: A Survey","volume":"1","author":"Goodrich","year":"2007","journal-title":"Found. Trends Hum. Comput. Interact."},{"key":"ref_14","unstructured":"Suzuki, T., Sekine, T., Fujii, T., Asama, H., and Endo, I. (2000, January 12\u201315). Cooperative formation among mobile robot teleoperation in inspection task. Proceedings of the 2000 IEEE Conference on Decision and Control, Sydney, Australia."},{"key":"ref_15","unstructured":"Kimura, H., and Ikeuchi, T. (1999, January 17\u201321). Task-model based Human-Robot Cooperation using vision. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems 1999\u2014IROS 99, Kyonju, Korea."},{"key":"ref_16","unstructured":"Sheridan, T.B. (1992). Telerobotics, Automation, and Human Supervisory Control, MIT Press."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1518\/001872007779598037","article-title":"Designing for flexible interaction between humans and automation: Delegation interfaces for supervisory control","volume":"49","author":"Miller","year":"2007","journal-title":"Hum. Factors J. Hum. Factors Ergon. Soc."},{"key":"ref_18","unstructured":"Cummings, M. (, January June). Human supervisory control of swarming networks. Proceedings of the 2nd Annual Swarming: Autonomous Intelligent Networked Systems Conference, Arlington, VA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1220","DOI":"10.1109\/TIE.2005.855694","article-title":"An application of Petri nets to supervisory control for human-computer interactive systems","volume":"52","author":"Lee","year":"2005","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Searle, J.R. (1969). Speech Acts, Cambridge University Press.","DOI":"10.1017\/CBO9781139173438"},{"key":"ref_21","unstructured":"Searle, J.R. (1975). Language Mind and Knowledge Minnesota, University of Minnesota Press. Studies in the Philosophy of Science Chapter A Taxonomy of Illocutionary Acts."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Austin, J.L. (1975). How to Do Things with Words, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780198245537.001.0001"},{"key":"ref_23","unstructured":"Murphy, R. (2000). Introduction to AI robotics, MIT Press."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Castelfranchi, C. (1994). Guarantees for autonomy in cognitive agent architecture. Intelligent Agents, Springer.","DOI":"10.1007\/3-540-58855-8_3"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1109\/TIE.2003.814768","article-title":"Multi-robot remote driving with collaborative control","volume":"50","author":"Fong","year":"2003","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Horvitz, E. (1999, January 15\u201320). Principles of mixed-initiative user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh, PA, USA.","DOI":"10.1145\/302979.303030"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fong, T., Thorpe, C., and Baur, C. (2002). Robot as partner: Vehicle teleoperation with collaborative control. Multi-Robot Systems: From Swarms to Intelligent Automata, Springer.","DOI":"10.1007\/978-94-017-2376-3_21"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1109\/TSMCA.2005.850587","article-title":"Validating human-robot interaction schemes in multitasking environments","volume":"35","author":"Crandall","year":"2005","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_29","unstructured":"Cheng, X. (2003). Image Segmentation and Pattern Recognition. A novel concept: The histogram of connected elements. Pattern Recognition and String Matching, Kluwer Academic Publishers."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1016\/j.imavis.2006.05.020","article-title":"A novel generalization of the grayscale histogram and its application to the automated visual measurement and inspection of wooden pallets","volume":"25","author":"Maravall","year":"2007","journal-title":"Image Vis. Comput."},{"key":"ref_31","first-page":"314","article-title":"Segmentation of Traffic images for automatic car driving","volume":"Volume 2809","author":"Moreno","year":"2004","journal-title":"Computer\u2014Aided Systems Theory. EUROCAST 2003"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sanchez-Lopez, J.L., Suarez-Fernandez, R.A., Bavle, H., Sampedro, C., Molina, M., Pestana, J., and Campoy, P. (2016, January 7\u201310). AEROSTACK: An Architecture and Open-Source Software Framework for Aerial Robotics. Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA.","DOI":"10.1109\/ICUAS.2016.7502591"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1007\/s10846-017-0551-4","article-title":"A Multi-Layered Component-Based Approach for the Development of Aerial Robotic Systems: The Aerostack Framework","volume":"88","author":"Molina","year":"2017","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_34","unstructured":"Arkin, R.C., Riseman, E.M., and Hansen, A. (1987, January 23). AuRA: An Architecture for Vision-Based Robot Navigation. Proceedings of the DARPA Image Understanding Work-shop, Los Angeles, CA, USA."},{"key":"ref_35","unstructured":"Kortenkamp, D., Bonnasso, R.P., and Murphy, R. (1998). On Three-Layer Architectures. Artificial Intelligence and Mobile Robots, AAAI MIT Press."},{"key":"ref_36","unstructured":"Duffy, B.R. (2001). The Social Robot. [Ph.D. Thesis, Department of Computer Science, University College Dublin]."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/JRA.1986.1087032","article-title":"A Robust Layer Control System for a Mobile Robot","volume":"2","author":"Brooks","year":"1986","journal-title":"IEEE J. Robot. Automat."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Mataric, M. (1994). Interaction and intelligent behavior. [Ph.D. Thesis, MIT].","DOI":"10.21236\/ADA290049"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Michaud, F., and Nicolescu, M. (2016). Behaviour-based systems. Springer Handbook of Robotics, Springer International Publishing.","DOI":"10.1007\/978-3-319-32552-1_13"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1108\/IJICC-03-2017-0025","article-title":"TML: A language to specify aerial robotic missions for the framework Aerostack","volume":"10","author":"Molina","year":"2017","journal-title":"Int. J. Intell. Comput. Cybern."},{"key":"ref_41","unstructured":"Molina, M., Diaz-Moreno, A., Palacios, D., Suarez-Fernandez, R.A., Sanchez-Lopez, J.L., Sampedro, C., Bavle, H., and Campoy, P. (2016, January 17\u201321). Specifying Complex Missions for Aerial Robotics in Dynamic Environments. Proceedings of the International Micro Air Vehicle Conference and Competition, IMAV 2016, Beijing, China."},{"key":"ref_42","unstructured":"Molina, M., Frau, P., Maravall, D., Sanchez Lopez, J.L., Bavle, H., and Campoy, P. (2017, January 18\u201321). Human-Robot Cooperation in Surface Inspection Aerial Missions. Proceedings of the 2017 International Micro Air Vehicle Conference and Flight Competition (IMAV), Toulouse, France."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/3\/893\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:57:29Z","timestamp":1760194649000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/3\/893"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,17]]},"references-count":42,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2018,3]]}},"alternative-id":["s18030893"],"URL":"https:\/\/doi.org\/10.3390\/s18030893","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,3,17]]}}}