{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:00:46Z","timestamp":1760238046482,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2020,7,1]],"date-time":"2020-07-01T00:00:00Z","timestamp":1593561600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010676","name":"H2020 Societal Challenges","doi-asserted-by":"publisher","award":["787021"],"award-info":[{"award-number":["787021"]}],"id":[{"id":"10.13039\/100010676","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Austrian security research programme KIRAS","award":["867026"],"award-info":[{"award-number":["867026"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Despite recent advances in image and video processing, the detection of people or cars in areas of dense vegetation is still challenging due to landscape, illumination changes and strong occlusion. In this paper, we address this problem with the use of a hyperspectral camera\u2014installed on the ground or possibly a drone\u2014and detection based on spectral signatures. We introduce a novel automatic method for annotating spectral signatures based on a combination of state-of-the-art deep learning methods. After we collected millions of samples with our method, we used a deep learning approach to train a classifier to detect people and cars. Our results show that, based only on spectral signature classification, we can achieve an Matthews Correlation Coefficient of 0.83. We evaluate our classification method in areas with varying vegetation and discuss the limitations and constraints that the current hyperspectral imaging technology has. We conclude that spectral signature classification is possible with high accuracy in uncontrolled outdoor environments. Nevertheless, even with state-of-the-art compact passive hyperspectral imaging technology, high dynamic range of illumination and relatively low image resolution continue to pose major challenges when developing object detection algorithms for areas of dense vegetation.<\/jats:p>","DOI":"10.3390\/rs12132111","type":"journal-article","created":{"date-parts":[[2020,7,2]],"date-time":"2020-07-02T02:44:25Z","timestamp":1593657865000},"page":"2111","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Automatic Annotation of Hyperspectral Images and Spectral Signal Classification of People and Vehicles in Areas of Dense Vegetation with Deep Learning"],"prefix":"10.3390","volume":"12","author":[{"given":"Adam","family":"Papp","sequence":"first","affiliation":[{"name":"AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Julian","family":"Pegoraro","sequence":"additional","affiliation":[{"name":"AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Bauer","sequence":"additional","affiliation":[{"name":"AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philip","family":"Taupe","sequence":"additional","affiliation":[{"name":"AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christoph","family":"Wiesmeyr","sequence":"additional","affiliation":[{"name":"AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andreas","family":"Kriechbaum-Zabini","sequence":"additional","affiliation":[{"name":"AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ad\u00e3o, T., Hru\u0161ka, J., P\u00e1dua, L., Bessa, J., Peres, E., Morais, R., and Sousa, J.J. (2017). Hyperspectral imaging: A review on UAV-based sensors, data processing and applications for agriculture and forestry. Remote Sens., 9.","DOI":"10.3390\/rs9111110"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kanning, M., K\u00fchling, I., Trautz, D., and Jarmer, T. (2018). High-resolution UAV-based hyperspectral imagery for LAI and chlorophyll estimations from wheat for yield prediction. Remote Sens., 10.","DOI":"10.3390\/rs10122000"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.compag.2017.11.027","article-title":"A novel approach for vegetation classification using UAV-based hyperspectral imaging","volume":"144","author":"Ishida","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.isprsjprs.2015.08.002","article-title":"Generating 3D hyperspectral information with lightweight UAV snapshot cameras for vegetation monitoring: From camera calibration to quality assurance","volume":"108","author":"Aasen","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Guo, M., Li, J., Sheng, C., Xu, J., and Wu, L. (2017). A review of wetland remote sensing. Sensors, 17.","DOI":"10.3390\/s17040777"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cao, J., Leng, W., Liu, K., Liu, L., He, Z., and Zhu, Y. (2018). Object-based mangrove species classification using unmanned aerial vehicle hyperspectral images and digital surface models. Remote Sens., 10.","DOI":"10.3390\/rs10010089"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"11434","DOI":"10.3390\/rs70911434","article-title":"Use of a portable camera for proximal soil sensing with hyperspectral image data","volume":"7","author":"Jung","year":"2015","journal-title":"Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1016\/j.tifs.2007.06.001","article-title":"Hyperspectral imaging\u2014An emerging process analytical tool for food quality and safety control","volume":"18","author":"Gowen","year":"2007","journal-title":"Trends Food Sci. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Al-Sarayreh, M., Reis, M.M., Yan, W.Q., and Klette, R. (2018, January 19\u201321). Deep spectral-spatial features of snapshot hyperspectral images for red-meat classification. Proceedings of the 2018 International Conference on Image and Vision Computing New Zealand (IVCNZ), Auckland, New Zealand.","DOI":"10.1109\/IVCNZ.2018.8634783"},{"key":"ref_10","unstructured":"Bradley, D., Thayer, S., Stentz, A., and Rander, P. (2004). Vegetation Detection for Mobile Robot Navigation, Robotics Institute, Carnegie Mellon University. Technical Report CMU-RI-TR-04-12."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gr\u00f6nwall, C., Hamoir, D., Steinvall, O., Larsson, H., Amselem, E., Lutzmann, P., Repasi, E., G\u00f6hler, B., Barb\u00e9, S., and Vaudelin, O. (2013, January 25\u201326). Measurements and analysis of active\/passive multispectral imaging. Proceedings of the Electro-Optical Remote Sensing, Photonic Technologies, and Applications VII; and Military Applications in Hyperspectral Imaging and High Spatial Resolution Sensing, Dresden, Germany.","DOI":"10.1117\/12.2028647"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"87","DOI":"10.5194\/isprs-archives-XLII-3-W3-87-2017","article-title":"Multimodal data fusion for effective surveillance of critical infrastructure","volume":"42","author":"Kandylakisa","year":"2017","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Puttonen, E., Hakala, T., Nevalainen, O., Kaasalainen, S., Krooks, A., Karjalainen, M., and Anttila, K. (2015). Artificial target detection with a hyperspectral LiDAR over 26-h measurement. Opt. Eng., 54.","DOI":"10.1117\/1.OE.54.1.013105"},{"key":"ref_14","unstructured":"Winkens, C., Sattler, F., and Paulus, D. (March, January 27). Hyperspectral Terrain Classification for Ground Vehicles. Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (5: VISAPP), Porto, Portugal."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cavigelli, L., Bernath, D., Magno, M., and Benini, L. (2016, January 26\u201327). Computationally efficient target classification in multispectral image data with Deep Neural Networks. Proceedings of the SPIE Target and Background Signatures II, Edinburgh, UK.","DOI":"10.1117\/12.2241383"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Howell, J.R., Menguc, M.P., and Siegel, R. (2015). Thermal Radiation Heat Transfer, CRC Press.","DOI":"10.1201\/b18835"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Meerdink, S.K., Hook, S.J., Roberts, D.A., and Abbott, E.A. (2019). The ECOSTRESS spectral library version 1.0. Remote Sens. Environ., 230.","DOI":"10.1016\/j.rse.2019.05.015"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1016\/j.rse.2008.11.007","article-title":"The ASTER spectral library version 2.0","volume":"113","author":"Baldridge","year":"2009","journal-title":"Remote Sens. Environ."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Geelen, B., Tack, N., and Lambrechts, A. (2014, January 3\u20135). A compact snapshot multispectral imager with a monolithically integrated per-pixel filter mosaic. Proceedings of the SPIE Advanced Fabrication Technologies for Micro\/Nano Optics and Photonics VII, San Francisco, CA, USA.","DOI":"10.1117\/12.2037607"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the IEEE international Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_22","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2020, June 29). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: https:\/\/static.googleusercontent.com\/media\/research.google.com\/en\/\/pubs\/archive\/45166.pdf."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/13\/2111\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:46:07Z","timestamp":1760175967000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/13\/2111"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,1]]},"references-count":22,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2020,7]]}},"alternative-id":["rs12132111"],"URL":"https:\/\/doi.org\/10.3390\/rs12132111","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2020,7,1]]}}}