{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T06:15:24Z","timestamp":1775024124646,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,3,31]],"date-time":"2022-03-31T00:00:00Z","timestamp":1648684800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000844","name":"European Space Agency","doi-asserted-by":"publisher","award":["4000132058\/20\/NL\/MH\/ic : \u2019Cultural Landscapes Scanner (CLS): Earth Observation and automated detection of subsoil undiscovered cultural heritage sites via AI approaches\u2019."],"award-info":[{"award-number":["4000132058\/20\/NL\/MH\/ic : \u2019Cultural Landscapes Scanner (CLS): Earth Observation and automated detection of subsoil undiscovered cultural heritage sites via AI approaches\u2019."]}],"id":[{"id":"10.13039\/501100000844","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Machine Learning-based workflows are being progressively used for the automatic detection of archaeological objects (intended as below-surface sites) in remote sensing data. Despite promising results in the detection phase, there is still a lack of a standard set of measures to evaluate the performance of object detection methods, since buried archaeological sites often have distinctive shapes that set them aside from other types of objects included in mainstream remote sensing datasets (e.g., Dataset of Object deTection in Aerial images, DOTA). Additionally, archaeological research relies heavily on geospatial information when validating the output of an object detection procedure, a type of information that is not normally considered in regular machine learning validation pipelines. This paper tackles these shortcomings by introducing two novel automatic evaluation measures, namely \u2018centroid-based\u2019 and \u2018pixel-based\u2019, designed to encode the salient aspects of the archaeologists\u2019 thinking process. To test their usability, an experiment with different object detection deep neural networks was conducted on a LiDAR dataset. The experimental results show that these two automatic measures closely resemble the semi-automatic one currently used by archaeologists and therefore can be adopted as fully automatic evaluation measures in archaeological remote sensing detection. Adoption will facilitate cross-study comparisons and close collaboration between machine learning and archaeological researchers, which in turn will encourage the development of novel human-centred archaeological object detection tools.<\/jats:p>","DOI":"10.3390\/rs14071694","type":"journal-article","created":{"date-parts":[[2022,3,31]],"date-time":"2022-03-31T21:34:29Z","timestamp":1648762469000},"page":"1694","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Deep Learning for Archaeological Object Detection on LiDAR: New Evaluation Measures and Insights"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1013-5147","authenticated-orcid":false,"given":"Marco","family":"Fiorucci","sequence":"first","affiliation":[{"name":"Center for Cultural Heritage Technology, Istituto Italiano di Tecnologia, 30170 Venice, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1053-3009","authenticated-orcid":false,"given":"Wouter B.","family":"Verschoof-van der Vaart","sequence":"additional","affiliation":[{"name":"Faculty of Archaeology, Leiden University, P.O. Box 9514, 2300 RA Leiden, The Netherlands"}]},{"given":"Paolo","family":"Soleni","sequence":"additional","affiliation":[{"name":"Center for Cultural Heritage Technology, Istituto Italiano di Tecnologia, 30170 Venice, Italy"}]},{"given":"Bertrand","family":"Le Saux","sequence":"additional","affiliation":[{"name":"ESA\/ESRIN, \u03d5-Lab, 00044 Frascati, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4508-1540","authenticated-orcid":false,"given":"Arianna","family":"Traviglia","sequence":"additional","affiliation":[{"name":"Center for Cultural Heritage Technology, Istituto Italiano di Tecnologia, 30170 Venice, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"18240","DOI":"10.1073\/pnas.2005583117","article-title":"Automated detection of archaeological mounds using machine-learning classification of multisensor and multitemporal satellite data","volume":"117","author":"Orengo","year":"2020","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bundzel, M., Ja\u0161\u010dur, M., Kov\u00e1\u010d, M., Lieskovsk\u00fd, T., Sin\u010d\u00e1k, P., and Tk\u00e1\u010dik, T. (2020). Semantic Segmentation of Airborne LiDAR Data in Maya Archaeology. Remote Sens., 12.","DOI":"10.3390\/rs12223685"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Somrak, M., D\u017eeroski, S., and Kokalj, Z. (2020). Learning to Classify Structures in ALS-Derived Visualizations of Ancient Maya Settlements with CNN. Remote Sens., 12.","DOI":"10.3390\/rs12142215"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Soroush, M., Mehrtash, A., Khazraee, E., and Ur, J.A. (2020). Deep Learning in Archaeological Remote Sensing: Automated Qanat Detection in the Kurdistan Region of Iraq. Remote Sens., 12.","DOI":"10.3390\/rs12030500"},{"key":"ref_5","unstructured":"Matsumoto, M., and Uleberg, E. (2018). Semi automatic mapping of charcoal kilns from airborne laser scanning data using deep learning. CAA 2016: Oceans of Data. Proceedings of the 44th Conference on Computer Applications and Quantitative Methods in Archaeology, Archaeopress."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Verschoof-van der Vaart, W.B., Lambers, K., Kowalczyk, W., and Bourgeois, Q.P. (2020). Combining Deep Learning and Location-Based Ranking for Large-Scale Archaeological Prospection of LiDAR Data from The Netherlands. ISPRS Int. J. Geo Inf., 9.","DOI":"10.3390\/ijgi9050293"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4580","DOI":"10.1109\/TGRS.2016.2545919","article-title":"Detection of fragmented rectangular enclosures in very-high-resolution remote sensing images","volume":"54","author":"Zingman","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W., and Frangi, A. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Springer.","DOI":"10.1007\/978-3-319-24571-3"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lambers, K., Verschoof-van der Vaart, W.B., and Bourgeois, Q.P.J. (2019). Integrating Remote Sensing, Machine Learning, and Citizen Science in Dutch Archaeological Prospection. Remote Sens., 11.","DOI":"10.3390\/rs11070794"},{"key":"ref_12","unstructured":"Verschoof-van der Vaart, W.B. (2022). Learning to Look at LiDAR. Combining CNN-Based Object Detection And GIS for Archaeological Prospection in Remotely-Sensed Data. [Ph.D. Thesis, Leiden University]. Available online: https:\/\/hdl.handle.net\/1887\/3256824."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.patrec.2020.02.017","article-title":"Machine Learning for Cultural Heritage: A Survey","volume":"133","author":"Fiorucci","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Randrianarivo, H., Le Saux, B., and Ferecatu, M. (2013, January 21\u201326). Urban structure detection with deformable part-based models. Proceedings of the 2013 IEEE International Geoscience and Remote Sensing Symposium\u2014IGARSS, Melbourne, VIC, Australia.","DOI":"10.1109\/IGARSS.2013.6721126"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1007\/s11263-014-0733-5","article-title":"The Pascal Visual Object Classes Challenge: A Retrospective","volume":"111","author":"Everingham","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ding, J., Xue, N., Xia, G., Bai, X., Yang, W., Yang, M.Y., Belongie, S.J., Luo, J., Datcu, M., and Pelillo, M. (2021). Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges. IEEE Trans. Pattern Anal. Mach. Intell., 1.","DOI":"10.1109\/TPAMI.2021.3117983"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014). Microsoft COCO: Common Objects in Context. Computer Vision\u2014ECCV 2014, Springer International Publishing.","DOI":"10.1007\/978-3-319-10602-1"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Rezatofighi, S.H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I.D., and Savarese, S. (2019, January 15\u201320). Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00075"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Gillings, M., Hacig\u00fczeller, P., and Lock, G. (2020). Archaeology and spatial analysis. Archaeological Spatial Analysis: A Methodological Guide, Routledge. Chapter 1.","DOI":"10.4324\/9781351243858-1"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yoo, D., and Kweon, I.S. (2019, January 15\u201320). Learning Loss for Active Learning. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00018"},{"key":"ref_21","unstructured":"Berendsen, H.J.A. (2004). De Vorming van het Land. Inleiding in de Geologie en de Geomorfologie, Koninklijke Van Gorcum. [4th ed.]."},{"key":"ref_22","first-page":"31","article-title":"Learning to look at LiDAR: The use of R-CNN in the automated detection of archaeological objects in LiDAR data from the Netherlands","volume":"2","author":"Lambers","year":"2019","journal-title":"J. Comput. Appl. Archaeol."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Giligny, F., Djindjian, F., Costa, L., Moscati, P., and Robert, S. (2015). Challenges and Perspectives of Woodland Archaeology Across Europe. CAA2014: 21st Century Archaeology, Concepts, Methods and Tools. Proceedings of the 42nd Annual Conference on Computer Applications and Quantitative Methods in Archaeology, Archaeopress.","DOI":"10.2307\/jj.15135883"},{"key":"ref_24","unstructured":"Nationaal Georegister (2022, March 27). Publieke Dienstverlening Op de Kaart (PDOK). Available online: https:\/\/www.pdok.nl\/."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1017\/ppr.2018.5","article-title":"The Fields that Outlived the Celts: The Use-histories of Later Prehistoric Field Systems (Celtic Fields or Raatakkers) in the Netherlands","volume":"84","author":"Arnoldussen","year":"2018","journal-title":"Proc. Prehist. Soc."},{"key":"ref_26","unstructured":"Bourgeois, Q.P.J. (2013). Monuments on the Horizon. The Formation of the Barrow Landscape throughout the 3rd and 2nd Millennium BC, Sidestone Press."},{"key":"ref_27","first-page":"15","article-title":"Applying automated object detection in archaeological practice: A case study from the southern Netherlands","volume":"29","author":"Lambers","year":"2021","journal-title":"Archaeol. Prospect."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"47","DOI":"10.2458\/azu_rc.57.17925","article-title":"The Tempo of Bronze Age Barrow Use: Modeling the Ebb and Flow in Monumental Funerary Landscapes","volume":"57","author":"Bourgeois","year":"2015","journal-title":"Radiocarbon"},{"key":"ref_29","first-page":"94","article-title":"Theoretical Repositioning of Automated Remote Sensing Archaeology: Shifting from Features to Ephemeral Landscapes","volume":"4","author":"Davis","year":"2021","journal-title":"J. Comput. Appl. Archaeol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Traviglia, A., and Torsello, A. (2017). Landscape Pattern Detection in Archaeological Remote Sensing. Geosciences, 7.","DOI":"10.3390\/geosciences7040128"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1002\/arp.374","article-title":"LiDAR-derived Local Relief Models\u2014A new tool for archaeological prospection","volume":"17","author":"Hesse","year":"2010","journal-title":"Archaeol. Prospect."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Opitz, R., and Cowley, D. (2013). Interpreting Archaeological Topography. Airborne Laser Scanning, 3D Data and Ground Observation, Oxbow Books.","DOI":"10.2307\/j.ctvh1dqdz"},{"key":"ref_33","unstructured":"QGIS Development Team (2022, March 27). QGIS Geographic Information System. Available online: http:\/\/qgis.org."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kokalj, \u017d., and Hesse, R. (2017). Airborne Laser Scanning Raster Data Visualization: A Guide to Good Practice, Zalo\u017eba ZRC.","DOI":"10.3986\/9789612549848"},{"key":"ref_35","unstructured":"van der Zon, N. (2013). Kwaliteitsdocument AHN-2, Rijkswaterstaat. Technical Report."},{"key":"ref_36","unstructured":"Tzutalin (2022, March 27). LabelImg. Git Code., Available online: https:\/\/github.com\/tzutalin\/labelImg."},{"key":"ref_37","unstructured":"Rijksdienst voor het Cultureel Erfgoed (2022, March 27). ArchIS and AMK. Available online: https:\/\/www.cultureelerfgoed.nl\/onderwerpen\/bronnen-en-kaarten\/overzicht."},{"key":"ref_38","unstructured":"Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2022, March 27). Detectron2. Available online: https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Tang, T., Zhou, S., Deng, Z., Zou, H., and Lei, L. (2017). Vehicle Detection in Aerial Images Based on Region Convolutional Neural Networks and Hard Negative Example Mining. Sensors, 17.","DOI":"10.3390\/s17020336"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2016). Aggregated Residual Transformations for Deep Neural Networks. arXiv.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22\u201329). Deformable Convolutional Networks. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.89"},{"key":"ref_47","unstructured":"Matsumoto, M., and Uleberg, E. (2018). Towards a national infrastructure for semi-automatic mapping of cultural heritage in\nNorway. Oceans of Data. Proceedings of the 44th Conference on Computer Applications and Quantitative Methods in Archaeology, Archaeopress."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1002\/arp.1806","article-title":"A modified Mask region-based convolutional neural network approach for the automated detection of archaeological sites on high-resolution light detection and ranging-derived digital elevation models in the North German Lowland","volume":"28","author":"Bonhage","year":"2021","journal-title":"Archaeol. Prospect."},{"key":"ref_49","first-page":"47","article-title":"Interferometric SAR and Machine Learning: Using Open Source Data to Detect Archaeological Looting and Destruction","volume":"4","year":"2021","journal-title":"J. Comput. Appl. Archaeol."},{"key":"ref_50","first-page":"274","article-title":"Implementing Advanced Deep Learning Approaches for Archaeological Object Detection in Remotely-Sensed Data: The Results of Cross-Domain Collaboration","volume":"4","author":"Olivier","year":"2021","journal-title":"J. Comput. Appl. Archaeol."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2218","DOI":"10.1109\/TGRS.2008.2010404","article-title":"Active Learning Methods for Remote Sensing Image Classification","volume":"47","author":"Tuia","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/arp.1731","article-title":"Using deep neural networks on airborne laser scanning data: Results from a case study of semi-automatic mapping of archaeological topography on Arran, Scotland","volume":"26","author":"Trier","year":"2019","journal-title":"Archaeol. Prospect."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1017\/aap.2021.6","article-title":"Machine Learning Arrives in Archaeology","volume":"9","author":"Bickler","year":"2021","journal-title":"Adv. Archaeol. Pract."},{"key":"ref_54","first-page":"109","article-title":"Making LiGHT Work of Large Area Survey? Developing Approaches to Rapid Archaeological Mapping and the Creation of Systematic National-scaled Heritage Data","volume":"3","author":"Cowley","year":"2020","journal-title":"J. Comput. Appl. Archaeol."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/7\/1694\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:47:48Z","timestamp":1760136468000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/7\/1694"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,31]]},"references-count":54,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["rs14071694"],"URL":"https:\/\/doi.org\/10.3390\/rs14071694","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,31]]}}}