{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T16:11:58Z","timestamp":1772208718512,"version":"3.50.1"},"reference-count":54,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T00:00:00Z","timestamp":1640217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005632","name":"Narodowe Centrum Bada\u0144 i Rozwoju","doi-asserted-by":"publisher","award":["DZP\/BIOSTRATEG-II\/390\/2015"],"award-info":[{"award-number":["DZP\/BIOSTRATEG-II\/390\/2015"]}],"id":[{"id":"10.13039\/501100005632","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Recent developments in computer hardware made it possible to assess the viability of permutation-based approaches in image classification. Such approaches sample a reference dataset multiple times in order to train an arbitrary number of machine learning models while assessing their accuracy. So-called iterative accuracy assessment techniques or Monte-Carlo-based approaches can be a useful tool when it comes to assessment of algorithm\/model performance but are lacking when it comes to actual image classification and map creation. Due to the multitude of models trained, one has to somehow reason which one of them, if any, should be used in the creation of a map. This poses an interesting challenge since there is a clear disconnect between algorithm assessment and the act of map creation. Our work shows one of the ways this disconnect can be bridged. We calculate how often a given pixel was classified as given class in all variations of a multitude of post-classification images delivered by models trained during the iterative assessment procedure. As a classification problem, a mapping of Calamagrostis epigejos, Rubus spp., Solidago spp. invasive plant species using three HySpex hyperspectral datasets collected in June, August and September was used. As a classification algorithm, the support vector machine approach was chosen, with training hyperparameters obtained using a grid search approach. The resulting maps obtained F1-scores ranging from 0.87 to 0.89 for Calamagrostis epigejos, 0.89 to 0.97 for Rubus spp. and 0.99 for Solidago spp.<\/jats:p>","DOI":"10.3390\/rs14010064","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T21:40:21Z","timestamp":1640295621000},"page":"64","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Mapping Invasive Plant Species with Hyperspectral Data Based on Iterative Accuracy Assessment Techniques"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8954-5047","authenticated-orcid":false,"given":"Anita","family":"Sabat-Tomala","sequence":"first","affiliation":[{"name":"Department of Geoinformatics, Cartography and Remote Sensing, Faculty of Geography and Regional Studies, University of Warsaw, ul. Krakowskie Przedmie\u015bcie 30, 00-927 Warsaw, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4843-9955","authenticated-orcid":false,"given":"Edwin","family":"Raczko","sequence":"additional","affiliation":[{"name":"Department of Geoinformatics, Cartography and Remote Sensing, Faculty of Geography and Regional Studies, University of Warsaw, ul. Krakowskie Przedmie\u015bcie 30, 00-927 Warsaw, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7882-5318","authenticated-orcid":false,"given":"Bogdan","family":"Zagajewski","sequence":"additional","affiliation":[{"name":"Department of Geoinformatics, Cartography and Remote Sensing, Faculty of Geography and Regional Studies, University of Warsaw, ul. Krakowskie Przedmie\u015bcie 30, 00-927 Warsaw, Poland"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,23]]},"reference":[{"key":"ref_1","unstructured":"Tokarska-Guzik, B., Dajdok, Z., Zaj\u0105c, M., Zaj\u0105c, A., Urbisz, A., Danielewicz, W., and Ho\u0142dy\u0144ski, C. (2012). Ro\u015bliny Obcego Pochodzenia w Polsce ze Szczeg\u00f3lnym Uwzgl\u0119dnieniem Gatunk\u00f3w Inwazyjnych."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.tree.2008.10.007","article-title":"Something in the way you move: Dispersal pathways affect invasion success","volume":"24","author":"Wilson","year":"2009","journal-title":"Trends Ecol. Evol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1111\/1365-2664.13025","article-title":"Developing a framework of minimum standards for the risk assessment of alien species","volume":"55","author":"Roy","year":"2017","journal-title":"J. Appl. Ecol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"151","DOI":"10.3391\/mbi.2019.10.1.10","article-title":"Modelling hot spot areas for the invasive alien plant Elodea nuttallii in the EU","volume":"10","author":"Steen","year":"2019","journal-title":"Manag. Biol. Invasions"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1126\/science.1171111","article-title":"Will Threat of Biological Invasions Unite the European Union?","volume":"324","author":"Hulme","year":"2009","journal-title":"Science"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1016\/S0034-4257(02)00061-5","article-title":"Estimation of leafy spurge cover from hyperspectral imagery using mixture tuned matched filtering","volume":"82","author":"Hunt","year":"2002","journal-title":"Remote Sens. Environ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/S0034-4257(03)00096-8","article-title":"Mapping nonnative plants using hyperspectral imagery","volume":"86","author":"Underwood","year":"2003","journal-title":"Remote Sens. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4301","DOI":"10.1016\/j.rse.2008.07.016","article-title":"The role of environmental context in mapping invasive plants with hyperspectral image data","volume":"112","author":"Andrew","year":"2008","journal-title":"Remote Sens. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5926","DOI":"10.1080\/01431161.2013.799790","article-title":"Early detection of the invasive alien plant Solidago altissima in moist tall grassland using hyperspectral imagery","volume":"34","author":"Ishii","year":"2013","journal-title":"Int. J. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2046","DOI":"10.3390\/rs70202046","article-title":"Classification of Herbaceous Vegetation Using Airborne Hyperspectral Imagery","volume":"7","author":"Burai","year":"2015","journal-title":"Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1839","DOI":"10.1080\/01431161.2016.1274447","article-title":"Subalpine and alpine vegetation classification based on hyperspectral APEX and simulated EnMAP images","volume":"38","author":"Zagajewski","year":"2017","journal-title":"Int. J. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Labont\u00e9, J., Drolet, G., Sylvain, J.-D., Thiffault, N., H\u00e9bert, F., and Girard, F. (2020). Phenology-Based Mapping of an Alien Invasive Species Using Time Series of Multispectral Satellite Data: A Case-Study with Glossy Buckthorn in Qu\u00e9bec, Canada. Remote Sens., 12.","DOI":"10.3390\/rs12060922"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Abeysinghe, T., Simic Milas, A., Arend, K., Hohman, B., Reil, P., Gregory, A., and V\u00e1zquez-Ortega, A. (2019). Mapping Invasive Phragmites australis in the Old Woman Creek Estuary Using UAV Remote Sensing and Machine Learning Classifiers. Remote Sens., 11.","DOI":"10.3390\/rs11111380"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2861","DOI":"10.1080\/01431161.2016.1271480","article-title":"Mapping of invasive phragmites (common reed) in Gulf of Mexico coastal wetlands using multispectral imagery and small unmanned aerial systems","volume":"38","author":"Samiappan","year":"2016","journal-title":"Int. J. Remote Sens."},{"key":"ref_15","first-page":"198","article-title":"Feature level image fusion of optical imagery and Synthetic Aperture Radar (SAR) for invasive alien plant species detection and mapping","volume":"10","author":"Rajah","year":"2018","journal-title":"Remote Sens. Appl. Soc. Environ."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Marcinkowska-Ochtyra, A., Zagajewski, B., Raczko, E., Ochtyra, A., and Jaroci\u0144ska, A. (2018). Classification of High-Mountain Vegetation Communities within a Diverse Giant Mountains Ecosystem Using Airborne APEX Hyperspectral Imagery. Remote Sens., 10.","DOI":"10.3390\/rs10040570"},{"key":"ref_17","first-page":"391","article-title":"Mapping Invasive Lupinus polyphyllus Lindl. in Semi-natural Grasslands Using Object-Based Image Analysis of UAV-borne Images","volume":"88","author":"Wijesingha","year":"2020","journal-title":"PFG J. Photogramm. Remote Sens. Geoinf. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2562","DOI":"10.1002\/ece3.4919","article-title":"Performances of machine learning algorithms for mapping fractional cover of an invasive plant species in a dryland ecosystem","volume":"9","author":"Shiferaw","year":"2019","journal-title":"Ecol. Evol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/JSTARS.2013.2257988","article-title":"Mapping Bugweed (Solanum mauritianum) Infestations in Pinus patula Plantations Using Hyperspectral Imagery and Support Vector Machines","volume":"7","author":"Atkinson","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1007\/s11273-020-09719-y","article-title":"Application of airborne hyperspectral data for mapping of invasive alien Spiraea tomentosa L.: A serious threat to peat bog plant communities","volume":"28","author":"Niedzielko","year":"2020","journal-title":"Wetl. Ecol. Manag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5739","DOI":"10.1080\/01431161.2018.1506951","article-title":"Assessment of PlanetScope images for benthic habitat and seagrass species mapping in a complex optically shallow water environment","volume":"39","author":"Wicaksono","year":"2018","journal-title":"Int. J. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sothe, C., Dalponte, M., De Almeida, C.M., Schimalski, M.B., Lima, C.L., Liesenberg, V., Miyoshi, G.T., and Tommaselli, A.M.G. (2019). Tree Species Classification in a Highly Diverse Subtropical Forest Integrating UAV-Based Photogrammetric Point Cloud and Hyperspectral Data. Remote Sens., 11.","DOI":"10.3390\/rs11111338"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2971","DOI":"10.1109\/JSTARS.2015.2432037","article-title":"Fusion of Hyperspectral and LiDAR Remote Sensing Data Using Multiple Feature Learning","volume":"8","author":"Khodadadzadeh","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_24","unstructured":"Nasrabadi, N.M. (2007, January 18\u201322). Kernel-Based Spectral Matched Signal Detectors for Hyperspectral Target Detection. Proceedings of the Lecture Notes in Computernat Scional Conference on Pattern Recognition and Machine Intelligence, Kolkata, India."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kwan, C., Gribben, D., Ayhan, B., Li, J., Bernabe, S., and Plaza, A. (2020). An Accurate Vegetation and Non-Vegetation Differentiation Approach Based on Land Cover Classification. Remote Sens., 12.","DOI":"10.3390\/rs12233880"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, M., and Stein, A. (2020). Mapping Land Use from High Resolution Satellite Images by Exploiting the Spatial Arrangement of Land Cover Objects. Remote Sens., 12.","DOI":"10.3390\/rs12244158"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_28","first-page":"49","article-title":"A framework for mapping tree species combining hyperspectral and LiDAR data: Role of selected classifiers and sensor across three spatial scales","volume":"26","author":"Ghosh","year":"2014","journal-title":"Int. J. Appl. Earth Obs. Geoinform."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.rse.2018.02.026","article-title":"A comparison of resampling methods for remote sensing classification and accuracy assessment","volume":"208","author":"Lyons","year":"2018","journal-title":"Remote Sens. Environ."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Raczko, E., and Zagajewski, B. (2018). Tree Species Classification of the UNESCO Man and the Biosphere Karkonoski National Park (Poland) Using Artificial Neural Networks and APEX Hyperspectral Images. Remote Sens., 10.","DOI":"10.3390\/rs10071111"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Laso, F.J., Ben\u00edtez, F.L., Rivas-Torres, G., Sampedro, C., and Arce-Nazario, J. (2019). Land Cover Classification of Complex Agroecosystems in the Non-Protected Highlands of the Galapagos Islands. Remote Sens., 12.","DOI":"10.3390\/rs12010065"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sabat-Tomala, A., Raczko, E., and Zagajewski, B. (2020). Comparison of Support Vector Machine and Random Forest Algorithms for Invasive and Expansive Species Classification Using Airborne Hyperspectral Data. Remote Sens., 12.","DOI":"10.3390\/rs12030516"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Paz-Kagan, T., Silver, M., Panov, N., and Karnieli, A. (2019). Multispectral Approach for Identifying Invasive Plant Species Based on Flowering Phenology Characteristics. Remote Sens., 11.","DOI":"10.3390\/rs11080953"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hunter, F.D., Mitchard, E.T., Tyrrell, P., and Russell, S. (2020). Inter-Seasonal Time Series Imagery Enhances Classification Accuracy of Grazing Resource and Land Degradation Maps in a Savanna Ecosystem. Remote Sens., 12.","DOI":"10.3390\/rs12010198"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"102345","DOI":"10.1016\/j.apgeog.2020.102345","article-title":"How to effectively use long-term remotely sensed data to analyze the process of tree and shrub encroachment into open protected wetlands","volume":"125","year":"2020","journal-title":"Appl. Geogr."},{"key":"ref_36","unstructured":"Lillesand, T., Kiefer, R.W., and Chipman, J. (2015). Remote Sensing and Image Interpretation, John Wiley and Sons."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ. Psychol. Meas."},{"key":"ref_38","first-page":"1","article-title":"The truth of the F-measure","volume":"1","author":"Sasaki","year":"2007","journal-title":"Teach. Tutor Mater."},{"key":"ref_39","unstructured":"Van Rijsbergen, C.J. (1979). Information Retrieval, Butterworth-Heinemann. [2nd ed.]."},{"key":"ref_40","first-page":"1","article-title":"Analysis of classification-based policy iteration algorithms","volume":"17","author":"Lazaric","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.rse.2018.11.014","article-title":"Joint Deep Learning for land cover and land use classification","volume":"221","author":"Zhang","year":"2019","journal-title":"Remote Sens. Environ."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2989","DOI":"10.1109\/TAC.2015.2418411","article-title":"Classification-Based Approximate Policy Iteration","volume":"60","author":"Farahmand","year":"2015","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1111\/jvs.12850","article-title":"Trait-based numerical classification of mesic and wet grasslands in Poland","volume":"31","author":"Lengyel","year":"2020","journal-title":"J. Veg. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Dabija, A., Kluczek, M., Zagajewski, B., Raczko, E., Kycko, M., Al-Sulttani, A., Tard\u00e0, A., Pineda, L., and Corbera, J. (2021). Comparison of Support Vector Machines and Random Forests for Corine Land Cover Mapping. Remote Sens., 13.","DOI":"10.3390\/rs13040777"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1080\/22797254.2017.1299557","article-title":"Comparison of support vector machine, random forest and neural network classifiers for tree species classification on airborne hyperspectral APEX images","volume":"50","author":"Raczko","year":"2017","journal-title":"Eur. J. Remote Sens."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Jaroci\u0144ska, A., Kope\u0107, D., Tokarska-Guzik, B., and Raczko, E. (2020). Intra-annual variabilities of Rubus caesius L. discrimination on hyperspectral and LiDAR data. Remote Sens., 13.","DOI":"10.3390\/rs13010107"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.isprsjprs.2007.01.004","article-title":"Discrimination of blackberry (Rubus fruticosus sp. agg.) using hyperspectral imagery in Kosciuszko National Park, NSW, Australia","volume":"62","author":"Dehaan","year":"2007","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1080\/03736245.2018.1461683","article-title":"Evaluating the potential of freely available multispectral remotely sensed imagery in mapping American bramble (Rubus cuneifolius)","volume":"100","author":"Rajah","year":"2018","journal-title":"S. Afr. Geogr. J."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Marcinkowska-Ochtyra, A., Jaroci\u0144ska, A., Bzd\u0119ga, K., and Tokarska-Guzik, B. (2018). Classification of expansive grassland species in different growth stages based on hyperspectral and LiDAR data. Remote Sens., 10.","DOI":"10.3390\/rs10122019"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Kope\u0107, D., Zakrzewska, A., Halladin-D\u0105browska, A., Wylaz\u0142owska, J., Kania, A., and Niedzielko, J. (2019). Using airborne hyperspectral imaging spectroscopy to accurately monitor invasive and expansive herb plants: Limitations and requirements of the method. Sensors, 19.","DOI":"10.3390\/s19132871"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"612","DOI":"10.3390\/rs5020612","article-title":"Remote Distinction of A Noxious Weed (Musk Thistle: CarduusNutans) Using Airborne Hyperspectral Imagery and the Support Vector Machine Classifier","volume":"5","author":"Mirik","year":"2013","journal-title":"Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Migas-Mazur, R., Kycko, M., Zwijacz-Kozica, T., and Zagajewski, B. (2021). Assessment of Sentinel-2 Images, Support Vector Machines and Change Detection Algorithms for Bark Beetle Outbreaks Mapping in the Tatra Mountains. Remote Sens., 13.","DOI":"10.3390\/rs13163314"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Zagajewski, B., Kluczek, M., Raczko, E., Njegovec, A., Dabija, A., and Kycko, M. (2021). Comparison of Random Forest, Support Vector Machines, and Neural Networks for Post-Disaster Forest Species Mapping of the Krkono\u0161e\/Karkonosze Transboundary Biosphere Reserve. Remote Sens., 13.","DOI":"10.3390\/rs13132581"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"3604","DOI":"10.5586\/asbp.3604","article-title":"Feasibility of hyperspectral vegetation indices for the detection of chlorophyll concentration in three high Arctic plants: Salix polaris, Bistorta vivipara, and Dryas octopetala","volume":"87","author":"Zagajewski","year":"2018","journal-title":"Acta Soc. Bot. Pol."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/64\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:52:28Z","timestamp":1760169148000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/64"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,23]]},"references-count":54,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["rs14010064"],"URL":"https:\/\/doi.org\/10.3390\/rs14010064","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,23]]}}}