{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T18:18:04Z","timestamp":1780337884246,"version":"3.54.1"},"reference-count":53,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,9,7]],"date-time":"2018-09-07T00:00:00Z","timestamp":1536278400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020","doi-asserted-by":"publisher","award":["644227"],"award-info":[{"award-number":["644227"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]},{"name":"The Swiss State Secretariat for Education, Research and Innovation","award":["15.0029"],"award-info":[{"award-number":["15.0029"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The ability to automatically monitor agricultural fields is an important capability in precision farming, enabling steps towards more sustainable agriculture. Precise, high-resolution monitoring is a key prerequisite for targeted intervention and the selective application of agro-chemicals. The main goal of this paper is developing a novel crop\/weed segmentation and mapping framework that processes multispectral images obtained from an unmanned aerial vehicle (UAV) using a deep neural network (DNN). Most studies on crop\/weed semantic segmentation only consider single images for processing and classification. Images taken by UAVs often cover only a few hundred square meters with either color only or color and near-infrared (NIR) channels. Although a map can be generated by processing single segmented images incrementally, this requires additional complex information fusion techniques which struggle to handle high fidelity maps due to their computational costs and problems in ensuring global consistency. Moreover, computing a single large and accurate vegetation map (e.g., crop\/weed) using a DNN is non-trivial due to difficulties arising from: (1) limited ground sample distances (GSDs) in high-altitude datasets, (2) sacrificed resolution resulting from downsampling high-fidelity images, and (3) multispectral image alignment. To address these issues, we adopt a stand sliding window approach that operates on only small portions of multispectral orthomosaic maps (tiles), which are channel-wise aligned and calibrated radiometrically across the entire map. We define the tile size to be the same as that of the DNN input to avoid resolution loss. Compared to our baseline model (i.e., SegNet with 3 channel RGB (red, green, and blue) inputs) yielding an area under the curve (AUC) of [background=0.607, crop=0.681, weed=0.576], our proposed model with 9 input channels achieves [0.839, 0.863, 0.782]. Additionally, we provide an extensive analysis of 20 trained models, both qualitatively and quantitatively, in order to evaluate the effects of varying input channels and tunable network hyperparameters. Furthermore, we release a large sugar beet\/weed aerial dataset with expertly guided annotations for further research in the fields of remote sensing, precision agriculture, and agricultural robotics.<\/jats:p>","DOI":"10.3390\/rs10091423","type":"journal-article","created":{"date-parts":[[2018,9,7]],"date-time":"2018-09-07T11:47:41Z","timestamp":1536320861000},"page":"1423","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":252,"title":["WeedMap: A Large-Scale Semantic Weed Mapping Framework Using Aerial Multispectral Imaging and Deep Neural Network for Precision Farming"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5429-0515","authenticated-orcid":false,"given":"Inkyu","family":"Sa","sequence":"first","affiliation":[{"name":"Autonomous Systems Lab., Department of Mechanical and Process Engineering, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marija","family":"Popovi\u0107","sequence":"additional","affiliation":[{"name":"Autonomous Systems Lab., Department of Mechanical and Process Engineering, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7217-2141","authenticated-orcid":false,"given":"Raghav","family":"Khanna","sequence":"additional","affiliation":[{"name":"Autonomous Systems Lab., Department of Mechanical and Process Engineering, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zetao","family":"Chen","sequence":"additional","affiliation":[{"name":"Vision for Robotics Lab., Department of Mechanical and Process Engineering, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0000-7491","authenticated-orcid":false,"given":"Philipp","family":"Lottes","sequence":"additional","affiliation":[{"name":"Institute of Geodesy and Geoinformation, University of Bonn, Bonn 53115, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frank","family":"Liebisch","sequence":"additional","affiliation":[{"name":"Crop Science, Department of Environmental Systems Science, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan","family":"Nieto","sequence":"additional","affiliation":[{"name":"Autonomous Systems Lab., Department of Mechanical and Process Engineering, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cyrill","family":"Stachniss","sequence":"additional","affiliation":[{"name":"Institute of Geodesy and Geoinformation, University of Bonn, Bonn 53115, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Achim","family":"Walter","sequence":"additional","affiliation":[{"name":"Crop Science, Department of Environmental Systems Science, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roland","family":"Siegwart","sequence":"additional","affiliation":[{"name":"Autonomous Systems Lab., Department of Mechanical and Process Engineering, ETHZ, Zurich 8092, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"De Castro, A.I., Torres-S\u00e1nchez, J., Pe\u00f1a, J.M., Jim\u00e9nez-Brenes, F.M., Csillik, O., and L\u00f3pez-Granados, F. (2018). An Automatic Random Forest-OBIA Algorithm for Early Weed Mapping between and within Crop Rows Using UAV Imagery. Remote Sens., 10.","DOI":"10.3390\/rs10020285"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kemker, R., Salvaggio, C., and Kanan, C. (2018). Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learning. ISPRS J. Photogramm. Remote Sens.","DOI":"10.1016\/j.isprsjprs.2018.04.014"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1007\/s11119-012-9274-5","article-title":"The application of small unmanned aerial systems for precision agriculture: A review","volume":"13","author":"Zhang","year":"2012","journal-title":"Precis. Agric."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.1365-3180.2010.00829.x","article-title":"Weed detection for site-specific weed management: Mapping and real-time approaches","volume":"51","year":"2011","journal-title":"Weed Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"6148","DOI":"10.1073\/pnas.1707462114","article-title":"Opinion: Smart farming is key to developing sustainable agriculture","volume":"114","author":"Walter","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1017\/S1466046615000174","article-title":"Bringing Unmanned Aerial Systems Closer to the Environment","volume":"17","author":"Detweiler","year":"2015","journal-title":"Environ. Pract."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lottes, P., Khanna, R., Pfeifer, J., Siegwart, R., and Stachniss, C. (June, January 29). UAV-based crop and weed classification for smart farming. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989347"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1109\/LRA.2017.2774979","article-title":"weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming","volume":"3","author":"Sa","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Joalland, S., Screpanti, C., Varella, H.V., Reuther, M., Schwind, M., Lang, C., Walter, A., and Liebisch, F. (2018). Aerial and Ground Based Sensing of Tolerance to Beet Cyst Nematode in Sugar Beet. Remote Sens., 10.","DOI":"10.3390\/rs10050787"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Carrio, A., Sampedro, C., Rodriguez-Ramos, A., and Campoy, P. (2017). A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles. J. Sens.","DOI":"10.1155\/2017\/3296874"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/gigascience\/gix083","article-title":"Deep machine learning provides state-of-the-art performance in image-based plant phenotyping","volume":"6","author":"Pound","year":"2017","journal-title":"Gigascience"},{"key":"ref_12","unstructured":"(2018, September 03). Remote Sensing 2018 Weed Map Dataset. Available online: https:\/\/goo.gl\/ZsgeCV."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.biosystemseng.2015.07.012","article-title":"Sugar beet (Beta vulgaris L.) and thistle (Cirsium arvensis L.) discrimination based on field spectral data","volume":"139","author":"Jose","year":"2015","journal-title":"Biosyst. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"11149","DOI":"10.1016\/j.eswa.2012.03.040","article-title":"Support Vector Machines for crop\/weeds identification in maize fields","volume":"39","author":"Guerrero","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1016\/j.asoc.2015.08.027","article-title":"A semi-supervised system for weed mapping in sunflower crops using unmanned aerial vehicles and a crop row detection method","volume":"37","author":"Gutierrez","year":"2015","journal-title":"Appl. Soft Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.eswa.2015.10.043","article-title":"Selecting patterns and features for between- and within- crop-row weed mapping using UAV-imagery","volume":"47","author":"Gutierrez","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sandino, J., Gonzalez, F., Mengersen, K., and Gaston, K.J. (2018). UAVs and Machine Learning Revolutionising Invasive Grass and Vegetation Surveys in Remote Arid Lands. Sensors, 18.","DOI":"10.3390\/s18020605"},{"key":"ref_18","first-page":"43","article-title":"Fusion of pixel and object-based features for weed mapping using unmanned aerial vehicle imagery","volume":"67","author":"Gao","year":"2018","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.compag.2018.02.016","article-title":"Deep learning in agriculture: A survey","volume":"147","author":"Kamilaris","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_20","unstructured":"Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (arXiv, 2016). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dai, J., He, K., and Sun, J. (2015, January 11\u201318). Boxsup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.191"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, X., Chen, H., .Qi, X., Dou, Q., Fu, C., and Heng, P.A. (arXiv, 2017). H-DenseUNet: Hybrid Densely Connected UNet for Liver and Liver Tumor Segmentation from CT Volumes, arXiv.","DOI":"10.1109\/TMI.2018.2845918"},{"key":"ref_26","unstructured":"Paszke, A., Chaurasia, A., Kim, S., and Culurciello, E. (arXiv, 2016). ENet: Deep Neural Network Architecture for Real-Time Semantic Segmentation, arXiv."},{"key":"ref_27","first-page":"234","article-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","volume":"Volume 9351","author":"Ronneberger","year":"2015","journal-title":"Medical Image Computing and Computer-Assisted Intervention"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Potena, C., Nardi, D., and Pretto, A. (2016, January 3\u20137). Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture. Proceedings of the International Conference on Intelligent Autonomous Systems, Shanghai, China.","DOI":"10.1007\/978-3-319-48036-7_9"},{"key":"ref_29","unstructured":"Mortensen, A., Dyrmann, M., Karstoft, H., J\u00f6rgensen, R.N., and Gislum, R. (2016, January 26\u201329). Semantic Segmentation of Mixed Crops using Deep Convolutional Neural Network. Proceedings of the International Conference on Agricultural Engineering (CIGR), Aarhus, Denmark."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Milioto, A., Lottes, P., and Stachniss, C. (2018, January 21\u201326). Real-time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs. Proceedings of the IEEE International Conference on Robotics & Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460962"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"McCool, C., Perez, T., and Upcroft, B. (2017). Mixtures of Lightweight Deep Convolutional Neural Networks: Applied to Agricultural Robotics. IEEE Robot. Autom. Lett.","DOI":"10.1109\/LRA.2017.2667039"},{"key":"ref_32","unstructured":"Cicco, M., Potena, C., Grisetti, G., and Pretto, A. (2017, January 24\u201328). Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lottes, P., Behley, J., Milioto, A., and Stachniss, C. (2018). Fully Convolutional Networks with Sequential Information for Robust Crop and Weed Detection in Precision Farming. IEEE Robot. Autom. Lett.","DOI":"10.1109\/LRA.2018.2846289"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_35","unstructured":"Rouse Jr, J.W., Haas, R.H., Schell, J., and Deering, D. (1973). Monitoring the Vernal Advancement and Retrogradation (Green Wave Effect) of Natural Vegetation, NASA."},{"key":"ref_36","unstructured":"(2018, September 03). MicaSense, Use of Calibrated Reflectance Panels For RedEdge Data. Available online: http:\/\/goo.gl\/EgNwtU."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hinzmann, T., Sch\u00f6nberger, J.L., Pollefeys, M., and Siegwart, R. (2017, January 12\u201315). Mapping on the Fly: Real-time 3D Dense Reconstruction, Digital Surface Map and Incremental Orthomosaic Generation for Unmanned Aerial Vehicles. Proceedings of the Field and Service Robotics\u2014Results of the 11th International Conference, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-67361-5_25"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1002\/rob.21765","article-title":"Robotic technologies for solar-powered UAVs: Fully autonomous updraft-aware aerial sensing for multiday search-and-rescue missions","volume":"35","author":"Oettershagen","year":"2017","journal-title":"J. Field Robot."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Snavely, N., Seitz, S.M., and Szeliski, R. (2006). Photo Tourism: Exploring Photo Collections in 3D, ACM Transactions on Graphics (TOG) ACM.","DOI":"10.1145\/1141911.1141964"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1109\/TPAMI.2009.161","article-title":"Accurate, dense, and robust multiview stereopsis","volume":"32","author":"Furukawa","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_41","unstructured":"(2018, September 03). Pix4Dmapper Software. Available online: https:\/\/pix4d.com."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/TITS.2017.2750080","article-title":"ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation","volume":"19","author":"Romera","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_43","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very deep convolutional networks for large-scale image recognition, arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., and Garcia-Rodriguez, J. (arXiv, 2017). A Review on Deep Learning Techniques Applied to Semantic Segmentation, arXiv.","DOI":"10.1016\/j.asoc.2018.05.018"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Eigen, D., and Fergus, R. (2015, January 11\u201318). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.304"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Khanna, R., Sa, I., Nieto, J., and Siegwart, R. (June, January 29). On field radiometric calibration for multispectral cameras. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989768"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Boyd, K., Eng, K.H., and Page, C.D. (2013). Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals. Machine Learning and Knowledge Discovery in Databases, Springer.","DOI":"10.1007\/978-3-642-40994-3_29"},{"key":"ref_48","unstructured":"(2018, September 03). MATLAB Expression. Available online: https:\/\/ch.mathworks.com\/help\/images\/image-coordinate-systems.html."},{"key":"ref_49","unstructured":"(2018, September 03). MATLAB Perfcurve. Available online: https:\/\/mathworks.com\/help\/stats\/perfcurve.html."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Csurka, G., Larlus, D., Perronnin, F., and Meylan, F. What is a good evaluation measure for semantic segmentation? In Proceedings of the 24th BMVC British Machine Vision Conference, Bristol, UK, 9\u201313 September 2013.","DOI":"10.5244\/C.27.32"},{"key":"ref_51","unstructured":"Wang, J., and Perez, L. (arXiv, 2017). The Effectiveness of Data Augmentation in Image Classification Using Deep Learning, arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Wong, S.C., Gatt, A., Stamatescu, V., and McDonnell, M.D. (December, January 30). Understanding Data Augmentation for Classification: When to Warp?. Proceedings of the 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Gold Coast, Australia.","DOI":"10.1109\/DICTA.2016.7797091"},{"key":"ref_53","unstructured":"(2018, September 03). NVIDIA Jetson Xavier. Available online: https:\/\/developer.nvidia.com\/jetson-xavier."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/10\/9\/1423\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:19:16Z","timestamp":1760195956000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/10\/9\/1423"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,7]]},"references-count":53,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["rs10091423"],"URL":"https:\/\/doi.org\/10.3390\/rs10091423","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,7]]}}}