{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T22:09:08Z","timestamp":1782252548105,"version":"3.54.5"},"reference-count":38,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2020,12,18]],"date-time":"2020-12-18T00:00:00Z","timestamp":1608249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"SMARAGD","award":["TKI AF16190"],"award-info":[{"award-number":["TKI AF16190"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Robotic plant-specific spraying can reduce herbicide usage in agriculture while minimizing labor costs and maximizing yield. Weed detection is a crucial step in automated weeding. Currently, weed detection algorithms are always evaluated at the image level, using conventional image metrics. However, these metrics do not consider the full pipeline connecting image acquisition to the site-specific operation of the spraying nozzles, which is vital for an accurate evaluation of the system. Therefore, we propose a novel application-specific image-evaluation method, which analyses the weed detections on the plant level and in the light of the spraying decision made by the robot. In this paper, a spraying robot is evaluated on three levels: (1) On image-level, using conventional image metrics, (2) on application-level, using our novel application-specific image-evaluation method, and (3) on field level, in which the weed-detection algorithm is implemented on an autonomous spraying robot and tested in the field. On image level, our detection system achieved a recall of 57% and a precision of 84%, which is a lower performance than detection systems reported in literature. However, integrated on an autonomous volunteer-potato sprayer-system we outperformed the state-of-the-art, effectively controlling 96% of the weeds while terminating only 3% of the crops. Using the application-level evaluation, an accurate indication of the field performance of the weed-detection algorithm prior to the field test was given and the type of errors produced by the spraying system was correctly predicted.<\/jats:p>","DOI":"10.3390\/s20247262","type":"journal-article","created":{"date-parts":[[2020,12,21]],"date-time":"2020-12-21T01:01:08Z","timestamp":1608512468000},"page":"7262","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["Application-Specific Evaluation of a Weed-Detection Algorithm for Plant-Specific Spraying"],"prefix":"10.3390","volume":"20","author":[{"given":"Thijs","family":"Ruigrok","sequence":"first","affiliation":[{"name":"Farm Technology, Department of Plant Sciences, Wageningen University and Research, 6700 AA Wageningen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eldert","family":"van Henten","sequence":"additional","affiliation":[{"name":"Farm Technology, Department of Plant Sciences, Wageningen University and Research, 6700 AA Wageningen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johan","family":"Booij","sequence":"additional","affiliation":[{"name":"Field Crops, Wageningen Plant Research, Wageningen University and Research, 8200 AK Lelystad, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Koen","family":"van Boheemen","sequence":"additional","affiliation":[{"name":"Agrosystems Research, Wageningen Plant Research, Wageningen University and Research, 6700 AA Wageningen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gert","family":"Kootstra","sequence":"additional","affiliation":[{"name":"Farm Technology, Department of Plant Sciences, Wageningen University and Research, 6700 AA Wageningen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.agee.2007.09.005","article-title":"Identifying and managing the conflicts between agriculture and biodiversity conservation in Europe-A review","volume":"124","author":"Henle","year":"2008","journal-title":"Agric. Ecosyst. Environ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/S0378-4290(00)00091-5","article-title":"Design of weed management systems with a reduced reliance on herbicides poses new challenges and prerequisites for modeling crop-weed interactions","volume":"67","author":"Bastiaans","year":"2000","journal-title":"Field Crop. Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1016\/S0921-8009(01)00238-5","article-title":"Why farmers continue to use pesticides despite environmental, health and sustainability costs","volume":"39","author":"Wilson","year":"2001","journal-title":"Ecol. Econ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1007\/s11540-018-9357-4","article-title":"Advances in Variable Rate Technology Application in Potato in The Netherlands","volume":"60","author":"Kempenaar","year":"2017","journal-title":"Potato Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1006\/jaer.1998.0334","article-title":"Reduced input of herbicides by use of optoelectronic sensors","volume":"71","author":"Biller","year":"1998","journal-title":"J. Agric. Eng. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1500","DOI":"10.13031\/2013.30344","article-title":"Machine vision and image processing for plant identification","volume":"29","author":"Guyer","year":"1986","journal-title":"Trans. ASAE"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1023\/A:1009977903204","article-title":"Robotic Weed Control System for Tomatoes","volume":"1","author":"Lee","year":"1999","journal-title":"Precis. Agric."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/S0168-1699(99)00068-X","article-title":"Colour and shape analysis techniques for weed detection in cereal fields","volume":"25","author":"Benlloch","year":"2000","journal-title":"Comput. Electron. Agric."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"259","DOI":"10.13031\/2013.27838","article-title":"Color Indices for Weed Identification Under Various Soil, Residue, and Lighting Conditions","volume":"38","author":"Woebbecke","year":"1995","journal-title":"Trans. ASAE"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/S0261-2194(98)00009-X","article-title":"A review of spectral properties of plants and their potential use for crop\/weed discrimination in row-crops","volume":"17","author":"Zwiggelaar","year":"1998","journal-title":"Crop Prot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Haug, S., Michaels, A., Biber, P., and Ostermann, J. (2014, January 24\u201326). Plant classification system for crop\/weed discrimination without segmentation. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, IEEE, Steamboat Springs, CO, USA.","DOI":"10.1109\/WACV.2014.6835733"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1160","DOI":"10.1002\/rob.21675","article-title":"Effective Vision-based Classification for Separating Sugar Beets and Weeds for Precision Farming","volume":"34","author":"Lottes","year":"2017","journal-title":"J. Field Robot."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lottes, P., and Stachniss, C. (2017, January 24\u201328). Semi-supervised online visual crop and weed classification in precision farming exploiting plant arrangement. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8206403"},{"key":"ref_14","unstructured":"Suh, H.K., Hofstee, J.W., IJselmuiden, J., and Van Henten, E.J. (2016, January 26\u201329). Discrimination between Volunteer Potato and Sugar Beet with a Bag-of-Visual-Words Model. Proceedings of the CIGR-AgEng 2016 Conference, Aarhus, Denmark."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.biosystemseng.2016.08.024","article-title":"Plant species classification using deep convolutional neural network","volume":"151","author":"Dyrmann","year":"2016","journal-title":"Biosyst. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1017\/S2040470017000206","article-title":"RoboWeedSupport\u2014Detection of weed locations in leaf occluded cereal crops using a fully convolutional neural network","volume":"8","author":"Dyrmann","year":"2017","journal-title":"Adv. Anim. Biosci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Milioto, A., Lottes, P., and Stachniss, C. (2017, January 4\u20137). Real-time blob-wise sugar beets VS weeds classification for monitoring field using convolutional neural networks. Proceedings of the ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, Bonn, Germany.","DOI":"10.5194\/isprs-annals-IV-2-W3-41-2017"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Milioto, A., Lottes, P., and Stachniss, C. (2018, January 21\u201325). Real-Time Semantic Segmentation of Crop and Weed for Precision Agriculture Robots Leveraging Background Knowledge in CNNs. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8460962"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.biosystemseng.2018.06.017","article-title":"Transfer learning for the classification of sugar beet and volunteer potato under field conditions","volume":"174","author":"Suh","year":"2018","journal-title":"Biosyst. Eng."},{"key":"ref_20","unstructured":"Voorhoeve, L. (2018). Plant Detection under Natural Illumination Conditions. [Master\u2019s Thesis, Wageningen University]."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"471","DOI":"10.3390\/agriengineering2030032","article-title":"A Deep Learning Approach for Weed Detection in Lettuce Crops Using Multispectral Images","volume":"2","author":"Osorio","year":"2020","journal-title":"AgriEngineering"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Teimouri, N., Dyrmann, M., Nielsen, P., Mathiassen, S., Somerville, G., and J\u00f8rgensen, R. (2018). Weed Growth Stage Estimator Using Deep Convolutional Neural Networks. Sensors, 18.","DOI":"10.3390\/s18051580"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Skovsen, S., Dyrmann, M., Mortensen, A., Steen, K., Green, O., Eriksen, J., Gislum, R., J\u00f8rgensen, R., and Karstoft, H. (2017). Estimation of the Botanical Composition of Clover-Grass Leys from RGB Images Using Data Simulation and Fully Convolutional Neural Networks. Sensors, 17.","DOI":"10.3390\/s17122930"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Liu, Y., Gong, C., Chen, Y., and Yu, H. (2020). Applications of Deep Learning for Dense Scenes Analysis in Agriculture: A Review. Sensors, 20.","DOI":"10.3390\/s20051520"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/S1474-6670(17)36047-0","article-title":"Intra-Row Weed Control: A Mechatronics Approach","volume":"31","author":"Bontsema","year":"1998","journal-title":"IFAC Proc. Vol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.biosystemseng.2010.06.011","article-title":"Performance evaluation of an automated detection and control system for volunteer potatoes in sugar beet fields","volume":"107","author":"Nieuwenhuizen","year":"2010","journal-title":"Biosyst. Eng."},{"key":"ref_27","unstructured":"Hemming, J., Nieuwenhuizen, A., and Struik, L. (2011, January 19\u201323). Image analysis system to determine crop row and plant positions for an intra-row weeding machine. Proceedings of the International Symposium on Sustainable Bioproduction, Tokyo, Japan."},{"key":"ref_28","unstructured":"Hemming, J., Jong, H., De Struik, L., Meer, J., and Van Henten, E.J. (2018, January 8\u201312). Field Performance Evaluation Method for a Machine Vision Based Intra Row Hoeing Machine. Proceedings of the AgEng2018, Wageningen, The Netherlands."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1002\/rob.20377","article-title":"A Robot to Detect and Control Broad-Leaved Dock (Rumex obtusifolius L.) in Grassland","volume":"28","author":"Samsom","year":"2011","journal-title":"J. Field Robot."},{"key":"ref_30","unstructured":"Underwood, J.P., Calleija, M., Taylor, Z., Hung, C., Nieto, J., Fitch, R., and Sukkarieh, S. (2015, January 26\u201330). Real-time target detection and steerable spray for vegetable crops. Proceedings of the International Conference on Robotics and Automation: Robotics in Agriculture Workshop, Seattle, WA, USA."},{"key":"ref_31","unstructured":"(2020, August 18). NVWA Teeltvoorschrift-Phytophthora-Infestans. Available online: https:\/\/www.nvwa.nl\/onderwerpen\/teeltvoorschriften-akkerbouw-en-tuinbouw\/teeltvoorschrift-phytophthora-infestans."},{"key":"ref_32","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_33","first-page":"1137","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst. 28"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017, January 22\u201329). Focal Loss for Dense Object Detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), IEEE, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_35","unstructured":"Jocher, G. (2019, August 19). YOLOv3 Ultralytics. Available online: https:\/\/github.com\/ultralytics\/yolov3."},{"key":"ref_36","unstructured":"Joseph, R., and Farhadi, A. (2019, May 06). YOLO: Real-Time Object Detection. Available online: https:\/\/pjreddie.com\/darknet\/yolo\/."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2014, January 6\u201312). Microsoft COCO: Common Objects in Context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2870","DOI":"10.1109\/LRA.2018.2846289","article-title":"Fully Convolutional Networks with Sequential Information for Robust Crop and Weed Detection in Precision Farming","volume":"3","author":"Lottes","year":"2018","journal-title":"IEEE Robot. Autom. Lett."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7262\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:46:45Z","timestamp":1760179605000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7262"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,18]]},"references-count":38,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20247262"],"URL":"https:\/\/doi.org\/10.3390\/s20247262","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,18]]}}}