{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T15:24:05Z","timestamp":1776525845217,"version":"3.51.2"},"reference-count":52,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,3,1]],"date-time":"2023-03-01T00:00:00Z","timestamp":1677628800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Government of Lower Austria and the private foundation Forum Morgen"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Smart farming (SF) applications rely on robust and accurate computer vision systems. An important computer vision task in agriculture is semantic segmentation, which aims to classify each pixel of an image and can be used for selective weed removal. State-of-the-art implementations use convolutional neural networks (CNN) that are trained on large image datasets. In agriculture, publicly available RGB image datasets are scarce and often lack detailed ground-truth information. In contrast to agriculture, other research areas feature RGB-D datasets that combine color (RGB) with additional distance (D) information. Such results show that including distance as an additional modality can improve model performance further. Therefore, we introduce WE3DS as the first RGB-D image dataset for multi-class plant species semantic segmentation in crop farming. It contains 2568 RGB-D images (color image and distance map) and corresponding hand-annotated ground-truth masks. Images were taken under natural light conditions using an RGB-D sensor consisting of two RGB cameras in a stereo setup. Further, we provide a benchmark for RGB-D semantic segmentation on the WE3DS dataset and compare it with a solely RGB-based model. Our trained models achieve up to 70.7% mean Intersection over Union (mIoU) for discriminating between soil, seven crop species, and ten weed species. Finally, our work confirms the finding that additional distance information improves segmentation quality.<\/jats:p>","DOI":"10.3390\/s23052713","type":"journal-article","created":{"date-parts":[[2023,3,2]],"date-time":"2023-03-02T02:10:59Z","timestamp":1677723059000},"page":"2713","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["WE3DS: An RGB-D Image Dataset for Semantic Segmentation in Agriculture"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8807-6904","authenticated-orcid":false,"given":"Florian","family":"Kitzler","sequence":"first","affiliation":[{"name":"Department of Sustainable Agricultural Systems, Institute of Agricultural Engineering, University of Natural Resources and Life Sciences Vienna, Peter-Jordan-Stra\u00dfe 82, 1190 Vienna, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0960-3564","authenticated-orcid":false,"given":"Norbert","family":"Barta","sequence":"additional","affiliation":[{"name":"Department of Sustainable Agricultural Systems, Institute of Agricultural Engineering, University of Natural Resources and Life Sciences Vienna, Peter-Jordan-Stra\u00dfe 82, 1190 Vienna, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9192-5716","authenticated-orcid":false,"given":"Reinhard W.","family":"Neugschwandtner","sequence":"additional","affiliation":[{"name":"Department of Crop Sciences, Institute of Agronomy, University of Natural Resources and Life Sciences Vienna, Konrad Lorenz-Stra\u00dfe 24, 3430 Tulln an der Donau, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6957-6815","authenticated-orcid":false,"given":"Andreas","family":"Gronauer","sequence":"additional","affiliation":[{"name":"Department of Sustainable Agricultural Systems, Institute of Agricultural Engineering, University of Natural Resources and Life Sciences Vienna, Peter-Jordan-Stra\u00dfe 82, 1190 Vienna, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4860-4247","authenticated-orcid":false,"given":"Viktoria","family":"Motsch","sequence":"additional","affiliation":[{"name":"Department of Sustainable Agricultural Systems, Institute of Agricultural Engineering, University of Natural Resources and Life Sciences Vienna, Peter-Jordan-Stra\u00dfe 82, 1190 Vienna, Austria"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kitzler, F., Wagentristl, H., Neugschwandtner, R.W., Gronauer, A., and Motsch, V. (2022). Influence of Selected Modeling Parameters on Plant Segmentation Quality Using Decision Tree Classifiers. Agriculture, 12.","DOI":"10.3390\/agriculture12091408"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1109\/LRA.2017.2774979","article-title":"weednet: Dense semantic weed classification using multispectral images and mav for smart farming","volume":"3","author":"Sa","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.biosystemseng.2019.08.014","article-title":"Plant-part segmentation using deep learning and multi-view vision","volume":"187","author":"Shi","year":"2019","journal-title":"Biosyst. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chiu, M.T., Xu, X., Wei, Y., Huang, Z., Schwing, A.G., Brunner, R., Khachatrian, H., Karapetyan, H., Dozier, I., and Rose, G. (2020, January 13\u201319). Agriculture-vision: A large aerial image database for agricultural pattern analysis. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00290"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.compag.2018.03.032","article-title":"A comparative study of fine-tuning deep learning models for plant disease identification","volume":"161","author":"Too","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_6","unstructured":"N\u00f8rremark, M., and Griepentrog, H.W. (2004, January 8\u201310). Analysis and definition of the close-to-crop area in relation to robotic weeding. Proceedings of the 6th Workshop of the EWRS Working Group \u2018Physical and Cultural Weed Control\u2019, Lillehammer, Norway."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/S0168-1699(00)00153-8","article-title":"Precision farming\u2014The environmental challenge","volume":"30","author":"Auernhammer","year":"2001","journal-title":"Comput. Electron. Agric."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/s11119-008-9077-x","article-title":"Soil heterogeneity at the field scale: A challenge for precision crop protection","volume":"9","author":"Patzold","year":"2008","journal-title":"Precis. Agric."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Saiz-Rubio, V., and Rovira-M\u00e1s, F. (2020). From smart farming towards agriculture 5.0: A review on crop data management. Agronomy, 10.","DOI":"10.3390\/agronomy10020207"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1017\/S0021859605005708","article-title":"Crop losses to pests","volume":"144","author":"Oerke","year":"2005","journal-title":"J. Agric. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1614\/WS-D-11-00155.1","article-title":"Reducing the risks of herbicide resistance: Best management practices and recommendations","volume":"60","author":"Norsworthy","year":"2012","journal-title":"Weed Sci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Alengebawy, A., Abdelkhalek, S.T., Qureshi, S.R., and Wang, M.Q. (2021). Heavy metals and pesticides toxicity in agricultural soil and plants: Ecological risks and human health implications. Toxics, 9.","DOI":"10.3390\/toxics9030042"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Heege, H.J., and Thiessen, E. (2013). Precision in Crop Farming, Springer.","DOI":"10.1007\/978-94-007-6760-7"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4009","DOI":"10.1021\/acs.jafc.5b03688","article-title":"Spot spraying reduces herbicide concentrations in runoff","volume":"64","author":"Melland","year":"2016","journal-title":"J. Agric. Food Chem."},{"key":"ref_15","unstructured":"Griepentrog, H.W., and Dedousis, A.P. (2010). Soil Engineering, Springer."},{"key":"ref_16","unstructured":"Langsenkamp, F., Sellmann, F., Kohlbrecher, M., Kielhorn, A., Strothmann, W., Michaels, A., Ruckelshausen, A., and Trautz, D. (2014, January 16\u201319). Tube Stamp for mechanical intra-row individual Plant Weed Control. Proceedings of the 18th World Congress of CIGR, Beijing, China."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.ecolind.2014.07.028","article-title":"Developing a set of simulation-based indicators to assess harmfulness and contribution to biodiversity of weed communities in cropping systems","volume":"48","author":"Petit","year":"2015","journal-title":"Ecol. Indic."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1614\/0043-1745(2003)051[0037:IORPAR]2.0.CO;2","article-title":"Interference of redroot pigweed (Amaranthus retroflexus), Palmer amaranth (A. palmeri), and common waterhemp (A. rudis) in soybean","volume":"51","author":"Bensch","year":"2003","journal-title":"Weed Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1002\/rob.21869","article-title":"Transfer learning between crop types for semantic segmentation of crops versus weeds in precision agriculture","volume":"37","author":"Bosilj","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lottes, P., Khanna, R., Pfeifer, J., Siegwart, R., and Stachniss, C. (June, January 29). UAV-based crop and weed classification for smart farming. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989347"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2870","DOI":"10.1109\/LRA.2018.2846289","article-title":"Fully convolutional networks with sequential information for robust crop and weed detection in precision farming","volume":"3","author":"Lottes","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Milioto, A., Lottes, P., and Stachniss, C. (2018, January 21\u201325). Real-time semantic segmentation of crop and weed for precision agriculture robots leveraging background knowledge in CNNs. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460962"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Munich, Germany.","DOI":"10.1007\/978-3-319-24553-9"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lameski, P., Zdravevski, E., Trajkovik, V., and Kulakov, A. (2017, January 18\u201323). Weed detection dataset with RGB images taken under variable light conditions. Proceedings of the International Conference on ICT Innovations, Skopje, Macedonia.","DOI":"10.1007\/978-3-319-67597-8_11"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Barchid, S., Mennesson, J., and Dj\u00e9raba, C. (2021). Review on Indoor RGB-D Semantic Segmentation with Deep Convolutional Neural Networks. arXiv.","DOI":"10.1109\/CBMI50038.2021.9461875"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Seichter, D., K\u00f6hler, M., Lewandowski, B., Wengefeld, T., and Gross, H.M. (June, January 30). Efficient rgb-d semantic segmentation for indoor scene analysis. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561675"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Song, S., Lichtenberg, S.P., and Xiao, J. (2015, January 7\u201312). Sun rgb-d: A rgb-d scene understanding benchmark suite. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298655"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. (2012, January 7\u201313). Indoor segmentation and support inference from rgbd images. Proceedings of the European Conference on Computer Vision, Florence, Italy.","DOI":"10.1007\/978-3-642-33715-4_54"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Silberman, N., and Fergus, R. (2011, January 6\u201313). Indoor scene segmentation using a structured light sensor. Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130298"},{"key":"ref_30","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (July, January 26). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.patrec.2008.04.005","article-title":"Semantic object classes in video: A high-definition ground truth database","volume":"30","author":"Brostow","year":"2009","journal-title":"Pattern Recognit. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Scharw\u00e4chter, T., Enzweiler, M., Franke, U., and Roth, S. (2013, January 3\u20136). Efficient multi-cue scene segmentation. Proceedings of the German Conference on Pattern Recognition, Saarbr\u00fccken, Germany.","DOI":"10.1007\/978-3-642-40602-7_46"},{"key":"ref_34","unstructured":"Bender, A., Whelan, B., and Sukkarieh, S. (2019). Ladybird Cobbitty 2017 Brassica Dataset, The University of Sydney."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1177\/0278364917720510","article-title":"Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields","volume":"36","author":"Chebrolu","year":"2017","journal-title":"Int. J. Robot. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wu, Z., Chen, Y., Zhao, B., Kang, X., and Ding, Y. (2021). Review of Weed Detection Methods Based on Computer Vision. Sensors, 21.","DOI":"10.3390\/s21113647"},{"key":"ref_37","first-page":"1","article-title":"BoniRob\u2014An autonomous field robot platform for individual plant phenotyping","volume":"9","author":"Ruckelshausen","year":"2009","journal-title":"Precis. Agric."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1002\/rob.21877","article-title":"A high-resolution, multimodal data set for agricultural robotics: A Ladybird\u2019s-eye view of Brassica","volume":"37","author":"Bender","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_39","unstructured":"Hensman, J., Matthews, A., and Ghahramani, Z. (2015, January 9\u201312). Scalable variational Gaussian process classification. Proceedings of the Artificial Intelligence and Statistics, PMLR, San Diego, CA, USA."},{"key":"ref_40","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7\u201312). Faster r-cnn: Towards real-time object detection with region proposal networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chen, X., Lin, K.Y., Wang, J., Wu, W., Qian, C., Li, H., and Zeng, G. (2020, January 23\u201328). Bi-directional cross-modality feature propagation with separation-and-aggregation gate for RGB-D semantic segmentation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58621-8_33"},{"key":"ref_42","unstructured":"Couprie, C., Farabet, C., Najman, L., and LeCun, Y. (2013). Indoor semantic segmentation using depth information. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hu, X., Yang, K., Fei, L., and Wang, K. (2019, January 22\u201325). Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.","DOI":"10.1109\/ICIP.2019.8803025"},{"key":"ref_44","unstructured":"Jiang, J., Zheng, L., Luo, F., and Zhang, Z. (2018). Rednet: Residual encoder-decoder network for indoor rgb-d semantic segmentation. arXiv."},{"key":"ref_45","unstructured":"Park, S.J., Hong, K.S., and Lee, S. (2017, January 22\u201329). Rdfnet: Rgb-d multi-level residual feature fusion for indoor semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy."},{"key":"ref_46","unstructured":"Steger, C., Ulrich, M., and Wiedemann, C. (2018). Machine Vision Algorithms and Applications, John Wiley & Sons."},{"key":"ref_47","unstructured":"Sekachev, B., Manovich, N., Zhiltsov, M., Zhavoronkov, A., Kalinin, D., Hoff, B., Kruchinin, D., Zankevich, A., and Sidnev, D. (2022, December 01). opencv\/cvat: v1.1.0. Available online: https:\/\/zenodo.org\/record\/4009388#.Y_bCTHZBxPY."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Gupta, S., Girshick, R., Arbel\u00e1ez, P., and Malik, J. (2014, January 6\u201312). Learning rich features from RGB-D images for object detection and segmentation. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10584-0_23"},{"key":"ref_49","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2713\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:45:29Z","timestamp":1760121929000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2713"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,1]]},"references-count":52,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052713"],"URL":"https:\/\/doi.org\/10.3390\/s23052713","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,1]]}}}