{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:18:02Z","timestamp":1760170682332,"version":"build-2065373602"},"reference-count":34,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T00:00:00Z","timestamp":1636416000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["No.CZ.02.1.01\/0.0\/0.0\/16_026\/0008446"],"award-info":[{"award-number":["No.CZ.02.1.01\/0.0\/0.0\/16_026\/0008446"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Automated analysis of small and optically variable plant organs, such as grain spikes, is highly demanded in quantitative plant science and breeding. Previous works primarily focused on the detection of prominently visible spikes emerging on the top of the grain plants growing in field conditions. However, accurate and automated analysis of all fully and partially visible spikes in greenhouse images renders a more challenging task, which was rarely addressed in the past. A particular difficulty for image analysis is represented by leaf-covered, occluded but also matured spikes of bushy crop cultivars that can hardly be differentiated from the remaining plant biomass. To address the challenge of automated analysis of arbitrary spike phenotypes in different grain crops and optical setups, here, we performed a comparative investigation of six neural network methods for pattern detection and segmentation in RGB images, including five deep and one shallow neural network. Our experimental results demonstrate that advanced deep learning methods show superior performance, achieving over 90% accuracy by detection and segmentation of spikes in wheat, barley and rye images. However, spike detection in new crop phenotypes can be performed more accurately than segmentation. Furthermore, the detection and segmentation of matured, partially visible and occluded spikes, for which phenotypes substantially deviate from the training set of regular spikes, still represent a challenge to neural network models trained on a limited set of a few hundreds of manually labeled ground truth images. Limitations and further potential improvements of the presented algorithmic frameworks for spike image analysis are discussed. Besides theoretical and experimental investigations, we provide a GUI-based tool (SpikeApp), which shows the application of pre-trained neural networks to fully automate spike detection, segmentation and phenotyping in images of greenhouse-grown plants.<\/jats:p>","DOI":"10.3390\/s21227441","type":"journal-article","created":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T21:39:07Z","timestamp":1636493947000},"page":"7441","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Towards Automated Analysis of Grain Spikes in Greenhouse Images Using Neural Network Approaches: A Comparative Investigation of Six Methods"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4906-721X","authenticated-orcid":false,"given":"Sajid","family":"Ullah","sequence":"first","affiliation":[{"name":"Plant Sciences Core Facility, CEITEC\u2014Central European Institute of Technology, Masaryk University, 60200 Brno, Czech Republic"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0673-3873","authenticated-orcid":false,"given":"Michael","family":"Henke","sequence":"additional","affiliation":[{"name":"Plant Sciences Core Facility, CEITEC\u2014Central European Institute of Technology, Masaryk University, 60200 Brno, Czech Republic"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7584-9461","authenticated-orcid":false,"given":"Narendra","family":"Narisetti","sequence":"additional","affiliation":[{"name":"Leibniz Institute of Plant Genetics and Crop Plant Research (IPK), 06466 Gatersleben, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7519-5162","authenticated-orcid":false,"given":"Kl\u00e1ra","family":"Panzarov\u00e1","sequence":"additional","affiliation":[{"name":"PSI (Photon Systems Instruments), spol. s r.o., 66424 Drasov, Czech Republic"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6294-052X","authenticated-orcid":false,"given":"Martin","family":"Trt\u00edlek","sequence":"additional","affiliation":[{"name":"PSI (Photon Systems Instruments), spol. s r.o., 66424 Drasov, Czech Republic"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2622-6046","authenticated-orcid":false,"given":"Jan","family":"Hejatko","sequence":"additional","affiliation":[{"name":"Plant Sciences Core Facility, CEITEC\u2014Central European Institute of Technology, Masaryk University, 60200 Brno, Czech Republic"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6153-727X","authenticated-orcid":false,"given":"Evgeny","family":"Gladilin","sequence":"additional","affiliation":[{"name":"Leibniz Institute of Plant Genetics and Crop Plant Research (IPK), 06466 Gatersleben, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-018-31977-3","article-title":"Manipulation and Prediction of Spike Morphology Traits for the Improvement of Grain Yield in Wheat","volume":"8","author":"Guo","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/MSP.2015.2405111","article-title":"Image Analysis: The New Bottleneck in Plant Phenotyping","volume":"32","author":"Minervini","year":"2015","journal-title":"IEEE Signal Proc. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1186\/s13007-018-0366-8","article-title":"Detection and Analysis of Wheat Spikes Using Convolutional Neural Networks","volume":"14","author":"Hasan","year":"2018","journal-title":"Plant Methods"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"259","DOI":"10.3389\/fpls.2020.00259","article-title":"Rapid Recognition of Field-grown Wheat Spikes Based on a Superpixel Segmentation Algorithm Using Digital Images","volume":"11","author":"Tan","year":"2020","journal-title":"Front. Plant Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Alharbi, N., Zhou, J., and Wang, W. (2018, January 16\u201318). Automatic Counting of Wheat Spikes From Wheat Growth Images. Proceedings of the ICPRAM, Funchal, Portugal.","DOI":"10.5220\/0006580403460355"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1933","DOI":"10.3389\/fpls.2018.01933","article-title":"Plant Phenotyping Research Trends, a Science Mapping Approach","volume":"9","author":"Costa","year":"2019","journal-title":"Front. Plant Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7507131","DOI":"10.34133\/2019\/7507131","article-title":"Plant Phenotyping: Past, Present, and Future","volume":"2019","author":"Pieruschka","year":"2019","journal-title":"Plant Phenomics"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/j.compag.2017.07.024","article-title":"Wheat Landraces Identification through Glumes Image Analysis","volume":"141","author":"Grillo","year":"2017","journal-title":"Comput. Electron. Agric."},{"key":"ref_9","first-page":"212","article-title":"Non-destructive Measurement of Wheat Spike Characteristics Based on Morphological Image Processing","volume":"26","author":"Bi","year":"2010","journal-title":"Trans. Chin. Soc. Agric. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13007-020-00582-9","article-title":"Spikesegnet\u2014A Deep Learning Approach Utilizing Encoder-decoder Network with Hourglass for Spike Segmentation and Counting in Wheat Plant from Visual Imaging","volume":"16","author":"Misra","year":"2020","journal-title":"Plant Methods"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pound, M.P., Atkinson, J.A., Wells, D.M., Pridmore, T.P., and French, A.P. (2017, January 22\u201329). Deep Learning For Multi-task Plant Phenotyping. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.241"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1186\/s13007-017-0231-1","article-title":"Detecting Spikes of Wheat Plants Using Neural Networks with Laws Texture Energy","volume":"13","author":"Qiongyan","year":"2017","journal-title":"Plant Methods"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"666","DOI":"10.3389\/fpls.2020.00666","article-title":"Automated Spike Detection In Diverse European Wheat Plants Using Textural Features And The Frangi Filter in 2D Greenhouse Images","volume":"11","author":"Narisetti","year":"2020","journal-title":"Front. Plant Sci."},{"key":"ref_14","first-page":"1097","article-title":"Imagenet Classification With Deep Convolutional Neural Networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_15","unstructured":"Everingham, M., and Williams, C.K. (2021, October 08). The Pascal Visual Object Classes Challenge 2007 (Voc2007) Results. Available online: http:\/\/host.robots.ox.ac.uk\/pascal\/VOC\/voc2007\/."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A Large-scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_17","unstructured":"(2019, December 01). Tzutalin. LabelImg. Free Software: MIT License. Available online: https:\/\/github.com\/tzutalin\/labelImg."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot Multibox Detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1007\/s11263-013-0620-5","article-title":"Selective Search For Object Recognition","volume":"104","author":"Uijlings","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal Visual Object Classes (Voc) Challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_21","first-page":"91","article-title":"Faster R-cnn: Towards Real-time Object Detection With Region Proposal Networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_22","unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An Incremental Improvement. arXiv."},{"key":"ref_23","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1109\/83.242353","article-title":"Texture Analysis And Classification With Tree-structured Wavelet Transform","volume":"2","author":"Chang","year":"1993","journal-title":"IEEE Trans. Image Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"902","DOI":"10.1109\/TMI.2003.815066","article-title":"Texture-based Classification of Atherosclerotic Carotid Plaques","volume":"22","author":"Christodoulou","year":"2003","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional Networks For Biomedical Image Segmentation. Proceedings of the International Conference on Medical image computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the ECCV, Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Holschneider, M., Kronland-Martinet, R., Morlet, J., and Tchamitchian, P. (1990). A Real-time Algorithm for Signal Analysis with the Help of the Wavelet Transform. Wavelets, Springer.","DOI":"10.1007\/978-3-642-75988-8_28"},{"key":"ref_30","unstructured":"Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic Image Segmentation With Deep Convolutional Nets And Fully Connected CRFs. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., and Guadarrama, S. (2017, January 21\u201326). Speed\/Accuracy Trade-offs For Modern Convolutional Object Detectors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.351"},{"key":"ref_32","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method For Stochastic Optimization. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Salehi, S.S.M., Erdogmus, D., and Gholipour, A. (2017, January 10). Tversky loss Function for Image Segmentation using 3D Fully Convolutional Deep Networks. Proceedings of the International Workshop on Machine Learning in Medical Imaging, Quebec City, QC, Canada.","DOI":"10.1007\/978-3-319-67389-9_44"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Azad, R., Asadi-Aghbolaghi, M., Fathy, M., and Escalera, S. (2020, January 23\u201328). Attention deeplabv3+: Multi-level context attention mechanism for skin lesion segmentation. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-66415-2_16"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/22\/7441\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:28:09Z","timestamp":1760167689000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/22\/7441"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,9]]},"references-count":34,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["s21227441"],"URL":"https:\/\/doi.org\/10.3390\/s21227441","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,11,9]]}}}