{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T02:00:07Z","timestamp":1793325607731,"version":"4.5.0"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,12,2]],"date-time":"2022-12-02T00:00:00Z","timestamp":1669939200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Plant Sci."],"abstract":"<jats:p>\n                    The accurate detection and segmentation of apples during growth stage is essential for yield estimation, timely harvesting, and retrieving growth information. However, factors such as the uncertain illumination, overlaps and occlusions of apples, homochromatic background and the gradual change in the ground color of apples from green to red, bring great challenges to the detection and segmentation of apples. To solve these problems, this study proposed an improved Mask Scoring region-based convolutional neural network (Mask Scoring R-CNN), known as MS-ADS, for accurate apple detection and instance segmentation in a natural environment. First, the ResNeSt, a variant of ResNet, combined with a feature pyramid network was used as backbone network to improve the feature extraction ability. Second, high-level architectures including R-CNN head and mask head were modified to improve the utilization of high-level features. Convolutional layers were added to the original R-CNN head to improve the accuracy of bounding box detection (\n                    <jats:italic>bbox_mAP<\/jats:italic>\n                    ), and the Dual Attention Network was added to the original mask head to improve the accuracy of instance segmentation (\n                    <jats:italic>mask_mAP<\/jats:italic>\n                    ). The experimental results showed that the proposed MS-ADS model effectively detected and segmented apples under various conditions, such as apples occluded by branches, leaves and other apples, apples with different ground colors and shadows, and apples divided into parts by branches and petioles. The\n                    <jats:italic>recall<\/jats:italic>\n                    ,\n                    <jats:italic>precision<\/jats:italic>\n                    , false detection rate, and\n                    <jats:italic>F<\/jats:italic>\n                    1 score were 97.4%, 96.5%, 3.5%, and 96.9%, respectively. A\n                    <jats:italic>bbox_mAP<\/jats:italic>\n                    and\n                    <jats:italic>mask_mAP<\/jats:italic>\n                    of 0.932 and 0.920, respectively, were achieved on the test set, and the average run-time was 0.27 s per image. The experimental results indicated that the MS-ADS method detected and segmented apples in the orchard robustly and accurately with real-time performance. This study lays a foundation for follow-up work, such as yield estimation, harvesting, and automatic and long-term acquisition of apple growth information.\n                  <\/jats:p>","DOI":"10.3389\/fpls.2022.1016470","type":"journal-article","created":{"date-parts":[[2022,12,2]],"date-time":"2022-12-02T03:52:09Z","timestamp":1669953129000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["Apple detection and instance segmentation in natural environments using an improved Mask Scoring R-CNN Model"],"prefix":"10.3389","volume":"13","author":[{"given":"Dandan","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongjian","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2022,12,2]]},"reference":[{"key":"B1","first-page":"269","article-title":"A review on fruit detection and segmentation techniques in agricultural field","volume-title":"Second international conference on image processing and capsule networks, (ICIPCN)","author":"Barbole","year":"2021"},{"key":"B2","doi-asserted-by":"crossref","first-page":"9157","DOI":"10.1109\/ICCV.2019.00925","article-title":"YOLACT: Real-time instance segmentation","volume-title":"In 2019 IEEE\/CVF international conference on computer vision (ICCV)","author":"Bolya","year":"2019"},{"key":"B3","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1016\/j.patrec.2021.04.022","article-title":"Deep learning-based apple detection using a suppression mask r-CNN","volume":"147","author":"Chu","year":"2021","journal-title":"Pattern Recong. Lett."},{"key":"B4","first-page":"2276","article-title":"The VIA annotation software for images, audio and video","author":"Dutta","year":"2019"},{"key":"B5","doi-asserted-by":"publisher","first-page":"105687","DOI":"10.1016\/j.compag.2020.105687","article-title":"Application of consumer RGB-d cameras for fruit detection and localization in field: A critical review","volume":"177","author":"Fu","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"B6","doi-asserted-by":"crossref","first-page":"3141","DOI":"10.1109\/CVPR.2019.00326","article-title":"Dual attention network for scene segmentation","volume-title":"In 2019 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Fu","year":"2019"},{"key":"B7","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1016\/j.compag.2015.10.022","article-title":"Apple crop-load estimation with over-the-row machine vision system","volume":"120","author":"Gongal","year":"2015","journal-title":"Comput. Electron. Agric."},{"key":"B8","volume-title":"Digital image processing","author":"Gonzalez","year":"2020"},{"key":"B9","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.neucom.2015.09.116","article-title":"Deep learning for visual understanding: A review","volume":"187","author":"Guo","year":"2016","journal-title":"Neurocomputing"},{"key":"B10","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1109\/TPAMI.2018.2844175","article-title":"Mask r-CNN","volume":"42","author":"He","year":"2020","journal-title":"IEEE T. Pattern Anal."},{"key":"B11","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"In proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)","author":"He","year":"2016"},{"key":"B12","doi-asserted-by":"crossref","first-page":"6402","DOI":"10.1109\/CVPR.2019.00657","article-title":"Mask scoring r-CNN","volume-title":"In 2019 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Huang","year":"2019"},{"key":"B13","doi-asserted-by":"crossref","first-page":"7132","DOI":"10.1109\/CVPR.2018.00745","article-title":"Squeeze-and-excitation networks","volume-title":"In 2018 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Hu","year":"2018"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2022.765523","article-title":"YOLOF-snake: An efficient segmentation model for green object fruit","volume":"13","author":"Jia","year":"2022","journal-title":"Front. Plant Sci."},{"key":"B15","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1007\/s11119-021-09849-0","article-title":"Fusion of the YOLO V4 network model and visual attention mechanism to detect low-quality young apples in a complex environment","volume":"23","author":"Jiang","year":"2022","journal-title":"Precis. Agric."},{"key":"B16","doi-asserted-by":"publisher","first-page":"105380","DOI":"10.1016\/j.compag.2020.105380","article-title":"Detection and segmentation of overlapped fruits based on optimized mask r-CNN application in apple harvesting robot","volume":"172","author":"Jia","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"B17","doi-asserted-by":"publisher","first-page":"5156","DOI":"10.1016\/j.jksuci.2022.01.005","article-title":"A fast and efficient green apple object detection model based on foveabox","volume":"34","author":"Jia","year":"2022","journal-title":"J. King Saud. Univ. Com."},{"key":"B18","doi-asserted-by":"publisher","first-page":"106488","DOI":"10.1016\/j.compag.2021.106488","article-title":"Foveamask: A fast and accurate deep learning model for green fruit instance segmentation","volume":"191","author":"Jia","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"B19","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1007\/s11119-021-09846-3","article-title":"RS-net: Robust segmentation of green overlapped apples","volume":"23","author":"Jia","year":"2022","journal-title":"Precis. Agric."},{"key":"B20","doi-asserted-by":"publisher","first-page":"4599","DOI":"10.3390\/s19204599","article-title":"Fruit detection and segmentation for apple harvesting using visual sensor in orchards","volume":"19","author":"Kang","year":"2019","journal-title":"Sensors"},{"key":"B21","doi-asserted-by":"publisher","first-page":"105302","DOI":"10.1016\/j.compag.2020.105302","article-title":"Fruit detection, segmentation and 3D visualisation of environments in apple orchards","volume":"171","author":"Kang","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"B22","doi-asserted-by":"publisher","first-page":"105900","DOI":"10.1016\/j.compag.2020.105900","article-title":"A novel green apple segmentation algorithm based on ensemble U-net under complex orchard environment","volume":"180","author":"Li","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"B23","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1016\/j.compag.2015.04.005","article-title":"Apple detection in nighttime tree images using the geometry of light patches around highlights","volume":"114","author":"Linker","year":"2015","journal-title":"Comput. Electron. Agric."},{"key":"B24","first-page":"740","article-title":"Microsoft COCO: Common objects in context","volume-title":"In European conference on computer vision","author":"Lin","year":"2014"},{"key":"B25","doi-asserted-by":"publisher","first-page":"106984","DOI":"10.1016\/j.compag.2022.106984","article-title":"An accurate detection and segmentation model of obscured green fruits","volume":"197","author":"Liu","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"B26","first-page":"11976","article-title":"A ConvNet for the 2020s","volume-title":"In proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Liu","year":"2022"},{"key":"B27","doi-asserted-by":"publisher","first-page":"7259","DOI":"10.1016\/j.jksuci.2021.09.023","article-title":"DLNet: Accurate segmentation of green fruit in obscured environments - ScienceDirect","volume":"34","author":"Liu","year":"2022","journal-title":"J. King Saud. Univ. Com."},{"key":"B28","doi-asserted-by":"crossref","first-page":"510","DOI":"10.1109\/CVPR.2019.00060","article-title":"Selective kernel networks","volume-title":"In 2019 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Li","year":"2019"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2021.684328","article-title":"Intelligent fruit yield estimation for orchards using deep learning based semantic segmentation techniques\u2013a review","volume":"12","author":"Maheswari","year":"2021","journal-title":"Front. Plant Sci."},{"key":"B30","doi-asserted-by":"publisher","first-page":"3443","DOI":"10.3390\/app10103443","article-title":"A review of convolutional neural network applied to fruit image processing","volume":"10","author":"Naranjo-Torres","year":"2020","journal-title":"Appl. Sci."},{"key":"B31","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.biosystemseng.2016.01.007","article-title":"). detection of red and bicolored apples on tree with an RGB-d camera","volume":"146","author":"Nguyen","year":"2016","journal-title":"Biosyst. Eng."},{"key":"B32","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1016\/j.compag.2011.01.007","article-title":"Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry","volume":"76","author":"Rakun","year":"2011","journal-title":"Comput. Electron. Agric."},{"key":"B33","first-page":"234","article-title":"U-Net: Convolutional networks for biomedical image segmentation","volume-title":"International conference on medical image computing and computer-assisted intervention","author":"Ronneberger","year":"2015"},{"key":"B34","doi-asserted-by":"crossref","first-page":"2203","DOI":"10.1109\/ICPR48806.2021.9412258","article-title":"A novel region of interest extraction layer for instance segmentation","volume-title":"2020 25th international conference on pattern recognition (ICPR)","author":"Rossi","year":"2021"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1007\/s11119-021-09806-x","article-title":"Automation in agriculture by machine and deep learning techniques: A review of recent developments","volume":"22","author":"Saleem","year":"2021","journal-title":"Precis. Agric."},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.3389\/fpls.2020.00510","article-title":"Recognition and localization methods for vision-based fruit picking robots: A review","volume":"11","author":"Tang","year":"2020","journal-title":"Front. Plant Sci."},{"key":"B37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.inpa.2019.09.006","article-title":"Computer vision technology in agricultural automation \u2014a review","volume":"7","author":"Tian","year":"2020","journal-title":"Inf. Process. Agric."},{"key":"B38","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1016\/j.biosystemseng.2020.03.008","article-title":"Instance segmentation of apple flowers using the improved mask r\u2013CNN model","volume":"193","author":"Tian","year":"2020","journal-title":"Biosyst. Eng."},{"key":"B39","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1016\/j.compag.2019.01.012","article-title":"Apple detection during different growth stages in orchards using the improved YOLO-V3 model","volume":"157","author":"Tian","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"B40","doi-asserted-by":"publisher","first-page":"3251","DOI":"10.3390\/s21093251","article-title":"Automatic detection and segmentation for group housed pigs based on PigMS r-CNN","volume":"21","author":"Tu","year":"2021","journal-title":"Sensors"},{"key":"B41","doi-asserted-by":"crossref","first-page":"7794","DOI":"10.1109\/CVPR.2018.00813","article-title":"Non-local neural networks","volume-title":"In 2018 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Wang","year":"2018"},{"key":"B42","doi-asserted-by":"publisher","first-page":"106864","DOI":"10.1016\/j.compag.2022.106864","article-title":"Fusion of mask r-CNN and attention mechanism for instance segmentation of apples under complex background","volume":"196","author":"Wang","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"B43","first-page":"1492","article-title":"Aggregated residual transformations for deep neural networks","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Xie","year":"2017"},{"key":"B44","first-page":"12193","article-title":"Polarmask: Single shot instance segmentation with polar representation","volume-title":"In proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"Xie","year":"2020"},{"key":"B45","first-page":"2736","article-title":"ResNeSt: Split-attention","volume-title":"In proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Zhang","year":"2022"},{"key":"B46","doi-asserted-by":"publisher","first-page":"568","DOI":"10.1007\/s11119-012-9269-2","article-title":"Using color features of CV. \u2018Gala\u2019 apple fruits in an orchard in image processing to predict yield","volume":"13","author":"Zhou","year":"2012","journal-title":"Precis. Agric."},{"key":"B47","first-page":"1","article-title":"An empirical study of spatial attention mechanisms in deep networks","volume-title":"In 2019 IEEE\/CVF international conference on computer vision (ICCV)","author":"Zhu","year":"2019"}],"container-title":["Frontiers in Plant Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2022.1016470\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,2]],"date-time":"2022-12-02T03:52:25Z","timestamp":1669953145000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpls.2022.1016470\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,2]]},"references-count":47,"alternative-id":["10.3389\/fpls.2022.1016470"],"URL":"https:\/\/doi.org\/10.3389\/fpls.2022.1016470","relation":{},"ISSN":["1664-462X"],"issn-type":[{"value":"1664-462X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,2]]},"article-number":"1016470"}}