{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T12:22:20Z","timestamp":1775218940786,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2021,11,25]],"date-time":"2021-11-25T00:00:00Z","timestamp":1637798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100012905","name":"Department of Science and Technology of Shandong Province","doi-asserted-by":"publisher","award":["2019GNC106103"],"award-info":[{"award-number":["2019GNC106103"]}],"id":[{"id":"10.13039\/100012905","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012905","name":"Department of Science and Technology of Shandong Province","doi-asserted-by":"publisher","award":["SD2019ZZ019"],"award-info":[{"award-number":["SD2019ZZ019"]}],"id":[{"id":"10.13039\/100012905","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012905","name":"Department of Science and Technology of Shandong Province","doi-asserted-by":"publisher","award":["2020KJTPY078"],"award-info":[{"award-number":["2020KJTPY078"]}],"id":[{"id":"10.13039\/100012905","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Since the mature green tomatoes have color similar to branches and leaves, some are shaded by branches and leaves, and overlapped by other tomatoes, the accurate detection and location of these tomatoes is rather difficult. This paper proposes to use the Mask R-CNN algorithm for the detection and segmentation of mature green tomatoes. A mobile robot is designed to collect images round-the-clock and with different conditions in the whole greenhouse, thus, to make sure the captured dataset are not only objects with the interest of users. After the training process, RestNet50-FPN is selected as the backbone network. Then, the feature map is trained through the region proposal network to generate the region of interest (ROI), and the ROIAlign bilinear interpolation is used to calculate the target region, such that the corresponding region in the feature map is pooled to a fixed size based on the position coordinates of the preselection box. Finally, the detection and segmentation of mature green tomatoes is realized by the parallel actions of ROI target categories, bounding box regression and mask. When the Intersection over Union is equal to 0.5, the performance of the trained model is the best. The experimental results show that the F1-Score of bounding box and mask region all achieve 92.0%. The image acquisition processes are fully unobservable, without any user preselection, which are a highly heterogenic mix, the selected Mask R-CNN algorithm could also accurately detect mature green tomatoes. The performance of this proposed model in a real greenhouse harvesting environment is also evaluated, thus facilitating the direct application in a tomato harvesting robot.<\/jats:p>","DOI":"10.3390\/s21237842","type":"journal-article","created":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T01:45:02Z","timestamp":1638323102000},"page":"7842","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6893-3817","authenticated-orcid":false,"given":"Linlu","family":"Zu","sequence":"first","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai\u2019an 271018, China"}]},{"given":"Yanping","family":"Zhao","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai\u2019an 271018, China"}]},{"given":"Jiuqin","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai\u2019an 271018, China"}]},{"given":"Fei","family":"Su","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai\u2019an 271018, China"}]},{"given":"Yan","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Shandong Agricultural University, Tai\u2019an 271018, China"}]},{"given":"Pingzeng","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Shandong Agricultural University, Tai\u2019an 271018, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1554","DOI":"10.3389\/fpls.2019.01554","article-title":"Tomato Fruit Development and Metabolism","volume":"10","author":"Quinet","year":"2019","journal-title":"Front. Plant Sci."},{"key":"ref_2","unstructured":"(2021, September 12). FAOSTAT. Available online: http:\/\/www.fao.org\/faostat\/en\/#home."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"S2091","DOI":"10.1111\/1750-3841.12638","article-title":"Consumer attitudes and preferences for fresh market tomatoes","volume":"79","author":"Oltman","year":"2014","journal-title":"J. Food Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.postharvbio.2013.07.009","article-title":"Nondestructive maturity determination in green tomatoes using a handheld visible and near infrared instrument","volume":"86","author":"Tiwari","year":"2013","journal-title":"Postharvest Biol. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"111287","DOI":"10.1016\/j.postharvbio.2020.111287","article-title":"Early discrimination of mature-and immature-green tomatoes (Solanum lycopersicum L.) using fluorescence imaging method","volume":"169","author":"Fatchurrahman","year":"2020","journal-title":"Postharvest Biol. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.postharvbio.2013.12.007","article-title":"Short period irradiation of single blue wavelength light extends the storage period of mature green tomatoes","volume":"90","author":"Dhakal","year":"2014","journal-title":"Postharvest Biol. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.biotechadv.2009.10.002","article-title":"Ripening of fleshy fruit: Molecular insight and the role of ethylene","volume":"28","author":"Bapat","year":"2010","journal-title":"Biotechnol. Adv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1002\/rob.21937","article-title":"Development of a sweet pepper harvesting robot","volume":"37","author":"Arad","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1002\/rob.21889","article-title":"An autonomous strawberry-harvesting robot: Design, development, integration, and field evaluation","volume":"37","author":"Xiong","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/1729881420925310","article-title":"Apple harvesting robot under information technology: A review","volume":"17","author":"Jia","year":"2020","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.compag.2016.06.022","article-title":"A review of key techniques of vision-based control for harvesting robot","volume":"127","author":"Zhao","year":"2016","journal-title":"Comput. Electron. Agric."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.compag.2018.01.011","article-title":"A methodology for fresh tomato maturity detection using computer vision","volume":"146","author":"Wan","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.biosystemseng.2016.05.001","article-title":"Detecting tomatoes in greenhouse scenes by combining AdaBoost classifier and colour analysis","volume":"148","author":"Zhao","year":"2016","journal-title":"Biosyst. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Gong, L., Huang, Y., and Liu, C. (2016). Robust Tomato Recognition for Robotic Harvesting Using Feature Images Fusion. Sensors, 16.","DOI":"10.3390\/s16020173"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"12191","DOI":"10.3390\/s140712191","article-title":"On plant detection of intact tomato fruits using image analysis and machine learning methods","volume":"14","author":"Yamamoto","year":"2014","journal-title":"Sensors"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103090","DOI":"10.1016\/j.micpro.2020.103090","article-title":"A microcontroller based machine vision approach for tomato grading and sorting using SVM classifier","volume":"76","author":"Esakkirajan","year":"2020","journal-title":"Microprocess. Microsyst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.compag.2015.05.021","article-title":"Sensors and systems for fruit detection and localization: A review","volume":"116","author":"Gongal","year":"2015","journal-title":"Comput. Electron. Agric."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liakos, K.G., Busato, P., Moshou, D., Pearson, S., and Bochtis, D. (2018). Machine Learning in Agriculture: A Review. Sensors, 18.","DOI":"10.3390\/s18082674"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.compag.2019.04.017","article-title":"Deep learning\u2014Method overview and review of use for fruit detection and yield estimation","volume":"162","author":"Koirala","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.compag.2018.02.016","article-title":"Deep learning in agriculture: A survey","volume":"147","author":"Kamilaris","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Stein, M., Bargoti, S., and Underwood, J. (2016). Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry. Sensors, 16.","DOI":"10.3390\/s16111915"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., and McCool, C. (2016). DeepFruits: A Fruit Detection System Using Deep Neural Networks. Sensors, 16.","DOI":"10.3390\/s16081222"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1107","DOI":"10.1007\/s11119-019-09642-0","article-title":"Deep learning for real-time fruit detection and orchard fruit load estimation: Benchmarking of \u2018MangoYOLO\u2019","volume":"20","author":"Koirala","year":"2019","journal-title":"Precis. Agric."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1016\/j.compag.2019.05.016","article-title":"Ruiz-Hidalgo, J.; Gregorio, E. Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities","volume":"162","author":"Vilaplana","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sun, J., He, X., Ge, X., Wu, X., Shen, J., and Song, Y. (2018). Detection of Key Organs in Tomato Based on Deep Migration Learning in a Complex Background. Agriculture, 8.","DOI":"10.20944\/preprints201810.0524.v1"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, G., Nouaze, J.C., Touko Mbouembe, P.L., and Kim, J.H. (2020). YOLO-Tomato: A Robust Algorithm for Tomato Detection Based on YOLOv3. Sensors, 20.","DOI":"10.3390\/s20072145"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"55289","DOI":"10.1109\/ACCESS.2020.2981823","article-title":"Fast Method of Detecting Tomatoes in a Complex Scene for Picking Robots","volume":"8","author":"Xu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"105302","DOI":"10.1016\/j.compag.2020.105302","article-title":"Fruit detection, segmentation and 3D visualisation of environments in apple orchards","volume":"171","author":"Kang","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"104846","DOI":"10.1016\/j.compag.2019.06.001","article-title":"Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN","volume":"163","author":"Yu","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kang, H., Zhou, H., Wang, X., and Chen, C. (2020). Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting. Sensors, 20.","DOI":"10.3390\/s20195670"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"207672","DOI":"10.1109\/ACCESS.2020.3038184","article-title":"Using Fuzzy Mask R-CNN Model to Automatically Identify Tomato Ripeness","volume":"8","author":"Huang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1759","DOI":"10.3389\/fpls.2020.571299","article-title":"Tomato Fruit Detection and Counting in Greenhouses Using Deep Learning","volume":"11","author":"Afonso","year":"2020","journal-title":"Front. Plant Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1007\/s00138-021-01202-9","article-title":"Automatic visual estimation of tomato cluster maturity in plant rows","volume":"32","author":"Tenorio","year":"2021","journal-title":"Mach. Vis. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Benavides, M., Cant\u00f3n-Garb\u00edn, M., S\u00e1nchez-Molina, J.A., and Rodr\u00edguez, F. (2020). Automatic tomato and peduncle location system based on computer vision for use in robotized harvesting. Appl. Sci., 10.","DOI":"10.3390\/app10175887"},{"key":"ref_35","unstructured":"Pete, W. (2017, December 14). How Many Images Do You Need to Train A Neural Network. Available online: https:\/\/petewarden.com."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sun, C., Shrivastava, A., Singh, S., and Gupta, A. (2017, January 22\u201329). Revisiting unreasonable effectiveness of data in deep learning era. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.97"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Weinstein, B.G., Marconi, S., Bohlman, S.A., Zare, A., and White, E.P. (2019). Individual Tree-Crown Detection in RGB Imagery Using Semi-Supervised Deep Learning Neural Networks. Remote Sens., 11.","DOI":"10.1101\/532952"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Protopapadakis, E., Doulamis, A., Doulamis, N., and Maltezos, E. (2021). Stacked autoencoders driven by semi-supervised learning for building extraction from near infrared remote sensing imagery. Remote Sens., 13.","DOI":"10.3390\/rs13030371"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kiran, B.R., Thomas, D.M., and Parakkal, R. (2018). An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. J. Imaging, 4.","DOI":"10.3390\/jimaging4020036"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7842\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:35:44Z","timestamp":1760168144000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7842"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,25]]},"references-count":39,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21237842"],"URL":"https:\/\/doi.org\/10.3390\/s21237842","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,25]]}}}