{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T11:24:56Z","timestamp":1772018696412,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T00:00:00Z","timestamp":1718323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000163","name":"ARC","doi-asserted-by":"publisher","award":["ITRH IH150100006"],"award-info":[{"award-number":["ITRH IH150100006"]}],"id":[{"id":"10.13039\/100000163","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study presents the Fast Fruit 3D Detector (FF3D), a novel framework that contains a 3D neural network for fruit detection and an anisotropic Gaussian-based next-best view estimator. The proposed one-stage 3D detector, which utilizes an end-to-end 3D detection network, shows superior accuracy and robustness compared to traditional 2D methods. The core of the FF3D is a 3D object detection network based on a 3D convolutional neural network (3D CNN) followed by an anisotropic Gaussian-based next-best view estimation module. The innovative architecture combines point cloud feature extraction and object detection tasks, achieving accurate real-time fruit localization. The model is trained on a large-scale 3D fruit dataset and contains data collected from an apple orchard. Additionally, the proposed next-best view estimator improves accuracy and lowers the collision risk for grasping. Thorough assessments on the test set and in a simulated environment validate the efficacy of our FF3D. The experimental results show an AP of 76.3%, an AR of 92.3%, and an average Euclidean distance error of less than 6.2 mm, highlighting the framework\u2019s potential to overcome challenges in orchard environments.<\/jats:p>","DOI":"10.3390\/s24123858","type":"journal-article","created":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T08:02:26Z","timestamp":1718352146000},"page":"3858","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["FF3D: A Rapid and Accurate 3D Fruit Detector for Robotic Harvesting"],"prefix":"10.3390","volume":"24","author":[{"given":"Tianhao","family":"Liu","sequence":"first","affiliation":[{"name":"Faculty of Engineering, Monash University, Clayton, VIC 3800, Australia"}]},{"given":"Xing","family":"Wang","sequence":"additional","affiliation":[{"name":"CSIRO\u2019s Data61, Level 5\/13 Garden St, Eveleigh, NSW 2015, Australia"}]},{"given":"Kewei","family":"Hu","sequence":"additional","affiliation":[{"name":"College of Engineering, South China Agriculture University, Guangzhou 510070, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6602-7714","authenticated-orcid":false,"given":"Hugh","family":"Zhou","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, Monash University, Clayton, VIC 3800, Australia"}]},{"given":"Hanwen","family":"Kang","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, Monash University, Clayton, VIC 3800, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1201-2039","authenticated-orcid":false,"given":"Chao","family":"Chen","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, Monash University, Clayton, VIC 3800, Australia"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,14]]},"reference":[{"key":"ref_1","unstructured":"Girshick, R. (1995, January 20\u201323). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Cambridge, MA, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_4","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kang, H., Zhou, H., Wang, X., and Chen, C. (2020). Real-time fruit recognition and grasping estimation for robotic apple harvesting. Sensors, 20.","DOI":"10.3390\/s20195670"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105165","DOI":"10.1016\/j.compag.2019.105165","article-title":"Fruit detection and 3D location using instance segmentation neural networks and structure-from-motion photogrammetry","volume":"169","author":"Morros","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, G., Nouaze, J.C., Touko Mbouembe, P.L., and Kim, J.H. (2020). YOLO-tomato: A robust algorithm for tomato detection based on YOLOv3. Sensors, 20.","DOI":"10.3390\/s20072145"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1002\/rob.22041","article-title":"Robotic harvesting of the occluded fruits with a precise shape and position reconstruction approach","volume":"39","author":"Gong","year":"2022","journal-title":"J. Field Robot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.compag.2018.02.016","article-title":"Deep learning in agriculture: A survey","volume":"147","author":"Kamilaris","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s40648-022-00233-9","article-title":"Automated harvesting by a dual-arm fruit harvesting robot","volume":"9","author":"Yoshida","year":"2022","journal-title":"Robomech J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1002\/rob.22178","article-title":"Development, integration, and field evaluation of an autonomous citrus-harvesting robot","volume":"40","author":"Yin","year":"2023","journal-title":"J. Field Robot."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Vrochidou, E., Tziridis, K., Nikolaou, A., Kalampokas, T., Papakostas, G.A., Pachidis, T.P., Mamalis, S., Koundouras, S., and Kaburlasos, V.G. (2021). An autonomous grape-harvester robot: Integrated system architecture. Electronics, 10.","DOI":"10.3390\/electronics10091056"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"102644","DOI":"10.1016\/j.mechatronics.2021.102644","article-title":"System design and control of an apple harvesting robot","volume":"79","author":"Zhang","year":"2021","journal-title":"Mechatronics"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.biosystemseng.2021.03.012","article-title":"A mango picking vision algorithm on instance segmentation and key point detection from RGB images in an open orchard","volume":"206","author":"Zheng","year":"2021","journal-title":"Biosyst. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1002\/rob.21973","article-title":"Performance improvements of a sweet pepper harvesting robot in protected cropping environments","volume":"37","author":"Lehnert","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rukhovich, D., Vorontsova, A., and Konushin, A. (2022, January 23\u201327). Fcaf3d: Fully convolutional anchor-free 3d object detection. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20080-9_28"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3592433","article-title":"3D Gaussian Splatting for Real-Time Radiance Field Rendering","volume":"42","author":"Kerbl","year":"2023","journal-title":"ACM Trans. Graph."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1856","DOI":"10.1007\/s11119-022-09913-3","article-title":"Intelligent robots for fruit harvesting: Recent developments and future challenges","volume":"23","author":"Zhou","year":"2022","journal-title":"Precis. Agric."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"510","DOI":"10.3389\/fpls.2020.00510","article-title":"Recognition and localization methods for vision-based fruit picking robots: A review","volume":"11","author":"Tang","year":"2020","journal-title":"Front. Plant Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.compag.2019.04.017","article-title":"Deep learning\u2013Method overview and review of use for fruit detection and yield estimation","volume":"162","author":"Koirala","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11119-019-09654-w","article-title":"Color-, depth-, and shape-based 3D fruit detection","volume":"21","author":"Lin","year":"2020","journal-title":"Precis. Agric."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"107450","DOI":"10.1016\/j.compag.2022.107450","article-title":"Accurate fruit localisation using high resolution LiDAR-camera fusion and instance segmentation","volume":"203","author":"Kang","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107233","DOI":"10.1016\/j.compag.2022.107233","article-title":"Mature pomegranate fruit detection and location combining improved F-PointNet with 3D point cloud clustering in orchard","volume":"200","author":"Yu","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"107884","DOI":"10.1016\/j.compag.2023.107884","article-title":"Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints","volume":"210","author":"Kok","year":"2023","journal-title":"Comput. Electron. Agric."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.compag.2019.01.012","article-title":"Apple detection during different growth stages in orchards using the improved YOLO-V3 model","volume":"157","author":"Tian","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"105687","DOI":"10.1016\/j.compag.2020.105687","article-title":"Application of consumer RGB-D cameras for fruit detection and localization in field: A critical review","volume":"177","author":"Fu","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18\u201323). Frustum pointnets for 3D object detection from rgb-d data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00102"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sukkar, F., Best, G., Yoo, C., and Fitch, R. (2019, January 20\u201324). Multi-robot region-of-interest reconstruction with Dec-MCTS. Proceedings of the 2019 International conference on robotics and automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793560"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Menon, R., Zaenker, T., Dengler, N., and Bennewitz, M. (2023, January 1\u20135). NBV-SC: Next best view planning based on shape completion for fruit mapping and reconstruction. Proceedings of the 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.","DOI":"10.1109\/IROS55552.2023.10341855"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1109\/JSEN.2020.3020626","article-title":"Deep 3D object detection networks using LiDAR data: A review","volume":"21","author":"Wu","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2018, January 18\u201323). Voxelnet: End-to-end learning for point cloud based 3d object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019, January 15\u201320). Pointpillars: Fast encoders for object detection from point clouds. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01298"},{"key":"ref_33","unstructured":"Qi, C.R., Litany, O., He, K., and Guibas, L.J. (November, January 27). Deep hough voting for 3D object detection in point clouds. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, Z., Zhang, Z., Cao, Y., Hu, H., and Tong, X. (2021, January 11\u201317). Group-free 3D object detection via transformers. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00294"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Choy, C., Gwak, J., and Savarese, S. (2019, January 15\u201320). 4D spatio-temporal convnets: Minkowski convolutional neural networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00319"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Gwak, J., Choy, C., and Savarese, S. (2020, January 23\u201328). Generative sparse detection networks for 3d single-shot object detection. Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part IV 16.","DOI":"10.1007\/978-3-030-58548-8_18"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"10812","DOI":"10.1109\/TNNLS.2022.3171553","article-title":"3D-DFM: Anchor-Free Multimodal 3-D Object Detection With Dynamic Fusion Module for Autonomous Driving","volume":"34","author":"Lin","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_38","unstructured":"Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2027","DOI":"10.3390\/s100302027","article-title":"Automatic chessboard detection for intrinsic and extrinsic camera parameter calibration","volume":"10","author":"Armingol","year":"2010","journal-title":"Sensors"},{"key":"ref_40","unstructured":"CloudCompare Development Team (2024, February 20). CloudCompare (Version 2.13.1) [GPL Software]. Available online: http:\/\/www.cloudcompare.org\/."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Song, S., Lichtenberg, S.P., and Xiao, J. (2015, January 7\u201312). SUN RGB-D: A RGB-D scene understanding benchmark suite. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298655"},{"key":"ref_42","unstructured":"Hahner, M., Dai, D., Liniger, A., and Gool, L.V. (2020). Quantifying Data Augmentation for LiDAR based 3D Object Detection. arXiv."},{"key":"ref_43","unstructured":"NVIDIA (2023). Isaac Sim User Guide, NVIDIA Corporation."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tang, J., Zhou, H., Chen, X., Hu, T., Ding, E., Wang, J., and Zeng, G. (2023). Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement. arXiv.","DOI":"10.1109\/ICCV51070.2023.01626"},{"key":"ref_45","unstructured":"Robotics, O. (2024, February 20). ROS Noetic Ninjemys, 2020. Version 1.15.11. Available online: https:\/\/www.ros.org\/noetic\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/12\/3858\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:58:54Z","timestamp":1760108334000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/12\/3858"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,14]]},"references-count":45,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24123858"],"URL":"https:\/\/doi.org\/10.3390\/s24123858","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,14]]}}}