{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T01:13:47Z","timestamp":1778030027566,"version":"3.51.4"},"reference-count":40,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,2,22]],"date-time":"2026-02-22T00:00:00Z","timestamp":1771718400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Accurate apple detection and precise three-dimensional (3D) localisation are essential for autonomous robotic harvesting in orchard environments, where occlusion, illumination variation, depth noise, and the similar colour appearance of fruits and surrounding leaves present significant challenges. This paper proposes a dual-detector vision framework combined with depth-aware back-projection to achieve robust apple detection and metric 3D localisation in real time. The method integrates the complementary strengths of YOLOv8 and Mask R-CNN through confidence-weighted fusion of bounding boxes and pixel-wise union of segmentation masks, producing stabilised two-dimensional (2D) apple representations under visually ambiguous conditions. The fusion results are converted into dense 3D representations through depth-guided projection within the camera coordinate system representing the visible fruit surface. A depth-consistency weighting strategy assigns higher influence to depth-reliable pixels during centroid computation, thereby suppressing noisy or occluded depth measurements and improving the stability of 3D fruit centre estimation, while local intensity normalisation standardises neighbourhood-level pixel intensities to reduce the impact of shadows, highlights, and uneven lighting, enabling more consistent segmentation and detection across varying illumination conditions. Experimental results demonstrate an accuracy of 98.9%, an mAP of 94.2%, an F1-score of 93.3%, and a recall of 92.8%, while achieving real-time performance at 86.42 FPS, confirming the suitability of the proposed method for robotic harvesting in challenging orchard environments.<\/jats:p>","DOI":"10.3390\/robotics15020047","type":"journal-article","created":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T11:39:10Z","timestamp":1771846750000},"page":"47","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Dual-Detector Vision and Depth-Aware Back-Projection for Accurate Apple Detection and 3D Localisation for Robotic Harvesting"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8718-0627","authenticated-orcid":false,"given":"Tagor","family":"Hossain","sequence":"first","affiliation":[{"name":"School of Electrical and Mechanical Engineering, Adelaide University, North Terrace Campus, Adelaide, SA 5005, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8218-586X","authenticated-orcid":false,"given":"Peng","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Electrical and Mechanical Engineering, Adelaide University, North Terrace Campus, Adelaide, SA 5005, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3188-0800","authenticated-orcid":false,"given":"Levente","family":"Kovacs","sequence":"additional","affiliation":[{"name":"Research and Innovation Center, Obuda University, 1034 Budapest, Hungary"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.compag.2016.06.022","article-title":"A Review of Key Techniques of Vision-Based Control for Harvesting Robot","volume":"127","author":"Zhao","year":"2016","journal-title":"Comput. Electron. Agric."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1016\/j.procs.2018.05.069","article-title":"Conceptual Understanding of Convolutional Neural Network\u2014A Deep Learning Approach","volume":"132","author":"Indolia","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"93688","DOI":"10.1109\/ACCESS.2025.3573376","article-title":"A Review of YOLO Algorithm and Its Applications in Autonomous Driving Object Detection","volume":"13","author":"Wei","year":"2025","journal-title":"IEEE Access"},{"key":"ref_4","first-page":"91","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_5","unstructured":"Abdulla, W. (2026, January 01). Mask R-CNN for Object Detection and Instance Segmentation on Keras and TensorFlow. GitHub Repository. Available online: https:\/\/github.com\/matterport\/Mask_RCNN."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1016\/j.procs.2021.02.100","article-title":"A Survey of LiDAR and Camera Fusion Enhancement","volume":"183","author":"Zhong","year":"2021","journal-title":"Procedia Comput. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Gen\u00e9-Mola, J., Llorens, J., Rosell-Polo, J.R., Gregorio, E., Arn\u00f3, J., Solanelles, F., Mart\u00ednez-Casasnovas, J.A., and Escol\u00e0, A. (2020). Assessing the Performance of RGB-D Sensors for 3D Fruit Crop Canopy Characterization under Different Operating and Lighting Conditions. Sensors, 20.","DOI":"10.3390\/s20247072"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s40648-022-00230-y","article-title":"Fruit Recognition Method for a Harvesting Robot with RGB-D Cameras","volume":"9","author":"Yoshida","year":"2022","journal-title":"ROBOMECH J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"105687","DOI":"10.1016\/j.compag.2020.105687","article-title":"Application of Consumer RGB-D Cameras for Fruit Detection and Localization in Field: A Critical Review","volume":"177","author":"Fu","year":"2020","journal-title":"Comput. Electron. Agric."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"107000","DOI":"10.1016\/j.compag.2022.107000","article-title":"A Novel Apple Fruit Detection and Counting Methodology Based on Deep Learning and Trunk Tracking in Modern Orchard","volume":"197","author":"Gao","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, T., Feng, Q., Qiu, Q., Xie, F., and Zhao, C. (2022). Occluded Apple Fruit Detection and Localization with a Frustum-Based Point-Cloud-Processing Approach for Robotic Harvesting. Remote Sens., 14.","DOI":"10.3390\/rs14030482"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4101","DOI":"10.1002\/rob.70008","article-title":"In Situ Detection and Measurement of Broccoli Heads under Different Lighting Conditions Using Proximal Remote Sensing","volume":"42","author":"Wang","year":"2025","journal-title":"J. Field Robot."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"101068","DOI":"10.1016\/j.atech.2025.101068","article-title":"FruitQuery: A Lightweight Query-Based Instance Segmentation Model for In-Field Fruit Ripeness Determination","volume":"12","author":"Zhao","year":"2025","journal-title":"Smart Agric. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ma, F., Cavalheiro, G.V., and Karaman, S. (2019). Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera. Proc. IEEE Int. Conf. Robot. Autom., 3288\u20133295.","DOI":"10.1109\/ICRA.2019.8793637"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Yu, C., Zhang, J., Liu, Y., Kan, J., Zou, X., Zhang, K., Li, H., Zhang, S., and Wu, F. (2024). Vision-Based Perception and Execution Decision-Making for Fruit Picking Robots Using Generative AI Models. Machines, 14.","DOI":"10.3390\/machines14010117"},{"key":"ref_16","first-page":"1439","article-title":"Automated Cattle Detection Using Mask R-CNN and IoU-Based Tracking with a Single Side-View Camera","volume":"20","author":"Myint","year":"2024","journal-title":"Int. J. Innov. Comput. Inf. Control"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"129584","DOI":"10.1016\/j.neucom.2025.129584","article-title":"Advances in Instance Segmentation: Technologies, Metrics and Applications in Computer Vision","volume":"587","author":"Molina","year":"2025","journal-title":"Neurocomputing"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mihajlovic, M., and Marjanovic, M. (2025). Enhancing Instance Segmentation in High-Resolution Images Using Slicing-Aided Hyper Inference and Spatial Mask Merging Optimized via R-Tree Indexing. Mathematics, 13.","DOI":"10.3390\/math13193079"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1007\/s42979-022-01407-3","article-title":"A Survey on Object Instance Segmentation","volume":"3","author":"Sharma","year":"2022","journal-title":"SN Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1007\/s11263-019-01188-y","article-title":"Self-Supervised Model Adaptation for Multimodal Semantic Segmentation","volume":"128","author":"Valada","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_21","first-page":"1573","article-title":"Improving a Faster R-CNN Model for Vehicle Detection and Human Action Recognition at Night via Infrared Thermal Imaging Using Transfer Learning","volume":"20","author":"Liu","year":"2024","journal-title":"Int. J. Innov. Comput. Inf. Control"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1109\/TITS.2019.2891788","article-title":"High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion","volume":"21","author":"Park","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mousavian, A., Anguelov, D., Flynn, J., and Kosecka, J. (2017). 3D Bounding Box Estimation Using Deep Learning and Geometry. Comput. Vis. Pattern Recognit., 5632\u20135640.","DOI":"10.1109\/CVPR.2017.597"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"106696","DOI":"10.1016\/j.compag.2022.106696","article-title":"Canopy-Attention-YOLOv4-Based Immature\/Mature Apple Fruit Detection on Dense-Foliage Tree Architectures for Early Crop Load Estimation","volume":"193","author":"Lu","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, Y., and Chen, Y. (2020). Fruit Morphological Measurement Based on Three-Dimensional Reconstruction. Agronomy, 10.","DOI":"10.3390\/agronomy10040455"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"111068","DOI":"10.1016\/j.compag.2025.111068","article-title":"Process, Challenges and Solutions of Fruit 3D Reconstruction: A Review","volume":"239","author":"Li","year":"2025","journal-title":"Comput. Electron. Agric."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"764","DOI":"10.1007\/s11119-022-09974-4","article-title":"Three-Dimensional Location Methods for the Vision System of Strawberry-Harvesting Robots: Development and Comparison","volume":"24","author":"Ge","year":"2023","journal-title":"Precis. Agric."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"He, X., Zhang, X., Wang, Y., Ji, H., Duan, X., and Guo, F. (2022). Spatial Attention Frustum: A 3D Object Detection Method Focusing on Occluded Objects. Sensors, 22.","DOI":"10.3390\/s22062366"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3063","DOI":"10.1002\/rob.22567","article-title":"Geometry-Aware 3D Point Cloud Learning for Precise Cutting-Point Detection in Unstructured Field Environments","volume":"42","author":"Wang","year":"2025","journal-title":"J. Field Robot."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"111213","DOI":"10.1016\/j.compag.2025.111213","article-title":"Multi-Model Ensembles for Object Detection in Multispectral Images: A Case Study for Precision Agriculture","volume":"240","author":"Scutelnic","year":"2026","journal-title":"Comput. Electron. Agric."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/s00138-021-01261-y","article-title":"MLMT-CNN for Object Detection and Segmentation in Multi-Layer and Multi-Spectral Images","volume":"33","author":"Almahasneh","year":"2022","journal-title":"Mach. Vis. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kutyrev, A., Andriyanov, N., Khort, D., Smirnov, I., and Zubina, V. (2025). Adaptive CNN Ensemble for Apple Detection: Enabling Sustainable Monitoring Orchard. AgriEngineering, 7.","DOI":"10.3390\/agriengineering7110369"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yang, X., Gao, Y., Yin, M., and Li, H. (2024). Automatic Apple Detection and Counting with AD-YOLO and MR-SORT. Sensors, 24.","DOI":"10.3390\/s24217012"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"110252","DOI":"10.1016\/j.scienta.2021.110252","article-title":"Real-Time Recognition of on-Branch Olive Ripening Stages by a Deep Convolutional Neural Network","volume":"287","author":"Khosravi","year":"2021","journal-title":"Sci. Hortic."},{"key":"ref_35","first-page":"1493","article-title":"Strip Surface Defect Detection Based on Improved YOLOv7","volume":"20","author":"Wu","year":"2024","journal-title":"Int. J. Innov. Comput. Inf. Control"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6921","DOI":"10.1109\/TCYB.2024.3424430","article-title":"Multilevel Fine-Grained Features-Based General Framework for Object Detection","volume":"54","author":"Zuo","year":"2024","journal-title":"IEEE Trans. Cybern."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4643","DOI":"10.1109\/TCYB.2023.3300963","article-title":"Multidomain Object Detection Framework Using Feature Domain Knowledge Distillation","volume":"54","author":"Jaw","year":"2024","journal-title":"IEEE Trans. Cybern."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yan, B., and Li, X. (2024). RGB-D Camera and Fractal-Geometry-Based Maximum Diameter Estimation Method of Apples for Robot Intelligent Selective Graded Harvesting. Fractal Fract., 8.","DOI":"10.3390\/fractalfract8110649"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"62151","DOI":"10.1109\/ACCESS.2020.2984556","article-title":"Visual Perception and Modeling for Autonomous Apple Harvesting","volume":"8","author":"Kang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sekharamantry, P.K., Melgani, F., Malacarne, J., Ricci, R., de Almeida Silva, R., and Marcato, J. (2024). A Seamless Deep Learning Approach for Apple Detection, Depth Estimation, and Tracking Using YOLO Models Enhanced by Multi-Head Attention Mechanism. Computers, 13.","DOI":"10.3390\/computers13030083"}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/15\/2\/47\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T05:25:24Z","timestamp":1772083524000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/15\/2\/47"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,22]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["robotics15020047"],"URL":"https:\/\/doi.org\/10.3390\/robotics15020047","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,22]]}}}