{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:33:07Z","timestamp":1760059987784,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T00:00:00Z","timestamp":1753142400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union\u2014NextGenerationEU through the Recovery and Resilience Facility","award":["NPOO.C3.2.R3-I1.04.0181"],"award-info":[{"award-number":["NPOO.C3.2.R3-I1.04.0181"]}]},{"DOI":"10.13039\/501100000780","name":"Croatian Science Foundation","doi-asserted-by":"publisher","award":["NPOO.C3.2.R3-I1.04.0181"],"award-info":[{"award-number":["NPOO.C3.2.R3-I1.04.0181"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>This paper presents the PRONOBIS project, an ultrasound-only, robotically assisted, deep learning-based system for prostate scanning and biopsy treatment planning. The proposed system addresses the challenges of precise prostate segmentation, reconstruction and inter-operator variability by performing fully automated prostate scanning, real-time CNN-transformer-based image processing, 3D prostate reconstruction, and biopsy needle position planning. Fully automated prostate scanning is achieved by using a robotic arm equipped with an ultrasound system. Real-time ultrasound image processing utilizes state-of-the-art deep learning algorithms with intelligent post-processing techniques for precise prostate segmentation. To create a high-quality prostate segmentation dataset, this paper proposes a deep learning-based medical annotation platform, MedAP. For precise segmentation of the entire prostate sweep, DAF3D and MicroSegNet models are evaluated, and additional image post-processing methods are proposed. Three-dimensional visualization and prostate reconstruction are performed by utilizing the segmentation results and robotic positional data, enabling robust, user-friendly biopsy treatment planning. The real-time sweep scanning and segmentation operate at 30 Hz, which enable complete scan in 15 to 20 s, depending on the size of the prostate. The system is evaluated on prostate phantoms by reconstructing the sweep and by performing dimensional analysis, which indicates 92% and 98% volumetric accuracy on the tested phantoms. Three-dimansional prostate reconstruction takes approximately 3 s and enables fast and detailed insight for precise biopsy needle position planning.<\/jats:p>","DOI":"10.3390\/robotics14080100","type":"journal-article","created":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T15:00:42Z","timestamp":1753196442000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["PRONOBIS: A Robotic System for Automated Ultrasound-Based Prostate Reconstruction and Biopsy Planning"],"prefix":"10.3390","volume":"14","author":[{"given":"Matija","family":"Markulin","sequence":"first","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"given":"Luka","family":"Matijevi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"given":"Janko","family":"Jurdana","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"given":"Luka","family":"\u0160iktar","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6653-0720","authenticated-orcid":false,"given":"Branimir","family":"\u0106aran","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"given":"Toni","family":"Zekuli\u0107","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Zagreb, \u0160alata 3, 10000 Zagreb, Croatia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8596-1972","authenticated-orcid":false,"given":"Filip","family":"\u0160uligoj","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"given":"Bojan","family":"\u0160ekoranja","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]},{"given":"Tvrtko","family":"Hudolin","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Zagreb, \u0160alata 3, 10000 Zagreb, Croatia"}]},{"given":"Tomislav","family":"Kuli\u0161","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Zagreb, \u0160alata 3, 10000 Zagreb, Croatia"}]},{"given":"Bojan","family":"Jerbi\u0107","sequence":"additional","affiliation":[{"name":"Croatian Academy of Sciences and Arts, Trg Nikole \u0160ubi\u0107a Zrinskog 11, 10000 Zagreb, Croatia"},{"name":"RONNA Medical Ltd., Slavonska avenija 6, 10000 Zagreb, Croatia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6761-4336","authenticated-orcid":false,"given":"Marko","family":"\u0160vaco","sequence":"additional","affiliation":[{"name":"Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Ul. Ivana Lu\u010di\u0107a 5, 10000 Zagreb, Croatia"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1336","DOI":"10.1002\/ijc.35278","article-title":"The GLOBOCAN 2022 cancer estimates: Data sources, methods, and a snapshot of the cancer burden worldwide","volume":"156","author":"Filho","year":"2025","journal-title":"Int. J. Cancer"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1097\/00000478-198812000-00001","article-title":"Zonal distribution of prostatic adenocarcinoma. Correlation with histologic pattern and direction of spread","volume":"12","author":"McNeal","year":"1988","journal-title":"Am. J. Surg. Pathol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1016\/j.eururo.2016.07.041","article-title":"Comparing Three Different Techniques for Magnetic Resonance Imaging-targeted Prostate Biopsies: A Systematic Review of In-bore versus Magnetic Resonance Imaging-transrectal Ultrasound fusion versus Cognitive Registration. Is There a Preferred Technique?","volume":"71","author":"Wegelin","year":"2017","journal-title":"Eur. Urol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1038\/s41585-022-00648-4","article-title":"Quality checkpoints in the MRI-directed prostate cancer diagnostic pathway","volume":"20","author":"Barrett","year":"2023","journal-title":"Nat. Rev. Urol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"601","DOI":"10.21037\/tau.2020.01.33","article-title":"Robotic-assisted magnetic resonance imaging ultrasound fusion results in higher significant cancer detection compared to cognitive prostate targeting in biopsy naive men","volume":"9","author":"Patel","year":"2020","journal-title":"Transl. Androl. Urol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.diii.2022.11.005","article-title":"Artificial intelligence algorithms aimed at characterizing or detecting prostate cancer on MRI: How accurate are they when tested on independent cohorts?\u2014A systematic review","volume":"104","author":"Jaouen","year":"2023","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1393","DOI":"10.1007\/s11548-021-02437-7","article-title":"Toward autonomous robotic prostate biopsy: A pilot study","volume":"16","author":"Maris","year":"2021","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wetterauer, C., Trotsenko, P., Matthias, M.O., Breit, C., Keller, N., Meyer, A., Brantner, P., Vlajnic, T., Bubendorf, L., and Winkel, D.J. (2021). Diagnostic accuracy and clinical implications of robotic assisted MRI-US fusion guided target saturation biopsy of the prostate. Sci. Rep., 11.","DOI":"10.1038\/s41598-021-99854-0"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1111\/bju.15118","article-title":"Multiparametric MRI-ultrasonography software fusion prostate biopsy: Initial results using a stereotactic robotic-assisted transperineal prostate biopsy platform comparing systematic vs. targeted biopsy","volume":"126","author":"Lee","year":"2020","journal-title":"BJU Int."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4833","DOI":"10.21873\/anticanres.11045","article-title":"Multiparametric-Magnetic Resonance\/Ultrasound Fusion Targeted Prostate Biopsy Improves Agreement Between Biopsy and Radical Prostatectomy Gleason Score","volume":"36","author":"Porpiglia","year":"2016","journal-title":"Anticancer Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"117","DOI":"10.21278\/TOF.481055523","article-title":"An Analytical Inverse Kinematics Solution with the Avoidance of Joint Limits, Singularity and the Simulation of 7-DOF Anthropomorphic Manipulators","volume":"48","author":"Chou","year":"2024","journal-title":"Trans. FAMENA"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"0036850420970366","DOI":"10.1177\/0036850420970366","article-title":"Design and experimental study of a novel 7-DOF manipulator for transrectal ultrasound probe","volume":"103","author":"Zhang","year":"2020","journal-title":"Sci. Prog."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"113667","DOI":"10.1109\/ACCESS.2023.3323312","article-title":"Continuous Body Type Prostate Biopsy Robot for Confined Space Operation","volume":"11","author":"Duan","year":"2023","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1203","DOI":"10.1016\/j.urology.2011.07.1389","article-title":"Robotic transperineal prostate biopsy: Pilot clinical study","volume":"78","author":"Ho","year":"2011","journal-title":"Urology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.eururo.2022.12.007","article-title":"Vector Prostate Biopsy: A Novel Magnetic Resonance Imaging\/Ultrasound Image Fusion Transperineal Biopsy Technique Using Electromagnetic Needle Tracking Under Local Anaesthesia","volume":"83","author":"Fletcher","year":"2023","journal-title":"Eur. Urol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ipsen, S., Wulff, D., Kuhlemann, I., Schweikard, A., and Ernst, F. (2021). Towards automated ultrasound imaging\u2014robotic image acquisition in liver and prostate for long-term motion monitoring. Phys. Med. Biol., 66.","DOI":"10.1088\/1361-6560\/abf277"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e2266","DOI":"10.1002\/rcs.2266","article-title":"Development of a transperineal prostate biopsy robot guided by MRI-TRUS image","volume":"17","author":"Wang","year":"2021","journal-title":"Int. J. Med. Robot. Comput. Assist. Surg."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1109\/TMECH.2016.2618362","article-title":"MR Safe Robot, FDA Clearance, Safety and Feasibility Prostate Biopsy Clinical Trial","volume":"22","author":"Stoianovici","year":"2017","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1002\/jmri.24770","article-title":"3T MR-guided in-bore transperineal prostate biopsy: A comparison of robotic and manual needle-guidance templates: Robotic Template for MRI-Guided Biopsy","volume":"42","author":"Tilak","year":"2015","journal-title":"J. Magn. Reson. Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2527","DOI":"10.1109\/TBME.2019.2891240","article-title":"Robotic Transrectal Ultrasound Guided Prostate Biopsy","volume":"66","author":"Lim","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3090","DOI":"10.1118\/1.4950721","article-title":"Segmentation of prostate from ultrasound images using level sets on active band and intensity variation across edges","volume":"43","author":"Li","year":"2016","journal-title":"Med. Phys."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TSMC.1979.4310076","article-title":"A Threshold Selection Method from Gray-Level Histograms","volume":"9","author":"Otsu","year":"1979","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_23","first-page":"3523","article-title":"Image Segmentation Using Deep Learning: A Survey","volume":"44","author":"Minaee","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Munich, Germany.","DOI":"10.1007\/978-3-319-24571-3"},{"key":"ref_25","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021). An Image is Worth 16 \u00d7 16 Words: Transformers for Image Recognition at Scale. arXiv."},{"key":"ref_26","unstructured":"Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"102326","DOI":"10.1016\/j.compmedimag.2024.102326","article-title":"MicroSegNet: A deep learning approach for prostate segmentation on micro-ultrasound images","volume":"112","author":"Jiang","year":"2024","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Avolio, P.P., Lughezzani, G., Paciotti, M., Maffei, D., Uleri, A., Frego, N., Hurle, R., Lazzeri, M., Saita, A., and Guazzoni, G. (2021). The use of 29 MHz transrectal micro-ultrasound to stratify the prostate cancer risk in patients with PI-RADS III lesions at multiparametric MRI: A single institutional analysis. Urol. Oncol. Semin. Orig. Investig., 39.","DOI":"10.1016\/S2666-1683(21)00792-8"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1679","DOI":"10.1001\/jama.2025.3579","article-title":"Microultrasonography-Guided vs MRI-Guided Biopsy for Prostate Cancer Diagnosis: The OPTIMUM Randomized Clinical Trial","volume":"333","author":"Kinnaird","year":"2025","journal-title":"JAMA"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.1097\/JU.0000000000001639","article-title":"Micro-Ultrasound-Guided vs Multiparametric Magnetic Resonance Imaging-Targeted Biopsy in the Detection of Prostate Cancer: A Systematic Review and Meta-Analysis","volume":"205","author":"Sountoulides","year":"2021","journal-title":"J. Urol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e2245","DOI":"10.1002\/rcs.2245","article-title":"Frameless stereotactic brain biopsy: A prospective study on robot-assisted brain biopsies performed on 32 patients by using the RONNA G4 system","volume":"17","author":"Dlaka","year":"2021","journal-title":"Int. J. Med. Robot. Comput. Assist. Surg. MRCAS"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"rjac151","DOI":"10.1093\/jscr\/rjac151","article-title":"Frameless stereotactic brain biopsy and external ventricular drainage placement using the RONNA G4 system","volume":"2022","author":"Dlaka","year":"2022","journal-title":"J. Surg. Case Rep."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"eabm6074","DOI":"10.1126\/scirobotics.abm6074","article-title":"Robot Operating System 2: Design, architecture, and uses in the wild","volume":"7","author":"Macenski","year":"2022","journal-title":"Sci. Robot."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"6138","DOI":"10.21105\/joss.06138","article-title":"LBR-Stack: ROS 2 and Python Integration of KUKA FRI for Med and IIWA Robots","volume":"9","author":"Huber","year":"2024","journal-title":"J. Open Source Softw."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.Y. (2023, January 1\u20136). Segment Anything. Proceedings of the 2023 IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2768","DOI":"10.1109\/TMI.2019.2913184","article-title":"Deep Attentive Features for Prostate Segmentation in 3D Transrectal Ultrasound","volume":"38","author":"Wang","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Dollar, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"67456","DOI":"10.1109\/ACCESS.2021.3077037","article-title":"RobUSt\u2014An Autonomous Robotic Ultrasound System for Medical Imaging","volume":"9","author":"Suligoj","year":"2021","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"155","DOI":"10.21278\/TOF.483057523","article-title":"Human-Robot Interaction of a Craniotomy Robot Based on Fuzzy Model Reference Learning Control","volume":"48","author":"Zhang","year":"2024","journal-title":"Trans. FAMENA"}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/8\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:14:07Z","timestamp":1760033647000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/14\/8\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,22]]},"references-count":39,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["robotics14080100"],"URL":"https:\/\/doi.org\/10.3390\/robotics14080100","relation":{},"ISSN":["2218-6581"],"issn-type":[{"type":"electronic","value":"2218-6581"}],"subject":[],"published":{"date-parts":[[2025,7,22]]}}}