{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T16:28:07Z","timestamp":1773246487167,"version":"3.50.1"},"reference-count":143,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T00:00:00Z","timestamp":1750636800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T00:00:00Z","timestamp":1750636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003151","name":"Fonds de recherche du Qu\u00e9bec\u2013 Nature et technologies","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003151","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["Horizon Grant"],"award-info":[{"award-number":["Horizon Grant"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["Horizon Grant"],"award-info":[{"award-number":["Horizon Grant"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["Horizon Grant"],"award-info":[{"award-number":["Horizon Grant"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["Horizon Grant"],"award-info":[{"award-number":["Horizon Grant"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004070","name":"Khalifa University of Science, Technology and Research","doi-asserted-by":"publisher","award":["Internal Grant"],"award-info":[{"award-number":["Internal Grant"]}],"id":[{"id":"10.13039\/501100004070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004070","name":"Khalifa University of Science, Technology and Research","doi-asserted-by":"publisher","award":["Internal Grant"],"award-info":[{"award-number":["Internal Grant"]}],"id":[{"id":"10.13039\/501100004070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004070","name":"Khalifa University of Science, Technology and Research","doi-asserted-by":"publisher","award":["Internal Grant"],"award-info":[{"award-number":["Internal Grant"]}],"id":[{"id":"10.13039\/501100004070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004070","name":"Khalifa University of Science, Technology and Research","doi-asserted-by":"publisher","award":["Internal Grant"],"award-info":[{"award-number":["Internal Grant"]}],"id":[{"id":"10.13039\/501100004070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002914","name":"Concordia University","doi-asserted-by":"publisher","award":["Internal Funding"],"award-info":[{"award-number":["Internal Funding"]}],"id":[{"id":"10.13039\/501100002914","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002914","name":"Concordia University","doi-asserted-by":"publisher","award":["Internal Funding"],"award-info":[{"award-number":["Internal Funding"]}],"id":[{"id":"10.13039\/501100002914","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002914","name":"Concordia University","doi-asserted-by":"publisher","award":["Internal Funding"],"award-info":[{"award-number":["Internal Funding"]}],"id":[{"id":"10.13039\/501100002914","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Medical Ultrasound (US) imaging has seen increasing demands over the past years, becoming one of the most preferred imaging modalities in clinical practice due to its affordability, portability, and real-time capabilities. However, it faces several challenges that limit its applicability, such as operator dependency, variability in interpretation, and limited resolution, which are amplified by the low availability of trained experts. This calls for the need of autonomous systems that are capable of reducing the dependency on humans for increased efficiency and throughput. Reinforcement Learning (RL) comes as a rapidly advancing field under Artificial Intelligence (AI) that allows the development of autonomous and intelligent agents through rewarded interactions with their environments. Several existing surveys on advancements in US imaging predominantly focus on partially autonomous AI solutions. However, none of these surveys explore the intersection between the stages of the US process and the recent advancements in RL solutions. To bridge this gap, this survey proposes a comprehensive taxonomy that integrates the stages of the US process with the RL development pipeline -including data preparation, problem formulation, simulation environment, RL training, validation and finetuning- and reviews current research efforts under this taxonomy. This work aims to highlight the potential of RL in building autonomous US solutions while identifying limitations and opportunities for further advancements in this field.<\/jats:p>","DOI":"10.1007\/s10462-025-11268-w","type":"journal-article","created":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T05:46:00Z","timestamp":1750657560000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Comprehensive review of reinforcement learning for medical ultrasound imaging"],"prefix":"10.1007","volume":"58","author":[{"given":"Hanae","family":"Elmekki","sequence":"first","affiliation":[]},{"given":"Saidul","family":"Islam","sequence":"additional","affiliation":[]},{"given":"Ahmed","family":"Alagha","sequence":"additional","affiliation":[]},{"given":"Hani","family":"Sami","sequence":"additional","affiliation":[]},{"given":"Amanda","family":"Spilkin","sequence":"additional","affiliation":[]},{"given":"Ehsan","family":"Zakeri","sequence":"additional","affiliation":[]},{"given":"Antonela Mariel","family":"Zanuttini","sequence":"additional","affiliation":[]},{"given":"Jamal","family":"Bentahar","sequence":"additional","affiliation":[]},{"given":"Lyes","family":"Kadem","sequence":"additional","affiliation":[]},{"given":"Wen-Fang","family":"Xie","sequence":"additional","affiliation":[]},{"given":"Philippe","family":"Pibarot","sequence":"additional","affiliation":[]},{"given":"Rabeb","family":"Mizouni","sequence":"additional","affiliation":[]},{"given":"Hadi","family":"Otrok","sequence":"additional","affiliation":[]},{"given":"Shakti","family":"Singh","sequence":"additional","affiliation":[]},{"given":"Azzam","family":"Mourad","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,6,23]]},"reference":[{"issue":"24","key":"11268_CR1","doi-asserted-by":"publisher","first-page":"21982","DOI":"10.1109\/JIOT.2023.3288050","volume":"10","author":"AA Abdellatif","year":"2023","unstructured":"Abdellatif AA, Mhaisen N, Mohamed A, Erbad A, Guizani M (2023) Reinforcement learning for intelligent healthcare systems: a review of challenges, applications, and open research issues. IEEE Internet Things J 10(24):21982\u201322007","journal-title":"IEEE Internet Things J"},{"key":"11268_CR2","volume-title":"Interventional endoscopic ultrasound","author":"DG Adler","year":"2018","unstructured":"Adler DG (2018) Interventional endoscopic ultrasound. Springer, Cham"},{"key":"11268_CR3","doi-asserted-by":"publisher","first-page":"7710","DOI":"10.1109\/JIOT.2023.3316078","volume":"11","author":"A Alagha","year":"2023","unstructured":"Alagha A, Bentahar J, Otrok H, Singh S, Mizouni R (2023) Blockchain-assisted demonstration cloning for multi-agent deep reinforcement learning. IEEE Internet Things J 11:7710","journal-title":"IEEE Internet Things J"},{"issue":"15","key":"11268_CR4","doi-asserted-by":"publisher","first-page":"13556","DOI":"10.1109\/JIOT.2023.3262663","volume":"10","author":"A Alagha","year":"2023","unstructured":"Alagha A, Mizouni R, Bentahar J, Otrok H, Singh S (2023) Multiagent deep reinforcement learning with demonstration cloning for target localization. IEEE Internet Things J 10(15):13556\u201313570","journal-title":"IEEE Internet Things J"},{"key":"11268_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2024.121107","volume":"679","author":"A Alagha","year":"2024","unstructured":"Alagha A, Otrok H, Singh S, Mizouni R, Bentahar J (2024) Blockchain-based crowdsourced deep reinforcement learning as a service. Inf Sci 679:121107","journal-title":"Inf Sci"},{"key":"11268_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.vehcom.2025.100874","volume":"52","author":"A Alagha","year":"2025","unstructured":"Alagha A, Kadadha M, Mizouni R, Singh S, Bentahar J, Otrok H (2025) Uav-assisted internet of vehicles: a framework empowered by reinforcement learning and blockchain. Vehic Commun 52:100874","journal-title":"Vehic Commun"},{"key":"11268_CR7","doi-asserted-by":"publisher","DOI":"10.1016\/j.iot.2024.101447","volume":"29","author":"A Alagha","year":"2024","unstructured":"Alagha A, Mizouni R, Singh S, Bentahar J, Otrok H (2024) Adaptive target localization under uncertainty using multi-agent deep reinforcement learning with knowledge transfer. Internet Things 29:101447","journal-title":"Internet Things"},{"key":"11268_CR8","first-page":"319","volume-title":"International conference on medical image computing and computer-assisted intervention","author":"AA Amadou","year":"2024","unstructured":"Amadou AA, Singh V, Ghesu FC, Kim Y-H, Stanciulescu L, Sai HP, Sharma P, Young A, Rajani R, Rhode K (2024) Goal-conditioned reinforcement learning for ultrasound navigation guidance. International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp 319\u2013329"},{"key":"11268_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2024.103345","volume":"99","author":"Y Ao","year":"2025","unstructured":"Ao Y, Esfandiari H, Carrillo F, Laux CJ, As Y, Li R, Van Assche K, Davoodi A, Cavalcanti NA, Farshad M (2025) Saferplan: safe deep reinforcement learning for intraoperative planning of pedicle screw placement. Med Image Anal 99:103345","journal-title":"Med Image Anal"},{"issue":"11","key":"11268_CR10","doi-asserted-by":"publisher","first-page":"1838","DOI":"10.1109\/JPROC.2021.3117472","volume":"109","author":"E Apostolidis","year":"2021","unstructured":"Apostolidis E, Adamantidou E, Metsai AI, Mezaris V, Patras I (2021) Video summarization using deep neural networks: a survey. Proc IEEE 109(11):1838\u20131863","journal-title":"Proc IEEE"},{"issue":"3","key":"11268_CR11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3447243","volume":"54","author":"D Avola","year":"2021","unstructured":"Avola D, Cinque L, Fagioli A, Foresti G, Mecca A (2021) Ultrasound medical imaging techniques: a survey. ACM Comput Surv (CSUR) 54(3):1\u201338","journal-title":"ACM Comput Surv (CSUR)"},{"key":"11268_CR12","first-page":"387","volume-title":"International workshop on machine learning in medical imaging","author":"M Bahrami","year":"2022","unstructured":"Bahrami M, Ghorbani M, Yeganeh Y, Navab N (2022) Intelligent masking: deep q-learning for context encoding in medical image analysis. International workshop on machine learning in medical imaging. Springer, Cham, pp 387\u2013396"},{"issue":"3","key":"11268_CR13","doi-asserted-by":"publisher","first-page":"6638","DOI":"10.1109\/LRA.2022.3176112","volume":"7","author":"Y Bi","year":"2022","unstructured":"Bi Y, Jiang Z, Gao Y, Wendler T, Karlas A, Navab N (2022) Vesnet-rl: simulation-based reinforcement learning for real-world us probe navigation. IEEE Robot Autom Lett 7(3):6638\u20136645","journal-title":"IEEE Robot Autom Lett"},{"key":"11268_CR14","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1146\/annurev-control-091523-100042","volume":"7","author":"Y Bi","year":"2024","unstructured":"Bi Y, Jiang Z, Duelmer F, Huang D, Navab N (2024) Machine learning in robotic ultrasound imaging: challenges and perspectives. Ann Rev Control, Robot, Autonom Syst 7:335","journal-title":"Ann Rev Control, Robot, Autonom Syst"},{"key":"11268_CR15","unstructured":"Bi Y, Qian C, Zhang Z, Navab N, Jiang Z (2024) Autonomous path planning for intercostal robotic ultrasound imaging using reinforcement learning. arXiv preprint arXiv:2404.09927"},{"key":"11268_CR16","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.cag.2024.01.003","volume":"118","author":"D Black","year":"2024","unstructured":"Black D, Nogami M, Salcudean S (2024) Mixed reality human teleoperation with device-agnostic remote ultrasound: communication and user interaction. Comput Graph 118:184\u2013193","journal-title":"Comput Graph"},{"issue":"2","key":"11268_CR17","doi-asserted-by":"publisher","first-page":"236","DOI":"10.3390\/diagnostics12020236","volume":"12","author":"RE Boh\u00eel\u0163ea","year":"2022","unstructured":"Boh\u00eel\u0163ea RE, Dima V, Ducu I, Iordache AM, Mihai BM, Munteanu O, Grigoriu C, Vedu\u0163\u0103 A, Pelinescu-Onciul D, Vl\u0103d\u0103reanu R (2022) Clinically relevant prenatal ultrasound diagnosis of umbilical cord pathology. Diagnostics 12(2):236","journal-title":"Diagnostics"},{"key":"11268_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2020.106767","volume":"87","author":"HR Boveiri","year":"2020","unstructured":"Boveiri HR, Khayami R, Javidan R, Mehdizadeh A (2020) Medical image registration using deep neural networks: a comprehensive review. Comput Electric Eng 87:106767","journal-title":"Comput Electric Eng"},{"key":"11268_CR19","doi-asserted-by":"publisher","first-page":"786","DOI":"10.1007\/s00261-018-1517-0","volume":"43","author":"LJ Brattain","year":"2018","unstructured":"Brattain LJ, Telfer BA, Dhyani M, Grajo JR, Samir AE (2018) Machine learning for medical ultrasound: status, methods, and future opportunities. Abdom Radiol 43:786\u2013799","journal-title":"Abdom Radiol"},{"issue":"5","key":"11268_CR20","first-page":"4446","volume":"35","author":"Q Cai","year":"2022","unstructured":"Cai Q, Cui C, Xiong Y, Wang W, Xie Z, Zhang M (2022) A survey on deep reinforcement learning for data processing and analytics. IEEE Trans Knowl Data Eng 35(5):4446\u20134465","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"11268_CR21","doi-asserted-by":"crossref","unstructured":"Chan V, Perlas A (2011) Basics of ultrasound imaging. Atlas of ultrasound-guided procedures in interventional pain management 13\u201319","DOI":"10.1007\/978-1-4419-1681-5_2"},{"key":"11268_CR22","unstructured":"Chaudhari K, Oza S, Dhumal Y, Shinde A, Sagar S, Chopade D (2023) The automated screening of ultrasound images for nuchal translucency using auxiliary u-net for semantic segmentation. In: 2023 10th International conference on computing for sustainable global development (INDIACom). IEEE, pp. 311\u2013315"},{"key":"11268_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2024.103385","volume":"100","author":"J Chen","year":"2024","unstructured":"Chen J, Liu Y, Wei S, Bian Z, Subramanian S, Carass A, Prince JL, Du Y (2024) A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond. Med Image Anal 100:103385","journal-title":"Med Image Anal"},{"key":"11268_CR24","doi-asserted-by":"crossref","unstructured":"Chen Y, Yang C, Li M, Dai S-L (2021) Learning to predict action based on b-ultrasound image information. In: 2021 6th IEEE International conference on advanced robotics and mechatronics (ICARM). IEEE, pp. 492\u2013497","DOI":"10.1109\/ICARM52023.2021.9536054"},{"key":"11268_CR25","doi-asserted-by":"crossref","unstructured":"Chiang AM, Chang PP, Broadstone SR (2000) Pc-based ultrasound imaging system in a probe. In: 2000 IEEE ultrasonics symposium. Proceedings. An International Symposium (Cat. No. 00CH37121), vol. 2. IEEE, pp. 1255\u20131260","DOI":"10.1109\/ULTSYM.2000.921551"},{"issue":"4","key":"11268_CR26","doi-asserted-by":"publisher","first-page":"865","DOI":"10.1016\/j.ultrasmedbio.2019.11.013","volume":"46","author":"K Christensen-Jeffries","year":"2020","unstructured":"Christensen-Jeffries K, Couture O, Dayton PA, Eldar YC, Hynynen K, Kiessling F, O\u2019Reilly M, Pinton GF, Schmitz G, Tang M-X (2020) Super-resolution ultrasound imaging. Ultrasound Med Biol 46(4):865\u2013891","journal-title":"Ultrasound Med Biol"},{"key":"11268_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2020.101964","volume":"109","author":"A Coronato","year":"2020","unstructured":"Coronato A, Naeem M, De Pietro G, Paragliola G (2020) Reinforcement learning for intelligent healthcare applications: a survey. Artif Intell Med 109:101964","journal-title":"Artif Intell Med"},{"issue":"3","key":"11268_CR28","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1007\/s10462-023-10635-9","volume":"57","author":"D Das","year":"2024","unstructured":"Das D, Iyengar MS, Majdi MS, Rodriguez JJ, Alsayed M (2024) Deep learning for thyroid nodule examination: a technical review. Artif Intell Rev 57(3):47","journal-title":"Artif Intell Rev"},{"key":"11268_CR29","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2024.3400342","author":"Z Deng","year":"2024","unstructured":"Deng Z, Hou X, Chen C, Gu X, Hou Z-G, Wang S (2024) A portable robot-assisted device with built-in intelligence for autonomous ultrasound acquisitions in follow-up diagnosis. IEEE Trans Instrum Meas. https:\/\/doi.org\/10.1109\/TIM.2024.3400342","journal-title":"IEEE Trans Instrum Meas"},{"key":"11268_CR30","doi-asserted-by":"crossref","unstructured":"Deng Z, Hou X, Wang S (2021) Workspace optimization of a 6-rss stewart-gough robotic platform to assist ultrasound diagnosis. In: 2021 IEEE 1st International conference on digital twins and parallel intelligence (DTPI). IEEE, pp. 230\u2013233","DOI":"10.1109\/DTPI52967.2021.9540100"},{"key":"11268_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.102900","volume":"70","author":"B Duan","year":"2021","unstructured":"Duan B, Xiong L, Guan X, Fu Y, Zhang Y (2021) Tele-operated robotic ultrasound system for medical diagnosis. Biomed Signal Process Control 70:102900","journal-title":"Biomed Signal Process Control"},{"key":"11268_CR32","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2024.3378915","author":"A Duan","year":"2024","unstructured":"Duan A, Yang C, Zhao J, Huo S, Zhou P, Ma W, Zheng Y, Navarro-Alarcon D (2024) Safe learning by constraint-aware policy optimization for robotic ultrasound imaging. IEEE Trans Autom Sci Eng. https:\/\/doi.org\/10.1109\/TASE.2024.3378915","journal-title":"IEEE Trans Autom Sci Eng"},{"issue":"9","key":"11268_CR33","first-page":"654","volume":"68","author":"D Esau","year":"2022","unstructured":"Esau D, Abramson BL (2022) Approach to risk stratification of atherosclerotic cardiovascular disease: use of biomarkers and imaging in a Canadian context. Can Fam Phys 68(9):654\u2013660","journal-title":"Can Fam Phys"},{"issue":"7","key":"11268_CR34","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1109\/TMM.2019.2960636","volume":"22","author":"R Furuta","year":"2019","unstructured":"Furuta R, Inoue N, Yamasaki T (2019) Pixelrl: Fully convolutional network with reinforcement learning for image processing. IEEE Trans Multimed 22(7):1704\u20131719","journal-title":"IEEE Trans Multimed"},{"key":"11268_CR35","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2022.879639","volume":"10","author":"W Gao","year":"2022","unstructured":"Gao W, Li X, Wang Y, Cai Y (2022) Medical image segmentation algorithm for three-dimensional multimodal using deep reinforcement learning and big data analytics. Front Public Health 10:879639","journal-title":"Front Public Health"},{"key":"11268_CR36","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2022.105572","volume":"117","author":"L Gao","year":"2023","unstructured":"Gao L, Yang S-L, Meng B, Tong G-X, Fan H-P, Yang G-S (2023) Frequency matching optimization model of ultrasonic scalpel transducer based on neural network and reinforcement learning. Eng Appl Artif Intell 117:105572","journal-title":"Eng Appl Artif Intell"},{"issue":"1","key":"11268_CR37","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1109\/TPAMI.2017.2782687","volume":"41","author":"F-C Ghesu","year":"2017","unstructured":"Ghesu F-C, Georgescu B, Zheng Y, Grbic S, Maier A, Hornegger J, Comaniciu D (2017) Multi-scale deep reinforcement learning for real-time 3d-landmark detection in ct scans. IEEE Trans Pattern Anal Mach Intell 41(1):176\u2013189","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"6","key":"11268_CR38","first-page":"31","volume":"35","author":"J Greenberg","year":"2023","unstructured":"Greenberg J (2023) Butterfly iq+: offering day-to-day portable ultrasound tech. OBG Manag 35(6):31\u201332","journal-title":"OBG Manag"},{"key":"11268_CR39","doi-asserted-by":"crossref","unstructured":"Hase H, Azampour MF, Tirindelli M, Paschali M, Simson W, Fatemizadeh E, Navab N (2020) Ultrasound-guided robotic navigation with deep reinforcement learning. In: 2020 IEEE\/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp. 5534\u20135541","DOI":"10.1109\/IROS45743.2020.9340913"},{"key":"11268_CR40","doi-asserted-by":"crossref","unstructured":"Herlin IL, Ayache N (1992) Features extraction and analysis methods for sequences of ultrasound images. In: Computer Vision\u2013ECCV\u201992: second European conference on computer vision Santa Margherita Ligure, Italy, May 19\u201322, 1992 Proceedings 2. Springer, pp. 43\u201357","DOI":"10.1007\/3-540-55426-2_6"},{"issue":"2","key":"11268_CR41","doi-asserted-by":"publisher","first-page":"13898","DOI":"10.1002\/acm2.13898","volume":"24","author":"M Hu","year":"2023","unstructured":"Hu M, Zhang J, Matkovic L, Liu T, Yang X (2023) Reinforcement learning in medical image analysis: concepts, applications, challenges, and future directions. J Appl Clin Med Phys 24(2):13898","journal-title":"J Appl Clin Med Phys"},{"key":"11268_CR42","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102137","volume":"72","author":"R Huang","year":"2021","unstructured":"Huang R, Lin Z, Dou H, Wang J, Miao J, Zhou G, Jia X, Xu W, Mei Z, Dong Y (2021) Aw3m: an auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound. Med Image Anal 72:102137","journal-title":"Med Image Anal"},{"key":"11268_CR43","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102490","volume":"80","author":"R Huang","year":"2022","unstructured":"Huang R, Ying Q, Lin Z, Zheng Z, Tan L, Tang G, Zhang Q, Luo M, Yi X, Liu P (2022) Extracting keyframes of breast ultrasound video using deep reinforcement learning. Med Image Anal 80:102490","journal-title":"Med Image Anal"},{"key":"11268_CR44","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126790","volume":"559","author":"Q Huang","year":"2023","unstructured":"Huang Q, Zhou J, Li Z (2023) Review of robot-assisted medical ultrasound imaging systems: technology and clinical applications. Neurocomputing 559:126790","journal-title":"Neurocomputing"},{"key":"11268_CR45","doi-asserted-by":"crossref","unstructured":"Huang J, Wang R, Jiang W, Shao S, Chen T (2023) Agent based fetal face segmentation for standard plane localization in 3d ultrasound. In: 2023 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp. 5317\u20135322","DOI":"10.1109\/SMC53992.2023.10393929"},{"key":"11268_CR46","doi-asserted-by":"crossref","unstructured":"Hu Y, Huang Y, Song A, Jones C, Siewerdsen J, Basar B, Helm P, Uneri A (2024) Probe positioning for robot-assisted intraoperative ultrasound imaging using deep reinforcement learning. In: Medical imaging 2024: image-guided procedures, robotic interventions, and modeling. SPIE, 12928, p. 1292803","DOI":"10.1117\/12.3006918"},{"key":"11268_CR47","unstructured":"iStock (2024) ovarian cancer ultrasound stock photos, images and royalty-free images - iStock. https:\/\/www.istockphoto.com. Accessed: 2024-11-03"},{"key":"11268_CR48","doi-asserted-by":"crossref","unstructured":"Jarosik P, Lewandowski M, Klimonda Z, Byra M (2021) Pixel-wise deep reinforcement learning approach for ultrasound image denoising. In: 2021 IEEE international ultrasonics symposium (IUS). IEEE, pp. 1\u20134","DOI":"10.1109\/IUS52206.2021.9593591"},{"issue":"8","key":"11268_CR49","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1007\/s10462-024-10807-1","volume":"57","author":"R Javed","year":"2024","unstructured":"Javed R, Abbas T, Khan AH, Daud A, Bukhari A, Alharbey R (2024) Deep learning for lungs cancer detection: a review. Artif Intell Rev 57(8):197","journal-title":"Artif Intell Rev"},{"key":"11268_CR50","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102878","volume":"89","author":"Z Jiang","year":"2023","unstructured":"Jiang Z, Salcudean SE, Navab N (2023) Robotic ultrasound imaging: state-of-the-art and future perspectives. Med Image Anal 89:102878","journal-title":"Med Image Anal"},{"key":"11268_CR51","doi-asserted-by":"crossref","unstructured":"Jiang B, Xu K, Taylor RH, Graham E, Unberath M, Boctor EM (2020) Standard plane extraction from 3d ultrasound with 6-dof deep reinforcement learning agent. In: 2020 IEEE international ultrasonics symposium (IUS). IEEE, pp. 1\u20134","DOI":"10.1109\/IUS46767.2020.9251555"},{"issue":"24","key":"11268_CR52","doi-asserted-by":"publisher","first-page":"3611","DOI":"10.3390\/diagnostics13243611","volume":"13","author":"N Karunanayake","year":"2023","unstructured":"Karunanayake N, Moodleah S, Makhanov SS (2023) Edge-driven multi-agent reinforcement learning: a novel approach to ultrasound breast tumor segmentation. Diagnostics 13(24):3611","journal-title":"Diagnostics"},{"key":"11268_CR53","first-page":"25","volume":"10","author":"P Kaushik","year":"2022","unstructured":"Kaushik P, Ratan R (2022) Eu-net: deep reinforcement learning aided breast tumor segmentation and attention based severity classification using fused ultrasound and mammography images. J Coast Life Med 10:25\u201345","journal-title":"J Coast Life Med"},{"key":"11268_CR54","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-024-20559-3","author":"C Kerdvibulvech","year":"2025","unstructured":"Kerdvibulvech C (2025) Multimodal ai model for zero-shot vehicle brand identification. Multimed Tools Appl. https:\/\/doi.org\/10.1007\/s11042-024-20559-3","journal-title":"Multimed Tools Appl"},{"key":"11268_CR55","doi-asserted-by":"publisher","DOI":"10.1016\/j.micron.2023.103583","volume":"178","author":"R Khajuria","year":"2023","unstructured":"Khajuria R, Sarwar A (2023) Review of reinforcement learning applications in segmentation, chemotherapy, and radiotherapy of cancer. Micron 178:103583","journal-title":"Micron"},{"issue":"7","key":"11268_CR56","doi-asserted-by":"publisher","first-page":"720","DOI":"10.3390\/biomedicines9070720","volume":"9","author":"M Komatsu","year":"2021","unstructured":"Komatsu M, Sakai A, Dozen A, Shozu K, Yasutomi S, Machino H, Asada K, Kaneko S, Hamamoto R (2021) Towards clinical application of artificial intelligence in ultrasound imaging. Biomedicines 9(7):720","journal-title":"Biomedicines"},{"key":"11268_CR57","unstructured":"K\u00f6rber M, Lange J, Rediske S, Steinmann S, Gl\u00fcck R (2021) Comparing popular simulation environments in the scope of robotics and reinforcement learning. arXiv preprint arXiv:2103.04616"},{"key":"11268_CR58","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2021.631813","volume":"11","author":"M Kuang","year":"2021","unstructured":"Kuang M, Hu H-T, Li W, Chen S-L, Lu X-Z (2021) Articles that use artificial intelligence for ultrasound: a reader\u2019s guide. Front Oncol 11:631813","journal-title":"Front Oncol"},{"key":"11268_CR59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.inffus.2022.03.003","volume":"85","author":"P Ladosz","year":"2022","unstructured":"Ladosz P, Weng L, Kim M, Oh H (2022) Exploration in deep reinforcement learning: a survey. Inform Fusion 85:1\u201322","journal-title":"Inform Fusion"},{"key":"11268_CR60","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-021-10061-9","author":"N Le","year":"2022","unstructured":"Le N, Rathour VS, Yamazaki K, Luu K, Savvides M (2022) Deep reinforcement learning in computer vision: a comprehensive survey. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-021-10061-9","journal-title":"Artif Intell Rev"},{"key":"11268_CR61","unstructured":"Leroy G, Alansary A (2020) Multi-agent deep reinforcement learning for anatomical landmark detection. Github"},{"key":"11268_CR62","doi-asserted-by":"crossref","unstructured":"Leroy G, Rueckert D, Alansary A (2020) Communicative reinforcement learning agents for landmark detection in brain images. In: Machine Learning in clinical neuroimaging and radiogenomics in neuro-oncology: third international workshop, MLCN 2020, and Second International Workshop, RNO-AI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4\u20138, 2020, Proceedings 3. Springer, pp. 177\u2013186","DOI":"10.1007\/978-3-030-66843-3_18"},{"issue":"2","key":"11268_CR63","doi-asserted-by":"publisher","first-page":"510","DOI":"10.1109\/TMRB.2021.3072190","volume":"3","author":"K Li","year":"2021","unstructured":"Li K, Xu Y, Meng MQ-H (2021) An overview of systems and techniques for autonomous robotic ultrasound acquisitions. IEEE Trans Med Robot Bion 3(2):510\u2013524","journal-title":"IEEE Trans Med Robot Bion"},{"issue":"2","key":"11268_CR64","doi-asserted-by":"publisher","first-page":"1526","DOI":"10.1109\/TASE.2023.3246089","volume":"21","author":"K Li","year":"2023","unstructured":"Li K, Li A, Xu Y, Xiong H, Meng MQ-H (2023) Rl-tee: autonomous probe guidance for transesophageal echocardiography based on attention-augmented deep reinforcement learning. IEEE Trans Autom Sci Eng 21(2):1526\u20131538","journal-title":"IEEE Trans Autom Sci Eng"},{"issue":"3","key":"11268_CR65","doi-asserted-by":"publisher","first-page":"580","DOI":"10.3390\/electronics13030580","volume":"13","author":"T Li","year":"2024","unstructured":"Li T, Zeng Q, Li J, Qian C, Yu H, Lu J, Zhang Y, Zhou S (2024) An adaptive control method and learning strategy for ultrasound-guided puncture robot. Electronics 13(3):580","journal-title":"Electronics"},{"key":"11268_CR66","doi-asserted-by":"crossref","unstructured":"Li W, Chen D, Le J (2020) Coronary heart disease prediction based on combined reinforcement multitask progressive networks. In: 2020 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, pp. 311\u2013318","DOI":"10.1109\/BIBM49941.2020.9313275"},{"key":"11268_CR67","doi-asserted-by":"crossref","unstructured":"Li J, Li Y, Guo C, Huang H, Lou H, Guo J (2024) Action-infused reinforcement learning for predicting vascular ultrasound images. In: 2024 IEEE international conference on cybernetics and intelligent systems (CIS) and IEEE international conference on robotics, automation and mechatronics (RAM). IEEE, pp. 457\u2013462","DOI":"10.1109\/CIS-RAM61939.2024.10672633"},{"key":"11268_CR68","doi-asserted-by":"crossref","unstructured":"Lin H, Xie Y, Wang Z, Wang S (2023) A deep reinforcement learning approach for pre-planning problems in robotic intra-operative ultrasound: a virtual environment-based analysis. In: 2023 IEEE 3rd international conference on digital twins and parallel intelligence (DTPI). IEEE, pp. 1\u20134","DOI":"10.1109\/DTPI59677.2023.10365457"},{"key":"11268_CR69","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","volume":"42","author":"G Litjens","year":"2017","unstructured":"Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, S\u00e1nchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60\u201388","journal-title":"Med Image Anal"},{"issue":"2","key":"11268_CR70","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/j.eng.2018.11.020","volume":"5","author":"S Liu","year":"2019","unstructured":"Liu S, Wang Y, Yang X, Lei B, Liu L, Li SX, Ni D, Wang T (2019) Deep learning in medical ultrasound analysis: a review. Engineering 5(2):261\u2013275","journal-title":"Engineering"},{"key":"11268_CR71","doi-asserted-by":"publisher","first-page":"1573","DOI":"10.1109\/TIP.2022.3143699","volume":"31","author":"T Liu","year":"2022","unstructured":"Liu T, Meng Q, Huang J-J, Vlontzos A, Rueckert D, Kainz B (2022) Video summarization through reinforcement learning with a 3d spatio-temporal u-net. IEEE Trans Image Process 31:1573\u20131586","journal-title":"IEEE Trans Image Process"},{"key":"11268_CR72","doi-asserted-by":"crossref","unstructured":"Liu T, Meng Q, Vlontzos A, Tan J, Rueckert D, Kainz B (2020) Ultrasound video summarization using deep reinforcement learning. In: Medical image computing and computer assisted intervention\u2013MICCAI 2020: 23rd international conference, Lima, Peru, October 4\u20138, 2020, Proceedings, Part III 23. Springer, pp. 483\u2013492","DOI":"10.1007\/978-3-030-59716-0_46"},{"key":"11268_CR73","doi-asserted-by":"crossref","unstructured":"Liu S, Ying Q, He S, Yang X, Ni D, Huang R (2023) Hierarchical agent-based reinforcement learning framework for automated quality assessment of fetal ultrasound video. In: 2023 IEEE 20th international symposium on biomedical imaging (ISBI). IEEE, pp. 1\u20135","DOI":"10.1109\/ISBI53787.2023.10230581"},{"key":"11268_CR74","doi-asserted-by":"crossref","unstructured":"Li K, Wang J, Xu Y, Qin H, Liu D, Liu L, Meng MQ-H (2021) Autonomous navigation of an ultrasound probe towards standard scan planes with deep reinforcement learning. In: 2021 IEEE international conference on robotics and automation (ICRA). IEEE, pp. 8302\u20138308","DOI":"10.1109\/ICRA48506.2021.9561295"},{"key":"11268_CR75","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2024.3370728","author":"C Luo","year":"2024","unstructured":"Luo C, Chen Y, Cao H, Al Sibahee MA, Xu W, Zhang J (2024) Multi-modal autonomous ultrasound scanning for efficient human-machine fusion interaction. IEEE Trans Autom Sci Eng. https:\/\/doi.org\/10.1109\/TASE.2024.3370728","journal-title":"IEEE Trans Autom Sci Eng"},{"key":"11268_CR76","unstructured":"Macura M (2024) Dynamic needle tracking in ultrasound using reinforcement learning"},{"issue":"4","key":"11268_CR77","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1016\/j.nic.2020.08.004","volume":"30","author":"F Maleki","year":"2020","unstructured":"Maleki F, Muthukrishnan N, Ovens K, Reinhold C, Forghani R (2020) Machine learning algorithm validation: from essentials to advanced applications and implications for regulatory certification and deployment. Neuroimaging Clin N Am 30(4):433\u2013445","journal-title":"Neuroimaging Clin N Am"},{"key":"11268_CR78","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpa.2021.100185","volume":"10","author":"RP Mathews","year":"2021","unstructured":"Mathews RP, Panicker MR, Hareendranathan AR (2021) vid-samgrah: a pytorch framework for multi-latent space reinforcement learning driven video summarization in ultrasound imaging. Softw Impacts 10:100185","journal-title":"Softw Impacts"},{"issue":"1","key":"11268_CR79","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1109\/JBHI.2022.3208779","volume":"27","author":"RP Mathews","year":"2022","unstructured":"Mathews RP, Panicker MR, Hareendranathan AR, Chen YT, Jaremko JL, Buchanan B, Narayan KV, Kesavadas C, Mathews G (2022) Unsupervised multi-latent space rl framework for video summarization in ultrasound imaging. IEEE J Biomed Health Inform 27(1):227\u2013238","journal-title":"IEEE J Biomed Health Inform"},{"key":"11268_CR80","doi-asserted-by":"crossref","unstructured":"Ma K, Wang J, Singh V, Tamersoy B, Chang Y-J, Wimmer A, Chen T (2017) Multimodal image registration with deep context reinforcement learning. In: Medical image computing and computer assisted intervention- MICCAI 2017: 20th international conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part I 20. Springer, pp. 240\u2013248","DOI":"10.1007\/978-3-319-66182-7_28"},{"key":"11268_CR81","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602"},{"key":"11268_CR82","doi-asserted-by":"publisher","first-page":"124","DOI":"10.3389\/fphy.2020.00124","volume":"8","author":"CM Moran","year":"2020","unstructured":"Moran CM, Thomson AJ (2020) Preclinical ultrasound imaging-a review of techniques and imaging applications. Front Phys 8:124","journal-title":"Front Phys"},{"issue":"3","key":"11268_CR83","doi-asserted-by":"publisher","first-page":"364","DOI":"10.3390\/diagnostics15030364","volume":"15","author":"S Muksimova","year":"2025","unstructured":"Muksimova S, Umirzakova S, Baltayev J, Cho Y-I (2025) Rl-cervix. net: a hybrid lightweight model integrating reinforcement learning for cervical cell classification. Diagnostics 15(3):364","journal-title":"Diagnostics"},{"key":"11268_CR84","doi-asserted-by":"publisher","first-page":"217897","DOI":"10.1109\/ACCESS.2020.3041867","volume":"8","author":"K Munadi","year":"2020","unstructured":"Munadi K, Muchtar K, Maulina N, Pradhan B (2020) Image enhancement for tuberculosis detection using deep learning. IEEE Access 8:217897\u2013217907","journal-title":"IEEE Access"},{"issue":"2","key":"11268_CR85","doi-asserted-by":"publisher","first-page":"1374","DOI":"10.1007\/s11227-022-04709-8","volume":"79","author":"C Narmatha","year":"2023","unstructured":"Narmatha C, Manimegalai P, Krishnadass J, Valsalan P, Manimurugan S, Mustafa M (2023) Ovarian cysts classification using novel deep reinforcement learning with Harris hawks optimization method. J Supercomput 79(2):1374\u20131397","journal-title":"J Supercomput"},{"issue":"12","key":"11268_CR86","doi-asserted-by":"publisher","first-page":"2189","DOI":"10.1007\/s11548-021-02462-6","volume":"16","author":"G Ning","year":"2021","unstructured":"Ning G, Chen J, Zhang X, Liao H (2021) Force-guided autonomous robotic ultrasound scanning control method for soft uncertain environment. Int J Comput Assist Radiol Surg 16(12):2189\u20132199","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"9","key":"11268_CR87","doi-asserted-by":"publisher","first-page":"2787","DOI":"10.1109\/TBME.2021.3054413","volume":"68","author":"G Ning","year":"2021","unstructured":"Ning G, Zhang X, Liao H (2021) Autonomic robotic ultrasound imaging system based on reinforcement learning. IEEE Trans Biomed Eng 68(9):2787\u20132797","journal-title":"IEEE Trans Biomed Eng"},{"issue":"11","key":"11268_CR88","doi-asserted-by":"publisher","first-page":"3166","DOI":"10.1109\/TBME.2023.3279114","volume":"70","author":"G Ning","year":"2023","unstructured":"Ning G, Liang H, Zhang X, Liao H (2023) Autonomous robotic ultrasound vascular imaging system with decoupled control strategy for external-vision-free environments. IEEE Trans Biomed Eng 70(11):3166\u20133177","journal-title":"IEEE Trans Biomed Eng"},{"issue":"2","key":"11268_CR89","doi-asserted-by":"publisher","first-page":"1686","DOI":"10.1109\/TIE.2023.3250767","volume":"71","author":"G Ning","year":"2023","unstructured":"Ning G, Liang H, Zhang X, Liao H (2023) Inverse-reinforcement-learning-based robotic ultrasound active compliance control in uncertain environments. IEEE Trans Ind Electron 71(2):1686\u20131696","journal-title":"IEEE Trans Ind Electron"},{"issue":"6","key":"11268_CR90","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3459991","volume":"54","author":"S Padakandla","year":"2021","unstructured":"Padakandla S (2021) A survey of reinforcement learning algorithms for dynamically varying environments. ACM Comput Surv (CSUR) 54(6):1\u201325","journal-title":"ACM Comput Surv (CSUR)"},{"key":"11268_CR91","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n71","author":"MJ Page","year":"2021","unstructured":"...Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hr\u00f3bjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D (2021) The prisma 2020 statement: an updated guideline for reporting systematic reviews. BMJ. https:\/\/doi.org\/10.1136\/bmj.n71","journal-title":"BMJ"},{"issue":"4","key":"11268_CR92","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3625287","volume":"56","author":"C Patr\u00edcio","year":"2023","unstructured":"Patr\u00edcio C, Neves JC, Teixeira LF (2023) Explainable deep learning methods in medical image classification: a survey. ACM Comput Surv 56(4):1\u201341","journal-title":"ACM Comput Surv"},{"issue":"1","key":"11268_CR93","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1038\/s41746-021-00388-6","volume":"4","author":"A Peine","year":"2021","unstructured":"Peine A, Hallawa A, Bickenbach J, Dartmann G, Fazlic LB, Schmeink A, Ascheid G, Thiemermann C, Schuppert A, Kindle R (2021) Development and validation of a reinforcement learning algorithm to dynamically optimize mechanical ventilation in critical care. NPJ Digit Med 4(1):32","journal-title":"NPJ Digit Med"},{"key":"11268_CR94","doi-asserted-by":"publisher","first-page":"58375","DOI":"10.1109\/ACCESS.2022.3178382","volume":"10","author":"E Petersen","year":"2022","unstructured":"Petersen E, Potdevin Y, Mohammadi E, Zidowitz S, Breyer S, Nowotka D, Henn S, Pechmann L, Leucker M, Rostalski P (2022) Responsible and regulatory conform machine learning for medicine: a survey of challenges and solutions. IEEE Access 10:58375\u201358418","journal-title":"IEEE Access"},{"key":"11268_CR95","doi-asserted-by":"publisher","DOI":"10.1109\/TMRB.2024.3464698","author":"D Raina","year":"2024","unstructured":"Raina D, Balakuntala MV, Kim BW, Wachs J, Voyles R (2024) Coaching a robotic sonographer: learning robotic ultrasound with sparse expert\u2019s feedback. IEEE Trans Med Robot Bion. https:\/\/doi.org\/10.1109\/TMRB.2024.3464698","journal-title":"IEEE Trans Med Robot Bion"},{"key":"11268_CR96","doi-asserted-by":"publisher","DOI":"10.1016\/j.ultras.2023.107012","volume":"132","author":"Q-Y Ran","year":"2023","unstructured":"Ran Q-Y, Miao J, Zhou S-P, Hua S-H, He S-Y, Zhou P, Wang H-X, Zheng Y-P, Zhou G-Q (2023) Automatic 3-d spine curve measurement in freehand ultrasound via structure-aware reinforcement learning spinous process localization. Ultrasonics 132:107012","journal-title":"Ultrasonics"},{"issue":"4","key":"11268_CR97","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1007\/s00607-022-01078-1","volume":"106","author":"G Rjoub","year":"2024","unstructured":"Rjoub G, Wahab OA, Bentahar J, Bataineh AS (2024) Trust-driven reinforcement selection strategy for federated learning on iot devices. Computing 106(4):1273\u20131295","journal-title":"Computing"},{"issue":"3","key":"11268_CR98","doi-asserted-by":"publisher","first-page":"390","DOI":"10.1093\/ehjdh\/ztab042","volume":"2","author":"F Sabov\u010dik","year":"2021","unstructured":"Sabov\u010dik F, Cauwenberghs N, Vens C, Kuznetsova T (2021) Echocardiographic phenogrouping by machine learning for risk stratification in the general population. Eur Heart J-Digit Health 2(3):390\u2013400","journal-title":"Eur Heart J-Digit Health"},{"key":"11268_CR99","first-page":"755","volume-title":"International conference on information processing in medical imaging","author":"SU Saeed","year":"2021","unstructured":"Saeed SU, Fu Y, Baum ZM, Yang Q, Rusu M, Fan RE, Sonn GA, Barratt DC, Hu Y (2021) Learning image quality assessment by reinforcing task amenable data selection. International conference on information processing in medical imaging. Springer, Cham, pp 755\u2013766"},{"key":"11268_CR100","doi-asserted-by":"crossref","unstructured":"Saeed SU, Fu Y, Stavrinides V, Baum ZM, Yang Q, Rusu M, Fan RE, Sonn GA, Noble JA, Barratt DC (2021) Adaptable image quality assessment using meta-reinforcement learning of task amenability. In: Simplifying medical ultrasound: second international workshop, ASMUS 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 2, pp. 191\u2013201. Springer","DOI":"10.1007\/978-3-030-87583-1_19"},{"key":"11268_CR101","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102427","volume":"78","author":"SU Saeed","year":"2022","unstructured":"Saeed SU, Fu Y, Stavrinides V, Baum ZM, Yang Q, Rusu M, Fan RE, Sonn GA, Noble JA, Barratt DC (2022) Image quality assessment for machine learning tasks using meta-reinforcement learning. Med Image Anal 78:102427","journal-title":"Med Image Anal"},{"key":"11268_CR102","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-1180-2","volume-title":"Ultrasound imaging","author":"JM Sanches","year":"2012","unstructured":"Sanches JM, Laine AF, Suri JS (2012) Ultrasound imaging. Springer, Cham"},{"key":"11268_CR103","unstructured":"Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347"},{"key":"11268_CR104","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1007\/s11548-019-02104-y","volume":"15","author":"PK Seitz","year":"2020","unstructured":"Seitz PK, Baumann B, Johnen W, Lissek C, Seidel J, Bendl R (2020) Development of a robot-assisted ultrasound-guided radiation therapy (usgrt). Int J Comput Assist Radiol Surg 15:491\u2013501","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"11268_CR105","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3388293","author":"S Shanmuganathan","year":"2024","unstructured":"Shanmuganathan S, Noga M, Boulanger P, Foster B, Becher H, Punithakumar K (2024) Two-step rigid and non-rigid image registration for the alignment of three-dimensional echocardiography sequences from multiple views. IEEE Access. https:\/\/doi.org\/10.1109\/ACCESS.2024.3388293","journal-title":"IEEE Access"},{"key":"11268_CR106","doi-asserted-by":"crossref","unstructured":"Shen C, Deng Z, Wang J, Wang S, Chen C (2023) Towards autonomous robotic ultrasound scanning using the reinforcement learning-based volumetric data navigation method. In: 2023 WRC symposium on advanced robotics and automation (WRC SARA). IEEE, pp. 335\u2013340","DOI":"10.1109\/WRCSARA60131.2023.10261822"},{"key":"11268_CR107","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-024-03275-z","author":"Y Shida","year":"2024","unstructured":"Shida Y, Kumagai S, Iwata H (2024) Robotic navigation with deep reinforcement learning in transthoracic echocardiography. Int J Comput Assist Radiol Surg. https:\/\/doi.org\/10.1007\/s11548-024-03275-z","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"2","key":"11268_CR108","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1007\/s10462-021-09997-9","volume":"55","author":"B Singh","year":"2022","unstructured":"Singh B, Kumar R, Singh VP (2022) Reinforcement learning in robotic applications: a comprehensive survey. Artif Intell Rev 55(2):945\u2013990","journal-title":"Artif Intell Rev"},{"issue":"5","key":"11268_CR109","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10462-025-11160-7","volume":"58","author":"N Sriraam","year":"2025","unstructured":"Sriraam N, Chinta B, Seshadri S, Suresh S (2025) A comprehensive review of artificial intelligence-based algorithm towards fetal facial anomalies detection (2013\u20132024). Artif Intell Rev 58(5):1\u201349","journal-title":"Artif Intell Rev"},{"key":"11268_CR110","doi-asserted-by":"crossref","unstructured":"Stevens TS, Chennakeshava N, Bruijn FJ, Peka\u0159 M, Sloun RJ (2022) Accelerated intravascular ultrasound imaging using deep reinforcement learning. In: ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp. 1216\u20131220","DOI":"10.1109\/ICASSP43922.2022.9746591"},{"issue":"1","key":"11268_CR111","doi-asserted-by":"publisher","first-page":"4004","DOI":"10.1038\/s41467-024-48421-y","volume":"15","author":"K Su","year":"2024","unstructured":"Su K, Liu J, Ren X, Huo Y, Du G, Zhao W, Wang X, Liang B, Li D, Liu PX (2024) A fully autonomous robotic ultrasound system for thyroid scanning. Nat Commun 15(1):4004","journal-title":"Nat Commun"},{"key":"11268_CR112","unstructured":"Sutton RS (2018) Reinforcement learning: an introduction. A Bradford Book"},{"key":"11268_CR113","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2020.101836","volume":"104","author":"M Tejedor","year":"2020","unstructured":"Tejedor M, Woldaregay AZ, Godtliebsen F (2020) Reinforcement learning application in diabetes blood glucose control: a systematic review. Artif Intell Med 104:101836","journal-title":"Artif Intell Med"},{"issue":"6","key":"11268_CR114","doi-asserted-by":"publisher","first-page":"3693","DOI":"10.3390\/app13063693","volume":"13","author":"R Tenajas","year":"2023","unstructured":"Tenajas R, Miraut D, Illana CI, Alonso-Gonzalez R, Arias-Valcayo F, Herraiz JL (2023) Recent advances in artificial intelligence-assisted ultrasound scanning. Appl Sci 13(6):3693","journal-title":"Appl Sci"},{"key":"11268_CR115","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102875","volume":"89","author":"K Wan","year":"2023","unstructured":"Wan K, Li L, Jia D, Gao S, Qian W, Wu Y, Lin H, Mu X, Gao X, Wang S (2023) Multi-target landmark detection with incomplete images via reinforcement learning and shape prior embedding. Med Image Anal 89:102875","journal-title":"Med Image Anal"},{"key":"11268_CR116","doi-asserted-by":"publisher","first-page":"54310","DOI":"10.1109\/ACCESS.2021.3071301","volume":"9","author":"Y Wang","year":"2021","unstructured":"Wang Y, Ge X, Ma H, Qi S, Zhang G, Yao Y (2021) Deep learning in medical ultrasound image analysis: a review. IEEE Access 9:54310\u201354324","journal-title":"IEEE Access"},{"issue":"4","key":"11268_CR117","doi-asserted-by":"publisher","first-page":"5064","DOI":"10.1109\/TNNLS.2022.3207346","volume":"35","author":"X Wang","year":"2022","unstructured":"Wang X, Wang S, Liang X, Zhao D, Huang J, Xu X, Dai B, Miao Q (2022) Deep reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 35(4):5064\u20135078","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"11268_CR118","doi-asserted-by":"crossref","unstructured":"Wang J, Miao J, Yang X, Li R, Zhou G, Huang Y, Lin Z, Xue W, Jia X, Zhou J (2020) Auto-weighting for breast cancer classification in multimodal ultrasound. In: Medical image computing and computer assisted intervention\u2013MICCAI 2020: 23rd International Conference, Lima, Peru, October 4\u20138, 2020, Proceedings, Part VI 23. Springer, pp. 190\u2013199","DOI":"10.1007\/978-3-030-59725-2_19"},{"issue":"6","key":"11268_CR119","doi-asserted-by":"publisher","first-page":"5023","DOI":"10.1007\/s10462-022-10299-x","volume":"56","author":"A Wong","year":"2023","unstructured":"Wong A, B\u00e4ck T, Kononova AV, Plaat A (2023) Deep multiagent reinforcement learning: challenges and directions. Artif Intell Rev 56(6):5023\u20135056","journal-title":"Artif Intell Rev"},{"key":"11268_CR120","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102793","volume":"89","author":"R Wright","year":"2023","unstructured":"Wright R, Gomez A, Zimmer VA, Toussaint N, Khanal B, Matthew J, Skelton E, Kainz B, Rueckert D, Hajnal JV (2023) Fast fetal head compounding from multi-view 3d ultrasound. Med Image Anal 89:102793","journal-title":"Med Image Anal"},{"key":"11268_CR121","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.124377","volume":"254","author":"J Wu","year":"2024","unstructured":"Wu J, Zhang H, Liu X, Lu M, Gao Z (2024) Segmentation-assisted hierarchical constrained state space approach for robust carotid artery wall motion measurement. Expert Syst Appl 254:124377","journal-title":"Expert Syst Appl"},{"key":"11268_CR122","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2023.3251323","author":"X Xie","year":"2023","unstructured":"Xie X, Tian Y, Ota K, Dong M, Liu Z, Jin H, Yao D (2023) Reinforced computer-aided framework for diagnosing thyroid cancer. IEEE\/ACM Trans Comput Biol Bioinform. https:\/\/doi.org\/10.1109\/TCBB.2023.3251323","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"issue":"22","key":"11268_CR123","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1088\/1361-6560\/ac9cb3","volume":"67","author":"L Xu","year":"2022","unstructured":"Xu L, Zhu S, Wen N (2022) Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey. Phys Med Biol 67(22):22","journal-title":"Phys Med Biol"},{"issue":"7","key":"11268_CR124","doi-asserted-by":"publisher","first-page":"1950","DOI":"10.1109\/TMI.2021.3069663","volume":"40","author":"X Yang","year":"2021","unstructured":"Yang X, Dou H, Huang R, Xue W, Huang Y, Qian J, Zhang Y, Luo H, Guo H, Wang T (2021a) Agent with warm start and adaptive dynamic termination for plane localization in 3d ultrasound. IEEE Trans Med Imaging 40(7):1950\u20131961","journal-title":"IEEE Trans Med Imaging"},{"key":"11268_CR125","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102119","volume":"72","author":"X Yang","year":"2021","unstructured":"Yang X, Huang Y, Huang R, Dou H, Li R, Qian J, Huang X, Shi W, Chen C, Zhang Y (2021b) Searching collaborative agents for multi-plane localization in 3d ultrasound. Med Image Anal 72:102119","journal-title":"Med Image Anal"},{"key":"11268_CR127","doi-asserted-by":"publisher","first-page":"858","DOI":"10.1109\/JBHI.2023.3336726","volume":"28","author":"F Yang","year":"2023","unstructured":"Yang F, Li X, Duan H, Xu F, Huang Y, Zhang X, Long Y, Zheng Y (2023) Mrl-seg: overcoming imbalance in medical image segmentation with multi-step reinforcement learning. IEEE J Biomed Health Inform 28:858","journal-title":"IEEE J Biomed Health Inform"},{"key":"11268_CR128","doi-asserted-by":"crossref","unstructured":"Yang H, Shan C, Kolen AF, With PH (2020) Deep q-network-driven catheter segmentation in 3d us by hybrid constrained semi-supervised learning and dual-unet. In: Medical image computing and computer assisted intervention\u2013MICCAI 2020: 23rd International Conference, Lima, Peru, October 4\u20138, 2020, Proceedings, Part I 23. Springer, pp. 646\u2013655","DOI":"10.1007\/978-3-030-59710-8_63"},{"key":"11268_CR129","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2024.3371375","author":"L Yao","year":"2024","unstructured":"Yao L, Zhao B, Wang Q, Wang Z, Zhang P, Qi X, Wong PK, Hu Y (2024) A decision-making algorithm for robotic breast ultrasound high-quality imaging via broad reinforcement learning from demonstration. IEEE Robot Autom Lett. https:\/\/doi.org\/10.1109\/LRA.2024.3371375","journal-title":"IEEE Robot Autom Lett"},{"key":"11268_CR130","first-page":"621","volume":"33","author":"D Ye","year":"2020","unstructured":"Ye D, Chen G, Zhang W, Chen S, Yuan B, Liu B, Chen J, Liu Z, Qiu F, Yu H (2020) Towards playing full moba games with deep reinforcement learning. Adv Neural Inf Process Syst 33:621\u2013632","journal-title":"Adv Neural Inf Process Syst"},{"issue":"1","key":"11268_CR131","doi-asserted-by":"publisher","first-page":"7","DOI":"10.14366\/usg.20102","volume":"40","author":"J Yi","year":"2020","unstructured":"Yi J, Kang HK, Kwon J-H, Kim K-S, Park MH, Seong YK, Kim DW, Ahn B, Ha K, Lee J (2020) Technology trends and applications of deep learning in ultrasonography: image quality enhancement, diagnostic support, and improving workflow efficiency. Ultrasonography 40(1):7","journal-title":"Ultrasonography"},{"key":"11268_CR132","doi-asserted-by":"crossref","unstructured":"Yi W, Stavrinides V, Baum ZM, Yang Q, Barratt DC, Clarkson MJ, Hu Y, Saeed SU (2023) Boundary-rl: Reinforcement learning for weakly-supervised prostate segmentation in trus images. In: International workshop on machine learning in medical imaging. Springer, pp. 277\u2013288","DOI":"10.1007\/978-3-031-45673-2_28"},{"key":"11268_CR133","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2024.3451138","author":"E Zakeri","year":"2024","unstructured":"Zakeri E, Spilkin A, Elmekki H, Zanuttini A, Kadem L, Bentahar J, Xie W-F, Pibarot P (2024) Ai-powered robust interaction force control of a cardiac ultrasound robotic system. IEEE Trans Ind Electron. https:\/\/doi.org\/10.1109\/TIE.2024.3451138","journal-title":"IEEE Trans Ind Electron"},{"key":"11268_CR134","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2025.3531925","author":"E Zakeri","year":"2025","unstructured":"Zakeri E, Spilkin A, Elmekki H, Zanuttini A, Kadem L, Bentahar J, Xie W-F, Pibarot P (2025) Robust deep feature ultrasound image-based visual servoing: focus on cardiac examination. IEEE\/ASME Trans Mechatron. https:\/\/doi.org\/10.1109\/TMECH.2025.3531925","journal-title":"IEEE\/ASME Trans Mechatron"},{"key":"11268_CR135","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11547-017-0805-z","volume":"123","author":"M Zanotel","year":"2018","unstructured":"Zanotel M, Bednarova I, Londero V, Linda A, Lorenzon M, Girometti R, Zuiani C (2018) Automated breast ultrasound: basic principles and emerging clinical applications. Radiol Med (Torino) 123:1\u201312","journal-title":"Radiol Med (Torino)"},{"key":"11268_CR136","doi-asserted-by":"publisher","first-page":"306","DOI":"10.29007\/12lv","volume":"4","author":"X Zeng","year":"2020","unstructured":"Zeng X, Vives M, Hacihaliloglu I (2020) Hierarchical 3-d registration of computed tomography to ultrasound using reinforcement learning. EPiC Series Health Sci 4:306\u2013311","journal-title":"EPiC Series Health Sci"},{"key":"11268_CR137","doi-asserted-by":"publisher","first-page":"135426","DOI":"10.1109\/ACCESS.2020.3011438","volume":"8","author":"F Zeng","year":"2020","unstructured":"Zeng F, Wang C, Ge SS (2020) A survey on visual navigation for artificial agents with deep reinforcement learning. IEEE Access 8:135426\u2013135442","journal-title":"IEEE Access"},{"issue":"1s","key":"11268_CR138","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3532625","volume":"19","author":"X Zhang","year":"2023","unstructured":"Zhang X, Gao W, Li G, Jiang Q, Cong R (2023) Image quality assessment-driven reinforcement learning for mixed distorted image restoration. ACM Trans Multimed Comput Commun Appl 19(1s):1\u201323","journal-title":"ACM Trans Multimed Comput Commun Appl"},{"key":"11268_CR139","doi-asserted-by":"crossref","unstructured":"Zhang Y, Amadou A-a, Voigt I, Mihalef V, Houle H, John M, Mansi T, Liao R (2020) A bottom-up approach for real-time mitral valve annulus modeling on 3d echo images. In: Medical image computing and computer assisted intervention\u2013MICCAI 2020: 23rd International Conference, Lima, Peru, October 4\u20138, 2020, Proceedings, Part VI 23. Springer, pp. 458\u2013467","DOI":"10.1007\/978-3-030-59725-2_44"},{"key":"11268_CR140","doi-asserted-by":"crossref","unstructured":"Zhang Q, Liu X, Wang Y (2023) Detection of benign and malignant thyroid nodules: an reinforcement region selection network framework. In: 2023 5th International conference on machine learning, big data and business intelligence (MLBDBI). IEEE, pp. 1\u20139","DOI":"10.1109\/MLBDBI60823.2023.10481678"},{"key":"11268_CR141","doi-asserted-by":"publisher","DOI":"10.1016\/j.compmedimag.2023.102275","volume":"108","author":"T Zheng","year":"2023","unstructured":"Zheng T, Chen W, Li S, Quan H, Zou M, Zheng S, Zhao Y, Gao X, Cui X (2023) Learning how to detect: a deep reinforcement learning method for whole-slide melanoma histopathology images. Comput Med Imaging Graph 108:102275","journal-title":"Comput Med Imaging Graph"},{"key":"11268_CR142","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102193","volume":"73","author":"SK Zhou","year":"2021","unstructured":"Zhou SK, Le HN, Luu K, Nguyen HV, Ayache N (2021) Deep reinforcement learning in medical imaging: a literature review. Med Image Anal 73:102193","journal-title":"Med Image Anal"},{"issue":"10","key":"11268_CR143","doi-asserted-by":"publisher","first-page":"3854","DOI":"10.1109\/JBHI.2021.3080703","volume":"25","author":"G-Q Zhou","year":"2021","unstructured":"Zhou G-Q, Miao J, Yang X, Li R, Huo E-Z, Shi W, Huang Y, Qian J, Chen C, Ni D (2021) Learn fine-grained adaptive loss for multiple anatomical landmark detection in medical images. IEEE J Biomed Health Inform 25(10):3854\u20133864","journal-title":"IEEE J Biomed Health Inform"},{"key":"11268_CR144","first-page":"300","volume-title":"International conference on medical image computing and computer-assisted intervention","author":"Y Zou","year":"2022","unstructured":"Zou Y, Dou H, Huang Y, Yang X, Qian J, Zhen C, Ji X, Ravikumar N, Chen G, Huang W (2022) Agent with tangent-based formulation and anatomical perception for standard plane localization in 3d ultrasound. International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp 300\u2013309"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-025-11268-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-025-11268-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-025-11268-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,6]],"date-time":"2025-09-06T21:34:13Z","timestamp":1757194453000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-025-11268-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,23]]},"references-count":143,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["11268"],"URL":"https:\/\/doi.org\/10.1007\/s10462-025-11268-w","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-6181857\/v1","asserted-by":"object"}]},"ISSN":["1573-7462"],"issn-type":[{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,23]]},"assertion":[{"value":"12 May 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 June 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"284"}}