{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T13:24:48Z","timestamp":1740144288385,"version":"3.37.3"},"reference-count":16,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/S021930\/1"],"award-info":[{"award-number":["EP\/S021930\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000289","name":"Cancer Research UK","doi-asserted-by":"publisher","award":["C28070\/A30912","C28070\/A30912"],"award-info":[{"award-number":["C28070\/A30912","C28070\/A30912"]}],"id":[{"id":"10.13039\/501100000289","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000289","name":"Cancer Research UK","doi-asserted-by":"publisher","award":["C28070\/A30912","C73666\/A31378"],"award-info":[{"award-number":["C28070\/A30912","C73666\/A31378"]}],"id":[{"id":"10.13039\/501100000289","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000289","name":"Cancer Research UK","doi-asserted-by":"publisher","award":["C73666\/A31378","C73666\/A31378","EP\/T029404\/1","EP\/T029404\/1"],"award-info":[{"award-number":["C73666\/A31378","C73666\/A31378","EP\/T029404\/1","EP\/T029404\/1"]}],"id":[{"id":"10.13039\/501100000289","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100020194","name":"Wellcome \/ EPSRC Centre for Interventional and Surgical Sciences","doi-asserted-by":"publisher","award":["203145Z\/16\/Z","203145Z\/16\/Z","203145Z\/16\/Z"],"award-info":[{"award-number":["203145Z\/16\/Z","203145Z\/16\/Z","203145Z\/16\/Z"]}],"id":[{"id":"10.13039\/501100020194","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Magnetic resonance (MR) imaging targeted prostate cancer (PCa) biopsy enables precise sampling of MR-detected lesions, establishing its importance in recommended clinical practice. Planning for the ultrasound-guided procedure involves pre-selecting needle sampling positions. However, performing this procedure is subject to a number of factors, including MR-to-ultrasound registration, intra-procedure patient movement and soft tissue motions. When a fixed <jats:italic>pre-procedure planning<\/jats:italic> is carried out without intra-procedure adaptation, these factors will lead to sampling errors which could cause false positives and false negatives. Reinforcement learning (RL) has been proposed for procedure plannings on similar applications such as this one, because intelligent agents can be trained for both pre-procedure and <jats:italic>intra-procedure planning<\/jats:italic>. However, it is not clear if RL is beneficial when it comes to addressing these intra-procedure errors.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>In this work, we develop and compare imitation learning (IL), supervised by demonstrations of predefined sampling strategy, and RL approaches, under varying degrees of intra-procedure motion and registration error, to represent sources of targeting errors likely to occur in an intra-operative procedure.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Based on results using imaging data from 567 PCa patients, we demonstrate the efficacy and value in adopting RL algorithms to provide intelligent intra-procedure action suggestions, compared to IL-based planning supervised by commonly adopted policies.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The improvement in biopsy sampling performance for intra-procedure planning has not been observed in experiments with only pre-procedure planning. These findings suggest a strong role for RL in future prospective studies which adopt intra-procedure planning. Our open source code implementation is available <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/i-gayo\/ImitationLearning\">here<\/jats:ext-link>.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-024-03084-4","type":"journal-article","created":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T06:01:52Z","timestamp":1709791312000},"page":"1003-1012","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["The distinct roles of reinforcement learning between pre-procedure and intra-procedure planning for prostate biopsy"],"prefix":"10.1007","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4622-2373","authenticated-orcid":false,"given":"Iani J. M. B.","family":"Gayo","sequence":"first","affiliation":[]},{"given":"Shaheer U.","family":"Saeed","sequence":"additional","affiliation":[]},{"given":"Ester","family":"Bonmati","sequence":"additional","affiliation":[]},{"given":"Dean C.","family":"Barratt","sequence":"additional","affiliation":[]},{"given":"Matthew J.","family":"Clarkson","sequence":"additional","affiliation":[]},{"given":"Yipeng","family":"Hu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,3,7]]},"reference":[{"key":"3084_CR1","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1016\/s0140-6736(16)32401-1","volume":"389","author":"HU Ahmed","year":"2017","unstructured":"Ahmed HU, El-Shater Bosaily A, Brown LC, Gabe R, Kaplan R, Parmar MK, Collaco-Moraes Y, Ward K, Hindley RG, Freeman A, Kirkham AP, Oldroyd R, Parker C, Emberton M (2017) Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. The Lancet 389:815\u2013822. https:\/\/doi.org\/10.1016\/s0140-6736(16)32401-1","journal-title":"The Lancet"},{"key":"3084_CR2","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1016\/j.juro.2016.08.097","volume":"197","author":"A Muthigi","year":"2017","unstructured":"Muthigi A, George AK, Sidana A, Kongnyuy M, Simon R, Moreno V, Merino MJ, Choyke PL, Turkbey B, Wood BJ, Pinto PA (2017) Missing the mark: prostate cancer upgrading by systematic biopsy over magnetic resonance imaging\/transrectal ultrasound fusion biopsy. J Urol 197:327\u2013334. https:\/\/doi.org\/10.1016\/j.juro.2016.08.097","journal-title":"J Urol"},{"key":"3084_CR3","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/j.eururo.2015.03.041","volume":"69","author":"E Baco","year":"2016","unstructured":"Baco E, Rud E, Eri LM, Moen G, Vlatkovic L, Svindland A, Eggesb\u00f8 HB, Ukimura O (2016) A randomized controlled trial to assess and compare the outcomes of two-core prostate biopsy guided by fused magnetic resonance and transrectal ultrasound images and traditional 12-core systematic biopsy. Eur Urol 69:149\u2013156. https:\/\/doi.org\/10.1016\/j.eururo.2015.03.041","journal-title":"Eur Urol"},{"key":"3084_CR4","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1016\/j.eururo.2019.09.021","volume":"77","author":"RRM Engels","year":"2020","unstructured":"Engels RRM, Isra\u00ebl B, Padhani AR, Barentsz JO (2020) Multiparametric magnetic resonance imaging for the detection of clinically significant prostate cancer: What urologists need to know. Part 1: Acquisition. Eur Urol 77:457\u2013468. https:\/\/doi.org\/10.1016\/j.eururo.2019.09.021","journal-title":"Eur Urol"},{"key":"3084_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.juro.2017.02.1160","author":"C Orczyk","year":"2017","unstructured":"Orczyk C, Hu YP, Gibson E, El-Shater Bosaily A, Kirkham A, Punwani S, Brown L, Bonmati E, Coraco-Moraes Y, Ward K, Kaplan R, Barratt D, Emberton M, Ahmed HU (2017) Mp38-07 should we aim for the centre of an MRI prostate lesion? Correlation between mpMRI and 3-dimensional 5mm transperineal prostate mapping biopsies from the PROMIS trial. J Urol. https:\/\/doi.org\/10.1016\/j.juro.2017.02.1160","journal-title":"J Urol"},{"key":"3084_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-17979-2_15","author":"IJ Gayo","year":"2022","unstructured":"Gayo IJ, Saeed SU, Barratt DC, Clarkson MJ, Hu Y (2022) Strategising template-guided needle placement for MR-targeted prostate biopsy. Cancer Prev Through Early Detect. https:\/\/doi.org\/10.1007\/978-3-031-17979-2_15","journal-title":"Cancer Prev Through Early Detect"},{"key":"3084_CR7","unstructured":"Chaitanya K, Audigier C, Balascuta LE, Mansi T (2022) Automatic planning of liver tumor thermal ablation using deep reinforcement learning. PMLR. https:\/\/proceedings.mlr.press\/v172\/chaitanya22a.html"},{"key":"3084_CR8","doi-asserted-by":"publisher","unstructured":"Lee Y, Tan X, Chng C-B, Chui C-K (2019) Simulation of robot-assisted flexible needle insertion using deep q-network. In: 2019 IEEE international conference on systems, man and cybernetics (SMC), pp 342\u2013346. https:\/\/doi.org\/10.1109\/SMC.2019.8914191","DOI":"10.1109\/SMC.2019.8914191"},{"key":"3084_CR9","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11757","author":"T Hester","year":"2018","unstructured":"Hester T, Vecerik M, Pietquin O, Lanctot M, Schaul T, Piot B, Horgan D, Quan J, Sendonaris A, Osband I, Dulac-Arnold G, Agapiou J, Leibo J, Gruslys A (2018) Deep q-learning from demonstrations. Proc AAAI Conf Artif Intell. https:\/\/doi.org\/10.1609\/aaai.v32i1.11757","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"3084_CR10","doi-asserted-by":"publisher","first-page":"1459","DOI":"10.1109\/TMI.2019.2950947","volume":"39","author":"L Liang","year":"2020","unstructured":"Liang L, Cool D, Kakani N, Wang G, Ding H, Fenster A (2020) Automatic radiofrequency ablation planning for liver tumors with multiple constraints based on set covering. IEEE Trans Med Imaging 39:1459\u20131471. https:\/\/doi.org\/10.1109\/TMI.2019.2950947","journal-title":"IEEE Trans Med Imaging"},{"key":"3084_CR11","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1016\/j.eururo.2018.08.007","volume":"75","author":"S Hamid","year":"2019","unstructured":"Hamid S, Donaldson IA, Hu Y, Rodell R, Villarini B, Bonmati E, Tranter P, Punwani S, Sidhu HS, Willis S, Meulen J, Hawkes D, McCartan N, Potyka I, Williams NR, Brew-Graves C, Freeman A, Moore CM, Barratt D, Emberton M, Ahmed HU (2019) The smarttarget biopsy trial: a prospective, within-person randomised, blinded trial comparing the accuracy of visual-registration and magnetic resonance imaging\/ultrasound image-fusion targeted biopsies for prostate cancer risk stratification. Eur Urol 75:733\u2013740. https:\/\/doi.org\/10.1016\/j.eururo.2018.08.007","journal-title":"Eur Urol"},{"key":"3084_CR12","doi-asserted-by":"publisher","unstructured":"Simmons LAM, Kanthabalan A, Arya M, Briggs T, Barratt D, Charman SC, Freeman A, Hawkes D, Hu Y, Jameson C, McCartan N, Moore CM, Punwani S, Muelen J, Emberton M, Ahmed HU (2018) Accuracy of transperineal targeted prostate biopsies, visual estimation and image fusion in men needing repeat biopsy in the picture trial. J Urol 200:1227\u20131234. https:\/\/doi.org\/10.1016\/j.juro.2018.07.001","DOI":"10.1016\/j.juro.2018.07.001"},{"key":"3084_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.103030","volume":"91","author":"W Yan","year":"2024","unstructured":"Yan W, Chiu B, Shen Z, Yang Q, Syer T, Min Z, Punwani S, Emberton M, Atkinson D, Barratt DC, Hu Y (2024) Combiner and hypercombiner networks: Rules to combine multimodality MR images for prostate cancer localisation. Medical Image Anal. 91:103030. https:\/\/doi.org\/10.1016\/j.media.2023.103030","journal-title":"Medical Image Anal."},{"key":"3084_CR14","unstructured":"Schulman, J., Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv:1707.06347"},{"key":"3084_CR15","doi-asserted-by":"publisher","unstructured":"Hu Y, Modat M, Gibson E, Li W, Ghavami N, Bonmati E, Wang G, Bandula S, Moore CM, Emberton M, Ourselin S, Noble JA, Barratt DC, Vercauteren T (2018) Weakly-supervised convolutional neural networks for multimodal image registration. Med Image Anal 49:1\u201313. https:\/\/doi.org\/10.1016\/j.media.2018.07.002","DOI":"10.1016\/j.media.2018.07.002"},{"key":"3084_CR16","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/s0262-8856(00)00057-3","volume":"19","author":"M Fornefett","year":"2001","unstructured":"Fornefett M, Rohr K, Stiehl HS (2001) Radial basis functions with compact support for elastic registration of medical images. Image Vis Comput 19:87\u201396. https:\/\/doi.org\/10.1016\/s0262-8856(00)00057-3","journal-title":"Image Vis Comput"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03084-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-024-03084-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03084-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T11:22:39Z","timestamp":1718364159000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-024-03084-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,7]]},"references-count":16,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["3084"],"URL":"https:\/\/doi.org\/10.1007\/s11548-024-03084-4","relation":{},"ISSN":["1861-6429"],"issn-type":[{"type":"electronic","value":"1861-6429"}],"subject":[],"published":{"date-parts":[[2024,3,7]]},"assertion":[{"value":"22 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors have no conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and\/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Informed consent was obtained from all individual participants included in the study","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}}]}}