{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T21:49:54Z","timestamp":1790286594255,"version":"4.1.0"},"reference-count":24,"publisher":"IOP Publishing","issue":"22","license":[{"start":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T00:00:00Z","timestamp":1762473600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T00:00:00Z","timestamp":1762473600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100000905","name":"Commonwealth Fund","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000905","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Phys. Med. Biol."],"published-print":{"date-parts":[[2025,11,16]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    <jats:italic>Objective.<\/jats:italic>\n                    To develop and evaluate a deep reinforcement learning (RL) framework for rapid and automatic machine parameter optimization of volumetric modulated arc therapy (VMAT) treatment plans for localized prostate cancer.\n                    <jats:italic>Approach.<\/jats:italic>\n                    A multi-task policy network combining convolution and long short-term memory was trained to sequentially predict the set of actions on the dose rate and multi-leaf collimator positions over the range of two arcs. The network uses as input the cumulative dose grid at the current gantry angle, contours of the planning target volume (PTV) and organs at risk, and the set of machine parameters at all preceding gantry angles. The method was evaluated on a set of 15 localized prostate cancer patients for a prescription dose of 60 Gy in 20 fractions. For each case, the final state dose distribution was compared against clinical plans. For seamless integration with the clinical workflow, the proposed model was integrated into a clinical treatment planning system (TPS), enabling dosimetric review and final plan adjustments.\n                    <jats:italic>Main results.<\/jats:italic>\n                    The RL framework produced deliverable dual-arc VMAT plans in an average of 20.7 \u00b1 5.0 s over the test set. Dosimetric comparison to clinical plans showed no statistically significant differences for the mean rectum dose as well as for the bladder V6160 Gy, indicating that the RL model was as efficient in sparing these structures as human planners. While the approach showed limitations in terms of PTV coverage and maximum body dose, our proposed integration to TPS showed the RL plans could be automatically refined to clinical quality in an additional 83.8 \u00b1 7.2 s.\n                    <jats:italic>Significance.<\/jats:italic>\n                    The accuracy and fast run time of the approach show the potential of the framework to significantly streamline VMAT treatment planning and enable adaptive radiation therapy.\n                  <\/jats:p>","DOI":"10.1088\/1361-6560\/ae1803","type":"journal-article","created":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T22:51:15Z","timestamp":1761605475000},"page":"225007","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Dual-arc VMAT machine parameter optimization for localized prostate cancer using deep reinforcement learning"],"prefix":"10.1088","volume":"70","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7670-6702","authenticated-orcid":true,"given":"Lina","family":"Mekki","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William T","family":"Hrinivich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3004-8625","authenticated-orcid":true,"given":"Junghoon","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2025,11,7]]},"reference":[{"key":"pmbae1803bib1","doi-asserted-by":"publisher","DOI":"10.1002\/acm2.13783","type":"journal-article","article-title":"Dosimetric and feasibility evaluation of a CBCT\u2010based daily adaptive radiotherapy protocol for locally advanced cervical cancer","volume":"24","author":"Branco","year":"2023","journal-title":"J. Appl. Clin. Med. Phys."},{"key":"pmbae1803bib2","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1120\/jacmp.v14i3.4053","type":"journal-article","article-title":"Prostate volumetric\u2010modulated arc therapy: dosimetry and radiobiological model variation between the single\u2010arc and double\u2010arc technique","volume":"14","author":"Chow","year":"2013","journal-title":"J. Appl. Clin. Med. Phys."},{"key":"pmbae1803bib3","doi-asserted-by":"publisher","first-page":"3877","DOI":"10.1118\/1.4921615","type":"journal-article","article-title":"Online adaptation and verification of VMAT","volume":"42","author":"Crijns","year":"2015","journal-title":"Med. Phys."},{"key":"pmbae1803bib4","article-title":"Multi-agent reinforcement learning meets leaf sequencing in radiotherapy","author":"Gao","year":"2024","type":"preprint"},{"key":"pmbae1803bib5","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","type":"journal-article","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"pmbae1803bib6","doi-asserted-by":"publisher","first-page":"3972","DOI":"10.1002\/mp.17100","type":"journal-article","article-title":"Clinical VMAT machine parameter optimization for localized prostate cancer using deep reinforcement learning","volume":"51","author":"Hrinivich","year":"2024","journal-title":"Med. Phys."},{"key":"pmbae1803bib7","doi-asserted-by":"publisher","first-page":"6140","DOI":"10.1002\/mp.14544","type":"journal-article","article-title":"Artificial intelligence\u2010based radiotherapy machine parameter optimization using reinforcement learning","volume":"47","author":"Hrinivich","year":"2020","journal-title":"Med. Phys."},{"key":"pmbae1803bib8","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ac2bb5","type":"journal-article","article-title":"Graph neural networks and deep reinforcement learning for simultaneous beam orientation and trajectory optimization of Cyberknife","volume":"66","author":"Kafaei","year":"2021","journal-title":"Phys. Med. Biol."},{"key":"pmbae1803bib9","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejmp.2024.104498","type":"journal-article","article-title":"Deep reinforcement learning in radiation therapy planning optimization: a comprehensive review","volume":"125","author":"Li","year":"2024","journal-title":"Phys. Med."},{"key":"pmbae1803bib10","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1016\/j.ijrobp.2009.10.075","type":"journal-article","article-title":"Consensus guidelines for delineation of clinical target volume for intensity-modulated pelvic radiotherapy for the definitive treatment of cervix cancer","volume":"79","author":"Lim","year":"2011","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbae1803bib11","doi-asserted-by":"publisher","first-page":"582","DOI":"10.1117\/12.3046982","type":"journal-article","article-title":"Deep reinforcement learning-based radiotherapy machine parameter optimization with a multitask policy network","volume":"13408","author":"Mekki","year":"2025","journal-title":"Proc. SPIE"},{"key":"pmbae1803bib12","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","type":"journal-article","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"pmbae1803bib13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ejmp.2021.12.009","type":"journal-article","article-title":"Deep reinforcement learning for treatment planning in high-dose-rate cervical brachytherapy","volume":"94","author":"Pu","year":"2022","journal-title":"Phys. Med."},{"key":"pmbae1803bib14","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2024.101550","type":"journal-article","article-title":"Using scorecards to tune ethos directive templates: an adaptive radiation therapy individualized approach-cervix dosimetric planning study","volume":"9","author":"Rayn","year":"2024","journal-title":"Adv. Radiat. Oncol."},{"key":"pmbae1803bib15","doi-asserted-by":"publisher","first-page":"1909","DOI":"10.1002\/mp.14712","type":"journal-article","article-title":"Improving efficiency of training a virtual treatment planner network via knowledge\u2010guided deep reinforcement learning for intelligent automatic treatment planning of radiotherapy","volume":"48","author":"Shen","year":"2021a","journal-title":"Med. Phys."},{"key":"pmbae1803bib16","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ac09a2","type":"journal-article","article-title":"A hierarchical deep reinforcement learning framework for intelligent automatic treatment planning of prostate cancer intensity modulated radiation therapy","volume":"66","author":"Shen","year":"2021b","journal-title":"Phys. Med. Biol."},{"key":"pmbae1803bib17","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ab18bf","type":"journal-article","article-title":"Intelligent inverse treatment planning via deep reinforcement learning, a proof-of-principle study in high dose-rate brachytherapy for cervical cancer","volume":"64","author":"Shen","year":"2019","journal-title":"Phys. Med. Biol."},{"key":"pmbae1803bib18","doi-asserted-by":"publisher","first-page":"2329","DOI":"10.1002\/mp.14114","type":"journal-article","article-title":"Operating a treatment planning system using a deep\u2010reinforcement learning\u2010based virtual treatment planner for prostate cancer intensity\u2010modulated radiation therapy treatment planning","volume":"47","author":"Shen","year":"2020","journal-title":"Med. Phys."},{"key":"pmbae1803bib19","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","type":"journal-article","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"pmbae1803bib20","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.meddos.2011.01.005","type":"journal-article","article-title":"RapidArc radiotherapy planning for prostate cancer: single-arc and double-arc techniques vs. intensity-modulated radiotherapy","volume":"37","author":"Sze","year":"2012","journal-title":"Med. Dosim."},{"key":"pmbae1803bib21","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","type":"journal-article","article-title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"pmbae1803bib22","doi-asserted-by":"publisher","first-page":"1997","DOI":"10.1002\/mp.17654","type":"journal-article","article-title":"Automating the optimization of proton PBS treatment planning for head and neck cancers using policy gradient\u2010based deep reinforcement learning","volume":"52","author":"Wang","year":"2025","journal-title":"Med. Phys."},{"key":"pmbae1803bib23","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1016\/j.radonc.2009.08.011","type":"journal-article","article-title":"Volumetric modulated arc therapy (VMAT) vs. serial tomotherapy, step-and-shoot IMRT and 3D-conformal RT for treatment of prostate cancer","volume":"93","author":"Wolff","year":"2009","journal-title":"Radiother. Oncol."},{"key":"pmbae1803bib24","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1002\/acm2.13425","type":"journal-article","article-title":"Initial analysis of the dosimetric benefit and clinical resource cost of CBCT\u2010based online adaptive radiotherapy for patients with cancers of the cervix or rectum","volume":"22","author":"Yock","year":"2021","journal-title":"J. Appl. Clin. Med. Phys."}],"container-title":["Physics in Medicine &amp; Biology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T09:53:35Z","timestamp":1762509215000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae1803"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,7]]},"references-count":24,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2025,11,7]]},"published-print":{"date-parts":[[2025,11,16]]}},"URL":"https:\/\/doi.org\/10.1088\/1361-6560\/ae1803","relation":{},"ISSN":["0031-9155","1361-6560"],"issn-type":[{"value":"0031-9155","type":"print"},{"value":"1361-6560","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,7]]},"assertion":[{"value":"Dual-arc VMAT machine parameter optimization for localized prostate cancer using deep reinforcement learning","name":"article_title","label":"Article Title"},{"value":"Physics in Medicine & Biology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published on behalf of Institute of Physics and Engineering in Medicine by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2025-05-20","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-10-27","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-11-07","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}