{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T00:28:50Z","timestamp":1773102530047,"version":"3.50.1"},"reference-count":40,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T00:00:00Z","timestamp":1769731200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T00:00:00Z","timestamp":1769731200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["aapm.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Medical Physics"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Volumetric modulated arc therapy (VMAT) machine parameter optimization (MPO) is a complex, high\u2010dimensional problem typically solved with inverse planning solutions that are both temporally and computationally expensive. While machine learning techniques have been explored to automate this process, they often supplement rather than replace conventional optimizers and are fundamentally limited by the quality and diversity of training data. Reinforcement learning (RL) offers a promising alternative, finding optimal strategies through trial\u2010and\u2010error by maximizing a narrowly tailored reward function, which can potentially discover novel solutions beyond mimicking features present in existing plans.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>The purpose of this study was to develop and validate a deep reinforcement learning\u2010based VMAT MPO algorithm capable of automatically generating clinically comparable treatment plans for prostate cancer that meet machine constraints, entirely independent of a commercial treatment planning system (TPS) optimizer.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A dataset comprised of 100 prostate cancer patients planned using the criteria from PACE\u2010B SBRT arm serve as the basis for network training using a 70\u201010\u201020 training\/validation\/testing split. An RL framework using a Proximal Policy Optimization (PPO) algorithm was developed to train two tandem convolutional neural networks that sequentially optimize multi\u2010leaf collimator (MLC) positions and monitor units (MUs) using current dose, contoured structure masks, and current machine parameters as inputs. Training was designed to predict MLC positions and MUs that maximize a dose\u2010volume histogram (DVH)\u2010based reward function tailored to prioritize meeting clinical objectives. The fully trained networks were executed on a test set of 20 patients and compared to reference plans optimized with a commercial TPS.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The RL algorithm generated plans in an average of 6.3\u00a0\u00b1\u00a04.7 s. Compared to the reference plans, the RL\u2010generated plans demonstrated improved sparing for both the bladder and rectum across their respective dosimetric endpoints. When normalizing to 95% coverage, the RL generated plans resulted in a statistically significant increase in the PTV , while achieving a significantly reduced  for the rectum. All RL plans successfully satisfied all clinical objectives used to optimize the reference plans.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We successfully developed and validated a deep RL framework for VMAT MPO. The algorithm rapidly generates VMAT prostate cancer treatment plans that meet clinical constraints and are dosimetrically comparable to manually optimized plans without the use of a commercial TPS optimizer. This work demonstrates the feasibility of RL as a tool to fully automate the VMAT planning process, offering the potential to decrease planning times while maintaining plan quality.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/mp.70306","type":"journal-article","created":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T13:39:04Z","timestamp":1769780344000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A tandem reinforcement learning framework for localized prostate cancer treatment planning and machine parameter optimization"],"prefix":"10.1002","volume":"53","author":[{"given":"Nathan","family":"Shaffer","sequence":"first","affiliation":[{"name":"Department of Biomedical Engineering University of Iowa  Iowa City Iowa USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Avinash Reddy","family":"Mudireddy","sequence":"additional","affiliation":[{"name":"The Iowa Initiative for Artificial Intelligence University of Iowa  Iowa City Iowa USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joel","family":"St\u2010Aubin","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology University of Iowa  Iowa City Iowa USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2026,1,30]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2008.02.047"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1118\/1.4908224"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1118\/1.3491675"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.1118\/1.4919742"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1088\/0031\u20109155\/57\/14\/4569"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.prro.2020.07.003"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.phro.2024.100534"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1533033819873922"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1118\/1.4938583"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361\u20106560\/ab039b"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.13597"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.13262"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1118\/1.4914140"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.12621"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/acm2.13583"},{"key":"e_1_2_8_17_1","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R","year":"2018"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586\u2010019\u20101724\u2010z"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature24270"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejmp.2024.104498"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.14114"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.17915"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361\u20106560\/ac09a2"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361\u20106560\/ab18bf"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361\u20106560\/ad4b90"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejmp.2021.12.009"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.17100"},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361\u20106560\/abd66d"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2403365"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.1088\/2057\u20101976\/ad9f6a"},{"key":"e_1_2_8_32_1","unstructured":"PignatelliE FerretJ GeistM et\u00a0al.a survey of temporal credit assignment in deep reinforcement learning. arXiv.2024. doi:10.48550\/arXiv.2312.01072. Preprint posted online."},{"key":"e_1_2_8_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/JRPROC.1961.287775"},{"key":"e_1_2_8_34_1","unstructured":"SchulmanJ MoritzP LevineS JordanM AbbeelP.High\u2010dimensional continuous control using generalized advantage estimation.arXiv.2018. doi:10.48550\/arXiv.1506.02438. Preprint posted online October 20 2018."},{"key":"e_1_2_8_35_1","unstructured":"SchulmanJ WolskiF DhariwalP RadfordA KlimovO.Proximal policy optimization algorithms. arXiv.2017. doi:10.48550\/arXiv.1707.06347. Preprint posted online August 28 2017."},{"key":"e_1_2_8_36_1","unstructured":"MnihV BadiaAP MirzaM et\u00a0al.Asynchronous methods for deep reinforcement learning. arXiv.2016. doi:10.48550\/arXiv.1602.01783. Preprint posted online June 16 2016."},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.3390\/ai6030046"},{"key":"e_1_2_8_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3054912"},{"key":"e_1_2_8_40_1","unstructured":"FlorenceP LynchC ZengA et\u00a0al .Implicit Behavioral Cloning.In: Proceedings of the 5th Conference on Robot Learning. PMLR;2022:158\u2010168. Accessed September 8 2025.https:\/\/proceedings.mlr.press\/v164\/florence22a.html"},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.3390\/math11051110"}],"container-title":["Medical Physics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/mp.70306","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/mp.70306","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/mp.70306","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T13:33:49Z","timestamp":1773063229000},"score":1,"resource":{"primary":{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/10.1002\/mp.70306"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,30]]},"references-count":40,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1002\/mp.70306"],"URL":"https:\/\/doi.org\/10.1002\/mp.70306","archive":["Portico"],"relation":{},"ISSN":["0094-2405","2473-4209"],"issn-type":[{"value":"0094-2405","type":"print"},{"value":"2473-4209","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,30]]},"assertion":[{"value":"2025-10-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70306"}}