{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:07:48Z","timestamp":1758845268373,"version":"3.44.0"},"reference-count":39,"publisher":"IOP Publishing","issue":"19","license":[{"start":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T00:00:00Z","timestamp":1758758400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T00:00:00Z","timestamp":1758758400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"Cancer Australia Priority-driven Collaborative Cancer Research Scheme","award":["1163065"],"award-info":[{"award-number":["1163065"]}]},{"DOI":"10.13039\/501100000925","name":"National Health & Medical Research Council","doi-asserted-by":"crossref","award":["APP1197652"],"award-info":[{"award-number":["APP1197652"]}],"id":[{"id":"10.13039\/501100000925","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100001755","name":"Mesothelioma Applied Research Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100001755","id-type":"DOI","asserted-by":"crossref"}]},{"name":"iCare Dust Diseases Board Fellowship"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Phys. Med. Biol."],"published-print":{"date-parts":[[2025,10,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>\n                  <jats:italic>Objective.<\/jats:italic> To investigate the use of reinforcement learning (RL) algorithms to optimise complex combination cancer therapies. The RL algorithm investigated the effect of varying the radiotherapy (RT) dose in each fraction when administered in conjunction with the immune checkpoint inhibitors (ICIs) anti-PD-1 and anti-CTLA-4. <jats:italic>Approach.<\/jats:italic> Data were available for BALB\/c mice inoculated with a syngeneic mesothelioma tumour on the flank, treated with combination RT and ICI with tumour growth subsequently measured. A deep <jats:italic>Q<\/jats:italic>-network (DQN) and a double DQN were trained using a mechanistic model fitted to the mesothelioma volumes to simulate the dynamics of the tumour microenvironment. Two reward functions were created for the RL algorithm to optimise: the first only considered tumour cell killing, while the second penalised treatment schedules with higher total RT dose. Comparison with experimental results was via the tumour control probability (TCP). <jats:italic>Main Results.<\/jats:italic> All the TCPs obtained with the RL algorithm exceeded the TCPs obtained with the same mechanistic model when only 1 or 2 fractions of RT were administered. However, the baseline schedule of 2 Gy per fraction outperformed the treatment schedules generated by RL. <jats:italic>Significance.<\/jats:italic> This study highlights the potential for RL to explore the vast solution space of possible treatment schedules, conceivably at the individual patient level.<\/jats:p>","DOI":"10.1088\/1361-6560\/ae0863","type":"journal-article","created":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T22:57:32Z","timestamp":1758149852000},"page":"195009","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Reinforcement learning with mechanistic models to optimise radiotherapy and immunotherapy combinations: a proof of concept"],"prefix":"10.1088","volume":"70","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5716-0802","authenticated-orcid":true,"given":"Allison M","family":"Ng","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3080-9655","authenticated-orcid":false,"given":"Du Q","family":"Huynh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6570-2140","authenticated-orcid":false,"given":"Rebecca A","family":"D\u2019Alonzo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3615-3618","authenticated-orcid":false,"given":"Synat","family":"Keam","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8306-7742","authenticated-orcid":true,"given":"Pejman","family":"Rowshanfarzad","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9317-9526","authenticated-orcid":false,"given":"Anna K","family":"Nowak","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9755-7232","authenticated-orcid":false,"given":"Suki","family":"Gill","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5339-5374","authenticated-orcid":true,"given":"Alistair M","family":"Cook","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6875-0719","authenticated-orcid":true,"given":"Martin A","family":"Ebert","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2025,9,25]]},"reference":[{"key":"pmbae0863bib1","doi-asserted-by":"publisher","first-page":"1110","DOI":"10.1016\/j.neo.2021.09.003","type":"journal-article","article-title":"Tumor-immune ecosystem dynamics define an individual radiation immune score to predict pan-cancer radiocurability","volume":"23","author":"Alfonso","year":"2021","journal-title":"Neoplasia"},{"key":"pmbae0863bib2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2020.100897","type":"journal-article","article-title":"On the immunological consequences of conventionally fractionated radiotherapy","volume":"23","author":"Alfonso","year":"2020","journal-title":"iScience"},{"key":"pmbae0863bib3","doi-asserted-by":"publisher","first-page":"902","DOI":"10.1093\/jrr\/rru020","type":"journal-article","article-title":"Clinical oxygen enhancement ratio of tumors in carbon ion radiotherapy: the influence of local oxygenation changes","volume":"55","author":"Antonovic","year":"2014","journal-title":"J. Radiat. Res."},{"key":"pmbae0863bib4","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1016\/S0140-6736(20)32714-8","type":"journal-article","article-title":"First-line nivolumab plus ipilimumab in unresectable malignant pleural mesothelioma (checkmate 743): a multicentre, randomised, open-label, phase 3 trial","volume":"397","author":"Baas","year":"2021","journal-title":"Lancet"},{"key":"pmbae0863bib5","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1038\/nrc3958","type":"journal-article","article-title":"The tumour microenvironment after radiotherapy: mechanisms of resistance and recurrence","volume":"15","author":"Barker","year":"2015","journal-title":"Nat. Rev. Cancer"},{"key":"pmbae0863bib6","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1007\/s00262-024-03889-x","type":"journal-article","article-title":"Fractionated low-dose radiotherapy primes the tumor microenvironment for immunotherapy in a murine mesothelioma model","volume":"74","author":"D\u2019Alonzo","year":"2025","journal-title":"Cancer Immunol. Immunother"},{"key":"pmbae0863bib7","doi-asserted-by":"publisher","DOI":"10.1016\/j.janxdis.2023.102763","type":"other","article-title":"Hyperparameters in reinforcement learning and how to tune them","author":"Eimer","year":"2023"},{"key":"pmbae0863bib8","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1561\/2200000071","type":"journal-article","article-title":"An introduction to deep reinforcement learning","volume":"11","author":"Francois-Lavet","year":"2018","journal-title":"Found. Trends Mach. Learn."},{"key":"pmbae0863bib9","doi-asserted-by":"publisher","DOI":"10.5772\/intechopen.92029","type":"book","article-title":"Biologically effective dose (bed) or radiation biological effect (rbef)?","author":"Frometa-Castillo","year":"2020"},{"year":"2019","author":"Geron","key":"pmbae0863bib10","type":"book"},{"key":"pmbae0863bib11","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2021.106455","type":"journal-article","article-title":"Clinically validated model predicts the effect of intratumoral heterogeneity on overall survival for non-small cell lung cancer (NSCLC) patients","volume":"212","author":"Ghaderi","year":"2021","journal-title":"Comput. Methods Programs Biomed."},{"key":"pmbae0863bib12","doi-asserted-by":"publisher","first-page":"1316","DOI":"10.3390\/ijms23031316","type":"journal-article","article-title":"A century of fractionated radiotherapy: how mathematical oncology can break the rules","volume":"23","author":"Ghaderi","year":"2022","journal-title":"Int. J. Mol. Sci."},{"key":"pmbae0863bib13","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1016\/j.clon.2008.01.011","type":"journal-article","article-title":"Concomitant boost radiotherapy compared with conventional radiotherapy in squamous cell carcinoma of the head and neck\u2014a phase iii trial from a single institution in India","volume":"20","author":"Ghoshal","year":"2008","journal-title":"Clin. Oncol."},{"article-title":"Replication data for \u201da biomathematical model of tumor response to radioimmunotherapy with (\u03b1)PDL1 and (\u03b1)CTLA4\u201d","year":"2022","author":"Gonzalez-Crespo","key":"pmbae0863bib14","type":"other"},{"key":"pmbae0863bib15","doi-asserted-by":"publisher","first-page":"808","DOI":"10.1109\/TCBB.2022.3174454","type":"journal-article","article-title":"A biomathematical model of tumor response to radioimmunotherapy with \u03b1PDL1 and \u03b1CTLA4","volume":"20","author":"Gonzalez-Crespo","year":"2023","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform."},{"key":"pmbae0863bib16","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1593\/tlo.12280","type":"journal-article","article-title":"A systemic complete response of metastatic melanoma to local radiation and immunotherapy","volume":"5","author":"Hiniker","year":"2012","journal-title":"Transl. Oncol."},{"key":"pmbae0863bib17","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.jtho.2022.11.018","type":"journal-article","article-title":"Considerations for imaging of malignant pleural mesothelioma: a consensus statement from the international mesothelioma interest group","volume":"18","author":"Katz","year":"2023","journal-title":"J. Thoracic Oncol."},{"key":"pmbae0863bib18","doi-asserted-by":"publisher","first-page":"e409","DOI":"10.1016\/j.ijrobp.2022.07.1587","type":"journal-article","article-title":"Low dose fractionated radiotherapy alters tumor blood vessel morphology, increases CD3+CD8+ t cell infiltration to the tumor, and sensitizes mesothelioma to immune checkpoint blockade","volume":"114","author":"Keam","year":"2022a","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbae0863bib19","doi-asserted-by":"publisher","DOI":"10.1016\/j.adro.2022.101013","type":"journal-article","article-title":"Effects of photon radiation on DNA damage, cell proliferation, cell survival and apoptosis of murine and human mesothelioma cell lines","volume":"7","author":"Keam","year":"2022b","journal-title":"Adv. Radiat. Oncol."},{"key":"pmbae0863bib20","doi-asserted-by":"publisher","first-page":"3521","DOI":"10.1073\/pnas.1611835114","type":"journal-article","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2017","journal-title":"Proc. Natl Acad. Sci."},{"key":"pmbae0863bib21","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.canlet.2020.03.017","type":"journal-article","article-title":"Hypoxia in tumor microenvironment regulates exosome biogenesis: molecular mechanisms and translational opportunities","volume":"479","author":"Kumar","year":"2020","journal-title":"Cancer Lett."},{"key":"pmbae0863bib22","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2022.916839","type":"journal-article","article-title":"Novel insights into mesothelioma therapy: emerging avenues and future prospects","volume":"12","author":"Kuryk","year":"2022","journal-title":"Front. Oncol."},{"key":"pmbae0863bib23","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1007\/s11425-019-1648-6","type":"journal-article","article-title":"Mathematical modeling of cancer treatment with radiation and pd-l1 inhibitor","volume":"63","author":"Lai","year":"2020","journal-title":"Sci. China Math."},{"key":"pmbae0863bib24","doi-asserted-by":"publisher","first-page":"644","DOI":"10.1038\/s41416-020-0912-9","type":"journal-article","article-title":"Impact of hypoxia on chemoresistance of mesothelioma mediated by the proton-coupled folate transporter, and preclinical activity of new anti-LDH-A compounds","volume":"123","author":"Li Petri","year":"2020","journal-title":"Br. J. Cancer"},{"key":"pmbae0863bib25","doi-asserted-by":"publisher","first-page":"581","DOI":"10.5306\/wjco.v12.i8.581","type":"journal-article","article-title":"GOECP\/SEOR clinical guidelines on radiotherapy for malignant pleural mesothelioma","volume":"12","author":"Luna","year":"2021","journal-title":"World J. Clin. Oncol."},{"article-title":"Mesothelioma in Australia 2023","year":"2024","author":"Australian Institute of Health and Welfare","key":"pmbae0863bib26","type":"web-resource"},{"article-title":"Playing atari with deep reinforcement learning","year":"2013","author":"Mnih","key":"pmbae0863bib27","type":"preprint"},{"key":"pmbae0863bib28","doi-asserted-by":"publisher","first-page":"214","DOI":"10.3390\/biomedicines9020214","type":"journal-article","article-title":"Reinforcement learning for radiotherapy dose fractioning automation","volume":"9","author":"Moreau","year":"2021","journal-title":"Biomedicines"},{"key":"pmbae0863bib29","doi-asserted-by":"publisher","first-page":"340","DOI":"10.18632\/genesandcancer.124","type":"journal-article","article-title":"Switching off malignant mesothelioma: exploiting the hypoxic microenvironment","volume":"7","author":"Nabavi","year":"2016","journal-title":"Genes Cancer."},{"article-title":"Mesothelioma: peritoneal, version 1.2024","year":"2024a","author":"National Comprehensive Cancer Network","key":"pmbae0863bib30","type":"web-resource"},{"article-title":"Mesothelioma: pleural, version 1.2024","year":"2024b","author":"National Comprehensive Cancer Network","key":"pmbae0863bib31","type":"web-resource"},{"key":"pmbae0863bib32","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.lungcan.2005.12.013","type":"journal-article","article-title":"Modeling of mesothelioma growth demonstrates weaknesses of current response criteria","volume":"52","author":"Oxnard","year":"2006","journal-title":"Lung Cancer"},{"key":"pmbae0863bib33","doi-asserted-by":"publisher","first-page":"1530","DOI":"10.1016\/j.ijrobp.2021.03.021","type":"journal-article","article-title":"A novel analytical population tumor control probability model includes cell density and volume variations: application to canine brain tumor","volume":"110","author":"Radonic","year":"2021","journal-title":"Int. J. Radiat. Oncol. Biol. Phys."},{"key":"pmbae0863bib34","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1007\/s42979-021-00592-x","type":"journal-article","article-title":"Machine learning: algorithms, real-world applications and research directions","volume":"2","author":"Sarker","year":"2021","journal-title":"SN Comput. Sci."},{"key":"pmbae0863bib35","doi-asserted-by":"publisher","first-page":"1625","DOI":"10.3389\/fimmu.2017.01625","type":"journal-article","article-title":"Hypoxic stress-induced tumor and immune plasticity, suppression, and impact on tumor heterogeneity","volume":"8","author":"Terry","year":"2017","journal-title":"Front. Immunol."},{"key":"pmbae0863bib36","type":"conference-proceedings","article-title":"Deep reinforcement learning with double q-learning","volume":"vol 30","author":"Van Hasselt","year":"2015"},{"key":"pmbae0863bib37","doi-asserted-by":"publisher","first-page":"563","DOI":"10.1016\/j.jacbts.2018.01.014","type":"journal-article","article-title":"Radiation-induced endothelial vascular injury: a review of possible mechanisms","volume":"3","author":"Venkatesulu","year":"2018","journal-title":"Jacc"},{"key":"pmbae0863bib38","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1136\/oem.17.4.260","type":"journal-article","article-title":"Diffuse pleural mesothelioma and asbestos exposure in the north western cape province","volume":"17","author":"Wagner","year":"1960","journal-title":"Br. J. Ind. Med."},{"key":"pmbae0863bib39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.neo.2014.12.004","type":"journal-article","article-title":"Spherical cancer models in tumor biology","volume":"17","author":"Weiswald","year":"2015","journal-title":"Neoplasia"}],"container-title":["Physics in Medicine &amp; Biology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T07:23:57Z","timestamp":1758785037000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/1361-6560\/ae0863"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,25]]},"references-count":39,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2025,9,25]]},"published-print":{"date-parts":[[2025,10,5]]}},"URL":"https:\/\/doi.org\/10.1088\/1361-6560\/ae0863","relation":{},"ISSN":["0031-9155","1361-6560"],"issn-type":[{"type":"print","value":"0031-9155"},{"type":"electronic","value":"1361-6560"}],"subject":[],"published":{"date-parts":[[2025,9,25]]},"assertion":[{"value":"Reinforcement learning with mechanistic models to optimise radiotherapy and immunotherapy combinations: a proof of concept","name":"article_title","label":"Article Title"},{"value":"Physics in Medicine & Biology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published on behalf of Institute of Physics and Engineering in Medicine by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2025-05-28","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-09-17","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-09-25","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}