{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T22:09:49Z","timestamp":1766268589930,"version":"3.37.3"},"reference-count":47,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T00:00:00Z","timestamp":1734566400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ANR","award":["ANR-21-CE40-005"],"award-info":[{"award-number":["ANR-21-CE40-005"]}]},{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["890462"],"award-info":[{"award-number":["890462"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Designing patient-specific follow-up strategies is key to personalized cancer care. Tools to assist doctors in treatment decisions and scheduling follow-ups based on patient preferences and medical data would be highly beneficial. These tools should incorporate realistic models of disease progression under treatment, multi-objective optimization of treatment strategies, and efficient algorithms to personalize follow-ups by considering patient history. We propose modeling cancer evolution using a Piecewise Deterministic Markov Process, where patients alternate between remission and relapse phases, and control the model via long-term cost function optimization. This considers treatment side effects, visit burden, and quality of life, using noisy blood marker measurements for feedback. Instead of discretizing the problem with a discrete Markov Decision Process, we apply the Partially-Observed Monte-Carlo Planning algorithm to solve the continuous-time, continuous-state problem, leveraging the near-deterministic nature of cancer progression. Our approach, tested on multiple myeloma patient data, outperforms exact solutions of the discrete model and allows greater flexibility in cost function modeling, enabling patient-specific follow-ups. This method can also be adapted to other diseases.<\/jats:p>","DOI":"10.1371\/journal.pone.0315661","type":"journal-article","created":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T18:26:36Z","timestamp":1734632796000},"page":"e0315661","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["A Monte-Carlo planning strategy for medical follow-up optimization: Illustration on multiple myeloma data"],"prefix":"10.1371","volume":"19","author":[{"given":"Beno\u00eete","family":"de Saporta","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2427-889X","authenticated-orcid":true,"given":"Aymar","family":"Thierry d\u2019Argenlieu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R\u00e9gis","family":"Sabbadin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8083-0204","authenticated-orcid":true,"given":"Alice","family":"Cleynen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2024,12,19]]},"reference":[{"issue":"12","key":"pone.0315661.ref001","first-page":"4675","article-title":"Analysis of circulating tumor DNA in plasma at diagnosis and during follow-up of lung cancer patients","volume":"61","author":"G Sozzi","year":"2001","journal-title":"Cancer research"},{"issue":"10","key":"pone.0315661.ref002","doi-asserted-by":"crossref","first-page":"1935","DOI":"10.1158\/1055-9965.EPI-06-0994","article-title":"Blood markers for early detection of colorectal cancer: a systematic review","volume":"16","author":"S Hundt","year":"2007","journal-title":"Cancer Epidemiology Biomarkers & Prevention"},{"issue":"6","key":"pone.0315661.ref003","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1074\/mcp.R300006-MCP200","article-title":"Tumor markers: from laboratory to clinical utility","volume":"2","author":"AS Schrohl","year":"2003","journal-title":"Molecular & Cellular Proteomics"},{"issue":"01","key":"pone.0315661.ref004","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4103\/0971-5851.56328","article-title":"Tumor markers in clinical practice: General principles and guidelines","volume":"30","author":"S Sharma","year":"2009","journal-title":"Indian journal of medical and paediatric oncology"},{"issue":"1","key":"pone.0315661.ref005","doi-asserted-by":"crossref","first-page":"17","DOI":"10.4103\/0975-5950.196135","article-title":"Tumor markers: A diagnostic tool","volume":"7","author":"M Nagpal","year":"2016","journal-title":"National journal of maxillofacial surgery"},{"issue":"8","key":"pone.0315661.ref006","doi-asserted-by":"crossref","first-page":"e170","DOI":"10.1371\/journal.pmed.0050170","article-title":"Cancer screening: a mathematical model relating secreted blood biomarker levels to tumor sizes","volume":"5","author":"AM Lutz","year":"2008","journal-title":"PLoS medicine"},{"issue":"1","key":"pone.0315661.ref007","doi-asserted-by":"crossref","first-page":"22830","DOI":"10.1038\/s41598-023-49832-5","article-title":"Prognostic value of normal levels of preoperative tumor markers in colorectal cancer","volume":"13","author":"G Ren","year":"2023","journal-title":"Scientific reports"},{"issue":"2","key":"pone.0315661.ref008","doi-asserted-by":"crossref","first-page":"e0149422","DOI":"10.1371\/journal.pone.0149422","article-title":"A mathematical model coupling tumor growth and angiogenesis","volume":"11","author":"J Xu","year":"2016","journal-title":"PloS one"},{"key":"pone.0315661.ref009","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1200\/CCI.19.00133","article-title":"Machine learning and mechanistic modeling for prediction of metastatic relapse in early-stage breast cancer","volume":"4","author":"C Nicol\u00f2","year":"2020","journal-title":"JCO clinical cancer informatics"},{"issue":"5","key":"pone.0315661.ref010","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.3390\/cancers15051368","article-title":"Predicting patient-specific tumor dynamics: how many measurements are necessary?","volume":"15","author":"I Harshe","year":"2023","journal-title":"Cancers"},{"issue":"1","key":"pone.0315661.ref011","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1001\/jamaoncol.2018.3395","article-title":"Optimizing cancer treatment using game theory: a review","volume":"5","author":"K Sta\u0148kov\u00e1","year":"2019","journal-title":"JAMA oncology"},{"issue":"2","key":"pone.0315661.ref012","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/s13235-021-00397-w","article-title":"The contribution of evolutionary game theory to understanding and treating cancer","volume":"12","author":"B W\u00f6lfl","year":"2022","journal-title":"Dynamic Games and Applications"},{"issue":"1","key":"pone.0315661.ref013","doi-asserted-by":"crossref","first-page":"1816","DOI":"10.1038\/s41467-017-01968-5","article-title":"Integrating evolutionary dynamics into treatment of metastatic castrate-resistant prostate cancer","volume":"8","author":"J Zhang","year":"2017","journal-title":"Nature communications"},{"issue":"4","key":"pone.0315661.ref014","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1158\/0008-5472.CAN-20-0806","article-title":"Turnover modulates the need for a cost of resistance in adaptive therapy","volume":"81","author":"MA Strobl","year":"2021","journal-title":"Cancer research"},{"issue":"4","key":"pone.0315661.ref015","doi-asserted-by":"crossref","first-page":"823","DOI":"10.3390\/cancers13040823","article-title":"Adaptive therapy for metastatic melanoma: predictions from patient calibrated mathematical models","volume":"13","author":"E Kim","year":"2021","journal-title":"Cancers"},{"key":"pone.0315661.ref016","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jtbi.2018.09.022","article-title":"Optimal control to develop therapeutic strategies for metastatic castrate resistant prostate cancer","volume":"459","author":"JJ Cunningham","year":"2018","journal-title":"Journal of theoretical biology"},{"issue":"3","key":"pone.0315661.ref017","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1111\/j.2517-6161.1984.tb01308.x","article-title":"Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models","volume":"46","author":"MH Davis","year":"1984","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"key":"pone.0315661.ref018","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1051\/proc\/201760225","article-title":"Probabilistic and piecewise deterministic models in biology","volume":"60","author":"B Cloez","year":"2017","journal-title":"ESAIM: Proceedings and Surveys"},{"key":"pone.0315661.ref019","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-61295-9","volume-title":"Piecewise deterministic processes in biological models","author":"R Rudnicki","year":"2017"},{"volume-title":"Markov models and optimization. vol. 49 of Monographs on Statistics and Applied Probability","year":"1993","author":"M Davis","key":"pone.0315661.ref020"},{"key":"pone.0315661.ref021","doi-asserted-by":"crossref","DOI":"10.1002\/9781119145066","volume-title":"Numerical methods for simulation and optimization of piecewise deterministic Markov processes: application to reliability","author":"B de Saporta","year":"2015"},{"issue":"14","key":"pone.0315661.ref022","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1056\/NEJMoa1611750","article-title":"Lenalidomide, bortezomib, and dexamethasone with transplantation for myeloma","volume":"376","author":"M Attal","year":"2017","journal-title":"New England Journal of Medicine"},{"key":"pone.0315661.ref023","first-page":"399","article-title":"Impulse control of piecewise deterministic Markov processes","author":"M Dempster","year":"1995","journal-title":"The Annals of Applied Probability"},{"issue":"2","key":"pone.0315661.ref024","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1137\/S0363012999364474","article-title":"A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes","volume":"40","author":"A Almudevar","year":"2001","journal-title":"SIAM Journal on Control and Optimization"},{"key":"pone.0315661.ref025","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-6983-4","volume-title":"Continuous average control of piecewise deterministic Markov processes","author":"OL do Valle Costa","year":"2013"},{"issue":"3","key":"pone.0315661.ref026","doi-asserted-by":"crossref","first-page":"417","DOI":"10.3934\/nhm.2017019","article-title":"Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via optogenetics","volume":"12","author":"V Renault","year":"2017","journal-title":"Netw Heterog Media"},{"key":"pone.0315661.ref027","unstructured":"Cleynen A, de Saporta B. Numerical method to solve impulse control problems for partially observed piecewise deterministic Markov processes. arXiv preprint arXiv:211209408. 2023;."},{"key":"pone.0315661.ref028","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-18324-9","volume-title":"Markov decision processes with applications to finance","author":"N B\u00e4uerle","year":"2011"},{"key":"pone.0315661.ref029","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.automatica.2018.08.011","article-title":"Change-point detection for Piecewise Deterministic Markov Processes","volume":"797","author":"A Cleynen","year":"2018","journal-title":"Automatica J IFAC"},{"key":"pone.0315661.ref030","article-title":"Monte-Carlo planning in large POMDPs","volume":"23","author":"D Silver","year":"2010","journal-title":"Advances in neural information processing systems"},{"issue":"27","key":"pone.0315661.ref031","first-page":"5436","article-title":"How I treat multiple myeloma in younger patients","volume":"114","author":"AK Stewart","year":"2009","journal-title":"Blood, The Journal of the American Society of Hematology"},{"issue":"13","key":"pone.0315661.ref032","first-page":"1536","article-title":"How I treat high-risk myeloma","volume":"126","author":"S Lonial","year":"2015","journal-title":"Blood, The Journal of the American Society of Hematology"},{"issue":"27-28","key":"pone.0315661.ref033","first-page":"470","article-title":"The diagnosis and treatment of multiple myeloma","volume":"113","author":"C Gerecke","year":"2016","journal-title":"Deutsches \u00c4rzteblatt International"},{"issue":"no. 55","key":"pone.0315661.ref034","first-page":"48","article-title":"Limit theorems for infinite-dimensional piecewise deterministic Markov processes. Applications to stochastic excitable membrane models","volume":"17","author":"MG Riedler","year":"2012","journal-title":"Electron J Probab"},{"issue":"1-2","key":"pone.0315661.ref035","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","article-title":"Planning and acting in partially observable stochastic domains","volume":"101","author":"LP Kaelbling","year":"1998","journal-title":"Artificial intelligence"},{"issue":"25","key":"pone.0315661.ref036","doi-asserted-by":"crossref","first-page":"2340","DOI":"10.1021\/j100540a008","article-title":"Exact stochastic simulation of coupled chemical reactions","volume":"81","author":"DT Gillespie","year":"1977","journal-title":"The journal of physical chemistry"},{"issue":"1","key":"pone.0315661.ref037","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1017\/apr.2019.55","article-title":"Thinning and multilevel Monte Carlo methods for piecewise deterministic (Markov) processes with an application to a stochastic Morris-Lecar model","volume":"52","author":"V Lemaire","year":"2020","journal-title":"Adv in Appl Probab"},{"key":"pone.0315661.ref038","first-page":"555","article-title":"Non Linear Filtering: Interacting Particle Solution","volume":"2","author":"P Del Moral","year":"1996","journal-title":"Markov Processes and Related Fields"},{"key":"pone.0315661.ref039","unstructured":"Cleynen A, Thierry d\u2019Argenlieu A. POMCP4POMDP, Implementation of POMCP algorithm adapted for PDMP models; 2023. Available from: https:\/\/github.com\/acleynen\/pomcp4pdmp."},{"volume-title":"Neuro-dynamic programming","year":"1996","author":"D Bertsekas","key":"pone.0315661.ref040"},{"volume-title":"Reinforcement learning: An introduction","year":"2018","author":"RS Sutton","key":"pone.0315661.ref041"},{"key":"pone.0315661.ref042","unstructured":"Katt S, Oliehoek FA, Amato C. Learning in POMDPs with Monte Carlo tree search. In: International Conference on Machine Learning. PMLR; 2017. p. 1819\u20131827."},{"issue":"1","key":"pone.0315661.ref043","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.clinthera.2021.11.002","article-title":"Reinforcement learning methods in public health","volume":"44","author":"J Weltz","year":"2022","journal-title":"Clinical therapeutics"},{"issue":"1","key":"pone.0315661.ref044","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1186\/s12911-022-01789-7","article-title":"Reinforcement learning evaluation of treatment policies for patients with hepatitis C virus","volume":"22","author":"B Oselio","year":"2022","journal-title":"BMC Medical Informatics and Decision Making"},{"key":"pone.0315661.ref045","article-title":"POMCPOW: An online algorithm for POMDPs with continuous state, action, and observation spaces","author":"Z Sunberg","year":"2017","journal-title":"ArXiv"},{"key":"pone.0315661.ref046","unstructured":"Lee J, Ahmed NR, Wray KH, Sunberg ZN. Rao-Blackwellized POMDP Planning; 2024. Available from: https:\/\/arxiv.org\/abs\/2409.16392."},{"volume-title":"Artificial intelligence: a modern approach","year":"2013","author":"S Russel","key":"pone.0315661.ref047"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0315661","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T18:26:55Z","timestamp":1734632815000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0315661"}},"subtitle":[],"editor":[{"given":"In\u00e9s P.","family":"Mari\u00f1o","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2024,12,19]]},"references-count":47,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12,19]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0315661","relation":{},"ISSN":["1932-6203"],"issn-type":[{"type":"electronic","value":"1932-6203"}],"subject":[],"published":{"date-parts":[[2024,12,19]]}}}