{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T08:42:28Z","timestamp":1792658548904,"version":"4.5.0"},"reference-count":50,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2024,2,9]],"date-time":"2024-02-09T00:00:00Z","timestamp":1707436800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["ascpt.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Clin Pharma and Therapeutics"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:p>The integration of pharmacokinetic\u2010pharmacodynamic (PK\u2010PD) modeling and simulations with artificial intelligence\/machine learning algorithms is one of the most attractive areas of the pharmacometric research. These hybrid techniques are currently under investigation to perform several tasks, among which precision dosing. In this scenario, this paper presents and evaluates a new framework embedding PK\u2010PD models into a reinforcement learning (RL) algorithm, Q\u2010learning (QL), to personalize pharmacological treatment. Each patient is represented with a set of PK\u2010PD parameters and has a personal QL agent which optimizes the individual treatment. In the training phase, leveraging PK\u2010PD simulations, the QL agent assesses different actions, defined consistently with the clinical knowledge to consider only plausible dose\u2010adjustments, in order to find the optimal rules. The proposed framework was evaluated to optimize the erdafitinib treatment in patients with metastatic urothelial carcinoma. This drug was approved by the US Food and Drug Administration (FDA) with a dose\u2010adaptive protocol based on monitoring the levels of serum phosphate, which represent a biomarker of both treatment efficacy and toxicity. To evaluate the flexibility of the methodology, a heterogeneous virtual population of 141 patients was generated using an erdafitinib population PK (PopPK)\u2010PD literature model. For each patient, treatment response was simulated by using both QL\u2010optimized protocol and the clinical one. QL agents outperform the approved dose\u2010adaptive rules, increasing more than 10% the efficacy and the safety of treatment at each end point. Results confirm the great potentialities of the integration of PopPK\u2010PD models and RL algorithms to optimize precision dosing tasks.<\/jats:p>","DOI":"10.1002\/cpt.3176","type":"journal-article","created":{"date-parts":[[2024,2,10]],"date-time":"2024-02-10T01:34:38Z","timestamp":1707528878000},"page":"825-838","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Reinforcement Learning and\n                    <scp>PK\u2010PD<\/scp>\n                    Models Integration to Personalize the Adaptive Dosing Protocol of Erdafitinib in Patients with Metastatic Urothelial Carcinoma"],"prefix":"10.1002","volume":"115","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7055-2140","authenticated-orcid":false,"given":"Alessandro","family":"De Carlo","sequence":"first","affiliation":[{"name":"Electrical, Computer, and Biomedical Engineering University of Pavia  Pavia Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9801-2135","authenticated-orcid":false,"given":"Elena Maria","family":"Tosca","sequence":"additional","affiliation":[{"name":"Electrical, Computer, and Biomedical Engineering University of Pavia  Pavia Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martina","family":"Fantozzi","sequence":"additional","affiliation":[{"name":"Electrical, Computer, and Biomedical Engineering University of Pavia  Pavia Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8931-4676","authenticated-orcid":false,"given":"Paolo","family":"Magni","sequence":"additional","affiliation":[{"name":"Electrical, Computer, and Biomedical Engineering University of Pavia  Pavia Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2024,2,9]]},"reference":[{"key":"e_1_2_9_1_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41573-019-0024-5"},{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.3390\/pharmaceutics13071101"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imed.2021.10.001"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.1803"},{"key":"e_1_2_9_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/cts.13431"},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.14801"},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.1002\/psp4.12418"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/psp4.12377"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.3390\/pharmaceutics14091814"},{"key":"e_1_2_9_10_1","unstructured":"Bartolucci R.et al.Artificial intelligence and machine learning: just a hype or a new opportunity for pharmacometrics?PAGE28(2019)."},{"key":"e_1_2_9_11_1","doi-asserted-by":"crossref","unstructured":"Ronchi D. Tosca E.M. Bartolucci R.B.&Magni P.Go beyond the limits of genetic algorithm in daily covariate selection practice. (2023)https:\/\/doi.org\/10.21203\/rs.3.rs\u20102520116\/v1.","DOI":"10.21203\/rs.3.rs-2520116\/v1"},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1097\/FTD.0000000000001078"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-pharmtox-010617-052446"},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/17512433.2018.1501271"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2020.00420"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2020.00177"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.1933"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2021.1004"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2125.2012.04237.x"},{"key":"e_1_2_9_20_1","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R.S.","year":"2018"},{"key":"e_1_2_9_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.1777"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2022.1094281"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2020.101964"},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.3390\/cancers13184624"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.2196\/18477"},{"key":"e_1_2_9_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/psp4.12858"},{"key":"e_1_2_9_27_1","doi-asserted-by":"publisher","DOI":"10.1002\/psp4.12588"},{"key":"e_1_2_9_28_1","unstructured":"Yauney G.&Shah P.Reinforcement Learning with Action\u2010Derived Rewards for Chemotherapy and Clinical Trial Dosing Regimen Selection.Proceedings of the 3rd Machine Learning for Healthcare Conference F. Doshi\u2010Velez J. Fackler K. Jung D. Kale R. Ranganath B. Wallace and J. Wiens Eds. in Proceedings of Machine Learning Research vol. 85. PMLR Aug. 2018 pp. 161\u2013226. <https:\/\/proceedings.mlr.press\/v85\/yauney18a.html>."},{"key":"e_1_2_9_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105443"},{"key":"e_1_2_9_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2019.03.004"},{"key":"e_1_2_9_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00280-021-04381-4"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/psp4.12727"},{"key":"e_1_2_9_33_1","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2014.60.7341"},{"key":"e_1_2_9_34_1","unstructured":"US Food and Drug Administration.FDA approves first targeted therapy for metastatic bladder cancer. FDA. <https:\/\/www.fda.gov\/news\u2010events\/press\u2010announcements\/fda\u2010approves\u2010first\u2010targeted\u2010therapy\u2010metastatic\u2010bladder\u2010cancer> Accessed April 20 2023."},{"key":"e_1_2_9_35_1","unstructured":"FDA \u2018Erdafitnib Prescribing Information\u2019. <https:\/\/www.accessdata.fda.gov\/drugsatfda_docs\/label\/2019\/212018s000lbl.pdf>."},{"key":"e_1_2_9_36_1","doi-asserted-by":"publisher","DOI":"10.1158\/1078-0432.CCR-18-3334"},{"key":"e_1_2_9_37_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1480"},{"key":"e_1_2_9_38_1","doi-asserted-by":"crossref","unstructured":"Erol T. Mendi A.F.&Do\u011fan D.The Digital Twin Revolution in Healthcare\u2019 in 2020 4th international symposium on multidisciplinary studies and innovative technologies (ISMSIT) Oct. 2020 pp. 1\u20137https:\/\/doi.org\/10.1109\/ISMSIT50672.2020.9255249.","DOI":"10.1109\/ISMSIT50672.2020.9255249"},{"key":"e_1_2_9_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/jcph.1547"},{"key":"e_1_2_9_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(94)00086-G"},{"key":"e_1_2_9_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-2607(97)00044-8"},{"key":"e_1_2_9_42_1","doi-asserted-by":"publisher","DOI":"10.1093\/toxsci\/kft161"},{"key":"e_1_2_9_43_1","doi-asserted-by":"publisher","DOI":"10.1002\/jbmr.1810"},{"key":"e_1_2_9_44_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa1817323"},{"key":"e_1_2_9_45_1","article-title":"Integrating reinforcement learning and PK\u2010PD modelling to enable precision dosing: a multi\u2010objective optimization for the treatment of polycithemia vera patients with givinostat","volume":"31","author":"De Carlo A.","journal-title":"PAGE"},{"key":"e_1_2_9_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.mbs.2017.08.004"},{"key":"e_1_2_9_47_1","first-page":"655","article-title":"Reinforcement learning for closed\u2010loop propofol anesthesia: a study in human volunteers","volume":"15","author":"Moore B.L.","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_9_48_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.28.1.1"},{"key":"e_1_2_9_49_1","doi-asserted-by":"publisher","DOI":"10.1002\/psp4.12745"},{"key":"e_1_2_9_50_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.2202"}],"container-title":["Clinical Pharmacology &amp; Therapeutics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascpt.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpt.3176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,9]],"date-time":"2025-08-09T07:22:45Z","timestamp":1754724165000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascpt.onlinelibrary.wiley.com\/doi\/10.1002\/cpt.3176"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,9]]},"references-count":50,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["10.1002\/cpt.3176"],"URL":"https:\/\/doi.org\/10.1002\/cpt.3176","archive":["Portico"],"relation":{},"ISSN":["0009-9236","1532-6535"],"issn-type":[{"value":"0009-9236","type":"print"},{"value":"1532-6535","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,9]]},"assertion":[{"value":"2023-08-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}