{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T05:19:11Z","timestamp":1787548751487,"version":"build-2736575974"},"reference-count":38,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T00:00:00Z","timestamp":1787529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Maintaining remission in ulcerative colitis (UC) with traditional Chinese herbal medicine is challenging because fixed-intensity prescriptions may not accommodate changes in inflammatory burden. This study aimed to predict 52-week relapse and develop a data-driven policy for sequential adjustment of herbal treatment intensity.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We analysed a development cohort of 1,181 patients receiving herbal maintenance therapy and an independent external validation cohort of 96 patients. Twenty-one baseline clinical, biomarker, herbal-dose and behavioral variables, expanded to 31 features after one-hot encoding, were used to train and compare seven machine-learning models through stratified five-fold cross-validation and external validation. Discrimination, calibration and decision-curve net benefit were evaluated. Remission maintenance was subsequently modelled as a sequential decision problem using 4,141 scheduled-visit transitions. A gradient-boosted reward model was used for model-based off-policy evaluation of a net-clinical-benefit policy that penalized unnecessary treatment intensification in clinically quiescent patients.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Calibrated logistic regression provided the best overall performance, with an internal cross-validation area under the receiver operating characteristic curve (AUROC) of 0.783 (95% confidence interval [CI], 0.755\u20130.809) and an external AUROC of 0.710 (95% CI, 0.600\u20130.820). The corresponding Brier scores were 0.180 (95% CI, 0.169\u20130.191) and 0.215 (95% CI, 0.170\u20130.263), respectively. Medication adherence was the strongest predictor of relapse, followed by fecal calprotectin, C-reactive protein and Coptidis Rhizoma dose. The learned policy increased herbal intensity with increasing fecal calprotectin and achieved a higher estimated mean reward than observed clinician behavior (0.552 \u00b1 0.018 versus 0.063 \u00b1 0.021). Policy-concordant visits were associated with a higher next-visit remission rate than discordant visits (70.0% versus 59.6%).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Combining interpretable relapse prediction with reinforcement learning offers a personalized decision-support framework for optimizing herbal treatment intensity during UC remission maintenance. Because the policy was derived and evaluated using observational data and model-based off-policy methods, prospective clinical evaluation is required before routine implementation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1911189","type":"journal-article","created":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T04:24:57Z","timestamp":1787545497000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Preventing relapse in ulcerative colitis remission maintenance: machine learning prediction and reinforcement-learning herbal prescription optimization with external validation"],"prefix":"10.3389","volume":"13","author":[{"given":"Yan","family":"Li","sequence":"first","affiliation":[{"name":"The Affiliated Yongchuan Hospital of Chongqing Medical University","place":["Chongqing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Liu","sequence":"additional","affiliation":[{"name":"People\u2019s Hospital of Shapingba District","place":["Chongqing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Kang","sequence":"additional","affiliation":[{"name":"Hospital of Chengdu University of Traditional Chinese Medicine","place":["Chengdu, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,24]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1016\/S0140-6736(23)00966-2","article-title":"Ulcerative colitis","volume":"402","author":"Le Berre","year":"2023","journal-title":"Lancet"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1001\/jama.2023.15389","article-title":"Ulcerative colitis in adults: a review","volume":"330","author":"Gros","year":"2023","journal-title":"JAMA"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1424","DOI":"10.1053\/j.gastro.2021.09.076","article-title":"Evolving short- and long-term goals of management in inflammatory bowel disease: getting it right, making it last","volume":"162","author":"Le Berre","year":"2022","journal-title":"Gastroenterology"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"3","DOI":"10.4103\/1319-3767.91726","article-title":"Herbal medicine in the treatment of ulcerative colitis","volume":"18","author":"Ke","year":"2012","journal-title":"Saudi J Gastroenterol"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"571","DOI":"10.3389\/fphar.2018.00571","article-title":"Berberine regulates Treg\/Th17 balance to treat ulcerative colitis through modulating the gut microbiota in the colon","volume":"9","author":"Cui","year":"2018","journal-title":"Front Pharmacol"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/1758-2946-6-13","article-title":"TCMSP: a database of systems pharmacology for drug discovery from herbal medicines","volume":"6","author":"Ru","year":"2014","journal-title":"J Cheminform"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"e0335139","DOI":"10.1371\/journal.pone.0335139","article-title":"Curcumin attenuates LPS-induced inflammation in RAW 264.7 cells: a multifaceted study integrating network pharmacology, molecular docking, molecular dynamics simulation, and experimental validation","volume":"20","author":"Gong","year":"2025","journal-title":"PLoS One"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"1493","DOI":"10.1053\/j.gastro.2021.12.238","article-title":"Artificial intelligence for disease assessment in inflammatory bowel disease: how will it change our practice?","volume":"162","author":"Stidham","year":"2022","journal-title":"Gastroenterology"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1093\/ibd\/izx007","article-title":"Predicting hospitalization and outpatient corticosteroid use in inflammatory bowel disease patients using machine learning","volume":"24","author":"Waljee","year":"2018","journal-title":"Inflamm Bowel Dis"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"108754","DOI":"10.1016\/j.cmpb.2025.108754","article-title":"Reinforcement learning using neural networks in estimating an optimal dynamic treatment regime in patients with sepsis","volume":"266","author":"Liang","year":"2025","journal-title":"Comput Methods Prog Biomed"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"203145","DOI":"10.1109\/ACCESS.2025.3638417","article-title":"A reinforcement learning framework for personalized anticoagulation dosing in critical care: integrating batch-constrained policy optimization and off-policy evaluation","volume":"13","author":"Lim","year":"2025","journal-title":"IEEE Access"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"3427","DOI":"10.3390\/jcm9113427","article-title":"Development of machine learning model to predict the 5-year risk of starting biologic agents in patients with inflammatory bowel disease: K-CDM network study","volume":"9","author":"Choi","year":"2020","journal-title":"J Clin Med"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"ref14","first-page":"785","author":"Chen","year":"2016"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"1925","DOI":"10.1093\/eurheartj\/ehu207","article-title":"Towards better clinical prediction models: seven steps for development and an ABCD for validation","volume":"35","author":"Steyerberg","year":"2014","journal-title":"Eur Heart J"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J Clin Epidemiol"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: the Achilles heel of predictive analytics","volume":"17","author":"Van Calster","year":"2019","journal-title":"BMC Med"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1038\/s41591-018-0310-5","article-title":"Guidelines for reinforcement learning in healthcare","volume":"25","author":"Gottesman","year":"2019","journal-title":"Nat Med"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"e18477","DOI":"10.2196\/18477","article-title":"Reinforcement learning for clinical decision support in critical care: comprehensive review","volume":"22","author":"Liu","year":"2020","journal-title":"J Med Internet Res"},{"key":"ref20","first-page":"2139","author":"Thomas","year":"2016"},{"key":"ref21","first-page":"652","author":"Jiang","year":"2016"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"592","DOI":"10.1016\/j.jep.2011.08.057","article-title":"Efficacy and safety of Fufangkushen colon-coated capsule in the treatment of ulcerative colitis compared with mesalazine: a double-blinded and randomized study","volume":"141","author":"Gong","year":"2012","journal-title":"J Ethnopharmacol"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"665102","DOI":"10.3389\/fphar.2021.665102","article-title":"In silico and in vivo studies on the mechanisms of Chinese medicine formula (Gegen Qinlian decoction) in the treatment of ulcerative colitis","volume":"12","author":"Liu","year":"2021","journal-title":"Front Pharmacol"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"935","DOI":"10.1053\/j.gastro.2017.11.024","article-title":"Efficacy of indigo naturalis in a multicenter randomized controlled trial of patients with ulcerative colitis","volume":"154","author":"Naganuma","year":"2018","journal-title":"Gastroenterology"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"g7594","DOI":"10.1136\/bmj.g7594","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement","volume":"350","author":"Collins","year":"2015","journal-title":"BMJ"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"W1","DOI":"10.7326\/M14-0698","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration","volume":"162","author":"Moons","year":"2015","journal-title":"Ann Intern Med"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1660","DOI":"10.1002\/ibd.20520","article-title":"Use of the noninvasive components of the Mayo score to assess clinical response in ulcerative colitis","volume":"14","author":"Lewis","year":"2008","journal-title":"Inflamm Bowel Dis"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"802","DOI":"10.1038\/ajg.2015.120","article-title":"C-reactive protein, fecal calprotectin, and stool lactoferrin for detection of endoscopic activity in symptomatic inflammatory bowel disease patients: a systematic review and meta-analysis","volume":"110","author":"Mosli","year":"2015","journal-title":"Am J Gastroenterol"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"CD000544","DOI":"10.1002\/14651858.CD000544.pub4","article-title":"Oral 5-aminosalicylic acid for maintenance of remission in ulcerative colitis","volume":"2016","author":"Wang","year":"2016","journal-title":"Cochrane Database Syst Rev"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1038\/ajg.2009.685","article-title":"Factors associated with non-adherence to oral medication for inflammatory bowel disease: a systematic review","volume":"105","author":"Jackson","year":"2010","journal-title":"Am J Gastroenterol"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"2929","DOI":"10.1111\/j.1572-0241.2001.04683.x","article-title":"Prevalence of nonadherence with maintenance mesalamine in quiescent ulcerative colitis","volume":"96","author":"Kane","year":"2001","journal-title":"Am J Gastroenterol"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"749.1","DOI":"10.1136\/gut.2005.082909","article-title":"The Montreal classification of inflammatory bowel disease: controversies, consensus, and implications","volume":"55","author":"Satsangi","year":"2006","journal-title":"Gut"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: a novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med Decis Mak"},{"key":"ref34","first-page":"4765","author":"Lundberg","year":"2017"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"1407","DOI":"10.1097\/MIB.0000000000000057","article-title":"Meta-analysis: fecal calprotectin for assessment of inflammatory bowel disease activity","volume":"20","author":"Lin","year":"2014","journal-title":"Inflamm Bowel Dis"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1053\/gast.2000.8523","article-title":"Surrogate markers of intestinal inflammation are predictive of relapse in patients with inflammatory bowel disease","volume":"119","author":"Tibble","year":"2000","journal-title":"Gastroenterology"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"e0257520","DOI":"10.1371\/journal.pone.0257520","article-title":"Replicating prediction algorithms for hospitalization and corticosteroid use in patients with inflammatory bowel disease","volume":"16","author":"Gan","year":"2021","journal-title":"PLoS One"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"1570","DOI":"10.1053\/j.gastro.2020.12.031","article-title":"STRIDE-II: an update on the selecting therapeutic targets in inflammatory bowel disease (STRIDE) initiative of the International Organization for the Study of IBD (IOIBD): determining therapeutic goals for treat-to-target strategies in IBD","volume":"160","author":"Turner","year":"2021","journal-title":"Gastroenterology"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1911189\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T04:24:58Z","timestamp":1787545498000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1911189\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,24]]},"references-count":38,"alternative-id":["10.3389\/fmed.2026.1911189"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1911189","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,24]]},"article-number":"1911189"}}