{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T00:25:27Z","timestamp":1791419127557,"version":"4.3.3"},"reference-count":24,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T00:00:00Z","timestamp":1780876800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100018227","name":"National Research Foundation of Ukraine","doi-asserted-by":"crossref","award":["2023.03\/0197"],"award-info":[{"award-number":["2023.03\/0197"]}],"id":[{"id":"10.13039\/100018227","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Managing an epidemic demands policies that respond at the pace of the outbreak. Conventional rule\u2011based interventions struggle to keep up, prompting interest in reinforcement learning (RL) for designing non\u2011pharmaceutical interventions (NPIs). However, current evidence is fragmented across diverse models and reporting styles.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To systematically map how RL is applied for epidemic NPI design, describe modeling choices, algorithm architectures, evaluation practices, and identify trends and research gaps.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Peer-reviewed studies (2014\u20132025, English) that applied deep RL to select NPIs were retrieved from IEEE Xplore, ACM Digital Library, ScienceDirect, and Scopus, searched on December 23, 2025. Reference list scanning supplemented database results. Predefined data items (bibliographic details, epidemic and RL model characteristics, experiments, validation methods, outcomes) were charted and summarized descriptively.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec004\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Of 512 retrieved records, 10 met the inclusion criteria, and three additional studies were identified via reference-list scanning, yielding 13. Five employed value\u2011based methods, four policy\u2011gradient, and four hybrid; one study additionally incorporated model-based planning. Six simulations relied on compartmental models, six on agent\u2011based models, and one on a hybrid model. Action spaces were predominantly discrete restriction levels. Five studies incorporated sequence-modeling techniques to include temporal context into a state space. Eleven studies designed reward functions as a trade-off between pandemic severity and socio-economic cost. According to the reviewed studies, RL policies across various settings outperform heuristic, rule-based, and historical baselines in reducing infections, deaths, or lockdown duration while limiting economic loss.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec005\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>RL shows promise for adaptive epidemic control. Comparison is hampered by simplified economic costs, inconsistent calibration rigor, varied evaluation metrics, and limited uncertainty or policy robustness analysis. Future work should establish common benchmark environments and reporting standards, incorporate empirically grounded economic and behavioral models, adopt uncertainty-aware and probabilistic RL, develop more sophisticated control spaces, investigate more advanced algorithms, and validate learned policies prospectively to enable real-world deployment.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0351176","type":"journal-article","created":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T17:34:12Z","timestamp":1780940052000},"page":"e0351176","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["Reinforcement learning for policymaking in epidemic control: A scoping review"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5952-7524","authenticated-orcid":true,"given":"Oleksandr","family":"Bolshov","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2623-3294","authenticated-orcid":true,"given":"Dmytro","family":"Chumachenko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,6,8]]},"reference":[{"key":"pone.0351176.ref001","doi-asserted-by":"crossref","first-page":"100546","DOI":"10.1016\/j.epidem.2022.100546","article-title":"Challenges for modelling interventions for future pandemics","volume":"38","author":"ME Kretzschmar","year":"2022","journal-title":"Epidemics"},{"issue":"1","key":"pone.0351176.ref002","doi-asserted-by":"crossref","first-page":"3102","DOI":"10.1038\/s41467-019-11067-2","article-title":"Using \u201coutbreak science\u201d to strengthen the use of models during epidemics","volume":"10","author":"C Rivers","year":"2019","journal-title":"Nat Commun"},{"issue":"1","key":"pone.0351176.ref003","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1038\/s41467-024-55461-x","article-title":"Integrating artificial intelligence with mechanistic epidemiological modeling: a scoping review of opportunities and challenges","volume":"16","author":"Y Ye","year":"2025","journal-title":"Nat Commun"},{"issue":"2","key":"pone.0351176.ref004","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1007\/s10729-025-09699-6","article-title":"Reinforcement learning for healthcare operations management: methodological framework, recent developments, and future research directions","volume":"28","author":"Q Wu","year":"2025","journal-title":"Health Care Manag Sci"},{"key":"pone.0351176.ref005","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1146\/annurev-publhealth-040617-014317","article-title":"Agent-Based Modeling in Public Health: Current Applications and Future Directions","volume":"39","author":"M Tracy","year":"2018","journal-title":"Annu Rev Public Health"},{"issue":"7","key":"pone.0351176.ref006","doi-asserted-by":"crossref","first-page":"467","DOI":"10.7326\/M18-0850","article-title":"PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation","volume":"169","author":"AC Tricco","year":"2018","journal-title":"Ann Intern Med"},{"key":"pone.0351176.ref007","doi-asserted-by":"crossref","DOI":"10.1109\/IJCNN60899.2024.10650164","article-title":"Advancing pandemic preparedness through a data-driven hybrid simulation model","author":"SB Shuvo","year":"2024"},{"key":"pone.0351176.ref008","article-title":"A Data-Driven Pandemic Simulator with Reinforcement Learning","volume":"13","author":"Y Zhang","year":"2024","journal-title":"Electronics"},{"key":"pone.0351176.ref009","first-page":"155","article-title":"Deep Reinforcement Learning for Large-Scale Epidemic Control","author":"PJK Libin","year":"2021"},{"key":"pone.0351176.ref010","doi-asserted-by":"crossref","first-page":"107960","DOI":"10.1016\/j.cie.2022.107960","article-title":"Reinforcement learning based framework for COVID-19 resource allocation","volume":"167","author":"K Zong","year":"2022","journal-title":"Comput Ind Eng"},{"issue":"8","key":"pone.0351176.ref011","doi-asserted-by":"crossref","first-page":"1064","DOI":"10.1177\/0272989X221107902","article-title":"PaCAR: COVID-19 Pandemic Control Decision Making via Large-Scale Agent-Based Modeling and Deep Reinforcement Learning","volume":"42","author":"X Guo","year":"2022","journal-title":"Med Decis Making"},{"key":"pone.0351176.ref012","doi-asserted-by":"crossref","first-page":"126974","DOI":"10.1016\/j.neucom.2023.126974","article-title":"Decentralized graph-based multi-agent reinforcement learning using reward machines","volume":"564","author":"J Hu","year":"2024","journal-title":"Neurocomputing"},{"key":"pone.0351176.ref013","doi-asserted-by":"crossref","first-page":"119065","DOI":"10.1016\/j.ins.2023.119065","article-title":"HRL4EC: Hierarchical reinforcement learning for multi-mode epidemic control","volume":"640","author":"X Du","year":"2023","journal-title":"Inf Sci (N Y)"},{"key":"pone.0351176.ref014","unstructured":"Kompella V, Capobianco R, Jong S, Browne J, Fox S, Meyers L. Reinforcement learning for optimization of COVID-19 mitigation policies. In: CEUR WORKSHOP PROCEEDINGS, 2020."},{"key":"pone.0351176.ref015","first-page":"1634","author":"R Wan","year":"2021"},{"key":"pone.0351176.ref016","doi-asserted-by":"crossref","first-page":"22106","DOI":"10.1038\/s41598-020-79147-8","article-title":"Exploring optimal control of epidemic spread using reinforcement learning","volume":"10","author":"AQ Ohi","year":"2020","journal-title":"Sci Rep"},{"key":"pone.0351176.ref017","doi-asserted-by":"crossref","first-page":"102312","DOI":"10.1016\/j.compenvurbsys.2025.102312","article-title":"Architecting urban epidemic defense: A hierarchical region-individual control framework for optimizing large-scale individual mobility interventions","volume":"121","author":"Y Luo","year":"2025","journal-title":"Computers, Environment and Urban Systems"},{"key":"pone.0351176.ref018","first-page":"145","author":"S Samadi","year":"2025"},{"key":"pone.0351176.ref019","doi-asserted-by":"crossref","first-page":"128432","DOI":"10.1016\/j.eswa.2025.128432","article-title":"H2-MARL: Multi-agent reinforcement learning for Pareto optimality in hospital capacity strain and human mobility during epidemic","volume":"291","author":"X Luo","year":"2025","journal-title":"Expert Systems with Applications"},{"key":"pone.0351176.ref020","volume-title":"Reinforcement Learning, Second Edition: An Introduction","author":"RS Sutton","year":"2018"},{"issue":"772","key":"pone.0351176.ref021","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1098\/rspa.1927.0118","article-title":"A contribution to the mathematical theory of epidemics","volume":"115","author":"WO Kermack","year":"1927","journal-title":"Proceedings of the Royal Society of London Series A, Containing Papers of a Mathematical and Physical Character"},{"key":"pone.0351176.ref022","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1137\/S0036144500371907","article-title":"The Mathematics of Infectious Diseases","volume":"42","author":"HW Hethcote","year":"2000","journal-title":"SIAM Review"},{"issue":"7","key":"pone.0351176.ref023","article-title":"Effects of non-pharmaceutical interventions on COVID-19 cases, deaths, and demand for hospital services in the UK: a modelling study","volume":"5","author":"Centre for the Mathematical Modelling of Infectious Diseases COVID-19 working group","year":"2020","journal-title":"Lancet Public Health"},{"issue":"7540","key":"pone.0351176.ref024","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"V Mnih","year":"2015","journal-title":"Nature"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0351176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T17:34:18Z","timestamp":1780940058000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0351176"}},"subtitle":[],"editor":[{"given":"Siew Ann","family":"Cheong","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,6,8]]},"references-count":24,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,6,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0351176","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,8]]}}}