{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T11:12:58Z","timestamp":1774955578199,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1012782","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T00:00:00Z","timestamp":1739836800000}}],"reference-count":60,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2025,2,13]],"date-time":"2025-02-13T00:00:00Z","timestamp":1739404800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"One Health Modelling Network for Emerging Infections"},{"DOI":"10.13039\/100013373","name":"Alberta Machine Intelligence Institute","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100013373","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000190","name":"University of Alberta","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000190","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["Individual Discovery Grant RGPIN-2020-03911 and Discovery Accelerator Supplement Award RGPAS-2020-00090"],"award-info":[{"award-number":["Individual Discovery Grant RGPIN-2020-03911 and Discovery Accelerator Supplement Award RGPAS-2020-00090"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001804","name":"Canada Research Chairs program","doi-asserted-by":"crossref","award":["Tier 1 Canada Research Chair Award"],"award-info":[{"award-number":["Tier 1 Canada Research Chair Award"]}],"id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000038","name":"a Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","award":["Discovery Grant"],"award-info":[{"award-number":["Discovery Grant"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Gilbert and Betty Kennedy Chair"},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RES0043537"],"award-info":[{"award-number":["RES0043537"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100013373","name":"Alberta Machine Intelligence Institute","doi-asserted-by":"publisher","award":["RES0054194"],"award-info":[{"award-number":["RES0054194"]}],"id":[{"id":"10.13039\/100013373","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007631","name":"Canadian Institute for Advanced Research","doi-asserted-by":"publisher","award":["ZAEHU"],"award-info":[{"award-number":["ZAEHU"]}],"id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Forecasting the occurrence and absence of novel disease outbreaks is essential for disease management, yet existing methods are often context-specific, require a long preparation time, and non-outbreak prediction remains understudied. To address this gap, we propose a novel framework using a feature-based time series classification (TSC) method to forecast outbreaks and non-outbreaks. We tested our methods on synthetic data from a Susceptible\u2013Infected\u2013Recovered (SIR) model for slowly changing, noisy disease dynamics. Outbreak sequences give a transcritical bifurcation within a specified future time window, whereas non-outbreak (null bifurcation) sequences do not. We identified incipient differences, reflected in 22 statistical features and 5 early warning signal indicators, in time series of infectives leading to future outbreaks and non-outbreaks. Classifier performance, given by the area under the receiver-operating curve (AUC), ranged from 0\u00a0.\u00a099 for large expanding windows of training data to 0\u00a0.\u00a07 for small rolling windows. The framework is further evaluated on four empirical datasets: COVID-19 incidence data from Singapore, 18 other countries, and Edmonton, Canada, as well as SARS data from Hong Kong, with two classifiers exhibiting consistently high accuracy. Our results highlight detectable statistical features distinguishing outbreak and non-outbreak sequences well before potential occurrence, in both synthetic and real-world datasets presented in this study.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1012782","type":"journal-article","created":{"date-parts":[[2025,2,13]],"date-time":"2025-02-13T18:50:37Z","timestamp":1739472637000},"page":"e1012782","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":3,"title":["Early detection of disease outbreaks and non-outbreaks using incidence data: A framework using feature-based time series classification and machine learning"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1705-8033","authenticated-orcid":true,"given":"Shan","family":"Gao","sequence":"first","affiliation":[]},{"given":"Amit K","family":"Chakraborty","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8327-934X","authenticated-orcid":true,"given":"Russell","family":"Greiner","sequence":"additional","affiliation":[]},{"given":"Mark A","family":"Lewis","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4132-6109","authenticated-orcid":true,"given":"Hao","family":"Wang","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2025,2,13]]},"reference":[{"issue":"18","key":"pcbi.1012782.ref001","doi-asserted-by":"crossref","first-page":"1681","DOI":"10.1056\/NEJMp2004361","article-title":"History in a crisis - lessons for COVID-19","volume":"382","author":"DS Jones","year":"2020","journal-title":"N Engl J Med"},{"key":"pcbi.1012782.ref002"},{"key":"pcbi.1012782.ref003","doi-asserted-by":"crossref","first-page":"101019","DOI":"10.1016\/j.jth.2021.101019","article-title":"Limiting mobility during COVID-19, when and to what level? An international comparative study using change point analysis","volume":"20","author":"SA Gargoum","year":"2021","journal-title":"J Transp Health"},{"key":"pcbi.1012782.ref004","doi-asserted-by":"crossref","first-page":"104773","DOI":"10.1016\/j.ssci.2020.104773","article-title":"Initial impacts of global risk mitigation measures taken during the combatting of the COVID-19 pandemic","volume":"128","author":"Y Bruinen de Bruin","year":"2020","journal-title":"Saf Sci"},{"issue":"6491","key":"pcbi.1012782.ref005","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1126\/science.abb6105","article-title":"An investigation of transmission control measures during the first 50 days of the COVID-19 epidemic in China","volume":"368","author":"H Tian","year":"2020","journal-title":"Science"},{"issue":"1","key":"pcbi.1012782.ref006","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1186\/s12879-023-08948-9","article-title":"The effectiveness and efficiency of asymptomatic SARS-CoV-2 testing strategies for patient and healthcare workers within acute NHS hospitals during an omicron-like period","volume":"24","author":"S Evans","year":"2024","journal-title":"BMC Infect Dis"},{"issue":"5628","key":"pcbi.1012782.ref007","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1038\/269471a0","article-title":"Thresholds and breakpoints in ecosystems with a multiplicity of stable states","volume":"269","author":"RM May","year":"1977","journal-title":"Nature"},{"key":"pcbi.1012782.ref008"},{"issue":"6856","key":"pcbi.1012782.ref009","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1038\/35098000","article-title":"Catastrophic shifts in ecosystems","volume":"413","author":"M Scheffer","year":"2001","journal-title":"Nature"},{"issue":"7159","key":"pcbi.1012782.ref010","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1038\/nature06111","article-title":"Spatial vegetation patterns and imminent desertification in Mediterranean arid ecosystems","volume":"449","author":"S K\u00e9fi","year":"2007","journal-title":"Nature"},{"issue":"5","key":"pcbi.1012782.ref011","doi-asserted-by":"crossref","first-page":"e1006917","DOI":"10.1371\/journal.pcbi.1006917","article-title":"The statistics of epidemic transitions","volume":"15","author":"JM Drake","year":"2019","journal-title":"PLoS Comput Biol"},{"issue":"3","key":"pcbi.1012782.ref012","doi-asserted-by":"crossref","first-page":"e1009958","DOI":"10.1371\/journal.pcbi.1009958","article-title":"Performance of early warning signals for disease re-emergence: A case study on COVID-19 data","volume":"18","author":"D Proverbio","year":"2022","journal-title":"PLoS Comput Biol"},{"issue":"4","key":"pcbi.1012782.ref013","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1098\/rsif.2005.0042","article-title":"Perspectives on the basic reproductive ratio","volume":"2","author":"JM Heffernan","year":"2005","journal-title":"J R Soc Interface"},{"issue":"6347","key":"pcbi.1012782.ref014","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1126\/science.aam8335","article-title":"Opportunities and challenges in modeling emerging infectious diseases","volume":"357","author":"CJE Metcalf","year":"2017","journal-title":"Science"},{"issue":"6","key":"pcbi.1012782.ref015","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1086\/516845","article-title":"Slow recovery from perturbations as a generic indicator of a nearby catastrophic shift","volume":"169","author":"EH van Nes","year":"2007","journal-title":"Am Nat"},{"issue":"3","key":"pcbi.1012782.ref016","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1111\/j.1461-0248.2005.00877.x","article-title":"Rising variance: a leading indicator of ecological transition","volume":"9","author":"SR Carpenter","year":"2006","journal-title":"Ecol Lett"},{"issue":"3","key":"pcbi.1012782.ref017","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1007\/s12080-009-0060-6","article-title":"Spatial correlation as leading indicator of catastrophic shifts","volume":"3","author":"V Dakos","year":"2009","journal-title":"Theor Ecol"},{"issue":"5","key":"pcbi.1012782.ref018","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1111\/j.1461-0248.2008.01160.x","article-title":"Changing skewness: an early warning signal of regime shifts in ecosystems","volume":"11","author":"V Guttal","year":"2008","journal-title":"Ecol Lett"},{"issue":"3","key":"pcbi.1012782.ref019","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1073\/pnas.0811729106","article-title":"Turning back from the brink: detecting an impending regime shift in time to avert it","volume":"106","author":"R Biggs","year":"2009","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"3","key":"pcbi.1012782.ref020","doi-asserted-by":"crossref","first-page":"20190713","DOI":"10.1098\/rsbl.2019.0713","article-title":"Early warning signals of malaria resurgence in Kericho, Kenya","volume":"16","author":"MJ Harris","year":"2020","journal-title":"Biol Lett"},{"issue":"12","key":"pcbi.1012782.ref021","doi-asserted-by":"crossref","first-page":"20210487","DOI":"10.1098\/rsbl.2021.0487","article-title":"Early warning signal reliability varies with COVID-19 waves","volume":"17","author":"DA O\u2019Brien","year":"2021","journal-title":"Biol Lett"},{"issue":"193","key":"pcbi.1012782.ref022","doi-asserted-by":"crossref","first-page":"20220123","DOI":"10.1098\/rsif.2022.0123","article-title":"Anticipating infectious disease re-emergence and elimination: a test of early warning signals using empirically based models","volume":"19","author":"AT Tredennick","year":"2022","journal-title":"J R Soc Interface"},{"issue":"2","key":"pcbi.1012782.ref023","doi-asserted-by":"crossref","first-page":"e1002360","DOI":"10.1371\/journal.pcbi.1002360","article-title":"Early warning signals for critical transitions: a generalized modeling approach","volume":"8","author":"SJ Lade","year":"2012","journal-title":"PLoS Comput Biol"},{"issue":"1659","key":"pcbi.1012782.ref024","doi-asserted-by":"crossref","first-page":"20130263","DOI":"10.1098\/rstb.2013.0263","article-title":"Resilience indicators: prospects and limitations for early warnings of regime shifts","volume":"370","author":"V Dakos","year":"2015","journal-title":"Phil Trans R Soc B"},{"issue":"4","key":"pcbi.1012782.ref025","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1111\/j.1461-0248.2010.01439.x","article-title":"Regime shifts in ecological systems can occur with no warning","volume":"13","author":"A Hastings","year":"2010","journal-title":"Ecol Lett"},{"issue":"39","key":"pcbi.1012782.ref026","doi-asserted-by":"crossref","first-page":"e2106140118","DOI":"10.1073\/pnas.2106140118","article-title":"Deep learning for early warning signals of tipping points","volume":"118","author":"TM Bury","year":"2021","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"2","key":"pcbi.1012782.ref027","doi-asserted-by":"crossref","first-page":"211475","DOI":"10.1098\/rsos.211475","article-title":"Machine learning methods trained on simple models can predict critical transitions in complex natural systems","volume":"9","author":"S Deb","year":"2022","journal-title":"Royal Society Open Science."},{"issue":"1","key":"pcbi.1012782.ref028","article-title":"Machine learning prediction of critical transition and system collapse","volume":"3","author":"L-W Kong","year":"2021","journal-title":"Phys Rev Research"},{"issue":"8","key":"pcbi.1012782.ref029","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1038\/s41558-021-01097-4","article-title":"Observation-based early-warning signals for a collapse of the Atlantic Meridional Overturning Circulation","volume":"11","author":"N Boers","year":"2021","journal-title":"Nat Clim Chang"},{"key":"pcbi.1012782.ref030"},{"issue":"772","key":"pcbi.1012782.ref031","first-page":"700","article-title":"A contribution to the mathematical theory of epidemics","volume":"115","author":"WO Kermack","year":"1927","journal-title":"Proceedings of the Royal Society of London Series A, Containing Papers of a Mathematical and Physical Character."},{"issue":"217","key":"pcbi.1012782.ref032","doi-asserted-by":"crossref","first-page":"20240199","DOI":"10.1098\/rsif.2024.0199","article-title":"An early warning indicator trained on stochastic disease-spreading models with different noises","volume":"21","author":"AK Chakraborty","year":"2024","journal-title":"J R Soc Interface"},{"issue":"6","key":"pcbi.1012782.ref033","doi-asserted-by":"crossref","first-page":"1821","DOI":"10.1007\/s10618-019-00647-x","article-title":"catch22: CAnonical Time-series CHaracteristics","volume":"33","author":"CH Lubba","year":"2019","journal-title":"Data Min Knowl Disc"},{"issue":"3","key":"pcbi.1012782.ref034","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach","volume":"44","author":"ER DeLong","year":"1988","journal-title":"Biometrics"},{"issue":"2","key":"pcbi.1012782.ref035","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1177\/003335490912400205","article-title":"Does influenza transmission occur from asymptomatic infection or prior to symptom onset?","volume":"124","author":"E Patrozou","year":"2009","journal-title":"Public Health Rep"},{"issue":"1968","key":"pcbi.1012782.ref036","first-page":"20211809","article-title":"Overlapping timescales obscure early warning signals of the second COVID-19 wave","volume":"289","author":"F Dablander","year":"2022","journal-title":"Proc Biol Sci"},{"issue":"3","key":"pcbi.1012782.ref037","first-page":"49","article-title":"Feature-based classification of time-series data","volume":"10","author":"A Nanopoulos","year":"2001","journal-title":"Int J Comput Res."},{"issue":"10","key":"pcbi.1012782.ref038","doi-asserted-by":"crossref","first-page":"e0002253","DOI":"10.1371\/journal.pgph.0002253","article-title":"The potential of resilience indicators to anticipate infectious disease outbreaks, a systematic review and guide","volume":"3","author":"C Delecroix","year":"2023","journal-title":"PLOS Glob Public Health"},{"issue":"182","key":"pcbi.1012782.ref039","doi-asserted-by":"crossref","first-page":"20210555","DOI":"10.1098\/rsif.2021.0555","article-title":"of Early warning signals infectious disease transitions: a review","volume":"18","author":"E Southall","year":"2021","journal-title":"J R Soc Interface"},{"issue":"50","key":"pcbi.1012782.ref040","doi-asserted-by":"crossref","first-page":"E8089","DOI":"10.1073\/pnas.1608242113","article-title":"Evaluating early-warning indicators of critical transitions in natural aquatic ecosystems","volume":"113","author":"AS Gsell","year":"2016","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"9","key":"pcbi.1012782.ref041","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1016\/S1471-4922(01)02077-3","article-title":"The development of Malaria Early Warning Systems for Africa","volume":"17","author":"MC Thomson","year":"2001","journal-title":"Trends Parasitol"},{"issue":"10","key":"pcbi.1012782.ref042","doi-asserted-by":"crossref","first-page":"4569","DOI":"10.1109\/TNNLS.2017.2764939","article-title":"Early Classification of Time Series by Simultaneously Optimizing the Accuracy and Earliness","volume":"29","author":"U Mori","year":"2018","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"pcbi.1012782.ref043","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.inffus.2021.05.008","article-title":"A review of uncertainty quantification in deep learning: Techniques, applications and challenges","volume":"76","author":"M Abdar","year":"2021","journal-title":"Information Fusion"},{"key":"pcbi.1012782.ref044"},{"key":"pcbi.1012782.ref045"},{"key":"pcbi.1012782.ref046","first-page":"5","article-title":"Misc functions of the Department of Statistics (e1071), TU Wien","volume":"1","author":"E Dimitriadou","year":"2008","journal-title":"R Package."},{"key":"pcbi.1012782.ref047"},{"key":"pcbi.1012782.ref048"},{"issue":"5934","key":"pcbi.1012782.ref049","doi-asserted-by":"crossref","first-page":"1557","DOI":"10.1126\/science.1176062","article-title":"Pandemic potential of a strain of influenza A (H1N1): early findings","volume":"324","author":"C Fraser","year":"2009","journal-title":"Science"},{"issue":"9393","key":"pcbi.1012782.ref050","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1016\/S0140-6736(03)14630-2","article-title":"Epidemiology and cause of severe acute respiratory syndrome (SARS) in Guangdong, People\u2019s Republic of China, in February, 2003","volume":"362","author":"NS Zhong","year":"2003","journal-title":"Lancet"},{"key":"pcbi.1012782.ref051"},{"key":"pcbi.1012782.ref052"},{"issue":"8","key":"pcbi.1012782.ref053","doi-asserted-by":"crossref","first-page":"e1011439","DOI":"10.1371\/journal.pcbi.1011439","article-title":"Estimating the epidemic reproduction number from temporally aggregated incidence data: A statistical modelling approach and software tool","volume":"19","author":"RK Nash","year":"2023","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1012782.ref054"},{"key":"pcbi.1012782.ref055","doi-asserted-by":"crossref","first-page":"e71345","DOI":"10.7554\/eLife.71345","article-title":"Estimation and worldwide monitoring of the effective reproductive number of SARS-CoV-2","volume":"11","author":"JS Huisman","year":"2022","journal-title":"Elife"},{"key":"pcbi.1012782.ref056"},{"issue":"5627","key":"pcbi.1012782.ref057","doi-asserted-by":"crossref","first-page":"1966","DOI":"10.1126\/science.1086616","article-title":"Transmission dynamics and control of severe acute respiratory syndrome","volume":"300","author":"M Lipsitch","year":"2003","journal-title":"Science"},{"issue":"11","key":"pcbi.1012782.ref058","doi-asserted-by":"crossref","first-page":"1857","DOI":"10.1016\/j.patcog.2005.01.025","article-title":"Clustering of time series data\u2014a survey","volume":"38","author":"T Warren Liao","year":"2005","journal-title":"Pattern Recognition"},{"key":"pcbi.1012782.ref059"},{"issue":"1","key":"pcbi.1012782.ref060","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/TAI.2020.3027279","article-title":"Approaches and applications of early classification of time series: a review","volume":"1","author":"A Gupta","year":"2020","journal-title":"IEEE Trans Artif Intell"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1012782","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T00:00:00Z","timestamp":1739836800000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1012782","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T18:55:41Z","timestamp":1739904941000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1012782"}},"subtitle":[],"editor":[{"given":"Ruian","family":"Ke","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2025,2,13]]},"references-count":60,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2,13]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1012782","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1012782","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,13]]}}}