{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T05:41:32Z","timestamp":1791265292926,"version":"4.1.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1014175","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T00:00:00Z","timestamp":1776816000000}}],"reference-count":43,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T00:00:00Z","timestamp":1776211200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100013154","name":"Center for Surveillance, Epidemiology, and Laboratory Services","doi-asserted-by":"publisher","award":["Cooperative Agreement Number 6 NU50CK000539"],"award-info":[{"award-number":["Cooperative Agreement Number 6 NU50CK000539"]}],"id":[{"id":"10.13039\/100013154","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>For infectious disease forecasting challenges, individual model performance typically varies across space and time. This phenomenon raises the question: are there properties of the target time series that contribute to a particular season, location, or disease being more difficult to forecast? Here we characterize a time series\u2019 future predictability using a forecastability metric that calculates the spectral entropy of the time series. Forecastability of syndromic influenza hospital admissions for the state of California varied widely across seasons and was positively correlated with peak burden. Next, using archived U.S. state and national forecasts targeting laboratory-confirmed COVID-19 and influenza hospital admissions, we investigated the relationship between forecastability and: (i) population size of the forecasting target, and (ii) forecast performance as measured by mean absolute error, weighted interval score (WIS), and scaled relative WIS. Forecastability increased with increasing population size of the forecasting target, and forecasting performance generally improved with higher forecastability when mitigating the effects of population size across scales. These preliminary results support the idea that some targets and respiratory virus seasons may be inherently more difficult to forecast and could help explain inter-seasonal variation in model performance.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1014175","type":"journal-article","created":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T17:56:45Z","timestamp":1776275805000},"page":"e1014175","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["Forecastability of infectious disease time series: are some seasons and pathogens intrinsically more difficult to forecast?"],"prefix":"10.1371","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3852-5374","authenticated-orcid":true,"given":"Lauren A.","family":"White","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7633-677X","authenticated-orcid":true,"given":"Tom\u00e1s M.","family":"Le\u00f3n","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,4,15]]},"reference":[{"issue":"1","key":"pcbi.1014175.ref001","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1186\/s12889-019-7966-8","article-title":"Applying infectious disease forecasting to public health: a path forward using influenza forecasting examples","volume":"19","author":"CS Lutz","year":"2019","journal-title":"BMC Public Health"},{"key":"pcbi.1014175.ref002","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1186\/s12879-016-1669-x","article-title":"Results from the Centers for Disease Control and Prevention\u2019s Predict the 2013-2014 Influenza Season Challenge","volume":"16","author":"M Biggerstaff","year":"2016","journal-title":"BMC Infect Dis"},{"issue":"15","key":"pcbi.1014175.ref003","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.2113561119","article-title":"Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States","volume":"119","author":"EY Cramer","year":"2022","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"48","key":"pcbi.1014175.ref004","doi-asserted-by":"crossref","first-page":"24268","DOI":"10.1073\/pnas.1909865116","article-title":"An open challenge to advance probabilistic forecasting for dengue epidemics","volume":"116","author":"MA Johansson","year":"2019","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1014175.ref005","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.epidem.2017.08.002","article-title":"The RAPIDD ebola forecasting challenge: Synthesis and lessons learnt","volume":"22","author":"C Viboud","year":"2018","journal-title":"Epidemics"},{"issue":"1","key":"pcbi.1014175.ref006","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/s13071-022-05630-y","article-title":"Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction","volume":"16","author":"KM Holcomb","year":"2023","journal-title":"Parasit Vectors"},{"issue":"1","key":"pcbi.1014175.ref007","doi-asserted-by":"crossref","first-page":"6289","DOI":"10.1038\/s41467-024-50601-9","article-title":"Title evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations","volume":"15","author":"SM Mathis","year":"2024","journal-title":"Nat Commun"},{"issue":"5","key":"pcbi.1014175.ref008","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1011200","article-title":"Challenges of COVID-19 case forecasting in the US, 2020\u20132021","volume":"20","author":"VK Lopez","year":"2024","journal-title":"PLOS Comput Biol"},{"issue":"11","key":"pcbi.1014175.ref009","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1007486","article-title":"Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.","volume":"15","author":"NG Reich","year":"2019","journal-title":"PLOS Comput Biol"},{"issue":"5","key":"pcbi.1014175.ref010","doi-asserted-by":"crossref","first-page":"052910","DOI":"10.1103\/PhysRevE.90.052910","article-title":"Model-free quantification of time-series predictability","volume":"90","author":"J Garland","year":"2014","journal-title":"Phys Rev E"},{"issue":"50","key":"pcbi.1014175.ref011","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1098\/rsif.2010.0142","article-title":"Modelling the influence of human behaviour on the spread of infectious diseases: a review","volume":"7","author":"S Funk","year":"2010","journal-title":"J R Soc Interface"},{"issue":"7","key":"pcbi.1014175.ref012","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1038\/s41562-021-01136-2","article-title":"A review and agenda for integrated disease models including social and behavioural factors","volume":"5","author":"J Bedson","year":"2021","journal-title":"Nat Hum Behav"},{"issue":"1","key":"pcbi.1014175.ref013","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1038\/s41467-019-08616-0","article-title":"On the predictability of infectious disease outbreaks","volume":"10","author":"SV Scarpino","year":"2019","journal-title":"Nat Commun"},{"issue":"17","key":"pcbi.1014175.ref014","doi-asserted-by":"crossref","first-page":"174102","DOI":"10.1103\/PhysRevLett.88.174102","article-title":"Permutation entropy: a natural complexity measure for time series","volume":"88","author":"C Bandt","year":"2002","journal-title":"Phys Rev Lett"},{"issue":"1","key":"pcbi.1014175.ref015","doi-asserted-by":"crossref","first-page":"89","DOI":"10.3390\/e22010089","article-title":"Time series complexities and their relationship to forecasting performance","volume":"22","author":"M Ponce-Flores","year":"2020","journal-title":"Entropy"},{"key":"pcbi.1014175.ref016","doi-asserted-by":"crossref","unstructured":"Kang Y, Hyndman RJ, Smith-Miles K. Visualising forecasting algorithm performance using time series instance spaces. 2016.","DOI":"10.1016\/j.ijforecast.2016.09.004"},{"issue":"3","key":"pcbi.1014175.ref017","doi-asserted-by":"crossref","first-page":"101349","DOI":"10.1016\/j.gsf.2022.101349","article-title":"Massive feature extraction for explaining and foretelling hydroclimatic time series forecastability at the global scale","volume":"13","author":"G Papacharalampous","year":"2022","journal-title":"Geosci Front"},{"issue":"1","key":"pcbi.1014175.ref018","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1109\/TKDE.2024.3484454","article-title":"Exploring progress in multivariate time series forecasting: Comprehensivebenchmarking and heterogeneity analysis","volume":"37","author":"Z Shao","year":"2025","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"3","key":"pcbi.1014175.ref019","doi-asserted-by":"crossref","first-page":"134","DOI":"10.3390\/e19030134","article-title":"Permutation entropy: new ideas and challenges","volume":"19","author":"K Keller","year":"2017","journal-title":"Entropy"},{"key":"pcbi.1014175.ref020","unstructured":"Mills C, Irons NJ, Tsui JLH, Sparrow S, Carvalho LM, Kucharski AJ, et al. From metric to action: An evaluation framework to translate infectious disease forecasts into policy decisions [Internet]. medRxiv. 2025 [cited 2025 Aug 7]. p. 2025.07.20.25331802. Available from: https:\/\/www.medrxiv.org\/content\/10.1101\/2025.07.20.25331802v1"},{"key":"pcbi.1014175.ref021","volume-title":"Forecastable Component Analysis. In: Proceedings of the 30th International Conference on Machine Learning. Atlanta, Georgia","author":"GM Goerg","year":"2013"},{"key":"pcbi.1014175.ref022","doi-asserted-by":"crossref","first-page":"437","DOI":"10.3389\/fnhum.2017.00437","article-title":"Spectral entropy can predict changes of working memory performance reduced by short-time training in the delayed-match-to-sample task","volume":"11","author":"Y Tian","year":"2017","journal-title":"Front Hum Neurosci"},{"issue":"3","key":"pcbi.1014175.ref023","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1016\/0013-4694(91)90138-T","article-title":"Quantification of EEG irregularity by use of the entropy of the power spectrum","volume":"79","author":"T Inouye","year":"1991","journal-title":"Electroencephalogr Clin Neurophysiol"},{"key":"pcbi.1014175.ref024","unstructured":"U.S. Department of Health & Human Services. COVID-19 Reported Patient Impact and Hospital Capacity by State Timeseries (RAW) [Internet]. 2024. Available from: https:\/\/healthdata.gov\/Hospital\/COVID-19-Reported-Patient-Impact-and-Hospital-Capa\/g62h-syeh\/"},{"issue":"1","key":"pcbi.1014175.ref025","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1038\/s41597-022-01517-w","article-title":"The United States COVID-19 Forecast Hub dataset","volume":"9","author":"EY Cramer","year":"2022","journal-title":"Sci Data"},{"issue":"1","key":"pcbi.1014175.ref026","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1186\/s12889-023-15649-0","article-title":"Assessing the accuracy of California county level COVID-19 hospitalization forecasts to inform public policy decision making","volume":"23","author":"LA White","year":"2023","journal-title":"BMC Public Health"},{"issue":"51","key":"pcbi.1014175.ref027","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.2111452118","article-title":"An open repository of real-time COVID-19 indicators","volume":"118","author":"A Reinhart","year":"2021","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1014175.ref028","unstructured":"U.S. Census Bureau. Population Estimate, July 1, 2021 [Internet]. [cited 2024 Dec 9]. Available from: https:\/\/api.census.gov\/data\/2021\/pep\/population\/variables.html"},{"key":"pcbi.1014175.ref029","unstructured":"State of California Department of Finance. California 2020 Population by Age [Internet]. [cited 2024 Dec 9]. Available from: https:\/\/dof.ca.gov\/forecasting\/demographics\/"},{"key":"pcbi.1014175.ref030","unstructured":"California Department of Public Health (CDPH) Influenza Surveillance Program. Historic Influenza and Other Respiratory Disease Surveillance Reports [Internet]. [cited 2025 Feb 7]. Available from: https:\/\/www.cdph.ca.gov\/Programs\/CID\/DCDC\/pages\/immunization\/flu-reports.aspx"},{"issue":"2","key":"pcbi.1014175.ref031","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1008618","article-title":"Evaluating epidemic forecasts in an interval format","volume":"17","author":"J Bracher","year":"2021","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1014175.ref032","author":"NI Bosse","year":"2022"},{"issue":"8","key":"pcbi.1014175.ref033","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1011393","article-title":"Scoring epidemiological forecasts on transformed scales","volume":"19","author":"NI Bosse","year":"2023","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1014175.ref034","author":"R Core Team","year":"2021"},{"key":"pcbi.1014175.ref035","doi-asserted-by":"crossref","DOI":"10.2196\/54340","article-title":"Responding to the Return of Influenza in the United States by Applying Centers for Disease Control and Prevention Surveillance, Analysis, and Modeling to Inform Understanding of Seasonal Influenza","volume":"10","author":"RK Borchering","year":"2024","journal-title":"JMIR Public Health Surveill"},{"key":"pcbi.1014175.ref036","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/978-981-96-1078-5_5","article-title":"Aliasing and Leakage.","author":"D. Sundararajan","year":"2025","journal-title":"Fourier Analysis\u2014A Signal Processing Approach [Internet]"},{"issue":"1","key":"pcbi.1014175.ref037","doi-asserted-by":"crossref","first-page":"7260","DOI":"10.1038\/s41467-023-42680-x","article-title":"Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty","volume":"14","author":"E Howerton","year":"2023","journal-title":"Nat Commun"},{"key":"pcbi.1014175.ref038","author":"A Srivastava","year":"2023"},{"issue":"10","key":"pcbi.1014175.ref039","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1093\/aje\/kwg239","article-title":"Prediction of the spread of influenza epidemics by the method of analogues","volume":"158","author":"C Viboud","year":"2003","journal-title":"Am J Epidemiol"},{"issue":"19","key":"pcbi.1014175.ref040","doi-asserted-by":"crossref","first-page":"529","DOI":"10.15585\/mmwr.mm7219e2","article-title":"Correlations and timeliness of COVID-19 surveillance data sources and indicators \u2015 United States, October 1, 2020\u2013March 22, 2023","volume":"72","author":"HM Scobie","year":"2023","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"pcbi.1014175.ref041","article-title":"Evaluation of a novel data source for national influenza surveillance: Influenza hospitalization data in the National Healthcare Safety Network, United States, September 2021\u2013April 2024","author":"AM Frutos","year":"2025","journal-title":"Am J Public Health"},{"key":"pcbi.1014175.ref042","doi-asserted-by":"crossref","unstructured":"White LA, Sun M, Burnor E, Murray EL, Hoover C, Penton C, et al. A new surveillance landscape for seasonal influenza?: Comparing lab-confirmed influenza hospitalizations with other syndromic and surveillance data sources for the state of California [Internet]. 2025 [cited 2025 Dec 16]. Available from: http:\/\/medrxiv.org\/lookup\/doi\/10.1101\/2025.10.28.25338991","DOI":"10.1101\/2025.10.28.25338991"},{"key":"pcbi.1014175.ref043","doi-asserted-by":"crossref","unstructured":"Stapper M, Funk S. Mind the Baseline: The Hidden Impact of Reference Model Selection on Forecast Assessment [Internet]. 2025 [cited 2025 Aug 26]. Available from: http:\/\/medrxiv.org\/lookup\/doi\/10.1101\/2025.08.01.25332807","DOI":"10.1101\/2025.08.01.25332807"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1014175","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T00:00:00Z","timestamp":1776816000000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1014175","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T17:50:21Z","timestamp":1776880221000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1014175"}},"subtitle":[],"editor":[{"given":"Samuel V.","family":"Scarpino","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,4,15]]},"references-count":43,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,15]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1014175","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,15]]}}}