{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T19:35:09Z","timestamp":1771616109077,"version":"3.50.1"},"reference-count":45,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2017,3,10]],"date-time":"2017-03-10T00:00:00Z","timestamp":1489104000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["U54 GM088491"],"award-info":[{"award-number":["U54 GM088491"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation (US)","doi-asserted-by":"crossref","award":["DGE-1252522"],"award-info":[{"award-number":["DGE-1252522"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["U54 GM088491"],"award-info":[{"award-number":["U54 GM088491"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"DOI":"10.1371\/journal.pcbi.1005248","type":"journal-article","created":{"date-parts":[[2017,3,10]],"date-time":"2017-03-10T13:25:24Z","timestamp":1489152324000},"page":"e1005248","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":53,"title":["A human judgment approach to epidemiological forecasting"],"prefix":"10.1371","volume":"13","author":[{"given":"David C.","family":"Farrow","sequence":"first","affiliation":[]},{"given":"Logan C.","family":"Brooks","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0377-897X","authenticated-orcid":true,"given":"Sangwon","family":"Hyun","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2158-8304","authenticated-orcid":true,"given":"Ryan J.","family":"Tibshirani","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5704-8094","authenticated-orcid":true,"given":"Donald S.","family":"Burke","sequence":"additional","affiliation":[]},{"given":"Roni","family":"Rosenfeld","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2017,3,10]]},"reference":[{"issue":"9524","key":"ref1","doi-asserted-by":"crossref","first-page":"1747","DOI":"10.1016\/S0140-6736(06)68770-9","article-title":"Global and regional burden of disease and risk factors, 2001: systematic analysis of population health data","volume":"367","author":"AD Lopez","year":"2006","journal-title":"The Lancet"},{"issue":"1","key":"ref2","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1001\/jama.281.1.61","article-title":"Trends in Infectious Disease Mortality in the United States During the 20th Century","volume":"281","author":"GL Armstrong","year":"1999","journal-title":"JAMA"},{"issue":"3","key":"ref3","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1001\/jama.1996.03530270029027","article-title":"Trends in Infectious Diseases Mortality in the United States","volume":"275","author":"RW Pinner","year":"1996","journal-title":"JAMA"},{"key":"ref4","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/S0065-308X(00)47013-2","article-title":"Forecasting disease risk for increased epidemic preparedness in public health","volume":"47","author":"MF Myers","year":"2000","journal-title":"Advances in Parasitology"},{"issue":"1","key":"ref5","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1186\/s12879-016-1669-x","article-title":"Results from the centers for disease control and prevention\u2019s predict the 2013\u20132014 Influenza Season Challenge","volume":"16","author":"M Biggerstaff","year":"2016","journal-title":"BMC Infectious Diseases"},{"key":"ref6","unstructured":"CDC Competition Encourages Use of Social Media to Predict Flu; 2013. Available from: http:\/\/www.cdc.gov\/flu\/news\/predict-flu-challenge.htm<\/ext-link>."},{"key":"ref7","unstructured":"Flu Activity Forecasting Website Launched; 2016. Available from: http:\/\/www.cdc.gov\/flu\/news\/flu-forecast-website-launched.htm<\/ext-link>."},{"key":"ref8","unstructured":"DARPA Forecasting Chikungunya Challenge | InnoCentive Challenge; 2014. Available from: https:\/\/www.innocentive.com\/ar\/challenge\/9933617?cc=DARPApress&utm_source=DARPA&utm_campaign=9933617&utm_medium=press<\/ext-link>."},{"key":"ref9","unstructured":"Servick K. Chikungunya threat inspires new DARPA challenge; 2014. Available from: http:\/\/www.sciencemag.org\/news\/2014\/08\/chikungunya-threat-inspires-new-darpa-challenge<\/ext-link>."},{"issue":"1","key":"ref10","doi-asserted-by":"crossref","DOI":"10.5210\/ojphi.v7i1.5677","article-title":"Advancing Epidemic Prediction and Forecasting: A New US Government Initiative","volume":"7","author":"JP Chretien","year":"2015","journal-title":"Online Journal of Public Health Informatics"},{"key":"ref11","unstructured":"Ebola, Zika modelers aim to inform policy decisions; 2016. Available from: http:\/\/www.fic.nih.gov\/News\/GlobalHealthMatters\/march-april-2016\/Pages\/disease-modeling-informs-health-policy.aspx<\/ext-link>."},{"key":"ref12","article-title":"Fundamentals of Numerical Weather Prediction","author":"J Coiffier","year":"2012"},{"key":"ref13","article-title":"An Introduction to Econometric Forecasting and Forecasting Models (The Wharton econometric studies series)","author":"LR Klein","year":"1980"},{"issue":"4","key":"ref14","doi-asserted-by":"crossref","first-page":"e94130","DOI":"10.1371\/journal.pone.0094130","article-title":"Influenza Forecasting in Human Populations: A Scoping Review","volume":"9","author":"JP Chretien","year":"2014","journal-title":"PLoS ONE"},{"issue":"3","key":"ref15","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1111\/irv.12226","article-title":"A systematic review of studies on forecasting the dynamics of influenza outbreaks","volume":"8","author":"EO Nsoesie","year":"2013","journal-title":"Influenza Other Respi Viruses"},{"issue":"5","key":"ref16","doi-asserted-by":"crossref","first-page":"e1648","DOI":"10.1371\/journal.pntd.0001648","article-title":"Surveillance of Dengue Fever Virus: A Review of Epidemiological Models and Early Warning Systems","volume":"6","author":"V Racloz","year":"2012","journal-title":"PLoS Negl Trop Dis"},{"key":"ref17","first-page":"450","article-title":"Vox populi (The wisdom of crowds)","volume":"75","author":"F Galton","year":"1907","journal-title":"Nature"},{"issue":"6","key":"ref18","doi-asserted-by":"crossref","first-page":"1115","DOI":"10.1175\/WAF-821.1","article-title":"Weather Forecasting by Humans\u2014Heuristics and Decision Making","volume":"19","author":"CA Doswell","year":"2004","journal-title":"Wea Forecasting"},{"key":"ref19","article-title":"The wisdom of crowds","author":"J Surowiecki","year":"2005","journal-title":"Anchor"},{"issue":"2","key":"ref20","first-page":"45","article-title":"Decisions 2.0: The power of collective intelligence","volume":"50","author":"E Bonabeau","year":"2009","journal-title":"MIT Sloan management review"},{"key":"ref21","article-title":"Neural Networks: A Comprehensive Foundation","author":"S Haykin","year":"1994"},{"key":"ref22","first-page":"1","article-title":"Multiple classifier systems","author":"TG Dietterich","year":"2000"},{"key":"ref23","first-page":"9","article-title":"Using prediction markets and Twitter to predict a swine flu pandemic","author":"J Ritterman","year":"2009"},{"issue":"2","key":"ref24","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1086\/510427","article-title":"Use of Prediction Markets to Forecast Infectious Disease Activity","volume":"44","author":"PM Polgreen","year":"2007","journal-title":"Clinical Infectious Diseases"},{"issue":"10","key":"ref25","doi-asserted-by":"crossref","first-page":"2124","DOI":"10.2105\/AJPH.2015.302696","article-title":"Flu Near You: Crowdsourced Symptom Reporting Spanning 2 Influenza Seasons","volume":"105","author":"MS Smolinski","year":"2015","journal-title":"American journal of public health"},{"issue":"1","key":"ref26","doi-asserted-by":"crossref","DOI":"10.5210\/ojphi.v5i1.4456","article-title":"Flu near you: an online self-reported influenza surveillance system in the USA","volume":"5","author":"R Chunara","year":"2013","journal-title":"Online Journal of Public Health Informatics"},{"key":"ref27","unstructured":"Overview of Influenza Surveillance in the United States; 2015. Available from: http:\/\/www.cdc.gov\/flu\/weekly\/overview.htm<\/ext-link>."},{"key":"ref28","unstructured":"Influenza National and Regional Level Graphs and Data; 2015. Available from: http:\/\/gis.cdc.gov\/grasp\/fluview\/fluportaldashboard.html<\/ext-link>."},{"key":"ref29","unstructured":"Chikungunya: Statistic Data; 2016. Available from: http:\/\/www.paho.org\/hq\/index.php?option=com_topics&view=readall&cid=5927&Itemid=40931&lang=en<\/ext-link>."},{"key":"ref30","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1111\/j.2517-6161.1952.tb00104.x","article-title":"Rational decisions","author":"IJ Good","year":"1952","journal-title":"Journal of the Royal Statistical Society Series B (Methodological)"},{"key":"ref31","article-title":"Center for Advanced Engineering Study, Massachusetts Institute of Technology","author":"M Tribus","year":"1961"},{"issue":"8","key":"ref32","doi-asserted-by":"crossref","first-page":"e1004382","DOI":"10.1371\/journal.pcbi.1004382","article-title":"Flexible Modeling of Epidemics with an Empirical Bayes Framework","volume":"11","author":"LC Brooks","year":"2015","journal-title":"PLOS Computational Biology"},{"key":"ref33","unstructured":"Farrow DC. Modeling the Past, Present, and Future of Influenza [PhD Thesis]. Carnegie Mellon University; 2016. Available from: http:\/\/reports-archive.adm.cs.cmu.edu\/anon\/cbd\/CMU-CB-16-101.pdf<\/ext-link>."},{"key":"ref34","unstructured":"Delphi Epicast\u2014Influenza; 2015. Available from: http:\/\/epicast.org\/<\/ext-link>."},{"key":"ref35","unstructured":"MMWR Week Fact Sheet; 2015. Available from: http:\/\/wwwn.cdc.gov\/nndss\/document\/MMWR_Week_overview.pdf<\/ext-link>."},{"key":"ref36","unstructured":"New Flu Activity Forecasts Available for 2016-17 Season; CDC Names Most Accurate Forecaster for 2015-16; 2016. Available from: https:\/\/www.cdc.gov\/flu\/spotlights\/flu-activity-forecasts-2016-2017.htm<\/ext-link>."},{"key":"ref37","unstructured":"CHIKV Challenge Announces Winners, Progress toward Forecasting the Spread of Infectious Diseases; 2015. Available from: http:\/\/www.darpa.mil\/news-events\/2015-05-27<\/ext-link>."},{"issue":"3","key":"ref38","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1287\/mnsc.9.3.458","article-title":"An experimental application of the Delphi method to the use of experts","volume":"9","author":"N Dalkey","year":"1963","journal-title":"Management science"},{"key":"ref39","first-page":"97","article-title":"User Modeling 2007","author":"P Symeonidis","year":"2007"},{"key":"ref40","first-page":"612","article-title":"SDM","volume":"vol. 12","author":"RH Li","year":"2012"},{"key":"ref41","doi-asserted-by":"crossref","unstructured":"Kittur A, Nickerson JV, Bernstein M, Gerber E, Shaw A, Zimmerman J, et al. The future of crowd work. In: Proceedings of the 2013 conference on Computer supported cooperative work. ACM; 2013. p. 1301\u20131318.","DOI":"10.1145\/2441776.2441923"},{"issue":"6268","key":"ref42","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1126\/science.aad6499","article-title":"The power of crowds","volume":"351","author":"P Michelucci","year":"2015","journal-title":"Science"},{"issue":"1","key":"ref43","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1037\/xge0000033","article-title":"Algorithm aversion: People erroneously avoid algorithms after seeing them err","volume":"144","author":"BJ Dietvorst","year":"2015","journal-title":"Journal of Experimental Psychology: General"},{"issue":"1","key":"ref44","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1037\/1040-3590.12.1.19","article-title":"Clinical versus mechanical prediction: A meta-analysis","volume":"12","author":"WM Grove","year":"2000","journal-title":"Psychological Assessment"},{"key":"ref45","article-title":"Thinking, fast and slow","author":"D Kahneman","year":"2011"}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/dx.plos.org\/10.1371\/journal.pcbi.1005248","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,9,19]],"date-time":"2019-09-19T12:00:47Z","timestamp":1568894447000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1005248"}},"subtitle":[],"editor":[{"given":"Samuel","family":"Alizon","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2017,3,10]]},"references-count":45,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2017,3,10]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1005248","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,3,10]]}}}