{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T17:26:48Z","timestamp":1776274008878,"version":"3.50.1"},"reference-count":33,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T00:00:00Z","timestamp":1678924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/publicdomain\/zero\/1.0\/"}],"funder":[{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["U24GM132013"],"award-info":[{"award-number":["U24GM132013"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Computational models of infectious diseases have become valuable tools for research and the public health response against epidemic threats. The reproducibility of computational models has been limited, undermining the scientific process and possibly trust in modeling results and related response strategies, such as vaccination. We translated published reproducibility guidelines from a wide range of scientific disciplines into an implementation framework for improving reproducibility of infectious disease computational models. The framework comprises 22 elements that should be described, grouped into 6 categories: computational environment, analytical software, model description, model implementation, data, and experimental protocol. The framework can be used by scientific communities to develop actionable tools for sharing computational models in a reproducible way.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010856","type":"journal-article","created":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T17:24:03Z","timestamp":1678987443000},"page":"e1010856","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":18,"title":["An implementation framework to improve the transparency and reproducibility of computational models of infectious diseases"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4699-8411","authenticated-orcid":true,"given":"Darya","family":"Pokutnaya","sequence":"first","affiliation":[]},{"given":"Bruce","family":"Childers","sequence":"additional","affiliation":[]},{"given":"Alice E.","family":"Arcury-Quandt","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8793-9982","authenticated-orcid":true,"given":"Harry","family":"Hochheiser","sequence":"additional","affiliation":[]},{"given":"Willem G.","family":"Van Panhuis","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2023,3,16]]},"reference":[{"key":"pcbi.1010856.ref001","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1016\/S0140-6736(20)30260-9","article-title":"Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in Wuhan, China: a modelling study","volume":"395","author":"JT Wu","year":"2020","journal-title":"Lancet"},{"key":"pcbi.1010856.ref002","doi-asserted-by":"crossref","first-page":"1893","DOI":"10.1001\/jama.2020.6585","article-title":"Predictive Mathematical Models of the COVID-19 Pandemic: Underlying Principles and Value of Projections","volume":"323","author":"NP Jewell","year":"2020","journal-title":"JAMA"},{"key":"pcbi.1010856.ref003","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.epidem.2018.05.007","article-title":"Modelling the global spread of diseases: A review of current practice and capability","volume":"25","author":"CE Walters","year":"2018","journal-title":"Epidemics"},{"key":"pcbi.1010856.ref004","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1016\/S1473-3099(21)00143-2","article-title":"Vaccination and non-pharmaceutical interventions for COVID-19: a mathematical modelling study","volume":"21","author":"S Moore","year":"2021","journal-title":"Lancet Infect Dis"},{"issue":"6060","key":"pcbi.1010856.ref005","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1126\/science.1213847","article-title":"Reproducible research in computational science","volume":"334","author":"RD Peng","year":"2011","journal-title":"Science"},{"key":"pcbi.1010856.ref006","unstructured":"National Academies of Sciences and Medicine E. Reproducibility and Replicability in Science. Washington, DC: The National Academies Press; 2019."},{"key":"pcbi.1010856.ref007","first-page":"725","article-title":"Artificial intelligence faces reproducibility crisis","volume":"359","author":"H. Matthew","year":"2018","journal-title":"Science (1979)"},{"issue":"7604","key":"pcbi.1010856.ref008","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1038\/533452a","article-title":"1,500 scientists lift the lid on reproducibility","volume":"533","author":"M. Baker","year":"2016","journal-title":"Nature"},{"key":"pcbi.1010856.ref009","article-title":"Covid-19 Retractions Raise Concerns About Data Oversight","volume":"19","author":"H Ledford","year":"2020","journal-title":"Nature"},{"key":"pcbi.1010856.ref010","doi-asserted-by":"crossref","first-page":"e0249937","DOI":"10.1371\/journal.pone.0249937","article-title":"The belief that politics drive scientific research & its impact on COVID-19 risk assessment","volume":"16","author":"DM McLaughlin","year":"2021","journal-title":"PLoS ONE"},{"key":"pcbi.1010856.ref011","doi-asserted-by":"crossref","first-page":"e0262823","DOI":"10.1371\/journal.pone.0262823","article-title":"An anchor in troubled times: Trust in science before and within the COVID-19 pandemic","volume":"17","author":"R Bromme","year":"2022","journal-title":"PLoS ONE"},{"key":"pcbi.1010856.ref012","article-title":"Many scientists citing two scandalous COVID-19 papers ignore their retractions","author":"C. Pillar","year":"2021","journal-title":"Science"},{"key":"pcbi.1010856.ref013","unstructured":"U.S Government Accountability Office. Opportunities to Improve Coordination and Ensure Reproducibility. 2020."},{"key":"pcbi.1010856.ref014","doi-asserted-by":"crossref","first-page":"e1003793","DOI":"10.1371\/journal.pmed.1003793","article-title":"Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines","volume":"18","author":"S Pollett","year":"2021","journal-title":"PLoS Med"},{"key":"pcbi.1010856.ref015","first-page":"1240","article-title":"Enhancing reproducibility for computational methods","volume":"354","author":"V Stodden","year":"2016","journal-title":"Science (1979)"},{"key":"pcbi.1010856.ref016","doi-asserted-by":"crossref","first-page":"5835","DOI":"10.1016\/j.vaccine.2017.09.020","article-title":"An ensemble approach to predicting the impact of vaccination on rotavirus disease in Niger","volume":"35","author":"J Park","year":"2017","journal-title":"Vaccine"},{"key":"pcbi.1010856.ref017","doi-asserted-by":"crossref","first-page":"e0189988","DOI":"10.1371\/journal.pone.0189988","article-title":"Ensemble method for dengue prediction","volume":"13","author":"AL Buczak","year":"2018","journal-title":"PLoS ONE"},{"key":"pcbi.1010856.ref018","doi-asserted-by":"crossref","first-page":"100379","DOI":"10.1016\/j.epidem.2019.100379","article-title":"Real-time forecasting of epidemic trajectories using computational dynamic ensembles","volume":"30","author":"G Chowell","year":"2020","journal-title":"Epidemics"},{"key":"pcbi.1010856.ref019","doi-asserted-by":"crossref","first-page":"7213","DOI":"10.1016\/j.vaccine.2020.09.031","article-title":"Ensemble forecast modeling for the design of COVID-19 vaccine efficacy trials","volume":"38","author":"NE Dean","year":"2020","journal-title":"Vaccine"},{"key":"pcbi.1010856.ref020","doi-asserted-by":"crossref","first-page":"e1007486","DOI":"10.1371\/journal.pcbi.1007486","article-title":"Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S","volume":"15","author":"NG Reich","year":"2019","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1010856.ref021","first-page":"577","article-title":"Harnessing multiple models for outbreak management","volume":"368","author":"K Shea","year":"2020","journal-title":"Science (1979)"},{"key":"pcbi.1010856.ref022","volume-title":"What is a CITATION.cff file?","author":"S. Druskat","year":"2022"},{"key":"pcbi.1010856.ref023","unstructured":"SPDX Workgroup a Linux Foundation Project. The Software Package Data Exchange (SPDX). 2021 [cited 2022 Dec 20]. https:\/\/spdx.dev\/"},{"key":"pcbi.1010856.ref024","doi-asserted-by":"crossref","first-page":"100322","DOI":"10.1016\/j.patter.2021.100322","article-title":"The role of metadata in reproducible computational research","volume":"2","author":"J Leipzig","year":"2021","journal-title":"Patterns (N Y)"},{"key":"pcbi.1010856.ref025","first-page":"256","article-title":"Playing catch-up in building an open research commons","volume":"377","author":"PE Bourne","year":"2022","journal-title":"Science (1979)"},{"key":"pcbi.1010856.ref026","doi-asserted-by":"crossref","first-page":"e1006750","DOI":"10.1371\/journal.pcbi.1006750","article-title":"Ten principles for machine-actionable data management plans","volume":"15","author":"T Miksa","year":"2019","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1010856.ref027","first-page":"1","author":"R. Darby","year":"2021","journal-title":"Checklist for an Open Research Action Plan"},{"key":"pcbi.1010856.ref028","doi-asserted-by":"crossref","first-page":"100402","DOI":"10.1016\/j.impact.2022.100402","article-title":"FAIR assessment tools: evaluating use and performance","volume":"27","author":"NA Krans","year":"2022","journal-title":"NanoImpact"},{"key":"pcbi.1010856.ref029","doi-asserted-by":"crossref","first-page":"W537","DOI":"10.1093\/nar\/gky379","article-title":"The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2018 update","volume":"46","author":"E Afgan","year":"2018","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1010856.ref030","doi-asserted-by":"crossref","unstructured":"Oliveira L, Wilkinson D, Moss\u00e9 D, Childers BR. Occam: Software environment for creating reproducible research. 2018 IEEE 14th International Conference on e-Science (e-Science). 2018:394\u2013395.","DOI":"10.1109\/eScience.2018.00117"},{"key":"pcbi.1010856.ref031","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.future.2014.10.008","article-title":"Pegasus, a workflow management system for science automation","volume":"46","author":"E Deelman","year":"2015","journal-title":"Future Gener Comput Syst"},{"key":"pcbi.1010856.ref032","article-title":"Active curation of artifacts and experiments is changing the way digital libraries will operate","volume":"1686","author":"BR Childers","year":"2016","journal-title":"CEUR Workshop Proc"},{"key":"pcbi.1010856.ref033","doi-asserted-by":"crossref","DOI":"10.1126\/sciadv.abd4563","article-title":"Model uncertainty, political contestation, and public trust in science: Evidence from the COVID-19 pandemic","volume":"6","author":"SE Kreps","year":"2020","journal-title":"Sci Adv"}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010856","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T17:24:28Z","timestamp":1678987468000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010856"}},"subtitle":[],"editor":[{"given":"James M.","family":"McCaw","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2023,3,16]]},"references-count":33,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3,16]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010856","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,16]]}}}