{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T14:25:19Z","timestamp":1771079119690,"version":"3.50.1"},"reference-count":65,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T00:00:00Z","timestamp":1635379200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["GM088849"],"award-info":[{"award-number":["GM088849"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["GM089636"],"award-info":[{"award-number":["GM089636"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Foundation Directorate for Biological Sciences - Emerging Frontiers","award":["EF-0949351"],"award-info":[{"award-number":["EF-0949351"]}]},{"name":"National Science Foundation Directorate for Biological Sciences - Biological Infrastructure","award":["DBI-1565146"],"award-info":[{"award-number":["DBI-1565146"]}]},{"name":"Director, Office of Science, Office of Basic Energy Science, of the U.S. Department of Energy","award":["DE-AC02-05CH11231"],"award-info":[{"award-number":["DE-AC02-05CH11231"]}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Experimental data about gene functions curated from the primary literature have enormous value for research scientists in understanding biology. Using the Gene Ontology (GO), manual curation by experts has provided an important resource for studying gene function, especially within model organisms. Unprecedented expansion of the scientific literature and validation of the predicted proteins have increased both data value and the challenges of keeping pace. Capturing literature-based functional annotations is limited by the ability of biocurators to handle the massive and rapidly growing scientific literature. Within the community-oriented wiki framework for GO annotation called the Gene Ontology Normal Usage Tracking System (GONUTS), we describe an approach to expand biocuration through crowdsourcing with undergraduates. This multiplies the number of high-quality annotations in international databases, enriches our coverage of the literature on normal gene function, and pushes the field in new directions. From an intercollegiate competition judged by experienced biocurators, Community Assessment of Community Annotation with Ontologies (CACAO), we have contributed nearly 5,000 literature-based annotations. Many of those annotations are to organisms not currently well-represented within GO. Over a 10-year history, our community contributors have spurred changes to the ontology not traditionally covered by professional biocurators. The CACAO principle of relying on community members to participate in and shape the future of biocuration in GO is a powerful and scalable model used to promote the scientific enterprise. It also provides undergraduate students with a unique and enriching introduction to critical reading of primary literature and acquisition of marketable skills.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009463","type":"journal-article","created":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T17:24:42Z","timestamp":1635441882000},"page":"e1009463","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":11,"title":["Crowdsourcing biocuration: The Community Assessment of Community Annotation with Ontologies (CACAO)"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3774-5896","authenticated-orcid":true,"given":"Jolene","family":"Ramsey","sequence":"first","affiliation":[]},{"given":"Brenley","family":"McIntosh","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3457-5884","authenticated-orcid":true,"given":"Daniel","family":"Renfro","sequence":"additional","affiliation":[]},{"given":"Suzanne A.","family":"Aleksander","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9618-3336","authenticated-orcid":true,"given":"Sandra","family":"LaBonte","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3035-6940","authenticated-orcid":true,"given":"Curtis","family":"Ross","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1931-2365","authenticated-orcid":true,"given":"Adrienne E.","family":"Zweifel","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5748-3010","authenticated-orcid":true,"given":"Nathan","family":"Liles","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0504-5473","authenticated-orcid":true,"given":"Shabnam","family":"Farrar","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9494-6053","authenticated-orcid":true,"given":"Jason J.","family":"Gill","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7280-7191","authenticated-orcid":true,"given":"Ivan","family":"Erill","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4042-1283","authenticated-orcid":true,"given":"Sarah","family":"Ades","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3837-8864","authenticated-orcid":true,"given":"Tanya Z.","family":"Berardini","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5482-365X","authenticated-orcid":true,"given":"Jennifer A.","family":"Bennett","sequence":"additional","affiliation":[]},{"given":"Siobhan","family":"Brady","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8983-9539","authenticated-orcid":true,"given":"Robert","family":"Britton","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8244-1536","authenticated-orcid":true,"given":"Seth","family":"Carbon","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9166-9137","authenticated-orcid":true,"given":"Steven M.","family":"Caruso","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9970-6657","authenticated-orcid":true,"given":"Dave","family":"Clements","sequence":"additional","affiliation":[]},{"given":"Ritu","family":"Dalia","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6407-4277","authenticated-orcid":true,"given":"Meredith","family":"Defelice","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9880-7440","authenticated-orcid":true,"given":"Erin L.","family":"Doyle","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1789-8000","authenticated-orcid":true,"given":"Iddo","family":"Friedberg","sequence":"additional","affiliation":[]},{"given":"Susan M. R.","family":"Gurney","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6357-2696","authenticated-orcid":true,"given":"Lee","family":"Hughes","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3779-7768","authenticated-orcid":true,"given":"Allison","family":"Johnson","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1538-7303","authenticated-orcid":true,"given":"Jason M.","family":"Kowalski","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3335-4537","authenticated-orcid":true,"given":"Donghui","family":"Li","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9791-0064","authenticated-orcid":true,"given":"Ruth C.","family":"Lovering","sequence":"additional","affiliation":[]},{"given":"Tamara L.","family":"Mans","sequence":"additional","affiliation":[]},{"given":"Fiona","family":"McCarthy","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1606-7771","authenticated-orcid":true,"given":"Sean D.","family":"Moore","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1156-4560","authenticated-orcid":true,"given":"Rebecca","family":"Murphy","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9701-0189","authenticated-orcid":true,"given":"Timothy D.","family":"Paustian","sequence":"additional","affiliation":[]},{"given":"Sarah","family":"Perdue","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7152-5387","authenticated-orcid":true,"given":"Celeste N.","family":"Peterson","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5244-766X","authenticated-orcid":true,"given":"Birgit M.","family":"Pr\u00fc\u00df","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0096-2667","authenticated-orcid":true,"given":"Margaret S.","family":"Saha","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4903-2658","authenticated-orcid":true,"given":"Robert R.","family":"Sheehy","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1288-6473","authenticated-orcid":true,"given":"John T.","family":"Tansey","sequence":"additional","affiliation":[]},{"given":"Louise","family":"Temple","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8102-8109","authenticated-orcid":true,"given":"Alexander William","family":"Thorman","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1073-8880","authenticated-orcid":true,"given":"Saul","family":"Trevino","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7502-8089","authenticated-orcid":true,"given":"Amy Cheng","family":"Vollmer","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1596-7279","authenticated-orcid":true,"given":"Virginia","family":"Walbot","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4544-4417","authenticated-orcid":true,"given":"Joanne","family":"Willey","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8935-0696","authenticated-orcid":true,"given":"Deborah A.","family":"Siegele","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9016-2684","authenticated-orcid":true,"given":"James C.","family":"Hu","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2021,10,28]]},"reference":[{"issue":"5","key":"pcbi.1009463.ref001","doi-asserted-by":"crossref","first-page":"e1002533","DOI":"10.1371\/journal.pcbi.1002533","article-title":"Quality of computationally inferred gene ontology annotations","volume":"8","author":"N \u0160kunca","year":"2012","journal-title":"PLoS Comput Biol"},{"issue":"8","key":"pcbi.1009463.ref002","doi-asserted-by":"crossref","first-page":"e1003715","DOI":"10.1371\/journal.pgen.1003715","article-title":"Global analysis of fission yeast mating genes reveals new autophagy factors","volume":"9","author":"L-L Sun","year":"2013","journal-title":"PLoS Genet"},{"issue":"3","key":"pcbi.1009463.ref003","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1080\/15548627.2017.1415189","article-title":"Exploring autophagy with Gene Ontology","volume":"14","author":"P Denny","year":"2018","journal-title":"Autophagy"},{"issue":"1","key":"pcbi.1009463.ref004","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.cub.2017.11.034","article-title":"Mechanisms connecting the conserved protein kinases Ssp1, Kin1, and Pom1 in fission yeast cell polarity and division","volume":"28","author":"ME Lee","year":"2018","journal-title":"Curr Biol"},{"issue":"1","key":"pcbi.1009463.ref005","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene Ontology: tool for the unification of biology","volume":"25","author":"M Ashburner","year":"2000","journal-title":"Nat Genet"},{"issue":"D1","key":"pcbi.1009463.ref006","doi-asserted-by":"crossref","first-page":"D325","DOI":"10.1093\/nar\/gkaa1113","article-title":"The Gene Ontology resource: enriching a GOld mine","volume":"49","author":"Gene Ontology Consortium","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1009463.ref007","doi-asserted-by":"crossref","first-page":"242","DOI":"10.12688\/f1000research.2-242.v1","article-title":"From zebrafish heart jogging genes to mouse and human orthologs: using Gene Ontology to investigate mammalian heart development","volume":"2","author":"VK Khodiyar","year":"2013","journal-title":"F1000Res"},{"issue":"15","key":"pcbi.1009463.ref008","doi-asserted-by":"crossref","first-page":"2256","DOI":"10.1016\/j.jmb.2018.03.004","article-title":"MetaGO: predicting Gene Ontology of non-homologous proteins through low-resolution protein structure prediction and protein-protein network mapping","volume":"430","author":"C Zhang","year":"2018","journal-title":"J Mol Biol"},{"issue":"12","key":"pcbi.1009463.ref009","doi-asserted-by":"crossref","first-page":"4186","DOI":"10.1021\/acs.jproteome.8b00453","article-title":"Structure and protein interaction-based Gene Ontology annotations reveal likely functions of uncharacterized proteins on human chromosome 17","volume":"17","author":"C Zhang","year":"2018","journal-title":"J Proteome Res"},{"issue":"W1","key":"pcbi.1009463.ref010","doi-asserted-by":"crossref","first-page":"W291","DOI":"10.1093\/nar\/gkx366","article-title":"COFACTOR: improved protein function prediction by combining structure, sequence and protein-protein interaction information","volume":"45","author":"C Zhang","year":"2017","journal-title":"Nucleic Acids Res"},{"issue":"2","key":"pcbi.1009463.ref011","doi-asserted-by":"crossref","first-page":"e001813","DOI":"10.1161\/CIRCGEN.117.001813","article-title":"Improving interpretation of cardiac phenotypes and enhancing discovery with expanded knowledge in the Gene Ontology","volume":"11","author":"RC Lovering","year":"2018","journal-title":"Circ Genom Precis Med"},{"issue":"3","key":"pcbi.1009463.ref012","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1111\/jnc.14666","article-title":"Quantitative interaction proteomics reveals differences in the interactomes of amyloid precursor protein isoforms","volume":"149","author":"RJ Andrew","year":"2019","journal-title":"J Neurochem"},{"key":"pcbi.1009463.ref013","first-page":"bas047","article-title":"Building an efficient curation workflow for the Arabidopsis literature corpus","volume":"2012","author":"D Li","year":"2012","journal-title":"Database"},{"key":"pcbi.1009463.ref014","first-page":"bas045","article-title":"Manual Gene Ontology annotation workflow at the Mouse Genome Informatics Database","volume":"2012","author":"for the Mouse Genome Informatics Database","year":"2012","journal-title":"Database"},{"issue":"10","key":"pcbi.1009463.ref015","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1038\/embor.2010.145","article-title":"Sustainable digital infrastructure","volume":"11","author":"R Bastow","year":"2010","journal-title":"EMBO Rep"},{"key":"pcbi.1009463.ref016","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1007\/978-1-4939-3743-1_21","volume-title":"The Gene Ontology Handbook. Methods in Molecular Biology","author":"SE Lewis","year":"2017"},{"issue":"21","key":"pcbi.1009463.ref017","doi-asserted-by":"crossref","first-page":"3454","DOI":"10.1093\/bioinformatics\/btx439","article-title":"On expert curation and scalability: UniProtKB\/Swiss-Prot as a case study","volume":"33","author":"S Poux","year":"2017","journal-title":"Bioinformatics"},{"issue":"3","key":"pcbi.1009463.ref018","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1093\/bioinformatics\/bty625","article-title":"TreeGrafter: phylogenetic tree-based annotation of proteins with Gene Ontology terms and other annotations","volume":"35","author":"H Tang","year":"2019","journal-title":"Bioinformatics"},{"issue":"5","key":"pcbi.1009463.ref019","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1093\/bib\/bbr042","article-title":"Phylogenetic-based propagation of functional annotations within the Gene Ontology consortium","volume":"12","author":"P Gaudet","year":"2011","journal-title":"Brief Bioinform"},{"key":"pcbi.1009463.ref020","first-page":"65","article-title":"Community-based ontology development, annotation and discussion with MediaWiki extension Ontokiwi and Ontokiwi-based Ontobedia","volume":"2016","author":"E Ong","year":"2016","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"issue":"4","key":"pcbi.1009463.ref021","doi-asserted-by":"crossref","first-page":"e2002846","DOI":"10.1371\/journal.pbio.2002846","article-title":"Biocuration: Distilling data into knowledge","volume":"16","author":"International Society for Biocuration","year":"2018","journal-title":"PLoS Biol"},{"issue":"8","key":"pcbi.1009463.ref022","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1056\/NEJMoa1107643","article-title":"Open-source genomic analysis of Shiga-toxin\u2013producing E. coli O104:H4","volume":"365","author":"H Rohde","year":"2011","journal-title":"N Engl J Med"},{"issue":"10","key":"pcbi.1009463.ref023","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1038\/nsmb.2119","article-title":"Crystal structure of a monomeric retroviral protease solved by protein folding game players","volume":"18","author":"Foldit Contenders Group","year":"2011","journal-title":"Nat Struct Mol Biol"},{"issue":"7070","key":"pcbi.1009463.ref024","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1038\/438900a","article-title":"Internet encyclopaedias go head to head","volume":"438","author":"J. Giles","year":"2005","journal-title":"Nature"},{"key":"pcbi.1009463.ref025","doi-asserted-by":"crossref","first-page":"D1255","DOI":"10.1093\/nar\/gkr925","article-title":"The Gene Wiki in 2011: community intelligence applied to human gene annotation","volume":"40","author":"BM Good","year":"2011","journal-title":"Nucleic Acids Res"},{"issue":"8","key":"pcbi.1009463.ref026","doi-asserted-by":"crossref","first-page":"1753","DOI":"10.1017\/S003329171100287X","article-title":"Quality of information sources about mental disorders: a comparison of Wikipedia with centrally controlled web and printed sources.","volume":"42","author":"NJ Reavley","year":"2011","journal-title":"Psychol Med"},{"issue":"2","key":"pcbi.1009463.ref027","doi-asserted-by":"crossref","first-page":"e0228713","DOI":"10.1371\/journal.pone.0228713","article-title":"Science through Wikipedia: A novel representation of open knowledge through co-citation networks","volume":"15","author":"W Arroyo-Machado","year":"2020","journal-title":"PLoS ONE"},{"issue":"5","key":"pcbi.1009463.ref028","doi-asserted-by":"crossref","first-page":"R89","DOI":"10.1186\/gb-2008-9-5-r89","article-title":"Calling on a million minds for community annotation in WikiProteins","volume":"9","author":"B Mons","year":"2008","journal-title":"Genome Biol"},{"issue":"7322","key":"pcbi.1009463.ref029","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1038\/468359a","article-title":"No rest for the bio-wikis","volume":"468","author":"E. Callaway","year":"2010","journal-title":"Nature"},{"key":"pcbi.1009463.ref030","author":"C Bauerle","year":"2011"},{"key":"pcbi.1009463.ref031","doi-asserted-by":"crossref","first-page":"bat054","DOI":"10.1093\/database\/bat054","article-title":"A guide to best practices for Gene Ontology (GO) manual annotation.","volume":"2013","author":"R Balakrishnan","year":"2013","journal-title":"Database"},{"key":"pcbi.1009463.ref032","article-title":"Gamifying critical reading through a genome annotation intercollegiate competition","volume":"39","author":"I Erill","year":"2018","journal-title":"Tested Studies in Laboratory Teaching"},{"key":"pcbi.1009463.ref033","doi-asserted-by":"crossref","first-page":"D1262","DOI":"10.1093\/nar\/gkr907","article-title":"GONUTS: the Gene Ontology Normal Usage Tracking System","volume":"40","author":"DP Renfro","year":"2012","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"pcbi.1009463.ref034","doi-asserted-by":"crossref","first-page":"10872","DOI":"10.1038\/s41598-018-28948-z","article-title":"GOATOOLS: A Python library for Gene Ontology analyses","volume":"8","author":"DV Klopfenstein","year":"2018","journal-title":"Sci Rep"},{"key":"pcbi.1009463.ref035","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1186\/1471-2105-12-325","article-title":"How the gene ontology evolves","volume":"12","author":"S Leonelli","year":"2011","journal-title":"BMC Bioinformatics"},{"issue":"D1","key":"pcbi.1009463.ref036","doi-asserted-by":"crossref","first-page":"D330","DOI":"10.1093\/nar\/gky1055","article-title":"The Gene Ontology Resource: 20 years and still GOing strong","volume":"47","author":"The Gene Ontology Consortium","year":"2019","journal-title":"Nucleic Acids Res"},{"issue":"3","key":"pcbi.1009463.ref037","first-page":"640","article-title":"Using the wisdom of the crowds to find critical errors in biomedical ontologies: a study of SNOMED CT. J Am Med Inform Assoc","volume":"22","author":"JM Mortensen","year":"2015","journal-title":"May"},{"key":"pcbi.1009463.ref038","doi-asserted-by":"crossref","first-page":"baaa028","DOI":"10.1093\/database\/baaa028","article-title":"Community curation in PomBase: enabling fission yeast experts to provide detailed, standardized, sharable annotation from research publications","author":"A Lock","year":"2020","journal-title":"Database"},{"issue":"Supplement_2","key":"pcbi.1009463.ref039","doi-asserted-by":"crossref","first-page":"S33","DOI":"10.1093\/infdis\/jiz174","article-title":"The science and value of diversity: closing the gaps in our understanding of inclusion and diversity","volume":"220","author":"TH Swartz","year":"2019","journal-title":"J Infect Dis"},{"key":"pcbi.1009463.ref040","doi-asserted-by":"crossref","first-page":"baw149","DOI":"10.1093\/database\/baw149","article-title":"Crowd-sourcing and author submission as alternatives to professional curation","author":"PD Karp","year":"2016","journal-title":"Database."},{"key":"pcbi.1009463.ref041","first-page":"bay013","article-title":"Micropublication: incentivizing community curation and placing unpublished data into the public domain","author":"D Raciti","year":"2018","journal-title":"Database"},{"issue":"7576","key":"pcbi.1009463.ref042","first-page":"S16","volume":"527","author":"PE Bourne","year":"2015","journal-title":"Perspective: Sustaining the big-data ecosystem. Nature"},{"key":"pcbi.1009463.ref043","doi-asserted-by":"crossref","first-page":"baw110","DOI":"10.1093\/database\/baw110","article-title":"How much does curation cost?","author":"PD Karp","year":"2016","journal-title":"Database"},{"issue":"D1","key":"pcbi.1009463.ref044","doi-asserted-by":"crossref","first-page":"D506","DOI":"10.1093\/nar\/gky1049","article-title":"UniProt: a worldwide hub of protein knowledge","volume":"47","author":"UniProt Consortium","year":"2019","journal-title":"Nucleic Acids Res"},{"issue":"D1","key":"pcbi.1009463.ref045","doi-asserted-by":"crossref","first-page":"D581","DOI":"10.1093\/nar\/gkw1105","article-title":"EuPathDB: the eukaryotic pathogen genomics database resource","volume":"45","author":"C Aurrecoechea","year":"2017","journal-title":"Nucleic Acids Res"},{"issue":"D1","key":"pcbi.1009463.ref046","doi-asserted-by":"crossref","first-page":"D535","DOI":"10.1093\/nar\/gkw1017","article-title":"Improvements to PATRIC, the all-bacterial Bioinformatics Database and Analysis Resource Center","volume":"45","author":"AR Wattam","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1009463.ref047","doi-asserted-by":"crossref","first-page":"D593","DOI":"10.1093\/nar\/gkr859","article-title":"ViPR: an open bioinformatics database and analysis resource for virology research","volume":"40","author":"BE Pickett","year":"2012","journal-title":"Nucleic Acids Res"},{"issue":"D1","key":"pcbi.1009463.ref048","doi-asserted-by":"crossref","first-page":"D466","DOI":"10.1093\/nar\/gkw857","article-title":"Influenza Research Database: An integrated bioinformatics resource for influenza virus research","volume":"45","author":"Y Zhang","year":"2017","journal-title":"Nucleic Acids Res"},{"issue":"7","key":"pcbi.1009463.ref049","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.tim.2009.04.003","article-title":"Applying the Gene Ontology in microbial annotation","volume":"17","author":"MG Giglio","year":"2009","journal-title":"Trends Microbiol"},{"issue":"4","key":"pcbi.1009463.ref050","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1187\/cbe.11-03-0027","article-title":"The C.R.E.A.T.E. approach to primary literature shifts undergraduates\u2019 self-assessed ability to read and analyze journal articles, attitudes about science, and epistemological beliefs","volume":"10","author":"SG Hoskins","year":"2011","journal-title":"CBE Life Sci Educ"},{"issue":"1","key":"pcbi.1009463.ref051","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1187\/cbe.11-07-0057","article-title":"Figure facts: encouraging undergraduates to take a data-centered approach to reading primary literature","volume":"12","author":"JE Round","year":"2013","journal-title":"CBE Life Sci Educ"},{"issue":"8","key":"pcbi.1009463.ref052","doi-asserted-by":"crossref","first-page":"e1000448","DOI":"10.1371\/journal.pbio.1000448","article-title":"Incorporating genomics and bioinformatics across the life sciences curriculum","volume":"8","author":"JL Ditty","year":"2010","journal-title":"PLoS Biol"},{"issue":"4","key":"pcbi.1009463.ref053","doi-asserted-by":"crossref","first-page":"e1006682","DOI":"10.1371\/journal.pcbi.1006682","article-title":"A quick guide for student-driven community genome annotation","volume":"15","author":"PS Hosmani","year":"2019","journal-title":"PLoS Comput Biol"},{"issue":"11","key":"pcbi.1009463.ref054","doi-asserted-by":"crossref","first-page":"e1008214","DOI":"10.1371\/journal.pcbi.1008214","article-title":"Galaxy and Apollo as a biologist-friendly interface for high-quality cooperative phage genome annotation","volume":"16","author":"J Ramsey","year":"2020","journal-title":"PLoS Comput Biol"},{"issue":"1","key":"pcbi.1009463.ref055","doi-asserted-by":"crossref","first-page":"e01051","DOI":"10.1128\/mBio.01051-13","article-title":"A broadly implementable research course in phage discovery and genomics for first-year undergraduate students","volume":"5","author":"TC Jordan","year":"2014","journal-title":"mBio"},{"issue":"51","key":"pcbi.1009463.ref056","doi-asserted-by":"crossref","first-page":"13531","DOI":"10.1073\/pnas.1718188115","article-title":"An inclusive Research Education Community (iREC): Impact of the SEA-PHAGES program on research outcomes and student learning","volume":"114","author":"SEA-PHAGES","year":"2017","journal-title":"Proc National Acad Sci U S A"},{"key":"pcbi.1009463.ref057","doi-asserted-by":"crossref","first-page":"160018","DOI":"10.1038\/sdata.2016.18","article-title":"The FAIR Guiding Principles for scientific data management and stewardship","volume":"3","author":"MD Wilkinson","year":"2016","journal-title":"Sci Data"},{"issue":"D1","key":"pcbi.1009463.ref058","doi-asserted-by":"crossref","first-page":"D650","DOI":"10.1093\/nar\/gkz813","article-title":"Alliance of Genome Resources Portal: unified model organism research platform.","volume":"48","author":"Alliance of Genome Resources Consortium","year":"2020","journal-title":"Nucleic Acids Res"},{"issue":"6","key":"pcbi.1009463.ref059","doi-asserted-by":"crossref","first-page":"1635","DOI":"10.1093\/molbev\/msw046","article-title":"ETE 3: Reconstruction, analysis, and visualization of phylogenomic data","volume":"33","author":"J Huerta-Cepas","year":"2016","journal-title":"Mol Biol Evol"},{"issue":"11","key":"pcbi.1009463.ref060","doi-asserted-by":"crossref","first-page":"1422","DOI":"10.1093\/bioinformatics\/btp163","article-title":"Biopython: freely available Python tools for computational molecular biology and bioinformatics","volume":"25","author":"PJA Cock","year":"2009","journal-title":"Bioinformatics"},{"key":"pcbi.1009463.ref061","doi-asserted-by":"crossref","first-page":"D1057","DOI":"10.1093\/nar\/gku1113","article-title":"The GOA database: Gene Ontology annotation updates for","volume":"43","author":"RP Huntley","year":"2015","journal-title":"Nucleic Acids Res"},{"issue":"3","key":"pcbi.1009463.ref062","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: fundamental algorithms for scientific computing in Python","volume":"17","author":"P Virtanen","year":"2020","journal-title":"Nat Methods"},{"issue":"1","key":"pcbi.1009463.ref063","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","article-title":"On a test of whether one of two random variables is stochastically larger than the other.","volume":"18","author":"HB Mann","year":"1947","journal-title":"Ann Math StatMar"},{"issue":"3","key":"pcbi.1009463.ref064","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: a 2D graphics environment.","volume":"9","author":"JD Hunter","year":"2007","journal-title":"Comput Sci Eng"},{"key":"pcbi.1009463.ref065","article-title":"mwaskom\/seaborn: v0.9.0","author":"M Waskom","year":"2018","journal-title":"Version v0.9.0"}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009463","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T17:25:24Z","timestamp":1635441924000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009463"}},"subtitle":[],"editor":[{"given":"Francis","family":"Ouellette","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,10,28]]},"references-count":65,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10,28]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009463","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,28]]}}}