{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T02:43:39Z","timestamp":1793155419753,"version":"4.5.0"},"reference-count":30,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2021,11,11]],"date-time":"2021-11-11T00:00:00Z","timestamp":1636588800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000936","name":"Gordon and Betty Moore Foundation","doi-asserted-by":"publisher","award":["GBMF3834"],"award-info":[{"award-number":["GBMF3834"]}],"id":[{"id":"10.13039\/100000936","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000879","name":"Alfred P. Sloan Foundation","doi-asserted-by":"publisher","award":["Grant 2013-10-27"],"award-info":[{"award-number":["Grant 2013-10-27"]}],"id":[{"id":"10.13039\/100000879","id-type":"DOI","asserted-by":"publisher"}]},{"name":"U.C. Berkeley UCSF and Johnson & Johnson","award":["Innovate for Health Data Science Fellowship"],"award-info":[{"award-number":["Innovate for Health Data Science Fellowship"]}]},{"DOI":"10.13039\/100000861","name":"Burroughs Wellcome Fund","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000861","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Functional, usable, and maintainable open-source software is increasingly essential to scientific research, but there is a large variation in formal training for software development and maintainability. Here, we propose 10 \u201crules\u201d centered on 2 best practice components: clean code and testing. These 2 areas are relatively straightforward and provide substantial utility relative to the learning investment. Adopting clean code practices helps to standardize and organize software code in order to enhance readability and reduce cognitive load for both the initial developer and subsequent contributors; this allows developers to concentrate on core functionality and reduce errors. Clean coding styles make software code more amenable to testing, including unit tests that work best with modular and consistent software code. Unit tests interrogate specific and isolated coding behavior to reduce coding errors and ensure intended functionality, especially as code increases in complexity; unit tests also implicitly provide example usages of code. Other forms of testing are geared to discover erroneous behavior arising from unexpected inputs or emerging from the interaction of complex codebases. Although conforming to coding styles and designing tests can add time to the software development project in the short term, these foundational tools can help to improve the correctness, quality, usability, and maintainability of open-source scientific software code. They also advance the principal point of scientific research: producing accurate results in a reproducible way. In addition to suggesting several tips for getting started with clean code and testing practices, we recommend numerous tools for the popular open-source scientific software languages Python, R, and Julia.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009481","type":"journal-article","created":{"date-parts":[[2021,11,11]],"date-time":"2021-11-11T18:25:23Z","timestamp":1636655123000},"page":"e1009481","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":34,"title":["Ten simple rules on writing clean and reliable open-source scientific software"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5853-3668","authenticated-orcid":true,"given":"Haley","family":"Hunter-Zinck","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1320-4347","authenticated-orcid":true,"given":"Alexandre Fioravante","family":"de Siqueira","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2108-6459","authenticated-orcid":true,"given":"V\u00e1leri N.","family":"V\u00e1squez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Barnes","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4296-998X","authenticated-orcid":true,"given":"Ciera C.","family":"Martinez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,11,11]]},"reference":[{"key":"pcbi.1009481.ref001","doi-asserted-by":"crossref","first-page":"1981","DOI":"10.1093\/bib\/bby063","article-title":"A brief history of bioinformatics","volume":"20","author":"J Gauthier","year":"2019","journal-title":"Brief Bioinform"},{"key":"pcbi.1009481.ref002","doi-asserted-by":"crossref","first-page":"100206","DOI":"10.1016\/j.patter.2021.100206","article-title":"Barely sufficient practices in scientific computing","volume":"2","author":"G Lee","year":"2021","journal-title":"Patterns (N Y)"},{"key":"pcbi.1009481.ref003","doi-asserted-by":"crossref","first-page":"e1002802","DOI":"10.1371\/journal.pcbi.1002802","article-title":"Ten Simple Rules for the Open Development of Scientific Software","volume":"8","author":"A Prli\u0107","year":"2012","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref004","doi-asserted-by":"crossref","first-page":"e1005412","DOI":"10.1371\/journal.pcbi.1005412","article-title":"Ten simple rules for making research software more robust","volume":"13","author":"M Taschuk","year":"2017","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref005","doi-asserted-by":"crossref","first-page":"e1005265","DOI":"10.1371\/journal.pcbi.1005265","article-title":"Ten Simple Rules for Developing Usable Software in Computational Biology","volume":"13","author":"M List","year":"2017","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref006","doi-asserted-by":"crossref","first-page":"e1006561","DOI":"10.1371\/journal.pcbi.1006561","article-title":"Ten simple rules for documenting scientific software","volume":"14","author":"BD Lee","year":"2018","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref007","doi-asserted-by":"crossref","first-page":"e1004947","DOI":"10.1371\/journal.pcbi.1004947","article-title":"Ten Simple Rules for Taking Advantage of Git and GitHub","volume":"12","author":"Y Perez-Riverol","year":"2016","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref008","doi-asserted-by":"crossref","first-page":"e1008549","DOI":"10.1371\/journal.pcbi.1008549","article-title":"Ten simple rules for quick and dirty scientific programming","volume":"17","author":"G Balaban","year":"2021","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref009","doi-asserted-by":"crossref","first-page":"e163","DOI":"10.7717\/peerj-cs.163","article-title":"Towards computational reproducibility: researcher perspectives on the use and sharing of software","volume":"4","author":"Y AINoamany","year":"2018","journal-title":"PeerJ Comput Sci"},{"key":"pcbi.1009481.ref010","doi-asserted-by":"crossref","first-page":"1856","DOI":"10.1126\/science.314.5807.1856","article-title":"A Scientist\u2019s Nightmare: Software Problem Leads to Five Retractions","volume":"314","author":"G. Miller","year":"2006","journal-title":"Science"},{"key":"pcbi.1009481.ref011","doi-asserted-by":"crossref","first-page":"7900","DOI":"10.1073\/pnas.1602413113","article-title":"Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates","volume":"113","author":"A Eklund","year":"2016","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"22","key":"pcbi.1009481.ref012","doi-asserted-by":"crossref","first-page":"2335","DOI":"10.1001\/jama.2018.17933","article-title":"Effect of a Program Combining Transitional Care and Long-term Self-management Support on Outcomes of Hospitalized Patients With Chronic Obstructive Pulmonary Disease: A Randomized Clinical Trial","volume":"320","author":"H Aboumatar","year":"2018","journal-title":"JAMA"},{"key":"pcbi.1009481.ref013","doi-asserted-by":"crossref","first-page":"e158","DOI":"10.1371\/journal.pcbi.0030158","article-title":"Retraction: Measures of Clade Confidence Do Not Correlate with Accuracy of Phylogenetic Trees","volume":"3","author":"BG Hall","year":"2007","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref014","first-page":"1758","article-title":"Algorithmic identification of discrepancies between published ratios and their reported confidence intervals and P-values.","volume":"34","author":"C Georgescu","year":"2018"},{"key":"pcbi.1009481.ref015","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1038\/nbt0308-274","article-title":"BLOSUM62 miscalculations improve search performance","volume":"26","author":"MP Styczynski","year":"2008","journal-title":"Nat Biotechnol"},{"key":"pcbi.1009481.ref016","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-94866-9","volume-title":"Advances in Neuroergonomics and Cognitive Engineering: Proceedings of the AHFE 2019 International Conference on Neuroergonomics and Cognitive Engineering, and the AHFE International Conference on Industrial Cognitive Ergonomics and Engineering Psychology, July 24\u201328, 2019","author":"H. Ayaz","year":"2019"},{"key":"pcbi.1009481.ref017","doi-asserted-by":"crossref","first-page":"2140","DOI":"10.1007\/s10664-019-09751-4","article-title":"Measuring the impact of lexical and structural inconsistencies on developers\u2019 cognitive load during bug localization","volume":"25","author":"S Fakhoury","year":"2020","journal-title":"Empir Softw Eng"},{"key":"pcbi.1009481.ref018","volume-title":"Building maintainable software: ten guidelines for future-proof code","author":"J. Visser","year":"2016"},{"key":"pcbi.1009481.ref019","volume-title":"Unit testing: principles, practices, and patterns.","author":"V. Khorikov","year":"2020"},{"key":"pcbi.1009481.ref020","volume-title":"Refactoring: improving the design of existing code","author":"M. Fowler","year":"2019","edition":"2"},{"key":"pcbi.1009481.ref021","author":"JO Villalobos","year":"2019","journal-title":"Defensive Programming Fundamentals"},{"key":"pcbi.1009481.ref022","unstructured":"Dubien N. Introduction to Property Based Testing. 2018 Mar 23 [cited 2021 Mar 10]. Available from: https:\/\/medium.com\/criteo-engineering\/introduction-to-property-based-testing-f5236229d237."},{"key":"pcbi.1009481.ref023","volume-title":"Test-driven development: by example.","author":"K. Beck","year":"2003"},{"key":"pcbi.1009481.ref024","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/MS.2019.2899838","article-title":"Naming the Pain in Developing Scientific Software","volume":"37","author":"I Wiese","year":"2020","journal-title":"IEEE Softw"},{"key":"pcbi.1009481.ref025","article-title":"rOpenSci Packages: Development, Maintenance, and Peer Review","author":"rOpenSci","year":"2021","journal-title":"Zenodo"},{"key":"pcbi.1009481.ref026","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1145\/2491411.2491444","volume-title":"Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering\u2014ESEC\/FSE 2013","author":"PC Rigby","year":"2013"},{"key":"pcbi.1009481.ref027","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/978-3-319-69926-4_9","volume-title":"Product-Focused Software Process Improvement","author":"T Baum","year":"2017"},{"key":"pcbi.1009481.ref028","doi-asserted-by":"crossref","first-page":"e1001745","DOI":"10.1371\/journal.pbio.1001745","article-title":"Best Practices for Scientific Computing","volume":"12","author":"G Wilson","year":"2014","journal-title":"PLoS Biol"},{"key":"pcbi.1009481.ref029","doi-asserted-by":"crossref","first-page":"e1005510","DOI":"10.1371\/journal.pcbi.1005510","article-title":"Good enough practices in scientific computing","volume":"13","author":"G Wilson","year":"2017","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009481.ref030","doi-asserted-by":"crossref","first-page":"e1008770","DOI":"10.1371\/journal.pcbi.1008770","article-title":"Principles for data analysis workflows","volume":"17","author":"S Stoudt","year":"2021","journal-title":"PLoS Comput Biol"}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009481","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,11]],"date-time":"2021-11-11T18:25:45Z","timestamp":1636655145000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009481"}},"subtitle":[],"editor":[{"given":"Scott","family":"Markel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,11,11]]},"references-count":30,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11,11]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009481","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,11]]}}}