{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T05:14:28Z","timestamp":1736918068308,"version":"3.33.0"},"reference-count":40,"publisher":"Brazilian Journals","issue":"2","license":[{"start":{"date-parts":[[2025,1,13]],"date-time":"2025-01-13T00:00:00Z","timestamp":1736726400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["jlsdgr"],"abstract":"<jats:p>Objective: This study investigates three different methods for provisioning computational environments in scientific research, focusing on the level of reproducibility offered by each one. The methods are manual setup, container-based, and one based on Infrastructure-as-Code principles, the Environment Code-First (ECF) framework.\n\u00a0\nMethod: The methodology adopted for this research comprises a comparative study design evaluating three distinct approaches to computational environment provisioning: manual setup, container-based setup, and the ECF framework. The study involved reproducing a research experiment using the Inception-V3 model to identify canine hip dysplasia from X-ray images across different computational setups. Data was collected through performance metrics such as reproducibility rate, consistency of results, and ease of setup.\n\u00a0\nResults and Discussion: The results revealed that while offering complete control over the environment, the manual setup needed to be more consistent and more accessible to reproduce, leading to variability in the results. The container-based method improved reproducibility but required manual intervention to set up the container infrastructure. The ECF-based approach, however, demonstrated superior performance by fully automating the environment provisioning process, ensuring 100% reproducibility and consistency across different platforms.\n\u00a0\nResearch Implications: The practical and theoretical implications of this research are discussed, providing insights into how the results can be applied to advance practices in computational research and scientific reproducibility. These implications could encompass the broader adoption of IaC tools in scientific experiments, potentially leading to more reliable and reproducible research outcomes.\n\u00a0\nOriginality\/Value: This study contributes to the literature by highlighting the innovative application of the Infrastructure-as-Code approach to achieving reproducibility in scientific research. The potential impact of adopting IaC tools on improving the reliability and consistency of research outcomes evidences the relevance and value of this research.\n\u00a0<\/jats:p>","DOI":"10.47172\/2965-730x.sdgsreview.v5.n02.pe02313","type":"journal-article","created":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T08:20:39Z","timestamp":1736842839000},"page":"e02313","source":"Crossref","is-referenced-by-count":0,"title":["Measuring the Reproducibility of Scientific Research Based on Computing Environment Provisioning Methods (SDG)"],"prefix":"10.47172","volume":"5","author":[{"given":"Daniel Adorno","family":"Gomes","sequence":"first","affiliation":[]},{"given":"Pedro","family":"Mestre","sequence":"additional","affiliation":[]},{"given":"Carlos","family":"Ser\u00f4dio","sequence":"additional","affiliation":[]}],"member":"30764","published-online":{"date-parts":[[2025,1,13]]},"reference":[{"key":"125897","unstructured":"Apple. (2024a). Hypervisor framework. https:\/\/developer.apple.com\/documentation\/hypervisor [Accessed: Jul. 13, 2024]."},{"key":"125898","unstructured":"Apple. (2024b). Virtualization framework. https:\/\/developer.apple.com\/documentation\/virtualization [Accessed: Jul. 13, 2024]."},{"key":"125899","doi-asserted-by":"crossref","unstructured":"Bernholdt, D. E., Doucet, M., Godoy, W. F., Malviya-Thakur, A., & Watson, G. R. (2022). A survey on sustainable software ecosystems to support experimental and observational science at Oak Ridge National Laboratory. Computational Science \u2013 ICCS 2022, 13353, 620-633. https:\/\/doi.org\/10.1007\/978-3-031-08760-8_46.","DOI":"10.1007\/978-3-031-08760-8_46"},{"key":"125900","doi-asserted-by":"crossref","unstructured":"Bobra, M. G., Mumford, S. J., Hewett, R. J., et al. (2020). A survey of computational tools in solar physics. Solar Physics, 295(57). https:\/\/doi.org\/10.1007\/s11207-020-01622-2.","DOI":"10.1007\/s11207-020-01622-2"},{"key":"125901","doi-asserted-by":"crossref","unstructured":"Bush, W. S., Wheeler, N., Beaulieu-Jones, B., & Darabos, C. (2020). Packaging biocomputing software to maximize distribution and reuse. In Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing (Vol. 25, pp. 739\u2013742).","DOI":"10.1142\/9789811215636_0066"},{"key":"125902","doi-asserted-by":"crossref","unstructured":"Cacho, J. R. F., & Taghva, K. (2018). Reproducible research in document analysis and recognition. In Information Technology: New Generations (pp. 389\u2013395). Springer, Berlin.","DOI":"10.1007\/978-3-319-77028-4_51"},{"key":"125903","unstructured":"Docker. (2024a). Troubleshoot topics for Docker Desktop. https:\/\/docs.docker.com\/desktop\/troubleshoot\/topics\/ [Accessed: Jul. 13, 2024]."},{"key":"125904","unstructured":"Docker. (2024b). Workarounds for common problems. https:\/\/docs.docker.com\/desktop\/troubleshoot\/workarounds\/ [Accessed: Jul. 13, 2024]."},{"key":"125905","unstructured":"Docker. (2024c). Known issues. https:\/\/docs.docker.com\/desktop\/troubleshoot\/known-issues\/ [Accessed: Jul. 13, 2024]."},{"key":"125906","unstructured":"Docker. (2024d). Docker installation. https:\/\/docs.docker.com\/engine\/install\/ [Accessed: Jul. 13, 2024]."},{"key":"125907","unstructured":"Docker. (2024e). Docker Desktop release notes. https:\/\/docs.docker.com\/desktop\/release-notes\/ [Accessed: Jul. 13, 2024]."},{"key":"125908","unstructured":"Docker. (2024f). Docker Desktop WSL 2 backend on Windows. https:\/\/docs.docker.com\/desktop\/wsl\/ [Accessed: Jul. 13, 2024]."},{"key":"125909","doi-asserted-by":"crossref","unstructured":"DuPre, E., Holdgraf, C., Karakuzu, A., Tetrel, L., Bellec, P., Stikov, N., & Poline, J. B. (2022). Beyond advertising: New infrastructures for publishing integrated research objects. PLoS Computational Biology, 18(1), e1009651. https:\/\/doi.org\/10.1371\/journal.pcbi.1009651.","DOI":"10.1371\/journal.pcbi.1009651"},{"key":"125910","unstructured":"Gomes, D. A. (2024). Canine Hip Dysplasia Reproducibility Experiment [GitHub repository]. https:\/\/github.com\/adornogomes\/CHD_Reproducibility [Accessed: Jul. 13, 2024]."},{"key":"125911","doi-asserted-by":"crossref","unstructured":"Gomes, D. A., Alves-Pimenta, M. S., Ginja, M., & Filipe, V. (2021). Predicting canine hip dysplasia in X-ray images using deep learning. In A. I. Pereira et al. (Eds.), Optimization, Learning Algorithms and Applications. OL2A 2021 (Vol. 1488, pp. 347-358). Cham: Springer. https:\/\/doi.org\/10.1007\/978-3-030-91885-9_29.","DOI":"10.1007\/978-3-030-91885-9_29"},{"key":"125912","unstructured":"Gomes, D. A., Mestre, P., & Ser\u00f4dio, C. (2019). Infrastructure-as-code for scientific computing environments. In CENTRIC 2019: The Twelfth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services. Nov. 2019."},{"key":"125913","unstructured":"Gomes, D. A., Mestre, P., & Ser\u00f4dio, C. (2022). Environment code-first framework: Provisioning scientific computational environments using the infrastructure-as-code approach. International Journal of Advanced Software, 15(1 & 2), 1-13."},{"key":"125914","doi-asserted-by":"crossref","unstructured":"Gomes, D. A., Mestre, P., & Ser\u00f4dio, C. (2024). Increasing the reproducibility of scientific research works: A case study using the environment code-first framework. International Journal of Professional Business Review, 9(5), e04662. https:\/\/doi.org\/10.26668\/businessreview\/2024.v9i5.4662.","DOI":"10.26668\/businessreview\/2024.v9i5.4662"},{"key":"125915","doi-asserted-by":"crossref","unstructured":"Hannay, J. E., MacLeod, C., Singer, J., Langtangen, H. P., Pfahl, D., & Wilson, G. (2009). How do scientists develop and use scientific software? In 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering (pp. 1-8). Vancouver, BC, Canada. https:\/\/doi.org\/10.1109\/SECSE.2009.5069155.","DOI":"10.1109\/SECSE.2009.5069155"},{"key":"125916","unstructured":"Humble, J., & Farley, D. (2010). Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Boston, MA: Pearson Education."},{"key":"125917","doi-asserted-by":"crossref","unstructured":"Kadri, S., Sboner, A., Sigaras, A., & Roy, S. (2022). Containers in bioinformatics: Applications, practical considerations, and best practices in molecular pathology. The Journal of Molecular Diagnostics, 24(5), 442\u2013454. https:\/\/doi.org\/10.1016\/j.jmoldx.2022.01.006.","DOI":"10.1016\/j.jmoldx.2022.01.006"},{"key":"125918","doi-asserted-by":"crossref","unstructured":"Marwick, B. (2017). Computational reproducibility in archaeological research: Basic principles and a case study of their implementation. Journal of Archaeological Method and Theory, 24, 424\u2013450. https:\/\/doi.org\/10.1007\/s10816-015-9272-9.","DOI":"10.1007\/s10816-015-9272-9"},{"key":"125919","unstructured":"Microsoft. (2024a). Hyper-V on Windows. https:\/\/learn.microsoft.com\/pt-br\/virtualization\/hyper-v-on-windows\/about\/ [Accessed: Jul. 13, 2024]."},{"key":"125920","unstructured":"Microsoft. (2024b). Windows Subsystem for Linux (WSL). https:\/\/learn.microsoft.com\/en-us\/windows\/wsl\/ [Accessed: Jul. 13, 2024]."},{"key":"125921","unstructured":"Microsoft. (2024c). Windows 10 Hyper-V system requirements. https:\/\/learn.microsoft.com\/pt-br\/virtualization\/hyper-v-on-windows\/reference\/hyper-v-requirements [Accessed: Jul. 13, 2024]."},{"key":"125922","unstructured":"Microsoft. (2024d). WSL version 2 troubleshooting. https:\/\/learn.microsoft.com\/en-us\/windows\/wsl\/troubleshooting [Accessed: Jul. 13, 2024]."},{"key":"125923","unstructured":"Morris, K. (2016). Infrastructure as Code: Managing Servers in the Cloud (1st ed.). O\u2019Reilly Media, Inc."},{"key":"125924","unstructured":"Morris, K. (2020). Infrastructure as Code: Dynamic Systems for the Cloud Age (2nd ed.). O\u2019Reilly Media, Inc."},{"key":"125925","doi-asserted-by":"crossref","unstructured":"\u00d6zdo\u011fan, E., Ceran, O., & \u00dcst\u00fcnda\u011f, M. T. (2023). Systematic analysis of infrastructure as code technologies. Gazi University Journal of Science Part A: Engineering and Innovation, 10(4), 452-471. https:\/\/doi.org\/10.54287\/gujsa.1373305.","DOI":"10.54287\/gujsa.1373305"},{"key":"125926","doi-asserted-by":"crossref","unstructured":"Powers, S. M., & Hampton, S. E. (2019). Open science, reproducibility, and transparency in ecology. Ecological Applications, 29(1), e01822. https:\/\/doi.org\/10.1002\/eap.1822.","DOI":"10.1002\/eap.1822"},{"key":"125927","unstructured":"Python. (2024). Python homepage. https:\/\/www.python.org\/ [Accessed: Jul. 13, 2024]."},{"key":"125928","doi-asserted-by":"crossref","unstructured":"Rahman, A., Mahdavi-Hezaveh, R., & Williams, L. (2019). A systematic mapping study of infrastructure as code research. Information and Software Technology, 108, 65-77. https:\/\/doi.org\/10.1016\/j.infsof.2018.12.004.","DOI":"10.1016\/j.infsof.2018.12.004"},{"key":"125929","doi-asserted-by":"crossref","unstructured":"Segal, J., & Morris, C. (2012). Developing software for a scientific community: Some challenges and solutions. In J. Leng & W. Sharrock (Eds.), Handbook of Research on Computational Science and Engineering: Theory and Practice (pp. 177-196). Hershey, PA: IGI Global. https:\/\/doi.org\/10.4018\/978-1-61350-116-0.ch008.","DOI":"10.4018\/978-1-61350-116-0.ch008"},{"key":"125930","doi-asserted-by":"crossref","unstructured":"Sparks, J. (2019). Enabling Docker for HPC. Concurrency and Computation: Practice and Experience, 31, e5018. https:\/\/doi.org\/10.1002\/cpe.5018.","DOI":"10.1002\/cpe.5018"},{"key":"125931","unstructured":"TensorFlow. (2024). TensorFlow homepage. https:\/\/www.tensorflow.org\/ [Accessed: Jul. 13, 2024]."},{"key":"125932","doi-asserted-by":"crossref","unstructured":"Tesser, R. K., & Borin, E. (2023). Containers in HPC: A survey. Journal of Supercomputing, 79, 5759\u20135827. https:\/\/doi.org\/10.1007\/s11227-022-04848-y.","DOI":"10.1007\/s11227-022-04848-y"},{"key":"125933","unstructured":"Virtualbox. (2024). Oracle Virtualbox homepage. https:\/\/www.virtualbox.org\/ [Accessed: Jul. 13, 2024]."},{"key":"125934","doi-asserted-by":"crossref","unstructured":"Wibbels, K., & Moreau, D. (2021). Leveraging containers for reproducible psychological research. Advances in Methods and Practices in Psychological Science. https:\/\/doi.org\/10.1177\/25152459211017853.","DOI":"10.1177\/25152459211017853"},{"key":"125935","unstructured":"Wiggins, A. (2017). The Twelve-Factor App official website. http:\/\/12factor.net [Accessed: Jul. 13, 2024]."},{"key":"125936","doi-asserted-by":"crossref","unstructured":"Ziemann, M., Poulain, P., & Bora, A. (2023). The five pillars of computational reproducibility: Bioinformatics and beyond. Briefings in Bioinformatics, 24(6), bbad375. https:\/\/doi.org\/10.1093\/bib\/bbad375.","DOI":"10.1093\/bib\/bbad375"}],"container-title":["Journal of Lifestyle and SDGs Review"],"original-title":[],"link":[{"URL":"https:\/\/sdgsreview.org\/LifestyleJournal\/article\/download\/2313\/2208","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/sdgsreview.org\/LifestyleJournal\/article\/download\/2313\/2208","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T08:21:02Z","timestamp":1736842862000},"score":1,"resource":{"primary":{"URL":"https:\/\/sdgsreview.org\/LifestyleJournal\/article\/view\/2313"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,13]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,11,1]]}},"URL":"https:\/\/doi.org\/10.47172\/2965-730x.sdgsreview.v5.n02.pe02313","relation":{},"ISSN":["2965-730X"],"issn-type":[{"value":"2965-730X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,13]]}}}