{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T07:27:45Z","timestamp":1768030065905,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":25,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,6,17]],"date-time":"2019-06-17T00:00:00Z","timestamp":1560729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1839010, 1421503, 1763788"],"award-info":[{"award-number":["1839010, 1421503, 1763788"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,6,17]]},"DOI":"10.1145\/3322790.3330595","type":"proceedings-article","created":{"date-parts":[[2019,6,21]],"date-time":"2019-06-21T12:45:07Z","timestamp":1561121107000},"page":"23-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Scientific Tests and Continuous Integration Strategies to Enhance Reproducibility in the Scientific Software Context"],"prefix":"10.1145","author":[{"given":"Matthew","family":"Krafczyk","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"August","family":"Shi","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adhithya","family":"Bhaskar","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Darko","family":"Marinov","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Victoria","family":"Stodden","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,6,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Accepted March","author":"Bailey David","year":"2013","unstructured":"David Bailey , Jonathan Borwein , and Victoria Stodden . 2013. Set the default to \"open. Notices of the AMS , Accepted March ( 2013 ). http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi=10.1.1.310.4101 David Bailey, Jonathan Borwein, and Victoria Stodden. 2013. Set the default to \"open. Notices of the AMS, Accepted March (2013). http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi=10.1.1.310.4101"},{"key":"e_1_3_2_1_2_1","unstructured":"Lorena A Barba. 2018. Praxis of Reproducible Computational Science.  Lorena A Barba. 2018. Praxis of Reproducible Computational Science."},{"key":"e_1_3_2_1_3_1","volume-title":"Terminologies for Reproducible Research. ArXiv e-prints (Feb","author":"Barba Lorena A.","year":"2018","unstructured":"Lorena A. Barba . 2018. Terminologies for Reproducible Research. ArXiv e-prints (Feb 2018 ). arxiv: cs.DL\/1802.03311 Lorena A. Barba. 2018. Terminologies for Reproducible Research. ArXiv e-prints (Feb 2018). arxiv: cs.DL\/1802.03311"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3188721"},{"key":"e_1_3_2_1_5_1","unstructured":"G. Booch. 1991. Object Oriented Design: With Applications .Benjamin\/Cummings Pub. 90034884 https:\/\/books.google.com\/books?id=w5VQAAAAMAAJ   G. Booch. 1991. Object Oriented Design: With Applications .Benjamin\/Cummings Pub. 90034884 https:\/\/books.google.com\/books?id=w5VQAAAAMAAJ"},{"key":"e_1_3_2_1_6_1","volume-title":"Lee","author":"Chiang Wei-Fan","year":"2013","unstructured":"Wei-Fan Chiang , Ganesh Gopalakrishnan , Zvonimir Rakamari\u0107 , Dong H. Ahn , and Gregory L . Lee . 2013 . Determinism and Reproducibility in Large-Scale HPC Systems . https:\/\/www.semanticscholar.org\/paper\/Determinism-and-Reproducibility-in-Large-Scale-HPC-Chiang-Gopalakrishnan\/9e8ea7d54dc67f672b31b223ed14edc758b0b28d Wei-Fan Chiang, Ganesh Gopalakrishnan, Zvonimir Rakamari\u0107, Dong H. Ahn, and Gregory L. Lee. 2013. Determinism and Reproducibility in Large-Scale HPC Systems. https:\/\/www.semanticscholar.org\/paper\/Determinism-and-Reproducibility-in-Large-Scale-HPC-Chiang-Gopalakrishnan\/9e8ea7d54dc67f672b31b223ed14edc758b0b28d"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2899401"},{"key":"e_1_3_2_1_8_1","volume-title":"Claerbout and Martin Karrenbach","author":"Jon","year":"1992","unstructured":"Jon F. Claerbout and Martin Karrenbach . 1992 . Electronic documents give reproducible research a new meaning. In SEG Technical Program Expanded Abstracts . 601--604. Jon F. Claerbout and Martin Karrenbach. 1992. Electronic documents give reproducible research a new meaning. In SEG Technical Program Expanded Abstracts. 601--604."},{"key":"e_1_3_2_1_9_1","unstructured":"K. Cranmer and L. Heinrich. 2017. Yadage and Packtivity - analysis preservation using parametrized workflows. ArXiv e-prints (June 2017). arxiv: physics.data-an\/1706.01878  K. Cranmer and L. Heinrich. 2017. Yadage and Packtivity - analysis preservation using parametrized workflows. ArXiv e-prints (June 2017). arxiv: physics.data-an\/1706.01878"},{"key":"e_1_3_2_1_10_1","article-title":"RECAST -- extending the impact of existing analyses","volume":"4","author":"Cranmer K.","year":"2011","unstructured":"K. Cranmer and I. Yavin . 2011 . RECAST -- extending the impact of existing analyses . Journal of High Energy Physics , Vol. 4 , Article 38 (April 2011), bibinfonumpages38 pages. arxiv: hep-ex\/1010.2506 K. Cranmer and I. Yavin. 2011. RECAST -- extending the impact of existing analyses. Journal of High Energy Physics, Vol. 4, Article 38 (April 2011), bibinfonumpages38 pages. arxiv: hep-ex\/1010.2506","journal-title":"Journal of High Energy Physics"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5334\/jors.73"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.3820"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2009.15"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5334\/jors.bt"},{"key":"e_1_3_2_1_15_1","volume-title":"Soltis","author":"Drew Bryan T.","year":"2013","unstructured":"Bryan T. Drew , Romina Gazis , Patricia Cabezas , Kristen S. Swithers , Jiabin Deng , Roseana Rodriguez , Laura A. Katz , Keith A. Crandall , David S. Hibbett , and Douglas E . Soltis . 2013 . Lost Branches on the Tree of Life. PLOS Biology , Vol. 11 , 9 (09 2013), 1--5. Bryan T. Drew, Romina Gazis, Patricia Cabezas, Kristen S. Swithers, Jiabin Deng, Roseana Rodriguez, Laura A. Katz, Keith A. Crandall, David S. Hibbett, and Douglas E. Soltis. 2013. Lost Branches on the Tree of Life. PLOS Biology, Vol. 11, 9 (09 2013), 1--5."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2016.10.022"},{"key":"e_1_3_2_1_17_1","unstructured":"Seth Green. {n.d.}. Five reproducibility lessons from a year of reviewing compute capsules. https:\/\/medium.com\/codeocean\/five-reproducibility-lessons-from-a-year-of-reviewing-compute-capsules-de71729ebd8a  Seth Green. {n.d.}. Five reproducibility lessons from a year of reviewing compute capsules. https:\/\/medium.com\/codeocean\/five-reproducibility-lessons-from-a-year-of-reviewing-compute-capsules-de71729ebd8a"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0377-0427(03)00650-2"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106270"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970358"},{"key":"e_1_3_2_1_21_1","volume-title":"PLOS Medicine","volume":"2","author":"Ioannidis John P. A.","year":"2005","unstructured":"John P. A. Ioannidis . 2005 . Why Most Published Research Findings Are False . PLOS Medicine , Vol. 2 , 8 (08 2005). John P. A. Ioannidis. 2005. Why Most Published Research Findings Are False. PLOS Medicine, Vol. 2, 8 (08 2005)."},{"key":"e_1_3_2_1_22_1","unstructured":"Damien Irving. {n.d.}. Best practices for scientific software. https:\/\/software.ac.uk\/blog\/2017--11--29-best-practices-scientific-software  Damien Irving. {n.d.}. Best practices for scientific software. https:\/\/software.ac.uk\/blog\/2017--11--29-best-practices-scientific-software"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.294.2.218"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1005412"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2016.08.012"}],"event":{"name":"HPDC '19: The 28th International Symposium on High-Performance Parallel and Distributed Computing","location":"Phoenix AZ USA","acronym":"HPDC '19","sponsor":["University of Arizona University of Arizona","SIGHPC ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing","SIGARCH ACM Special Interest Group on Computer Architecture"]},"container-title":["Proceedings of the 2nd International Workshop on Practical Reproducible Evaluation of Computer Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3322790.3330595","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3322790.3330595","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3322790.3330595","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:02:25Z","timestamp":1750208545000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3322790.3330595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,17]]},"references-count":25,"alternative-id":["10.1145\/3322790.3330595","10.1145\/3322790"],"URL":"https:\/\/doi.org\/10.1145\/3322790.3330595","relation":{},"subject":[],"published":{"date-parts":[[2019,6,17]]},"assertion":[{"value":"2019-06-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}