{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T12:28:56Z","timestamp":1768220936313,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","funder":[{"name":"Department of Energy by Lawrence Livermore National Laboratory","award":["DE-AC52-07NA27344"],"award-info":[{"award-number":["DE-AC52-07NA27344"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,7,29]]},"DOI":"10.1145\/3736731.3746148","type":"proceedings-article","created":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T11:58:19Z","timestamp":1761047899000},"page":"76-84","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Implementing a Reproducibility Initiative in HPC: Experiences from SC'24"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8862-3277","authenticated-orcid":false,"given":"Guillaume","family":"Pallez","sequence":"first","affiliation":[{"name":"Inria Rennes, Rennes, France"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4398-4432","authenticated-orcid":false,"given":"Judith","family":"Hill","sequence":"additional","affiliation":[{"name":"Lawrence Livermore National Laboratory, Livermore, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5280-3855","authenticated-orcid":false,"given":"Sascha","family":"Hunold","sequence":"additional","affiliation":[{"name":"TU Wien, Vienna, Austria"}]}],"member":"320","published-online":{"date-parts":[[2025,10,21]]},"reference":[{"key":"e_1_3_3_2_2_2","unstructured":"Association for Computing Machinery. 2020. Artifact Review and Badging Version 1.1. https:\/\/www.acm.org\/publications\/policies\/artifact-review-and-badging-current Accessed: 2025-03-30."},{"key":"e_1_3_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Alberto Baccini Giuseppe De\u00a0Nicolao and Eugenio Petrovich. 2019. Citation gaming induced by bibliometric evaluation: A country-level comparative analysis. PLoS One 14 9 (2019) e0221212.","DOI":"10.1371\/journal.pone.0221212"},{"key":"e_1_3_3_2_4_2","doi-asserted-by":"crossref","unstructured":"Robin Bo\u00ebzennec Fanny Dufoss\u00e9 and Guillaume Pallez. 2024. Qualitatively analyzing optimization objectives in the design of hpc resource manager. ACM Transactions on Modeling and Performance Evaluation of Computing Systems 9 4 (2024) 1\u201328.","DOI":"10.1145\/3701986"},{"key":"e_1_3_3_2_5_2","volume-title":"Indicateurs de suivi de l\u2019activit\u00e9 scientifique de l\u2019Inria","author":"Brotcorne Luce","year":"2020","unstructured":"Luce Brotcorne, Anne Canteaut, Aline\u00a0Carneiro Viana, C\u00e9line Grandmont, Benjamin Guedj, St\u00e9phane Huot, Val\u00e9rie Issarny, Guillaume Pallez, Val\u00e9rie Perrier, Vivien Quema, Jean-Baptiste Pomet, Xavier Rival, Sylvain Salvati, and Emmanuel Thom\u00e9. 2020. Indicateurs de suivi de l\u2019activit\u00e9 scientifique de l\u2019Inria. Research Report. Inria. https:\/\/inria.hal.science\/hal-03033764"},{"key":"e_1_3_3_2_6_2","unstructured":"Jan Ciesko. 2024. Artifact Evaluation Report of: MIXQ: Taming Dynamic Outliers in Mixed-Precision Quantization by Online Prediction. Reproducibility report@SC24."},{"key":"e_1_3_3_2_7_2","doi-asserted-by":"publisher","unstructured":"Christian Collberg and Todd\u00a0A. Proebsting. 2016. Repeatability in computer systems research. Commun. ACM 59 3 (Feb. 2016) 62\u201369. 10.1145\/2812803","DOI":"10.1145\/2812803"},{"key":"e_1_3_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-27308-2_47"},{"key":"e_1_3_3_2_9_2","unstructured":"Roberto\u00a0R. Exp\u00f3sito and Keren Zhou. 2024. Artifact Evaluation Report of: Optimizing quantum fourier transformation (qft) kernels for modern nisq and ft architectures. Reproducibility report@SC24."},{"key":"e_1_3_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807623"},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641525.3663631"},{"key":"e_1_3_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807644"},{"key":"e_1_3_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC.COMPANION.2012.81"},{"key":"e_1_3_3_2_14_2","doi-asserted-by":"publisher","unstructured":"Sascha Hunold and Alexandra Carpen-Amarie. 2016. Reproducible MPI Benchmarking is Still Not as Easy as You Think. IEEE Trans. Parallel Distributed Syst. 27 12 (2016) 3617\u20133630. 10.1109\/TPDS.2016.2539167","DOI":"10.1109\/TPDS.2016.2539167"},{"key":"e_1_3_3_2_15_2","doi-asserted-by":"publisher","unstructured":"Sascha Hunold and Jesper\u00a0Larsson Tr\u00e4ff. 2013. On the State and Importance of Reproducible Experimental Research in Parallel Computing. CoRR abs\/1308.3648 (2013). 10.48550\/arxiv.1308.3648 arxiv:https:\/\/arXiv.org\/abs\/1308.3648","DOI":"10.48550\/arxiv.1308.3648"},{"key":"e_1_3_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Evaristo Jim\u00e9nez-Contreras Emilio Delgado L\u00f3pez-C\u00f3zar Rafael Ruiz-P\u00e9rez and V\u00edctor\u00a0M Fern\u00e1ndez. 2002. Impact-factor rewards affect Spanish research. Nature 417 6892 (2002) 898\u2013898.","DOI":"10.1038\/417898b"},{"key":"e_1_3_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/E-SCIENCE58273.2023.10254846"},{"key":"e_1_3_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICNP.2019.8888067"},{"key":"e_1_3_3_2_19_2","unstructured":"Klaus Kra\u00dfnitzer. 2025. AutoAppendix: Towards one-click Reproduction of Computational Artifacts. arxiv:https:\/\/arXiv.org\/abs\/2504.00876\u00a0[cs.DC] https:\/\/arxiv.org\/abs\/2504.00876"},{"key":"e_1_3_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC41406.2024.00022"},{"key":"e_1_3_3_2_21_2","unstructured":"Arnaud Legrand. 2024. Reproducibility Issues and Publication Evolutions (in HPC). Slides available.https:\/\/orap.irisa.fr\/52ieme-forum-reproductibilite\/ Accessed March 2025.."},{"key":"e_1_3_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.3789\/niso-rp-31-2021"},{"key":"e_1_3_3_2_23_2","doi-asserted-by":"publisher","unstructured":"Manish Parashar. 2019. The Reproducibility Initiative. Computer 52 11 (2019) 7\u20138. 10.1109\/MC.2019.2935265","DOI":"10.1109\/MC.2019.2935265"},{"key":"e_1_3_3_2_24_2","unstructured":"Richard Pausch. 2024. Artifact Evaluation Report of: A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization. Reproducibility report@SC24."},{"key":"e_1_3_3_2_25_2","doi-asserted-by":"publisher","unstructured":"Beth\u00a0A. Plale Tanu Malik and Line\u00a0C. Pouchard. 2021. Reproducibility Practice in High-Performance Computing: Community Survey Results. Comput. Sci. Eng. 23 5 (2021) 55\u201360. 10.1109\/MCSE.2021.3096678","DOI":"10.1109\/MCSE.2021.3096678"},{"key":"e_1_3_3_2_26_2","unstructured":"Benjamin Schwaller and Alessio Orsino. 2024. Artifact Evaluation Report of: LLM-Pilot: Characterize and Optimize Performance of your LLM Inference Services. Reproducibility report@SC24."},{"key":"e_1_3_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Arfon\u00a0M Smith Kyle\u00a0E Niemeyer Daniel\u00a0S Katz Lorena\u00a0A Barba George Githinji Melissa Gymrek Kathryn\u00a0D Huff Christopher\u00a0R Madan Abigail\u00a0Cabunoc Mayes Kevin\u00a0M Moerman et\u00a0al. 2018. Journal of Open Source Software (JOSS): design and first-year review. PeerJ Computer Science 4 (2018) e147.","DOI":"10.7717\/peerj-cs.147"},{"key":"e_1_3_3_2_28_2","doi-asserted-by":"publisher","unstructured":"Mengyi Sun Jainabou\u00a0Barry Danfa and Misha Teplitskiy. 2022. Does double-blind peer review reduce bias? Evidence from a top computer science conference. J. Assoc. Inf. Sci. Technol. 73 6 (2022) 811\u2013819. 10.1002\/ASI.24582","DOI":"10.1002\/ASI.24582"},{"key":"e_1_3_3_2_29_2","doi-asserted-by":"crossref","unstructured":"Susan Van\u00a0Rooyen Fiona Godlee Stephen Evans Nick Black and Richard Smith. 1999. Effect of open peer review on quality of reviews and on reviewers\u2019 recommendations: a randomised trial. Bmj 318 7175 (1999) 23\u201327.","DOI":"10.1136\/bmj.318.7175.23"},{"key":"e_1_3_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/10968987_3"}],"event":{"name":"ACM REP '25: ACM Conference on Reproducibility and Replicability","location":"Vancouver Canada","acronym":"ACM REP '25","sponsor":["EIGREP Emerging Interest Group on Reproducibility and Replicability"]},"container-title":["Proceedings of the 3rd ACM Conference on Reproducibility and Replicability"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3736731.3746148","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T18:05:41Z","timestamp":1767981941000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3736731.3746148"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,29]]},"references-count":29,"alternative-id":["10.1145\/3736731.3746148","10.1145\/3736731"],"URL":"https:\/\/doi.org\/10.1145\/3736731.3746148","relation":{},"subject":[],"published":{"date-parts":[[2025,7,29]]},"assertion":[{"value":"2025-10-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}