{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:17:37Z","timestamp":1750220257000,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":19,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T00:00:00Z","timestamp":1657238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,8]]},"DOI":"10.1145\/3491418.3530760","type":"proceedings-article","created":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T16:36:23Z","timestamp":1657298183000},"page":"1-7","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Automatic Benchmark Testing with Performance Notification for a Research Computing Center"],"prefix":"10.1145","author":[{"given":"Eugene","family":"Min","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, USA"}]},{"given":"Fang","family":"Liu","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, USA"}]},{"given":"Ronald","family":"Rahaman","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,7,8]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"2022. GitLab CI\/CD. https:\/\/docs.gitlab.com\/ee\/ci\/  2022. GitLab CI\/CD. https:\/\/docs.gitlab.com\/ee\/ci\/"},{"key":"e_1_3_2_1_2_1","unstructured":"2022. ReFrame: A Framework for system regression tests and benchmarks on HPC systems.https:\/\/reframe-hpc.readthedocs.io\/en\/stable\/index.html  2022. ReFrame: A Framework for system regression tests and benchmarks on HPC systems.https:\/\/reframe-hpc.readthedocs.io\/en\/stable\/index.html"},{"key":"e_1_3_2_1_3_1","volume-title":"Continuous Integration and Continuous Delivery Pipeline Automation for Agile Software Project Management. In 2018 Moratuwa Engineering Research Conference (MERCon). 156\u2013161","author":"S.","year":"2018","unstructured":"S.A.I.B. S. Arachchi and Indika Perera. 2018 . Continuous Integration and Continuous Delivery Pipeline Automation for Agile Software Project Management. In 2018 Moratuwa Engineering Research Conference (MERCon). 156\u2013161 . https:\/\/doi.org\/10.1109\/MERCon. 2018 .8421965 10.1109\/MERCon.2018.8421965 S.A.I.B.S. Arachchi and Indika Perera. 2018. Continuous Integration and Continuous Delivery Pipeline Automation for Agile Software Project Management. In 2018 Moratuwa Engineering Research Conference (MERCon). 156\u2013161. https:\/\/doi.org\/10.1109\/MERCon.2018.8421965"},{"key":"e_1_3_2_1_4_1","volume-title":"A data\u2010driven support strategy for a sustainable research software repository. Concurrency and Computation: Practice and Experience 31 (05","author":"Belgin Mehmet","year":"2019","unstructured":"Mehmet Belgin , Tyler Perini , Fang Liu , Nuyun Zhang , Semir Sarajlic , Andre McNeill , Paul Manno , and Neil Bright . 2019. A data\u2010driven support strategy for a sustainable research software repository. Concurrency and Computation: Practice and Experience 31 (05 2019 ). https:\/\/doi.org\/10.1002\/cpe.5338 10.1002\/cpe.5338 Mehmet Belgin, Tyler Perini, Fang Liu, Nuyun Zhang, Semir Sarajlic, Andre McNeill, Paul Manno, and Neil Bright. 2019. A data\u2010driven support strategy for a sustainable research software repository. Concurrency and Computation: Practice and Experience 31 (05 2019). https:\/\/doi.org\/10.1002\/cpe.5338"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.3245"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2015.27"},{"key":"#cr-split#-e_1_3_2_1_7_1.1","doi-asserted-by":"crossref","unstructured":"Todd Evans William\u00a0L. Barth James\u00a0C. Browne Robert\u00a0L. Deleon Thomas\u00a0R. Furlani Steven\u00a0M. Gallo Matthew\u00a0D. Jones and Abani\u00a0K. Patra. 2014. Comprehensive resource use monitoring for HPC systems with TACC stats. Proceedings of HUST 2014: 1st International Workshop on HPC User Support Tools - Held in Conjunction with SC 2014: The International Conference for High Performance Computing Networking Storage and Analysis 13-21. https:\/\/doi.org\/10.1109\/HUST.2014.7 10.1109\/HUST.2014.7","DOI":"10.1109\/HUST.2014.7"},{"key":"#cr-split#-e_1_3_2_1_7_1.2","doi-asserted-by":"crossref","unstructured":"Todd Evans William\u00a0L. Barth James\u00a0C. Browne Robert\u00a0L. Deleon Thomas\u00a0R. Furlani Steven\u00a0M. Gallo Matthew\u00a0D. Jones and Abani\u00a0K. Patra. 2014. Comprehensive resource use monitoring for HPC systems with TACC stats. Proceedings of HUST 2014: 1st International Workshop on HPC User Support Tools - Held in Conjunction with SC 2014: The International Conference for High Performance Computing Networking Storage and Analysis 13-21. https:\/\/doi.org\/10.1109\/HUST.2014.7","DOI":"10.1109\/HUST.2014.7"},{"key":"e_1_3_2_1_8_1","unstructured":"Martin Fowler. 2022. Continuous Integration. https:\/\/martinfowler.com\/articles\/continuousIntegration.html  Martin Fowler. 2022. Continuous Integration. https:\/\/martinfowler.com\/articles\/continuousIntegration.html"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.2871"},{"key":"e_1_3_2_1_10_1","volume-title":"ProvBench: A performance provenance capturing framework for heterogeneous research computing environments. Concurrency and Computation: Practice and Experience 10 (01","author":"Belgin Mehmet","year":"2022","unstructured":"Fang\u00a0(Cherry) Liu, Mehmet Belgin , Nuyun Zhang , Kevin Manalo , Ruben Lara , Christopher\u00a0 P. Stone , and Paul Manno . 2022. ProvBench: A performance provenance capturing framework for heterogeneous research computing environments. Concurrency and Computation: Practice and Experience 10 (01 2022 ), 18. https:\/\/doi.org\/10.1002\/cpe.6820 10.1002\/cpe.6820 Fang\u00a0(Cherry) Liu, Mehmet Belgin, Nuyun Zhang, Kevin Manalo, Ruben Lara, Christopher\u00a0P. Stone, and Paul Manno. 2022. ProvBench: A performance provenance capturing framework for heterogeneous research computing environments. Concurrency and Computation: Practice and Experience 10 (01 2022), 18. https:\/\/doi.org\/10.1002\/cpe.6820"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219104.3219147"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2005.1526010"},{"key":"e_1_3_2_1_13_1","volume-title":"Managing computational gateway resources with XDMoD. Future Generation Computer Systems 98 (9","author":"Sperhac M.","year":"2019","unstructured":"Jeanette\u00a0 M. Sperhac , Robert\u00a0 L. DeLeon , Thomas\u00a0 R. Furlani , Steven\u00a0 M. Gallo , Martins Innus , Matthew\u00a0 D. Jones , Jeffrey\u00a0 T. Palmer , Abani Patra , Benjamin\u00a0 D. Plessinger , Ryan Rathsam , Nikolay Simakov , Joseph\u00a0 P. White , Rudra Chakraborty , and Gregary Dean . 2019. Managing computational gateway resources with XDMoD. Future Generation Computer Systems 98 (9 2019 ), 154\u2013166. https:\/\/doi.org\/10.1016\/j.future.2019.03.029 10.1016\/j.future.2019.03.029 Jeanette\u00a0M. Sperhac, Robert\u00a0L. DeLeon, Thomas\u00a0R. Furlani, Steven\u00a0M. Gallo, Martins Innus, Matthew\u00a0D. Jones, Jeffrey\u00a0T. Palmer, Abani Patra, Benjamin\u00a0D. Plessinger, Ryan Rathsam, Nikolay Simakov, Joseph\u00a0P. White, Rudra Chakraborty, and Gregary Dean. 2019. Managing computational gateway resources with XDMoD. Future Generation Computer Systems 98 (9 2019), 154\u2013166. https:\/\/doi.org\/10.1016\/j.future.2019.03.029"},{"key":"#cr-split#-e_1_3_2_1_14_1.1","doi-asserted-by":"crossref","unstructured":"Jake Tronge Jieyang Chen Patricia Grubel Tim Randles Rusty Davis Quincy Wofford Steven Anaya and Qiang Guan. 2021. BeeSwarm: Enabling Parallel Scaling Performance Measurement in Continuous Integration for HPC Applications. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1136-1140. https:\/\/doi.org\/10.1109\/ASE51524.2021.9678805 10.1109\/ASE51524.2021.9678805","DOI":"10.1109\/ASE51524.2021.9678805"},{"key":"#cr-split#-e_1_3_2_1_14_1.2","doi-asserted-by":"crossref","unstructured":"Jake Tronge Jieyang Chen Patricia Grubel Tim Randles Rusty Davis Quincy Wofford Steven Anaya and Qiang Guan. 2021. BeeSwarm: Enabling Parallel Scaling Performance Measurement in Continuous Integration for HPC Applications. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1136-1140. https:\/\/doi.org\/10.1109\/ASE51524.2021.9678805","DOI":"10.1109\/ASE51524.2021.9678805"},{"key":"#cr-split#-e_1_3_2_1_15_1.1","doi-asserted-by":"crossref","unstructured":"Ozan Tuncer Emre Ates Yijia Zhang Ata Turk Jim Brandt Vitus Leung Manuel Egele and Kaan Co\u015fkun. 2017. Diagnosing Performance Variations in HPC Applications Using Machine Learning. In ISC. 355-373. https:\/\/doi.org\/10.1007\/978-3-319-58667-0_19 10.1007\/978-3-319-58667-0_19","DOI":"10.1007\/978-3-319-58667-0_19"},{"key":"#cr-split#-e_1_3_2_1_15_1.2","doi-asserted-by":"crossref","unstructured":"Ozan Tuncer Emre Ates Yijia Zhang Ata Turk Jim Brandt Vitus Leung Manuel Egele and Kaan Co\u015fkun. 2017. Diagnosing Performance Variations in HPC Applications Using Machine Learning. In ISC. 355-373. https:\/\/doi.org\/10.1007\/978-3-319-58667-0_19","DOI":"10.1007\/978-3-319-58667-0_19"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126946"}],"event":{"name":"PEARC '22: Practice and Experience in Advanced Research Computing","sponsor":["SIGAPP ACM Special Interest Group on Applied Computing","SIGHPC ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing"],"location":"Boston MA USA","acronym":"PEARC '22"},"container-title":["Practice and Experience in Advanced Research Computing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491418.3530760","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3491418.3530760","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:48Z","timestamp":1750188648000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491418.3530760"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,8]]},"references-count":19,"alternative-id":["10.1145\/3491418.3530760","10.1145\/3491418"],"URL":"https:\/\/doi.org\/10.1145\/3491418.3530760","relation":{},"subject":[],"published":{"date-parts":[[2022,7,8]]},"assertion":[{"value":"2022-07-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}