{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T10:58:13Z","timestamp":1781693893986,"version":"3.54.5"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"1-2","funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["DFG (ID 470857344), and DFG (ID 44071968)"],"award-info":[{"award-number":["DFG (ID 470857344), and DFG (ID 44071968)"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["SNSF (ID 200021L_204817)"],"award-info":[{"award-number":["SNSF (ID 200021L_204817)"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003475","name":"Hasler Stiftung","doi-asserted-by":"publisher","award":["2024-07-08-113"],"award-info":[{"award-number":["2024-07-08-113"]}],"id":[{"id":"10.13039\/501100003475","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Perform. Eval. Comput. Syst."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>We present a methodology for automated performance regression testing at the unit level, designed to integrate seamlessly into the development workflow of high-performance computing (HPC) applications. The proposed framework emphasizes the integration of performance tests with existing functional unit tests by leveraging their shared structure, thereby enabling the development of performance test suites with minimal developer effort and supporting the early detection of performance degradations. As a proof of concept, we implement and release the framework as a publicly available Julia package. It generates architecture-agnostic test suites that remain valid across heterogeneous hardware environments. To demonstrate its effectiveness, we apply it to representative HPC use cases, including memory throughput analysis in stencil-based computations and roofline modeling of a sparse linear system solver. Experiments conducted on a diverse set of CPU and GPU architectures confirm the framework\u2019s ability to detect performance regressions consistently, validating its potential for use in continuous performance testing pipelines.<\/jats:p>","DOI":"10.1145\/3801098","type":"journal-article","created":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T20:51:51Z","timestamp":1772830311000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Generating Architecture-Agnostic Performance Tests from Functional Unit Tests"],"prefix":"10.1145","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2103-6023","authenticated-orcid":false,"given":"Daniel","family":"Vega Rodriguez","sequence":"first","affiliation":[{"name":"Institute of Computing, Universit\u00e0 della Svizzera italiana","place":["Lugano, Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8969-5619","authenticated-orcid":false,"given":"Samuel","family":"Omlin","sequence":"additional","affiliation":[{"name":"Swiss National Supercomputing Centre (CSCS), ETH Zurich","place":["Lugano, Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8580-1023","authenticated-orcid":false,"given":"Dimosthenis","family":"Pasadakis","sequence":"additional","affiliation":[{"name":"Institute of Computing, Universit\u00e0 della Svizzera italiana","place":["Lugano, Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8636-1023","authenticated-orcid":false,"given":"Olaf","family":"Schenk","sequence":"additional","affiliation":[{"name":"Institute of Computing, Universit\u00e0 della Svizzera italiana","place":["Lugano, Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,22]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aam9744"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TETC.2015.2479093"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2016.2543725"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543873.3587583"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3363554"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1562764.1562783"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/5514284"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3030207.3030213"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3185768.3186409"},{"key":"e_1_3_2_11_2","volume-title":"Development of Massively Parallel Near Peak Performance Solvers for Three-Dimensional Geodynamic Modelling","author":"Omlin Samuel","year":"2017","unstructured":"Samuel Omlin. 2017. Development of Massively Parallel Near Peak Performance Solvers for Three-Dimensional Geodynamic Modelling. Ph.D. Dissertation. University of Laussanne. arxiv:physics.geo-ph\/2207.08716"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1498765.1498785"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.3185"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/974044.974059"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227197"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568232"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3184407.3184422"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416573"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931070"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1002\/smr.2215"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2002.10046"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3185768.3186404"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3505285"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-015-0188-0"},{"key":"e_1_3_2_25_2","unstructured":"Google. 2025. google\/benchmark. (Feb.2025). Retrieved March 18 2026 from https:\/\/github.com\/google\/benchmark"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2016.46"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3427921.3450243"},{"key":"e_1_3_2_28_2","unstructured":"Jiahao Chen and Jarrett Revels. 2016. Robust benchmarking in noisy environments. The 2016 IEEE High Performance Extreme Computing Conference (HPEC\u201816). Retrieved from https:\/\/ieee-hpec.org\/2016\/techprog2016\/index_htm_files\/R-main.pdf"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPPW.2010.38"},{"key":"e_1_3_2_30_2","volume-title":"Presented on SC19 Conference","author":"Gruber Thomas","year":"2019","unstructured":"Thomas Gruber, Jan Eitzinger, Georg Hager, and Gerhard Wellein. 2019. LIKWID 5: Lightweight performance tools. In Presented on SC19 Conference."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-56702-0_1"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2751205.2751240"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3184407.3184417"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/SmartCloud.2016.36"},{"key":"e_1_3_2_35_2","first-page":"19","article-title":"Memory bandwidth and machine balance in current high performance computers","author":"McCalpin John D.","year":"1995","unstructured":"John D. McCalpin. 1995. Memory bandwidth and machine balance in current high performance computers. IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter (December 1995), 19\u201325.","journal-title":"IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2014.6844463"},{"key":"e_1_3_2_37_2","volume-title":"PerfTest.jl: Automated Performance Test Generation from Unit Tests","author":"Vega D. S.","year":"2025","unstructured":"D. S. Vega, S. Omlin, D. Pasadakis, and O. Schenk. 2025. PerfTest.jl: Automated Performance Test Generation from Unit Tests. Retrieved March 18, 2026 from https:\/\/github.com\/JuliaPerf\/PerfTest.jl"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1137\/141000671"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2018.2872064"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00602"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-023-00239-3"},{"key":"e_1_3_2_42_2","volume-title":"GFlops.jl","author":"F\u00e9votte Fran\u00e7ois","year":"2019","unstructured":"Fran\u00e7ois F\u00e9votte. 2019. GFlops.jl. Retrieved March 1, 2026 from https:\/\/github.com\/triscale-innov\/GFlops.jl"},{"key":"e_1_3_2_43_2","volume-title":"Cassette.jl","year":"2018","unstructured":"JuliaLabs. 2018. Cassette.jl. Retrieved May 22, 2025 from https:\/\/github.com\/JuliaLabs\/Cassette.jl"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.21105\/jcon.00137"},{"key":"e_1_3_2_45_2","volume-title":"Pardiso.jl: Pardiso Julia interface","year":"2025","unstructured":"JuliaSparse. 2025. Pardiso.jl: Pardiso Julia interface. Retrieved May 22, 2025 from https:\/\/github.com\/JuliaSparse\/Pardiso.jl"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2023.3263969"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocs.2021.101389"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-022-10192-1"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-43736-7_1"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/567806.567807"},{"key":"e_1_3_2_51_2","volume-title":"Intel\u00ae Math Kernel Library","author":"Corporation Intel","year":"2023","unstructured":"Intel Corporation. 2023. Intel\u00ae Math Kernel Library. Retrieved March 18, 2026 from https:\/\/www.intel.com\/content\/www\/us\/en\/developer\/tools\/oneapi\/onemkl.html"},{"key":"e_1_3_2_52_2","volume-title":"BLAS and LAPACK - Basic Linear Algebra Subprograms and Linear Algebra PACKage","year":"2023","unstructured":"Netlib. 2023. BLAS and LAPACK - Basic Linear Algebra Subprograms and Linear Algebra PACKage. Retrieved March 18, 2026 from https:\/\/www.netlib.org\/blas\/"},{"key":"e_1_3_2_53_2","unstructured":"Carsten Bauer. 2023. STREAMbenchmark.jl. Retrieved March 18 2026 from https:\/\/github.com\/JuliaPerf\/STREAMBenchmark.jl\/tree\/master"}],"container-title":["ACM Transactions on Modeling and Performance Evaluation of Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3801098","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T10:46:34Z","timestamp":1781693194000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3801098"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,22]]},"references-count":52,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3801098"],"URL":"https:\/\/doi.org\/10.1145\/3801098","relation":{},"ISSN":["2376-3639","2376-3647"],"issn-type":[{"value":"2376-3639","type":"print"},{"value":"2376-3647","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,22]]},"assertion":[{"value":"2025-07-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-22","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}