{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T15:03:29Z","timestamp":1779894209889,"version":"3.53.1"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"U.S. Department of Energy, Office of Science, Office of High Energy Physics","award":["89243024CSC000002"],"award-info":[{"award-number":["89243024CSC000002"]}]},{"name":"U.S. Department of Energy, Office of Science, Office of Workforce Development for Teachers and Scientists (WDTS), Science Undergraduate Laboratory Internships Program"},{"name":"National Science Foundation","award":["CNS-1730158, ACI-1540112, ACI-1541349, OAC-1826967, OAC-2112167, CNS-2100237, CNS-2120019"],"award-info":[{"award-number":["CNS-1730158, ACI-1540112, ACI-1541349, OAC-1826967, OAC-2112167, CNS-2100237, CNS-2120019"]}]},{"DOI":"10.13039\/100014576","name":"University of California Office of the President","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100014576","id-type":"DOI","asserted-by":"crossref"}]},{"name":"University of California San Diego, California Institute for Telecommunications and Information Technology \/ Qualcomm Institute"},{"name":"National Science Foundation","award":["2112356, 2411377"],"award-info":[{"award-number":["2112356, 2411377"]}]},{"name":"Fermi Research Alliance, LLC, U.S. Department of Energy (DOE), Office of Science, Office of High Energy Physics","award":["DE-AC02-07CH11359"],"award-info":[{"award-number":["DE-AC02-07CH11359"]}]},{"name":"U.S. Department of Energy (DOE), Early Career Research Program","award":["DE-0000247070"],"award-info":[{"award-number":["DE-0000247070"]}]},{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"crossref","award":["KA2401045"],"award-info":[{"award-number":["KA2401045"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"crossref"}]},{"name":"U.S. Department of Energy (DOE), Office of Science, Office of Advanced Scientific Computing Research","award":["DE-FOA-0002501, DE-SC0021396"],"award-info":[{"award-number":["DE-FOA-0002501, DE-SC0021396"]}]},{"name":"Research Corporation for Science Advancement","award":["CS-CSA-2023-109"],"award-info":[{"award-number":["CS-CSA-2023-109"]}]},{"DOI":"10.13039\/100000879","name":"Alfred P. Sloan Foundation","doi-asserted-by":"crossref","award":["FG-2023-20452"],"award-info":[{"award-number":["FG-2023-20452"]}],"id":[{"id":"10.13039\/100000879","id-type":"DOI","asserted-by":"crossref"}]},{"name":"U.S. Department of Energy (DOE), Office of Science, Office of High Energy Physics, Early Career Research Program","award":["DE-SC0021187"],"award-info":[{"award-number":["DE-SC0021187"]}]},{"name":"National Science Foundation (NSF), Harnessing the Data Revolution (HDR) Institute for Accelerating AI Algorithms for Data Driven Discovery","award":["PHY-2117997"],"award-info":[{"award-number":["PHY-2117997"]}]},{"name":"U.S. Department of Energy (DOE), Office of High-Energy Physics","award":["DE-SC-0023527"],"award-info":[{"award-number":["DE-SC-0023527"]}]},{"DOI":"10.13039\/501100001804","name":"Canada Research Chairs Program","doi-asserted-by":"crossref","award":["950-232805"],"award-info":[{"award-number":["950-232805"]}],"id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Reconfigurable Technol. Syst."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75th percentile within several percent of the synthesized resources on the synthetic test dataset.<\/jats:p>","DOI":"10.1145\/3787490","type":"journal-article","created":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T11:16:58Z","timestamp":1776079018000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5700-0288","authenticated-orcid":false,"given":"Benjamin","family":"Hawks","sequence":"first","affiliation":[{"name":"Fermi National Accelerator Laboratory, Batavia, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6315-3562","authenticated-orcid":false,"given":"Jason","family":"Weitz","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9453-9755","authenticated-orcid":false,"given":"Dmitri","family":"Demler","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2249-9450","authenticated-orcid":false,"given":"Karla","family":"Tame-Narvaez","sequence":"additional","affiliation":[{"name":"Fermi National Accelerator Laboratory, Batavia, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2610-8226","authenticated-orcid":false,"given":"Dennis","family":"Plotnikov","sequence":"additional","affiliation":[{"name":"Johns Hopkins University, Baltimore, Maryland, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6582-8322","authenticated-orcid":false,"given":"Mohammad Mehdi","family":"Rahimifar","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Sherbrooke, Sherbrooke, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0352-725X","authenticated-orcid":false,"given":"Hamza Ezzaoui","family":"Rahali","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Sherbrooke, Sherbrooke, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6698-8400","authenticated-orcid":false,"given":"Audrey C.","family":"Therrien","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Sherbrooke, Sherbrooke, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-6719-5769","authenticated-orcid":false,"given":"Donovan","family":"Sproule","sequence":"additional","affiliation":[{"name":"Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8720-6615","authenticated-orcid":false,"given":"Elham E.","family":"Khoda","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0653-7033","authenticated-orcid":false,"given":"Keegan A.","family":"Smith","sequence":"additional","affiliation":[{"name":"Texas A&amp;M University, College Station, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3364-7463","authenticated-orcid":false,"given":"Russell","family":"Marroquin","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5749-1432","authenticated-orcid":false,"given":"Giuseppe","family":"Di Guglielmo","sequence":"additional","affiliation":[{"name":"Fermi National Accelerator Laboratory, Batavia, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8440-6854","authenticated-orcid":false,"given":"Nhan","family":"Tran","sequence":"additional","affiliation":[{"name":"Fermi National Accelerator Laboratory, Batavia, Illinois, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5076-7096","authenticated-orcid":false,"given":"Javier","family":"Duarte","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3651-0232","authenticated-orcid":false,"given":"Vladimir","family":"Loncar","sequence":"additional","affiliation":[{"name":"CERN, Geneve, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,27]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Stefan Abi-Karam Rishov Sarkar Allison Seigler Sean Lowe Zhigang Wei Hanqiu Chen Nanditha Rao Lizy John Aman Arora and Cong Hao. 2024. HLSFactory: A framework empowering high-level synthesis datasets for machine learning and beyond. arXiv:2405.00820. Retrieved from https:\/\/arxiv.org\/abs\/2405.00820","DOI":"10.1145\/3670474.3685961"},{"key":"e_1_3_2_3_2","unstructured":"H. Abidi A. Boveia V. Cavaliere D. Furletov A. Gekow C. W. Kalderon and S. Yoo. 2022. Charged particle tracking with machine learning on FPGAs. arXiv:2212.02348. Retrieved from https:\/\/arxiv.org\/abs\/2212.02348"},{"key":"e_1_3_2_4_2","volume-title":"Vitis High-Level Synthesis User Guide (UG1399)","author":"Advanced Micro Devices (Xilinx)","year":"2024","unstructured":"Advanced Micro Devices (Xilinx). 2024. Vitis High-Level Synthesis User Guide (UG1399) (2024.2 ed.). Advanced Micro Devices. Retrieved from https:\/\/docs.amd.com\/r\/en-US\/ug1399-vitis-hls","edition":"2024"},{"key":"e_1_3_2_5_2","unstructured":"Jimmy Lei Ba Jamie Ryan Kiros and Geoffrey E. Hinton. 2016. Layer Normalization. arXiv:1607.06450. Retrieved from https:\/\/arxiv.org\/abs\/1607.06450"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.52202\/075280-1962"},{"key":"e_1_3_2_7_2","first-page":"112","volume-title":"International Symposium on Logic-Based Program Synthesis and Transformation","author":"Barany Gerg\u00f6","year":"2017","unstructured":"Gerg\u00f6 Barany. 2017. Liveness-driven random program generation. In International Symposium on Logic-Based Program Synthesis and Transformation. Springer, 112\u2013127."},{"key":"e_1_3_2_8_2","volume-title":"3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys)","author":"Borras Hendrik","year":"2022","unstructured":"Hendrik Borras, Giuseppe Di Guglielmo, Javier Duarte, Nicol\u00f2 Ghielmetti, Ben Hawks, Scott Hauck, Shih-Chieh Hsu, Ryan Kastner, Jason Liang, Andres Meza, et al. 2022. Open-source FPGA-ML codesign for the MLPerf tiny benchmark. In 3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys). arXiv:2206.11791. Retrieved from https:\/\/arxiv.org\/abs\/2206.11791"},{"key":"e_1_3_2_9_2","volume-title":"International Conference on Learning Representations","author":"Brody Shaked","year":"2022","unstructured":"Shaked Brody, Uri Alon, and Eran Yahav. 2022. How attentive are graph attention networks? In International Conference on Learning Representations. arXiv:2105.14491. Retrieved from https:\/\/openreview.net\/forum?id=F72ximsx7C1"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3662000"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBCAS.2023.3299084"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.623"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","unstructured":"Claudionor N. Coelho Jr. Aki Kuusela Shan Li Hao Zhuang Thea Aarrestad Vladimir Loncar Jennifer Ngadiuba Maurizio Pierini Adrian Alan Pol and Sioni Summers. 2021. Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors. Nature Machine Intelligence 3 (2021) 675. arXiv:2006.10159. Retrieved from https:\/\/arxiv.org\/abs\/2006.10159","DOI":"10.1038\/s42256-021-00356-5"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/FCCM.2018.00029"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPSW55747.2022.00022"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1088\/1748-0221\/13\/07\/P07027"},{"key":"e_1_3_2_17_2","volume-title":"3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys)","author":"Duarte Javier","year":"2022","unstructured":"Javier Duarte, Nhan Tran, Ben Hawks, Christian Herwig, Jules Muhizi, Shvetank Prakash, and Vijay Janapa Reddi. 2022. FastML science benchmarks: Accelerating real-time scientific edge machine learning. In 3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys). arXiv:2207.07958. Retrieved from https:\/\/arxiv.org\/abs\/2207.07958"},{"key":"e_1_3_2_18_2","volume-title":"1st TinyML Research Symposium","author":"Fahim Farah","year":"2021","unstructured":"Farah Fahim, Benjamin Hawks, Christian Herwig, James Hirschauer, Sergo Jindariani, Nhan Tran, Luca P. Carloni, Giuseppe Di Guglielmo, Philip Harris, Jeffrey Krupa, et al. 2021. hls4ml: An open-source codesign workflow to empower scientific low-power machine learning devices. In 1st TinyML Research Symposium. arXiv:2103.05579. Retrieved from https:\/\/arxiv.org\/abs\/2103.05579"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"Lorenzo Ferretti Jihye Kwon Giovanni Ansaloni Giuseppe Di Guglielmo Luca Carloni and Laura Pozzi. 2021. DB4HLS: A database of high-level synthesis design space explorations. arXiv:2101.00587. Retrieved from https:\/\/arxiv.org\/abs\/2101.00587","DOI":"10.1109\/LES.2021.3066882"},{"key":"e_1_3_2_20_2","volume-title":"ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds (RLGM Workshop)","author":"Fey Matthias","year":"2019","unstructured":"Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. In ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds (RLGM Workshop). arXiv:1903.02428. Retrieved from https:\/\/arxiv.org\/abs\/1903.02428"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/NSS\/MIC44845.2022.10399237"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.2197\/ipsjjip.17.242"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3303840"},{"key":"e_1_3_2_24_2","volume-title":"3rd International Conference for Learning Representations","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference for Learning Representations. arXiv:1412.6980. Retrieved from https:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_3_2_25_2","unstructured":"Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. Retrieved from https:\/\/openreview.net\/forum?id=Bkg6RiCqY7"},{"key":"e_1_3_2_26_2","unstructured":"Jason Moss. 2024. xilinx-docker. Retrieved from https:\/\/gitlab.com\/rjmoss\/xilinx-docker"},{"key":"e_1_3_2_27_2","volume-title":"Advances in Neural Information Processing Systems","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems. H. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32, Curran Associates, Inc. arXiv:1912.01703. Retrieved from https:\/\/arxiv.org\/abs\/1912.01703"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.5120\/ijca2017915495"},{"key":"e_1_3_2_29_2","unstructured":"Louis-Noel Pouchet and Tomofumi Yuki. 2018. PolyBench\/C. Retrieved from https:\/\/sourceforge.net\/projects\/polybench\/"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.nima.2023.168829"},{"key":"e_1_3_2_31_2","doi-asserted-by":"crossref","unstructured":"Mohammad Mehdi Rahimifar Hamza Ezzaoui Rahali and Audrey C. Therrien. 2024. rule4ml: An open-source tool for resource utilization and latency estimation for ML models on FPGA. arXiv:2408.05314. Retrieved from https:\/\/arxiv.org\/abs\/2408.05314","DOI":"10.1088\/2632-2153\/ada71c"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2014.6983050"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA56546.2023.10071015"},{"key":"e_1_3_2_34_2","volume-title":"2022 59th ACM\/IEEE Design Automation Conference (DAC)","author":"Sohrabizadeh Atefeh","year":"2022","unstructured":"Atefeh Sohrabizadeh, Yunsheng Bai, Yizhou Sun, and Jason Cong. 2022. Automated accelerator optimization aided by graph neural networks. In 2022 59th ACM\/IEEE Design Automation Conference (DAC)."},{"key":"e_1_3_2_35_2","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2023. Attention is all you need. arXiv:1706.03762. Retrieved from https:\/\/arxiv.org\/abs\/1706.03762"},{"key":"e_1_3_2_36_2","volume-title":"International Conference on Learning Representations","author":"Velic\u0306kovi\u0107 Petar","year":"2018","unstructured":"Petar Velic\u0306kovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li\u00f2, and Yoshua Bengio. 2018. Graph attention networks. In International Conference on Learning Representations. arXiv:1710.10903. Retrieved from https:\/\/arxiv.org\/abs\/1710.10903"},{"key":"e_1_3_2_37_2","first-page":"307","volume-title":"7th Symposium on Operating Systems Design and Implementation","author":"Weil Sage A.","year":"2006","unstructured":"Sage A. Weil, Scott A. Brandt, Ethan L. Miller, Darrell D. E. Long, and Carlos Maltzahn. 2006. Ceph: A scalable, high-performance distributed file system. In 7th Symposium on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, 307."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3489517.3530408"},{"key":"e_1_3_2_39_2","unstructured":"AMD Xilinx. 2023. Alveo U250 Datasheet. Retrieved October 06 2024 from https:\/\/docs.amd.com\/r\/en-US\/ds962-u200-u250\/Alveo-Product-Details"}],"container-title":["ACM Transactions on Reconfigurable Technology and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3787490","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T14:04:20Z","timestamp":1779890660000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3787490"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,27]]},"references-count":38,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3787490"],"URL":"https:\/\/doi.org\/10.1145\/3787490","relation":{},"ISSN":["1936-7406","1936-7414"],"issn-type":[{"value":"1936-7406","type":"print"},{"value":"1936-7414","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,27]]},"assertion":[{"value":"2025-06-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}