{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T00:56:37Z","timestamp":1778028997882,"version":"3.51.4"},"reference-count":92,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,12,28]],"date-time":"2024-12-28T00:00:00Z","timestamp":1735344000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"ECE-Huawei Research Initiative"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,1,31]]},"abstract":"<jats:p>Bindings for machine learning frameworks (such as TensorFlow and PyTorch) allow developers to integrate a framework\u2019s functionality using a programming language different from the framework\u2019s default language (usually Python). In this article, we study the impact of using TensorFlow and PyTorch bindings in C#, Rust, Python and JavaScript on the software quality in terms of correctness (training and test accuracy) and time cost (training and inference time) when training and performing inference on five widely used deep learning models. Our experiments show that a model can be trained in one binding and used for inference in another binding for the same framework without losing accuracy. Our study is the first to show that using a non-default binding can help improve machine learning software quality from the time cost perspective compared to the default Python binding while still achieving the same level of correctness.<\/jats:p>","DOI":"10.1145\/3678168","type":"journal-article","created":{"date-parts":[[2024,7,13]],"date-time":"2024-07-13T14:38:20Z","timestamp":1720881500000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4468-5972","authenticated-orcid":false,"given":"Hao","family":"Li","sequence":"first","affiliation":[{"name":"University of Alberta, Edmonton, AB, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1812-5365","authenticated-orcid":false,"given":"Gopi Krishnan","family":"Rajbahadur","sequence":"additional","affiliation":[{"name":"Centre for Software Excellence, Huawei Canada, Kingston, ON, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0474-5718","authenticated-orcid":false,"given":"Cor-Paul","family":"Bezemer","sequence":"additional","affiliation":[{"name":"University of Alberta, Edmonton, AB, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,12,28]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"265","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI \u201916)","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI \u201916). USENIX Association, 265\u2013283."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICECCE52056.2021.9514261"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.suscom.2022.100725"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196445"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2016.31"},{"key":"e_1_3_2_7_2","first-page":"1","volume-title":"Proceedings of the International Workshop on Smalltalk Technologies","author":"Bruni Camillo","year":"2013","unstructured":"Camillo Bruni, St\u00e9phane Ducasse, Igor Stasenko, and Luc Fabresse. 2013. Language-side foreign function interfaces with nativeboost. In Proceedings of the International Workshop on Smalltalk Technologies, 1\u201312."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CEIT.2018.8751930"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510163"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587155"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ECAI52376.2021.9515098"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-4012"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510042"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390177"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.14778\/1454159.1454226"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2897336.2897338"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-021-03240-4"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0316-z"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510121"},{"key":"e_1_3_2_21_2","first-page":"4977","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Fedorov Igor","year":"2019","unstructured":"Igor Fedorov, Ryan P. Adams, Matthew Mattina, and Paul Whatmough. 2019. SpArSe: Sparse architecture search for CNNs on resource-constrained microcontrollers. In Proceedings of the Advances in Neural Information Processing Systems. Curran Associates, Inc., Red Hook, NY, 4977\u20134989."},{"key":"e_1_3_2_22_2","first-page":"3259","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Frankle Jonathan","year":"2020","unstructured":"Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. 2020. Linear mode connectivity and the lottery ticket hypothesis. In Proceedings of the 37th International Conference on Machine Learning, Vol. 119. PMLR, 3259\u20133269."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.22363\/2658-4670-2019-27-4-305-315"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387473"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/2660252.2660256"},{"key":"e_1_3_2_26_2","first-page":"2374","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"32","author":"Gundersen Odd E.","year":"2018","unstructured":"Odd E. Gundersen and Sigbj\u00f8rn Kjensmo. 2018. State of the art: Reproducibility in artificial intelligence. Proceedings of the AAAI Conference on Artificial Intelligence 32, 1 (Apr. 2018). 2374\u20133468"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00080"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.array.2021.100057"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2019.00013"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2021.08.002"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSSD47982.2019.9003171"},{"key":"e_1_3_2_33_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Hu Edward J.","year":"2022","unstructured":"Edward J. Hu, Yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the International Conference on Learning Representations, 1\u201313."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511598"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/eScience.2019.00017"},{"key":"e_1_3_2_37_2","unstructured":"Md J. Islam Hoan A. Nguyen Rangeet Pan and Hridesh Rajan. 2019. What do developers ask about ml libraries? A large-scale study using stack overflow. arXiv:1906.11940. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1906.11940"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2019.8891042"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-59410-7_40"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110935"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2021.3079164"},{"key":"e_1_3_2_42_2","unstructured":"Keras. 2021. About Keras. Retrieved March 28 2022 from https:\/\/keras.io\/about\/"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.114805"},{"key":"e_1_3_2_44_2","unstructured":"Alex Krizhevsky. 2012. Learning Multiple Layers of Features from Tiny Images (Technical Report). (April 2012)."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_2_46_2","unstructured":"Yann LeCun Corinna Cortes and Christopher J. C. Burges. 1998. The MNIST Database of Handwritten Digits. Retrieved March 28 2022 from http:\/\/yann.lecun.com\/exdb\/mnist\/"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-018-3354-z"},{"key":"e_1_3_2_48_2","unstructured":"Hao Li and Cor-Paul Bezemer. 2022. Studying popular open source machine learning libraries and their cross-ecosystem bindings. arXiv:2201.07201. Retrieved from https:\/\/doi.org\/10.48550\/ARXIV.2201.07201"},{"key":"e_1_3_2_49_2","unstructured":"Hao Li Gopi K. Rajbahadur and Cor-Paul Bezemer. 2024. The Replication Package of Our Study on Bindings for TensorFlow and PyTorch. Retrieved from https:\/\/github.com\/asgaardlab\/CmpMLBindings"},{"key":"e_1_3_2_50_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Li Xiaoyun","year":"2022","unstructured":"Xiaoyun Li, Belhal Karimi, and Ping Li. 2022. On distributed adaptive optimization with gradient compression. In Proceedings of the International Conference on Learning Representations, 1\u201322."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-020-01637-z"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477535"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377815.3381377"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1002\/0471264385.wei0225"},{"key":"e_1_3_2_55_2","unstructured":"Chunjie Luo Xiwen He Jianfeng Zhan Lei Wang Wanling Gao and Jiahui Dai. 2020. Comparison and benchmarking of AI models and frameworks on mobile devices. arXiv:2005.05085. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2005.05085"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1126\/scitranslmed.abb1655"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.1809.05676"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.90"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-018-09679-z"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.3390\/app10082749"},{"issue":"1","key":"e_1_3_2_63_2","doi-asserted-by":"crossref","first-page":"5821","DOI":"10.1038\/s41467-022-33417-3","article-title":"Public attitudes value interpretability but prioritize accuracy in artificial intelligence","volume":"13","author":"Nussberger Anne-Marie","year":"2022","unstructured":"Anne-Marie Nussberger, Lan Luo, L. E. Celis, and Molly J. Crockett. 2022. Public attitudes value interpretability but prioritize accuracy in artificial intelligence. Nature Communications 13, 1 (2022), 5821.","journal-title":"Nature Communications"},{"key":"e_1_3_2_64_2","first-page":"8024","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 32 (NeurIPS \u201919)","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the Advances in Neural Information Processing Systems 32 (NeurIPS \u201919). Curran Associates, Inc., 8024\u20138035."},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3136014.3136031"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416545"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/2.876288"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.3390\/info11040193"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/1542476.1542516"},{"key":"e_1_3_2_71_2","first-page":"1","volume-title":"Proceedings of the Annual Meeting of the Southern Association for Institutional Research","author":"Romano Jeanine","year":"2006","unstructured":"Jeanine Romano, Jeffrey D. Kromrey, Jesse Coraggio, Jeff Skowronek, and Linda Devine. 2006. Exploring methods for evaluating group differences on the NSSE and other surveys: Are the t-test and Cohen\u2019sd indices the most appropriate choices. In Proceedings of the Annual Meeting of the Southern Association for Institutional Research. Citeseer, 1\u201351."},{"key":"e_1_3_2_72_2","first-page":"19","volume-title":"Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI \u201919)","author":"Arish S.","year":"2019","unstructured":"S. Arish, Sharad Sinha, and K. G. Smitha. 2019. Optimization of convolutional neural networks on resource constrained devices. In Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI \u201919). 19\u201324. https:\/\/doi.org\/10.1109\/ISVLSI.2019.00013"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.5555\/1367985.1367993"},{"key":"e_1_3_2_74_2","first-page":"2503","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NIPS\u201915)","volume":"2","author":"Sculley D.","year":"2015","unstructured":"D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Fran\u00e7ois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems. In Proceedings of the Advances in Neural Information Processing Systems (NIPS\u201915), Vol. 2, MIT Press, Cambridge, MA, 2503\u20132511."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.ps.46.020195.003021"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00049"},{"key":"e_1_3_2_77_2","first-page":"691","volume-title":"Proceedings of the IEEE\/ACM 42nd International Conference on Software Engineering (ICSE \u201920)","author":"Shrestha Nischal","year":"2020","unstructured":"Nischal Shrestha, Colton Botta, Titus Barik, and Chris Parnin. 2020. Here we go again: Why is it difficult for developers to learn another programming language? In Proceedings of the IEEE\/ACM 42nd International Conference on Software Engineering (ICSE \u201920). 691\u2013701."},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2018.00100"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-021-09557-y"},{"key":"e_1_3_2_80_2","first-page":"1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915)","author":"Simonyan Karen","year":"2015","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915), 1\u201314."},{"key":"e_1_3_2_81_2","first-page":"1","volume-title":"Proceedings of the Reproducibility in Machine Learning Workshop at ICML 2018","author":"Tatman Rachael","year":"2018","unstructured":"Rachael Tatman, J. Vanderplas, and Sohier Dane. 2018. A practical taxonomy of reproducibility for machine learning research. In Proceedings of the Reproducibility in Machine Learning Workshop at ICML 2018, 1\u20135."},{"key":"e_1_3_2_82_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288 [cs.CL]. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2307.09288"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2021.3054828"},{"key":"e_1_3_2_84_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Wang Benyou","year":"2021","unstructured":"Benyou Wang, Lifeng Shang, Christina Lioma, Xin Jiang, Hao Yang, Qun Liu, and Jakob Grue Simonsen. 2021. On position embeddings in BERT. In Proceedings of the International Conference on Learning Representations, 1\u201321."},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1002\/int.22883"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571852"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00012"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2021.02.011"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2017.04.002"},{"key":"e_1_3_2_91_2","first-page":"10577","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Yan Zhanglu","year":"2021","unstructured":"Zhanglu Yan, Jun Zhou, and Weng-Fai Wong. 2021. Near lossless transfer learning for spiking neural networks. Proceedings of the AAAI Conference on Artificial Intelligence 35, 12 (May 2021), 10577\u201310584."},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00040"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00020"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678168","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678168","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:08Z","timestamp":1750287248000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678168"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,28]]},"references-count":92,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1,31]]}},"alternative-id":["10.1145\/3678168"],"URL":"https:\/\/doi.org\/10.1145\/3678168","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,28]]},"assertion":[{"value":"2023-01-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}