{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:08:03Z","timestamp":1772906883658,"version":"3.50.1"},"reference-count":83,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T00:00:00Z","timestamp":1713398400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61825201, 62332001"],"award-info":[{"award-number":["61825201, 62332001"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>\n            Deep learning (DL) frameworks have become the cornerstone of the rapidly developing DL field. Through installation dependencies specified in the distribution metadata, numerous packages directly or transitively depend on DL frameworks, layer after layer, forming DL package supply chains (SCs), which are critical for DL frameworks to remain competitive. However, vital knowledge on how to nurture and sustain DL package SCs is still lacking. Achieving this knowledge may help DL frameworks formulate effective measures to strengthen their SCs to remain competitive and shed light on dependency issues and practices in the DL SC for researchers and practitioners. In this paper, we explore the domains, clusters, and disengagement of packages in two representative PyPI DL package SCs to bridge this knowledge gap. We analyze the metadata of nearly six million PyPI package distributions and construct version-sensitive SCs for two popular DL frameworks: TensorFlow and PyTorch. We find that popular packages (measured by the number of monthly downloads) in the two SCs cover 34 domains belonging to eight categories.\n            <jats:italic>Applications<\/jats:italic>\n            ,\n            <jats:italic>Infrastructure<\/jats:italic>\n            , and\n            <jats:italic>Sciences<\/jats:italic>\n            categories account for over 85% of popular packages in either SC and TensorFlow and PyTorch SC have developed specializations on\n            <jats:italic>Infrastructure<\/jats:italic>\n            and\n            <jats:italic>Applications<\/jats:italic>\n            packages, respectively. We employ the Leiden community detection algorithm and detect 131 and 100 clusters in the two SCs. The clusters mainly exhibit four shapes: Arrow, Star, Tree, and Forest with increasing dependency complexity. Most clusters are Arrow or Star, while Tree and Forest clusters account for most packages (Tensorflow SC: 70.7%, PyTorch SC: 92.9%). We identify three groups of reasons why packages disengage from the SC (i.e., remove the DL framework and its dependents from their installation dependencies): dependency issues, functional improvements, and ease of installation. The most common reason in TensorFlow SC is dependency incompatibility and in PyTorch SC is to simplify functionalities and reduce installation size. Our study provides rich implications for DL framework vendors, researchers, and practitioners on the maintenance and dependency management practices of PyPI DL SCs.\n          <\/jats:p>","DOI":"10.1145\/3640336","type":"journal-article","created":{"date-parts":[[2024,1,10]],"date-time":"2024-01-10T12:26:16Z","timestamp":1704889576000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Characterizing Deep Learning Package Supply Chains in PyPI: Domains, Clusters, and Disengagement"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0942-7890","authenticated-orcid":false,"given":"Kai","family":"Gao","sequence":"first","affiliation":[{"name":"School of Software &amp; Microelectronics, Peking University, Beijing, China and Key Laboratory of High Confidence Software Technologies, Ministry of Education, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6181-6519","authenticated-orcid":false,"given":"Runzhi","family":"He","sequence":"additional","affiliation":[{"name":"School of Computer Science, Peking University, Beijing, China and Key Laboratory of High Confidence Software Technologies, Ministry of Education, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2988-2575","authenticated-orcid":false,"given":"Bing","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Computer Science, Peking University, Beijing, China and Key Laboratory of High Confidence Software Technologies, Ministry of Education, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6324-3964","authenticated-orcid":false,"given":"Minghui","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computer Science, Peking University, Beijing, China and Key Laboratory of High Confidence Software Technologies, Ministry of Education, China"}]}],"member":"320","published-online":{"date-parts":[[2024,4,18]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"2013. PEP 440-Version Identification and Dependency Specification. https:\/\/peps.python.org\/pep-0440\/(Accessed on 02\/02\/2023)."},{"key":"e_1_3_2_3_2","unstructured":"2015. PEP 508 \u2013 Dependency Specification for Python Software Packages. https:\/\/peps.python.org\/pep-0508(Accessed on 2022-08-04)."},{"key":"e_1_3_2_4_2","unstructured":"2019. Microsoft\/CNTK: Microsoft Cognitive Toolkit (CNTK) an Open Source Deep-Learning Toolkit. https:\/\/github.com\/microsoft\/CNTK#disclaimer(Accessed on 12\/15\/2022)."},{"key":"e_1_3_2_5_2","unstructured":"2019. Preferred Networks Migrates its Deep Learning Research Platform to PyTorch - Preferred Networks Inc.https:\/\/www.preferred.jp\/en\/news\/pr20191205\/(Accessed on 12\/15\/2022)."},{"key":"e_1_3_2_6_2","unstructured":"2021. SolarWinds Orion Security Breach: A Shift in the Software Supply Chain Paradigm. https:\/\/snyk.io\/blog\/solarwinds-orion-security-breach-a-shift-in-the-software-supply-chain-paradigm\/(Accessed on 2022-08-02)."},{"key":"e_1_3_2_7_2","unstructured":"2022. Core Metadata Specifications \u2013 Python Packaging User Guide. https:\/\/packaging.python.org\/en\/latest\/specifications\/core-metadata\/(Accessed on 08\/02\/2022)."},{"key":"e_1_3_2_8_2","unstructured":"2022. Glossary \u2013 Python Packaging User Guide. https:\/\/packaging.python.org\/en\/latest\/glossary\/(Accessed on 08\/03\/2022)."},{"key":"e_1_3_2_9_2","unstructured":"2022. onnx.ai. https:\/\/onnx.ai\/(Accessed on 2022-09-01)."},{"key":"e_1_3_2_10_2","unstructured":"2022. PyPI \\(\\cdot\\) The Python Package Index. https:\/\/pypi.org\/(Accessed on 12\/22\/2022)."},{"key":"e_1_3_2_11_2","unstructured":"2022. Python Package Index (PyPI) \u2013 Marketplace \u2013 Google Cloud Console. https:\/\/console.cloud.google.com\/marketplace\/product\/gcp-public-data-pypi\/pypi(Accessed on 2022-08-04)."},{"key":"e_1_3_2_12_2","unstructured":"2022. PyTorch. https:\/\/pytorch.org\/(Accessed on 12\/27\/2022)."},{"key":"e_1_3_2_13_2","unstructured":"2022. PyTorch\/Vision: Datasets Transforms and Models Specific to Computer Vision. https:\/\/github.com\/pytorch\/vision(Accessed on 08\/14\/2022)."},{"key":"e_1_3_2_14_2","unstructured":"2022. Stack Overflow Developer Survey 2022. https:\/\/survey.stackoverflow.co\/2022\/#section-most-popular-technologies-other-frameworks-and-libraries(Accessed on 01\/31\/2023)."},{"key":"e_1_3_2_15_2","unstructured":"2022. TensorFlow. https:\/\/www.tensorflow.org\/(Accessed on 12\/27\/2022)."},{"key":"e_1_3_2_16_2","unstructured":"2023. API Documentation | TensorFlow v2.12.0. https:\/\/www.tensorflow.org\/api_docs(Accessed on 06\/13\/2023)."},{"key":"e_1_3_2_17_2","unstructured":"2023. Ecosystem | PyTorch. https:\/\/pytorch.org\/ecosystem\/(Accessed on 02\/03\/2023)."},{"key":"e_1_3_2_18_2","unstructured":"2023. Igraph \u2013 Network Analysis Software. https:\/\/igraph.org\/(Accessed on 06\/27\/2023)."},{"key":"e_1_3_2_19_2","unstructured":"2023. Let\u2019s Permit \u2018+Local.Version.Label\u2018 in Version Specifiers - Packaging. https:\/\/discuss.python.org\/t\/lets-permit-local-version-label-in-version-specifiers\/22781(Accessed on 02\/02\/2023)."},{"key":"e_1_3_2_20_2","unstructured":"2023. Packaging. https:\/\/packaging.pypa.io\/en\/stable\/(Accessed on 02\/03\/2023)."},{"key":"e_1_3_2_21_2","unstructured":"2023. PyTorch C++ API \u2013 PyTorch main documentation. https:\/\/pytorch.org\/cppdocs\/(Accessed on 06\/13\/2023)."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-7099-1_1"},{"key":"e_1_3_2_23_2","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.)."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409738"},{"key":"e_1_3_2_25_2","first-page":"1","article-title":"Towards better dependency management: A first look at dependency smells in Python projects","author":"Cao Yulu","year":"2022","unstructured":"Yulu Cao, Lin Chen, Wanwangying Ma, Yanhui Li, Yuming Zhou, and Linzhang Wang. 2022. Towards better dependency management: A first look at dependency smells in Python projects. IEEE Transactions on Software Engineering (2022), 1\u201326.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3240471"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409759"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00068"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.5555\/3203489"},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1109\/ESEM.2011.36","volume-title":"2011 International Symposium on Empirical Software Engineering and Measurement","author":"Cruzes Daniela S.","year":"2011","unstructured":"Daniela S. Cruzes and Tore Dyba. 2011. Recommended steps for thematic synthesis in software engineering. In 2011 International Symposium on Empirical Software Engineering and Measurement. 275\u2013284."},{"key":"e_1_3_2_31_2","series-title":"ECSAW \u201916","volume-title":"Proceedings of the 10th European Conference on Software Architecture Workshops","author":"Decan Alexandre","year":"2016","unstructured":"Alexandre Decan, Tom Mens, and Maelick Claes. 2016. On the topology of package dependency networks: A comparison of three programming language ecosystems. In Proceedings of the 10th European Conference on Software Architecture Workshops (Copenhagen, Denmark) (ECSAW \u201916). Association for Computing Machinery, New York, NY, USA, Article 21, 4 pages."},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1109\/SANER.2017.7884604","volume-title":"2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Decan Alexandre","year":"2017","unstructured":"Alexandre Decan, Tom Mens, and Ma\u00eblick Claes. 2017. An empirical comparison of dependency issues in OSS packaging ecosystems. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). 2\u201312."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196401"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9589-y"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00482"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3273934.3273942"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453478"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3163576"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00064"},{"key":"e_1_3_2_40_2","first-page":"868","volume-title":"2020 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Han Junxiao","year":"2020","unstructured":"Junxiao Han, Shuiguang Deng, David Lo, Chen Zhi, Jianwei Yin, and Xin Xia. 2020. An empirical study of the dependency networks of deep learning libraries. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). 868\u2013878."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09819-6"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468571"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3278129"},{"issue":"2","key":"e_1_3_2_44_2","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"Holm Sture","year":"1979","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 2 (1979), 65\u201370. http:\/\/www.jstor.org\/stable\/4615733","journal-title":"Scandinavian Journal of Statistics"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616325"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_3_2_48_2","series-title":"ICSE \u201920","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/3377811.3380378","volume-title":"Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering","author":"Islam Md. Johirul","year":"2020","unstructured":"Md. Johirul Islam, Rangeet Pan, Giang Nguyen, and Hridesh Rajan. 2020. Repairing deep neural networks: Fix patterns and challenges. In Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE \u201920). Association for Computing Machinery, New York, NY, USA, 1135\u20131146."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-59410-7_40"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110935"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.55"},{"key":"e_1_3_2_52_2","unstructured":"Klaus Krippendorff. 2011. Computing Krippendorff\u2019s alpha-reliability. (2011)."},{"key":"e_1_3_2_53_2","volume-title":"Content Analysis: An Introduction to its Methodology","author":"Krippendorff Klaus","year":"2018","unstructured":"Klaus Krippendorff. 2018. Content Analysis: An Introduction to its Methodology. Sage publications."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409711"},{"key":"e_1_3_2_55_2","series-title":"ICSE \u201922","first-page":"1545","volume-title":"Proceedings of the 44th International Conference on Software Engineering","author":"Li Zhenming","year":"2022","unstructured":"Zhenming Li, Ying Wang, Zeqi Lin, Shing-Chi Cheung, and Jian-Guang Lou. 2022. Nufix: Escape from NuGet dependency maze. In Proceedings of the 44th International Conference on Software Engineering (Pittsburgh, Pennsylvania) (ICSE \u201922). Association for Computing Machinery, New York, NY, USA, 1545\u20131557."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510142"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417051"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2012.031"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1002\/j.2158-1592.2001.tb00001.x"},{"key":"e_1_3_2_61_2","volume-title":"Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC\u201906)","author":"Passonneau Rebecca","year":"2006","unstructured":"Rebecca Passonneau. 2006. Measuring agreement on set-valued items (MASI) for semantic and pragmatic annotation. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC\u201906). European Language Resources Association (ELRA), Genoa, Italy."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468589"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09914-8"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884833"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6430"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510199"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510205"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-41695-z"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236062"},{"key":"e_1_3_2_70_2","first-page":"1394","volume-title":"2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Vu Duc-Ly","year":"2021","unstructured":"Duc-Ly Vu. 2021. py2src: Towards the automatic (and reliable) identification of sources for PyPI package. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1394\u20131396."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468592"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSPW51379.2020.00074"},{"key":"e_1_3_2_73_2","series-title":"ICSE \u201920","first-page":"125","volume-title":"Proceedings of the 42nd International Conference on Software Engineering","author":"Wang Ying","year":"2020","unstructured":"Ying Wang, Ming Wen, Yepang Liu, Yibo Wang, Zhenming Li, Chao Wang, Hai Yu, Shing-Chi Cheung, Chang Xu, and Zhiliang Zhu. 2020. Watchman: Monitoring dependency conflicts for Python library ecosystem. In Proceedings of the 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE \u201920). Association for Computing Machinery, New York, NY, USA, 125\u2013135."},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2901743"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527919"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00051"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00175"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.107004"},{"key":"e_1_3_2_79_2","first-page":"331","volume-title":"2022 IEEE\/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)","author":"Zahan Nusrat","year":"2022","unstructured":"Nusrat Zahan, Thomas Zimmermann, Patrice Godefroid, Brendan Murphy, Chandra Maddila, and Laurie Williams. 2022. What are weak links in the npm supply chain?. In 2022 IEEE\/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 331\u2013340."},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380362"},{"key":"e_1_3_2_81_2","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1109\/ISSRE.2019.00020","volume-title":"2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)","author":"Zhang Tianyi","year":"2019","unstructured":"Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael Lyu, and Miryung Kim. 2019. An empirical study of common challenges in developing deep learning applications. In 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE). 104\u2013115."},{"key":"e_1_3_2_82_2","series-title":"ICSE \u201921","first-page":"359","volume-title":"Proceedings of the 43rd International Conference on Software Engineering","author":"Zhang Xiaoyu","year":"2021","unstructured":"Xiaoyu Zhang, Juan Zhai, Shiqing Ma, and Chao Shen. 2021. AutoTrainer: An automatic DNN training problem detection and repair system. In Proceedings of the 43rd International Conference on Software Engineering (Madrid, Spain) (ICSE \u201921). IEEE Press, 359\u2013371."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2946156"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3640336","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3640336","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:26Z","timestamp":1750287026000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3640336"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,18]]},"references-count":83,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3640336"],"URL":"https:\/\/doi.org\/10.1145\/3640336","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,18]]},"assertion":[{"value":"2023-06-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-18","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}