{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T11:35:57Z","timestamp":1784547357389,"version":"3.55.0"},"reference-count":88,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T00:00:00Z","timestamp":1726185600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nd\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digit. Gov.: Res. Pract."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>This paper investigates the challenges of designing a computer supported collaborative environment aimed at facilitating co-production processes, i.e., those collaborative processes between Public Administrations, private stakeholders and citizens that aim at the design of public services, their implementation, and their shared delivery to the community. We argue that, for such a digital platform, different types of socio-technical requirements should be considered, i.e., those related to governance models and associated collaboration dynamics; requirements that may emerge from the specific type of public service to design; as well as user and technical requirements common to all e-government platforms. This research informed the development and testing of a digital collaboration platform that offers guidance on how to organize a co-production initiative and a network of stakeholders, functionalities to support collaborative work, and enablers (in the form of reusable knowledge and digital resources) to perform the sequence of steps to produce a public service. The lessons learned from the iterative platform development process and a preliminary evaluation study conducted with three Public Administrations in different European countries pointed at specific functionalities that are perceived as most crucial and at different appropriation practices that depend on the organizational structure of the involved Public Administrations and related multi-stakeholder networks. The innovation that is brought about with respect to general-purpose platforms for computer supported cooperative work is represented by the operationalization of co-production processes, with step-by-step guidance and potential reuse (with adaptation) of ready to use resources and processes. Based on the results of our research, general guidelines are also proposed for the design of future digital platforms supporting co-production.<\/jats:p>","DOI":"10.1145\/3664612","type":"journal-article","created":{"date-parts":[[2024,5,13]],"date-time":"2024-05-13T11:09:11Z","timestamp":1715598551000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Designing a Digital Environment to Support the Co-production of Public Services: Balancing Multiple Requirements and Governance Concepts"],"prefix":"10.1145","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2067-4390","authenticated-orcid":false,"given":"Elena","family":"Not","sequence":"first","affiliation":[{"name":"Fondazione Bruno Kessler, Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6142-7038","authenticated-orcid":false,"given":"Chiara","family":"Leonardi","sequence":"additional","affiliation":[{"name":"Fondazione Bruno Kessler, Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8055-6823","authenticated-orcid":false,"given":"Diego","family":"L\u00f3pez-De-Ipi\u00f1a","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Deusto, Bilbao, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7240-590X","authenticated-orcid":false,"given":"Daniel","family":"Silva Palacios","sequence":"additional","affiliation":[{"name":"DeustoTech, University of Deusto, Bilbao, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6361-0901","authenticated-orcid":false,"given":"Ruben","family":"S\u00e1nchez-Corcuera","sequence":"additional","affiliation":[{"name":"DeustoTech, University of Deusto, Bilbao, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8502-2396","authenticated-orcid":false,"given":"Raman","family":"Kazhamiakin","sequence":"additional","affiliation":[{"name":"Digital Society Center, Fondazione Bruno Kessler, Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7741-0436","authenticated-orcid":false,"given":"Matteo","family":"Gerosa","sequence":"additional","affiliation":[{"name":"Digital Society Center, Fondazione Bruno Kessler, Trento, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,9,13]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1080\/14719037.2013.806578"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.3390\/admsci10020025"},{"key":"e_1_3_3_4_2","article-title":"Guidelines on the acquisition and reuse of software for public administrations","author":"Bajo Giovanni","year":"2020","unstructured":"Giovanni Bajo, Viviana De Paola, Daniela Intravaia, Guido Pera, Umberto Rosini, and Guido Scorza. 2020. Guidelines on the acquisition and reuse of software for public administrations. Italia. Agency for Digital Italy and the Digital Transformation Team. https:\/\/docs.italia.it\/italia\/developers-italia\/gl-acquisition-and-reuse-software-for-pa-docs\/en\/stabile\/index.html (accessed 21st February 2023).","journal-title":"Italia. Agency for Digital Italy and the Digital Transformation Team"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2018.01.016"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2010.07.003"},{"key":"e_1_3_3_7_2","article-title":"Advanced governance model. Deliverable D2.2","author":"Brandsen Taco","year":"2023","unstructured":"Taco Brandsen, Noortje Hoevens, Marlies Honingh, Ina Radtke, and Sawssane Bargach. 2023. Advanced governance model. Deliverable D2.2. Interlink project, Horizon 2020 - DT-GOVERNANCE-05-2020 - Grant Agreement No 959201","journal-title":"Interlink project, Horizon 2020 - DT-GOVERNANCE-05-2020 - Grant Agreement No 959201"},{"key":"e_1_3_3_8_2","article-title":"Preliminary governance model. Deliverable D2.1","author":"Brandsen Taco","year":"2022","unstructured":"Taco Brandsen, Noortje Hoevens, Marlies Honingh, Ina Radtke, Chiara Leonardi, Elena Not, Pauli Misikangas, Diego L\u00f3pez de Ipi\u00f1a, Ana Ortiz de Guinea Lopez de Arana. 2022. Preliminary governance model. Deliverable D2.1. Interlink project, Horizon 2020 - DT-GOVERNANCE-05-2020 - Grant Agreement No 959201. https:\/\/zenodo.org\/records\/10668560","journal-title":"Interlink project, Horizon 2020 - DT-GOVERNANCE-05-2020 - Grant Agreement No 959201"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1111\/puar.12465"},{"key":"e_1_3_3_10_2","doi-asserted-by":"crossref","unstructured":"Taco Brandsen and Marlies Honingh. 2018. Definitions of co-production and co-creation. In: T. Brandsen B. Verschuere and T. Steen (2018). Co-production and co-creation: Engaging citizens in public service delivery 9\u201317. New York London: Routledge.","DOI":"10.4324\/9781315204956-2"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0953-5438(00)00024-2"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11266-012-9309-6"},{"key":"e_1_3_3_13_2","volume-title":"NESTA, New Economics Foundation","author":"Boyle David","year":"2009","unstructured":"David Boyle and Michael Harris. 2009. The challenge of co-production: How equal partnerships between professionals and the public are crucial to improving public service. NESTA, New Economics Foundation, The Lab, London."},{"key":"e_1_3_3_14_2","article-title":"Log design for accountability","author":"Butin Denis","year":"2013","unstructured":"Denis Butin, Marcos Chicote, and Daniel Le M\u00e9tayer. 2013. Log design for accountability. DUMA13 - 4th International Workshop on Data Usage Management, May 2013, San Francisco, CA, USA.","journal-title":"DUMA13 - 4th International Workshop on Data Usage Management"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/517699"},{"key":"e_1_3_3_16_2","first-page":"12","article-title":"Privacy by design: The 7 foundational principles","volume":"5","author":"Cavoukian Ann","year":"2009","unstructured":"Ann Cavoukian. 2009. Privacy by design: The 7 foundational principles. Information and Privacy Commissioner of Ontario, Canada 5, (2009), 12.","journal-title":"Information and Privacy Commissioner of Ontario, Canada"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.3233\/IP-190122"},{"key":"e_1_3_3_18_2","article-title":"About Face 3: The essentials of interaction design","author":"Cooper Alan","year":"2007","unstructured":"Alan Cooper, Robert Reimann, and David Cronin. 2007. About Face 3: The essentials of interaction design. Wiley.","journal-title":"Wiley"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1108\/DPRG-02-2021-0028"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/0-306-47374-7_16"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1080\/13511610.2017.1348936"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.14236\/ewic\/HCI2007.53"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9248.1996.tb00334.x"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.3390\/admsci11010022"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.3390\/admsci13020043"},{"key":"e_1_3_3_26_2","unstructured":"European Commission. 2016. EU eGovernment action plan 2016-2020: Accelerating the digital transformation of government. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=CELEX%3A52016DC0179 (accessed on the 21st of February 2023)"},{"key":"e_1_3_3_27_2","unstructured":"European Commission. 2017. European interoperability framework \u2013 implementation strategy. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=COM:2017:134:FIN (accessed on the 21st of February 2023)."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","unstructured":"European Commission. 2018. Directorate-general for employment social affairs and inclusion. Co-production: Enhancing the Role of Citizens in Governance and Service Delivery. Technical Dossier no. 4 May 2018 Publications Office 10.2767\/128036 (accessed on the 12th of July 2023).","DOI":"10.2767\/128036"},{"key":"e_1_3_3_29_2","unstructured":"European Commission. 2023. About the Digital Europe Programme. https:\/\/ec.europa.eu\/digital-building-blocks\/wikis\/display\/DIGITAL\/About+us (accessed on the 3rd November 2023)"},{"key":"e_1_3_3_30_2","unstructured":"European Union. 2014. Regulation (EU) No 910\/2014 of the European Parliament and of the Council of 23 July 2014 on electronic identification and trust services for electronic transactions in the internal market and repealing Directive 1999\/93\/EC https:\/\/eur-lex.europa.eu\/eli\/reg\/2014\/910\/oj (accessed on the 12th of July 2023)."},{"key":"e_1_3_3_31_2","unstructured":"European Union. 2016. Regulation (EU) 2016\/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data and repealing Directive 95\/46\/EC (General Data Protection Regulation). https:\/\/eur-lex.europa.eu\/eli\/reg\/2016\/679\/2016-05-04 (accessed on the 3rd of November 2023)."},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-30558-5"},{"issue":"2","key":"e_1_3_3_33_2","first-page":"1","article-title":"E-government in digital era: Concept, practice, and development","volume":"10","author":"Fang Zhiyuan","year":"2002","unstructured":"Zhiyuan Fang. 2002. E-government in digital era: Concept, practice, and development. International Journal of the Computer, the Internet and Management 10, 2 (2002), 1\u201322.","journal-title":"International Journal of the Computer, the Internet and Management"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315204956-9"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40900-021-00259-0"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.giq.2016.01.006"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/2.291294"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315204956-25"},{"key":"e_1_3_3_39_2","volume-title":"Paper presented at the NUS-FPZG UNESCO Chair Workshop on The Governance of Collaboration: Co-Production, Contracting, Commissioning and Certification","author":"Howlett Michael","year":"2017","unstructured":"Michael Howlett and Michael Ramesh. 2017. The Achilles heels of collaboration: Overcoming critical capacity deficits in collaborative governance arrangements? Paper presented at the NUS-FPZG UNESCO Chair Workshop on The Governance of Collaboration: Co-Production, Contracting, Commissioning and Certification, Dubrovnik, Croatia."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.giq.2015.07.001"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISC2.2015.7366196"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.3390\/info8020047"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jart.2015.06.001"},{"key":"e_1_3_3_44_2","article-title":"Kanban and scrum-making the most of both","author":"Kniberg Henrik","year":"2010","unstructured":"Henrik Kniberg and Mattias Skarin. 2010. Kanban and scrum-making the most of both. InfoQ.","journal-title":"InfoQ"},{"key":"e_1_3_3_45_2","unstructured":"International Standards Organisation. 2019. ISO 9241-210:2019(en) Ergonomics of human-system interaction \u2014 Part 210: Human-centred design for interactive systems. ISO Geneva Switzerland. https:\/\/www.iso.org\/obp\/ui\/en\/#iso:std:iso:9241:-210:ed-2:v1:en"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/0160-7383(94)00067-3"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315204956-33"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1080\/14719037.2019.1619807"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543434.3543462"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3593743.3593747"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2037556.2037581"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.23919\/SpliTech55088.2022.9854297"},{"key":"e_1_3_3_53_2","unstructured":"Diego L\u00f3pez de Ipi\u00f1a Roberto Carballedo Igone Porto Daniel Silva Julen Badiola Elena Not Chiara Leonardi Angelo Mazzoli Michele Cappuccio Mat\u012bss Veigurs Ana Jim\u00e9nez Raquel Povar Noortje Hoevens Serge Sushkov and Pauli Misikangas. 2022. Use-case deployment and operation report v1. INTERLINK project. Deliverable D5.3. Interlink project Horizon 2020 - DT-GOVERNANCE-05-2020 - Grant Agreement No 959201"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-6210.2011.02391.x"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1332\/030557321X16115951032181"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1111\/puar.12765"},{"key":"e_1_3_3_57_2","unstructured":"Nesta and IDEO. 2017. Designing for public services: A practical guide. https:\/\/www.nesta.org.uk\/toolkit\/designing-for-public-services-a-practical-guide\/accessed17thFebruary2023"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1080\/10967494.2021.1937413"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/191666.191729"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/97243.97281"},{"key":"e_1_3_3_61_2","volume-title":"McGinnis, ed., Polycentric Governance and Development","author":"Ostrom Elinor","year":"1999","unstructured":"Elinor Ostrom. 1999. Crossing the great divide: Coproduction, synergy, and development, Ch. 15 in M.D. McGinnis, ed., Polycentric Governance and Development. Readings from the Workshop in Political Theory and Policy Analysis, Univ. of Michigan Press, Ann Arbor, MI."},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-40265-9_10"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503823.3503901"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222240302"},{"issue":"3","key":"e_1_3_3_65_2","first-page":"27","article-title":"The barriers and enablers of co-design for services","volume":"10","author":"Pirinen Antti","year":"2016","unstructured":"Antti Pirinen. 2016. The barriers and enablers of co-design for services. International Journal of Design 10, 3 (2016), 27\u201342.","journal-title":"International Journal of Design"},{"issue":"3","key":"e_1_3_3_66_2","first-page":"1","article-title":"Design enters the city: Requisites and points of friction in deepening public sector design","volume":"16","author":"Pirinen Antti","year":"2022","unstructured":"Antti Pirinen, Kaisa Savolainen, Sampsa Hyysalo, and Tuuli Mattelm\u00e4ki. 2022. Design enters the city: Requisites and points of friction in deepening public sector design. International Journal of Design 16, 3 (2022), 1\u201319.","journal-title":"International Journal of Design"},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.3390\/admsci13030069"},{"key":"e_1_3_3_68_2","article-title":"Mastering the requirements process: Getting requirements right","author":"Robertson Suzanne","year":"2012","unstructured":"Suzanne Robertson and James Robertson. 2012. Mastering the requirements process: Getting requirements right. Addison-Wesley.","journal-title":"Addison-Wesley"},{"key":"e_1_3_3_69_2","article-title":"Usability engineering: Scenario-based development of human-computer interaction","author":"Rosson Mary Beth","year":"2002","unstructured":"Mary Beth Rosson and John M. Carroll. 2002. Usability engineering: Scenario-based development of human-computer interaction. Morgan Kaufmann.","journal-title":"Morgan Kaufmann"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376770"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.22215\/timreview\/1355"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.giq.2021.101650"},{"key":"e_1_3_3_73_2","article-title":"Enabling social innovation: Ecosystems for community-led territorial development","author":"Sgaragli Fabio","year":"2014","unstructured":"Fabio Sgaragli (Ed.). 2014. Enabling social innovation: Ecosystems for community-led territorial development. Quaderni della Fondazione Giacomo Brodolini.","journal-title":"Quaderni della Fondazione Giacomo Brodolini"},{"key":"e_1_3_3_74_2","doi-asserted-by":"publisher","DOI":"10.1177\/0020852314566008"},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.23919\/SpliTech58164.2023.10193212"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.3897\/rio.8.e93937"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1080\/14494035.2018.1521676"},{"key":"e_1_3_3_78_2","unstructured":"Kaisa Spilling Maija Bergstr\u00f6m and Kubo Creative Agency. 2021. Life and innovations. Smart Kalasatama \u2013 A district for experimentation. https:\/\/forumvirium.fi\/en\/publication\/smartkalasatama-final-report\/(accessed 16th November 2023)"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315204956-45"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2017.08.095"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.9774\/GLEAF.4700.2005.au.00013"},{"key":"e_1_3_3_82_2","unstructured":"Pieter Raymaekers and Steven Van de Walle. 2019. Pleio: An open source collaboration platform for public administrations and public policy in The Netherlands. https:\/\/tropico-project.eu\/cases\/administration-costs-for-bureaucracy\/pleio-an-open-source-collaboration-platform-for-public-administrations-and-public-policy-in-the-netherlands\/(accessed 15th November 2023)"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1177\/0095399716680057"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1080\/14719037.2019.1619810"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1080\/0044929031000"},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.1080\/13645579.1998.10846874"},{"key":"e_1_3_3_87_2","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.11"},{"key":"e_1_3_3_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3325112.3325232"},{"key":"e_1_3_3_89_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-28244-0_2"}],"container-title":["Digital Government: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664612","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3664612","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:45Z","timestamp":1750291425000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3664612"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,13]]},"references-count":88,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3664612"],"URL":"https:\/\/doi.org\/10.1145\/3664612","relation":{},"ISSN":["2691-199X","2639-0175"],"issn-type":[{"value":"2691-199X","type":"print"},{"value":"2639-0175","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,13]]},"assertion":[{"value":"2023-07-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}