{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T23:26:33Z","timestamp":1780961193957,"version":"3.54.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,4,25]],"date-time":"2024-04-25T00:00:00Z","timestamp":1714003200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>Immersive technologies like eXtended Reality (XR) are the next step in videoconferencing. In this context, understanding the effect of delay on communication is crucial. This article presents the first study on the impact of delay on collaborative tasks using a realistic Social XR system. Specifically, we design an experiment and evaluate the impact of end-to-end delays of 300, 600, 900, 1,200, and 1,500 ms on the execution of a standardized task involving the collaboration of two remote users that meet in a virtual space and construct block-based shapes. To measure the impact of the delay in this communication scenario, objective and subjective data were collected. As objective data, we measured the time required to execute the tasks and computed conversational characteristics by analyzing the recorded audio signals. As subjective data, a questionnaire was prepared and completed by every user to evaluate different factors such as overall quality, perception of delay, annoyance using the system, level of presence, cybersickness, and other subjective factors associated with social interaction. The results show a clear influence of the delay on the perceived quality and a significant negative effect as the delay increases. Specifically, the results indicate that the acceptable threshold for end-to-end delay should not exceed 900 ms. This article additionally provides guidelines for developing standardized XR tasks for assessing interaction in Social XR environments.<\/jats:p>","DOI":"10.1145\/3651164","type":"journal-article","created":{"date-parts":[[2024,3,6]],"date-time":"2024-03-06T12:07:03Z","timestamp":1709726823000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["Delay Threshold for Social Interaction in Volumetric eXtended Reality Communication"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8867-0559","authenticated-orcid":false,"given":"Carlos","family":"Cort\u00e9s","sequence":"first","affiliation":[{"name":"Universidad Polit\u00e9cnica de Madrid, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8990-2665","authenticated-orcid":false,"given":"Irene","family":"Viola","sequence":"additional","affiliation":[{"name":"Centrum Wiskunde &amp; Informatica, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7878-4712","authenticated-orcid":false,"given":"Jes\u00fas","family":"Guti\u00e9rrez","sequence":"additional","affiliation":[{"name":"Universidad Polit\u00e9cnica de Madrid, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7006-2560","authenticated-orcid":false,"given":"Jack","family":"Jansen","sequence":"additional","affiliation":[{"name":"Centrum Wiskunde &amp; Informatica, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3653-7630","authenticated-orcid":false,"given":"Shishir","family":"Subramanyam","sequence":"additional","affiliation":[{"name":"Centrum Wiskunde &amp; Informatica, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5561-9711","authenticated-orcid":false,"given":"Evangelos","family":"Alexiou","sequence":"additional","affiliation":[{"name":"Netherlands Organisation for Applied Scientific Research, Den Haag, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3502-6791","authenticated-orcid":false,"given":"Pablo","family":"P\u00e9rez","sequence":"additional","affiliation":[{"name":"eXtended Reality Labs, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0397-894X","authenticated-orcid":false,"given":"Narciso","family":"Garc\u00eda","sequence":"additional","affiliation":[{"name":"Universidad Polit\u00e9cnica de Madrid, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1752-6837","authenticated-orcid":false,"given":"Pablo","family":"C\u00e9sar","sequence":"additional","affiliation":[{"name":"Centrum Wiskunde &amp; Informatica and the Delft University of Technology, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,25]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3212804"},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-319-58475-1_1","volume-title":"Engineering Psychology and Cognitive Ergonomics: Cognition and Design","author":"Attig Christiane","year":"2017","unstructured":"Christiane Attig, Nadine Rauh, Thomas Franke, and Josef F. Krems. 2017. System latency guidelines then and now\u2014Is zero latency really considered necessary? In Engineering Psychology and Cognitive Ergonomics: Cognition and Design, Don Harris (Ed.). Springer International Publishing, Cham, 3\u201314."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934556"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10055-019-00395-9"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.33"},{"key":"e_1_3_1_7_2","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/PV.2012.6229740","volume-title":"Proceedings of the 2012 19th International Packet Video Workshop (PV\u201912)","author":"Berndtsson Gunilla","year":"2012","unstructured":"Gunilla Berndtsson, Mats Folkesson, and Valentin Kulyk. 2012. Subjective quality assessment of video conferences and telemeetings. In Proceedings of the 2012 19th International Packet Video Workshop (PV\u201912). 25\u201330. DOI:10.1109\/PV.2012.6229740"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.image.2020.116005"},{"key":"e_1_3_1_9_2","volume-title":"Qualinet White Paper on Definitions of Quality of Experience","author":"Brunnstr\u00f6m Kjell","year":"2013","unstructured":"Kjell Brunnstr\u00f6m, Katrien De Moor, Ann Dooms, Sebastian Egger-Lampl, Marie-Neige Garcia, Tobias Hossfeld, Satu Jumisko-Pyykk\u00f6, Christian Keimel, Chaker Larabi, Bob Lawlor, Patrick Le Callet, Sebastian M\u00f6ller, Fernando Pereira, Manuela Pereira, Andrew Perkis, Antonio Pinheiro, Ulrich Reiter, Peter Reichl, Raimund Schatz, and Andrej Zgank. 2013. Qualinet White Paper on Definitions of Quality of Experience. Qualinet."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3079711"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/VRW52623.2021.00014"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10772-006-6368-3"},{"key":"e_1_3_1_13_2","first-page":"3131","volume-title":"Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP\u201922)","author":"Cort\u00e9s Carlos","year":"2022","unstructured":"Carlos Cort\u00e9s, Jes\u00fas Guti\u00e9rrez, Pablo P\u00e9rez, Irene Viola, Pablo C\u00e9sar, and Narciso Garc\u00eda. 2022. Impact of self-view latency on quality of experience: Analysis of natural interaction in XR environments. In Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP\u201922). 3131\u20133135. DOI:10.1109\/ICIP46576.2022.9897983"},{"key":"e_1_3_1_14_2","first-page":"281","volume-title":"Proceedings of the 2019 IEEE 9th International Conference on Consumer Electronics (ICCE-Berlin\u201919)","author":"Cort\u00e9s C.","year":"2019","unstructured":"C. Cort\u00e9s, P. P\u00e9rez, and N. Garc\u00eda. 2019. Unity3D-based app for 360VR subjective quality assessment with customizable questionnaires. In Proceedings of the 2019 IEEE 9th International Conference on Consumer Electronics (ICCE-Berlin\u201919). 281\u2013282. DOI:10.1109\/ICCE-Berlin47944.2019.8966170"},{"key":"e_1_3_1_15_2","first-page":"1","volume-title":"Proceedings of the 2020 12th International Conference on Quality of Multimedia Experience (QoMEX\u201920)","author":"Cort\u00e9s Carlos","year":"2020","unstructured":"Carlos Cort\u00e9s, Pablo P\u00e9rez, Jes\u00fas Guti\u00e9rrez, and Narciso Garc\u00eda. 2020. Influence of video delay on quality, presence, and sickness in viewport adaptive immersive streaming. In Proceedings of the 2020 12th International Conference on Quality of Multimedia Experience (QoMEX\u201920). 1\u20134. DOI:10.1109\/QoMEX48832.2020.9123114"},{"key":"e_1_3_1_16_2","volume-title":"Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Garg Shrikant","year":"2022","unstructured":"Shrikant Garg, Ayushi Srivastava, Mashhuda Glencross, and Ojaswa Sharma. 2022. A study of the effects of network latency on visual task performance in video conferencing. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems(CHI EA\u201922). ACM, New York, NY, USA, Article 213, 7 pages. DOI:10.1145\/3491101.3519678"},{"key":"e_1_3_1_17_2","volume-title":"SPSS for Windows Step by Step: A Simple Study Guide and Reference, 17.0 Update, 10\/e","author":"George Darren","year":"2011","unstructured":"Darren George. 2011. SPSS for Windows Step by Step: A Simple Study Guide and Reference, 17.0 Update, 10\/e. Pearson Education India."},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1145\/3210825.3213566","volume-title":"Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video","author":"Gunkel Simon","year":"2018","unstructured":"Simon Gunkel, Hans Stokking, Martin Prins, Omar Niamut, Ernestasia Siahaan, and Pablo Cesar. 2018. Experiencing virtual reality together: Social VR use case study. In Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video(TVX\u201918). ACM, New York, NY, USA, 233\u2013238. DOI:10.1145\/3210825.3213566"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2593778"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3093717"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11747-022-00908-0"},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1109\/ISMAR-Adjunct54149.2021.00104","volume-title":"Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct\u201921)","author":"Kachach Redouane","year":"2021","unstructured":"Redouane Kachach, Sandra Morcuende, Diego Gonzalez-Morin, Pablo Perez-Garcia, Ester Gonzalez-Sosa, Francisco Pereira, and Alvaro Villegas. 2021. The Owl: Immersive telepresence communication for hybrid conferences. In Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct\u201921). 451\u2013452. DOI:10.1109\/ISMAR-Adjunct54149.2021.00104"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1145\/3292147.3292203","volume-title":"Proceedings of the 30th Australian Conference on Computer-Human Interaction","author":"Lawrence Louise","year":"2018","unstructured":"Louise Lawrence, Arindam Dey, and Mark Billinghurst. 2018. The effect of video placement in AR conferencing applications. In Proceedings of the 30th Australian Conference on Computer-Human Interaction(OzCHI\u201918). ACM, New York, NY, USA, 453\u2013457. DOI:10.1145\/3292147.3292203"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1177\/154193128903301812"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-32-391755-1.00028-6"},{"key":"e_1_3_1_26_2","first-page":"284","volume-title":"Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR\u201921)","author":"Li Jie","year":"2021","unstructured":"Jie Li, Shishir Subramanyam, Jack Jansen, Yanni Mei, Ignacio Reimat, Kinga \u0141awicka, and Pablo Cesar. 2021. Evaluating the user experience of a photorealistic social VR movie. In Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR\u201921). 284\u2013293. DOI:10.1109\/ISMAR52148.2021.00044"},{"key":"e_1_3_1_27_2","article-title":"2019 Impact of Video Conferencing Report","year":"2019","unstructured":"Lifesize. 2019. 2019 Impact of Video Conferencing Report. Industrial Report. Retrieved March 15, 2024 from https:\/\/blog.tmcnet.com\/blog\/rich-tehrani\/wp-content\/uploads\/2019\/09\/2019-Impact-of-Video-Conferencing-Report-Lifesize_FINAL.pdf","journal-title":"Industrial Report"},{"key":"e_1_3_1_28_2","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1145\/2483977.2483980","volume-title":"Proceedings of the 4th ACM Multimedia Systems Conference","author":"Mekuria Rufael","year":"2013","unstructured":"Rufael Mekuria, Michele Sanna, Stefano Asioli, Ebroul Izquierdo, Dick C. A. Bulterman, and Pablo Cesar. 2013. A 3D tele-immersion system based on live captured mesh geometry. In Proceedings of the 4th ACM Multimedia Systems Conference(MMSys\u201913). ACM, New York, NY, USA, 24\u201335. DOI:10.1145\/2483977.2483980"},{"key":"e_1_3_1_29_2","volume-title":"OBS Studio","unstructured":"OBS. n.d. OBS Studio. Retrieved March 15, 2024 from https:\/\/github.com\/obsproject\/obs-studio"},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1145\/2984511.2984517","volume-title":"Proceedings of the 29th Annual Symposium on User Interface Software and Technology","year":"2016","unstructured":"Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, Sameh Khamis, Mingson Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cern Keskin, and Shahram Izadi. 2016. Holoportation: Virtual 3D teleportation in real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology(UIST\u201916). ACM, New York, NY, USA, 741\u2013754. DOI:10.1145\/2984511.2984517"},{"key":"e_1_3_1_31_2","first-page":"179","volume-title":"Proceedings of the 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR\u201921)","year":"2021","unstructured":"Pablo P\u00e9rez, Ester Gonzalez-Sosa, Redouane Kachach, Francisco Pereira, and Alvaro Villegas. 2021. Ecological validity through gamification: An experiment with a mixed reality escape room. In Proceedings of the 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR\u201921). 179\u2013183. DOI:10.1109\/AIVR52153.2021.00040"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.3389\/frsip.2022.917684"},{"key":"e_1_3_1_33_2","unstructured":"ITU. 2021. Relative Timing of Sound and Vision for Broadcasting. Rec. ITU-R BT.1359. ITU."},{"key":"e_1_3_1_34_2","unstructured":"ITU. 2023. Methodology for the Subjective Assessment of the Quality of Television Pictures. Rec. ITU-R BT.500-15. ITU."},{"key":"e_1_3_1_35_2","unstructured":"RITU. 2021. Influencing Factors on Quality of Experience for Virtual Reality Services. Rec. ITU-T G.1035. ITU."},{"key":"e_1_3_1_36_2","unstructured":"ITU. 2015. The E-Model: A Computational Model for Use in Transmission Planning. Rec. ITU-T G.107. ITU."},{"key":"e_1_3_1_37_2","unstructured":"ITU. 2017. Subjective Quality Evaluation of Audio and Audiovisual Multiparty Telemeetings. Rec. ITU-T P.1301. ITU."},{"key":"e_1_3_1_38_2","unstructured":"ITU. 2016. Effect of Delays on Telemeeting Quality. Rec. ITU-T P.1305. ITU."},{"key":"e_1_3_1_39_2","unstructured":"ITU. 2016. Method for the Measurement of the Communication Effectiveness of Multiparty Telemeetings Using Task Performance. Rec. ITU-T P.1312. ITU."},{"key":"e_1_3_1_40_2","unstructured":"ITU. 1993. Objective Measurement of Active Speech Level. Rec. ITU-T P.56. ITU."},{"key":"e_1_3_1_41_2","unstructured":"ITU. 2008. Subjective Video Quality Assessment Methods for Multimedia Applications. Rec. ITU-T P.910. ITU."},{"key":"e_1_3_1_42_2","unstructured":"ITU. 2000. Interactive Test Methods for Audiovisual Communications. Rec. ITU-T P.920. ITU."},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.3389\/FRVIR.2021.692641"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2017.2777466"},{"key":"e_1_3_1_45_2","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1145\/2661126.2661135","volume-title":"Proceedings of the 3rd International Workshop on Socially-Aware Multimedia","author":"Schmitt Marwin","year":"2014","unstructured":"Marwin Schmitt, Simon Gunkel, Pablo Cesar, and Dick Bulterman. 2014. The influence of interactivity patterns on the quality of experience in multi-party video-mediated conversations under symmetric delay conditions. In Proceedings of the 3rd International Workshop on Socially-Aware Multimedia(SAM\u201914). ACM, New York, NY, USA, 13\u201316. DOI:10.1145\/2661126.2661135"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2014.02.004"},{"key":"e_1_3_1_47_2","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/QoMEX.2014.6982282","volume-title":"Proceedings of the 2014 6th International Workshop on Quality of Multimedia Experience (QoMEX\u201914)","author":"Schoenenberg K.","year":"2014","unstructured":"K. Schoenenberg, A. Raake, and P. Lebreton. 2014. Conversational quality and visual interaction of video-telephony under synchronous and asynchronous transmission delay. In Proceedings of the 2014 6th International Workshop on Quality of Multimedia Experience (QoMEX\u201914). 31\u201336. DOI:10.1109\/QoMEX.2014.6982282"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/j.pragma.2020.11.005","article-title":"Whose turn is it anyway? Latency and the organization of turn-taking in video-mediated interaction","volume":"172","author":"Seuren Lucas M.","year":"2021","unstructured":"Lucas M. Seuren, Joseph Wherton, Trisha Greenhalgh, and Sara E. Shaw. 2021. Whose turn is it anyway? Latency and the organization of turn-taking in video-mediated interaction. Journal of Pragmatics 172 (Jan.2021), 63\u201378.","journal-title":"Journal of Pragmatics"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3176369"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2520090"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2212776.2223750"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-32-391755-1.00021-3"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2023.3263943"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1093\/iwcomp\/iwaa012"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICBDIE52740.2021.00162"},{"key":"e_1_3_1_56_2","first-page":"365","volume-title":"Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP\u201916)","author":"Zioulis Nikolaos","year":"2016","unstructured":"Nikolaos Zioulis, Dimitrios Alexiadis, Alexandros Doumanoglou, Georgios Louizis, Konstantinos Apostolakis, Dimitrios Zarpalas, and Petros Daras. 2016. 3D tele-immersion platform for interactive immersive experiences between remote users. In Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP\u201916). 365\u2013369. DOI:10.1109\/ICIP.2016.7532380"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3651164","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3651164","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:49:53Z","timestamp":1750286993000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3651164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,25]]},"references-count":55,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3651164"],"URL":"https:\/\/doi.org\/10.1145\/3651164","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,25]]},"assertion":[{"value":"2023-10-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-15","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}