{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,18]],"date-time":"2025-12-18T14:20:35Z","timestamp":1766067635519,"version":"3.41.0"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2023,2,27]],"date-time":"2023-02-27T00:00:00Z","timestamp":1677456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2018YFC1603601"],"award-info":[{"award-number":["2018YFC1603601"]}]},{"name":"Program for Innovative Research Team in University of the Ministry of Education","award":["IRT_17R32"],"award-info":[{"award-number":["IRT_17R32"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62076085 and 62120106008"],"award-info":[{"award-number":["62076085 and 62120106008"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2023,10,30]]},"abstract":"<jats:p>Crowdsourcing truth inference aims to assign a correct answer to each task from candidate answers that are provided by crowdsourced workers. A common approach is to generate workers\u2019 reliabilities to represent the quality of answers. Although crowdsourced triples can be converted into various crowdsourced relationships, the available related methods are not effective in capturing these relationships to alleviate the harm to inference that is caused by conflicting answers. In this research, we propose a<jats:bold>Re<\/jats:bold>liability-driven<jats:bold>M<\/jats:bold>ulti-view<jats:bold>G<\/jats:bold>raph<jats:bold>E<\/jats:bold>mbedding framework for<jats:bold>T<\/jats:bold>ruth<jats:bold>i<\/jats:bold>nference (TiReMGE), which explores multiple crowdsourced relationships by organically integrating worker reliabilities into a graph space that is constructed from crowdsourced triples. Specifically, to create an interactive environment, we propose a reliability-driven initialization criterion for initializing vectors of tasks and workers as interactive carriers of reliabilities. From the perspective of multiple crowdsourced relationships, a multi-view graph embedding framework is proposed for reliability information interaction on a task-worker graph, which encodes latent crowdsourced relationships into vectors of workers and tasks for reliability update and truth inference. A heritable reliability updating method based on the Lagrange multiplier method is proposed to obtain reliabilities that match the quality of workers for interaction by a novel constraint law. Our ultimate goal is to minimize the Euclidean distance between the encoded task vector and the answer that is provided by a worker with high reliability. Extensive experimental results on nine real-world datasets demonstrate that TiReMGE significantly outperforms the nine state-of-the-art baselines.<\/jats:p>","DOI":"10.1145\/3565576","type":"journal-article","created":{"date-parts":[[2022,10,4]],"date-time":"2022-10-04T11:08:20Z","timestamp":1664881700000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Crowdsourcing Truth Inference via Reliability-Driven Multi-View Graph Embedding"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2289-1679","authenticated-orcid":false,"given":"Gongqing","family":"Wu","sequence":"first","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8349-3597","authenticated-orcid":false,"given":"Xingrui","family":"Zhuo","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1590-083X","authenticated-orcid":false,"given":"Xianyu","family":"Bao","sequence":"additional","affiliation":[{"name":"Shenzhen Academy of Inspection and Quarantine, Shenzhen, Guangdong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5421-6171","authenticated-orcid":false,"given":"Xuegang","family":"Hu","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5461-3986","authenticated-orcid":false,"given":"Richang","family":"Hong","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2396-1704","authenticated-orcid":false,"given":"Xindong","family":"Wu","sequence":"additional","affiliation":[{"name":"Hefei University of Technology, Hefei, Anhui, China"}]}],"member":"320","published-online":{"date-parts":[[2023,2,27]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"2946","volume-title":"Proceedings of the Association for the Advancement of Artificial Intelligence","author":"Aydin Bahadir Ismail","year":"2014","unstructured":"Bahadir Ismail Aydin, Yavuz Selim Yilmaz, Yaliang Li, Qi Li, Jing Gao, and Murat Demirbas. 2014. Crowdsourcing for multiple-choice question answering. In Proceedings of the Association for the Advancement of Artificial Intelligence. AAAI Press, 2946\u20132953."},{"key":"e_1_3_2_3_2","first-page":"1937","volume-title":"Proceedings of the Web Conference WWW","author":"Balayn Agathe","year":"2021","unstructured":"Agathe Balayn, Panagiotis Soilis, Christoph Lofi, Jie Yang, and Alessandro Bozzon. 2021. What do you mean? Interpreting image classification with crowdsourced concept extraction and analysis. In Proceedings of the Web Conference WWW. ACM, 1937\u20131948."},{"key":"e_1_3_2_4_2","volume-title":"Convex Optimization","author":"Boyd Stephen P.","year":"2014","unstructured":"Stephen P. Boyd and Lieven Vandenberghe. 2014. Convex Optimization. Cambridge University Press."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2807452"},{"key":"e_1_3_2_6_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Cao Peng","year":"2019","unstructured":"Peng Cao, Yilun Xu, Yuqing Kong, and Yizhou Wang. 2019. Max-MIG: An information theoretic approach for joint learning from crowds. In Proceedings of the International Conference on Learning Representations. OpenReview.net."},{"key":"e_1_3_2_7_2","first-page":"5832","volume-title":"Proceedings of the Association for the Advancement of Artificial Intelligence","author":"Chu Zhendong","year":"2021","unstructured":"Zhendong Chu, Jing Ma, and Hongning Wang. 2021. Learning from crowds by modeling common confusions. In Proceedings of the Association for the Advancement of Artificial Intelligence. AAAI Press, 5832\u20135840."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.2307\/2346806"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1145\/2187836.2187900","volume-title":"Proceedings of the International World Wide Web Conference","author":"Demartini Gianluca","year":"2012","unstructured":"Gianluca Demartini, Djellel Eddine Difallah, and Philippe Cudr\u00e9-Mauroux. 2012. ZenCrowd: Leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In Proceedings of the International World Wide Web Conference. ACM, 469\u2013478."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1977.tb01600.x"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482032"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687690"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989331"},{"key":"e_1_3_2_14_2","first-page":"1729","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence","author":"Gemalmaz Meric Altug","year":"2021","unstructured":"Meric Altug Gemalmaz and Ming Yin. 2021. Accounting for confirmation bias in crowdsourced label aggregation. In Proceedings of the International Joint Conference on Artificial Intelligence. ijcai.org, 1729\u20131735."},{"key":"e_1_3_2_15_2","first-page":"1904","volume-title":"Proceedings of the International World Wide Web Conference","author":"Hong Chi","year":"2021","unstructured":"Chi Hong, Amirmasoud Ghiassi, Yichi Zhou, Robert Birke, and Lydia Y. Chen. 2021. Online label aggregation: A variational Bayesian approach. In Proceedings of the International World Wide Web Conference. ACM, 1904\u20131915."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3478322"},{"key":"e_1_3_2_17_2","first-page":"1953","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Karger David R.","year":"2011","unstructured":"David R. Karger, Sewoong Oh, and Devavrat Shah. 2011. Iterative learning for reliable crowdsourcing systems. In Proceedings of the Advances in Neural Information Processing Systems. Curran Associates, Inc., 1953\u20131961."},{"key":"e_1_3_2_18_2","first-page":"1272","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence","author":"Kawase Yasushi","year":"2019","unstructured":"Yasushi Kawase, Yuko Kuroki, and Atsushi Miyauchi. 2019. Graph mining meets crowdsourcing: Extracting experts for answer aggregation. In Proceedings of the International Joint Conference on Artificial Intelligence. ijcai.org, 1272\u20131279."},{"key":"e_1_3_2_19_2","first-page":"619","volume-title":"AISTATS (JMLR Proceedings)","volume":"22","author":"Kim Hyun-Chul","year":"2012","unstructured":"Hyun-Chul Kim and Zoubin Ghahramani. 2012. Bayesian classifier combination. In AISTATS (JMLR Proceedings), Vol. 22. JMLR.org, 619\u2013627."},{"key":"e_1_3_2_20_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Kipf Thomas N.","year":"2017","unstructured":"Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the International Conference on Learning Representations. OpenReview.net."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.5555\/1795555"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403125"},{"key":"e_1_3_2_23_2","first-page":"1821","volume-title":"Proceedings of the International World Wide Web Conference","author":"Li Dan","year":"2021","unstructured":"Dan Li, Zhaochun Ren, and Evangelos Kanoulas. 2021. CrowdGP: A Gaussian process model for inferring relevance from crowd annotations. In Proceedings of the International World Wide Web Conference. ACM, 1821\u20131832."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482288"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.14778\/2735496.2735505"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610509"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2897350.2897352"},{"key":"e_1_3_2_28_2","first-page":"675","volume-title":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Li Yaliang","year":"2015","unstructured":"Yaliang Li, Qi Li, Jing Gao, Lu Su, Bo Zhao, Wei Fan, and Jiawei Han. 2015. On the discovery of evolving truth. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 675\u2013684."},{"key":"e_1_3_2_29_2","first-page":"3886","volume-title":"Proceedings of the International Conference on Machine Learning","volume":"97","author":"Li Yuan","year":"2019","unstructured":"Yuan Li, Benjamin I. P. Rubinstein, and Trevor Cohn. 2019. Exploiting worker correlation for label aggregation in crowdsourcing. In Proceedings of the International Conference on Machine Learning, Vol. 97. PMLR, 3886\u20133895."},{"key":"e_1_3_2_30_2","first-page":"1028","volume-title":"Proceedings of the International World Wide Web Conference","author":"Li Yuan","year":"2019","unstructured":"Yuan Li, Benjamin I. P. Rubinstein, and Trevor Cohn. 2019. Truth inference at scale: A Bayesian model for adjudicating highly redundant crowd annotations. In Proceedings of the International World Wide Web Conference. ACM, 1028\u20131038."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403102"},{"key":"e_1_3_2_32_2","first-page":"701","volume-title":"Proceedings of the Neural Information Processing Systems","author":"Liu Qiang","year":"2012","unstructured":"Qiang Liu, Jian Peng, and Alexander T. Ihler. 2012. Variational inference for crowdsourcing. In Proceedings of the Neural Information Processing Systems. Curran Associates, Inc., 701\u2013709."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3133069"},{"issue":"3","key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"1264","DOI":"10.1109\/TKDE.2019.2936189","article-title":"Truth discovery by claim and source embedding","volume":"33","author":"Lyu Shanshan","year":"2021","unstructured":"Shanshan Lyu, Wentao Ouyang, Yongqing Wang, Huawei Shen, and Xueqi Cheng. 2021. Truth discovery by claim and source embedding. IEEE Transactions on Knowledge and Data Engineering 33, 3 (2021), 1264\u20131275.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Tengfei Ma Cao Xiao Jiayu Zhou and Fei Wang. 2018. Drug similarity integration through attentive multi-view graph auto-encoders. In IJCAI . ijcai.org 3477\u20133483.","DOI":"10.24963\/ijcai.2018\/483"},{"key":"e_1_3_2_36_2","first-page":"211","volume-title":"Proceedings of the Conference on Innovative Data Systems Research","author":"Marcus Adam","year":"2011","unstructured":"Adam Marcus, Eugene Wu, Samuel Madden, and Robert C. Miller. 2011. Crowdsourced databases: Query processing with people. In Proceedings of the Conference on Innovative Data Systems Research. 211\u2013214. www.cidrdb.org."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2809695.2809715"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"1203","DOI":"10.1145\/2396761.2398421","volume-title":"Proceedings of the 21st ACM International Conference on Information and Knowledge Management","author":"Parameswaran Aditya G.","year":"2012","unstructured":"Aditya G. Parameswaran, Hyunjung Park, Hector Garcia-Molina, Neoklis Polyzotis, and Jennifer Widom. 2012. Deco: Declarative crowdsourcing. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management. ACM, 1203\u20131212."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2593674"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3103315"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/1951365.1951414"},{"key":"e_1_3_2_42_2","first-page":"857","volume-title":"Proceedings of the International World Wide Web Conference","author":"Shi Zheyuan Ryan","year":"2021","unstructured":"Zheyuan Ryan Shi, Leah Lizarondo, and Fei Fang. 2021. A recommender system for crowdsourcing food rescue platforms. In Proceedings of the International World Wide Web Conference. ACM, 857\u2013865."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357920"},{"issue":"5","key":"e_1_3_2_44_2","first-page":"2295","article-title":"Two-sided online micro-task assignment in spatial crowdsourcing","volume":"33","author":"Tong Yongxin","year":"2021","unstructured":"Yongxin Tong, Yuxiang Zeng, Bolin Ding, Libin Wang, and Lei Chen. 2021. Two-sided online micro-task assignment in spatial crowdsourcing. IEEE Trans. Knowl. Data Eng. 33, 5 (2021), 2295\u20132309.","journal-title":"IEEE Trans. Knowl. Data Eng."},{"issue":"3","key":"e_1_3_2_45_2","first-page":"1156","article-title":"Quality-assured synchronized task assignment in crowdsourcing","volume":"33","author":"Tu Jiayang","year":"2021","unstructured":"Jiayang Tu, Peng Cheng, and Lei Chen. 2021. Quality-assured synchronized task assignment in crowdsourcing. IEEE Transactions on Knowledge and Data Engineering 33, 3 (2021), 1156\u20131168.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_46_2","first-page":"155","volume-title":"Proceedings of the International World Wide Web Conference","author":"Venanzi Matteo","year":"2014","unstructured":"Matteo Venanzi, John Guiver, Gabriella Kazai, Pushmeet Kohli, and Milad Shokouhi. 2014. Community-based Bayesian aggregation models for crowdsourcing. In Proceedings of the International World Wide Web Conference. ACM, 155\u2013164."},{"key":"e_1_3_2_47_2","first-page":"44","volume-title":"Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing","author":"Venanzi Matteo","year":"2015","unstructured":"Matteo Venanzi, Oliver Parson, Alex Rogers, and Nick R. Jennings. 2015. The ActiveCrowdToolkit: An open-source tool for benchmarking active learning algorithms for crowdsourcing research. In Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing. AAAI Press, 44\u201345."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000001"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.5555\/2602339.2602344"},{"key":"e_1_3_2_50_2","first-page":"2946","volume-title":"Proceedings of the International World Wide Web Conference","author":"Wang Ping","year":"2021","unstructured":"Ping Wang, Khushbu Agarwal, Colby Ham, Sutanay Choudhury, and Chandan K. Reddy. 2021. Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks. In Proceedings of the International World Wide Web Conference. ACM, 2946\u20132957."},{"key":"e_1_3_2_51_2","first-page":"202","volume-title":"Proceedings of the 13th International Symposium on Information Processing in Sensor Networks","author":"Wang Shiguang","year":"2015","unstructured":"Shiguang Wang, Lu Su, Shen Li, Shaohan Hu, Md. Tanvir Al Amin, Hongwei Wang, Shuochao Yao, Lance M. Kaplan, and Tarek F. Abdelzaher. 2015. Scalable social sensing of interdependent phenomena. In Proceedings of the 13th International Symposium on Information Processing in Sensor Networks. ACM, 202\u2013213."},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1145\/3394486.3403052","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Wang Yue","year":"2020","unstructured":"Yue Wang, Ke Wang, and Chunyan Miao. 2020. Truth discovery against strategic sybil attack in crowdsourcing. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 95\u2013104."},{"key":"e_1_3_2_53_2","unstructured":"Jacob Whitehill Paul Ruvolo Tingfan Wu Jacob Bergsma and Javier R. Movellan. 2009. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In NIPS . Curran Associates Inc. 2035\u20132043."},{"issue":"2","key":"e_1_3_2_54_2","first-page":"27:1\u201327:18","article-title":"Exploiting heterogeneous graph neural networks with latent worker\/task correlation information for label aggregation in crowdsourcing","volume":"16","author":"Wu Hanlu","year":"2022","unstructured":"Hanlu Wu, Tengfei Ma, Lingfei Wu, Fangli Xu, and Shouling Ji. 2022. Exploiting heterogeneous graph neural networks with latent worker\/task correlation information for label aggregation in crowdsourcing. ACM Transactions on Knowledge Discovery from Data 16, 2 (2022), 27:1\u201327:18.","journal-title":"ACM Transactions on Knowledge Discovery from Data"},{"issue":"1","key":"e_1_3_2_55_2","first-page":"15:1\u201315:23","article-title":"CrowdTC: Crowd-powered learning for text classification","volume":"16","author":"Yang Keyu","year":"2022","unstructured":"Keyu Yang, Yunjun Gao, Lei Liang, Song Bian, Lu Chen, and Baihua Zheng. 2022. CrowdTC: Crowd-powered learning for text classification. ACM Transactions on Knowledge Discovery from Data 16, 1 (2022), 15:1\u201315:23.","journal-title":"ACM Transactions on Knowledge Discovery from Data"},{"key":"e_1_3_2_56_2","first-page":"4805","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Ying Zhitao","year":"2018","unstructured":"Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and Jure Leskovec. 2018. Hierarchical graph representation learning with differentiable pooling. In Proceedings of the Advances in Neural Information Processing Systems. Curran Associates, Inc., 4805\u20134815."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080679"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2018.2890148"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2504974"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.14778\/3055540.3055547"},{"key":"e_1_3_2_61_2","first-page":"2204","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Zhou Dengyong","year":"2012","unstructured":"Dengyong Zhou, John C. Platt, Sumit Basu, and Yi Mao. 2012. Learning from the wisdom of crowds by minimax entropy. In Proceedings of the Advances in Neural Information Processing Systems. Curran Associates, Inc., 2204\u20132212."}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3565576","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3565576","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:51Z","timestamp":1750182531000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3565576"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,27]]},"references-count":60,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,10,30]]}},"alternative-id":["10.1145\/3565576"],"URL":"https:\/\/doi.org\/10.1145\/3565576","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"type":"print","value":"1556-4681"},{"type":"electronic","value":"1556-472X"}],"subject":[],"published":{"date-parts":[[2023,2,27]]},"assertion":[{"value":"2022-01-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}