{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,11]],"date-time":"2026-01-11T01:24:32Z","timestamp":1768094672762,"version":"3.49.0"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,4,21]],"date-time":"2023-04-21T00:00:00Z","timestamp":1682035200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Debiased recommendation has recently attracted increasing attention from both industry and academic communities. Traditional models mostly rely on the inverse propensity score (IPS), which can be hard to estimate and may suffer from the high variance issue. To alleviate these problems, in this article, we propose a novel debiased recommendation framework based on user feature balancing. The general idea is to introduce a projection function to adjust user feature distributions, such that the ideal unbiased learning objective can be upper bounded by a solvable objective purely based on the offline dataset. In the upper bound, the projected user distributions are expected to be equal given different items. From the causal inference perspective, this requirement aims to remove the causal relation from the user to the item, which enables us to achieve unbiased recommendation, bypassing the computation of IPS. To efficiently balance the user distributions upon each item pair, we propose three strategies, including clipping, sampling, and adversarial learning to improve the training process. For more robust optimization, we deploy an explicit model to capture the potential latent confounders in recommendation systems. To the best of our knowledge, this article is the first work on debiased recommendation based on confounder balancing. In the experiments, we compare our framework with many state-of-the-art methods based on synthetic, semi-synthetic, and real-world datasets. Extensive experiments demonstrate that our model is effective in promoting the recommendation performance.<\/jats:p>","DOI":"10.1145\/3580594","type":"journal-article","created":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T23:50:11Z","timestamp":1676505011000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Debiased Recommendation with User Feature Balancing"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4175-8398","authenticated-orcid":false,"given":"Mengyue","family":"Yang","sequence":"first","affiliation":[{"name":"University College London, London, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9000-857X","authenticated-orcid":false,"given":"Guohao","family":"Cai","sequence":"additional","affiliation":[{"name":"Noah\u2019s Ark Lab, Huawei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3997-3822","authenticated-orcid":false,"given":"Furui","family":"Liu","sequence":"additional","affiliation":[{"name":"Zhejiang Laboratory, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6458-1586","authenticated-orcid":false,"given":"Jiarui","family":"Jin","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2231-4663","authenticated-orcid":false,"given":"Zhenhua","family":"Dong","sequence":"additional","affiliation":[{"name":"Noah\u2019s Ark Lab, Huawei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4115-8205","authenticated-orcid":false,"given":"Xiuqiang","family":"He","sequence":"additional","affiliation":[{"name":"Noah\u2019s Ark Lab, Huawei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0422-8235","authenticated-orcid":false,"given":"Jianye","family":"Hao","sequence":"additional","affiliation":[{"name":"Noah\u2019s Ark Lab, Huawei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1225-6997","authenticated-orcid":false,"given":"Weiqi","family":"Shao","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4021-4228","authenticated-orcid":false,"given":"Jun","family":"Wang","sequence":"additional","affiliation":[{"name":"University College London, London, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0144-1775","authenticated-orcid":false,"given":"Xu","family":"Chen","sequence":"additional","affiliation":[{"name":"Beijing Key Laboratory of Big Data Management and Analysis Methods, Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,4,21]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"884","volume-title":"ICML (Proceedings of Machine Learning Research)","volume":"119","author":"Bica Ioana","year":"2020","unstructured":"Ioana Bica, Ahmed M. Alaa, and Mihaela van der Schaar. 2020. Time series deconfounder: Estimating treatment effects over time in the presence of hidden confounders. In ICML (Proceedings of Machine Learning Research), Vol. 119. PMLR, 884\u2013895. Retrieved from http:\/\/proceedings.mlr.press\/v119\/bica20a.html."},{"key":"e_1_3_2_3_2","article-title":"AutoDebias: Learning to debias for recommendation","author":"Chen Jiawei","year":"2021","unstructured":"Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021. AutoDebias: Learning to debias for recommendation. arXiv preprint arXiv:2105.04170 (2021).","journal-title":"arXiv preprint arXiv:2105.04170"},{"key":"e_1_3_2_4_2","article-title":"Bias and debias in recommender system: A survey and future directions","author":"Chen Jiawei","year":"2020","unstructured":"Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and debias in recommender system: A survey and future directions. arXiv preprint arXiv:2010.03240 (2020).","journal-title":"arXiv preprint arXiv:2010.03240"},{"key":"e_1_3_2_5_2","first-page":"4613","volume-title":"ICCV","author":"Chen Long","year":"2019","unstructured":"Long Chen, Hanwang Zhang, Jun Xiao, Xiangnan He, Shiliang Pu, and Shih-Fu Chang. 2019. Counterfactual critic multi-agent training for scene graph generation. In ICCV. 4613\u20134623."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2022.11.005"},{"key":"e_1_3_2_7_2","first-page":"2230","volume-title":"IJCAI","author":"Ding Jingtao","year":"2019","unstructured":"Jingtao Ding, Yuhan Quan, Xiangnan He, Yong Li, and Depeng Jin. 2019. Reinforced negative sampling for recommendation with exposure data. In IJCAI. 2230\u20132236."},{"key":"e_1_3_2_8_2","article-title":"Causal incremental graph convolution for recommender system retraining","author":"Ding Sihao","year":"2021","unstructured":"Sihao Ding, Fuli Feng, Xiangnan He, Yong Liao, Jun Shi, and Yongdong Zhang. 2021. Causal incremental graph convolution for recommender system retraining. arXiv preprint arXiv:2108.06889 (2021).","journal-title":"arXiv preprint arXiv:2108.06889"},{"key":"e_1_3_2_9_2","article-title":"Doubly robust policy evaluation and learning","author":"Dud\u00edk Miroslav","year":"2011","unstructured":"Miroslav Dud\u00edk, John Langford, and Lihong Li. 2011. Doubly robust policy evaluation and learning. arXiv preprint arXiv:1103.4601 (2011).","journal-title":"arXiv preprint arXiv:1103.4601"},{"key":"e_1_3_2_10_2","volume-title":"ISIT.","author":"Fuglede Bent","year":"2004","unstructured":"Bent Fuglede and Flemming Topsoe. 2004. Jensen-Shannon divergence and Hilbert space embedding. In ISIT. IEEE."},{"key":"e_1_3_2_11_2","article-title":"Bounds on the Jensen gap, and implications for mean-concentrated distributions","author":"Gao Xiang","year":"2017","unstructured":"Xiang Gao, Meera Sitharam, and Adrian E. Roitberg. 2017. Bounds on the Jensen gap, and implications for mean-concentrated distributions. arXiv preprint arXiv:1712.05267 (2017).","journal-title":"arXiv preprint arXiv:1712.05267"},{"issue":"12","key":"e_1_3_2_12_2","article-title":"A survey of accuracy evaluation metrics of recommendation tasks.","volume":"10","author":"Gunawardana Asela","year":"2009","unstructured":"Asela Gunawardana and Guy Shani. 2009. A survey of accuracy evaluation metrics of recommendation tasks.J. Mach. Learn. Res. 10, 12 (2009).","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_13_2","first-page":"232","volume-title":"WSDM","author":"Guo Ruocheng","year":"2020","unstructured":"Ruocheng Guo, Jundong Li, and Huan Liu. 2020. Learning individual causal effects from networked observational data. In WSDM. 232\u2013240."},{"key":"e_1_3_2_14_2","article-title":"Outer product-based neural collaborative filtering","author":"He Xiangnan","year":"2018","unstructured":"Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer product-based neural collaborative filtering. arXiv preprint arXiv:1808.03912 (2018).","journal-title":"arXiv preprint arXiv:1808.03912"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"Xiangnan He Lizi Liao Hanwang Zhang Liqiang Nie Xia Hu and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW . 173\u2013182.","DOI":"10.1145\/3038912.3052569"},{"key":"e_1_3_2_16_2","unstructured":"Fredrik Johansson Uri Shalit and David Sontag. 2016. Learning representations for counterfactual inference. In ICML . PMLR 3020\u20133029."},{"key":"e_1_3_2_17_2","article-title":"Generalization bounds and representation learning for estimation of potential outcomes and causal effects","volume":"2001","author":"Johansson Fredrik D.","year":"2020","unstructured":"Fredrik D. Johansson, Uri Shalit, Nathan Kallus, and David A. Sontag. 2020. Generalization bounds and representation learning for estimation of potential outcomes and causal effects. CoRR abs\/2001.07426 (2020).","journal-title":"CoRR"},{"key":"e_1_3_2_18_2","first-page":"3020","volume-title":"ICML (JMLR Workshop and Conference Proceedings)","volume":"48","author":"Johansson Fredrik D.","year":"2016","unstructured":"Fredrik D. Johansson, Uri Shalit, and David A. Sontag. 2016. Learning representations for counterfactual inference. In ICML (JMLR Workshop and Conference Proceedings), Vol. 48. JMLR.org, 3020\u20133029. Retrieved from http:\/\/proceedings.mlr.press\/v48\/johansson16.html."},{"key":"e_1_3_2_19_2","volume-title":"EMNLP","author":"Lawrence Carolin","year":"2017","unstructured":"Carolin Lawrence, Artem Sokolov, and Stefan Riezler. 2017. Counterfactual learning from bandit feedback under deterministic logging: A case study in statistical machine translation. In EMNLP."},{"key":"e_1_3_2_20_2","volume-title":"ICLR","author":"Lee Byung-Jun","year":"2021","unstructured":"Byung-Jun Lee, Jongmin Lee, and Kee-Eung Kim. 2021. Representation balancing offline model-based reinforcement learning. In ICLR. OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=QpNz8r_Ri2Y."},{"key":"e_1_3_2_21_2","first-page":"2649","volume-title":"NIPS","author":"Liu Yao","year":"2018","unstructured":"Yao Liu, Omer Gottesman, Aniruddh Raghu, Matthieu Komorowski, Aldo A. Faisal, Finale Doshi-Velez, and Emma Brunskill. 2018. Representation balancing MDPs for off-policy policy evaluation. In NIPS. 2649\u20132658. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/980ecd059122ce2e50136bda65c25e07-Abstract.html."},{"key":"e_1_3_2_22_2","first-page":"166","volume-title":"WSDM","author":"Ma Jing","year":"2021","unstructured":"Jing Ma, Ruocheng Guo, Chen Chen, Aidong Zhang, and Jundong Li. 2021. Deconfounding with networked observational data in a dynamic environment. In WSDM. 166\u2013174."},{"key":"e_1_3_2_23_2","article-title":"Collaborative filtering and the missing at random assumption","volume":"1206","author":"Marlin Benjamin M.","year":"2012","unstructured":"Benjamin M. Marlin, Richard S. Zemel, Sam T. Roweis, and Malcolm Slaney. 2012. Collaborative filtering and the missing at random assumption. CoRR abs\/1206.5267 (2012).","journal-title":"CoRR"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014634"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/1642718"},{"key":"e_1_3_2_26_2","article-title":"Multiple causal inference with latent confounding","author":"Ranganath Rajesh","year":"2018","unstructured":"Rajesh Ranganath and Adler Perotte. 2018. Multiple causal inference with latent confounding. arXiv preprint arXiv:1805.08273 (2018).","journal-title":"arXiv preprint arXiv:1805.08273"},{"key":"e_1_3_2_27_2","article-title":"BPR: Bayesian personalized ranking from implicit feedback","author":"Rendle Steffen","year":"2012","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).","journal-title":"arXiv preprint arXiv:1205.2618"},{"key":"e_1_3_2_28_2","first-page":"309","volume-title":"SIGIR","author":"Saito Yuta","year":"2020","unstructured":"Yuta Saito. 2020. Asymmetric tri-training for debiasing missing-not-at-random explicit feedback. In SIGIR. 309\u2013318."},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","unstructured":"Yuta Saito Suguru Yaginuma Yuta Nishino Hayato Sakata and Kazuhide Nakata. 2020. Unbiased recommender learning from missing-not-at-random implicit feedback. In WSDM . 501\u2013509.","DOI":"10.1145\/3336191.3371783"},{"key":"e_1_3_2_30_2","first-page":"1670","volume-title":"ICML (JMLR Workshop and Conference Proceedings)","volume":"48","author":"Schnabel Tobias","year":"2016","unstructured":"Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In ICML (JMLR Workshop and Conference Proceedings), Vol. 48. JMLR.org, 1670\u20131679. Retrieved from http:\/\/proceedings.mlr.press\/v48\/schnabel16.html."},{"key":"e_1_3_2_31_2","first-page":"3076","volume-title":"ICML","author":"Shalit Uri","year":"2017","unstructured":"Uri Shalit, Fredrik D. Johansson, and David Sontag. 2017. Estimating individual treatment effect: Generalization bounds and algorithms. In ICML. PMLR, 3076\u20133085."},{"key":"e_1_3_2_32_2","unstructured":"Bharath Sriperumbudur Kenji Fukumizu Arthur Gretton Bernhard Sch\u00f6lkopf and Gert Lanckriet. 2009. On integral probability metrics divergences and binary classification. arXiv preprint arXiv:0901.2698 (2009)."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/2789272.2886805"},{"key":"e_1_3_2_34_2","article-title":"Counterfactual risk minimization: Learning from logged bandit feedback","volume":"1502","author":"Swaminathan Adith","year":"2015","unstructured":"Adith Swaminathan and Thorsten Joachims. 2015. Counterfactual risk minimization: Learning from logged bandit feedback. CoRR abs\/1502.02362 (2015).","journal-title":"CoRR"},{"key":"e_1_3_2_35_2","first-page":"3231","volume-title":"NIPS","author":"Swaminathan Adith","year":"2015","unstructured":"Adith Swaminathan and Thorsten Joachims. 2015. The self-normalized estimator for counterfactual learning. In NIPS. 3231\u20133239. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2015\/hash\/39027dfad5138c9ca0c474d71db915c3-Abstract.html."},{"key":"e_1_3_2_36_2","article-title":"Deconfounded recommendation for alleviating bias amplification","author":"Wang Wenjie","year":"2021","unstructured":"Wenjie Wang, Fuli Feng, Xiangnan He, Xiang Wang, and Tat-Seng Chua. 2021. Deconfounded recommendation for alleviating bias amplification. arXiv preprint arXiv:2105.10648 (2021).","journal-title":"arXiv preprint arXiv:2105.10648"},{"key":"e_1_3_2_37_2","first-page":"1288","volume-title":"SIGIR","author":"Wang Wenjie","year":"2021","unstructured":"Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2021. Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue. In SIGIR. 1288\u20131297."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159732"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2019.1686987"},{"key":"e_1_3_2_40_2","article-title":"Probabilistic and variational recommendation denoising","author":"Wang Yu","year":"2021","unstructured":"Yu Wang, Xin Xin, Zaiqiao Meng, Xiangnan He, Joemon Jose, and Fuli Feng. 2021. Probabilistic and variational recommendation denoising. arXiv preprint arXiv:2105.09605 (2021).","journal-title":"arXiv preprint arXiv:2105.09605"},{"key":"e_1_3_2_41_2","volume-title":"NeurIPS","author":"Wang Zifeng","year":"2020","unstructured":"Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan E. Kuruoglu, and Yefeng Zheng. 2020. Information theoretic counterfactual learning from missing-not-at-random feedback. In NeurIPS. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/13f3cf8c531952d72e5847c4183e6910-Abstract.html."},{"key":"e_1_3_2_42_2","first-page":"2195","volume-title":"WWW","author":"Wang Zhenlei","year":"2022","unstructured":"Zhenlei Wang, Shiqi Shen, Zhipeng Wang, Bo Chen, Xu Chen, and Ji-Rong Wen. 2022. Unbiased sequential recommendation with latent confounders. In WWW. 2195\u20132204."},{"key":"e_1_3_2_43_2","first-page":"1791","volume-title":"SIGIR","author":"Wei Tianxin","year":"2021","unstructured":"Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In SIGIR. 1791\u20131800."},{"key":"e_1_3_2_44_2","first-page":"2260","volume-title":"CIKM","author":"Xu Chen","year":"2022","unstructured":"Chen Xu, Jun Xu, Xu Chen, Zhenghua Dong, and Ji-Rong Wen. 2022. Dually enhanced propensity score estimation in sequential recommendation. In CIKM. 2260\u20132269."},{"key":"e_1_3_2_45_2","first-page":"13515","article-title":"Adversarial counterfactual learning and evaluation for recommender system","volume":"33","author":"Xu Da","year":"2020","unstructured":"Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. 2020. Adversarial counterfactual learning and evaluation for recommender system. Adv. Neural Inf. Process. Syst. 33 (2020), 13515\u201313526.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_46_2","article-title":"A survey on causal inference","author":"Yao Liuyi","year":"2020","unstructured":"Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. 2020. A survey on causal inference. arXiv preprint arXiv:2002.02770 (2020).","journal-title":"arXiv preprint arXiv:2002.02770"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3358058"},{"key":"e_1_3_2_48_2","article-title":"Causal intervention for leveraging popularity bias in recommendation","author":"Zhang Yang","year":"2021","unstructured":"Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal intervention for leveraging popularity bias in recommendation. arXiv preprint arXiv:2105.06067 (2021).","journal-title":"arXiv preprint arXiv:2105.06067"},{"key":"e_1_3_2_49_2","article-title":"Popularity bias is not always evil: Disentangling benign and harmful bias for recommendation","author":"Zhao Zihao","year":"2021","unstructured":"Zihao Zhao, Jiawei Chen, Sheng Zhou, Xiangnan He, Xuezhi Cao, Fuzheng Zhang, and Wei Wu. 2021. Popularity bias is not always evil: Disentangling benign and harmful bias for recommendation. arXiv preprint arXiv:2109.07946 (2021).","journal-title":"arXiv preprint arXiv:2109.07946"},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","first-page":"2980","DOI":"10.1145\/3442381.3449788","volume-title":"WWW","author":"Zheng Yu","year":"2021","unstructured":"Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Yong Li, and Depeng Jin. 2021. Disentangling user interest and conformity for recommendation with causal embedding. In WWW. 2980\u20132991."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330852"},{"key":"e_1_3_2_52_2","first-page":"696","volume-title":"SIGIR","author":"Zou Hao","year":"2019","unstructured":"Hao Zou, Kun Kuang, Boqi Chen, Peixuan Chen, and Peng Cui. 2019. Focused context balancing for robust offline policy evaluation. In SIGIR. 696\u2013704."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580594","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580594","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:36Z","timestamp":1750178796000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,21]]},"references-count":51,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3580594"],"URL":"https:\/\/doi.org\/10.1145\/3580594","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,21]]},"assertion":[{"value":"2022-02-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-06","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}