{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:28:09Z","timestamp":1750220889126,"version":"3.41.0"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T00:00:00Z","timestamp":1593993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100007515","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1218393, 1514204, 1924928, 1938167, 1934600"],"award-info":[{"award-number":["1218393, 1514204, 1924928, 1938167, 1934600"]}],"id":[{"id":"10.13039\/100007515","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2020,10,31]]},"abstract":"<jats:p>The goal of metric learning is to learn a good distance metric that can capture the relationships among instances, and its importance has long been recognized in many fields. An implicit assumption in the traditional settings of metric learning is that the associated labels of the instances are deterministic. However, in many real-world applications, the associated labels come naturally with probabilities instead of deterministic values, which makes it difficult for the existing metric-learning methods to work well in these applications. To address this challenge, in this article, we study how to effectively learn the distance metric from datasets that contain probabilistic information, and then propose several novel metric-learning mechanisms for two types of probabilistic labels, i.e., the instance-wise probabilistic label and the group-wise probabilistic label. Compared with the existing metric-learning methods, our proposed mechanisms are capable of learning distance metrics directly from the probabilistic labels with high accuracy. We also theoretically analyze the proposed mechanisms and conduct extensive experiments on real-world datasets to verify the desirable properties of these mechanisms.<\/jats:p>","DOI":"10.1145\/3364320","type":"journal-article","created":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T21:19:53Z","timestamp":1594070393000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Learning Distance Metrics from Probabilistic Information"],"prefix":"10.1145","volume":"14","author":[{"given":"Mengdi","family":"Huai","sequence":"first","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}]},{"given":"Chenglin","family":"Miao","sequence":"additional","affiliation":[{"name":"State University of New York at Buffalo, Buffalo, NY, USA"}]},{"given":"Yaliang","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group, Bellevue, WA, USA"}]},{"given":"Qiuling","family":"Suo","sequence":"additional","affiliation":[{"name":"State University of New York at Buffalo, Buffalo, NY, USA"}]},{"given":"Lu","family":"Su","sequence":"additional","affiliation":[{"name":"State University of New York at Buffalo, Buffalo, NY, USA"}]},{"given":"Aidong","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}]}],"member":"320","published-online":{"date-parts":[[2020,7,6]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence. 1217--1222","author":"Baghshah Mahdieh Soleymani","year":"2009","unstructured":"Mahdieh Soleymani Baghshah and Saeed Bagheri Shouraki . 2009 . Semi-supervised metric learning using pairwise constraints . In Proceedings of the International Joint Conference on Artificial Intelligence. 1217--1222 . Mahdieh Soleymani Baghshah and Saeed Bagheri Shouraki. 2009. Semi-supervised metric learning using pairwise constraints. In Proceedings of the International Joint Conference on Artificial Intelligence. 1217--1222."},{"key":"e_1_2_1_2_1","first-page":"937","article-title":"Learning a mahalanobis metric from equivalence constraints","author":"Bar-Hillel Aharon","year":"2005","unstructured":"Aharon Bar-Hillel , Tomer Hertz , Noam Shental , and Daphna Weinshall . 2005 . Learning a mahalanobis metric from equivalence constraints . Journal of Machine Learning Research 6 , Jun (2005), 937 -- 965 . Aharon Bar-Hillel, Tomer Hertz, Noam Shental, and Daphna Weinshall. 2005. Learning a mahalanobis metric from equivalence constraints. Journal of Machine Learning Research 6, Jun (2005), 937--965.","journal-title":"Journal of Machine Learning Research 6"},{"volume-title":"Nonlinear Programming","author":"Bertsekas Dimitri P.","key":"e_1_2_1_3_1","unstructured":"Dimitri P. Bertsekas . 1999. Nonlinear Programming . Athena Scientific , Belmont . Dimitri P. Bertsekas. 1999. Nonlinear Programming. Athena Scientific, Belmont."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-015-5499-7"},{"key":"e_1_2_1_5_1","first-page":"203","article-title":"Optimization techniques for semi-supervised support vector machines","author":"Chapelle Olivier","year":"2008","unstructured":"Olivier Chapelle , Vikas Sindhwani , and Sathiya S. Keerthi . 2008 . Optimization techniques for semi-supervised support vector machines . Journal of Machine Learning Research 9 , Feb (2008), 203 -- 233 . Olivier Chapelle, Vikas Sindhwani, and Sathiya S. Keerthi. 2008. Optimization techniques for semi-supervised support vector machines. Journal of Machine Learning Research 9, Feb (2008), 203--233.","journal-title":"Journal of Machine Learning Research 9"},{"volume-title":"Proceedings of the 24th International Conference on Machine Learning. ACM, 209--216","author":"Davis Jason V.","key":"e_1_2_1_6_1","unstructured":"Jason V. Davis , Brian Kulis , Prateek Jain , Suvrit Sra , and Inderjit S. Dhillon . 2007. Information-theoretic metric learning . In Proceedings of the 24th International Conference on Machine Learning. ACM, 209--216 . Jason V. Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S. Dhillon. 2007. Information-theoretic metric learning. In Proceedings of the 24th International Conference on Machine Learning. ACM, 209--216."},{"key":"e_1_2_1_7_1","first-page":"2899","article-title":"Efficient online and batch learning using forward backward splitting","author":"Duchi John","year":"2009","unstructured":"John Duchi and Yoram Singer . 2009 . Efficient online and batch learning using forward backward splitting . Journal of Machine Learning Research 10 , Dec (2009), 2899 -- 2934 . John Duchi and Yoram Singer. 2009. Efficient online and batch learning using forward backward splitting. Journal of Machine Learning Research 10, Dec (2009), 2899--2934.","journal-title":"Journal of Machine Learning Research 10"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/232"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/1886063.1886112"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219976"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611975321.31"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/352"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 1262--1270","author":"Huang Chen","year":"2016","unstructured":"Chen Huang , Chen Change Loy , and Xiaoou Tang . 2016 . Local similarity-aware deep feature embedding . In Proceedings of the Advances in Neural Information Processing Systems. 1262--1270 . Chen Huang, Chen Change Loy, and Xiaoou Tang. 2016. Local similarity-aware deep feature embedding. In Proceedings of the Advances in Neural Information Processing Systems. 1262--1270."},{"key":"e_1_2_1_14_1","volume-title":"Anagnostopoulos","author":"Huang Yinjie","year":"2013","unstructured":"Yinjie Huang , Cong Li , Michael Georgiopoulos , and Georgios C . Anagnostopoulos . 2013 . Reduced-rank local distance metric learning. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer , 224--239. Yinjie Huang, Cong Li, Michael Georgiopoulos, and Georgios C. Anagnostopoulos. 2013. Reduced-rank local distance metric learning. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 224--239."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939806"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206684"},{"volume-title":"Proceedings of the Conference on Computer Vision and Pattern Recognition.","author":"Law Marc T.","key":"e_1_2_1_17_1","unstructured":"Marc T. Law , Yaoliang Yu , Raquel Urtasun , Richard S. Zemel , and Eric P. Xing . 2017. Efficient multiple instance metric learning using weakly supervised data . In Proceedings of the Conference on Computer Vision and Pattern Recognition. Marc T. Law, Yaoliang Yu, Raquel Urtasun, Richard S. Zemel, and Eric P. Xing. 2017. Efficient multiple instance metric learning using weakly supervised data. In Proceedings of the Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_1_18_1","volume-title":"Multi-view metric learning for multi-instance image classification. Arxiv Preprint Arxiv:1610.06671","author":"Li Dewei","year":"2016","unstructured":"Dewei Li and Yingjie Tian . 2016. Multi-view metric learning for multi-instance image classification. Arxiv Preprint Arxiv:1610.06671 ( 2016 ). Dewei Li and Yingjie Tian. 2016. Multi-view metric learning for multi-instance image classification. Arxiv Preprint Arxiv:1610.06671 (2016)."},{"volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence. 2800--2806","author":"Liu Weiwei","key":"e_1_2_1_19_1","unstructured":"Weiwei Liu and Ivor W. Tsang . 2015. Large margin metric learning for multi-label prediction . In Proceedings of the AAAI Conference on Artificial Intelligence. 2800--2806 . Weiwei Liu and Ivor W. Tsang. 2015. Large margin metric learning for multi-label prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. 2800--2806."},{"key":"e_1_2_1_20_1","first-page":"148","article-title":"On the method of bounded differences","volume":"141","author":"McDiarmid Colin","year":"1989","unstructured":"Colin McDiarmid . 1989 . On the method of bounded differences . Surveys in Combinatorics 141 , 1 (1989), 148 -- 188 . Colin McDiarmid. 1989. On the method of bounded differences. Surveys in Combinatorics 141, 1 (1989), 148--188.","journal-title":"Surveys in Combinatorics"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00614"},{"volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 1822--1830","author":"Noh Yung-Kyun","key":"e_1_2_1_22_1","unstructured":"Yung-Kyun Noh , Byoung-Tak Zhang , and Daniel D. Lee . 2010. Generative local metric learning for nearest neighbor classification . In Proceedings of the Advances in Neural Information Processing Systems. 1822--1830 . Yung-Kyun Noh, Byoung-Tak Zhang, and Daniel D. Lee. 2010. Generative local metric learning for nearest neighbor classification. In Proceedings of the Advances in Neural Information Processing Systems. 1822--1830."},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 190--198","author":"Patrini Giorgio","year":"2014","unstructured":"Giorgio Patrini , Richard Nock , Paul Rivera , and Tiberio Caetano . 2014 . (Almost) no label no cry . In Proceedings of the Advances in Neural Information Processing Systems. 190--198 . Giorgio Patrini, Richard Nock, Paul Rivera, and Tiberio Caetano. 2014. (Almost) no label no cry. In Proceedings of the Advances in Neural Information Processing Systems. 190--198."},{"key":"e_1_2_1_24_1","volume-title":"Raymond Chi-Wing Wong, and Phillp S. Yu","author":"Peng Peng","year":"2014","unstructured":"Peng Peng , Raymond Chi-Wing Wong, and Phillp S. Yu . 2014 . Learning on probabilistic labels. In Proceedings of the 2014 SIAM International Conference on Data Mining. 307--315. Peng Peng, Raymond Chi-Wing Wong, and Phillp S. Yu. 2014. Learning on probabilistic labels. In Proceedings of the 2014 SIAM International Conference on Data Mining. 307--315."},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the International Conference on Machine Learning. 433--441","author":"Rodrigues Filipe","year":"2014","unstructured":"Filipe Rodrigues , Francisco Pereira , and Bernardete Ribeiro . 2014 . Gaussian process classification and active learning with multiple annotators . In Proceedings of the International Conference on Machine Learning. 433--441 . Filipe Rodrigues, Francisco Pereira, and Bernardete Ribeiro. 2014. Gaussian process classification and active learning with multiple annotators. In Proceedings of the International Conference on Machine Learning. 433--441."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104322.3104438"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.56"},{"volume-title":"Understanding Machine Learning: From Theory to Algorithms","author":"Shalev-Shwartz Shai","key":"e_1_2_1_28_1","unstructured":"Shai Shalev-Shwartz and Shai Ben-David . 2014. Understanding Machine Learning: From Theory to Algorithms . Cambridge University Press . Shai Shalev-Shwartz and Shai Ben-David. 2014. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-017-0708-2"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/WI.2018.00-21"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 1857--1865","author":"Sohn Kihyuk","year":"2016","unstructured":"Kihyuk Sohn . 2016 . Improved deep metric learning with multi-class n-pair loss objective . In Proceedings of the Advances in Neural Information Processing Systems. 1857--1865 . Kihyuk Sohn. 2016. Improved deep metric learning with multi-class n-pair loss objective. In Proceedings of the Advances in Neural Information Processing Systems. 1857--1865."},{"volume-title":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1097--1104","author":"St Joseph","key":"e_1_2_1_32_1","unstructured":"Joseph St . Amand and Jun Huan. 2017. Sparse compositional local metric learning . In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1097--1104 . Joseph St. Amand and Jun Huan. 2017. Sparse compositional local metric learning. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1097--1104."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2408736.2408740"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2017.54"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNB.2018.2837622"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM.2017.8217759"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2018.00063"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 1621--1629","author":"Tian Tian","year":"2015","unstructured":"Tian Tian and Jun Zhu . 2015 . Max-margin majority voting for learning from crowds . In Proceedings of the Advances in Neural Information Processing Systems. 1621--1629 . Tian Tian and Jun Zhu. 2015. Max-margin majority voting for learning from crowds. In Proceedings of the Advances in Neural Information Processing Systems. 1621--1629."},{"key":"e_1_2_1_39_1","volume-title":"Tropp et\u00a0al","author":"Joel","year":"2015","unstructured":"Joel A. Tropp et\u00a0al . 2015 . An introduction to matrix concentration inequalities. Foundations and Trends\u00ae in Machine Learning 8, 1\u20132 (2015), 1--230. Joel A. Tropp et\u00a0al. 2015. An introduction to matrix concentration inequalities. Foundations and Trends\u00ae in Machine Learning 8, 1\u20132 (2015), 1--230."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/GlobalSIP.2018.8646444"},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence. 1321--1327","author":"Wang Dong","year":"2014","unstructured":"Dong Wang and Xiaoyang Tan . 2014 . Robust distance metric learning in the presence of label noise . In Proceedings of the AAAI Conference on Artificial Intelligence. 1321--1327 . Dong Wang and Xiaoyang Tan. 2014. Robust distance metric learning in the presence of label noise. In Proceedings of the AAAI Conference on Artificial Intelligence. 1321--1327."},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 1601--1609","author":"Wang Jun","year":"2012","unstructured":"Jun Wang , Alexandros Kalousis , and Adam Woznica . 2012 . Parametric local metric learning for nearest neighbor classification . In Proceedings of the Advances in Neural Information Processing Systems. 1601--1609 . Jun Wang, Alexandros Kalousis, and Adam Woznica. 2012. Parametric local metric learning for nearest neighbor classification. In Proceedings of the Advances in Neural Information Processing Systems. 1601--1609."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220008"},{"key":"e_1_2_1_44_1","volume-title":"Saul","author":"Weinberger Kilian Q.","year":"2006","unstructured":"Kilian Q. Weinberger , John Blitzer , and Lawrence K . Saul . 2006 . Distance metric learning for large margin nearest neighbor classification. In Proceedings of the Advances in Neural Information Processing Systems . 1473--1480. Kilian Q. Weinberger, John Blitzer, and Lawrence K. Saul. 2006. Distance metric learning for large margin nearest neighbor classification. In Proceedings of the Advances in Neural Information Processing Systems. 1473--1480."},{"key":"e_1_2_1_45_1","first-page":"207","article-title":"Distance metric learning for large margin nearest neighbor classification","author":"Weinberger Kilian Q.","year":"2009","unstructured":"Kilian Q. Weinberger and Lawrence K. Saul . 2009 . Distance metric learning for large margin nearest neighbor classification . Journal of Machine Learning Research 10 , Feb (2009), 207 -- 244 . Kilian Q. Weinberger and Lawrence K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10, Feb (2009), 207--244.","journal-title":"Journal of Machine Learning Research 10"},{"key":"e_1_2_1_46_1","first-page":"2543","article-title":"Dual averaging methods for regularized stochastic learning and online optimization","author":"Xiao Lin","year":"2010","unstructured":"Lin Xiao . 2010 . Dual averaging methods for regularized stochastic learning and online optimization . Journal of Machine Learning Research 11 , Oct (2010), 2543 -- 2596 . Lin Xiao. 2010. Dual averaging methods for regularized stochastic learning and online optimization. Journal of Machine Learning Research 11, Oct (2010), 2543--2596.","journal-title":"Journal of Machine Learning Research 11"},{"key":"e_1_2_1_47_1","volume-title":"Ng","author":"Xing Eric P.","year":"2003","unstructured":"Eric P. Xing , Michael I. Jordan , Stuart J. Russell , and Andrew Y . Ng . 2003 . Distance metric learning with application to clustering with side-information. In Proceedings of the Advances in Neural Information Processing Systems . 521--528. Eric P. Xing, Michael I. Jordan, Stuart J. Russell, and Andrew Y. Ng. 2003. Distance metric learning with application to clustering with side-information. In Proceedings of the Advances in Neural Information Processing Systems. 521--528."},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 2633--2643","author":"Yao Liuyi","year":"2018","unstructured":"Liuyi Yao , Sheng Li , Yaliang Li , Mengdi Huai , Jing Gao , and Aidong Zhang . 2018 . Representation learning for treatment effect estimation from observational data . In Proceedings of the Advances in Neural Information Processing Systems. 2633--2643 . Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. 2018. Representation learning for treatment effect estimation from observational data. In Proceedings of the Advances in Neural Information Processing Systems. 2633--2643."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611975673.55"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/3042817.3042993"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the International Conference on Machine Learning. 2464--2471","author":"Zadeh Pourya","year":"2016","unstructured":"Pourya Zadeh , Reshad Hosseini , and Suvrit Sra . 2016 . Geometric mean metric learning . In Proceedings of the International Conference on Machine Learning. 2464--2471 . Pourya Zadeh, Reshad Hosseini, and Suvrit Sra. 2016. Geometric mean metric learning. In Proceedings of the International Conference on Machine Learning. 2464--2471."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2016.0182"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219948"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the International Conference on Machine Learning. 262--270","author":"Zhou Dengyong","year":"2014","unstructured":"Dengyong Zhou , Qiang Liu , John Platt , and Christopher Meek . 2014 . Aggregating ordinal labels from crowds by minimax conditional entropy . In Proceedings of the International Conference on Machine Learning. 262--270 . Dengyong Zhou, Qiang Liu, John Platt, and Christopher Meek. 2014. Aggregating ordinal labels from crowds by minimax conditional entropy. In Proceedings of the International Conference on Machine Learning. 262--270."}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3364320","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3364320","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:27Z","timestamp":1750203867000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3364320"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,6]]},"references-count":54,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,10,31]]}},"alternative-id":["10.1145\/3364320"],"URL":"https:\/\/doi.org\/10.1145\/3364320","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"type":"print","value":"1556-4681"},{"type":"electronic","value":"1556-472X"}],"subject":[],"published":{"date-parts":[[2020,7,6]]},"assertion":[{"value":"2018-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-07-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}