{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T14:50:56Z","timestamp":1776783056126,"version":"3.51.2"},"publisher-location":"New York, NY, USA","reference-count":56,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T00:00:00Z","timestamp":1658102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,18]]},"DOI":"10.1145\/3533767.3534225","type":"proceedings-article","created":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T14:28:50Z","timestamp":1657895330000},"page":"327-339","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["One step further: evaluating interpreters using metamorphic testing"],"prefix":"10.1145","author":[{"given":"Ming","family":"Fan","sequence":"first","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"given":"Jiali","family":"Wei","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"given":"Wuxia","family":"Jin","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"given":"Zhou","family":"Xu","sequence":"additional","affiliation":[{"name":"Chongqing University, China"}]},{"given":"Wenying","family":"Wei","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]},{"given":"Ting","family":"Liu","sequence":"additional","affiliation":[{"name":"Xi'an Jiaotong University, China"}]}],"member":"320","published-online":{"date-parts":[[2022,7,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2014.2339736"},{"key":"e_1_3_2_1_2_1","volume-title":"Advances in Neural Information Processing Systems","author":"Adebayo Julius","year":"2018","unstructured":"Julius Adebayo , Justin Gilmer , Michael Muelly , Ian Goodfellow , Moritz Hardt , and Been Kim . 2018. Sanity Checks for Saliency Maps . In Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). 31, Curran Associates, Inc. . https:\/\/proceedings.neurips.cc\/paper\/ 2018 \/file\/294a8ed24b1ad22ec2e7efea049b8737-Paper.pdf Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity Checks for Saliency Maps. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). 31, Curran Associates, Inc.. https:\/\/proceedings.neurips.cc\/paper\/2018\/file\/294a8ed24b1ad22ec2e7efea049b8737-Paper.pdf"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00023"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377325.3377519"},{"key":"e_1_3_2_1_5_1","volume-title":"Advances in Neural Information Processing Systems","author":"Melis David Alvarez","year":"2018","unstructured":"David Alvarez Melis and Tommi Jaakkola . 2018. Towards Robust Interpretability with Self-Explaining Neural Networks . In Advances in Neural Information Processing Systems , S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). 31, Curran Associates, Inc. . https:\/\/proceedings.neurips.cc\/paper\/ 2018 \/file\/3e9f0fc9b2f89e043bc6233994dfcf76-Paper.pdf David Alvarez Melis and Tommi Jaakkola. 2018. Towards Robust Interpretability with Self-Explaining Neural Networks. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). 31, Curran Associates, Inc.. https:\/\/proceedings.neurips.cc\/paper\/2018\/file\/3e9f0fc9b2f89e043bc6233994dfcf76-Paper.pdf"},{"key":"e_1_3_2_1_6_1","unstructured":"Marco Ancona Enea Ceolini Cengiz \u00d6ztireli and Markus Gross. 2017. Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104. Marco Ancona Enea Ceolini Cengiz \u00d6ztireli and Markus Gross. 2017. Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2018.713"},{"key":"e_1_3_2_1_8_1","unstructured":"Oana-Maria Camburu. 2020. Explaining deep neural networks. arXiv preprint arXiv:2010.01496. Oana-Maria Camburu. 2020. Explaining deep neural networks. arXiv preprint arXiv:2010.01496."},{"key":"e_1_3_2_1_9_1","unstructured":"Oana-Maria Camburu Eleonora Giunchiglia Jakob Foerster Thomas Lukasiewicz and Phil Blunsom. 2019. Can I trust the explainer? Verifying post-hoc explanatory methods. arXiv preprint arXiv:1910.02065. Oana-Maria Camburu Eleonora Giunchiglia Jakob Foerster Thomas Lukasiewicz and Phil Blunsom. 2019. Can I trust the explainer? Verifying post-hoc explanatory methods. arXiv preprint arXiv:1910.02065."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Aditya Chattopadhay Anirban Sarkar Prantik Howlader and Vineeth N Balasubramanian. 2018. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV). 839\u2013847. Aditya Chattopadhay Anirban Sarkar Prantik Howlader and Vineeth N Balasubramanian. 2018. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV). 839\u2013847.","DOI":"10.1109\/WACV.2018.00097"},{"key":"e_1_3_2_1_12_1","unstructured":"Tsong Y Chen Shing C Cheung and Shiu Ming Yiu. 2020. Metamorphic testing: a new approach for generating next test cases. arXiv preprint arXiv:2002.12543. Tsong Y Chen Shing C Cheung and Shiu Ming Yiu. 2020. Metamorphic testing: a new approach for generating next test cases. arXiv preprint arXiv:2002.12543."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3143561","article-title":"Metamorphic testing: A review of challenges and opportunities","volume":"51","author":"Chen Tsong Yueh","year":"2018","unstructured":"Tsong Yueh Chen , Fei-Ching Kuo , Huai Liu , Pak-Lok Poon , Dave Towey , TH Tse , and Zhi Quan Zhou . 2018 . Metamorphic testing: A review of challenges and opportunities . ACM Computing Surveys (CSUR) , 51 , 1 (2018), 1 \u2013 27 . Tsong Yueh Chen, Fei-Ching Kuo, Huai Liu, Pak-Lok Poon, Dave Towey, TH Tse, and Zhi Quan Zhou. 2018. Metamorphic testing: A review of challenges and opportunities. ACM Computing Surveys (CSUR), 51, 1 (2018), 1\u201327.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"e_1_3_2_1_14_1","volume-title":"Deep neural networks segment neuronal membranes in electron microscopy images. Advances in neural information processing systems, 25","author":"Ciresan Dan","year":"2012","unstructured":"Dan Ciresan , Alessandro Giusti , Luca Gambardella , and J\u00fcrgen Schmidhuber . 2012. Deep neural networks segment neuronal membranes in electron microscopy images. Advances in neural information processing systems, 25 ( 2012 ), 2843\u20132851. Dan Ciresan, Alessandro Giusti, Luca Gambardella, and J\u00fcrgen Schmidhuber. 2012. Deep neural networks segment neuronal membranes in electron microscopy images. Advances in neural information processing systems, 25 (2012), 2843\u20132851."},{"key":"e_1_3_2_1_15_1","unstructured":"Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371. Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213858"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2021.3103064"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2020.3021924"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00304"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_3_2_1_21_1","first-page":"2925","article-title":"Fooling neural network interpretations via adversarial model manipulation","volume":"32","author":"Heo Juyeon","year":"2019","unstructured":"Juyeon Heo , Sunghwan Joo , and Taesup Moon . 2019 . Fooling neural network interpretations via adversarial model manipulation . Advances in Neural Information Processing Systems , 32 (2019), 2925 \u2013 2936 . Juyeon Heo, Sunghwan Joo, and Taesup Moon. 2019. Fooling neural network interpretations via adversarial model manipulation. Advances in Neural Information Processing Systems, 32 (2019), 2925\u20132936.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_22_1","unstructured":"Robert R Hoffman Shane T Mueller Gary Klein and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608. Robert R Hoffman Shane T Mueller Gary Klein and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608."},{"key":"e_1_3_2_1_23_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=4dXmpCDGNp7","author":"Hsieh Cheng-Yu","year":"2021","unstructured":"Cheng-Yu Hsieh , Chih-Kuan Yeh , Xuanqing Liu , Pradeep Kumar Ravikumar , Seungyeon Kim , Sanjiv Kumar , and Cho-Jui Hsieh . 2021 . Evaluations and Methods for Explanation through Robustness Analysis . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=4dXmpCDGNp7 Cheng-Yu Hsieh, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Kumar Ravikumar, Seungyeon Kim, Sanjiv Kumar, and Cho-Jui Hsieh. 2021. Evaluations and Methods for Explanation through Robustness Analysis. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=4dXmpCDGNp7"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSESS.2013.6615365"},{"key":"e_1_3_2_1_25_1","volume-title":"International conference on machine learning. 2668\u20132677","author":"Kim Been","year":"2018","unstructured":"Been Kim , Martin Wattenberg , Justin Gilmer , Carrie Cai , James Wexler , and Fernanda Viegas . 2018 . Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) . In International conference on machine learning. 2668\u20132677 . Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, and Fernanda Viegas. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning. 2668\u20132677."},{"key":"e_1_3_2_1_26_1","unstructured":"Pieter-Jan Kindermans Kristof Sch\u00fctt Klaus-Robert M\u00fcller and Sven D\u00e4hne. 2016. Investigating the influence of noise and distractors on the interpretation of neural networks. arXiv preprint arXiv:1611.07270. Pieter-Jan Kindermans Kristof Sch\u00fctt Klaus-Robert M\u00fcller and Sven D\u00e4hne. 2016. Investigating the influence of noise and distractors on the interpretation of neural networks. arXiv preprint arXiv:1611.07270."},{"key":"e_1_3_2_1_27_1","volume-title":"Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E Hinton . 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25 ( 2012 ), 1097\u20131105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25 (2012), 1097\u20131105."},{"key":"e_1_3_2_1_28_1","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 7, 59\u201367","author":"Lage Isaac","year":"2019","unstructured":"Isaac Lage , Emily Chen , Jeffrey He , Menaka Narayanan , Been Kim , Samuel J Gershman , and Finale Doshi-Velez . 2019 . Human evaluation of models built for interpretability . In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 7, 59\u201367 . Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Samuel J Gershman, and Finale Doshi-Velez. 2019. Human evaluation of models built for interpretability. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 7, 59\u201367."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Pingchuan Ma Shuai Wang and Jin Liu. 2020. Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models.. In IJCAI. 458\u2013465. Pingchuan Ma Shuai Wang and Jin Liu. 2020. Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models.. In IJCAI. 458\u2013465.","DOI":"10.24963\/ijcai.2020\/64"},{"key":"e_1_3_2_1_31_1","volume-title":"Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421.","author":"Petsiuk Vitali","year":"2018","unstructured":"Vitali Petsiuk , Abir Das , and Kate Saenko . 2018 . Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421. Vitali Petsiuk, Abir Das, and Kate Saenko. 2018. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3110335"},{"key":"e_1_3_2_1_34_1","volume-title":"Advances in Neural Information Processing Systems","author":"Sabour Sara","year":"2017","unstructured":"Sara Sabour , Nicholas Frosst , and Geoffrey E Hinton . 2017. Dynamic Routing Between Capsules . In Advances in Neural Information Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). 30, Curran Associates, Inc. . https:\/\/proceedings.neurips.cc\/paper\/ 2017 \/file\/2cad8fa47bbef282badbb8de5374b894-Paper.pdf Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic Routing Between Capsules. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). 30, Curran Associates, Inc.. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/2cad8fa47bbef282badbb8de5374b894-Paper.pdf"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.2352\/ISSN.2470-1173.2017.19.AVM-023"},{"key":"e_1_3_2_1_36_1","volume-title":"Evaluating the visualization of what a deep neural network has learned","author":"Samek Wojciech","year":"2016","unstructured":"Wojciech Samek , Alexander Binder , Gr\u00e9goire Montavon , Sebastian Lapuschkin , and Klaus-Robert M\u00fcller . 2016. Evaluating the visualization of what a deep neural network has learned . IEEE transactions on neural networks and learning systems, 28, 11 ( 2016 ), 2660\u20132673. Wojciech Samek, Alexander Binder, Gr\u00e9goire Montavon, Sebastian Lapuschkin, and Klaus-Robert M\u00fcller. 2016. Evaluating the visualization of what a deep neural network has learned. IEEE transactions on neural networks and learning systems, 28, 11 (2016), 2660\u20132673."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2532875"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2013.6639100"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_2_1_40_1","volume-title":"In Workshop at International Conference on Learning Representations.","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan , Andrea Vedaldi , and Andrew Zisserman . 2014 . Deep inside convolutional networks: Visualising image classification models and saliency maps . In In Workshop at International Conference on Learning Representations. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. In In Workshop at International Conference on Learning Representations."},{"key":"e_1_3_2_1_41_1","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556."},{"key":"e_1_3_2_1_42_1","unstructured":"J Springenberg Alexey Dosovitskiy Thomas Brox and M Riedmiller. 2015. Striving for Simplicity: The All Convolutional Net. In ICLR (workshop track). J Springenberg Alexey Dosovitskiy Thomas Brox and M Riedmiller. 2015. Striving for Simplicity: The All Convolutional Net. In ICLR (workshop track)."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASWEC.2018.00021"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380420"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_1_47_1","volume-title":"Metamorphic Object Insertion for Testing Object Detection Systems. In 2020 35th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1053\u20131065","author":"Wang Shuai","year":"2020","unstructured":"Shuai Wang and Zhendong Su . 2020 . Metamorphic Object Insertion for Testing Object Detection Systems. In 2020 35th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1053\u20131065 . Shuai Wang and Zhendong Su. 2020. Metamorphic Object Insertion for Testing Object Detection Systems. In 2020 35th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 1053\u20131065."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00577"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00457"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-017-1059-x"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00920"},{"key":"e_1_3_2_1_54_1","volume-title":"29th USENIX Security Symposium (USENIX Security 20)","author":"Zhang Xinyang","year":"2020","unstructured":"Xinyang Zhang , Ningfei Wang , Hua Shen , Shouling Ji , Xiapu Luo , and Ting Wang . 2020 . Interpretable Deep Learning under Fire . In 29th USENIX Security Symposium (USENIX Security 20) . USENIX Association, 1659\u20131676. isbn:978-1-939133-17-5 https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/zhang-xinyang Xinyang Zhang, Ningfei Wang, Hua Shen, Shouling Ji, Xiapu Luo, and Ting Wang. 2020. Interpretable Deep Learning under Fire. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 1659\u20131676. isbn:978-1-939133-17-5 https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/zhang-xinyang"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.319"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3241979"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2478001"}],"event":{"name":"ISSTA '22: 31st ACM SIGSOFT International Symposium on Software Testing and Analysis","location":"Virtual South Korea","acronym":"ISSTA '22","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3533767.3534225","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3533767.3534225","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T18:43:40Z","timestamp":1750272220000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3533767.3534225"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,18]]},"references-count":56,"alternative-id":["10.1145\/3533767.3534225","10.1145\/3533767"],"URL":"https:\/\/doi.org\/10.1145\/3533767.3534225","relation":{},"subject":[],"published":{"date-parts":[[2022,7,18]]},"assertion":[{"value":"2022-07-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}