{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T08:35:14Z","timestamp":1790757314990,"version":"4.1.0"},"reference-count":93,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,2,28]],"date-time":"2023-02-28T00:00:00Z","timestamp":1677542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation Graduate Research Fellowship Program","award":["DGE1745016"],"award-info":[{"award-number":["DGE1745016"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput.-Hum. Interact."],"published-print":{"date-parts":[[2023,2,28]]},"abstract":"<jats:p>\n            Data scientists require rich mental models of how AI systems behave to effectively train, debug, and work with them. Despite the prevalence of AI analysis tools, there is no general theory describing how people make sense of what their models have learned. We frame this process as a form of sensemaking and derive a framework describing how data scientists develop mental models of AI behavior. To evaluate the framework, we show how existing AI analysis tools fit into this sensemaking process and use it to design\n            <jats:sc>AIFinnity<\/jats:sc>\n            , a system for analyzing image-and-text models. Lastly, we explored how data scientists use a tool developed with the framework through a think-aloud study with 10 data scientists tasked with using\n            <jats:sc>AIFinnity<\/jats:sc>\n            to pick an image captioning model. We found that\n            <jats:sc>AIFinnity<\/jats:sc>\n            \u2019s sensemaking workflow reflected participants\u2019 mental processes and enabled them to discover and validate diverse AI behaviors.\n          <\/jats:p>","DOI":"10.1145\/3542921","type":"journal-article","created":{"date-parts":[[2022,6,11]],"date-time":"2022-06-11T22:49:57Z","timestamp":1654987797000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":41,"title":["What Did My AI Learn? How Data Scientists Make Sense of Model Behavior"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0348-3362","authenticated-orcid":false,"given":"\u00c1ngel Alexander","family":"Cabrera","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3301-1297","authenticated-orcid":false,"given":"Marco","family":"Tulio Ribeiro","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, Washington"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4217-627X","authenticated-orcid":false,"given":"Bongshin","family":"Lee","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, Washington"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8885-8367","authenticated-orcid":false,"given":"Robert","family":"Deline","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, Washington"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8369-3847","authenticated-orcid":false,"given":"Adam","family":"Perer","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5022-9343","authenticated-orcid":false,"given":"Steven M.","family":"Drucker","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, Washington"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,3,7]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702509"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.5465\/amle.2011.0007"},{"key":"e_1_3_2_5_2","unstructured":"Julia Angwin Jeff Larson Surya Mattu and Lauren Kirchner. 2016. Machine Bias. ProPublica. Retrieved June 2 2021 from https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2019.2942288"},{"key":"e_1_3_2_7_2","unstructured":"Josh Attenberg Panagiotis G. Ipeirotis and Foster Provost. 2011. Beat the machine: Challenging workers to find the unknown unknowns. In Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_2_8_2","first-page":"1463","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI\u20192018)","author":"Bansal Gagan","year":"2018","unstructured":"Gagan Bansal and Daniel S. Weld. 2018. A coverage-based utility model for identifying unknown unknowns. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI\u20192018). 1463\u20131470."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3502102"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/989863.989865"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3375624"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.186"},{"key":"e_1_3_2_13_2","first-page":"77","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency.","volume":"81","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency.Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, 77\u201391. Retrieved from https:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3479569"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/VAST47406.2019.8986948"},{"key":"e_1_3_2_16_2","first-page":"249","volume-title":"Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics","author":"Callison-Burch Chris","year":"2006","unstructured":"Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of bleu in machine translation research. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Trento, Italy, 249\u2013256. Retrieved from https:\/\/aclanthology.org\/E06-1032."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030443"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415865"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1978967"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172950"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDAR.2017.157"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2019.00139"},{"key":"e_1_3_2_23_2","article-title":"Interactive model cards: A human-centered approach to model documentation","author":"Crisan Anamaria","year":"2022","unstructured":"Anamaria Crisan, Margaret Drouhard, Jesse Vig, and Nazneen Rajani. 2022. Interactive model cards: A human-centered approach to model documentation. arXiv:2205.02894. Retrieved from https:\/\/arxiv.org\/abs\/2205.02894.","journal-title":"arXiv:2205.02894"},{"key":"e_1_3_2_24_2","article-title":"The spotlight: A general method for discovering systematic errors in deep learning models","author":"d\u2019Eon Greg","year":"2021","unstructured":"Greg d\u2019Eon, Jason d\u2019Eon, James R. Wright, and Kevin Leyton-Brown. 2021. The spotlight: A general method for discovering systematic errors in deep learning models. arXiv:2107.00758. Retrieved from https:\/\/arxiv.org\/abs\/2107.00758.","journal-title":"arXiv:2107.00758"},{"key":"e_1_3_2_25_2","article-title":"Domino: Discovering systematic errors with cross-modal embeddings","author":"Eyuboglu Sabri","year":"2022","unstructured":"Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher R\u00e9. 2022. Domino: Discovering systematic errors with cross-modal embeddings. arXiv:2203.14960. Retrieved from https:\/\/arxiv.org\/abs\/2203.14960.","journal-title":"arXiv:2203.14960"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207711"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3134680"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458723"},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","unstructured":"Karan Goel Nazneen Rajani Jesse Vig Samson Tan Jason Wu Stephan Zheng Caiming Xiong Mohit Bansal and Christopher R\u00e9. 2021. Robustness gym: Unifying the NLP evaluation landscape. arXiv:2101.04840. Retrieved from http:\/\/arxiv.org\/abs\/2101.04840.","DOI":"10.18653\/v1\/2021.naacl-demos.6"},{"key":"e_1_3_2_30_2","first-page":"1","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow Ian J.","year":"2015","unstructured":"Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. InProceedings of the 3rd International Conference on Learning Representations (2015), 1\u201311.","journal-title":"Proceedings of the 3rd International Conference on Learning Representations"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2147783.2147788"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501823"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702561"},{"issue":"3","key":"e_1_3_2_34_2","first-page":"309","article-title":"Organizational culture and individual sensemaking : A schema-based perspective","volume":"5","author":"Harris Stanley G.","year":"1994","unstructured":"Stanley G. Harris and Stanley G. Harris. 1994. Organizational culture and individual sensemaking : A schema-based perspective. INFORMS 5, 3 (1994), 309\u2013321.","journal-title":"INFORMS"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380339"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3012063"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376177"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445735"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3392878"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462527"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939502.2939503"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_2_45_2","article-title":"Sensible AI: Re-imagining interpretability and explainability using sensemaking theory","author":"Kaur Harmanpreet","year":"2022","unstructured":"Harmanpreet Kaur, Eytan Adar, Eric Gilbert, and Cliff Lampe. 2022. Sensible AI: Re-imagining interpretability and explainability using sensemaking theory. arXiv:2205.05057. Retrieved from https:\/\/arxiv.org\/abs\/2205.05057.","journal-title":"arXiv:2205.05057"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274357"},{"key":"e_1_3_2_47_2","article-title":"Dynabench: Rethinking benchmarking in NLP","author":"Kiela Douwe","year":"2021","unstructured":"Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. 2021. Dynabench: Rethinking benchmarking in NLP. arXiv:2104.14337. Retrieved from https:\/\/arxiv.org\/abs\/2104.14337.","journal-title":"arXiv:2104.14337"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2754374"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531644"},{"key":"e_1_3_2_50_2","first-page":"2124","volume-title":"Proceedings of the 31st AAAI Conference on Artificial Intelligence","author":"Lakkaraju Himabindu","year":"2017","unstructured":"Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Eric Horvitz. 2017. Identifying unknown unknowns in the open world: Representations and policies for guided exploration. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. AAAI, 2124\u20132132."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3028948"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274374"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380306"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308561.3354629"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376445"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.5465\/19416520.2014.873177"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300356"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2019.8870157"},{"key":"e_1_3_2_60_2","unstructured":"National Transportation Safety Board. 2019. Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian. Retrieved June 6 2021 from https:\/\/www.ntsb.gov\/news\/events\/Pages\/2019-HWY18MH010-BMG.aspx."},{"key":"e_1_3_2_61_2","first-page":"10","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","author":"Nushi Besmira","year":"2018","unstructured":"Besmira Nushi, Ece Kamar, and Eric Horvitz. 2018. Towards accountable AI: Hybrid human-machine analyses for characterizing system failure. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing. 10."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368555.3384468"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449205"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13398-014-0173-7.2"},{"key":"e_1_3_2_65_2","unstructured":"Mahima Pushkarna James Wexler and Jimbo Wilson. 2017. Facets: An Open Source Visualization Tool for Machine Learning Training Data. (2017). Retrieved June 18 2021 from https:\/\/pair-code.github.io\/facets\/."},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1138-y"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598828"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.442"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/288392.288596"},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-02197-8","volume-title":"HCI Theory: classical, modern, and contemporary","author":"Rogers Yvonne.","year":"2012","unstructured":"Yvonne. Rogers. 2012. HCI Theory: classical, modern, and contemporary. Synthesis Lectures on Human-Centered Informatics 5, 2 (2012), 1\u2013129."},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/169059.169209"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445518"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382494.3410681"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1038\/515151a"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3415224"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-019-0197-0"},{"key":"e_1_3_2_78_2","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Singla Sahil","year":"2021","unstructured":"Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, and Eric Horvitz. 2021. Understanding failures of deep networks via robust feature extraction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation\/IEEE. Retrieved from https:\/\/openaccess.thecvf.com\/content\/CVPR2021\/papers\/Singla_Understanding_Failures_of_Deep_Networks_via_Robust_Feature_Extraction_CVPR_2021_paper.pdf."},{"key":"e_1_3_2_79_2","volume-title":"Grounded Theory in Practice","author":"Strauss Anselm L.","year":"1997","unstructured":"Anselm L. Strauss and Juliet M. Corbin (Eds.). 1997. Grounded Theory in Practice. Sage Publications, Thousand Oaks."},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1177\/0735275112457914"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1177\/0149206312457823"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2937083"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00871"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-011-9718-x"},{"key":"e_1_3_2_86_2","volume-title":"Sensemaking in Organizations","author":"Weick Karl E.","year":"1995","unstructured":"Karl E. Weick. 1995. Sensemaking in Organizations. Sage Publications, Thousand Oaks."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.1050.0133"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934619"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.5465\/amj.2008.0843"},{"key":"e_1_3_2_90_2","first-page":"747","article-title":"{E}rrudite: Scalable, reproducible, and testable error analysis","author":"Wu Tongshuang","year":"2019","unstructured":"Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel Weld. 2019. {E}rrudite: Scalable, reproducible, and testable error analysis. Proceedings of the 57th Conference of the Association for Computational Linguistics. 57 (2019), 747\u2013763. Retrieved from https:\/\/www.aclweb.org\/anthology\/P19-1073.","journal-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196729"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_3_2_94_2","article-title":"Software engineering practice in the development of deep learning applications","author":"Zhang Xufan","year":"2019","unstructured":"Xufan Zhang, Yilin Yang, Yang Feng, and Zhenyu Chen. 2019. Software engineering practice in the development of deep learning applications. arXiv:1910.03156. Retrieved from https:\/\/arxiv.org\/abs\/1910.03156.","journal-title":"arXiv:1910.03156"}],"container-title":["ACM Transactions on Computer-Human Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3542921","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3542921","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:22Z","timestamp":1750186942000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3542921"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,28]]},"references-count":93,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2,28]]}},"alternative-id":["10.1145\/3542921"],"URL":"https:\/\/doi.org\/10.1145\/3542921","relation":{},"ISSN":["1073-0516","1557-7325"],"issn-type":[{"value":"1073-0516","type":"print"},{"value":"1557-7325","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,28]]},"assertion":[{"value":"2021-04-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-05","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}