{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,16]],"date-time":"2026-11-16T16:47:08Z","timestamp":1794847628163,"version":"4.9.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,2,7]],"date-time":"2023-02-07T00:00:00Z","timestamp":1675728000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61732008, 62002194"],"award-info":[{"award-number":["61732008, 62002194"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Tsinghua University Guoqiang Research Institute","id":[{"id":"https:\/\/ror.org\/03cve4549","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>Legal case retrieval, which aims to retrieve relevant cases given a query case, has drawn increasing research attention in recent years. While much research has worked on developing automatic retrieval models, how to characterize relevance in this specialized information retrieval (IR) task is still an open question. Towards an in-depth understanding of relevance judgments, we conduct a laboratory user study that involves 72 participants of different domain expertise. In the user study, we collect the relevance score along with detailed explanations for the relevance judgment and various measures of the judgment process. From the collected data, we observe that both the subjective (e.g., domain expertise) and objective (e.g., query\/case property) factors influence the relevance judgment process. By investigating the collected user explanations, we identify task-specific patterns of user attention distribution and re-think the criteria for relevance judgments. Moreover, we investigate the similarity in attention distribution between models and users. Further, we propose a two-stage framework that utilizes user attention to improve relevance estimation for legal case retrieval. Our study sheds light on understanding relevance judgments in legal case retrieval and provides implications for improving the design of corresponding retrieval systems.<\/jats:p>","DOI":"10.1145\/3569929","type":"journal-article","created":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T07:47:07Z","timestamp":1666943227000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":24,"title":["Understanding Relevance Judgments in Legal Case Retrieval"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1727-8311","authenticated-orcid":false,"given":"Yunqiu","family":"Shao","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, Institute for Internet Judiciary, Tsinghua University, Zhongguancun Laboratory, Quan Cheng Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2539-8954","authenticated-orcid":false,"given":"Yueyue","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Institute for Internet Judiciary, Tsinghua University, Zhongguancun Laboratory, Quan Cheng Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0140-4512","authenticated-orcid":false,"given":"Yiqun","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Institute for Internet Judiciary, Tsinghua University, Zhongguancun Laboratory, Quan Cheng Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9257-5498","authenticated-orcid":false,"given":"Jiaxin","family":"Mao","sequence":"additional","affiliation":[{"name":"Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8762-8268","authenticated-orcid":false,"given":"Shaoping","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Institute for Internet Judiciary, Tsinghua University, Zhongguancun Laboratory, Quan Cheng Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,2,7]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR\u201915)","author":"Allegretti Marco","year":"2015","unstructured":"Marco Allegretti, Yashar Moshfeghi, Maria Hadjigeorgieva, Frank E. Pollick, Joemon M. Jose, and Gabriella Pasi. 2015. When relevance judgement is happening? An EEG-based study. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR\u201915). 719\u2013722."},{"key":"e_1_3_2_3_2","first-page":"797","article-title":"Open access in a closed universe: Lexis, Westlaw, law schools, and the legal information market","volume":"10","author":"Arewa Olufunmilayo B.","year":"2006","unstructured":"Olufunmilayo B. Arewa. 2006. Open access in a closed universe: Lexis, Westlaw, law schools, and the legal information market. Lewis & Clark L. Rev. 10 (2006), 797.","journal-title":"Lewis & Clark L. Rev."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.5555\/186141.186147"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-012-9131-x"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Paheli Bhattacharya Kripabandhu Ghosh Saptarshi Ghosh Arindam Pal Parth Mehta Arnab Bhattacharya and Prasenjit Majumder. 2019. FIRE 2019 AILA Track: Artificial Intelligence for Legal Assistance. In Proceedings of the 11th Forum for Information Retrieval Evaluation (FIRE\u201919) . 4\u20136.","DOI":"10.1145\/3368567.3368587"},{"key":"e_1_3_2_7_2","first-page":"85","volume-title":"Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM\u201920)","author":"Bolotova Valeria","year":"2020","unstructured":"Valeria Bolotova, Vladislav Blinov, Yukun Zheng, W. Bruce Croft, Falk Scholer, and Mark Sanderson. 2020. Do people and neural nets pay attention to the same words: Studying eye-tracking data for non-factoid QA evaluation. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM\u201920). 85\u201394."},{"issue":"3","key":"e_1_3_2_8_2","article-title":"The IIR evaluation model: A framework for evaluation of interactive information retrieval systems","volume":"8","author":"Borlund Pia","year":"2003","unstructured":"Pia Borlund. 2003. The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research 8, 3 (2003).","journal-title":"Information Research"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"James Carifio and Rocco Perla. 2008. Resolving the 50-year debate around using and misusing Likert scales. Medical Education 42 12 (2008) 1150\u20131152.","DOI":"10.1111\/j.1365-2923.2008.03172.x"},{"key":"e_1_3_2_10_2","article-title":"Factors affecting relevance judgment: A report from TREC legal track","author":"Chu Heting","year":"2011","unstructured":"Heting Chu. 2011. Factors affecting relevance judgment: A report from TREC legal track. Journal of Documentation 67, 2 (2011), 264\u2013278.","journal-title":"Journal of Documentation"},{"key":"e_1_3_2_11_2","unstructured":"Factors Determining the Performance of Indexing Systems 1966 Aslib\u2013Cranfield research project"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(99)00072-2"},{"key":"e_1_3_2_13_2","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1177\/1536867X1501500117"},{"key":"e_1_3_2_15_2","first-page":"229","article-title":"WESTLAW and the American Digest classification scheme","volume":"84","author":"Doyle John","year":"1992","unstructured":"John Doyle. 1992. WESTLAW and the American Digest classification scheme. Law. Libr. J. 84 (1992), 229.","journal-title":"Law. Libr. J."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0031619"},{"key":"e_1_3_2_17_2","volume-title":"Introduction to Information Behaviour","author":"Ford Nigel","year":"2015","unstructured":"Nigel Ford. 2015. Introduction to Information Behaviour. Facet Publishing."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0053912"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2637002.2637011"},{"issue":"3","key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1111\/jels.12230","article-title":"The German federal courts dataset 1950\u20132019: From paper archives to linked open data","volume":"16","author":"Hamann Hanjo","year":"2019","unstructured":"Hanjo Hamann. 2019. The German federal courts dataset 1950\u20132019: From paper archives to linked open data. Journal of Empirical Legal Studies 16, 3 (2019), 671\u2013688.","journal-title":"Journal of Empirical Legal Studies"},{"key":"e_1_3_2_21_2","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","author":"Holm Sture","year":"1979","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 2 (1979), 65\u201370.","journal-title":"Scandinavian Journal of Statistics"},{"key":"e_1_3_2_22_2","article-title":"HLDC: Hindi legal documents corpus","author":"Kapoor Arnav","year":"2022","unstructured":"Arnav Kapoor, Mudit Dhawan, Anmol Goel, T. H. Arjun, Akshala Bhatnagar, Vibhu Agrawal, Amul Agrawal, Arnab Bhattacharya, Ponnurangam Kumaraguru, and Ashutosh Modi. 2022. HLDC: Hindi legal documents corpus. arXiv preprint arXiv:2204.00806 (2022).","journal-title":"arXiv preprint arXiv:2204.00806"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1952.10483441"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609631"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271764"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1967.10482916"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463064"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463250"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10049"},{"key":"e_1_3_2_30_2","first-page":"1","article-title":"Unsupervised approaches for measuring textual similarity between legal court case reports","author":"Mandal Arpan","year":"2021","unstructured":"Arpan Mandal, Kripabandhu Ghosh, Saptarshi Ghosh, and Sekhar Mandal. 2021. Unsupervised approaches for measuring textual similarity between legal court case reports. Artificial Intelligence and Law 29, 1 (2021), 1\u201335.","journal-title":"Artificial Intelligence and Law"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3223045"},{"issue":"2","key":"e_1_3_2_33_2","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1561\/1500000025","article-title":"Information retrieval for e-discovery","volume":"7","author":"Oard Douglas W.","year":"2013","unstructured":"Douglas W. Oard and William Webber. 2013. Information retrieval for e-discovery. Information Retrieval 7, 2\u20133 (2013), 99\u2013237.","journal-title":"Information Retrieval"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3295750.3298946"},{"key":"e_1_3_2_35_2","volume-title":"JSAI International Symposium on Artificial Intelligence (JSAI-isAI\u201919)","author":"Rabelo Juliano","year":"2019","unstructured":"Juliano Rabelo, Mi-Young Kim, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. 2019. A summary of the COLIEE 2019 competition. In JSAI International Symposium on Artificial Intelligence (JSAI-isAI\u201919). Springer, 34\u201349."},{"key":"e_1_3_2_36_2","volume-title":"Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks (LREC\u201910)","author":"\u0158eh\u016f\u0159ek Radim","year":"2010","unstructured":"Radim \u0158eh\u016f\u0159ek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks (LREC\u201910). ELRA, Valletta, Malta, 45\u201350. http:\/\/is.muni.cz\/publication\/884893\/en."},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"Stephen E. Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval (SIGIR\u201994) . Springer 232\u2013241.","DOI":"10.1007\/978-1-4471-2099-5_24"},{"key":"e_1_3_2_38_2","article-title":"Legal information retrieval with generalized language models","author":"Rossi Juline","year":"2019","unstructured":"Juline Rossi and Evangelos Kanoulas. 2019. Legal information retrieval with generalized language models. In Proceedings of the 6th Competition on Legal Information Extraction\/Entailment (COLIEE\u201919).","journal-title":"Proceedings of the 6th Competition on Legal Information Extraction\/Entailment (COLIEE\u201919)"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(88)90021-0"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.5555\/1315930.1315947"},{"issue":"3","key":"e_1_3_2_41_2","first-page":"i\u2013109","article-title":"The notion of relevance in information science: Everybody knows what relevance is. But, what is it really?","volume":"8","author":"Saracevic Tefko","year":"2016","unstructured":"Tefko Saracevic. 2016. The notion of relevance in information science: Everybody knows what relevance is. But, what is it really? Synthesis Lectures on Information Concepts, Retrieval, and Services 8, 3 (2016), i\u2013109.","journal-title":"Synthesis Lectures on Information Concepts, Retrieval, and Services"},{"key":"e_1_3_2_42_2","volume-title":"Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL\u201921)","author":"Savelka Jaromir","year":"2021","unstructured":"Jaromir Savelka, Hannes Westermann, Karim Benyekhlef, Charlotte S. Alexander, Jayla C. Grant, David Restrepo Amariles, Rajaa El Hamdani, S\u00e9bastien Mee\u00f9s, Aurore Troussel, Micha\u0142 Araszkiewicz, et\u00a0al. 2021. Lex Rosetta: Transfer of predictive models across languages, jurisdictions, and legal domains. In Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL\u201921). 129\u2013138."},{"key":"e_1_3_2_43_2","volume-title":"Proceedings of the ASIS Annual Meeting","author":"Schamber Linda","year":"1991","unstructured":"Linda Schamber. 1991. Users\u2019 criteria for evaluation in a multimedia environment. Proceedings of the ASIS Annual Meeting 28 (1991), 126\u201333."},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Yunqiu Shao Jiaxin Mao Yiqun Liu Weizhi Ma Ken Satoh Min Zhang and Shaoping Ma. 2020. BERT-PLI: Modeling paragraph-level interactions for legal case retrieval. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI\u201920) . 3501\u20133507.","DOI":"10.24963\/ijcai.2020\/484"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462876"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/186141.186151"},{"issue":"1","key":"e_1_3_2_47_2","first-page":"n1","article-title":"A study of the information search behaviour of the millennial generation.","volume":"17","author":"Taylor Arthur","year":"2012","unstructured":"Arthur Taylor. 2012. A study of the information search behaviour of the millennial generation. Information Research: An International Electronic Journal 17, 1 (2012), n1.","journal-title":"Information Research: An International Electronic Journal"},{"key":"e_1_3_2_48_2","volume-title":"Proceedings of the 17th International Conference on Artificial Intelligence and Law (ICAIL\u201919)","author":"Tran Vu","year":"2019","unstructured":"Vu Tran, Minh Le Nguyen, and Ken Satoh. 2019. Building legal case retrieval systems with lexical matching and summarization using a pre-trained phrase scoring model. In Proceedings of the 17th International Conference on Artificial Intelligence and Law (ICAIL\u201919). 275\u2013282."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-017-9195-8"},{"key":"e_1_3_2_50_2","volume-title":"Advances in Neural Information Processing Systems (NIPS\u201917)","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NIPS\u201917). 5998\u20136008."},{"key":"e_1_3_2_51_2","first-page":"1275","volume-title":"Proceedings of the International Conference on Scientific Information (ICSI\u201959)","volume":"2","author":"Vickery Brian C.","year":"1959","unstructured":"Brian C. Vickery. 1959. The structure of information retrieval systems. In Proceedings of the International Conference on Scientific Information (ICSI\u201959), Vol. 2. 1275\u20131290."},{"issue":"2","key":"e_1_3_2_52_2","first-page":"19","article-title":"A survey on similarity measures in text mining","volume":"3","author":"Vijaymeena M. K.","year":"2016","unstructured":"M. K. Vijaymeena and K. Kavitha. 2016. A survey on similarity measures in text mining. Machine Learning and Applications: An International Journal 3, 2 (2016), 19\u201328.","journal-title":"Machine Learning and Applications: An International Journal"},{"key":"e_1_3_2_53_2","article-title":"Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned","author":"Voita Elena","year":"2019","unstructured":"Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv:1905.09418 (2019).","journal-title":"arXiv preprint arXiv:1905.09418"},{"key":"e_1_3_2_54_2","volume-title":"Proceedings of SIGIR Information Retrieval for E-discovery Workshop (SIRE\u201911)","volume":"1","author":"Wang Jianqiang","year":"2011","unstructured":"Jianqiang Wang. 2011. Accuracy, agreement, speed, and perceived difficulty of users\u2019 relevance judgments for e-discovery. In Proceedings of SIGIR Information Retrieval for E-discovery Workshop (SIRE\u201911), Vol. 1. Citeseer."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.5555\/1920331.1920438"},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","unstructured":"Robert Warren. 2011. University of waterloo at TREC 2011: A social networking approach to the legal learning track. In Text Retrieval Conference (TREC\u201911) .","DOI":"10.6028\/NIST.SP.500-296.legal-waterloo"},{"key":"e_1_3_2_57_2","volume-title":"JSAI International Symposium on Artificial Intelligence (ISAI-isAI\u201920)","author":"Westermann Hannes","year":"2020","unstructured":"Hannes Westermann, Jaromir Savelka, and Karim Benyekhlef. 2020. Paragraph similarity scoring and fine-tuned BERT for legal information retrieval and entailment. In JSAI International Symposium on Artificial Intelligence (ISAI-isAI\u201920). Springer, 269\u2013285."},{"key":"e_1_3_2_58_2","volume-title":"Proceedings of the 2nd ACM International Conference on Web Search and Data Mining (WSDM\u201909)","author":"White Ryen W.","year":"2009","unstructured":"Ryen W. White, Susan T. Dumais, and Jaime Teevan. 2009. Characterizing the influence of domain expertise on web search behavior. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining (WSDM\u201909). 132\u2013141."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401305"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3086512.3086540"},{"key":"e_1_3_2_61_2","volume-title":"Open Chinese Language Pre-trained Model Zoo","author":"Zhong Haoxi","year":"2019","unstructured":"Haoxi Zhong, Zhengyan Zhang, Zhiyuan Liu, and Maosong Sun. 2019. Open Chinese Language Pre-trained Model Zoo. Technical Report. https:\/\/github.com\/thunlp\/openclap."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569929","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3569929","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T14:07:50Z","timestamp":1750169270000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569929"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,7]]},"references-count":60,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3569929"],"URL":"https:\/\/doi.org\/10.1145\/3569929","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,7]]},"assertion":[{"value":"2021-10-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-18","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}