{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T10:49:29Z","timestamp":1779101369670,"version":"3.51.4"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T00:00:00Z","timestamp":1618272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,4,13]]},"abstract":"<jats:p>Human labeling of training data is often a time-consuming, expensive part of machine learning. In this paper, we study \"batch labeling\", an AI-assisted UX paradigm, that aids data labelers by allowing a single labeling action to apply to multiple records. We ran a large scale study on Mechanical Turk with 156 participants to investigate labeler-AI-batching system interaction. We investigate the efficacy of the system when compared to a single-item labeling interface (i.e., labeling one record at-a-time), and evaluate the impact of batch labeling on accuracy and time. We further investigate the impact of AI algorithm quality and its effects on the labelers' overreliance, as well as potential mechanisms for mitigating it. Our work offers implications for the design of batch labeling systems and for work practices focusing on labeler-AI-batching system interaction.<\/jats:p>","DOI":"10.1145\/3449163","type":"journal-article","created":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T17:51:09Z","timestamp":1619113869000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":42,"title":["AI-Assisted Human Labeling"],"prefix":"10.1145","volume":"5","author":[{"given":"Zahra","family":"Ashktorab","sequence":"first","affiliation":[{"name":"IBM Research, Yorktown Heights, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Desmond","sequence":"additional","affiliation":[{"name":"IBM Research, Yorktown Heights, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Josh","family":"Andres","sequence":"additional","affiliation":[{"name":"IBM Research Australia, Melbourne, VIC, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Muller","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Narendra Nath","family":"Joshi","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michelle","family":"Brachman","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aabhas","family":"Sharma","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kristina","family":"Brimijoin","sequence":"additional","affiliation":[{"name":"IBM Research, Hastings on Hudson, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qian","family":"Pan","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christine T.","family":"Wolf","sequence":"additional","affiliation":[{"name":"Independent Researcher, San Jose, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Evelyn","family":"Duesterwald","sequence":"additional","affiliation":[{"name":"IBM Research, Yorktown Heights, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Casey","family":"Dugan","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Werner","family":"Geyer","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Darrell","family":"Reimer","sequence":"additional","affiliation":[{"name":"IBM Research, Yorktown Heights, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,4,22]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Samy S Abu-Naser, and Mousa Abu Laban.","author":"Abu Ghali Mahmoud J","year":"2018","unstructured":"Mahmoud J Abu Ghali, Abdullah Abu Ayyad, Samy S Abu-Naser, and Mousa Abu Laban. 2018. An intelligent tutoring system for teaching english grammar. (2018)."},{"key":"e_1_2_1_2_1","volume-title":"Artificial intelligence and the future of humans","author":"Anderson Janna","year":"2018","unstructured":"Janna Anderson, Lee Rainie, and Alex Luchsinger. 2018. Artificial intelligence and the future of humans. Pew Research Center (2018)."},{"key":"e_1_2_1_3_1","volume-title":"Scenario-based XAI for Humanitarian Aid Forecasting. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1--8.","author":"Andres Josh","year":"2020","unstructured":"Josh Andres, Christine T Wolf, Sergio Cabrero Barros, Erick Oduor, Rahul Nair, Alexander Kj\u00e6rum, Anders Bech Tharsgaard, and Bo Schwartz Madsen. 2020. Scenario-based XAI for Humanitarian Aid Forecasting. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1--8."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487600"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the First Workshop on Metaphor in NLP. 77--86","author":"Badryzlova Yulia","year":"2013","unstructured":"Yulia Badryzlova, Natalia Shekhtman, Yekaterina Isaeva, and Ruslan Kerimov. 2013. Annotating a Russian corpus of conceptual metaphor: A bottom-up approach. In Proceedings of the First Workshop on Metaphor in NLP. 77--86."},{"key":"e_1_2_1_6_1","volume-title":"Driving aviation forward","author":"Banks Victoria A","year":"2019","unstructured":"Victoria A Banks, Katherine L Plant, and Neville A Stanton. 2019. Driving aviation forward; contrasting driving automation and aviation automation. Theoretical issues in ergonomics science, Vol. 20, 3 (2019), 250--264."},{"key":"e_1_2_1_7_1","volume-title":"NIPS workshop on computational social science and the wisdom of crowds.","author":"Brew Anthony","year":"2010","unstructured":"Anthony Brew, Derek Greene, and P\u00e1draig Cunningham. 2010. The interaction between supervised learning and crowdsourcing. In NIPS workshop on computational social science and the wisdom of crowds."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)","author":"Caines Andrew","year":"2016","unstructured":"Andrew Caines, Christian Bentz, Calbert Graham, Tim Polzehl, and Paula Buttery. 2016. Crowdsourcing a multi-lingual speech corpus: Recording, transcription and annotation of the crowdis corpora. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16). 2145--2152."},{"key":"e_1_2_1_10_1","volume-title":"Universal Sentence Encoder. arXiv preprint","author":"Cer Daniel","year":"2018","unstructured":"Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal Sentence Encoder. arXiv preprint, Vol. abs\/1803.11175 (2018). arxiv: 1803.11175 http:\/\/arxiv.org\/abs\/1803.11175"},{"key":"e_1_2_1_11_1","unstructured":"Alan Chamberlain Alessio Malizia and David De Roure. 2017. An agent on my shoulder: AI privacy and the application of human-like computing technologies to music creation. (2017)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300789"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-012-1352-1"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027063.3027072"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3185517"},{"key":"e_1_2_1_16_1","first-page":"226","article-title":"A density-based algorithm for discovering clusters in large spatial databases with noise","volume":"96","author":"Ester Martin","year":"1996","unstructured":"Martin Ester, Hans-Peter Kriegel, J\u00f6rg Sander, Xiaowei Xu, et almbox. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise.. In Kdd, Vol. 96. 226--231.","journal-title":"Kdd"},{"key":"e_1_2_1_17_1","volume-title":"Constructing large scale biomedical knowledge bases from scratch with rapid annotation of interpretable patterns. arXiv preprint arXiv:1907.01417","author":"Fauqueur Julien","year":"2019","unstructured":"Julien Fauqueur, Ashok Thillaisundara, and Theodosia Togia. 2019. Constructing large scale biomedical knowledge bases from scratch with rapid annotation of interpretable patterns. arXiv preprint arXiv:1907.01417 (2019)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/1866696.1866709"},{"key":"e_1_2_1_19_1","volume-title":"Collaborative Annotation for Reliable Natural Language Processing: Technical and Sociological Aspects","author":"Fort Kar\u00ebn","unstructured":"Kar\u00ebn Fort. 2016. Collaborative Annotation for Reliable Natural Language Processing: Technical and Sociological Aspects. John Wiley & Sons."},{"key":"e_1_2_1_20_1","unstructured":"J Glanz M Suhartono and H Beech. 2018. In Indonesia Lion Air crash black box data reveal pilots' struggle to regain control. The New York Times (2018)."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000089"},{"key":"e_1_2_1_22_1","volume-title":"A survey of methods for explaining black box models. ACM computing surveys (CSUR)","author":"Guidotti Riccardo","year":"2018","unstructured":"Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys (CSUR), Vol. 51, 5 (2018), 1--42."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1807184115"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376177"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376428"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/302979.303030"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300469"},{"key":"e_1_2_1_28_1","unstructured":"Kaggle. 2017. 2017 Kaggle ML & DS Survey. https:\/\/www.kaggle.com\/kaggle\/kaggle-survey-2017"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063860"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884783"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300717"},{"key":"e_1_2_1_32_1","volume-title":"Advances in experimental social psychology.","author":"Langer Ellen J","unstructured":"Ellen J Langer. 1989. Minding matters: The consequences of mindlessness--mindfulness. In Advances in experimental social psychology. Vol. 22. Elsevier, 137--173."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300649"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-89350-9_6"},{"key":"e_1_2_1_35_1","unstructured":"George Lawton. 2018. The nine roles you need on your data science research team. TechTarget. https:\/\/searchcio.techtarget.com\/news\/252445605\/The-nine-roles-you-need-on-your-data-science-research-team."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/00140139208967392"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2835776.2835797"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the fifth Berkeley symposium on mathematical statistics and probability","volume":"1","author":"James","unstructured":"James MacQueen et almbox. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Vol. 1. Oakland, CA, USA, 281--297."},{"key":"e_1_2_1_39_1","volume-title":"Introduction to information retrieval","author":"Manning Christopher D","unstructured":"Christopher D Manning, Prabhakar Raghavan, and Hinrich Sch\u00fctze. 2008. Introduction to information retrieval. Cambridge university press."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2756506.2767971"},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the 1st international conference on augmented cognition. 22--27","author":"Miller Christopher A","year":"2005","unstructured":"Christopher A Miller, Harry Funk, Robert Goldman, John Meisner, and Peggy Wu. 2005. Implications of adaptive vs. adaptable UIs on decision making: Why ?automated adaptiveness\" is not always the right answer. In Proceedings of the 1st international conference on augmented cognition. 22--27."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702553"},{"key":"e_1_2_1_43_1","volume-title":"Human-Centered Study of Data Science Work Practices. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, W15","author":"Muller Michael","year":"2019","unstructured":"Michael Muller, Melanie Feinberg, Timothy George, Steven J Jackson, Bonnie E John, Mary Beth Kery, and Samir Passi. 2019. Human-Centered Study of Data Science Work Practices. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, W15."},{"key":"e_1_2_1_44_1","volume-title":"Running experiments on amazon mechanical turk. Judgment and Decision making","author":"Paolacci Gabriele","year":"2010","unstructured":"Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running experiments on amazon mechanical turk. Judgment and Decision making, Vol. 5, 5 (2010), 411--419."},{"key":"e_1_2_1_45_1","volume-title":"Complacency and bias in human use of automation: An attentional integration. Human factors","author":"Parasuraman Raja","year":"2010","unstructured":"Raja Parasuraman and Dietrich H Manzey. 2010. Complacency and bias in human use of automation: An attentional integration. Human factors, Vol. 52, 3 (2010), 381--410."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327108ijap0301_1"},{"key":"e_1_2_1_47_1","volume-title":"Humans and automation: Use, misuse, disuse, abuse. Human factors","author":"Parasuraman Raja","year":"1997","unstructured":"Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human factors, Vol. 39, 2 (1997), 230--253."},{"key":"e_1_2_1_48_1","volume-title":"Building data science teams. \"O'Reilly Media","author":"Patil DJ","unstructured":"DJ Patil. 2011. Building data science teams. \"O'Reilly Media, Inc.\"."},{"key":"e_1_2_1_49_1","volume-title":"Machine Teaching by Domain Experts: Towards More Humane, Inclusive, and Intelligent Machine Learning Systems. arXiv preprint arXiv:1908.08931","author":"Pinhanez Claudio","year":"2019","unstructured":"Claudio Pinhanez. 2019. Machine Teaching by Domain Experts: Towards More Humane, Inclusive, and Intelligent Machine Learning Systems. arXiv preprint arXiv:1908.08931 (2019)."},{"key":"e_1_2_1_50_1","volume-title":"Interactive machine teaching: a human-centered approach to building machine-learned models. Human-Computer Interaction","author":"Ramos Gonzalo","year":"2020","unstructured":"Gonzalo Ramos, Christopher Meek, Patrice Simard, Jina Suh, and Soroush Ghorashi. 2020. Interactive machine teaching: a human-centered approach to building machine-learned models. Human-Computer Interaction (2020), 1--39."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-019-00552-1"},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL). 410--420","author":"Rosenberg Andrew","year":"2007","unstructured":"Andrew Rosenberg and Julia Hirschberg. 2007. V-measure: A conditional entropy-based external cluster evaluation measure. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL). 410--420."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376290"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1515\/9783035610673"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.9781\/ijimai.2017.09.001"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1741118"},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. 102--107","author":"Stenetorp Pontus","year":"2012","unstructured":"Pontus Stenetorp, Sampo Pyysalo, Goran Topi\u0107, Tomoko Ohta, Sophia Ananiadou, and Jun'ichi Tsujii. 2012. BRAT: a web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. 102--107."},{"key":"e_1_2_1_60_1","volume-title":"Don't freak over Boeing's self-flying plane-robots already run the skies'. Wired,(online","author":"Stewart Jack","year":"2017","unstructured":"Jack Stewart. 2017. Don't freak over Boeing's self-flying plane-robots already run the skies'. Wired,(online, 6 September 2017) (2017)."},{"key":"e_1_2_1_61_1","volume-title":"Healthy Data: Policy solutions for big data and AI innovation in health","author":"Stinson Catherine","year":"2018","unstructured":"Catherine Stinson. 2018. Healthy Data: Policy solutions for big data and AI innovation in health. Mowat Centre for Policy Innovation."},{"key":"e_1_2_1_62_1","volume-title":"Plans and situated actions: The problem of human-machine communication","author":"Suchman Lucy A","unstructured":"Lucy A Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge university press."},{"key":"e_1_2_1_63_1","volume-title":"What is human-machine interaction. Cognition, computing, and cooperation","author":"Suchman Lucy A","year":"1990","unstructured":"Lucy A Suchman. 1990. What is human-machine interaction. Cognition, computing, and cooperation (1990), 25--55."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300768"},{"key":"e_1_2_1_65_1","volume-title":"Semi-automatic ground truth generation using unsupervised clustering and limited manual labeling: Application to handwritten character recognition. Pattern recognition letters","author":"Vajda Szil\u00e1rd","year":"2015","unstructured":"Szil\u00e1rd Vajda, Yves Rangoni, and Hubert Cecotti. 2015. Semi-automatic ground truth generation using unsupervised clustering and limited manual labeling: Application to handwritten character recognition. Pattern recognition letters, Vol. 58 (2015), 23--28."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359313"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377325.3377538"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJODE.2010.035191"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2011.207"},{"key":"e_1_2_1_70_1","volume-title":"Mouni Reddy, and Geoff Zweig.","author":"Williams Jason D","year":"2015","unstructured":"Jason D Williams, Nobal B Niraula, Pradeep Dasigi, Aparna Lakshmiratan, Carlos Garcia Jurado Suarez, Mouni Reddy, and Geoff Zweig. 2015. Rapidly scaling dialog systems with interactive learning. In Natural Language Dialog Systems and Intelligent Assistants. Springer, 1--13."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1017\/S104909651500116X"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359245"},{"key":"e_1_2_1_73_1","volume-title":"Proceedings of the Fourth BioCreative Challenge Evaluation Workshop","volume":"1","author":"Yepes Antonio Jimeno","year":"2013","unstructured":"Antonio Jimeno Yepes, Mariana Neves, and Karin Verspoor. 2013. Brat2BioC: conversion tool between brat and BioC. In Proceedings of the Fourth BioCreative Challenge Evaluation Workshop, Vol. 1. 46--53."},{"key":"e_1_2_1_74_1","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 1--6.","author":"Yimam Seid Muhie","year":"2013","unstructured":"Seid Muhie Yimam, Iryna Gurevych, Richard Eckart de Castilho, and Chris Biemann. 2013. Webanno: A flexible, web-based and visually supported system for distributed annotations. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 1--6."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300509"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392826"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449163","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449163","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:17Z","timestamp":1750191497000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449163"}},"subtitle":["Batching for Efficiency without Overreliance"],"short-title":[],"issued":{"date-parts":[[2021,4,13]]},"references-count":74,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2021,4,13]]}},"alternative-id":["10.1145\/3449163"],"URL":"https:\/\/doi.org\/10.1145\/3449163","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,13]]},"assertion":[{"value":"2021-04-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}