{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T08:24:41Z","timestamp":1781079881172,"version":"3.54.1"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,3,17]],"date-time":"2023-03-17T00:00:00Z","timestamp":1679011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput.-Hum. Interact."],"published-print":{"date-parts":[[2023,4,30]]},"abstract":"<jats:p>This article describes the successful collaboration \u201cin the wild\u201d between Clinical Documentation Integrity Specialists (CDIS) and an Artificial Intelligence (AI)-embedded software to conduct knowledge work. CDIS review patient charts in near real-time to improve clinicians\u2019 documentation, with the goal to make medical documentation more accurate, consistent and complete. CDIS collaborate with an AI-embedded \u201cComputer Assisted Coding\u201d (CAC) system that scans records from the Electronic Healthcare Record and auto-suggests codes based on natural language processing. CDIS find the CAC's suggestions are often inaccurate\u2014often humorously so. Still, they find the CAC to be a useful helper, like Robin is to Batman. This human-AI collaboration is contingent on several factors: the flexible integration of the AI into the workflow similar to the notion of unremarkable AI; supporting the CDIS\u2019 sensemaking; the CDIS\u2019 knowledge about the CAC being predictably unreliable, an experience by the CDIS of the AI's value; humans remaining in control; and ability to experiment with the AI, which spurs reflection and learning for these knowledge workers.<\/jats:p>","DOI":"10.1145\/3569892","type":"journal-article","created":{"date-parts":[[2022,10,27]],"date-time":"2022-10-27T12:27:56Z","timestamp":1666873676000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":53,"title":["Batman and Robin in Healthcare Knowledge Work: Human-AI Collaboration by Clinical Documentation Integrity Specialists"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3698-0675","authenticated-orcid":false,"given":"Claus","family":"Bossen","sequence":"first","affiliation":[{"name":"Digital Design &amp; Information Studies, Aarhus University, Aarhus N, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2172-8100","authenticated-orcid":false,"given":"Kathleen H.","family":"Pine","sequence":"additional","affiliation":[{"name":"College of Health Solutions, Arizona State University, Phoenix, AZ, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,3,17]]},"reference":[{"key":"e_1_3_3_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300233"},{"key":"e_1_3_3_3_1","volume-title":"ACI Open","unstructured":"Sally L. Baxter, Jeremy S. Bass, and Amy M. Sitapati. 2020. Barriers to implementing an artificial intelligence model for unplanned readmissions. ACI Open 4, 2 (2020), e108--e113."},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445717"},{"key":"e_1_3_3_5_1","volume-title":"Social Problems","unstructured":"S. Brayne and A. Christin. 2021. Technologies of crime prediction: The reception of algorithms in policing and criminal courts. Social Problems 68, 3 (2021), 608--624."},{"key":"e_1_3_3_6_1","volume-title":"Cac 2010-11 Industry Outlook and Resources Report","year":"2011","unstructured":"June Bronnert, Bonnie S. Cassidy, Shirley Eichenwald Maki, Heather Eminger James Flanagan, Mark Morsch, Kathleen Peterson, Kozie V. Phibbs, Hilip Resnik, Rita Scichilone, and Gail Smith. 2011. Cac 2010-11 Industry Outlook and Resources Report. American Health Information Management Association."},{"issue":"6","key":"e_1_3_3_7_1","first-page":"10","article-title":"Computer-assisted coding reality check","volume":"90","author":"Butler Mary","year":"2019","unstructured":"Mary Butler. 2019. Computer-assisted coding reality check. Journal of merican Health Information Management Association 90, 6 (2019), 10\u201313. Retrieved from http:\/\/bok.ahima.org\/doc?oid=302747#.XsbQ8cB7mT8. Accessed January 18, 2022.","journal-title":"Journal of merican Health Information Management Association"},{"key":"e_1_3_3_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_3_3_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1833358319851305"},{"key":"e_1_3_3_10_1","doi-asserted-by":"publisher","DOI":"10.1089\/wound.2013.0478"},{"key":"e_1_3_3_11_1","doi-asserted-by":"publisher","DOI":"10.3389\/fimmu.2021.694222"},{"key":"e_1_3_3_12_1","doi-asserted-by":"crossref","first-page":"014","DOI":"10.1055\/s-0039-1677892","article-title":"The price of artificial intelligence","volume":"28","author":"Coiera Enrico","year":"2019","unstructured":"Enrico Coiera. 2019. The price of artificial intelligence. Yearbook of Medical Informatics 28, 1 (2019), 014\u2013015.","journal-title":"Yearbook of Medical Informatics"},{"key":"e_1_3_3_13_1","volume-title":"Thinking for a Living: How to Get Better Performances And Results from Knowledge Workers","author":"Davenport Thomas H.","year":"2005","unstructured":"Thomas H. Davenport. 2005. Thinking for a Living: How to Get Better Performances And Results from Knowledge Workers. Cambridge, MA: Harvard Business Review Press."},{"key":"e_1_3_3_14_1","unstructured":"Dominik Dellermann Adrian Calma Nikolaus Lipusch Thorsten Weber Sascha Weigel and Philipp Ebel. 2021. The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems. arXiv: 2105.03354. Retrieved January 18 2021 from https:\/\/arxiv.org\/abs\/2105.03354\/"},{"key":"e_1_3_3_15_1","unstructured":"Erik Drysdale Elham Dolatabadi Corey Chivers Vincent Liu Such Saria Mark Sendak Jenna Wiens Michael Brudno Amelia Hoyt Mjaye Mazwi Muhammad Mamdani Devin Singh Vanessa Allen Carolyn McGregor Heather Ross Antonio Szeto Amol Anand Verma Bo Wang P. Alison Paprica and Anna Goldenberg. 2019. Implementing AI in Healthcare Vector-SickKids Health AI Deployment Symposium Toronto. Retrieved January 27 2021 http:\/\/www.erikdrysdale.com\/figures\/implementing-ai-in-healthcare.pdf."},{"key":"e_1_3_3_16_1","article-title":"Repairing innovation: A study of integrating AI in clinical care","author":"Elish Madeleine C.","year":"2020","unstructured":"Madeleine C. Elish and Elizabeth A. Watkins. 2020. Repairing innovation: A study of integrating AI in clinical care. Data and Society. Retrieved January 27, 2021 from https:\/\/datasociety.net\/wp-content\/uploads\/2020\/09\/Repairing-Innovation-DataSociety-20200930-1.pdf.","journal-title":"Data and Society"},{"key":"e_1_3_3_17_1","unstructured":"Journal of Medical Internet Research"},{"key":"e_1_3_3_18_1","first-page":"627","volume-title":"Computer Supported Cooperative Work","author":"Erickson Ingrid","year":"2019","unstructured":"Ingrid Erickson, Deepti Menezes, Raghav Raheja, and Thanushree Shetty. 2019. Flexible turtles and elastic octopi: Exploring agile practice in knowledge work. Computer Supported Cooperative Work 28, 3 (2019), 627\u2013653."},{"key":"e_1_3_3_19_1","doi-asserted-by":"crossref","DOI":"10.4324\/9781315146027","volume-title":"Ethnography Principles in Practice Tavistock","author":"Hammersley Martyn","year":"2019","unstructured":"Martyn Hammersley and Paul Atkinson. 2019. Ethnography Principles in Practice Tavistock. London, England"},{"key":"e_1_3_3_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0307-0"},{"key":"e_1_3_3_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bushor.2018.03.007"},{"key":"e_1_3_3_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/3306127.3331663"},{"key":"e_1_3_3_23_1","volume-title":"International Journal of Management Reviews","unstructured":"E. K. Kelloway and J. Barling. 2000. Knowledge work as organizational behavior. International Journal of Management Reviews 2, 3 (2000), 287--304."},{"issue":"5","key":"e_1_3_3_24_1","article-title":"Application of machine and deep learning algorithms in intelligent clinical decision support systems in healthcare","volume":"9","author":"Kim Jong Taek","year":"2018","unstructured":"Jong Taek Kim. 2018. Application of machine and deep learning algorithms in intelligent clinical decision support systems in healthcare. Journal of Health and Medical Informatics 9, 5 (2018), 6.","journal-title":"Journal of Health and Medical Informatics"},{"key":"e_1_3_3_25_1","doi-asserted-by":"publisher","DOI":"10.1080\/13675567.2017.1384451"},{"key":"e_1_3_3_26_1","unstructured":"Journal of AHIMA"},{"key":"e_1_3_3_27_1","unstructured":"Sarah Lebovitz. 2019. Diagnostic doubt and artificial intelligence: An inductive field study of radiology work. In Proceedings of the (ICIS'19) Vol. 11 18 pages. https:\/\/aisel.aisnet.org\/icis2019\/future_of_work\/future_work\/11."},{"key":"e_1_3_3_28_1","article-title":"AI in context: The labor of integrating new technologies","author":"Mateescu Alexandra","year":"2019","unstructured":"Alexandra Mateescu and Madeleine Elish. 2019. AI in context: The labor of integrating new technologies. Data and Society Research Institute. 54 pages. Retrieved January 30, 2021 from https:\/\/datasociety.net\/library\/ai-in-context\/.","journal-title":"Data and Society Research Institute."},{"key":"e_1_3_3_29_1","first-page":"143","volume-title":"Proceedings of the Mind Design.","author":"McDermott Drew","year":"1981","unstructured":"Drew McDermott. 1981. Artificial intelligence meets natural stupidity. In Proceedings of the Mind Design. J. Haugland (Ed.), MIT Press, Cambridge, 143\u2013160."},{"key":"e_1_3_3_30_1","first-page":"506","volume-title":"Proceedings of the Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing.","author":"Park Sun Young","year":"2019","unstructured":"Sun Young Park, Pei-Yi Kuo, Andrea Barbarin, Elizabeth Kaziunas, Astrid Chow, Karandeep Singh, Lauren Wilcox, and Walter S. Lasecki. 2019. Identifying challenges and opportunities in human-AI collaboration in healthcare. In Proceedings of the Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing. 506\u2013510."},{"key":"e_1_3_3_31_1","volume-title":"Healthcare Financial Management","unstructured":"T. Payne. 2010. Improving clinical documentation in an EMR world: one health system used a three-step process to integrate a clinical documentation improvement program with its new electronic medical record system, with significant results. Healthcare Financial Management 64, 2 (2010), 70--75."},{"key":"e_1_3_3_32_1","doi-asserted-by":"publisher","DOI":"10.1177\/0950017018817489"},{"key":"e_1_3_3_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3464385.3464738"},{"key":"e_1_3_3_34_1","doi-asserted-by":"publisher","DOI":"10.1177\/18333583211067"},{"issue":"2","key":"e_1_3_3_35_1","doi-asserted-by":"crossref","first-page":"205395172096561","DOI":"10.1177\/2053951720965616","article-title":"Good organizational reasons for better medical records: The data work of clinical documentation integrity specialists","volume":"7","author":"Pine Kathleen H.","year":"2020","unstructured":"Kathleen H. Pine and Claus Bossen. 2020. Good organizational reasons for better medical records: The data work of clinical documentation integrity specialists. Big Data and Society 7, 2 (2020), 2053951720965616.","journal-title":"Big Data and Society"},{"key":"e_1_3_3_36_1","volume-title":"Redefining Health Care: Creating Value-Based Competition on Results","author":"Porter Michael E.","year":"2006","unstructured":"Michael E. Porter and Elizabeth Olmsted Teisberg. 2006. Redefining Health Care: Creating Value-Based Competition on Results. Harvard Business Press."},{"key":"e_1_3_3_37_1","volume-title":"Journal of Enterprise Transformation","unstructured":"W. B. Rouse and J. C. Spohrer. 2018. Automating versus augmenting intelligence. Journal of Enterprise Transformation 8, 1--2 (2018), 1--21."},{"key":"e_1_3_3_38_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10050561"},{"key":"e_1_3_3_39_1","first-page":"272","volume-title":"Proceedings of the Healthcare Management Forum","volume":"32","author":"Racine Eric","year":"2019","unstructured":"Eric Racine, Wren Boehlen, and Matthew Sample. 2019. Healthcare uses of artificial intelligence: Challenges and opportunities for growth. In Proceedings of the Healthcare Management Forum. Sage CA: Los Angeles, CA: SAGE Publications, 32, 5 (2019), 272\u2013275."},{"key":"e_1_3_3_40_1","volume-title":"Proceedings of the International Symposium on Distributed Computing and Artificial Intelligence","author":"Ramalho Andr\u00e9","year":"2020","unstructured":"Andr\u00e9 Ramalho, J\u00falio Souza, and Alberto. Freitas. 2020. The use of artificial intelligence for clinical coding automation: A bibliometric analysis. In Proceedings of the International Symposium on Distributed Computing and Artificial Intelligence."},{"key":"e_1_3_3_41_1","doi-asserted-by":"publisher","DOI":"10.1177\/0141076818815510"},{"key":"e_1_3_3_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372827"},{"key":"e_1_3_3_43_1","first-page":"14","article-title":"Wishful mnemonics and autonomous killing machines","volume":"136","author":"Sharkey Noel","year":"2013","unstructured":"Noel Sharkey and Lucy Suchman, 2013. Wishful mnemonics and autonomous killing machines. In Proceedings of the AISB 136 (2013), 14\u201322.","journal-title":"Proceedings of the AISB"},{"key":"e_1_3_3_44_1","doi-asserted-by":"publisher","DOI":"10.1177\/0730888417726119"},{"issue":"4","key":"e_1_3_3_45_1","first-page":"1","article-title":"Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy Human-Centered AI systems","volume":"10","year":"2021","unstructured":"Ben Shneiderman. 2021. Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy Human-Centered AI systems. ACM Transactions on Interactive Intelligent Systems 10, 4 (2021), 1\u201331.","journal-title":"ACM Transactions on Interactive Intelligent Systems"},{"key":"e_1_3_3_46_1","volume-title":"Human-Machine Reconfigurations: Plans and situated actions","author":"Suchman Lucy","year":"2007","unstructured":"Lucy Suchman. 2007. Human-Machine Reconfigurations: Plans and situated actions. Expanded edition, Cambridge University Press."},{"key":"e_1_3_3_47_1","doi-asserted-by":"publisher","DOI":"10.2308\/jeta-52311"},{"key":"e_1_3_3_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/503376.503448"},{"key":"e_1_3_3_49_1","volume-title":"Artificial Intelligence as Augmenting Automation: Implications for Employment","author":"Tschang Feichin Ted","year":"2020","unstructured":"Feichin Ted Tschang and Esteve Almirall Mezquita. 2020. Artificial Intelligence as Augmenting Automation: Implications for Employment. Academy of Management Perspectives, jan."},{"key":"e_1_3_3_50_1","volume-title":"Qualitative Research Methods: Collecting Evidence, Crafting analysis, Communicating Impact","author":"Tracy Sarah J.","year":"2019","unstructured":"Sarah J. Tracy. 2019. Qualitative Research Methods: Collecting Evidence, Crafting analysis, Communicating Impact. John Wiley and Sons."},{"key":"e_1_3_3_51_1","doi-asserted-by":"crossref","DOI":"10.4324\/9781351109598","volume-title":"Artificial intelligence in organizations: New opportunities for phenomenon-based theorizing","author":"Von Krogh Georg","year":"2018","unstructured":"Georg Von Krogh. 2018. Artificial intelligence in organizations: New opportunities for phenomenon-based theorizing. Academy of Management Discoveries."},{"key":"e_1_3_3_52_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3359313","article-title":"Human-ai collaboration in data science: Exploring data scientists' perceptions of automated ai","volume":"3","author":"Wang Dakuo","year":"2019","unstructured":"Dakuo Wang, Justin D. Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-ai collaboration in data science: Exploring data scientists' perceptions of automated ai. Proceedings of the ACM on Human-Computer Interaction 3 (2019), 1\u201324.","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"e_1_3_3_53_1","first-page":"1","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","year":"2020","unstructured":"Dakuo Wang, Liuping Wang, Zhan Zhang, Ding Wang, Haiyi Zhu, Yvonne Gao, Xiangmin Fan, and Feng Tian. 2020. \u201cBrilliant AI Doctor\u201d in Rural Clinics: Challenges in AI-Powered clinical decision support system deployment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1\u201318."},{"key":"e_1_3_3_54_1","volume-title":"Sensemaking in Organizations","author":"Weick Karl E.","year":"1995","unstructured":"Karl E. Weick. 1995. Sensemaking in Organizations. Sage, Vol. 3, 1995."},{"key":"e_1_3_3_55_1","first-page":"130","volume-title":"Proceedings of the 2020 13th International Conference on Human System Interaction","author":"Wickramasinghe Chathurika S.","year":"2020","unstructured":"Chathurika S. Wickramasinghe, Daniel L. Marino, Javier Grandio, and Milos Manic. 2020. Trustworthy AI development guidelines for human system interaction. In Proceedings of the 2020 13th International Conference on Human System Interaction. IEEE, 130\u2013136."},{"key":"e_1_3_3_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bushor.2018.07.001"},{"key":"e_1_3_3_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300468"},{"key":"e_1_3_3_58_1","first-page":"1","volume-title":"Proceedings of the CHI Conference on Human Factors in Computing Systems.","author":"Zhang Qiaoning","year":"2022","unstructured":"Qiaoning Zhang, Matthew L. Lee, and Scott Carter. 2022. You complete Me: Human-AI teams and complementary expertise. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1\u201328."}],"container-title":["ACM Transactions on Computer-Human Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569892","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3569892","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:58Z","timestamp":1750182538000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569892"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,17]]},"references-count":57,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4,30]]}},"alternative-id":["10.1145\/3569892"],"URL":"https:\/\/doi.org\/10.1145\/3569892","relation":{},"ISSN":["1073-0516","1557-7325"],"issn-type":[{"value":"1073-0516","type":"print"},{"value":"1557-7325","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,17]]},"assertion":[{"value":"2021-10-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-19","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}