{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T03:07:56Z","timestamp":1768792076696,"version":"3.49.0"},"reference-count":88,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"DOI":"10.13039\/100000001","name":"US National Science Foundation","doi-asserted-by":"crossref","award":["IIS-2202451, DRL-2506865"],"award-info":[{"award-number":["IIS-2202451, DRL-2506865"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2026,1,31]]},"abstract":"<jats:p>This study introduces AutoCLC, an AI-powered system designed to assess and provide feedback on Closed-Loop Communication (CLC) in professional learning environments. CLC, where a sender\u2019s Call-Out statement is acknowledged by the receiver\u2019s Check-Back statement, is a critical safety protocol in high-reliability domains, including emergency medicine resuscitation teams. Existing methods for evaluating CLC lack quantifiable metrics and depend heavily on human observation. AutoCLC addresses these limitations by leveraging natural language processing and large language models to analyze audio recordings from Advanced Cardiovascular Life Support (ACLS) simulation training. The system identifies CLC instances, measures their frequency and rate per minute, and categorizes communications as effective, incomplete, or missed. Technical evaluations demonstrate that AutoCLC achieves 78.9% precision for identifying Call-Outs and 74.3% for Check-Backs, with a performance gap of only 5% compared to human annotations. A user study involving 11 cardiac arrest instructors across three training sites supported the need for automated CLC assessment. Instructors found AutoCLC reports valuable for quantifying CLC frequency and quality, as well as for providing actionable, example-based feedback. Participants rated AutoCLC highly, with a System Usability Scale score of 76.4%, reflecting above-average usability. This work represents a significant step toward developing scalable, data-driven feedback systems that enhance individual skills and team performance in high-reliability settings.<\/jats:p>","DOI":"10.1145\/3768164","type":"journal-article","created":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T13:21:05Z","timestamp":1758633665000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["AutoCLC: Towards Automated Assessment and Feedback on Closed-Loop Communication in Team-based Healthcare Simulation Training"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0729-8471","authenticated-orcid":false,"given":"Kaylee Yaxuan","family":"Li","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3404-2634","authenticated-orcid":false,"given":"Kunpeng","family":"Huang","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0094-0337","authenticated-orcid":false,"given":"Bryan","family":"Harmer","sequence":"additional","affiliation":[{"name":"Medical School, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-3020-8112","authenticated-orcid":false,"given":"Casey","family":"Ducharme","sequence":"additional","affiliation":[{"name":"Medical School, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0738-653X","authenticated-orcid":false,"given":"Benjamin","family":"Heasman","sequence":"additional","affiliation":[{"name":"Medical School, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5070-6025","authenticated-orcid":false,"given":"Eleanor","family":"Falahee","sequence":"additional","affiliation":[{"name":"Medical School, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4761-7185","authenticated-orcid":false,"given":"James","family":"Cooke","sequence":"additional","affiliation":[{"name":"Department of Learning Health Sciences, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4146-3632","authenticated-orcid":false,"given":"Michael","family":"Cole","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8046-0538","authenticated-orcid":false,"given":"Alanson","family":"Sample","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2348-5285","authenticated-orcid":false,"given":"Vitaliy","family":"Popov","sequence":"additional","affiliation":[{"name":"Medical School, University of Michigan-Ann Arbor, Ann Arbor, Michigan, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,1,14]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"20","volume-title":"The ACLS Instructor Manual","year":"2023","unstructured":"American Heart Association. 2023. The ACLS Instructor Manual. American Heart Association. This product has been updated with new science from the 2020 Guidelines for CPR and ECC and replaces the previous version (15-1003). A digital version of this product is also available: 20\u20133101."},{"key":"e_1_3_1_3_2","unstructured":"OpenAI. 2023. GPT-4 is OpenAI\u2019s most advanced system producing safer and more useful responses. Retrieved May 24 2024 from https:\/\/openai.com\/index\/gpt-4\/"},{"key":"e_1_3_1_4_2","unstructured":"SciPy. 2024. scipy.stats.mannwhitneyu. Retrieved May 24 2024 from https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.stats.mannwhitneyu.html"},{"key":"e_1_3_1_5_2","unstructured":"Maria Celine Aas. 2024. The effectiveness of closed-loop communication training for healthcare providers on use medical errors and task performance: A systematic review. Retrieved from https:\/\/munin.uit.no\/handle\/10037\/33922"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.2196\/48291"},{"issue":"1","key":"e_1_3_1_7_2","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.jsurg.2017.06.025","article-title":"Closed-loop communication improves task completion in pediatric trauma resuscitation","volume":"75","author":"El-Shafy Ibrahim Abd","year":"2018","unstructured":"Ibrahim Abd El-Shafy, Jennifer Delgado, Meredith Akerman, Francesca Bullaro, Nathan A. M. Christopherson, and Jose M. Prince. 2018. Closed-loop communication improves task completion in pediatric trauma resuscitation. Journal of Surgical Education 75, 1 (2018), 58\u201364.","journal-title":"Journal of Surgical Education"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.18608\/jla.2018.52.5"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.emc.2023.03.011"},{"issue":"4","key":"e_1_3_1_11_2","doi-asserted-by":"crossref","first-page":"1576","DOI":"10.1111\/j.1475-6773.2006.00566.x","article-title":"Teamwork as an essential component of high-reliability organizations","volume":"41","author":"Baker David P.","year":"2006","unstructured":"David P. Baker, Rachel Day, and Eduardo Salas. 2006. Teamwork as an essential component of high-reliability organizations. Health Services Research 41, 4p2 (2006), 1576\u20131598.","journal-title":"Health Services Research"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.edurev.2022.100489"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s43678-022-00295-z"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","DOI":"10.1093\/acrefore\/9780190228637.013.1570","article-title":"Maintaining resilience in times of crisis: Insights from high-reliability organizations","author":"Bisbey Tiffany M.","year":"2021","unstructured":"Tiffany M. Bisbey, Molly P. Kilcullen, and Eduardo Salas. 2021. Maintaining resilience in times of crisis: Insights from high-reliability organizations. Oxford Research Encyclopedia of Politics (2021).","journal-title":"Oxford Research Encyclopedia of Politics"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1518\/001872098779649265"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1097\/SIH.0000000000000347"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"issue":"1","key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"i96","DOI":"10.1136\/qshc.2004.009829","article-title":"How to turn a team of experts into an expert medical team: Guidance from the aviation and military communities","volume":"13","author":"Burke C. S.","year":"2004","unstructured":"C. S. Burke, Eduardo Salas, K. Wilson-Donnelly, and H. Priest. 2004. How to turn a team of experts into an expert medical team: Guidance from the aviation and military communities. Quality and Safety in Health Care 13, Suppl 1 (2004), i96\u2013i104.","journal-title":"Quality and Safety in Health Care"},{"key":"e_1_3_1_19_2","article-title":"Implementation, evaluation, and outcome of TeamSTEPPS in interprofessional education: A scoping review","author":"Chen Alissa S.","year":"2019","unstructured":"Alissa S. Chen, Bernice Yau, Lee Revere, and Jennifer Swails. 2019. Implementation, evaluation, and outcome of TeamSTEPPS in interprofessional education: A scoping review. Journal of Interprofessional Care (2019).","journal-title":"Journal of Interprofessional Care"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3610196"},{"issue":"16","key":"e_1_3_1_21_2","first-page":"S551","article-title":"Part 6: Resuscitation education science: 2020 American heart association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care","volume":"142","author":"Cheng Adam","year":"2020","unstructured":"Adam Cheng, David J. Magid, Marc Auerbach, Farhan Bhanji, Blair L. Bigham, Audrey L. Blewer, Katie N. Dainty, Emily Diederich, Yiqun Lin, Marion Leary, et al. 2020. Part 6: Resuscitation education science: 2020 American heart association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 142, 16 Suppl 2 (2020), S551\u2013S579.","journal-title":"Circulation"},{"issue":"1","key":"e_1_3_1_22_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/S0300-9572(99)00080-5","article-title":"Leadership of resuscitation teams: \u2018Lighthouse Leadership\u2019","volume":"42","author":"Cooper Simon","year":"1999","unstructured":"Simon Cooper and Alan Wakelam. 1999. Leadership of resuscitation teams: \u2018Lighthouse Leadership\u2019. Resuscitation 42, 1 (1999), 27\u201345.","journal-title":"Resuscitation"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10639-022-11000-z"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-6393(01)00088-3"},{"key":"e_1_3_1_25_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171\u20134186."},{"issue":"6","key":"e_1_3_1_26_2","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1177\/1062860620912480","article-title":"Impact of simulation-based closed-loop communication training on medical errors in a pediatric emergency department","volume":"35","author":"Diaz Maria Carmen G.","year":"2020","unstructured":"Maria Carmen G. Diaz and Kimberly Dawson. 2020. Impact of simulation-based closed-loop communication training on medical errors in a pediatric emergency department. American Journal of Medical Quality 35, 6 (2020), 474\u2013478.","journal-title":"American Journal of Medical Quality"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.teln.2023.06.019"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1002\/ccd.28298"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11301-021-00235-8"},{"issue":"1","key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/MPRV.2022.3217454","article-title":"HuCETA: A framework for human-centered embodied teamwork analytics","volume":"22","author":"Echeverria Vanessa","year":"2022","unstructured":"Vanessa Echeverria, Roberto Martinez-Maldonado, Lixiang Yan, Linxuan Zhao, Gloria Fernandez-Nieto, Dragan Ga\u0161evi\u0107, and Simon Buckingham Shum. 2022. HuCETA: A framework for human-centered embodied teamwork analytics. IEEE Pervasive Computing 22, 1 (2022), 39\u201349.","journal-title":"IEEE Pervasive Computing"},{"key":"e_1_3_1_31_2","unstructured":"Debora Field Simon Worgan Nick Webb M. Hepple and Yorick Wilks. 2008. Automatic Induction of Dialogue Structure from the Companions Dialogue Corpus. Retrieved from https:\/\/www.researchgate.net\/publication\/262485912_Automatic_Induction_of_Dialogue_Structure_from_the_Companions_Dialogue_Corpus"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1080\/00140139.2015.1015621"},{"issue":"3","key":"e_1_3_1_33_2","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1177\/0310057X1504300303","article-title":"Debriefing after simulation-based non-technical skill training in healthcare: A systematic review of effective practice","volume":"43","author":"Garden A. L.","year":"2015","unstructured":"A. L. Garden, D. M. Le Fevre, H. L. Waddington, and J. M. Weller. 2015. Debriefing after simulation-based non-technical skill training in healthcare: A systematic review of effective practice. Anaesthesia and Intensive Care 43, 3 (2015), 300\u2013308.","journal-title":"Anaesthesia and Intensive Care"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1177\/0018720819858274"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1080\/13561820.2016.1249280"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1111\/aas.12336"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000089"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1097\/PTS.0000000000000541"},{"issue":"10","key":"e_1_3_1_39_2","doi-asserted-by":"crossref","first-page":"e003525","DOI":"10.1136\/bmjopen-2013-003525","article-title":"Communication in interdisciplinary teams: Exploring closed-loop communication during in situ trauma team training","volume":"3","author":"H\u00e4rgestam Maria","year":"2013","unstructured":"Maria H\u00e4rgestam, Marie Lindkvist, Christine Brulin, Maritha Jacobsson, and Magnus Hultin. 2013. Communication in interdisciplinary teams: Exploring closed-loop communication during in situ trauma team training. BMJ Open 3, 10 (2013), e003525.","journal-title":"BMJ Open"},{"key":"e_1_3_1_40_2","doi-asserted-by":"crossref","first-page":"5049","DOI":"10.1109\/ICASSP.2008.4518793","volume-title":"2008 IEEE International Conference on Acoustics, Speech and Signal Processing","author":"Hsueh Pei-Yun","year":"2008","unstructured":"Pei-Yun Hsueh. 2008. Audio-based unsupervised segmentation of multiparty dialogue. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 5049\u20135052."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmjstel-2019-000498"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacc.2011.03.017"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2013-003525"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3272973.3274071"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1037\/apl0000821"},{"issue":"5","key":"e_1_3_1_46_2","doi-asserted-by":"crossref","first-page":"953","DOI":"10.1007\/s10869-021-09784-x","article-title":"A meta-analysis of task and training characteristics that contribute to or attenuate the effectiveness of the after-action review (or debrief)","volume":"37","author":"Keiser Nathanael L.","year":"2022","unstructured":"Nathanael L. Keiser and Winfred Arthur Jr. 2022. A meta-analysis of task and training characteristics that contribute to or attenuate the effectiveness of the after-action review (or debrief). Journal of Business and Psychology 37, 5 (2022), 953\u2013976.","journal-title":"Journal of Business and Psychology"},{"key":"e_1_3_1_47_2","article-title":"Non-linear, team-based VR training for cardiac arrest care with enhanced CRM toolkit","author":"Kentros Mike","year":"2025","unstructured":"Mike Kentros, Manos Kamarianakis, Michael Cole, Vitaliy Popov, Antonis Protopsaltis, and George Papagiannakis. 2025. Non-linear, team-based VR training for cardiac arrest care with enhanced CRM toolkit. Eurographics Technical Report Series (2025).","journal-title":"Eurographics Technical Report Series"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.2312\/egs.20251054"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeai.2022.100074"},{"key":"e_1_3_1_50_2","unstructured":"Heidi B. King James Battles David P. Baker Alexander Alonso Eduardo Salas John Webster Lauren Toomey and Mary Salisbury. 2008. TeamSTEPPS\u2122: Team Strategies and Tools to Enhance Performance and Patient Safety. Retrieved from https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK43686\/"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.17226\/9728"},{"issue":"3","key":"e_1_3_1_52_2","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1136\/bmjqs-2021-014393","article-title":"Helping healthcare teams to debrief effectively: Associations of debriefers\u2019 actions and participants\u2019 reflections during team debriefings","volume":"32","author":"Kolbe Michaela","year":"2023","unstructured":"Michaela Kolbe, Bastian Grande, Nale Lehmann-Willenbrock, and Julia Carolin Seelandt. 2023. Helping healthcare teams to debrief effectively: Associations of debriefers\u2019 actions and participants\u2019 reflections during team debriefings. BMJ Quality and Safety 32, 3 (2023), 160\u2013172.","journal-title":"BMJ Quality and Safety"},{"key":"e_1_3_1_53_2","unstructured":"Jim Lara. 2023. Achieve Perfection with Simplified Closed-Loop Communication Principles. Retrieved from https:\/\/www.gray stoneadvisors.com\/blog\/closed-loop-communication\/"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231184991"},{"issue":"2","key":"e_1_3_1_55_2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1111\/nhs.12662","article-title":"Comparing the learning effects of debriefing modalities for the care of premature infants","volume":"22","author":"Lee Myung-Nam","year":"2020","unstructured":"Myung-Nam Lee, Shin-Jeong Kim, Kyung-Ah Kang, and Sunghee Kim. 2020. Comparing the learning effects of debriefing modalities for the care of premature infants. Nursing and Health Sciences 22, 2 (2020), 243\u2013253.","journal-title":"Nursing and Health Sciences"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2018.1455307"},{"issue":"1","key":"e_1_3_1_57_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3622784","article-title":"Lessons learnt from a multimodal learning analytics deployment in-the-wild","volume":"31","author":"Martinez-Maldonado Roberto","year":"2023","unstructured":"Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez-Nieto, Lixiang Yan, Linxuan Zhao, Riordan Alfredo, Xinyu Li, Samantha Dix, Hollie Jaggard, Rosie Wotherspoon, et al. 2023. Lessons learnt from a multimodal learning analytics deployment in-the-wild. ACM Transactions on Computer-Human Interaction 31, 1 (2023), 1\u201341.","journal-title":"ACM Transactions on Computer-Human Interaction"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbah.2024.100053"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.3115\/1075178.1075186"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2021.05.043"},{"issue":"1","key":"e_1_3_1_61_2","first-page":"361","article-title":"Using the closed loop communication system to prevent accidents caused by human errors","volume":"21","author":"Mozhey Dmitry","year":"2024","unstructured":"Dmitry Mozhey, Jos\u00e9 Agust\u00edn Gonz\u00e1lez Almeida, Mar\u00eda del Cristo, Adri\u00e1n de Ganzo, Alejandro U. G\u00f3mez Correa, and Javier Almenara de Luz. 2024. Using the closed loop communication system to prevent accidents caused by human errors. Journal of Maritime Research 21, 1 (2024), 361\u2013369.","journal-title":"Journal of Maritime Research"},{"key":"e_1_3_1_62_2","doi-asserted-by":"crossref","unstructured":"Julius Murumba. 2022. Learning Analytics and Educational Data Visualization in the Digital Era. Journal of Innovations in Data Science and Big Data Management 1 3 (2022). Retrieved from https:\/\/www.researchgate.net\/publication\/359093879_Learning_Analytics_and_Educational_Data_Visualization_in_the_Digital_Era","DOI":"10.46610\/JIDSBDM.2022.v01i01.001"},{"issue":"2","key":"e_1_3_1_63_2","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1161\/CIRCULATIONAHA.118.033674","article-title":"How do resuscitation teams at top-performing hospitals for in-hospital cardiac arrest succeed? A qualitative study","volume":"138","author":"Nallamothu Brahmajee K.","year":"2018","unstructured":"Brahmajee K. Nallamothu, Timothy C. Guetterman, Molly Harrod, Joan E. Kellenberg, Jessica L. Lehrich, Steven L. Kronick, Sarah L. Krein, Theodore J. Iwashyna, Sanjay Saint, and Paul S. Chan. 2018. How do resuscitation teams at top-performing hospitals for in-hospital cardiac arrest succeed? A qualitative study. Circulation 138, 2 (2018), 154\u2013163.","journal-title":"Circulation"},{"key":"e_1_3_1_64_2","unstructured":"Inc. Otter.ai. 2024. Otter.ai. Retrieved May 24 2024 from https:\/\/otter.ai\/"},{"issue":"1","key":"e_1_3_1_65_2","doi-asserted-by":"crossref","first-page":"2424450","DOI":"10.1080\/07853890.2024.2424450","article-title":"Metaverse-based simulation: A scoping review of charting medical education over the last two decades in the lens of the \u2018marvelous medical education machine","volume":"56","author":"Popov Vitaliy","year":"2024","unstructured":"Vitaliy Popov, Natalie Mateju, Caris Jeske, and Kadriye O. Lewis. 2024. Metaverse-based simulation: A scoping review of charting medical education over the last two decades in the lens of the \u2018marvelous medical education machine. Annals of Medicine 56, 1 (2024), 2424450.","journal-title":"Annals of Medicine"},{"key":"e_1_3_1_66_2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/s10488-010-0319-7","article-title":"Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda","volume":"38","author":"Proctor Enola","year":"2011","unstructured":"Enola Proctor, Hiie Silmere, Ramesh Raghavan, Peter Hovmand, Greg Aarons, Alicia Bunger, Richard Griffey, and Melissa Hensley. 2011. Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health 38 (2011), 65\u201376.","journal-title":"Administration and Policy in Mental Health"},{"key":"e_1_3_1_67_2","first-page":"28492","volume-title":"International Conference on Machine Learning","author":"Radford Alec","year":"2023","unstructured":"Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning. PMLR, 28492\u201328518."},{"issue":"4","key":"e_1_3_1_68_2","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1097\/ACM.0000000000001033","article-title":"Improving anesthesiologists\u2019 ability to speak up in the operating room: A randomized controlled experiment of a simulation-based intervention and a qualitative analysis of hurdles and enablers","volume":"91","author":"Raemer Daniel B.","year":"2016","unstructured":"Daniel B. Raemer, Michaela Kolbe, Rebecca D. Minehart, Jenny W. Rudolph, and May C. M. Pian-Smith. 2016. Improving anesthesiologists\u2019 ability to speak up in the operating room: A randomized controlled experiment of a simulation-based intervention and a qualitative analysis of hurdles and enablers. Academic Medicine 91, 4 (2016), 530\u2013539.","journal-title":"Academic Medicine"},{"key":"e_1_3_1_69_2","first-page":"1","article-title":"An automated content-based measure of closed loop communication among critical care air transport teams","author":"Robinson F. Eric","year":"2024","unstructured":"F. Eric Robinson, David Grimm, Dain Horning, Jamie C. Gorman, Jennifer Winner, and Christopher W. Wiese. 2024. An automated content-based measure of closed loop communication among critical care air transport teams. Military Psychology 37 (2024), 1\u20137.","journal-title":"Military Psychology"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.anclin.2007.03.007"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.2196\/30082"},{"key":"e_1_3_1_72_2","unstructured":"Irim Salik and John Ashurst. 2020. Closed Loop Communication Training in Medical Simulation. Retrieved from https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK549899\/"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.resuscitation.2014.03.180"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.5272\/jimab.2017233.1623"},{"key":"e_1_3_1_75_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-540-71062-2","volume-title":"Crisis Management in Acute Care Settings: Human Factors and Team Psychology in a High Stakes Environment","author":"St. Pierre Michael","year":"2008","unstructured":"Michael St. Pierre, Gesine Hofinger, and Cornelius Buerschaper. 2008. Crisis Management in Acute Care Settings: Human Factors and Team Psychology in a High Stakes Environment. Springer."},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1177\/0018720812448394"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-024-02063-1"},{"key":"e_1_3_1_78_2","doi-asserted-by":"publisher","DOI":"10.3233\/APC210017"},{"key":"e_1_3_1_79_2","first-page":"1","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS \u201922)","volume":"1800","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed. H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS \u201922). Curran Associates Inc., Red Hook, NY, Article 1800, 1\u201314."},{"key":"e_1_3_1_80_2","first-page":"1","article-title":"Psychometric assessment of three newly developed implementation outcome measures","volume":"12","author":"Weiner Bryan J.","year":"2017","unstructured":"Bryan J. Weiner, Cara C. Lewis, Cameo Stanick, Byron J. Powell, Caitlin N. Dorsey, Alecia S. Clary, Marcella H. Boynton, and Heather Halko. 2017. Psychometric assessment of three newly developed implementation outcome measures. Implementation Science 12 (2017), 1\u201312.","journal-title":"Implementation Science"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.288.23.3035"},{"issue":"1061","key":"e_1_3_1_82_2","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1136\/postgradmedj-2012-131168","article-title":"Teams, tribes and patient safety: Overcoming barriers to effective teamwork in healthcare","volume":"90","author":"Weller Jennifer","year":"2014","unstructured":"Jennifer Weller, Matt Boyd, and David Cumin. 2014. Teams, tribes and patient safety: Overcoming barriers to effective teamwork in healthcare. Postgraduate Medical Journal 90, 1061 (2014), 149\u2013154.","journal-title":"Postgraduate Medical Journal"},{"key":"e_1_3_1_83_2","first-page":"21","volume-title":"Proceedings of the 11th International Symposium on Human Factors and Ergonomics in Health Care","author":"Winner Jennifer","year":"2022","unstructured":"Jennifer Winner, Jayde King, Jamie Gorman, and David Grimm. 2022. Team coordination dynamics measurement in enroute care training: Defining requirements and usability study. In Proceedings of the 11th International Symposium on Human Factors and Ergonomics in Health Care. SAGE Publications, Los Angeles, CA, 21\u201325."},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0035-1565997"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(23)00225-X"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998328"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.13476"},{"key":"e_1_3_1_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576050.3576076"},{"key":"e_1_3_1_89_2","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al. 2023. A survey of large language models. arXiv:2303.18223. Retrieved from https:\/\/arxiv.org\/abs\/2303.18223"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3768164","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T12:54:50Z","timestamp":1768395290000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3768164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,14]]},"references-count":88,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,31]]}},"alternative-id":["10.1145\/3768164"],"URL":"https:\/\/doi.org\/10.1145\/3768164","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"value":"2691-1957","type":"print"},{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,14]]},"assertion":[{"value":"2025-03-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}