{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T13:29:18Z","timestamp":1791379758894,"version":"4.3.3"},"reference-count":52,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Hum Factors"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Adverse events refer to incidents with potential or actual harm to patients in hospitals. These events are typically documented through patient safety event (PSE) reports, which consist of detailed narratives providing contextual information on the occurrences. Accurate classification of PSE reports is crucial for patient safety monitoring. However, this process faces challenges due to inconsistencies in classifications and the sheer volume of reports. Recent advancements in text representation, particularly contextual text representation derived from transformer-based language models, offer a promising solution for more precise PSE report classification. Integrating the machine learning (ML) classifier necessitates a balance between human expertise and artificial intelligence (AI). Central to this integration is the concept of explainability, which is crucial for building trust and ensuring effective human-AI collaboration.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to investigate the efficacy of ML classifiers trained using contextual text representation in automatically classifying PSE reports. Furthermore, the study presents an interface that integrates the ML classifier with the explainability technique to facilitate human-AI collaboration for PSE report classification.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This study used a data set of 861 PSE reports from a large academic hospital\u2019s maternity units in the Southeastern United States. Various ML classifiers were trained with both static and contextual text representations of PSE reports. The trained ML classifiers were evaluated with multiclass classification metrics and the confusion matrix. The local interpretable model-agnostic explanations (LIME) technique was used to provide the rationale for the ML classifier\u2019s predictions. An interface that integrates the ML classifier with the LIME technique was designed for incident reporting systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The top-performing classifier using contextual representation was able to obtain an accuracy of 75.4% (95\/126) compared to an accuracy of 66.7% (84\/126) by the top-performing classifier trained using static text representation. A PSE reporting interface has been designed to facilitate human-AI collaboration in PSE report classification. In this design, the ML classifier recommends the top 2 most probable event types, along with the explanations for the prediction, enabling PSE reporters and patient safety analysts to choose the most suitable one. The LIME technique showed that the classifier occasionally relies on arbitrary words for classification, emphasizing the necessity of human oversight.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>This study demonstrates that training ML classifiers with contextual text representations can significantly enhance the accuracy of PSE report classification. The interface designed in this study lays the foundation for human-AI collaboration in the classification of PSE reports. The insights gained from this research enhance the decision-making process in PSE report classification, enabling hospitals to more efficiently identify potential risks and hazards and enabling patient safety analysts to take timely actions to prevent patient harm.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/53378","type":"journal-article","created":{"date-parts":[[2024,1,25]],"date-time":"2024-01-25T09:46:16Z","timestamp":1706175976000},"page":"e53378","source":"Crossref","is-referenced-by-count":29,"title":["A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports: Algorithm Development and Validation Study"],"prefix":"10.2196","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-5823-9406","authenticated-orcid":false,"given":"Hongbo","family":"Chen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5767-6683","authenticated-orcid":false,"given":"Eldan","family":"Cohen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4726-7848","authenticated-orcid":false,"given":"Dulaney","family":"Wilson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0045-0426","authenticated-orcid":false,"given":"Myrtede","family":"Alfred","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2024,1,25]]},"reference":[{"key":"ref1","author":"Kohn, LT","year":"2000","journal-title":"To Err Is Human: Building a Safer Health System.  Volume 627"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-140-10-200405180-00009"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.2196\/28632"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.i2139"},{"key":"ref5","author":"Slawomirski, L","year":"2017","journal-title":"The Economics of Patient Safety: Strengthening a Value-based Approach to Reducing Patient Harm at National Level"},{"key":"ref6","unstructured":"Medication without harmWorld Health Organization20232023-03-08https:\/\/www.who.int\/initiatives\/medication-without-harm"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2014-003279"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2011-000213"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.2196\/38144"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.39071.441609.80"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1136\/qshc.2009.036376"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1136\/qshc.2009.036657"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1016\/s1553-7250(12)38044-6"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.09.011"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.12927\/hcq.2011.22491"},{"key":"ref16","unstructured":"Conceptual framework for the international classification for patient safety version 1.1: final technical report January 2009World Health Organization20102023-03-08https:\/\/apps.who.int\/iris\/handle\/10665\/70882"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1097\/PTS.0000000000000731"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1136\/bmjoq-2022-001945"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.5811\/westjem.2020.7.48217"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1186\/2056-5917-1-7"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-009-9398-y"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-022-07822-9"},{"key":"ref23","unstructured":"Biannual incident reportClinical Excellence Commission2023-05-16https:\/\/www.cec.health.nsw.gov.au\/Review-incidents\/Biannual-Incident-Report"},{"key":"ref24","unstructured":"Organisation patient safety incident report up to March 2022NHS England20222023-03-08https:\/\/www.england.nhs.uk\/publication\/organisation-patient-safety-incident-report-up-to-march-2022\/"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1177\/1460458219833102"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-017-0483-8"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1145\/3439726"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2003.07278"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.2196\/15154"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.2196\/40238"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231186520"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.09082"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-020-01332-6"},{"key":"ref34","unstructured":"DevlinJChangMWLeeKToutanovaKBERT: pre-training of deep bidirectional transformers for language understanding20192019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language TechnologiesJune 2-7, 2019Minneapolis, MinnesotaMinneapolis, MinnesotaAssociation for Computational Linguistics41714186"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/d14-1162"},{"key":"ref36","unstructured":"ModelsHugging Face20222023-03-09https:\/\/huggingface.co\/models"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.5260\/chara.21.2.8"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1088\/1757-899x\/1099\/1\/012077"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.2196\/38161"},{"key":"ref41","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.2196\/30470"},{"key":"ref43","unstructured":"The collaborative interface design toolFigma2023-09-24https:\/\/www.figma.com\/"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1177\/1062860619891995"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.3109\/13814788.2015.1043728"},{"key":"ref46","unstructured":"HasegawaTFujitaSPatient safety policies\u2012experiences, effects and priorities; lessons from OECD member states \u2012 (version 2.0)Patient Safety Global Ministerial Summit 201820182023-12-21https:\/\/www.mhlw.go.jp\/psgms2018\/pdf\/document\/5_Document.pdf"},{"key":"ref47","first-page":"609","volume":"245","author":"Wang, Y","year":"2017","journal-title":"Stud Health Technol Inform"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40942-4_4"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2023.1250725"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.24251\/hicss.2021.046"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-18751-2"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445472"}],"container-title":["JMIR Human Factors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/humanfactors.jmir.org\/2024\/1\/e53378","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:39:03Z","timestamp":1782841143000},"score":1,"resource":{"primary":{"URL":"https:\/\/humanfactors.jmir.org\/2024\/1\/e53378"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,25]]},"references-count":52,"URL":"https:\/\/doi.org\/10.2196\/53378","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.53378","asserted-by":"object"}]},"ISSN":["2292-9495"],"issn-type":[{"value":"2292-9495","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,25]]}}}