{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T03:31:37Z","timestamp":1782790297243,"version":"3.54.5"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2019,8,28]],"date-time":"2019-08-28T00:00:00Z","timestamp":1566950400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Australian National Health and Medical Research Council"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To evaluate the feasibility of a convolutional neural network (CNN) with word embedding to identify the type and severity of patient safety incident reports.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>A CNN with word embedding was applied to identify 10 incident types and 4 severity levels. Model training and validation used data sets (n_type\u2009=\u20092860, n_severity\u2009=\u20091160) collected from a statewide incident reporting system. Generalizability was evaluated using an independent hospital-level reporting system. CNN architectures were examined by varying layer size and hyperparameters. Performance was evaluated by F score, precision, recall, and compared to binary support vector machine (SVM) ensembles on 3 testing data sets (type\/severity: n_benchmark = 286\/116, n_original = 444\/4837, n_independent = 6000\/5950).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>A CNN with 6 layers was the most effective architecture, outperforming SVMs with better generalizability to identify incidents by type and severity. The CNN achieved high F scores (&amp;gt; 85%) across all test data sets when identifying common incident types including falls, medications, pressure injury, and aggression. When identifying common severity levels (medium\/low), CNN outperformed SVMs, improving F scores by 11.9%\u201345.1% across all 3 test data sets.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Automated identification of incident reports using machine learning is challenging because of a lack of large labelled training data sets and the unbalanced distribution of incident classes. The standard classification strategy is to build multiple binary classifiers and pool their predictions. CNNs can extract hierarchical features and assist in addressing class imbalance, which may explain their success in identifying incident report types.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>A CNN with word embedding was effective in identifying incidents by type and severity, providing better generalizability than SVMs.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocz146","type":"journal-article","created":{"date-parts":[[2019,7,25]],"date-time":"2019-07-25T11:28:00Z","timestamp":1564054080000},"page":"1600-1608","source":"Crossref","is-referenced-by-count":20,"title":["Using convolutional neural networks to identify patient safety incident reports by type and severity"],"prefix":"10.1093","volume":"26","author":[{"given":"Ying","family":"Wang","sequence":"first","affiliation":[{"name":"Centre for Health Informatics Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enrico","family":"Coiera","sequence":"additional","affiliation":[{"name":"Centre for Health Informatics Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farah","family":"Magrabi","sequence":"additional","affiliation":[{"name":"Centre for Health Informatics Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,8,28]]},"reference":[{"issue":"4","key":"2020110613094281100_ocz146-B1","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1093\/qjmed\/hcu145","article-title":"Adverse events in healthcare: learning from mistakes","volume":"108","author":"Rafter","year":"2015","journal-title":"QJM"},{"key":"2020110613094281100_ocz146-B2","volume-title":"Safety and Ethics in Health care: A Guide to Getting It Right","author":"Runciman","year":"2007"},{"key":"2020110613094281100_ocz146-B3","author":"Clinical Excellence Commission NSW","year":"2016"},{"key":"2020110613094281100_ocz146-B4","doi-asserted-by":"crossref","first-page":"i82","DOI":"10.1136\/qshc.2005.017467","article-title":"An integrated framework for safety, quality and risk management: an information and incident management system based on a universal patient safety classification","volume":"15 Suppl 1","author":"Runciman","year":"2006","journal-title":"Qual Saf Health Care"},{"key":"2020110613094281100_ocz146-B5","volume-title":"Advances in Patient Safety: New Directions and Alternative Approaches","author":"Pronovost"},{"issue":"2","key":"2020110613094281100_ocz146-B6","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1136\/bmjqs-2015-004405","article-title":"Patient safety incident reporting: a qualitative study of thoughts and perceptions of experts 15 years after \u2018To Err is Human\u2019","volume":"25","author":"Mitchell","year":"2016","journal-title":"BMJ Qual Saf"},{"issue":"1","key":"2020110613094281100_ocz146-B7","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1093\/bja\/aeq133","article-title":"Critical incident reporting and learning","volume":"105","author":"Mahajan","year":"2010","journal-title":"Br J Anaesth"},{"key":"2020110613094281100_ocz146-B8","author":"Agency for Healthcare Research and Quality"},{"issue":"4","key":"2020110613094281100_ocz146-B9","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1007\/s10916-009-9398-y","article-title":"Data consistency in a voluntary medical incident reporting system","volume":"35","author":"Gong","year":"2011","journal-title":"J Med Syst"},{"issue":"3","key":"2020110613094281100_ocz146-B10","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1111\/j.1532-5415.2008.02142.x","article-title":"Inconsistency in classification and reporting of in-hospital falls","volume":"57","author":"Haines","year":"2009","journal-title":"J Am Geriatr Soc"},{"issue":"5","key":"2020110613094281100_ocz146-B11","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1093\/intqhc\/mzp034","article-title":"Medication errors: how reliable are the severity ratings reported to the national reporting and learning system?","volume":"21","author":"Williams","year":"2009","journal-title":"Int J Qual Health C"},{"issue":"1","key":"2020110613094281100_ocz146-B12","doi-asserted-by":"crossref","first-page":"84.","DOI":"10.1186\/s12911-017-0483-8","article-title":"Using multiclass classification to automate the identification of patient safety incident reports by type and severity","volume":"17","author":"Wang","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613094281100_ocz146-B13","first-page":"609","article-title":"Automating the identification of patient safety incident reports using multi-label classification","volume":"245","author":"Wang","year":"2017","journal-title":"Stud Health Technol Inform"},{"issue":"1","key":"2020110613094281100_ocz146-B14","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1097\/PTS.0000000000000104","article-title":"Screening electronic health record-related patient safety reports using machine learning","volume":"13","author":"Marella","year":"2017","journal-title":"J Patient Saf"},{"issue":"6","key":"2020110613094281100_ocz146-B15","first-page":"e55.","article-title":"Automated categorisation of clinical incident reports using statistical text classification","volume":"19","author":"Ong","year":"2010","journal-title":"Qual Saf Health Care"},{"issue":"e1","key":"2020110613094281100_ocz146-B16","doi-asserted-by":"crossref","first-page":"e110","DOI":"10.1136\/amiajnl-2011-000562","article-title":"Automated identification of extreme-risk events in clinical incident reports","volume":"19","author":"Ong","year":"2012","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"2020110613094281100_ocz146-B17","doi-asserted-by":"crossref","first-page":"338","DOI":"10.3414\/ME15-01-0010","article-title":"An evaluation of patient safety event report categories using unsupervised topic modeling","volume":"54","author":"Fong","year":"2015","journal-title":"Methods Inf Med"},{"issue":"5","key":"2020110613094281100_ocz146-B18","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1136\/amiajnl-2012-001409","article-title":"Using statistical text classification to identify health information technology incidents","volume":"20","author":"Chai","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613094281100_ocz146-B19","first-page":"1024","article-title":"Initializing and growing a database of health information technology (HIT) events by using TF-IDF and biterm topic modeling","volume":"2017","author":"Kang","journal-title":"AMIA Annu SympProc"},{"key":"2020110613094281100_ocz146-B20","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.jbi.2015.09.011","article-title":"Exploring methods for identifying related patient safety events using structured and unstructured data","volume":"58","author":"Fong","year":"2015","journal-title":"J Biomed Inform"},{"issue":"2","key":"2020110613094281100_ocz146-B21","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1136\/amiajnl-2014-002963","article-title":"Connecting the dots': leveraging visual analytics to make sense of patient safety event reports","volume":"22","author":"Ratwani","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613094281100_ocz146-B22","first-page":"1070","article-title":"Automated classification of multi-labeled patient safety reports: a shift from quantity to quality measure","volume":"245","author":"Liang","year":"2017","journal-title":"Stud Health Technol Inform"},{"issue":"06","key":"2020110613094281100_ocz146-B23","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1142\/S0219622013500338","article-title":"A multiclass support vector machine: theory and model","volume":"12","author":"Sun","year":"2013","journal-title":"Int J Inf Tech Dec Mak"},{"key":"2020110613094281100_ocz146-B24","author":"Kim","year":"2014"},{"issue":"10","key":"2020110613094281100_ocz146-B25","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1093\/jamia\/ocy068","article-title":"Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review","volume":"25","author":"Xiao","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613094281100_ocz146-B26","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1007\/978-3-319-59740-9_39","article-title":"Brain white matter lesion segmentation with 2D\/3D CNN","volume":"10337","author":"Lopez-Zorrilla","year":"2017","journal-title":"Nat Artif Comput Biomed Neurosci"},{"key":"2020110613094281100_ocz146-B27","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.media.2016.10.004","article-title":"Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation","volume":"36","author":"Kamnitsas","year":"2017","journal-title":"Med Image Anal"},{"key":"2020110613094281100_ocz146-B28","first-page":"111","article-title":"A deep CNN based multiclass classification of Alzheimer's disease using MRI","author":"Farooq","year":"2017","journal-title":"IEEE Conference on Imaging Systems"},{"key":"2020110613094281100_ocz146-B29","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.jbi.2017.11.003","article-title":"Predicting age by mining electronic medical records with deep learning characterizes differences between chronological and physiological age","volume":"76","author":"Wang","year":"2017","journal-title":"J Biomed Inform"},{"key":"2020110613094281100_ocz146-B30","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1016\/j.jbi.2017.04.001","article-title":"Predicting health care trajectories from medical records: A deep learning approach","volume":"69","author":"Pham","year":"2017","journal-title":"J Biomed Inform"},{"key":"2020110613094281100_ocz146-B31","author":"Le","year":"2017"},{"issue":"6","key":"2020110613094281100_ocz146-B32","doi-asserted-by":"crossref","first-page":"242","DOI":"10.3390\/e19060242","article-title":"A framework for designing the architectures of deep convolutional neural networks","volume":"19","author":"Albelwi","year":"2017","journal-title":"Entropy-Switz"},{"issue":"3","key":"2020110613094281100_ocz146-B33","doi-asserted-by":"crossref","first-page":"1.","DOI":"10.1145\/2716262","article-title":"A tutorial on multilabel learning","volume":"47","author":"Gibaja","year":"2015","journal-title":"ACM Comput Surv"},{"key":"2020110613094281100_ocz146-B34","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4615-0907-3","volume-title":"Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms","author":"Joachims","year":"2002"},{"key":"2020110613094281100_ocz146-B35","doi-asserted-by":"crossref","first-page":"1796","DOI":"10.1109\/BigData.2017.8258123","article-title":"Improving Text Classification with Word Embedding","author":"Ge","year":"2017","journal-title":"2017 IEEE International Conference on Big Data"},{"issue":"1","key":"2020110613094281100_ocz146-B36","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1093\/intqhc\/mzn057","article-title":"Towards an international classification for patient safety: key concepts and terms","volume":"21","author":"Runciman","year":"2009","journal-title":"Int J Qual Health Care"},{"issue":"10","key":"2020110613094281100_ocz146-B37","first-page":"522","article-title":"Developing and deploying a patient safety program in a large health care delivery system: you can't fix what you don't know about","volume":"27","author":"Bagian","year":"2001","journal-title":"Jt Comm J Qual Improv"},{"key":"2020110613094281100_ocz146-B38","author":"Goldberg","year":"2014"},{"key":"2020110613094281100_ocz146-B39","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1016\/j.procs.2017.08.009","article-title":"Comparative study of word embedding methods in topic segmentation","volume":"112","author":"Naili","year":"2017","journal-title":"Procedia Comput Sci"},{"key":"2020110613094281100_ocz146-B40","author":"Xu"},{"key":"2020110613094281100_ocz146-B41","author":"Landola","year":"2017"},{"key":"2020110613094281100_ocz146-B42","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1016\/j.neucom.2015.09.096","article-title":"Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification","volume":"174","author":"Wang","year":"2016","journal-title":"Neurocomputing"},{"key":"2020110613094281100_ocz146-B43","author":"Danushka","year":"2015"},{"key":"2020110613094281100_ocz146-B44","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jbi.2018.09.008","article-title":"A comparison of word embeddings for the biomedical natural language processing","volume":"87","author":"Wang","year":"2018","journal-title":"J Biomed Inform"},{"key":"2020110613094281100_ocz146-B45","volume-title":"Evaluation of the Safety Improvement Program in New South Wales: Study No 6 Report on Program Outcomes","author":"Centre for Clinical Governance Research in Health","year":"2005"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/12\/1600\/34152218\/ocz146.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/12\/1600\/34152218\/ocz146.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:28:38Z","timestamp":1604690918000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/26\/12\/1600\/5555866"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,28]]},"references-count":45,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2019,8,28]]},"published-print":{"date-parts":[[2019,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz146","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,12]]},"published":{"date-parts":[[2019,8,28]]}}}