{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T22:51:20Z","timestamp":1774392680441,"version":"3.50.1"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2021,5,19]],"date-time":"2021-05-19T00:00:00Z","timestamp":1621382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["18H03336"],"award-info":[{"award-number":["18H03336"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["9610406"],"award-info":[{"award-number":["9610406"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,30]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>This study aims to improve the classification of the fall incident severity level by considering data imbalance issues and structured features through machine learning.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We present an incident report classification (IRC) framework to classify the in-hospital fall incident severity level by addressing the imbalanced class problem and incorporating structured attributes. After text preprocessing, bag-of-words features, structured text features, and structured clinical features were extracted from the reports. Next, resampling techniques were incorporated into the training process. Machine learning algorithms were used to build classification models. IRC systems were trained, validated, and tested using a repeated and randomly stratified shuffle-split cross-validation method. Finally, we evaluated the system performance using the F1-measure, precision, and recall over 15 stratified test sets.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The experimental results demonstrated that the classification system setting considering both data imbalance issues and structured features outperformed the other system settings (with a mean macro-averaged F1-measure of 0.733). Considering the structured features and resampling techniques, this classification system setting significantly improved the mean F1-measure for the rare class by 30.88% (P value\u2009&amp;lt;\u2009.001) and the mean macro-averaged F1-measure by 8.26% from the baseline system setting (P value &amp;lt; .001). In general, the classification system employing the random forest algorithm and random oversampling method outperformed the others.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Structured features provide essential information for categorizing the fall incident severity level. Resampling methods help rebalance the class distribution of the original incident report data, which improves the performance of machine learning models. The IRC framework presented in this study effectively automates the identification of fall incident reports by the severity level.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocab048","type":"journal-article","created":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T15:43:54Z","timestamp":1619624634000},"page":"1756-1764","source":"Crossref","is-referenced-by-count":20,"title":["Evaluating resampling methods and structured features to improve fall incident report identification by the severity level"],"prefix":"10.1093","volume":"28","author":[{"given":"Jiaxing","family":"Liu","sequence":"first","affiliation":[{"name":"School of Statistics and Mathematics, Zhongnan University of Economics and Law, Wuhan, China"},{"name":"School of Data Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zoie S Y","family":"Wong","sequence":"additional","affiliation":[{"name":"Graduate School of Public Health, St. Luke\u2019s International University, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"H Y","family":"So","sequence":"additional","affiliation":[{"name":"Alice Ho Miu Ling Nethersole Hospital, New Territories, Hong Kong SAR, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kwok Leung","family":"Tsui","sequence":"additional","affiliation":[{"name":"School of Data Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,5,19]]},"reference":[{"key":"2021073020264412600_ocab048-B1","volume-title":"Patient Safety and Quality: An Evidence-Based Handbook for Nurses","author":"Currie","year":"2008"},{"issue":"6","key":"2021073020264412600_ocab048-B2","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1136\/qshc.2007.024695","article-title":"Falls in English and Welsh hospitals: a national observational study based on retrospective analysis of 12 months of patient safety incident reports","volume":"17","author":"Healey","year":"2008","journal-title":"Qual Saf Health Care"},{"issue":"4","key":"2021073020264412600_ocab048-B3","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1111\/jep.12144","article-title":"Falls in hospital increase length of stay regardless of degree of harm","volume":"20","author":"Dunne","year":"2014","journal-title":"J Eval Clin Pract"},{"issue":"3","key":"2021073020264412600_ocab048-B4","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1071\/AH070471","article-title":"Falls in the acute hospital setting\u2013impact on resource utilisation","volume":"31","author":"Hill","year":"2007","journal-title":"Aust Health Review"},{"issue":"6","key":"2021073020264412600_ocab048-B5","first-page":"e51","article-title":"A 10-year cohort study of the burden and risk of in-hospital falls and fractures using routinely collected hospital data","volume":"19","author":"Brand","year":"2010","journal-title":"Quality Saf Health Care"},{"issue":"4","key":"2021073020264412600_ocab048-B6","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1080\/03610730500206881","article-title":"Incidence and consequence of falls in inpatient rehabilitation of stroke patients","volume":"31","author":"Suzuki","year":"2005","journal-title":"Exp Aging Res"},{"issue":"1","key":"2021073020264412600_ocab048-B7","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1136\/qshc.2007.022400","article-title":"Trends in healthcare incident reporting and relationship to safety and quality data in acute hospitals: results from the National Reporting and Learning System","volume":"18","author":"Hutchinson","year":"2009","journal-title":"Qual Saf Health Care"},{"issue":"1-2","key":"2021073020264412600_ocab048-B8","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.jbi.2003.08.003","article-title":"Detecting adverse events for patient safety research: a review of current methodologies","volume":"36","author":"Murff","year":"2003","journal-title":"J Biomed Inform"},{"key":"2021073020264412600_ocab048-B9","volume-title":"Advances in Patient Safety: New Directions and Alternative Approaches. (Vol. 1: Assessment)","author":"Pronovost","year":"2008"},{"issue":"e1","key":"2021073020264412600_ocab048-B10","doi-asserted-by":"crossref","first-page":"e110","DOI":"10.1136\/amiajnl-2011-000562","article-title":"Automated identification of extreme-risk events in clinical incident reports","volume":"19","author":"Ong","year":"2012","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2021073020264412600_ocab048-B11","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1186\/s12911-017-0483-8","article-title":"Using multiclass classification to automate the identification of patient safety incident reports by type and severity","volume":"17","author":"Wang","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"issue":"12","key":"2021073020264412600_ocab048-B12","doi-asserted-by":"crossref","first-page":"1600","DOI":"10.1093\/jamia\/ocz146","article-title":"Using convolutional neural networks to identify patient safety incident reports by type and severity","volume":"26","author":"Wang","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2021073020264412600_ocab048-B13","first-page":"639","article-title":"Scoring patient fall reports using quality rubric and machine learning","volume":"264","author":"Klock","year":"2019","journal-title":"Stud Health Technol Inform"},{"issue":"4","key":"2021073020264412600_ocab048-B14","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1007\/s10916-009-9398-y","article-title":"Data consistency in a voluntary medical incident reporting system","volume":"35","author":"Gong","year":"2011","journal-title":"J Med Syst"},{"issue":"8","key":"2021073020264412600_ocab048-B15","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1177\/1049732310369232","article-title":"Multiple accountabilities in incident reporting and management","volume":"20","author":"Hor","year":"2010","journal-title":"Qual Health Res"},{"key":"2021073020264412600_ocab048-B16","doi-asserted-by":"crossref","first-page":"103089","DOI":"10.1016\/j.jbi.2018.12.003","article-title":"A comprehensive data level analysis for cancer diagnosis on imbalanced data","volume":"90","author":"Fotouhi","year":"2019","journal-title":"J Biomed Inform"},{"key":"2021073020264412600_ocab048-B17","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1016\/j.eswa.2018.09.034","article-title":"Clinical text classification research trends: systematic literature review and open issues","volume":"116","author":"Mujtaba","year":"2019","journal-title":"Expert Syst Appl"},{"issue":"11","key":"2021073020264412600_ocab048-B18","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1093\/jamia\/ocz149","article-title":"Evaluating shallow and deep learning strategies for the 2018 n2c2 shared task on clinical text classification","volume":"26","author":"Oleynik","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"2021073020264412600_ocab048-B19","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1159\/000129954","article-title":"Characteristics of hospital inpatient falls across clinical departments","volume":"54","author":"Schwendimann","year":"2008","journal-title":"Gerontology"},{"issue":"12","key":"2021073020264412600_ocab048-B20","doi-asserted-by":"crossref","first-page":"2403","DOI":"10.1111\/ggi.13085","article-title":"Analysis of falls that caused serious events in hospitalized patients","volume":"17","author":"Kobayashi","year":"2017","journal-title":"Geriatr Gerontol Int"},{"issue":"3","key":"2021073020264412600_ocab048-B21","first-page":"291","article-title":"Incidence and characteristics of accidental falls in hospitalizations","volume":"79","author":"Kobayashi","year":"2017","journal-title":"Nagoya J Med Sci"},{"issue":"2","key":"2021073020264412600_ocab048-B22","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1177\/1460458214555040","article-title":"Statistical classification of drug incidents due to look-alike sound-alike mix-ups","volume":"22","author":"Wong","year":"2016","journal-title":"Health Informatics J"},{"issue":"5","key":"2021073020264412600_ocab048-B23","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1016\/j.jbi.2009.08.007","article-title":"What can natural language processing do for clinical decision support?","volume":"42","author":"Demner-Fushman","year":"2009","journal-title":"J Biomed Inform"},{"issue":"3","key":"2021073020264412600_ocab048-B24","doi-asserted-by":"crossref","first-page":"1777","DOI":"10.1177\/1460458219889798","article-title":"Medication-rights detection using incident reports: A natural language processing and deep neural network approach","volume":"26","author":"Wong","year":"2020","journal-title":"Health Informatics J"},{"issue":"6","key":"2021073020264412600_ocab048-B25","first-page":"e55","article-title":"Automated categorisation of clinical incident reports using statistical text classification","volume":"19","author":"Ong","year":"2010","journal-title":"Qual Saf Health Care"},{"issue":"9","key":"2021073020264412600_ocab048-B26","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2021073020264412600_ocab048-B27","doi-asserted-by":"crossref","first-page":"6275435","DOI":"10.1155\/2018\/6275435","article-title":"A framework of rebalancing imbalanced healthcare data for rare events\u2019 classification: a case of look-alike sound-alike mix-up incident detection","volume":"2018","author":"Zhao","year":"2018","journal-title":"J Healthc Eng"},{"key":"2021073020264412600_ocab048-B28","volume-title":"Minimal Information Model for Patient Safety Incident Reporting and Learning Systems: user Guide. Document Number: WHO: HIS\/SDS\/2016.22","year":"2016"},{"key":"2021073020264412600_ocab048-B29","author":"Aristotelis","year":"2016"},{"key":"2021073020264412600_ocab048-B30","doi-asserted-by":"crossref","first-page":"105662","DOI":"10.1016\/j.asoc.2019.105662","article-title":"An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets","volume":"83","author":"Kov\u00e1cs","year":"2019","journal-title":"Appl Soft Comput"},{"issue":"11","key":"2021073020264412600_ocab048-B31","first-page":"769","article-title":"Two modifications of CNN","volume":"6","author":"Tomek","year":"1976","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"2021073020264412600_ocab048-B32","first-page":"321","author":"Chawla"},{"key":"2021073020264412600_ocab048-B33","author":"Han","year":"23\u201326, 2005; ,"},{"key":"2021073020264412600_ocab048-B34","author":"Barua","year":"14\u201317, 2013;  ,"},{"key":"2021073020264412600_ocab048-B35","author":"Gazzah","year":"2008"},{"issue":"1","key":"2021073020264412600_ocab048-B36","first-page":"559","article-title":"Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning","volume":"18","author":"Lema\u00eetre","year":"2017","journal-title":"J Mach Learn Res"},{"key":"2021073020264412600_ocab048-B37","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1016\/j.neucom.2019.06.100","article-title":"SMOTE-variants: A python implementation of 85 minority oversampling techniques","volume":"366","author":"Kov\u00e1cs","year":"2019","journal-title":"Neurocomputing"},{"issue":"4","key":"2021073020264412600_ocab048-B38","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inform Process Manag"},{"issue":"4","key":"2021073020264412600_ocab048-B39","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1109\/MCI.2018.2866730","article-title":"Cross-validation for imbalanced datasets: avoiding overoptimistic and overfitting approaches [research frontier]","volume":"13","author":"Santos","year":"2018","journal-title":"IEEE Comput Intell Mag"},{"issue":"1","key":"2021073020264412600_ocab048-B40","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"issue":"8964","key":"2021073020264412600_ocab048-B41","doi-asserted-by":"crossref","first-page":"1549","DOI":"10.1016\/S0140-6736(95)91091-3","article-title":"Falls, a community care perspective","volume":"345","author":"Van Weel","year":"1995","journal-title":"Lancet"},{"issue":"1","key":"2021073020264412600_ocab048-B42","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1186\/1471-2105-14-106","article-title":"SMOTE for high-dimensional class-imbalanced data","volume":"14","author":"Blagus","year":"2013","journal-title":"BMC Bioinformatics"},{"issue":"4","key":"2021073020264412600_ocab048-B43","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1111\/j.1365-2125.2011.04166.x","article-title":"A review of medication incidents reported to the National Reporting and Learning System in England and Wales over 6 years (2005\u20132010)","volume":"74","author":"Cousins","year":"2012","journal-title":"Br J Clin Pharmacol"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/8\/1756\/39502196\/ocab048.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/8\/1756\/39502196\/ocab048.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,30]],"date-time":"2021-07-30T23:04:49Z","timestamp":1627686289000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/8\/1756\/6278433"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,19]]},"references-count":43,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,5,19]]},"published-print":{"date-parts":[[2021,7,30]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab048","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,8,1]]},"published":{"date-parts":[[2021,5,19]]}}}