{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T07:02:02Z","timestamp":1787986922559,"version":"build-2784847793"},"reference-count":43,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Bleeding complications are a major contributor to adverse drug events among older inpatients, particularly in those treated with antithrombotic agents. Timely and accurate detection of bleeding events is essential for improving drug safety surveillance and clinical risk management.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The study aimed to develop and validate automated algorithms for detecting major bleeding (MB) and clinically relevant nonmajor bleeding (CRNMB) events from electronic medical records (EMRs) by combining structured data-based rule models and a natural language processing (NLP) approach, and to evaluate their performance and generalizability against a manually reviewed gold standard and an external dataset.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n We conducted a multicenter retrospective study using routinely collected EMR data from 3 Swiss university hospitals. Patients 65 years or older who received at least one antithrombotic agent and were hospitalized between January 2015 and December 2016 were included. To detect MB and CRNMB events, rule-based algorithms were developed using structured data (\n International Statistical Classification of Diseases, 10th Revision, German Modification<\/jats:italic>\n [\n ICD-10-GM<\/jats:italic>\n ] codes, laboratory values, transfusion records, and antihemorrhagic prescriptions), with variables and cutoff values defined according to adapted International Society on Thrombosis and Haemostasis definitions and expert consensus. In parallel, a supervised NLP model was applied to discharge summaries from one hospital. A manual review of 754 EMRs served as the reference standard for internal validation, and the algorithm performance of the structured data algorithms (SDA), NLP, and their combination (SDA+NLP) was evaluated against this manually reviewed gold standard using standard performance metrics. External validation was performed on an independent dataset from the Lausanne University Hospital to assess model robustness and generalizability.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Among 36,039 inpatient stays, SDA identified 8.26% (n=2979) as MB and 15.04% (n=5419) as CRNMB cases.\n ICD-10-GM<\/jats:italic>\n codes alone detected 28.5% (n=849) of MB and 31.48% (n=1706) of CRNMB cases, while laboratory data contributed most to event detection (n=1994, 66.94% for MB and n=3663, 67.60% for CRNMB). Integrating SDA with NLP improved detection, identifying 12.2% (920\/7513) of MB and 27.4% (2062\/7513) of CRNMB cases at 1 hospital. The combined model achieved the best performance (sensitivity 0.84, positive predictive value 0.51,\n F<\/jats:italic>\n 1<\/jats:sub>\n -score 0.64). External validation on Lausanne University Hospital 2021\u20102022 data (n=24,054 stays) confirmed the algorithms\u2019 reproducibility; the prevalence of MB decreased while CRNMB increased, reflecting evolving clinical practices and antithrombotic use patterns.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Our integrated approach, combining SDA with NLP, enhances the detection of hemorrhagic events in older hospitalized patients treated with antithrombotic agents, suggesting its potential usefulness for drug safety monitoring and clinical risk management.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/77809","type":"journal-article","created":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T22:50:08Z","timestamp":1764283808000},"page":"e77809-e77809","source":"Crossref","is-referenced-by-count":1,"title":["Detection of Antithrombotic-Related Bleeding in Older Inpatients: Multicenter Retrospective Study Using Structured and Unstructured Electronic Health Record Data"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2303-7848","authenticated-orcid":false,"given":"Claire","family":"Coumau","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0225-7294","authenticated-orcid":false,"given":"Frederic","family":"Gaspar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4758-6851","authenticated-orcid":false,"given":"Mehdi","family":"Zayene","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6523-2443","authenticated-orcid":false,"given":"Elliott","family":"Bertrand","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9686-9920","authenticated-orcid":false,"given":"Lorenzo","family":"Alberio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2681-8076","authenticated-orcid":false,"given":"Christian","family":"Lovis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6097-2480","authenticated-orcid":false,"given":"Patrick E","family":"Beeler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5718-5462","authenticated-orcid":false,"given":"Fabio","family":"Rinaldi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7063-578X","authenticated-orcid":false,"given":"Monika","family":"Lutters","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1672-3644","authenticated-orcid":false,"given":"Marie-Annick","family":"Le Pogam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0660-082X","authenticated-orcid":false,"given":"Chantal","family":"Csajka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"name":"SwissMADE Collaborators","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,1,29]]},"reference":[{"issue":"6","key":"R1","doi-asserted-by":"publisher","first-page":"948","DOI":"10.1093\/ageing\/afaa188","article-title":"In-hospital adverse drug reactions in older adults; prevalence, presentation and associated drugs-a systematic review and meta-analysis","volume":"49","author":"Jennings","journal-title":"Age Ageing"},{"issue":"5","key":"R2","doi-asserted-by":"publisher","first-page":"542","DOI":"10.1093\/intqhc\/mzt056","article-title":"What is known about adverse events in older medical hospital inpatients? A systematic review of the literature","volume":"25","author":"Long","journal-title":"Int J Qual Health Care"},{"issue":"46","key":"R3","doi-asserted-by":"publisher","first-page":"3238","DOI":"10.1093\/eurheartj\/ehv304","article-title":"Antithrombotic therapy in the elderly: expert position paper of the European Society of Cardiology Working Group on Thrombosis","volume":"36","author":"Andreotti","journal-title":"Eur Heart J"},{"issue":"5","key":"R4","doi-asserted-by":"publisher","first-page":"385","DOI":"10.2165\/00002018-200629050-00003","article-title":"Under-reporting of adverse drug reactions: a systematic review","volume":"29","author":"Hazell","journal-title":"Drug Saf"},{"issue":"1-2","key":"R5","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1016\/j.jbi.2003.08.003","article-title":"Detecting adverse events for patient safety research: a review of current methodologies","volume":"36","author":"Murff","journal-title":"J Biomed Inform"},{"issue":"6","key":"R6","doi-asserted-by":"publisher","first-page":"1728","DOI":"10.1016\/j.surg.2022.08.021","article-title":"Development and validation of a model for surveillance of postoperative bleeding complications using structured electronic health records data","volume":"172","author":"Dyas","journal-title":"Surgery"},{"issue":"5-06","key":"R7","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1055\/a-2165-5552","article-title":"Performance characteristics of a rule-based electronic health record algorithm to identify patients with gross and microscopic hematuria","volume":"62","author":"Kashkoush","journal-title":"Methods Inf Med"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2019.8857244","article-title":"Predicting gastrointestinal bleeding events from multimodal in-hospital electronic health records using deep fusion networks","volume":"2019","author":"Hung","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"issue":"5","key":"R9","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1007\/s40290-022-00441-z","article-title":"The use of artificial intelligence in pharmacovigilance: a systematic review of the literature","volume":"36","author":"Salas","journal-title":"Pharmaceut Med"},{"issue":"6","key":"R10","doi-asserted-by":"publisher","first-page":"1590","DOI":"10.1111\/jgh.15313","article-title":"Early identification of patients with acute gastrointestinal bleeding using natural language processing and decision rules","volume":"36","author":"Shung","journal-title":"J Gastroenterol Hepatol"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","DOI":"10.2196\/10788","article-title":"Detection of bleeding events in electronic health record notes using convolutional neural network models enhanced with recurrent neural network autoencoders: deep learning approach","volume":"7","author":"Li","journal-title":"JMIR Med Inform"},{"issue":"7","key":"R12","doi-asserted-by":"publisher","DOI":"10.2196\/27527","article-title":"Relation classification for bleeding events from electronic health records using deep learning systems: an empirical study","volume":"9","author":"Mitra","journal-title":"JMIR Med Inform"},{"issue":"6","key":"R13","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2018.3451","article-title":"Comparison of 2 natural language processing methods for identification of bleeding among critically Ill patients","volume":"1","author":"Taggart","journal-title":"JAMA Netw Open"},{"key":"R14","doi-asserted-by":"crossref","unstructured":"Walker AL Watson C Butcher R Abedin Z Yandell M Shah RU . Use of commercially available natural language processing software to identify bleeding from the medical record. medRxiv. Preprint posted online on Nov 19, 2021. [doi: 10.1101\/2021.11.19.21266586]","DOI":"10.1101\/2021.11.19.21266586"},{"issue":"8","key":"R15","doi-asserted-by":"publisher","DOI":"10.1002\/pds.5875","article-title":"Validation of a major and clinically relevant nonmajor bleeding phenotyping algorithm on electronic health records","volume":"33","author":"Yap","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1016\/j.jns.2023.120807","article-title":"Clinical narratives as a predictor for prognosticating functional outcomes after intracerebral hemorrhage","volume":"453","author":"Hung","journal-title":"J Neurol Sci"},{"key":"R17","volume":"2017","author":"Lee","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.3389\/fnagi.2022.798132","article-title":"Development and validation of an automatic system for intracerebral hemorrhage medical text recognition and treatment plan output","volume":"14","author":"Deng","journal-title":"Front Aging Neurosci"},{"key":"R19","doi-asserted-by":"publisher","DOI":"10.1016\/j.thromres.2024.109182","article-title":"Identification of hematuria with a natural language processing model and validation of hematuria diagnosecodes","volume":"244","author":"Hansen","journal-title":"Thromb Res"},{"issue":"23","key":"R20","doi-asserted-by":"publisher","first-page":"2736","DOI":"10.1161\/CIRCULATIONAHA.110.009449","article-title":"Standardized bleeding definitions for cardiovascular clinical trials: a consensus report from the Bleeding Academic Research Consortium","volume":"123","author":"Mehran","journal-title":"Circulation"},{"key":"R21","unstructured":"Le CHUV en chiffres [Web Page in French]. Centre Hospitalier Universitaire Vaudois. 2023. URL: https:\/\/www.chuv.ch\/fr\/a-propos\/le-chuv-en-chiffres [Accessed 10-01-2026]"},{"key":"R22","unstructured":"Facts, figures and dates. H\u00f4pitaux Universitaires De Gen\u00e8ve. URL: https:\/\/www.hug.ch\/en\/facts-figures-and-dates [Accessed 10-01-2026]"},{"key":"R23","unstructured":"About the USZ 2025. Universit\u00e4ts Spital Z\u00fcrich. URL: https:\/\/www.usz.ch\/en\/about-university-hospital-zurich [Accessed 07-10-2025]"},{"key":"R24","unstructured":"KSB weiterhin auf wachstumskurs [Web Page in German]. Kantonsspital Baden. 2024. URL: https:\/\/www.kantonsspitalbaden.ch\/news\/ksb-weiterhin-auf-wachstumskurs [Accessed 10-01-2026]"},{"issue":"11","key":"R25","doi-asserted-by":"publisher","DOI":"10.2196\/40456","article-title":"Automatic detection of adverse drug events in geriatric care: study proposal","volume":"11","author":"Gaspar","journal-title":"JMIR Res Protoc"},{"issue":"4","key":"R26","doi-asserted-by":"publisher","first-page":"692","DOI":"10.1111\/j.1538-7836.2005.01204.x","article-title":"Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients","volume":"3","author":"Schulman","journal-title":"J Thromb Haemost"},{"issue":"11","key":"R27","doi-asserted-by":"publisher","first-page":"2119","DOI":"10.1111\/jth.13140","article-title":"Definition of clinically relevant non-major bleeding in studies of anticoagulants in atrial fibrillation and venous thromboembolic disease in non-surgical patients: communication from the SSC of the ISTH","volume":"13","author":"Kaatz","journal-title":"J Thromb Haemost"},{"issue":"1","key":"R28","first-page":"411","volume":"7","author":"Honnibal","journal-title":"Sentometrics Res"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.2196\/67837","article-title":"Natural language processing and coding for detecting bleeding events in discharge summaries: comparative cross-sectional study","volume":"13","author":"Gaspar","journal-title":"JMIR Med Inform"},{"issue":"1","key":"R30","doi-asserted-by":"publisher","first-page":"E1","DOI":"10.1177\/0272989X11426176","article-title":"Test result-based sampling: an efficient design for estimating the accuracy of patient safety indicators","volume":"32","author":"Taff\u00e9","journal-title":"Med Decis Making"},{"issue":"1","key":"R31","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","journal-title":"Biometrics"},{"key":"R32","unstructured":"The use of the WHO-UMC system for standardised case causality assessment. World Health Organization - Uppsala Monitoring Centre; 2018. URL: https:\/\/www.who.int\/docs\/default-source\/medicines\/pharmacovigilance\/whocausality-assessment.pdf [Accessed 10-01-2026]"},{"issue":"5","key":"R33","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/0021-9681(87)90171-8","article-title":"A new method of classifying prognostic comorbidity in longitudinal studies: development and validation","volume":"40","author":"Charlson","journal-title":"J Chronic Dis"},{"issue":"1","key":"R34","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1097\/00005650-199801000-00004","article-title":"Comorbidity measures for use with administrative data","volume":"36","author":"Elixhauser","journal-title":"Med Care"},{"issue":"2","key":"R35","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1182\/bloodadvances.2022007263","article-title":"Incidence and clinical impact of bleeding events in older patients with acute venous thromboembolism","volume":"7","author":"Ferrazzini","journal-title":"Blood Adv"},{"issue":"4","key":"R36","doi-asserted-by":"publisher","first-page":"1017","DOI":"10.1007\/s11239-020-02274-6","article-title":"Predicting the risk for major bleeding in elderly patients with venous thromboembolism using the Charlson index. Findings from the RIETE","volume":"51","author":"G\u00f3mez-Cuervo","journal-title":"J Thromb Thrombolysis"},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.1016\/j.thromres.2019.08.024","article-title":"Very elderly patients with venous thromboembolism on oral anticoagulation with VKAs or DOACs: results from the prospective multicenter START2-Register Study","volume":"183","author":"Poli","journal-title":"Thromb Res"},{"issue":"1","key":"R38","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s40266-023-01076-8","article-title":"Prevalence of and risk factors for drug-related readmissions in older adults: a systematic review and meta-analysis","volume":"41","author":"Prasad","journal-title":"Drugs Aging"},{"issue":"7","key":"R39","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1002\/pds.4783","article-title":"Assessment of ICD-10-CM code assignment validity for case finding of outpatient anticoagulant-related bleeding among Medicare beneficiaries","volume":"28","author":"Shehab","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"R40","doi-asserted-by":"publisher","DOI":"10.1016\/j.thromres.2019.07.021","article-title":"Accuracy of ICD-10 codes for identifying hospitalizations for acute anticoagulation therapy-related bleeding events","volume":"181","author":"Joos","journal-title":"Thromb Res"},{"key":"R41","volume":"2020","author":"Mitra","journal-title":"AMIA Annu Symp Proc"},{"issue":"2","key":"R42","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1007\/s10557-021-07232-9","article-title":"Severe bleeding risk of direct oral anticoagulants versus vitamin K antagonists for stroke prevention and treatment in patients with atrial fibrillation: a systematic review and network meta-analysis","volume":"37","author":"Xu","journal-title":"Cardiovasc Drugs Ther"},{"issue":"6","key":"R43","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1093\/intqhc\/13.6.475","article-title":"Process versus outcome indicators in the assessment of quality of health care","volume":"13","author":"Mant","journal-title":"Int J Qual Health Care"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e77809","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:10:08Z","timestamp":1782843008000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e77809"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,29]]},"references-count":43,"URL":"https:\/\/doi.org\/10.2196\/77809","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,29]]},"article-number":"v28i6e77809"}}