{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T03:17:17Z","timestamp":1776309437802,"version":"3.50.1"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2024,7,4]],"date-time":"2024-07-04T00:00:00Z","timestamp":1720051200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R35GM131905"],"award-info":[{"award-number":["R35GM131905"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The aim of this project was to create time-aware, individual-level risk score models for adverse drug events related to multiple sclerosis disease-modifying therapy and to provide interpretable explanations for model prediction behavior.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We used temporal sequences of observational medical outcomes partnership common data model (OMOP CDM) concepts derived from an electronic health record as model features. Each concept was assigned an embedding representation that was learned from a graph convolution network trained on a knowledge graph (KG) of OMOP concept relationships. Concept embeddings were fed into long short-term memory networks for 1-year adverse event prediction following drug exposure. Finally, we implemented a novel extension of the local interpretable model agnostic explanation (LIME) method, knowledge graph LIME (KG-LIME) to leverage the KG and explain individual predictions of each model.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>For a set of 4859 patients, we found that our model was effective at predicting 32 out of 56 adverse event types (P\u2009&amp;lt;\u2009.05) when compared to demographics and past diagnosis as variables. We also assessed discrimination in the form of area under the curve (AUC\u2009=\u20090.77\u2009\u00b1\u20090.15) and area under the precision-recall curve (AUC-PR\u2009=\u20090.31\u2009\u00b1\u20090.27) and assessed calibration in the form of Brier score (BS\u2009=\u20090.04\u2009\u00b1\u20090.04). Additionally, KG-LIME generated interpretable literature-validated lists of relevant medical concepts used for prediction.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion and Conclusion<\/jats:title>\n                  <jats:p>Many of our risk models demonstrated high calibration and discrimination for adverse event prediction. Furthermore, our novel KG-LIME method was able to utilize the knowledge graph to highlight concepts that were important to prediction. Future work will be required to further explore the temporal window of adverse event occurrence beyond the generic 1-year window used here, particularly for short-term inpatient adverse events and long-term severe adverse events.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae155","type":"journal-article","created":{"date-parts":[[2024,7,4]],"date-time":"2024-07-04T23:07:54Z","timestamp":1720134474000},"page":"1693-1703","source":"Crossref","is-referenced-by-count":8,"title":["KG-LIME: predicting individualized risk of adverse drug events for multiple sclerosis disease-modifying therapy"],"prefix":"10.1093","volume":"31","author":[{"given":"Jason","family":"Patterson","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York, NY 10032,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicholas","family":"Tatonetti","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York, NY 10032,","place":["United States"]},{"name":"Department of Computational Biomedicine, Cedars-Sinai Cancer, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,7,4]]},"reference":[{"key":"2024110509562886200_ocae155-B1","doi-asserted-by":"crossref","first-page":"1816","DOI":"10.1177\/1352458520970841","article-title":"Rising prevalence of multiple sclerosis worldwide: insights from the Atlas of MS, third edition","volume":"26","author":"Walton","year":"2020","journal-title":"Mult Scler"},{"issue":"4","key":"2024110509562886200_ocae155-B2","doi-asserted-by":"crossref","first-page":"919","DOI":"10.1016\/j.ncl.2016.06.016","article-title":"Epidemiology of multiple sclerosis","volume":"34","author":"Howard","year":"2016","journal-title":"Neurol Clin"},{"key":"2024110509562886200_ocae155-B3","year":"2023"},{"issue":"9","key":"2024110509562886200_ocae155-B4","doi-asserted-by":"crossref","first-page":"a028928","DOI":"10.1101\/cshperspect.a028928","article-title":"Clinical course of multiple sclerosis","volume":"8","author":"Klineova","year":"2018","journal-title":"Cold Spring Harb Perspect Med"},{"key":"2024110509562886200_ocae155-B5","doi-asserted-by":"crossref","first-page":"s53","DOI":"10.7861\/clinmedicine.16-6-s53","article-title":"Multiple sclerosis, a treatable disease","volume":"16(Suppl 6)","author":"Doshi","year":"2016","journal-title":"Clin Med"},{"key":"2024110509562886200_ocae155-B6","author":"National Multiple Sclerosis Society","year":"2023"},{"issue":"6","key":"2024110509562886200_ocae155-B7","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1007\/s00415-015-7986-y","article-title":"Optimizing treatment success in multiple sclerosis","volume":"263","author":"Ziemssen","year":"2016","journal-title":"J Neurol"},{"key":"2024110509562886200_ocae155-B8","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1177\/1352458520949986","article-title":"Adverse event profile differences between rituximab and ocrelizumab: findings from the FDA Adverse Event Reporting Database","volume":"27","author":"Caldito","year":"2021","journal-title":"Mult Scler."},{"key":"2024110509562886200_ocae155-B9","doi-asserted-by":"crossref","first-page":"104015","DOI":"10.1016\/j.msard.2022.104015","article-title":"Ocrelizumab-related neutropenia: effects of age, sex and bodyweight using the FDA Adverse Event Reporting System (FAERS)","volume":"65","author":"Hammer","year":"2022","journal-title":"Mult Scler Relat Disord"},{"key":"2024110509562886200_ocae155-B10","doi-asserted-by":"crossref","first-page":"693017","DOI":"10.3389\/fneur.2021.693017","article-title":"Early high efficacy treatment in multiple sclerosis is the best predictor of future disease activity over 1 and 2 years in a Norwegian population-based registry","volume":"12","author":"Simonsen","year":"2021","journal-title":"Front Neurol"},{"issue":"4","key":"2024110509562886200_ocae155-B11","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/S1474-4422(20)30067-3","article-title":"Timing of high-efficacy therapy for multiple sclerosis: a retrospective observational cohort study","volume":"19","author":"He","year":"2020","journal-title":"Lancet Neurol"},{"issue":"11","key":"2024110509562886200_ocae155-B12","doi-asserted-by":"crossref","first-page":"e050176","DOI":"10.1136\/bmjopen-2021-050176","article-title":"OPTIMISE: MS study protocol: a pragmatic, prospective observational study to address the need for, and challenges with, real world pharmacovigilance in multiple sclerosis","volume":"11","author":"Dobson","year":"2021","journal-title":"BMJ Open"},{"key":"2024110509562886200_ocae155-B13","first-page":"75","author":"."},{"issue":"1","key":"2024110509562886200_ocae155-B14","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1007\/s40290-022-00456-6","article-title":"Real-world evidence: a primer","volume":"37","author":"Dang","year":"2023","journal-title":"Pharm Med"},{"issue":"1-2","key":"2024110509562886200_ocae155-B15","doi-asserted-by":"crossref","first-page":"108","DOI":"10.3121\/cmr.2014.1250.ps1-11","article-title":"PS1-11: a comparison of electronic medical records vs claims data for rheumatoid arthritis patients in a large healthcare system: an exploratory analysis","volume":"12","author":"Maeng","year":"2014","journal-title":"Clin Med Res"},{"key":"2024110509562886200_ocae155-B16","doi-asserted-by":"crossref","first-page":"105246","DOI":"10.1016\/j.ijmedinf.2023.105246","article-title":"Machine learning models to detect and predict patient safety events using electronic health records: a systematic review","volume":"180","author":"Deimazar","year":"2023","journal-title":"Int J Med Inform"},{"issue":"5","key":"2024110509562886200_ocae155-B17","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1093\/jamia\/ocad014","article-title":"Electronic health record-based prediction models for in-hospital adverse drug event diagnosis or prognosis: a systematic review","volume":"30","author":"Yasrebi-de Kom","year":"2023","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2024110509562886200_ocae155-B18","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s12911-018-0717-4","article-title":"A classification framework for exploiting sparse multi-variate temporal features with application to adverse drug event detection in medical records","volume":"19","author":"Bagattini","year":"2019","journal-title":"BMC Med Inform Decis Making"},{"issue":"3","key":"2024110509562886200_ocae155-B19","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1002\/bimj.201700067","article-title":"Variable selection\u2014a review and recommendations for the practicing statistician","volume":"60","author":"Heinze","year":"2018","journal-title":"Biom J"},{"key":"2024110509562886200_ocae155-B20","author":"Pang","year":"2021"},{"key":"2024110509562886200_ocae155-B21","author":"Monnin","year":"2021"},{"key":"2024110509562886200_ocae155-B22","author":"Tanaka","year":"2024"},{"key":"2024110509562886200_ocae155-B23"},{"key":"2024110509562886200_ocae155-B24","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1038\/s41746-019-0110-4","article-title":"Semantic integration of clinical laboratory tests from electronic health records for deep phenotyping and biomarker discovery","volume":"2","author":"Zhang","year":"2019","journal-title":"NPJ Digital Med."},{"issue":"3","key":"2024110509562886200_ocae155-B25","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1093\/jamia\/ocu023","article-title":"Feasibility and utility of applications of the common data model to multiple, disparate observational health databases","volume":"22","author":"Voss","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2024110509562886200_ocae155-B26","author":"Schlichtkrull","year":"2017"},{"key":"2024110509562886200_ocae155-B27","author":"Zhang","year":"2022"},{"key":"2024110509562886200_ocae155-B28","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"2024110509562886200_ocae155-B29"},{"key":"2024110509562886200_ocae155-B30","author":"Ribeiro","year":"2016"},{"key":"2024110509562886200_ocae155-B31"},{"issue":"4","key":"2024110509562886200_ocae155-B32","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1111\/bcp.12530","article-title":"Pharmacodynamics of cytarabine induced leucopenia: a retrospective cohort study","volume":"79","author":"Shepshelovich","year":"2015","journal-title":"Br J Clin Pharmacol"},{"key":"2024110509562886200_ocae155-B33","year":"2023"},{"issue":"2","key":"2024110509562886200_ocae155-B34","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1093\/oxfordjournals.jjco.a039131","article-title":"Phase II study of mitoxantrone in patients with non-small cell lung cancer","volume":"16","author":"Suga","year":"1986","journal-title":"Jpn J Clin Oncol"},{"issue":"10","key":"2024110509562886200_ocae155-B35","doi-asserted-by":"crossref","first-page":"3400","DOI":"10.1016\/j.transproceed.2014.07.070","article-title":"Incidence and management of leukopenia\/neutropenia in 233 kidney transplant patients following single dose alemtuzumab induction","volume":"46","author":"Smith","year":"2014","journal-title":"Transpl Proc"},{"issue":"2","key":"2024110509562886200_ocae155-B36","first-page":"189","article-title":"Natalizumab for the treatment of relapsing multiple sclerosis","volume":"2","author":"Rudick","year":"2008","journal-title":"Biologics"},{"issue":"8","key":"2024110509562886200_ocae155-B37","doi-asserted-by":"crossref","first-page":"2121","DOI":"10.1007\/s00213-021-05836-5","article-title":"Glatiramer acetate attenuates depressive\/anxiety-like behaviors and cognitive deficits induced by post-weaning social isolation in male mice","volume":"238","author":"Salihu","year":"2021","journal-title":"Psychopharmacology (Berl)"},{"key":"2024110509562886200_ocae155-B38","first-page":"1"},{"issue":"5","key":"2024110509562886200_ocae155-B39","doi-asserted-by":"crossref","first-page":"E940","DOI":"10.3390\/ijms18050940","article-title":"Natalizumab in multiple sclerosis: long-term management","volume":"18","author":"Clerico","year":"2017","journal-title":"Int J Mol Sci"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/8\/1693\/60421108\/ocae155.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/8\/1693\/60421108\/ocae155.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,5]],"date-time":"2024-11-05T09:56:50Z","timestamp":1730800610000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/8\/1693\/7706266"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,4]]},"references-count":39,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,7,4]]},"published-print":{"date-parts":[[2024,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae155","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,8]]},"published":{"date-parts":[[2024,7,4]]}}}