{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T23:45:47Z","timestamp":1780962347590,"version":"3.54.1"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"e1","content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2015,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Objectives To improve the accuracy of mining structured and unstructured components of the electronic medical record (EMR) by adding temporal features to automatically identify patients with rheumatoid arthritis (RA) with methotrexate-induced liver transaminase abnormalities.<\/jats:p>\n               <jats:p>Materials and methods Codified information and a string-matching algorithm were applied to a RA cohort of 5903 patients from Partners HealthCare to select 1130 patients with potential liver toxicity. Supervised machine learning was applied as our key method. For features, Apache clinical Text Analysis and Knowledge Extraction System (cTAKES) was used to extract standard vocabulary from relevant sections of the unstructured clinical narrative. Temporal features were further extracted to assess the temporal relevance of event mentions with regard to the date of transaminase abnormality. All features were encapsulated in a 3-month-long episode for classification. Results were summarized at patient level in a training set (N=480 patients) and evaluated against a test set (N=120 patients).<\/jats:p>\n               <jats:p>Results The system achieved positive predictive value (PPV) 0.756, sensitivity 0.919, F1 score 0.829 on the test set, which was significantly better than the best baseline system (PPV 0.590, sensitivity 0.703, F1 score 0.642). Our innovations, which included framing the phenotype problem as an episode-level classification task, and adding temporal information, all proved highly effective.<\/jats:p>\n               <jats:p>Conclusions Automated methotrexate-induced liver toxicity phenotype discovery for patients with RA based on structured and unstructured information in the EMR shows accurate results. Our work demonstrates that adding temporal features significantly improved classification results.<\/jats:p>","DOI":"10.1136\/amiajnl-2014-002642","type":"journal-article","created":{"date-parts":[[2014,10,26]],"date-time":"2014-10-26T01:16:52Z","timestamp":1414286212000},"page":"e151-e161","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":53,"title":["Automatic identification of methotrexate-induced liver toxicity in patients with rheumatoid arthritis from the electronic medical record"],"prefix":"10.1093","volume":"22","author":[{"given":"Chen","family":"Lin","sequence":"first","affiliation":[{"name":"Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA"},{"name":"*CL, EWK and DD are co-first authors"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elizabeth W","family":"Karlson","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Immunology and Allergy, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"*CL, EWK and DD are co-first authors"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dmitriy","family":"Dligach","sequence":"additional","affiliation":[{"name":"Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"*CL, EWK and DD are co-first authors"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Monica P","family":"Ramirez","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Immunology and Allergy, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy A","family":"Miller","sequence":"additional","affiliation":[{"name":"Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huan","family":"Mo","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Natalie S","family":"Braggs","sequence":"additional","affiliation":[{"name":"Department of Medicine, Vanderbilt University, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Cagan","sequence":"additional","affiliation":[{"name":"Research Computing, Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vivian","family":"Gainer","sequence":"additional","affiliation":[{"name":"Research Computing, Partners HealthCare, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua C","family":"Denny","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA"},{"name":"Department of Medicine, Vanderbilt University, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guergana K","family":"Savova","sequence":"additional","affiliation":[{"name":"Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2014,10,24]]},"reference":[{"key":"2020110613012934600_R1","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1002\/art.1780310203","article-title":"Long-term prospective trial of low-dose methotrexate in rheumatoid arthritis","volume":"31","author":"Weinblatt","year":"1988","journal-title":"Arthritis Rheum"},{"key":"2020110613012934600_R2","first-page":"1727","article-title":"Risk factors for methotrexate-induced abnormal laboratory monitoring results in patients with rheumatoid arthritis","volume":"31","author":"Kent","year":"2004","journal-title":"J Rheumatol"},{"key":"2020110613012934600_R3","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1136\/ard.2008.101378","article-title":"Elevated liver enzyme tests among patients with rheumatoid arthritis or psoriatic arthritis treated with methotrexate and\/or leflunomide","volume":"69","author":"Curtis","year":"2010","journal-title":"Ann Rheum Dis"},{"key":"2020110613012934600_R4","first-page":"459","article-title":"Significant changes in serum AST across hepatic histological biopsy grades: prospective analysis of 3 cohorts receiving methotrexate therapy for rheumatoid arthritis","volume":"23","author":"Kremer","year":"1996","journal-title":"J Rheumatol"},{"key":"2020110613012934600_R5","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1002\/art.1780370304","article-title":"Methotrexate for rheumatoid arthritis. Suggested guidelines for monitoring liver toxicity. American College of Rheumatology","volume":"37","author":"Kremer","year":"1994","journal-title":"Arthritis Rheum"},{"key":"2020110613012934600_R6","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1136\/ard.2008.094474","article-title":"Multinational evidence-based recommendations for the use of methotrexate in rheumatic disorders with a focus on rheumatoid arthritis: integrating systematic literature research and expert opinion of a broad international panel of rheumatologists in the 3E Initiative","volume":"68","author":"Visser","year":"2009","journal-title":"Ann Rheum Dis"},{"key":"2020110613012934600_R7","first-page":"497","article-title":"A highly specific algorithm for identifying asthma cases and controls for genome-wide association studies","volume":"2009","author":"Pacheco","year":"2009","journal-title":"AMIA Annual Symp Proc"},{"key":"2020110613012934600_R8","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/1471-2415-11-32","article-title":"Cataract research using electronic health records","volume":"11","author":"Waudby","year":"2011","journal-title":"BMC Ophthalmol"},{"key":"2020110613012934600_R9","doi-asserted-by":"crossref","first-page":"79re1","DOI":"10.1126\/scitranslmed.3001807","article-title":"et al. Electronic medical records for genetic research: results of the eMERGE consortium","volume":"3","author":"Kho","year":"2011","journal-title":"Sci Transl Med"},{"key":"2020110613012934600_R10","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1136\/jamia.2010.004366","article-title":"Leveraging informatics for genetic studies: use of the electronic medical record to enable a genome-wide association study of peripheral arterial disease","volume":"17","author":"Kullo","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R11","doi-asserted-by":"crossref","first-page":"e69932","DOI":"10.1371\/journal.pone.0069932","article-title":"Automatic prediction of rheumatoid arthritis disease activity from the electronic medical records","volume":"8","author":"Lin","year":"2013","journal-title":"PLoS One"},{"key":"2020110613012934600_R12","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1136\/amiajnl-2011-000208","article-title":"Facilitating pharmacogenetic studies using electronic health records and natural-language processing: a case study of warfarin","volume":"18","author":"Xu","year":"2011","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R13","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1038\/clpt.2010.260","article-title":"The emerging role of electronic medical records in pharmacogenomics","volume":"89","author":"Wilke","year":"2011","journal-title":"Clin Pharmacol Ther"},{"key":"2020110613012934600_R14","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1002\/acr.20184","article-title":"Electronic medical records for discovery research in rheumatoid arthritis","volume":"62","author":"Liao","year":"2010","journal-title":"Arthritis Care Res"},{"key":"2020110613012934600_R15","doi-asserted-by":"crossref","first-page":"e162","DOI":"10.1136\/amiajnl-2011-000583","article-title":"et al. Portability of an algorithm to identify rheumatoid arthritis in electronic health records","volume":"19","author":"Carroll","year":"2012","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R16","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1097\/MIB.0b013e31828133fd","article-title":"Improving case definition of Crohn's disease and ulcerative colitis in electronic medical records using natural language processing: a novel informatics approach","volume":"19","author":"Ananthakrishnan","year":"2013","journal-title":"Inflamm Bowel Dis"},{"key":"2020110613012934600_R17","first-page":"1921","article-title":"Normalization of plasma 25-hydroxy vitamin D is associated with reduced risk of surgery in Crohn's disease","volume":"19","author":"Ananthakrishnan","year":"2013","journal-title":"Inflamm Bowel Dis"},{"key":"2020110613012934600_R18","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1038\/ajg.2012.471","article-title":"Similar risk of depression and anxiety following surgery or hospitalization for Crohn's disease and ulcerative colitis","volume":"108","author":"Ananthakrishnan","year":"2013","journal-title":"Am J Gastroenterol"},{"key":"2020110613012934600_R19","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1111\/apt.12195","article-title":"Psychiatric co-morbidity is associated with increased risk of surgery in Crohn's disease","volume":"37","author":"Ananthakrishnan","year":"2013","journal-title":"Aliment Pharmacol Ther"},{"key":"2020110613012934600_R20","doi-asserted-by":"crossref","first-page":"e341","DOI":"10.1136\/amiajnl-2013-001939","article-title":"et al. Normalization and standardization of electronic health records for high-throughput phenotyping: the SHARPn consortium","volume":"20","author":"Pathak","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R21","article-title":"Feature engineering and selection for rheumatoid arthritis disease activity classification using electronic medical records","author":"Lin","year":"2012"},{"key":"2020110613012934600_R22","article-title":"Maximal information coefficient for feature selection for clinical document classification (extended abstract)","author":"Lin","year":"2012"},{"key":"2020110613012934600_R23","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1136\/amiajnl-2013-001635","article-title":"A hybrid system for temporal information extraction from clinical text","volume":"20","author":"Tang","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R24","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1136\/amiajnl-2013-001627","article-title":"Eventual situations for timeline extraction from clinical reports","volume":"20","author":"Grouin","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R25","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1136\/amiajnl-2013-001622","article-title":"Comprehensive temporal information detection from clinical text: medical events, time, and TLINK identification","volume":"20","author":"Sohn","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R26","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1136\/amiajnl-2012-001607","article-title":"An end-to-end system to identify temporal relation in discharge summaries: 2012 i2b2 challenge","volume":"20","author":"Xu","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R27","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1136\/amiajnl-2013-001625","article-title":"Combining rules and machine learning for extraction of temporal expressions and events from clinical narratives","volume":"20","author":"Kovacevic","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R28","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1136\/amiajnl-2013-001619","article-title":"A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text","volume":"20","author":"Roberts","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R29","first-page":"328","article-title":"TN-TIES: a system for extracting temporal information from emergency department triage notes","author":"Irvine","year":"2008"},{"key":"2020110613012934600_R30","volume-title":"It's All Relative: Usage of Relative Temporal Expressions in Triage Notes","author":"Sullivan","year":"2008"},{"key":"2020110613012934600_R31","author":"Apache Clinical Text Analysis and Knowledge Extraction System (cTAKES)","year":"2013"},{"key":"2020110613012934600_R32","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R33","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1002\/anr.1780320202","article-title":"Liver histology in rheumatoid arthritis patients receiving long-term methotrexate therapy. A prospective study with baseline and sequential biopsy samples","volume":"32","author":"Kremer","year":"1989","journal-title":"Arthritis Rheum"},{"key":"2020110613012934600_R34","author":"Unified Medical Language System (UMLS)","year":"2013"},{"key":"2020110613012934600_R35","article-title":"Discovering Temporal Narrative Containers in Clinical Text","author":"Miller","year":"2013"},{"key":"2020110613012934600_R36","author":"THYME","year":"2013"},{"key":"2020110613012934600_R37","doi-asserted-by":"crossref","DOI":"10.1162\/tacl_a_00172","article-title":"Temporal annotations in the clinical domain","volume-title":"Trans Assoc Comput Linguist","author":"Styler","year":"2014"},{"key":"2020110613012934600_R38","first-page":"313","article-title":"Building a large annotated corpus of English: the Penn Treebank","volume":"19","author":"Marcus","year":"1993","journal-title":"Comput Linguistic"},{"key":"2020110613012934600_R39","first-page":"4","article-title":"LIBLINEAR: a library for large linear classification","volume":"9","author":"Fan","year":"2008","journal-title":"J Mach Learn Res"},{"key":"2020110613012934600_R40","doi-asserted-by":"crossref","first-page":"e243","DOI":"10.1136\/amiajnl-2013-001930","article-title":"A collaborative approach to developing an electronic health record phenotyping algorithm for drug-induced liver injury","volume":"20","author":"Overby","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613012934600_R41","doi-asserted-by":"crossref","first-page":"e1002823","DOI":"10.1371\/journal.pcbi.1002823","article-title":"Chapter 13: mining electronic health records in the genomics era","volume":"8","author":"Denny","year":"2012","journal-title":"PLoS Comput Biol"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/e1\/e151\/34146153\/amiajnl-2014-002642.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/e1\/e151\/34146153\/amiajnl-2014-002642.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T18:49:38Z","timestamp":1604688578000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/22\/e1\/e151\/701000"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,10,24]]},"references-count":41,"journal-issue":{"issue":"e1","published-online":{"date-parts":[[2014,10,24]]},"published-print":{"date-parts":[[2015,4,1]]}},"URL":"https:\/\/doi.org\/10.1136\/amiajnl-2014-002642","relation":{},"ISSN":["1527-974X","1067-5027"],"issn-type":[{"value":"1527-974X","type":"electronic"},{"value":"1067-5027","type":"print"}],"subject":[],"published-other":{"date-parts":[[2015,4]]},"published":{"date-parts":[[2014,10,24]]}}}