{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T16:08:21Z","timestamp":1784390901188,"version":"3.55.0"},"reference-count":27,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T00:00:00Z","timestamp":1692835200000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000092","name":"NLM","doi-asserted-by":"publisher","award":["R01LM009886"],"award-info":[{"award-number":["R01LM009886"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,11,17]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Outcomes are important clinical study information. Despite progress in automated extraction of PICO (Population, Intervention, Comparison, and Outcome) entities from PubMed, rarely are these entities encoded by standard terminology to achieve semantic interoperability. This study aims to evaluate the suitability of the Unified Medical Language System (UMLS) and SNOMED-CT in encoding outcome concepts in randomized controlled trial (RCT) abstracts.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We iteratively developed and validated an outcome annotation guideline and manually annotated clinically significant outcome entities in the Results and Conclusions sections of 500 randomly selected RCT abstracts on PubMed. The extracted outcomes were fully, partially, or not mapped to the UMLS via MetaMap based on established heuristics. Manual UMLS browser search was performed for select unmapped outcome entities to further differentiate between UMLS and MetaMap errors.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Only 44% of 2617 outcome concepts were fully covered in the UMLS, among which 67% were complex concepts that required the combination of 2 or more UMLS concepts to represent them. SNOMED-CT was present as a source in 61% of the fully mapped outcomes.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Domains such as Metabolism and Nutrition, and Infections and Infectious Diseases need expanded outcome concept coverage in the UMLS and MetaMap. Future work is warranted to similarly assess the terminology coverage for P, I, C entities.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Computational representation of clinical outcomes is important for clinical evidence extraction and appraisal and yet faces challenges from the inherent complexity and lack of coverage of these concepts in UMLS and SNOMED-CT, as demonstrated in this study.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad161","type":"journal-article","created":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T15:29:36Z","timestamp":1692890976000},"page":"1895-1903","source":"Crossref","is-referenced-by-count":12,"title":["The suitability of UMLS and SNOMED-CT for encoding outcome concepts"],"prefix":"10.1093","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0290-4965","authenticated-orcid":false,"given":"Abigail","family":"Newbury","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York City, NY 10032, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1975-1272","authenticated-orcid":false,"given":"Hao","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York City, NY 10032, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4318-5987","authenticated-orcid":false,"given":"Betina","family":"Idnay","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York City, NY 10032, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9624-0214","authenticated-orcid":false,"given":"Chunhua","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York City, NY 10032, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,8,23]]},"reference":[{"key":"2023111709553238900_ocad161-B1","doi-asserted-by":"crossref","first-page":"A12","DOI":"10.7326\/ACPJC-1995-123-3-A12","article-title":"The well-built clinical question: a key to evidence-based decisions","volume":"123","author":"Richardson","year":"1995","journal-title":"ACP J Club"},{"issue":"3","key":"2023111709553238900_ocad161-B2","doi-asserted-by":"crossref","first-page":"273","DOI":"10.2217\/cer.13.17","article-title":"Modernizing the systematic review process to inform comparative effectiveness: tools and methods","volume":"2","author":"Wallace","year":"2013","journal-title":"J Comp Eff Res"},{"key":"2023111709553238900_ocad161-B3","doi-asserted-by":"publisher","first-page":"197","DOI":"10.18653\/v1\/P18-1019","author":"Nye","year":"2018"},{"key":"2023111709553238900_ocad161-B4","author":"Zlabinger","year":"2018"},{"issue":"1","key":"2023111709553238900_ocad161-B5","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1186\/s13326-022-00271-7","article-title":"An annotated corpus of clinical trial publications supporting schema-based relational information extraction","volume":"13","author":"Sanchez-Graillet","year":"2022","journal-title":"J Biomed Semantics"},{"key":"2023111709553238900_ocad161-B6","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1186\/1472-6947-10-29","article-title":"Combining classifiers for robust PICO element detection","volume":"10","author":"Boudin","year":"2010","journal-title":"BMC Med Inform Decis Mak"},{"key":"2023111709553238900_ocad161-B7","doi-asserted-by":"publisher","first-page":"67","DOI":"10.18653\/v1\/W18-2308","author":"Jin","year":"2018"},{"issue":"13","key":"2023111709553238900_ocad161-B8","doi-asserted-by":"crossref","first-page":"e1434","DOI":"10.1212\/WNL.0000000000008872","article-title":"Randomized controlled trial of N-acetylcysteine therapy for RYR1-related myopathies","volume":"94","author":"Todd","year":"2020","journal-title":"Neurology"},{"key":"2023111709553238900_ocad161-B9","author":"About PCORI","year":"2021"},{"key":"2023111709553238900_ocad161-B10","author":"COMET Initiative | Home"},{"key":"2023111709553238900_ocad161-B11","first-page":"58"},{"key":"2023111709553238900_ocad161-B12","author":"UMLS \u2013 Metathesaurus Vocabulary FAQ"},{"issue":"8","key":"2023111709553238900_ocad161-B13","doi-asserted-by":"crossref","first-page":"e20675","DOI":"10.2196\/20675","article-title":"The Unified Medical Language system at 30 years and how it is used and published: systematic review and content analysis","volume":"9","author":"Jing","year":"2021","journal-title":"JMIR Med Inform"},{"issue":"9","key":"2023111709553238900_ocad161-B14","doi-asserted-by":"crossref","first-page":"2017","DOI":"10.1093\/jamia\/ocab084","article-title":"The use of SNOMED CT, 2013-2020: a literature review","volume":"28","author":"Chang","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2023111709553238900_ocad161-B15","first-page":"435","article-title":"An evaluation of the UMLS in representing corpus derived clinical concepts","volume":"2011","author":"Friedlin","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"key":"2023111709553238900_ocad161-B16","volume-title":"E-Utilities Quick Start","author":"Sayers","year":"2018"},{"key":"2023111709553238900_ocad161-B17","doi-asserted-by":"crossref","first-page":"104235","DOI":"10.1016\/j.jbi.2022.104235","article-title":"Ontology-based categorization of clinical studies by their conditions","volume":"135","author":"Liu","year":"2022","journal-title":"J Biomed Inform"},{"issue":"4","key":"2023111709553238900_ocad161-B18","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/j.jbi.2010.02.005","article-title":"UMLS content views appropriate for NLP processing of the biomedical literature vs. clinical text","volume":"43","author":"Demner-Fushman","year":"2010","journal-title":"J Biomed Inform"},{"key":"2023111709553238900_ocad161-B19","author":"Rios"},{"key":"2023111709553238900_ocad161-B20","author":"Word Sense Disambiguation Server (WSD Server)"},{"key":"2023111709553238900_ocad161-B21","author":"pattern-en"},{"key":"2023111709553238900_ocad161-B22","author":"String \u2013 Common String Operations"},{"key":"2023111709553238900_ocad161-B23","author":"UMLS API Home"},{"key":"2023111709553238900_ocad161-B24","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med"},{"issue":"3","key":"2023111709553238900_ocad161-B25","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1162\/089120105774321109","article-title":"Evaluating discourse and dialogue coding schemes","volume":"31","author":"Craggs","year":"2005","journal-title":"Comput Linguist"},{"key":"2023111709553238900_ocad161-B26","first-page":"392","article-title":"Data-driven modeling of randomized controlled trial outcomes","volume":"294","author":"Chen","year":"2022","journal-title":"Stud Health Technol Inform"},{"issue":"3","key":"2023111709553238900_ocad161-B27","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1136\/jamia.2009.002733","article-title":"An overview of MetaMap: historical perspective and recent advances","volume":"17","author":"Aronson","year":"2010","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/12\/1895\/53477613\/ocad161.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/12\/1895\/53477613\/ocad161.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T13:28:51Z","timestamp":1700227731000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/12\/1895\/7249289"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,23]]},"references-count":27,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,8,23]]},"published-print":{"date-parts":[[2023,11,17]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad161","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,12,1]]},"published":{"date-parts":[[2023,8,23]]}}}