{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T18:26:52Z","timestamp":1778005612461,"version":"3.51.4"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2022,12,20]],"date-time":"2022-12-20T00:00:00Z","timestamp":1671494400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2047001"],"award-info":[{"award-number":["2047001"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM013335"],"award-info":[{"award-number":["R01LM013335"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"publisher","award":["R01NS116287"],"award-info":[{"award-number":["R01NS116287"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR000371"],"award-info":[{"award-number":["UL1TR000371"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["U01TR002393"],"award-info":[{"award-number":["U01TR002393"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004917","name":"Cancer Prevention and Research Institute of Texas","doi-asserted-by":"publisher","award":["RP170668"],"award-info":[{"award-number":["RP170668"]}],"id":[{"id":"10.13039\/100004917","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Reynolds and Reynolds Professorship in Clinical Informatics"},{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,2,16]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>SNOMED CT is the largest clinical terminology worldwide. Quality assurance of SNOMED CT is of utmost importance to ensure that it provides accurate domain knowledge to various SNOMED CT-based applications. In this work, we introduce a deep learning-based approach to uncover missing is-a relations in SNOMED CT.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>Our focus is to identify missing is-a relations between concept-pairs exhibiting a containment pattern (ie, the set of words of one concept being a proper subset of that of the other concept). We use hierarchically related containment concept-pairs as positive instances and hierarchically unrelated containment concept-pairs as negative instances to train a model predicting whether an is-a relation exists between 2 concepts with containment pattern. The model is a binary classifier leveraging concept name features, hierarchical features, enriched lexical attribute features, and logical definition features. We introduce a cross-validation inspired approach to identify missing is-a relations among all hierarchically unrelated containment concept-pairs.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We trained and applied our model on the Clinical finding subhierarchy of SNOMED CT (September 2019 US edition). Our model (based on the validation sets) achieved a precision of 0.8164, recall of 0.8397, and F1 score of 0.8279. Applying the model to predict actual missing is-a relations, we obtained a total of 1661 potential candidates. Domain experts performed evaluation on randomly selected 230 samples and verified that 192 (83.48%) are valid.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>The results showed that our deep learning approach is effective in uncovering missing is-a relations between containment concept-pairs in SNOMED CT.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocac248","type":"journal-article","created":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:46:35Z","timestamp":1671583595000},"page":"475-484","source":"Crossref","is-referenced-by-count":8,"title":["A deep learning approach to identify missing<i>is-a<\/i>relations in SNOMED CT"],"prefix":"10.1093","volume":"30","author":[{"given":"Rashmie","family":"Abeysinghe","sequence":"first","affiliation":[{"name":"Department of Neurology, McGovern Medical School, The University of Texas Health Science Center at Houston , Houston, Texas, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fengbo","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston , Houston, Texas, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elmer V","family":"Bernstam","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston , Houston, Texas, USA"},{"name":"Division of General Internal Medicine, McGovern Medical School, The University of Texas Health Science Center at Houston , Houston, Texas, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jay","family":"Shi","sequence":"additional","affiliation":[{"name":"Intermountain Healthcare , Denver, Colorado, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Olivier","family":"Bodenreider","sequence":"additional","affiliation":[{"name":"National Library of Medicine, National Institutes of Health , Bethesda, Maryland, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5549-8780","authenticated-orcid":false,"given":"Licong","family":"Cui","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston , Houston, Texas, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,12,20]]},"reference":[{"issue":"01","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1055\/s-0038-1638585","article-title":"Biomedical ontologies in action: role in knowledge management, data integration and decision support","volume":"17","author":"Bodenreider","year":"2008","journal-title":"Yearb Med Inform"},{"issue":"4","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1093\/jamia\/ocw175","article-title":"Mining non-lattice subgraphs for detecting missing hierarchical relations and concepts in SNOMED CT","volume":"24","author":"Cui","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1055\/s-0038-1667077","article-title":"Recent developments in clinical terminologies\u2014SNOMED CT, LOINC, and RxNorm","volume":"27","author":"Bodenreider","year":"2018","journal-title":"Yearb Med Inform"},{"key":"2023021611062238400_","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-30370-3","volume-title":"Principles of Health Interoperability: SNOMED CT, HL7 and FHIR","author":"Benson","year":"2016"},{"key":"2023021611062238400_","year":"2020"},{"key":"2023021611062238400_","year":"2020"},{"issue":"3","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.jbi.2009.03.003","article-title":"A review of auditing methods applied to the content of controlled biomedical terminologies","volume":"42","author":"Zhu","year":"2009","journal-title":"J Biomed Inform"},{"issue":"3","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"628","DOI":"10.1136\/amiajnl-2014-003173","article-title":"A tribal abstraction network for SNOMED CT target hierarchies without attribute relationships","volume":"22","author":"Ochs","year":"2015","journal-title":"J Am Med Inform Assoc"},{"issue":"Pt 2","key":"2023021611062238400_","first-page":"1070","article-title":"Using the abstraction network in complement to description logics for quality assurance in biomedical terminologies\u2014a case study in SNOMED CT","volume":"160","author":"Wei","year":"2010","journal-title":"Stud Health Technol Inform"},{"issue":"3","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/amiajnl-2014-003151","article-title":"Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies","volume":"22","author":"Ochs","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2023021611062238400_","first-page":"1071","article-title":"Scalability of abstraction-network-based quality assurance to large SNOMED hierarchies","volume":"2013","author":"Ochs","year":"2013","journal-title":"AMIA Annu Symp Proc"},{"key":"2023021611062238400_","author":"Bodenreider","year":"2016"},{"key":"2023021611062238400_","first-page":"476","article-title":"Algorithmic detection of inconsistent modeling among SNOMED CT concepts by combining lexical and structural indicators","volume":"2015","author":"Agrawal","year":"2015","journal-title":"IEEE Int Conf Bioinformatics Biomed"},{"key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.jbi.2017.12.010","article-title":"Auditing SNOMED CT hierarchical relations based on lexical features of concepts in non-lattice subgraphs","volume":"78","author":"Cui","year":"2018","journal-title":"J Biomed Inform"},{"key":"2023021611062238400_","first-page":"1392","article-title":"A lexical-based approach for exhaustive detection of missing hierarchical IS-A relations in SNOMED CT","volume":"2020","author":"Zheng","year":"2020","journal-title":"AMIA Annu Symp Proc"},{"key":"2023021611062238400_","first-page":"808","article-title":"Exploring deep learning-based approaches for predicting concept names in SNOMED CT","volume":"2018","author":"Zheng","year":"2018","journal-title":"IEEE Int Conf Bioinformatics Biomed"},{"key":"2023021611062238400_","first-page":"750","article-title":"Using convolutional neural networks to support insertion of new concepts into SNOMED CT","volume":"2018","author":"Liu","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"key":"2023021611062238400_","first-page":"972","article-title":"Training a convolutional neural network with terminology summarization data improves SNOMED CT enrichment","volume":"2019","author":"Zheng","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2023021611062238400_","first-page":"1129","article-title":"Transfer learning from BERT to support insertion of new concepts into SNOMED CT","volume":"2019","author":"Liu","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2023021611062238400_","first-page":"378","article-title":"Validating auto-suggested changes for SNOMED CT in non-lattice subgraphs using relational machine learning","volume":"264","author":"Sun","year":"2019","journal-title":"Stud Health Technol Inform"},{"key":"2023021611062238400_","author":"Liu","year":"2018"},{"key":"2023021611062238400_","year":"2021"},{"key":"2023021611062238400_","year":"2022"},{"key":"2023021611062238400_","first-page":"1188","author":"Le","year":"2014"},{"key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.aiopen.2021.01.001","article-title":"Graph neural networks: a review of methods and applications","volume":"1","author":"Zhou","year":"2020","journal-title":"AI Open"},{"issue":"1","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TNNLS.2020.2978386","article-title":"A comprehensive survey on graph neural networks","volume":"32","author":"Wu","year":"2021","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"2023021611062238400_","first-page":"982","article-title":"Leveraging non-lattice subgraphs to audit hierarchical relations in NCI thesaurus","volume":"2019","author":"Abeysinghe","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2023021611062238400_","year":"2021"},{"key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.jbi.2017.08.013","article-title":"Integrating cancer diagnosis terminologies based on logical definitions of SNOMED CT concepts","volume":"74","author":"Nikiema","year":"2017","journal-title":"J Biomed Inform"},{"key":"2023021611062238400_","first-page":"167","article-title":"Benchmark and best practices for biomedical knowledge graph embeddings","volume":"2020","author":"Chang","year":"2020","journal-title":"Proc Conf Assoc Comput Linguist Meet"},{"key":"2023021611062238400_","first-page":"739","author":"Zheng"},{"key":"2023021611062238400_","first-page":"2787","article-title":"Translating embeddings for modeling multi-relational data","volume":"26","author":"Bordes","year":"2013","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023021611062238400_","author":"Wang","year":"2019"},{"issue":"5","key":"2023021611062238400_","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/0895-4356(93)90018-V","article-title":"Bias, prevalence and kappa","volume":"46","author":"Byrt","year":"1993","journal-title":"J Clin Epidemiol"},{"key":"2023021611062238400_","year":"2022"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/3\/475\/49198809\/ocac248.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/3\/475\/49198809\/ocac248.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T12:42:00Z","timestamp":1678970520000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/3\/475\/6947996"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,20]]},"references-count":35,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,12,20]]},"published-print":{"date-parts":[[2023,2,16]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocac248","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,3,1]]},"published":{"date-parts":[[2022,12,20]]}}}