{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T12:41:24Z","timestamp":1768826484366,"version":"3.49.0"},"reference-count":133,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2020,8,7]],"date-time":"2020-08-07T00:00:00Z","timestamp":1596758400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"University of Alabama School of Medicine Informatics Institute and by the Center for Clinical and Translational Sciences","award":["UL1TR001417"],"award-info":[{"award-number":["UL1TR001417"]}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"name":"University of Florida Clinical and Translational Science Institute"},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR001427"],"award-info":[{"award-number":["UL1TR001427"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["R01AG064529"],"award-info":[{"award-number":["R01AG064529"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["R21AG061431"],"award-info":[{"award-number":["R21AG061431"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>The study sought to describe the literature related to the development of methods for auditing the Unified Medical Language System (UMLS), with particular attention to identifying errors and inconsistencies of attributes of the concepts in the UMLS Metathesaurus.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>We applied the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) approach by searching\u00a0the MEDLINE database and Google Scholar for studies referencing the UMLS and any of several terms related to auditing, error detection, and quality assurance. A qualitative analysis and summarization of articles that met inclusion criteria were performed.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Eighty-three studies were reviewed in detail. We first categorized techniques based on various aspects including concepts, concept names, and synonymy (n\u2009=\u200937), semantic type assignments (n\u2009=\u200936), hierarchical relationships (n\u2009=\u200924), lateral relationships (n\u2009=\u200912), ontology enrichment (n\u2009=\u20098), and ontology alignment (n\u2009=\u200918). We also categorized the methods according to their level of automation (ie, automated systematic, automated heuristic, or manual) and the type of knowledge used (ie, intrinsic or extrinsic knowledge).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>This study is a comprehensive review of the published methods for auditing the various conceptual aspects of the UMLS. Categorizing the auditing techniques according to the various aspects will enable the curators of the UMLS as well as researchers comprehensive easy access to this wealth of knowledge (eg, for auditing lateral relationships in the UMLS). We also reviewed ontology enrichment and alignment techniques due to their critical use of and impact on the UMLS.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocaa108","type":"journal-article","created":{"date-parts":[[2020,5,14]],"date-time":"2020-05-14T11:36:04Z","timestamp":1589456164000},"page":"1625-1638","source":"Crossref","is-referenced-by-count":13,"title":["A review of auditing techniques for the Unified Medical Language System"],"prefix":"10.1093","volume":"27","author":[{"given":"Ling","family":"Zheng","sequence":"first","affiliation":[{"name":"Department of Computer Science and Software Engineering, Monmouth University, West Long Branch, New Jersey, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhe","family":"He","sequence":"additional","affiliation":[{"name":"School of Information, Florida State University, Tallahassee, Florida, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Duo","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Business, Stockton University, Galloway, New Jersey, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vipina","family":"Keloth","sequence":"additional","affiliation":[{"name":"Department of Computer Science, New Jersey Institute of Technology, Newark, New Jersey, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jung-Wei","family":"Fan","sequence":"additional","affiliation":[{"name":"Division of Digital Health Sciences, Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luke","family":"Lindemann","sequence":"additional","affiliation":[{"name":"Center for Biomedical Data Science, Yale School of Medicine, New Haven, Connecticut, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinxin","family":"Zhu","sequence":"additional","affiliation":[{"name":"Center for Biomedical Data Science, Yale School of Medicine, New Haven, Connecticut, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4101-1622","authenticated-orcid":false,"given":"James J","family":"Cimino","sequence":"additional","affiliation":[{"name":"Informatics Institute, University of Alabama at Birmingham, Birmingham, Alabama, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yehoshua","family":"Perl","sequence":"additional","affiliation":[{"name":"Department of Computer Science, New Jersey Institute of Technology, Newark, New Jersey, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,8,7]]},"reference":[{"issue":"90001","key":"2020110613095165000_ocaa108-B1","doi-asserted-by":"crossref","first-page":"267D","DOI":"10.1093\/nar\/gkh061","article-title":"The Unified Medical Language System (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2020110613095165000_ocaa108-B2","first-page":"815","article-title":"Beyond synonymy: exploiting the UMLS semantics in mapping vocabularies","author":"Bodenreider","year":"1998","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B3","first-page":"78","article-title":"Assessing and enhancing the value of the UMLS Knowledge Sources","author":"Humphreys","year":"1991","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"issue":"4","key":"2020110613095165000_ocaa108-B4","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1055\/s-0038-1634945","article-title":"The Unified Medical Language System","volume":"32","author":"Lindberg","year":"1993","journal-title":"Methods Inf Med"},{"key":"2020110613095165000_ocaa108-B5","first-page":"475","article-title":"Building the unified medical language system","author":"Humphreys","year":"1989","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"issue":"1","key":"2020110613095165000_ocaa108-B6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1136\/jamia.1998.0050001","article-title":"The Unified Medical Language System: an informatics research collaboration","volume":"5","author":"Humphreys","year":"1998","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613095165000_ocaa108-B7","first-page":"503","article-title":"The UMLS semantic network","author":"McCray","year":"1989","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"key":"2020110613095165000_ocaa108-B8","first-page":"126","article-title":"The scope and structure of the first version of the UMLS semantic network","author":"McCray","year":"1990","journal-title":"Proc Annu Symp Comput Appl Med Care"},{"issue":"1","key":"2020110613095165000_ocaa108-B9","first-page":"80","article-title":"An upper-level ontology for the biomedical domain","volume":"4","author":"McCray","year":"2003","journal-title":"Int J Genomics"},{"issue":"Pt 1","key":"2020110613095165000_ocaa108-B10","first-page":"327","article-title":"Aligning knowledge sources in the UMLS: methods, quantitative results, and applications","volume":"107","author":"Bodenreider","year":"2004","journal-title":"Stud Health Technol Inform"},{"issue":"1","key":"2020110613095165000_ocaa108-B11","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1136\/jamia.1998.0050041","article-title":"Auditing the Unified Medical Language System with semantic methods","volume":"5","author":"Cimino","year":"1998","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110613095165000_ocaa108-B12","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1197\/jamia.M2202","article-title":"Analysis of a study of the users, uses, and future agenda of the UMLS","volume":"14","author":"Chen","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110613095165000_ocaa108-B13","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1016\/j.jbi.2009.04.006","article-title":"Special issue on auditing of terminologies","volume":"42","author":"Geller","year":"2009","journal-title":"J Biomed Inform"},{"issue":"3","key":"2020110613095165000_ocaa108-B14","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.jbi.2009.03.003","article-title":"A review of auditing methods applied to the content of controlled biomedical terminologies","volume":"42","author":"Zhu","year":"2009","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbi.2018.02.010","article-title":"Assessing the practice of biomedical ontology evaluation: Gaps and opportunities","volume":"80","author":"Amith","year":"2018","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B16","first-page":"251","volume-title":"International Conference on Knowledge Engineering and Knowledge Management","author":"Maedche","year":"2002"},{"issue":"e2","key":"2020110613095165000_ocaa108-B17","doi-asserted-by":"crossref","first-page":"e185","DOI":"10.1136\/amiajnl-2013-002227","article-title":"Auditing the multiply-related concepts within the UMLS","volume":"21","author":"Mougin","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110613095165000_ocaa108-B18","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1016\/j.jbi.2009.03.008","article-title":"Analyzing polysemous concepts from a clinical perspective: application to auditing concept categorization in the UMLS","volume":"42","author":"Mougin","year":"2009","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B19","first-page":"825","author":"Wei","year":"2012"},{"issue":"2","key":"2020110613095165000_ocaa108-B20","doi-asserted-by":"crossref","first-page":"158","DOI":"10.3414\/ME14-01-0104","article-title":"Quality assurance of UMLS semantic type assignments using SNOMED CT hierarchies","volume":"55","author":"Gu","year":"2016","journal-title":"Methods Inf Med"},{"issue":"2","key":"2020110613095165000_ocaa108-B21","doi-asserted-by":"crossref","first-page":"e181","DOI":"10.5210\/ojphi.v6i2.5412","article-title":"Sculpting the UMLS refined semantic network","volume":"6","author":"He","year":"2014","journal-title":"Online J Public Health Inform"},{"issue":"1","key":"2020110613095165000_ocaa108-B22","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.jbi.2012.09.007","article-title":"Rule-based support system for multiple UMLS semantic type assignments","volume":"46","author":"Geller","year":"2013","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B23","first-page":"48","article-title":"Categorizing the relationships between structurally congruent concepts from pairs of terminologies for semantic harmonization","author":"He","year":"2014","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"issue":"1","key":"2020110613095165000_ocaa108-B24","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.artmed.2015.03.002","article-title":"A comparative analysis of the density of the SNOMED CT conceptual content for semantic harmonization","volume":"64","author":"He","year":"2015","journal-title":"Artif Intell Med"},{"key":"2020110613095165000_ocaa108-B25","first-page":"618","article-title":"Topological-pattern-based recommendation of UMLS concepts for National Cancer Institute thesaurus","author":"He","year":"2016","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B26","first-page":"713","author":"Gu","year":"2012"},{"key":"2020110613095165000_ocaa108-B27","first-page":"120","article-title":"Battling Scylla and Charybdis: the search for redundancy and ambiguity in the 2001 UMLS metathesaurus","author":"Cimino","year":"2001","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B28","first-page":"354","article-title":"Discovering missed synonymy in a large concept-oriented Metathesaurus","author":"Hole","year":"2000","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B29","first-page":"339","article-title":"Piecewise synonyms for enhanced UMLS source terminology integration","author":"Huang","year":"2007","journal-title":"Proc AMIA Symp"},{"issue":"2","key":"2020110613095165000_ocaa108-B30","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.artmed.2008.11.008","article-title":"Using WordNet synonym substitution to enhance UMLS source integration","volume":"46","author":"Huang","year":"2009","journal-title":"Artif Intell Med"},{"key":"2020110613095165000_ocaa108-B31","first-page":"61","article-title":"Evaluation of the UMLS as a terminology and knowledge resource for biomedical informatics","author":"Bodenreider","year":"2002","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B32","first-page":"57","article-title":"Circular hierarchical relationships in the UMLS: etiology, diagnosis, treatment, complications and prevention","author":"Bodenreider","year":"2001","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B33","first-page":"550","article-title":"Approaches to eliminating cycles in the UMLS Metathesaurus: naive vs. formal","author":"Mougin","year":"2005","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B34","first-page":"101","article-title":"Strength in numbers: exploring redundancy in hierarchical relations across biomedical terminologies","author":"Bodenreider","year":"2003","journal-title":"Proc AMIA Symp"},{"issue":"1","key":"2020110613095165000_ocaa108-B35","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1136\/jamia.1998.0050076","article-title":"Evaluation of the Unified Medical Language System as a medical knowledge source","volume":"5","author":"Bodenreider","year":"1998","journal-title":"J Am Med Inform Assoc"},{"issue":"1\u20133","key":"2020110613095165000_ocaa108-B36","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/S1386-5056(02)00051-5","article-title":"Assessing the consistency of a biomedical terminology through lexical knowledge","volume":"67","author":"Bodenreider","year":"2002","journal-title":"Int J Med Inform"},{"issue":"1","key":"2020110613095165000_ocaa108-B37","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.jbi.2008.06.001","article-title":"Structural group auditing of a UMLS semantic type's extent","volume":"42","author":"Chen","year":"2009","journal-title":"J Biomed Inform"},{"issue":"2-3","key":"2020110613095165000_ocaa108-B38","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1016\/S1386-5056(01)00201-5","article-title":"Medical knowledge reengineering\u2014converting major portions of the UMLS into a terminological knowledge base","volume":"64","author":"Schulz","year":"2001","journal-title":"Int J Med Inform"},{"issue":"1","key":"2020110613095165000_ocaa108-B39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0169-023X(02)00153-2","article-title":"Semantic refinement and error correction in large terminological knowledge bases","volume":"45","author":"Geller","year":"2003","journal-title":"Data Knowl Eng"},{"key":"2020110613095165000_ocaa108-B40","first-page":"82","article-title":"Modeling the UMLS using an OODB","author":"Gu","year":"1999","journal-title":"Proc AMIA Symp"},{"issue":"1","key":"2020110613095165000_ocaa108-B41","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1136\/jamia.2000.0070066","article-title":"Representing the UMLS as an object-oriented database: modeling issues and advantages","volume":"7","author":"Gu","year":"2000","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"2020110613095165000_ocaa108-B42","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.jbi.2003.11.001","article-title":"Consistency across the hierarchies of the UMLS Semantic Network and Metathesaurus","volume":"36","author":"Cimino","year":"2003","journal-title":"J Biomed Inform"},{"issue":"1","key":"2020110613095165000_ocaa108-B43","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.artmed.2004.02.002","article-title":"Auditing concept categorizations in the UMLS","volume":"31","author":"Gu","year":"2004","journal-title":"Artif Intell Med"},{"key":"2020110613095165000_ocaa108-B44","first-page":"294","article-title":"Evaluation of a UMLS auditing process of semantic type assignments","author":"Gu","year":"2007","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B45","first-page":"310","article-title":"Using the metaschema to audit UMLS classification errors","author":"Gu","year":"2002","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B46","first-page":"612","article-title":"Auditing the UMLS for redundant classifications","author":"Peng","year":"2002","journal-title":"AMIA Annu Symp Proc"},{"issue":"1","key":"2020110613095165000_ocaa108-B47","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1197\/jamia.M2604","article-title":"Modeling multi-typed structurally viewed chemicals with the UMLS Refined Semantic Network","volume":"16","author":"Chen","year":"2009","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613095165000_ocaa108-B48","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1186\/1471-2105-8-264","article-title":"Using contextual and lexical features to restructure and validate the classification of biomedical concepts","volume":"8","author":"Fan","year":"2007","journal-title":"BMC Bioinformatics"},{"key":"2020110613095165000_ocaa108-B49","doi-asserted-by":"crossref","first-page":"b2535","DOI":"10.1136\/bmj.b2535","article-title":"Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement","volume":"339","author":"Moher","year":"2009","journal-title":"BMJ"},{"key":"2020110613095165000_ocaa108-B50","first-page":"662","article-title":"SNOMED clinical terms: overview of the development process and project status","author":"Stearns","year":"2001","journal-title":"Proc AMIA Symp"},{"issue":"1","key":"2020110613095165000_ocaa108-B51","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene Ontology: tool for the unification of biology","volume":"25","author":"Ashburner","year":"2000","journal-title":"Nat Genet"},{"key":"2020110613095165000_ocaa108-B52","first-page":"321","article-title":"Auditing SNOMED Integration into the UMLS for duplicate concepts","author":"Huang","year":"2010","journal-title":"Proc AMIA Symp"},{"issue":"6","key":"2020110613095165000_ocaa108-B53","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/j.jbi.2003.11.002","article-title":"Exploring semantic groups through visual approaches","volume":"36","author":"Bodenreider","year":"2003","journal-title":"J Biomed Inform"},{"issue":"Pt 1","key":"2020110613095165000_ocaa108-B54","first-page":"216","article-title":"Aggregating UMLS semantic types for reducing conceptual complexity","volume":"84","author":"McCray","year":"2001","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B55","first-page":"1927","author":"Erdogan","year":"2010"},{"issue":"Pt 1","key":"2020110613095165000_ocaa108-B56","first-page":"749","article-title":"Exploiting UMLS semantics for checking semantic consistency among UMLS concepts","volume":"160","author":"Erdogan","year":"2010","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B57","first-page":"393","article-title":"A study of abbreviations in the UMLS","author":"Liu","year":"2001","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B58","doi-asserted-by":"crossref","first-page":"7","DOI":"10.5210\/disco.v4i0.2663","article-title":"Concepts and synonymy in the UMLS metathesaurus","volume":"4","author":"Merrill","year":"2009","journal-title":"J Biomed Discov Collab"},{"key":"2020110613095165000_ocaa108-B59","first-page":"45","volume-title":"High Performance Medical Libraries: Advances in Information Management for the Virtual Era","author":"McCray","year":"1993"},{"issue":"1\/2","key":"2020110613095165000_ocaa108-B60","first-page":"193","article-title":"The representation of meaning in the UMLS","volume":"34","author":"McCray","year":"1995","journal-title":"Methods Inf Med"},{"issue":"1","key":"2020110613095165000_ocaa108-B61","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artmed.2015.03.005","article-title":"Abstraction networks for terminologies: Supporting management of big knowledge","volume":"64","author":"Halper","year":"2015","journal-title":"Artif Intell Med"},{"issue":"6","key":"2020110613095165000_ocaa108-B62","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1016\/j.jbi.2012.05.006","article-title":"A study of terminology auditors' performance for UMLS semantic type assignments","volume":"45","author":"Gu","year":"2012","journal-title":"J Biomed Inform"},{"issue":"3","key":"2020110613095165000_ocaa108-B63","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1016\/j.jbi.2009.01.006","article-title":"The Neighborhood Auditing Tool: a hybrid interface for auditing the UMLS","volume":"42","author":"Morrey","year":"2009","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B64","first-page":"529","article-title":"Auditing hierarchical cycles to locate other inconsistencies in the UMLS","author":"Halper","year":"2011","journal-title":"Proc AMIA Symp"},{"issue":"4","key":"2020110613095165000_ocaa108-B65","first-page":"3","article-title":"A relationship-centric hybrid interface for browsing and auditing the UMLS","volume":"15","author":"Ochs","year":"2011","journal-title":"J Integr Des Process Sci"},{"key":"2020110613095165000_ocaa108-B66","first-page":"234","article-title":"A metaschema of the UMLS based on a partition of its semantic network","author":"Halper","year":"2001","journal-title":"Proc AMIA Symp"},{"issue":"3","key":"2020110613095165000_ocaa108-B67","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/S1532-0464(02)00528-2","article-title":"The cohesive metaschema: a higher-level abstraction of the UMLS semantic network","volume":"35","author":"Perl","year":"2002","journal-title":"J Biomed Inform"},{"issue":"5","key":"2020110613095165000_ocaa108-B68","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1197\/jamia.M2951","article-title":"Expanding the extent of a UMLS semantic type via group neighborhood auditing","volume":"16","author":"Chen","year":"2009","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613095165000_ocaa108-B69","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.jbi.2011.08.021","article-title":"Overcoming an obstacle in expanding a UMLS semantic type extent","volume":"45","author":"Chen","year":"2012","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B70","first-page":"193","article-title":"Comparing inconsistent relationship configurations indicating UMLS errors","author":"Geller","year":"2009","journal-title":"Proc AMIA Symp"},{"issue":"1\/2","key":"2020110613095165000_ocaa108-B71","first-page":"43","article-title":"Validating UMLS semantic type assignments using SNOMED CT semantic tags","volume":"57","author":"Gu","year":"2018","journal-title":"Methods Inf Med"},{"key":"2020110613095165000_ocaa108-B72","first-page":"825","article-title":"The potential of the digital anatomist foundational model for assuring consistency in UMLS sources","author":"Mejino","year":"1998","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B73","first-page":"820","article-title":"The digital anatomist foundational model: principles for defining and structuring its concept domain","author":"Rosse","year":"1998","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B74","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/978-1-84628-885-2_4","volume-title":"Anatomy Ontologies for Bioinformatics","author":"Rosse","year":"2008"},{"issue":"3","key":"2020110613095165000_ocaa108-B75","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.artmed.2011.05.003","article-title":"Resolution of redundant semantic type assignments for organic chemicals in the UMLS","volume":"52","author":"Morrey","year":"2011","journal-title":"Artif Intell Med"},{"issue":"1","key":"2020110613095165000_ocaa108-B76","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/1758-2946-4-9","article-title":"A chemical specialty semantic network for the Unified Medical Language System","volume":"4","author":"Morrey","year":"2012","journal-title":"J Cheminform"},{"issue":"4","key":"2020110613095165000_ocaa108-B77","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1197\/jamia.M2314","article-title":"Semantic classification of biomedical concepts using distributional similarity","volume":"14","author":"Fan","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"Pt 1","key":"2020110613095165000_ocaa108-B78","first-page":"519","article-title":"Using distributional analysis to semantically classify UMLS concepts","volume":"129","author":"Fan","year":"2007","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B79","first-page":"231","article-title":"Combining contextual and lexical features to classify UMLS concepts","author":"Fan","year":"2007","journal-title":"Proc AMIA Symp"},{"issue":"17","key":"2020110613095165000_ocaa108-B80","doi-asserted-by":"crossref","first-page":"1971","DOI":"10.1093\/bioinformatics\/btn343","article-title":"Semantic reclassification of the UMLS concepts","volume":"24","author":"Fan","year":"2008","journal-title":"Bioinformatics"},{"key":"2020110613095165000_ocaa108-B81","first-page":"1262","author":"He","year":"2017"},{"key":"2020110613095165000_ocaa108-B82","first-page":"810","article-title":"An ontological analysis of the UMLS Metathesaurus","author":"Pisanelli","year":"1998","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B83","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1186\/s13040-016-0110-8","article-title":"FEDRR: fast, exhaustive detection of redundant hierarchical relations for quality improvement of large biomedical ontologies","volume":"9","author":"Xing","year":"2016","journal-title":"BioData Min"},{"key":"2020110613095165000_ocaa108-B84","first-page":"456","article-title":"COHeRE: cross-ontology hierarchical relation examination for ontology quality assurance","author":"Cui","year":"2015","journal-title":"Proc AMIA Symp"},{"issue":"3","key":"2020110613095165000_ocaa108-B85","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1016\/j.jbi.2008.08.006","article-title":"Structural group-based auditing of missing hierarchical relationships in UMLS","volume":"42","author":"Chen","year":"2009","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B86","first-page":"475","article-title":"Method for automatic management of the semantic network ambiguity in the UMLS: possible application for information retrieval on the Web","volume":"95","author":"Mary","year":"2003","journal-title":"Stud Health Technol Inform"},{"issue":"3","key":"2020110613095165000_ocaa108-B87","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1016\/j.jbi.2009.01.004","article-title":"Auditing associative relations\u00a0across two knowledge sources","volume":"42","author":"Vizenor","year":"2009","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B88","first-page":"863","article-title":"Perceiving the usefulness of the National Cancer Institute metathesaurus for enriching NCIt with topological patterns","volume":"245","author":"He","year":"2017","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B89","first-page":"389","article-title":"Preliminary analysis of difficulty of importing pattern-based concepts into the National Cancer Institute Thesaurus","volume":"228","author":"He","year":"2016","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B90","first-page":"1641","author":"He","year":"2018"},{"key":"2020110613095165000_ocaa108-B91","first-page":"644","article-title":"Leveraging horizontal density differences between ontologies to identify missing child concepts: a proof of concept","author":"Keloth","year":"2018","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B92","doi-asserted-by":"crossref","first-page":"103193","DOI":"10.1016\/j.jbi.2019.103193","article-title":"Alternative classification of identical concepts in different terminologies: different ways to view the world","volume":"94","author":"Keloth","year":"2019","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B93","first-page":"804","article-title":"Enhancing biomedical ontologies through alignment of semantic relationships: exploratory approaches","author":"Vizenor","year":"2006","journal-title":"Proc AMIA Symp"},{"issue":"12","key":"2020110613095165000_ocaa108-B94","doi-asserted-by":"crossref","first-page":"i69","DOI":"10.1093\/bioinformatics\/btp194","article-title":"Alignment of the UMLS semantic network with BioTop: methodology and assessment","volume":"25","author":"Schulz","year":"2009","journal-title":"Bioinformatics"},{"issue":"4","key":"2020110613095165000_ocaa108-B95","doi-asserted-by":"crossref","first-page":"205","DOI":"10.3233\/AO-2008-0057","article-title":"BioTop: An upper domain ontology for the life sciences","volume":"3","author":"Beisswanger","year":"2008","journal-title":"Appl Ontol"},{"key":"2020110613095165000_ocaa108-B96","first-page":"103","author":"Schulz","year":"2006"},{"issue":"3","key":"2020110613095165000_ocaa108-B97","doi-asserted-by":"crossref","first-page":"R29","DOI":"10.1186\/gb-2005-6-3-r29","article-title":"The Adult Mouse Anatomical Dictionary: a tool for annotating and integrating data","volume":"6","author":"Hayamizu","year":"2005","journal-title":"Genome Biol"},{"issue":"Pt 1","key":"2020110613095165000_ocaa108-B98","first-page":"67","article-title":"The MeSH translation maintenance system: structure, interface design, and implementation","volume":"107","author":"Nelson","year":"2004","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B99","author":"WHO Collaborating Centre for Drug Statistics Methodology"},{"issue":"1\u20132","key":"2020110613095165000_ocaa108-B100","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/0169-2607(94)90020-5","article-title":"The GALEN project","volume":"45","author":"Rector","year":"1994","journal-title":"Comput Methods Prog Biomed"},{"key":"2020110613095165000_ocaa108-B101","first-page":"61","article-title":"Of mice and men: aligning mouse and human anatomies","author":"Bodenreider","year":"2005","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B102","author":"Winnenburg","year":"2013"},{"key":"2020110613095165000_ocaa108-B103","first-page":"864","article-title":"Alignment of multiple ontologies of anatomy: Deriving indirect mappings from direct mappings to a reference","author":"Zhang","year":"2005","journal-title":"Proc AMIA Symp"},{"key":"2020110613095165000_ocaa108-B104","first-page":"153","author":"Zhang","year":"2006"},{"key":"2020110613095165000_ocaa108-B105","first-page":"201","author":"Zhang","year":"2006"},{"key":"2020110613095165000_ocaa108-B106","first-page":"139","author":"Zhang","year":"2007"},{"issue":"3","key":"2020110613095165000_ocaa108-B107","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.artmed.2006.12.002","article-title":"Comparing 2 approaches for aligning representations of anatomy","volume":"39","author":"Zhang","year":"2007","journal-title":"Artif Intell Med"},{"key":"2020110613095165000_ocaa108-B108","first-page":"102","author":"Zhang","year":"2004"},{"issue":"2","key":"2020110613095165000_ocaa108-B109","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/jswis.2007040101","article-title":"Experience in aligning anatomical ontologies","volume":"3","author":"Zhang","year":"2007","journal-title":"Int J Semant Web Inf Syst"},{"issue":"Pt 1","key":"2020110613095165000_ocaa108-B110","first-page":"822","article-title":"Lessons learned from cross-validating alignments between large anatomical ontologies","volume":"129","author":"Zhang","year":"2007","journal-title":"Stud Health Technol Inform"},{"key":"2020110613095165000_ocaa108-B111","first-page":"51","article-title":"Issues in mapping LOINC laboratory tests to SNOMED CT","author":"Bodenreider","year":"2008","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B112","author":"Jim\u00e9nez-Ruiz","year":"2012"},{"key":"2020110613095165000_ocaa108-B113","doi-asserted-by":"crossref","first-page":"S2","DOI":"10.1186\/2041-1480-2-S1-S2","article-title":"Logic-based assessment of the compatibility of UMLS ontology sources","volume":"2 (Suppl 1","author":"Jim\u00e9nez-Ruiz","year":"2011","journal-title":"J Biomed Semantics"},{"key":"2020110613095165000_ocaa108-B114","author":"Jimenez-Ruiz","year":"2010"},{"key":"2020110613095165000_ocaa108-B115","first-page":"697","article-title":"Shortest Path Edit Distance for Enhancing UMLS Integration and Audit","author":"Rudniy","year":"2010","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B116","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.jbi.2018.09.006","article-title":"Quality assurance of biomedical terminologies and ontologies","volume":"86","author":"Geller","year":"2018","journal-title":"J Biomed Inform"},{"issue":"4\/5","key":"2020110613095165000_ocaa108-B117","first-page":"394","article-title":"Desiderata for controlled medical vocabularies in the twenty-first century","volume":"37","author":"Cimino","year":"1998","journal-title":"Methods Inf Med"},{"key":"2020110613095165000_ocaa108-B118","author":"Agrawal"},{"issue":"4","key":"2020110613095165000_ocaa108-B119","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1136\/jamia.1999.0060283","article-title":"Benefits of an object-oriented database representation for controlled medical terminologies","volume":"6","author":"Gu","year":"1999","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"2020110613095165000_ocaa108-B120","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1197\/jamia.M2036","article-title":"Auditing as part of the terminology design life cycle","volume":"13","author":"Min","year":"2006","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110613095165000_ocaa108-B121","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1016\/j.jbi.2006.12.003","article-title":"Structural methodologies for auditing SNOMED","volume":"40","author":"Wang","year":"2007","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B122","first-page":"750","article-title":"Using convolutional neural networks to support insertion of new concepts into SNOMED CT","author":"Liu","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B123","first-page":"1186","article-title":"Identifying similar non-lattice subgraphs in gene ontology based on structural isomorphism and semantic similarity of concept labels","author":"Abeysinghe","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B124","first-page":"1129","article-title":"Transfer learning from BERT to support insertion of new concepts into SNOMED CT","author":"Liu","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B125","first-page":"972","article-title":"Training a convolutional neural network with terminology summarization data improves SNOMED CT enrichment","author":"Zheng","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"issue":"4","key":"2020110613095165000_ocaa108-B126","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1093\/jamia\/ocw175","article-title":"Mining non-lattice subgraphs for detecting missing hierarchical relations and concepts in SNOMED CT","volume":"24","author":"Cui","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613095165000_ocaa108-B127","first-page":"476","article-title":"Algorithmic detection of inconsistent modeling among SNOMED CT concepts by combining lexical and structural indicators","author":"Agrawal","year":"2015","journal-title":"Proceedings (IEEE Int Conf Bioinformatics Biomed)"},{"key":"2020110613095165000_ocaa108-B128","first-page":"17","article-title":"Identifying Inconsistencies in SNOMED CT problem lists using structural indicators","author":"Agrawal","year":"2013","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613095165000_ocaa108-B129","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.jbi.2013.11.003","article-title":"Contrasting lexical similarity and formal definitions in SNOMED CT: consistency and implications","volume":"47","author":"Agrawal","year":"2014","journal-title":"J Biomed Inform"},{"key":"2020110613095165000_ocaa108-B130","first-page":"754","article-title":"MaPLE: A MapReduce pipeline for lattice-based evaluation and its application to SNOMED CT","author":"Zhang","year":"2014"},{"issue":"4","key":"2020110613095165000_ocaa108-B131","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2768830","article-title":"Biomedical ontology quality assurance using a big data approach","volume":"10","author":"Cui","year":"2016","journal-title":"ACM Trans Knowl Discov Data"},{"key":"2020110613095165000_ocaa108-B132","first-page":"46","article-title":"Mining relation reversals in the evolution of SNOMED CT using MapReduce","author":"Tao","year":"2015","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"2020110613095165000_ocaa108-B133","first-page":"216","article-title":"NEO: systematic non-lattice embedding of ontologies for comparing the subsumption relationship in SNOMED CT and in FMA using MapReduce","author":"Zhu","year":"2015","journal-title":"AMIA Jt Summits Transl Sci Proc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/10\/1625\/34153270\/ocaa108.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/10\/1625\/34153270\/ocaa108.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,1]],"date-time":"2023-10-01T02:42:30Z","timestamp":1696128150000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/10\/1625\/5882052"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,7]]},"references-count":133,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2020,8,7]]},"published-print":{"date-parts":[[2020,10,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa108","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,10]]},"published":{"date-parts":[[2020,8,7]]}}}