{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T07:08:33Z","timestamp":1763017713808,"version":"3.37.3"},"reference-count":27,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2017,2,21]],"date-time":"2017-02-21T00:00:00Z","timestamp":1487635200000},"content-version":"vor","delay-in-days":2,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["ACI-1626364"],"award-info":[{"award-number":["ACI-1626364"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Objective: Quality assurance of large ontological systems such as SNOMED CT is an indispensable part of the terminology management lifecycle. We introduce a hybrid structural-lexical method for scalable and systematic discovery of missing hierarchical relations and concepts in SNOMED CT.<\/jats:p>\n               <jats:p>Material and Methods: All non-lattice subgraphs (the structural part) in SNOMED CT are exhaustively extracted using a scalable MapReduce algorithm. Four lexical patterns (the lexical part) are identified among the extracted non-lattice subgraphs. Non-lattice subgraphs exhibiting such lexical patterns are often indicative of missing hierarchical relations or concepts. Each lexical pattern is associated with a potential specific type of error.<\/jats:p>\n               <jats:p>Results: Applying the structural-lexical method to SNOMED CT (September 2015 US edition), we found 6801 non-lattice subgraphs that matched these lexical patterns, of which 2046 were amenable to visual inspection. We evaluated a random sample of 100 small subgraphs, of which 59 were reviewed in detail by domain experts. All the subgraphs reviewed contained errors confirmed by the experts. The most frequent type of error was missing is-a relations due to incomplete or inconsistent modeling of the concepts.<\/jats:p>\n               <jats:p>Conclusions: Our hybrid structural-lexical method is innovative and proved effective not only in detecting errors in SNOMED CT, but also in suggesting remediation for these errors.<\/jats:p>","DOI":"10.1093\/jamia\/ocw175","type":"journal-article","created":{"date-parts":[[2016,12,3]],"date-time":"2016-12-03T20:05:30Z","timestamp":1480795530000},"page":"788-798","source":"Crossref","is-referenced-by-count":42,"title":["Mining non-lattice subgraphs for detecting missing hierarchical relations and concepts in SNOMED CT"],"prefix":"10.1093","volume":"24","author":[{"given":"Licong","family":"Cui","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Kentucky, Lexington, KY, USA"},{"name":"Institute for Biomedical Informatics, University of Kentucky"}]},{"given":"Wei","family":"Zhu","sequence":"additional","affiliation":[{"name":"Institute for Biomedical Informatics, University of Kentucky"}]},{"given":"Shiqiang","family":"Tao","sequence":"additional","affiliation":[{"name":"Institute for Biomedical Informatics, University of Kentucky"},{"name":"Division of Biomedical Informatics, College of Medicine, University of Kentucky"}]},{"given":"James T","family":"Case","sequence":"additional","affiliation":[{"name":"National Library of Medicine, Bethesda, MD, USA"}]},{"given":"Olivier","family":"Bodenreider","sequence":"additional","affiliation":[{"name":"National Library of Medicine, Bethesda, MD, USA"}]},{"given":"Guo-Qiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute for Biomedical Informatics, University of Kentucky"},{"name":"Division of Biomedical Informatics, College of Medicine, University of Kentucky"}]}],"member":"286","published-online":{"date-parts":[[2017,2,19]]},"reference":[{"key":"2020110612362557600_ocw175-B1","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1016\/j.jbi.2009.04.006","article-title":"Special issue on auditing of terminologies","volume":"42","author":"Geller","year":"2009","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B2","first-page":"67","article-title":"Biomedical ontologies in action: role in knowledge management, data integration and decision support","author":"Bodenreider","year":"2008","journal-title":"Yearb Med Inform."},{"key":"2020110612362557600_ocw175-B3","doi-asserted-by":"crossref","first-page":"e11","DOI":"10.1136\/amiajnl-2013-001636","article-title":"Literature review of SNOMED CT use","volume":"21","author":"Lee","year":"2014","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612362557600_ocw175-B4","first-page":"1497","article-title":"Metrics for assessing the quality of value sets in clinical quality measures","author":"Winnenburg","year":"2013","journal-title":"AMIA Annu Symp Proc."},{"key":"2020110612362557600_ocw175-B5","unstructured":"Health Information Technology for Economic and Clinical Health (HITECH) Act. 2009. http:\/\/www.healthit.gov\/sites\/default\/files\/hitech_act_excerpt_from_arra_with_index.pdf. Accessed April 6, 2015."},{"key":"2020110612362557600_ocw175-B6","unstructured":"ONC Stage 2 Meaningful Use Final Rule. 2012. http:\/\/www.gpo.gov\/fdsys\/pkg\/FR-2012-09-04\/pdf\/2012-20982.pdf. Accessed April 6, 2015."},{"key":"2020110612362557600_ocw175-B7","unstructured":"SNOMED CT Starter Guide. 2014 http:\/\/ihtsdo.org\/fileadmin\/user_upload\/doc\/download\/doc_StarterGuide_Current-en-US_INT_20141202.pdf. Accessed April 6, 2015."},{"key":"2020110612362557600_ocw175-B8","first-page":"513","article-title":"Designing an introspective, controlled medical vocabulary","volume-title":"Proceedings of the Thirteenth Annual SCAMC","author":"Cimino","year":"1989"},{"key":"2020110612362557600_ocw175-B9","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1136\/jamia.1998.0050041","article-title":"Auditing the unified medical language system with semantic methods","volume":"5","author":"Cimino","year":"1998","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612362557600_ocw175-B10","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.jbi.2009.03.003","article-title":"A review of auditing methods applied to the content of controlled biomedical terminologies","volume":"42","author":"Zhu","year":"2009","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B11","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/S1386-5056(02)00051-5","article-title":"Assessing the consistency of a biomedical terminology through lexical knowledge","volume":"67","author":"Bodenreider","year":"2002","journal-title":"Int J Med Inform."},{"key":"2020110612362557600_ocw175-B12","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.jbi.2013.11.003","article-title":"Contrasting lexical similarity and formal definitions in SNOMED CT: Consistency and implications","volume":"47","author":"Agrawal","year":"2014","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B13","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1197\/jamia.M2541","article-title":"Auditing the semantic completeness of SNOMED CT using formal concept analysis","volume":"16","author":"Jiang","year":"2009","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612362557600_ocw175-B14","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.jbi.2011.10.002","article-title":"Lexically suggest, logically define: quality assurance of the use of qualifiers and expected results of post-coordination in SNOMED CT","volume":"45","author":"Rector","year":"2012","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B15","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1016\/j.jbi.2006.12.003","article-title":"Structural methodologies for auditing SNOMED","volume":"40","author":"Wang","year":"2007","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B16","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.jbi.2011.08.013","article-title":"Abstraction of complex concepts with a refined partial-area taxonomy of SNOMED","volume":"45","author":"Wang","year":"2012","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbi.2011.08.016","article-title":"Auditing complex concepts of SNOMED using a refined hierarchical abstraction network","volume":"45","author":"Wang","year":"2012","journal-title":"J Biomed Inform."},{"key":"2020110612362557600_ocw175-B18","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/amiajnl-2014-003151","article-title":"Scalable quality assurance for large SNOMED CT hierarchies using subject-based subtaxonomies","volume":"22","author":"Ochs","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612362557600_ocw175-B19","doi-asserted-by":"crossref","first-page":"628","DOI":"10.1136\/amiajnl-2014-003173","article-title":"A tribal abstraction network for SNOMED CT target hierarchies without attribute relationships","volume":"22","author":"Ochs","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612362557600_ocw175-B20","first-page":"273","article-title":"Using SPARQL to Test for Lattices: application to quality assurance in biomedical ontologies","author":"Zhang","year":"2010","journal-title":"The Semantic Web-ISWC."},{"key":"2020110612362557600_ocw175-B21","first-page":"922","article-title":"Large-scale, exhaustive lattice-based structural auditing of SNOMED CT","author":"Zhang","year":"2010","journal-title":"AMIA Annu Symp Proc."},{"key":"2020110612362557600_ocw175-B22","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1055\/s-0038-1634577","article-title":"Issues in the structuring and acquisition of an ontology for medical language understanding","volume":"34","author":"Zweigenbaum","year":"1995","journal-title":"Methods Inform Med."},{"key":"2020110612362557600_ocw175-B23","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-59830-2","volume-title":"Formal Concept Analysis","author":"Ganter","year":"1999"},{"key":"2020110612362557600_ocw175-B24","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1007\/978-3-540-73681-3_16","article-title":"Faster concept analysis","volume-title":"Conceptual Structures: Knowledge Architectures for Smart Applications","author":"Troy","year":"2007"},{"key":"2020110612362557600_ocw175-B25","first-page":"754","article-title":"MaPLE: A MapReduce Pipeline for Lattice-based Evaluation and Its Application to SNOMED CT","author":"Zhang","year":"2014","journal-title":"IEEE BigData."},{"key":"2020110612362557600_ocw175-B26","first-page":"41","article-title":"Biomedical ontology quality assurance using a big data approach","volume":"10","author":"Cui","year":"2016","journal-title":"ACM Transact Knowledge Discov Data."},{"key":"2020110612362557600_ocw175-B27","unstructured":"The CORE Problem List Subset of SNOMED CT. 2016. https:\/\/www.nlm.nih.gov\/research\/umls\/Snomed\/core_subset.html. Accessed October 3, 2016."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/4\/788\/34148599\/ocw175.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/4\/788\/34148599\/ocw175.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T17:43:59Z","timestamp":1604684639000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/4\/788\/3038204"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,19]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2017,2,19]]},"published-print":{"date-parts":[[2017,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw175","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"type":"print","value":"1067-5027"},{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2017,7]]},"published":{"date-parts":[[2017,2,19]]}}}