{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T10:14:41Z","timestamp":1755598481877,"version":"3.37.3"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,8,13]],"date-time":"2022-08-13T00:00:00Z","timestamp":1660348800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,8,13]],"date-time":"2022-08-13T00:00:00Z","timestamp":1660348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2047001"],"award-info":[{"award-number":["2047001"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"U.S. National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM013335"],"award-info":[{"award-number":["R01LM013335"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"publisher","award":["R01NS116287"],"award-info":[{"award-number":["R01NS116287"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Biomed Semant"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>The Vaccine Ontology (VO) is a biomedical ontology that standardizes vaccine annotation. Errors in VO will affect a multitude of applications that it is being used in. Quality assurance of VO is imperative to ensure that it provides accurate domain knowledge to these downstream tasks. Manual review to identify and fix quality issues (such as missing hierarchical is-a relations) is challenging given the complexity of the ontology. Automated approaches are highly desirable to facilitate the quality assurance of VO.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We developed an automated lexical approach that identifies potentially missing <jats:italic>is-a<\/jats:italic> relations in VO. First, we construct two types of VO concept-pairs: (1) linked; and (2) unlinked. Each concept-pair further derives an Acquired Term Pair (ATP) based on their lexical features. If the same ATP is obtained by a linked concept-pair and an unlinked concept-pair, this is considered to indicate a potentially missing <jats:italic>is-a<\/jats:italic> relation between the unlinked pair of concepts.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Applying this approach on the 1.1.192 version of VO, we were able to identify 232 potentially missing <jats:italic>is-a<\/jats:italic> relations. A manual review by a VO domain expert on a random sample of 70 potentially missing <jats:italic>is-a<\/jats:italic> relations revealed that 65 of the cases were valid missing <jats:italic>is-a<\/jats:italic> relations in VO (a precision of 92.86%).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The results indicate that our approach is highly effective in identifying missing <jats:italic>is-a<\/jats:italic> relation in VO.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s13326-022-00276-2","type":"journal-article","created":{"date-parts":[[2022,8,13]],"date-time":"2022-08-13T06:02:45Z","timestamp":1660370565000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Identification of missing hierarchical relations in the vaccine ontology using acquired term pairs"],"prefix":"10.1186","volume":"13","author":[{"given":"Warren","family":"Manuel","sequence":"first","affiliation":[]},{"given":"Rashmie","family":"Abeysinghe","sequence":"additional","affiliation":[]},{"given":"Yongqun","family":"He","sequence":"additional","affiliation":[]},{"given":"Cui","family":"Tao","sequence":"additional","affiliation":[]},{"given":"Licong","family":"Cui","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,8,13]]},"reference":[{"key":"276_CR1","unstructured":"Immunization Basics | CDC. https:\/\/www.cdc.gov\/vaccines\/vac-gen\/imz-basics.htm. Accessed 29 Mar 2022."},{"key":"276_CR2","unstructured":"Global Vaccine Action Plan Monitoring, Evaluation & Accountability: Secretariat Annual Report 2020. https:\/\/www.who.int\/publications\/i\/item\/global-vaccine-action-plan-monitoring-evaluation-accountability-secretariat-annual-report-2020. Accessed 29 Mar 2022."},{"key":"276_CR3","doi-asserted-by":"crossref","unstructured":"He Y, Cowell L, Diehl AD, Mobley H, Peters B, Ruttenberg A, et al.VO: vaccine ontology. In: The 1st International Conference on Biomedical Ontology (ICBO-2009).\u00a0Buffalo: ICBO;\u00a02009. p. 24\u20136.","DOI":"10.1038\/npre.2009.3553"},{"issue":"1","key":"276_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/2041-1480-3-17","volume":"3","author":"Y Lin","year":"2012","unstructured":"Lin Y, He Y. Ontology representation and analysis of vaccine formulation and administration and their effects on vaccine immune responses. J Biomed Semantic. 2012; 3(1):1\u201315.","journal-title":"J Biomed Semantic"},{"key":"276_CR5","unstructured":"Introduction to Vaccine Ontology. https:\/\/www.violinet.org\/vaccineontology\/introduction.php. Accessed 11 Apr 2022."},{"key":"276_CR6","doi-asserted-by":"crossref","unstructured":"De Matos P, Alc\u00e1ntara R, Dekker A, Ennis M, Hastings J, Haug K, et al.Chemical entities of biological interest: an update. 38 Suppl 1. 2010:D249\u2013D254.","DOI":"10.1093\/nar\/gkp886"},{"key":"276_CR7","volume-title":"Anatomy Ontologies for Bioinformatics","author":"C Rosse","year":"2008","unstructured":"Rosse C, Mejino JL. The foundational model of anatomy ontology In: Burger A, Davidson D, Baldock R, editors. Anatomy Ontologies for Bioinformatics. New York: Springer: 2008. p. 59\u2013117."},{"key":"276_CR8","volume-title":"Infectious disease informatics","author":"LG Cowell","year":"2010","unstructured":"Cowell LG, Smith B. Infectious disease ontology In: Sintchenko V, editor. Infectious disease informatics. New York: Springer: 2010. p. 373\u201395."},{"key":"276_CR9","doi-asserted-by":"publisher","first-page":"D1073","DOI":"10.1093\/nar\/gkq944","volume":"39 Suppl 1","author":"B Yang","year":"2011","unstructured":"Yang B, Sayers S, Xiang Z, He Y. Protegen: a web-based protective antigen database and analysis system. Nucleic Acids Res. 2011; 39 Suppl 1:D1073\u2013D1078.","journal-title":"Nucleic Acids Res"},{"key":"276_CR10","doi-asserted-by":"publisher","first-page":"831486","DOI":"10.1155\/2012\/831486","volume":"2012","author":"S Sayers","year":"2012","unstructured":"Sayers S, Ulysse G, Xiang Z, He Y. Vaxjo: a web-based vaccine adjuvant database and its application for analysis of vaccine adjuvants and their uses in vaccine development. J Biomed Biotechnol. 2012; 2012:831486.","journal-title":"J Biomed Biotechnol"},{"issue":"5","key":"276_CR11","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1016\/j.vaccine.2012.11.066","volume":"31","author":"R Racz","year":"2013","unstructured":"Racz R, Chung M, Xiang Z, He Y. Systematic annotation and analysis of \u201cvirmugens\u201d\u2014Virulence factors whose mutants can be used as live attenuated vaccines. Vaccine. 2013; 31(5):797\u2013805.","journal-title":"Vaccine"},{"issue":"D1","key":"276_CR12","doi-asserted-by":"publisher","first-page":"D1124","DOI":"10.1093\/nar\/gkt1133","volume":"42","author":"Y He","year":"2014","unstructured":"He Y, Racz R, Sayers S, Lin Y, Todd T, Hur J, et al.Updates on the web-based VIOLIN vaccine database and analysis system. Nucleic Acids Res. 2014; 42(D1):D1124\u2013D1132.","journal-title":"Nucleic Acids Res"},{"issue":"2","key":"276_CR13","first-page":"1","volume":"2","author":"A \u00d6zg\u00fcr","year":"2011","unstructured":"\u00d6zg\u00fcr A, Xiang Z, Radev DR, He Y. Mining of vaccine-associated IFN- \u03b3 gene interaction networks using the Vaccine Ontology. J Biomed Semantic. 2011; 2(2):1\u201316.","journal-title":"J Biomed Semantic"},{"issue":"1","key":"276_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2172-12-49","volume":"12","author":"J Hur","year":"2011","unstructured":"Hur J, Xiang Z, Feldman EL, He Y. Ontology-based Brucella vaccine literature indexing and systematic analysis of gene-vaccine association network. BMC Immunol. 2011; 12(1):1\u201318.","journal-title":"BMC Immunol"},{"key":"276_CR15","unstructured":"Noy NF, Crub\u00e9zy M, Fergerson RW, Knublauch H, Tu SW, Vendetti J, et al.Prot\u00e9g\u00e9-2000: an open-source ontology-development and knowledge-acquisition environment. In: AMIA Annu Symp Proc.\u00a0Rockville: AMIA;\u00a02003. p. 953."},{"issue":"4","key":"276_CR16","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1145\/2757001.2757003","volume":"1","author":"MA Musen","year":"2015","unstructured":"Musen MA. The prot\u00e9g\u00e9 project: a look back and a look forward. AI Matters. 2015; 1(4):4\u201312.","journal-title":"AI Matters"},{"key":"276_CR17","unstructured":"Horrocks I, Chen J, Jaehun L. Tool support for ontology design and quality assurance. In: ICBO 2020 integrated food ontology workshop (IFOW). Bozen-Bolzano: ICBO;\u00a02020."},{"key":"276_CR18","unstructured":"Gene Ontology Resource. http:\/\/geneontology.org\/stats.html. Accessed 29 Mar 2022."},{"issue":"D1","key":"276_CR19","doi-asserted-by":"publisher","first-page":"D325","DOI":"10.1093\/nar\/gkaa1113","volume":"49","author":"Consortium TGO","year":"2020","unstructured":"Consortium TGO. The Gene Ontology resource: enriching a GOld mine. Nucleic Acids Res. 2020; 49(D1):D325\u2013D334.","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"276_CR20","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/75556","volume":"25","author":"M Ashburner","year":"2000","unstructured":"Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, et al.Gene ontology: tool for the unification of biology. Nat Genet. 2000; 25(1):25\u20139.","journal-title":"Nat Genet"},{"key":"276_CR21","unstructured":"UMLS Metathesaurus - SNOMEDCT_US (SNOMED CT, US Edition) - Statistics. https:\/\/www.nlm.nih.gov\/research\/umls\/sourcereleasedocs\/current\/SNOMEDCT_US\/stats.html. Accessed 29 Mar 2022."},{"key":"276_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jbi.2018.02.010","volume":"80","author":"M Amith","year":"2018","unstructured":"Amith M, He Z, Bian J, Lossio-Ventura JA, Tao C. Assessing the practice of biomedical ontology evaluation: Gaps and opportunities. J Biomed Inform. 2018; 80:1\u201313.","journal-title":"J Biomed Inform"},{"issue":"3","key":"276_CR23","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1055\/s-0038-1634078","volume":"45","author":"JE Rogers","year":"2006","unstructured":"Rogers JE. Quality assurance of medical ontologies. Methods Inf Med. 2006; 45(3):267\u201374.","journal-title":"Methods Inf Med"},{"issue":"1","key":"276_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artmed.2015.03.005","volume":"64","author":"M Halper","year":"2015","unstructured":"Halper M, Gu H, Perl Y, Ochs C. Abstraction networks for terminologies: supporting management of \u201cbig knowledge\u201d. Artif Intell Med. 2015; 64(1):1\u201316.","journal-title":"Artif Intell Med"},{"issue":"6","key":"276_CR25","doi-asserted-by":"publisher","first-page":"676","DOI":"10.1197\/jamia.M2036","volume":"13","author":"H Min","year":"2006","unstructured":"Min H, Perl Y, Chen Y, Halper M, Geller J, Wang Y. Auditing as part of the terminology design life cycle. J Am Med Inform Assoc. 2006; 13(6):676\u201390.","journal-title":"J Am Med Inform Assoc"},{"key":"276_CR26","first-page":"384","volume":"228","author":"M Quesada-Mart\u00ednez","year":"2016","unstructured":"Quesada-Mart\u00ednez M, Fernandez-Breis J, Karlsson D. Suggesting Missing Relations in Biomedical Ontologies Based on Lexical Regularities. Stud Health Technol Inform. 2016; 228:384\u20138.","journal-title":"Stud Health Technol Inform"},{"issue":"2","key":"276_CR27","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1016\/j.jbi.2011.10.002","volume":"45","author":"A Rector","year":"2012","unstructured":"Rector A, Iannone L. Lexically suggest, logically define: Quality assurance of the use of qualifiers and expected results of post-coordination in SNOMED CT. J Biomed Inform. 2012; 45(2):199\u2013209.","journal-title":"J Biomed Inform"},{"issue":"4","key":"276_CR28","doi-asserted-by":"publisher","first-page":"788","DOI":"10.1093\/jamia\/ocw175","volume":"24","author":"L Cui","year":"2017","unstructured":"Cui L, Zhu W, Tao S, Case JT, Bodenreider O, Zhang GQ. Mining non-lattice subgraphs for detecting missing hierarchical relations and concepts in SNOMED CT. J Am Med Inform Assoc. 2017; 24(4):788\u201398.","journal-title":"J Am Med Inform Assoc"},{"key":"276_CR29","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1016\/j.jbi.2017.12.010","volume":"78","author":"L Cui","year":"2018","unstructured":"Cui L, Bodenreider O, Shi J, Zhang GQ. Auditing SNOMED CT hierarchical relations based on lexical features of concepts in non-lattice subgraphs. J Biomed Inform. 2018; 78:177\u201384.","journal-title":"J Biomed Inform"},{"key":"276_CR30","unstructured":"Abeysinghe R, Brooks MA, Talbert J, Licong C. Quality assurance of NCI Thesaurus by mining structural-lexical patterns. In: AMIA Annu Symp Proc. vol. 2017.\u00a0Rockville: American Medical Informatics Association;\u00a02017. p. 364\u201373."},{"key":"276_CR31","doi-asserted-by":"crossref","unstructured":"Zheng F, Abeysinghe R, Cui L. A hybrid method to detect missing hierarchical relations in NCI Thesaurus. In: Proceedings (IEEE Int Conf Bioinformatics Biomed).\u00a0Piscataway: IEEE;\u00a02019. p. 1948\u201353.","DOI":"10.1109\/BIBM47256.2019.8983265"},{"key":"276_CR32","doi-asserted-by":"crossref","unstructured":"Hao X, Abeysinghe R, Zheng F, Cui L. Leveraging non-lattice subgraphs for suggestion of new concepts for SNOMED CT. In: Proceedings (IEEE Int Conf Bioinformatics Biomed).\u00a0Piscataway: IEEE;\u00a02021. p. 1805\u201312.","DOI":"10.1109\/BIBM52615.2021.9669407"},{"key":"276_CR33","unstructured":"Abeysinghe R, Zheng F, Cui L. A Comparison of Exhaustive and Non-lattice-based Methods for Auditing Hierarchical Relations in Gene Ontology. In: AMIA Annu Symp Proc. vol. 2021.\u00a0Rockville: American Medical Informatics Association;\u00a02021. p. 177\u201386."},{"key":"276_CR34","doi-asserted-by":"crossref","unstructured":"Zhang GQ, Bodenreider O. Large-scale, exhaustive lattice-based structural auditing of SNOMED CT. In: AMIA Annu Symp Proc. vol. 2010.\u00a0Rockville: American Medical Informatics Association;\u00a02010. p. 922\u20136.","DOI":"10.1007\/978-3-642-15280-1_61"},{"key":"276_CR35","doi-asserted-by":"crossref","unstructured":"Abeysinghe R, Hinderer EW, Moseley HN, Cui L. Auditing subtype inconsistencies among gene ontology concepts. In: Proceedings (IEEE Int Conf Bioinformatics Biomed).\u00a0Piscataway: IEEE;\u00a02017. p. 1242\u20135.","DOI":"10.1109\/BIBM.2017.8217835"},{"key":"276_CR36","doi-asserted-by":"crossref","unstructured":"Abeysinghe R, Zheng F, Hinderer EW, Moseley HN, Cui L. A lexical approach to identifying subtype inconsistencies in biomedical terminologies. In: Proceedings (IEEE Int Conf Bioinformatics Biomed).\u00a0Piscataway: IEEE;\u00a02018. p. 1982\u201389.","DOI":"10.1109\/BIBM.2018.8621511"},{"key":"276_CR37","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.artmed.2017.07.002","volume":"80","author":"JB Lamy","year":"2017","unstructured":"Lamy JB. Owlready: Ontology-oriented programming in Python with automatic classification and high level constructs for biomedical ontologies. Artif Intell Med. 2017; 80:11\u201328.","journal-title":"Artif Intell Med"},{"issue":"D1","key":"276_CR38","doi-asserted-by":"publisher","first-page":"D136","DOI":"10.1093\/nar\/gkr1178","volume":"40","author":"S Federhen","year":"2012","unstructured":"Federhen S. The NCBI taxonomy database. Nucleic Acids Res. 2012; 40(D1):D136\u2013D143.","journal-title":"Nucleic Acids Res"},{"key":"276_CR39","unstructured":"Zhang GQ, Huang Y, Cui L. Can snomed ct changes be used as a surrogate standard for evaluating the performance of its auditing methods? In: AMIA Annu Symp Proc. vol. 2017. American Medical Informatics Association: 2017. p. 1903\u201312."}],"container-title":["Journal of Biomedical Semantics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13326-022-00276-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13326-022-00276-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13326-022-00276-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,13]],"date-time":"2022-08-13T06:17:51Z","timestamp":1660371471000},"score":1,"resource":{"primary":{"URL":"https:\/\/jbiomedsem.biomedcentral.com\/articles\/10.1186\/s13326-022-00276-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,13]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["276"],"URL":"https:\/\/doi.org\/10.1186\/s13326-022-00276-2","relation":{},"ISSN":["2041-1480"],"issn-type":[{"type":"electronic","value":"2041-1480"}],"subject":[],"published":{"date-parts":[[2022,8,13]]},"assertion":[{"value":"1 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 July 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 August 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"22"}}