{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T15:43:39Z","timestamp":1758815019750,"version":"3.37.3"},"reference-count":95,"publisher":"Springer Science and Business Media LLC","issue":"S1","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR003017"],"award-info":[{"award-number":["UL1TR003017"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Two years into the COVID-19 pandemic and with more than five million deaths worldwide, the healthcare establishment continues to struggle with every new wave of the pandemic resulting from a new coronavirus variant. Research has demonstrated that there are variations in the symptoms, and even in the order of symptom presentations, in COVID-19 patients infected by different SARS-CoV-2 variants (e.g., Alpha and Omicron). Textual data in the form of admission notes and physician notes in the Electronic Health Records (EHRs) is rich in information regarding the symptoms and their orders of presentation. Unstructured EHR data is often underutilized in research due to the lack of annotations that enable automatic extraction of useful information from the available extensive volumes of textual data.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We present the design of a COVID Interface Terminology (CIT), not just a generic COVID-19 terminology, but one serving a specific purpose of enabling automatic annotation of EHRs of COVID-19 patients. CIT was constructed by integrating existing COVID-related ontologies and mining additional fine granularity concepts from clinical notes. The iterative mining approach utilized the techniques of 'anchoring' and 'concatenation' to identify potential fine granularity concepts to be added to the CIT. We also tested the generalizability of our approach on a hold-out dataset and compared the annotation coverage to the coverage obtained for the dataset used to build the CIT.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Our experiments demonstrate that this approach results in higher annotation coverage compared to existing ontologies such as SNOMED CT and Coronavirus Infectious Disease Ontology (CIDO). The final version of CIT achieved about 20% more coverage than SNOMED CT and 50% more coverage than CIDO. In the future, the concepts mined and added into CIT could be used as training data for machine learning models for mining even more concepts into CIT and further increasing the annotation coverage.\n<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>In this paper, we demonstrated the construction of a COVID interface terminology that can be utilized for automatically annotating EHRs of COVID-19 patients. The techniques presented can identify frequently documented fine granularity concepts that are missing in other ontologies thereby increasing the annotation coverage.\n<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-023-02136-0","type":"journal-article","created":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T10:02:49Z","timestamp":1677232969000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Mining of EHR for interface terminology concepts for annotating EHRs of COVID patients"],"prefix":"10.1186","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6919-1122","authenticated-orcid":false,"given":"Vipina K.","family":"Keloth","sequence":"first","affiliation":[]},{"given":"Shuxin","family":"Zhou","sequence":"additional","affiliation":[]},{"given":"Luke","family":"Lindemann","sequence":"additional","affiliation":[]},{"given":"Ling","family":"Zheng","sequence":"additional","affiliation":[]},{"given":"Gai","family":"Elhanan","sequence":"additional","affiliation":[]},{"given":"Andrew J.","family":"Einstein","sequence":"additional","affiliation":[]},{"given":"James","family":"Geller","sequence":"additional","affiliation":[]},{"given":"Yehoshua","family":"Perl","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,2,24]]},"reference":[{"key":"2136_CR1","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1136\/amiajnl-2011-000203","volume":"18","author":"O Uzuner","year":"2011","unstructured":"Uzuner O, South BR, Shen S, DuVall SL. 2010 i2b2\/VA challenge on concepts, assertions, and relations in clinical text. J Am Med Inform Assoc. 2011;18:552\u20136.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR2","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1093\/jamia\/ocz166","volume":"27","author":"S Henry","year":"2020","unstructured":"Henry S, Buchan K, Filannino M, Stubbs A, Uzuner O. 2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records. J Am Med Inform Assoc. 2020;27:3\u201312.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR3","doi-asserted-by":"publisher","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","volume":"22","author":"B Shickel","year":"2018","unstructured":"Shickel B, Tighe PJ, Bihorac A, Rashidi P. Deep EHR: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE J Biomed Health Inform. 2018;22:1589\u2013604.","journal-title":"IEEE J Biomed Health Inform"},{"key":"2136_CR4","doi-asserted-by":"publisher","first-page":"103301","DOI":"10.1016\/j.jbi.2019.103301","volume":"100","author":"S Datta","year":"2019","unstructured":"Datta S, Bernstam EV, Roberts K. A frame semantic overview of NLP-based information extraction for cancer-related EHR notes. J Biomed Inform. 2019;100:103301.","journal-title":"J Biomed Inform"},{"key":"2136_CR5","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1136\/amiajnl-2013-001628","volume":"20","author":"W Sun","year":"2013","unstructured":"Sun W, Rumshisky A, Uzuner O. Evaluating temporal relations in clinical text: 2012 i2b2 Challenge. J Am Med Inform Assoc. 2013;20:806\u201313.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR6","doi-asserted-by":"crossref","unstructured":"Pradhan S, Elhadad N, Chapman WW, Manandhar S, Savova GK. SemEval-2014 Task 7: analysis of clinical text. *SEMEVAL2014.","DOI":"10.3115\/v1\/S14-2007"},{"key":"2136_CR7","doi-asserted-by":"publisher","first-page":"1075","DOI":"10.1007\/s40264-017-0558-6","volume":"40","author":"Y Luo","year":"2017","unstructured":"Luo Y, Thompson WK, Herr TM, Zeng Z, Berendsen MA, Jonnalagadda SR, et al. Natural language processing for ehr-based pharmacovigilance: a structured review. Drug Saf. 2017;40:1075\u201389.","journal-title":"Drug Saf"},{"key":"2136_CR8","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1136\/amiajnl-2011-000501","volume":"18","author":"L Ohno-Machado","year":"2011","unstructured":"Ohno-Machado L. Realizing the full potential of electronic health records: the role of natural language processing. J Am Med Inform Assoc. 2011;18:539.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR9","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1197\/jamia.M3028","volume":"16","author":"X Wang","year":"2009","unstructured":"Wang X, Hripcsak G, Markatou M, Friedman C. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study. J Am Med Inform Assoc. 2009;16:328\u201337.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR10","doi-asserted-by":"publisher","first-page":"e26","DOI":"10.2196\/jmir.8669","volume":"20","author":"J Chen","year":"2018","unstructured":"Chen J, Druhl E, Polepalli Ramesh B, Houston TK, Brandt CA, Zulman DM, et al. A natural language processing system that links medical terms in electronic health record notes to lay definitions: system development using physician reviews. J Med Internet Res. 2018;20:e26.","journal-title":"J Med Internet Res"},{"key":"2136_CR11","first-page":"56","volume":"2009","author":"C Jonquet","year":"2009","unstructured":"Jonquet C, Shah NH, Musen MA. The open biomedical annotator. Summit Transl Bioinform. 2009;2009:56\u201360.","journal-title":"Summit Transl Bioinform"},{"key":"2136_CR12","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1136\/jamia.2009.002733","volume":"17","author":"AR Aronson","year":"2010","unstructured":"Aronson AR, Lang FM. An overview of MetaMap: historical perspective and recent advances. J Am Med Inform Assoc. 2010;17:229\u201336.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR13","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1136\/jamia.2009.001560","volume":"17","author":"GK Savova","year":"2010","unstructured":"Savova GK, Masanz JJ, Ogren PV, Zheng J, Sohn S, Kipper-Schuler KC, et al. Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc. 2010;17:507\u201313.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR14","doi-asserted-by":"publisher","first-page":"D267","DOI":"10.1093\/nar\/gkh061","volume":"32","author":"O Bodenreider","year":"2004","unstructured":"Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004;32:D267\u201370.","journal-title":"Nucleic Acids Res"},{"key":"2136_CR15","first-page":"279","volume":"121","author":"K Donnelly","year":"2006","unstructured":"Donnelly K. SNOMED-CT: the advanced terminology and coding system for eHealth. Stud Health Technol Inform. 2006;121:279\u201390.","journal-title":"Stud Health Technol Inform"},{"key":"2136_CR16","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1038\/s41597-020-0523-6","volume":"7","author":"Y He","year":"2020","unstructured":"He Y, Yu H, Ong E, Wang Y, Liu Y, Huffman A, et al. CIDO, a community-based ontology for coronavirus disease knowledge and data integration, sharing, and analysis. Sci Data. 2020;7:181.","journal-title":"Sci Data"},{"key":"2136_CR17","doi-asserted-by":"publisher","first-page":"E262","DOI":"10.1148\/radiol.2021204522","volume":"299","author":"JP Kanne","year":"2021","unstructured":"Kanne JP, Bai H, Bernheim A, Chung M, Haramati LB, Kallmes DF, et al. COVID-19 imaging: what we know now and what remains unknown. Radiology. 2021;299:E262\u201379.","journal-title":"Radiology"},{"key":"2136_CR18","doi-asserted-by":"publisher","first-page":"142","DOI":"10.4329\/wjr.v12.i8.142","volume":"12","author":"AE Kaufman","year":"2020","unstructured":"Kaufman AE, Naidu S, Ramachandran S, Kaufman DS, Fayad ZA, Mani V. Review of radiographic findings in COVID-19. World J Radiol. 2020;12:142\u201355.","journal-title":"World J Radiol"},{"key":"2136_CR19","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1186\/s12890-020-01286-5","volume":"20","author":"LA Rousan","year":"2020","unstructured":"Rousan LA, Elobeid E, Karrar M, Khader Y. Chest x-ray findings and temporal lung changes in patients with COVID-19 pneumonia. BMC Pulm Med. 2020;20:245.","journal-title":"BMC Pulm Med"},{"key":"2136_CR20","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1016\/j.cognition.2011.11.003","volume":"122","author":"F Mathy","year":"2012","unstructured":"Mathy F, Feldman J. What\u2019s magic about magic numbers? Chunking and data compression in short-term memory. Cognition. 2012;122:346\u201362.","journal-title":"Cognition"},{"key":"2136_CR21","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1037\/h0083231","volume":"16","author":"E Tulving","year":"1962","unstructured":"Tulving E, Patkau JE. Concurrent effects of contextual constraint and word frequency on immediate recall and learning of verbal material. Can J Psychol. 1962;16:83\u201395.","journal-title":"Can J Psychol"},{"key":"2136_CR22","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1016\/S1364-6613(00)01662-4","volume":"5","author":"F Gobet","year":"2001","unstructured":"Gobet F, Lane PC, Croker S, Cheng PC, Jones G, Oliver I, et al. Chunking mechanisms in human learning. Trends Cogn Sci. 2001;5:236\u201343.","journal-title":"Trends Cogn Sci"},{"key":"2136_CR23","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1093\/jamia\/ocw177","volume":"24","author":"D Demner-Fushman","year":"2017","unstructured":"Demner-Fushman D, Rogers WJ, Aronson AR. MetaMap Lite: an evaluation of a new Java implementation of MetaMap. J Am Med Inform Assoc. 2017;24:841\u20134.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR24","unstructured":"SNOMED CT Compositional Grammer Specification and Guide. https:\/\/confluence.ihtsdotools.org\/display\/DOCSCG (accessed June 15th, 2020). 2021."},{"key":"2136_CR25","unstructured":"Spackman KA, Campbell KE. Compositional concept representation using SNOMED: towards further convergence of clinical terminologies. Proc AMIA Symp. 1998;740\u20134."},{"key":"2136_CR26","first-page":"446","volume":"235","author":"JA Minarro-Gimenez","year":"2017","unstructured":"Minarro-Gimenez JA, Martinez-Costa C, Lopez-Garcia P, Schulz S. Building SNOMED CT post-coordinated expressions from annotation groups. Stud Health Technol Inform. 2017;235:446\u201350.","journal-title":"Stud Health Technol Inform"},{"key":"2136_CR27","unstructured":"Radiopeadia. https:\/\/radiopaedia.org\/ (accessed Jun 15th, 2020). 2020."},{"key":"2136_CR28","unstructured":"COVID-19 Database. https:\/\/www.sirm.org\/category\/senza-categoria\/covid-19\/ (accessed Nov 15th, 2019). 2021."},{"key":"2136_CR29","doi-asserted-by":"crossref","unstructured":"Keloth V, Zhou S, Lindemann L, Elhanan G, Einstein A, Geller J, et al. Mining Concepts for a COVID Interface Terminology for Annotation of EHRs. In: 2020 IEEE International Conference on Big Data (Big Data). 2020;3753\u201360.","DOI":"10.1109\/BigData50022.2020.9377981"},{"key":"2136_CR30","doi-asserted-by":"crossref","unstructured":"Wang L, Foer D, MacPhaul E, Lo Y-C, Bates D, Zhou L. PASCLex: a comprehensive post-acute sequelae of COVID-19 (PASC) symptom lexicon derived from electronic health record clinical notes. J Biomed Inform. 2021.","DOI":"10.1101\/2021.07.29.21261260"},{"key":"2136_CR31","first-page":"1639","volume":"2011","author":"L Zhou","year":"2011","unstructured":"Zhou L, Plasek JM, Mahoney LM, Karipineni N, Chang F, Yan X, et al. Using medical text extraction, reasoning and mapping system (MTERMS) to process medication information in outpatient clinical notes. AMIA Annu Symp Proc. 2011;2011:1639\u201348.","journal-title":"AMIA Annu Symp Proc"},{"key":"2136_CR32","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1017\/S1351324900000061","volume":"1","author":"C Friedman","year":"1995","unstructured":"Friedman C, Hripcsak G, DuMouchel W, Johnson SB, Clayton PD. Natural language processing in an operational clinical information system. Nat Lang Eng. 1995;1:83\u2013108.","journal-title":"Nat Lang Eng"},{"key":"2136_CR33","unstructured":"Health Information Text Extraction (HITEx). https:\/\/www.i2b2.org\/software\/projects\/hitex\/hitex_manual.html (accessed Jan 10th, 2020). 2006."},{"key":"2136_CR34","unstructured":"Soldaini L. QuickUMLS: a fast, unsupervised approach for medical concept extraction. 2016."},{"key":"2136_CR35","doi-asserted-by":"publisher","first-page":"W170","DOI":"10.1093\/nar\/gkp440","volume":"37","author":"NF Noy","year":"2009","unstructured":"Noy NF, Shah NH, Whetzel PL, Dai B, Dorf M, Griffith N, et al. BioPortal: ontologies and integrated data resources at the click of a mouse. Nucleic Acids Res. 2009;37:W170\u20133.","journal-title":"Nucleic Acids Res"},{"key":"2136_CR36","doi-asserted-by":"publisher","first-page":"W587","DOI":"10.1093\/nar\/gkz389","volume":"47","author":"CH Wei","year":"2019","unstructured":"Wei CH, Allot A, Leaman R, Lu Z. PubTator central: automated concept annotation for biomedical full text articles. Nucleic Acids Res. 2019;47:W587\u201393.","journal-title":"Nucleic Acids Res"},{"key":"2136_CR37","doi-asserted-by":"publisher","first-page":"73729","DOI":"10.1109\/ACCESS.2019.2920708","volume":"7","author":"D Kim","year":"2019","unstructured":"Kim D, Lee J, So CH, Jeon H, Jeong M, Choi Y, et al. A neural named entity recognition and multi-type normalization tool for biomedical text mining. IEEE Access. 2019;7:73729\u201340.","journal-title":"IEEE Access"},{"key":"2136_CR38","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1093\/jamia\/ocx132","volume":"25","author":"E Soysal","year":"2018","unstructured":"Soysal E, Wang J, Jiang M, Wu Y, Pakhomov S, Liu H, et al. CLAMP - a toolkit for efficiently building customized clinical natural language processing pipelines. J Am Med Inform Assoc. 2018;25:331\u20136.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR39","first-page":"27","volume":"136","author":"AS Kanter","year":"2008","unstructured":"Kanter AS, Wang AY, Masarie FE, Naeymi-Rad FF, Safran C. Interface terminologies: bridging the gap between theory and reality for Africa. Stud Health Technol Inform. 2008;136:27\u201332.","journal-title":"Stud Health Technol Inform"},{"key":"2136_CR40","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1504\/IJWS.2012.052531","volume":"1","author":"L Zemmouchi-Ghomari","year":"2012","unstructured":"Zemmouchi-Ghomari L, Ghomari AR. Ontology versus terminology, from the perspective of ontologists. Int J Web Sci. 2012;1:315\u201331.","journal-title":"Int J Web Sci"},{"key":"2136_CR41","doi-asserted-by":"publisher","first-page":"375","DOI":"10.3233\/AO-2012-0119","volume":"7","author":"N Grabar","year":"2012","unstructured":"Grabar N, Hamon T, Bodenreider O. Ontologies and terminologies: continuum or dichotomy? Appl Ontol. 2012;7:375\u201386.","journal-title":"Appl Ontol"},{"key":"2136_CR42","first-page":"132","volume":"8","author":"S Schulz","year":"2013","unstructured":"Schulz S, Jansen L. Formal ontologies in biomedical knowledge representation. Yearb Med Inform. 2013;8:132\u201346.","journal-title":"Yearb Med Inform"},{"key":"2136_CR43","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1197\/jamia.M1957","volume":"13","author":"ST Rosenbloom","year":"2006","unstructured":"Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. Interface terminologies: facilitating direct entry of clinical data into electronic health record systems. J Am Med Inform Assoc. 2006;13:277\u201388.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR44","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1197\/jamia.M2694","volume":"16","author":"ST Rosenbloom","year":"2009","unstructured":"Rosenbloom ST, Brown SH, Froehling D, Bauer BA, Wahner-Roedler DL, Gregg WM, et al. Using SNOMED CT to represent two interface terminologies. J Am Med Inform Assoc. 2009;16:81\u20138.","journal-title":"J Am Med Inform Assoc"},{"issue":"Suppl 1","key":"2136_CR45","doi-asserted-by":"publisher","first-page":"S3","DOI":"10.1186\/1472-6947-8-S1-S3","volume":"8","author":"G Wade","year":"2008","unstructured":"Wade G, Rosenbloom ST. Experiences mapping a legacy interface terminology to SNOMED CT. BMC Med Inform Decis Mak. 2008;8(Suppl 1):S3.","journal-title":"BMC Med Inform Decis Mak"},{"key":"2136_CR46","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1016\/j.jbi.2009.03.004","volume":"42","author":"G Wade","year":"2009","unstructured":"Wade G, Rosenbloom ST. The impact of SNOMED CT revisions on a mapped interface terminology: terminology development and implementation issues. J Biomed Inform. 2009;42:490\u20133.","journal-title":"J Biomed Inform"},{"key":"2136_CR47","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1197\/jamia.M2506","volume":"15","author":"ST Rosenbloom","year":"2008","unstructured":"Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. A model for evaluating interface terminologies. J Am Med Inform Assoc. 2008;15:65\u201376.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR48","doi-asserted-by":"publisher","first-page":"e178","DOI":"10.1136\/amiajnl-2012-001384","volume":"20","author":"ST Rosenbloom","year":"2013","unstructured":"Rosenbloom ST, Miller RA, Adams P, Madani S, Khan N, Shultz EK. Implementing an interface terminology for structured clinical documentation. J Am Med Inform Assoc. 2013;20:e178\u201382.","journal-title":"J Am Med Inform Assoc"},{"key":"2136_CR49","unstructured":"BioPortal webpage of CIDO. https:\/\/bioportal.bioontology.org\/ontologies\/CIDO (accessed Dec 20th, 2020). 2008."},{"key":"2136_CR50","doi-asserted-by":"publisher","first-page":"1251","DOI":"10.1038\/nbt1346","volume":"25","author":"B Smith","year":"2007","unstructured":"Smith B, Ashburner M, Rosse C, Bard J, Bug W, Ceusters W, et al. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol. 2007;25:1251\u20135.","journal-title":"Nat Biotechnol"},{"key":"2136_CR51","doi-asserted-by":"publisher","first-page":"D344","DOI":"10.1093\/nar\/gkm791","volume":"36","author":"K Degtyarenko","year":"2008","unstructured":"Degtyarenko K, de Matos P, Ennis M, Hastings J, Zbinden M, McNaught A, et al. ChEBI: a database and ontology for chemical entities of biological interest. Nucleic Acids Res. 2008;36:D344\u201350.","journal-title":"Nucleic Acids Res"},{"key":"2136_CR52","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.artmed.2017.05.002","volume":"79","author":"G Elhanan","year":"2017","unstructured":"Elhanan G, Ochs C, Mejino JLV, Liu H, Mungall CJ, Perl Y. From SNOMED CT to Uberon: transferability of evaluation methodology between similarly structured ontologies. Artif Intell Med. 2017;79:9\u201314.","journal-title":"Artif Intell Med"},{"issue":"3","key":"2136_CR53","doi-asserted-by":"publisher","first-page":"1642001","DOI":"10.1142\/S0219720016420014","volume":"14","author":"C Ochs","year":"2016","unstructured":"Ochs C, Perl Y, Halper M, Geller J, Lomax J. Quality assurance of the gene ontology using abstraction networks. J Bioinform Comput Biol. 2016;14(3):1642001.","journal-title":"J Bioinform Comput Biol"},{"key":"2136_CR54","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/j.jbi.2017.07.013","volume":"73","author":"L Zheng","year":"2017","unstructured":"Zheng L, Yumak H, Chen L, Ochs C, Geller J, Kapusnik-Uner J, et al. Quality assurance of chemical ingredient classification for the national drug file-reference terminology. J Biomed Inform. 2017;73:30\u201342.","journal-title":"J Biomed Inform"},{"key":"2136_CR55","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1016\/j.ajhg.2008.09.017","volume":"83","author":"PN Robinson","year":"2008","unstructured":"Robinson PN, Kohler S, Bauer S, Seelow D, Horn D, Mundlos S. The Human phenotype ontology: a tool for annotating and analyzing human hereditary disease. Am J Hum Genet. 2008;83:610\u20135.","journal-title":"Am J Hum Genet"},{"key":"2136_CR56","doi-asserted-by":"publisher","first-page":"103861","DOI":"10.1016\/j.jbi.2021.103861","volume":"120","author":"L Zheng","year":"2021","unstructured":"Zheng L, Perl Y, He YO, Ochs C, Geller J, Liu H, et al. Visual comprehension and orientation into the COVID-19 CIDO ontology. J Biomed Inform. 2021;120:103861.","journal-title":"J Biomed Inform"},{"key":"2136_CR57","unstructured":"COVID-19 Ontology. http:\/\/bioportal.bioontology.org\/ontologies\/COVID-19 (accessed Sept 30, 2020). 2020"},{"issue":"24","key":"2136_CR58","doi-asserted-by":"publisher","first-page":"5703","DOI":"10.1093\/bioinformatics\/btaa1057","volume":"36","author":"A Sargsyan","year":"2020","unstructured":"Sargsyan A, Kodamullil AT, Baksi S, Darms J, Madan S, Gebel S, et al. The COVID-19 ontology. Bioinformatics. 2020;36(24):5703\u20135.","journal-title":"Bioinformatics"},{"key":"2136_CR59","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/s13326-021-00245-1","volume":"12","author":"S Babcock","year":"2021","unstructured":"Babcock S, Beverley J, Cowell LG, Smith B. The infectious disease ontology in the age of COVID-19. J Biomed Semant. 2021;12:13.","journal-title":"J Biomed Semant"},{"key":"2136_CR60","unstructured":"WHO COVID-19 rapid version CRF semantic data model. https:\/\/bioportal.bioontology.org\/ontologies\/COVIDCRFRAPID (accessed Sept 30, 2020). 2020."},{"key":"2136_CR61","unstructured":"Infectious Disease Ontology. https:\/\/bioportal.bioontology.org\/ontologies\/IDO (accessed Sept 30, 2020). 2020."},{"key":"2136_CR62","unstructured":"Virus Infectious Disease Ontology. https:\/\/bioportal.bioontology.org\/ontologies\/VIDO (accessed Sept 30, 2020). 2020."},{"key":"2136_CR63","doi-asserted-by":"publisher","first-page":"e18606","DOI":"10.2196\/18606","volume":"6","author":"S de Lusignan","year":"2020","unstructured":"de Lusignan S, Lopez Bernal J, Zambon M, Akinyemi O, Amirthalingam G, Andrews N, et al. Emergence of a novel coronavirus (COVID-19): protocol for extending surveillance used by the royal college of general practitioners research and surveillance centre and public health England. JMIR Public Health Surveill. 2020;6:e18606.","journal-title":"JMIR Public Health Surveill"},{"key":"2136_CR64","doi-asserted-by":"crossref","unstructured":"Dutta B, DeBellis M. CODO: an ontology for collection and analysis of Covid-19 data. ArXiv. 2020;abs\/2009.01210.","DOI":"10.5220\/0010112500760085"},{"key":"2136_CR65","unstructured":"ACT COVID Ontology v3.0. https:\/\/github.com\/shyamvis\/ACT-COVID-Ontology\/tree\/master\/ontology (accessed Sept 30, 2020). 2020."},{"key":"2136_CR66","unstructured":"WHO. International Classification of Diseases. http:\/\/www.who.int\/classifications\/icd\/en\/ (accessed Sept 30, 2020). 2020."},{"key":"2136_CR67","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1373\/49.4.624","volume":"49","author":"CJ McDonald","year":"2003","unstructured":"McDonald CJ, Huff SM, Suico JG, Hill G, Leavelle D, Aller R, et al. LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clin Chem. 2003;49:624\u201333.","journal-title":"Clin Chem"},{"key":"2136_CR68","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1136\/neurintsurg-2014-011156","volume":"7","author":"JA Hirsch","year":"2015","unstructured":"Hirsch JA, Leslie-Mazwi TM, Nicola GN, Barr RM, Bello JA, Donovan WD, et al. Current procedural terminology; a primer. J Neurointerv Surg. 2015;7:309\u201312.","journal-title":"J Neurointerv Surg"},{"key":"2136_CR69","unstructured":"National Drug Code Database Background Information. https:\/\/www.fda.gov\/drugs\/development-approval-process-drugs\/national-drug-code-database-background-information (accessed Sept 30, 2020). 2017."},{"key":"2136_CR70","unstructured":"Wang LL, Lo K, Chandrasekhar Y, Reas R, Yang J, Eide D, et al. CORD-19: the COVID-19 open research dataset. ArXiv. 2020."},{"key":"2136_CR71","unstructured":"Global literature on coronavirus disease. https:\/\/search.bvsalud.org\/global-literature-on-novel-coronavirus-2019-ncov\/ (accessed Jun 15th, 2021). 2021."},{"issue":"118","key":"2136_CR72","doi-asserted-by":"publisher","first-page":"103790","DOI":"10.1016\/j.jbi.2021.103790","volume":"1","author":"Y Sun","year":"2021","unstructured":"Sun Y, Butler A, Stewart LA, Liu H, Yuan C, Southard CT, Kim JH, Weng C. Building an OMOP common data model-compliant annotated corpus for COVID-19 clinical trials. J Biomed Inform. 2021;1(118):103790.","journal-title":"J Biomed Inform"},{"key":"2136_CR73","doi-asserted-by":"crossref","unstructured":"Lee J, Kim JH, Liu C, Hripcsak G, Ta C, Weng C. COHD-COVID: Columbia Open Health Data for COVID-19 Research. medRxiv. 2020.","DOI":"10.1101\/2020.11.17.20232983"},{"key":"2136_CR74","doi-asserted-by":"crossref","unstructured":"Lybarger K, Ostendorf M, Thompson M, Yetisgen M. Extracting COVID-19 diagnoses and symptoms from clinical text: a new annotated corpus and neural event extraction framework. ArXiv. 2021.","DOI":"10.1016\/j.jbi.2021.103761"},{"key":"2136_CR75","unstructured":"Centers for Disease Control and Prevention. https:\/\/www.cdc.gov\/coronavirus\/2019-ncov\/need-extra-precautions\/people-with-medical-conditions.http (accessed Jun 1st, 2021). 2020."},{"key":"2136_CR76","doi-asserted-by":"crossref","unstructured":"Daintith J. Kleene star. A dictionary of computing. 6th edN. Oxford University Press; 2008.","DOI":"10.1093\/acref\/9780199234004.001.0001"},{"key":"2136_CR77","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"ML McHugh","year":"2012","unstructured":"McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22:276\u201382.","journal-title":"Biochem Med (Zagreb)"},{"issue":"2","key":"2136_CR78","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1037\/h0043158","volume":"63","author":"GA Miller","year":"1956","unstructured":"Miller GA. The magical number seven plus or minus two: some limits on our capacity for processing information. Psychol Rev. 1956;63(2):81\u201397.","journal-title":"Psychol Rev"},{"key":"2136_CR79","doi-asserted-by":"crossref","unstructured":"Sung M, Jeon H, Lee J, Kang J. Biomedical entity representations with synonym marginalization. arXiv 2020. arXiv preprint arXiv:2005.00239. 2021.","DOI":"10.18653\/v1\/2020.acl-main.335"},{"issue":"21","key":"2136_CR80","doi-asserted-by":"publisher","first-page":"3856","DOI":"10.1093\/bioinformatics\/btab474","volume":"37","author":"Z Miftahutdinov","year":"2021","unstructured":"Miftahutdinov Z, Kadurin A, Kudrin R, Tutubalina E. Medical concept normalization in clinical trials with drug and disease representation learning. Bioinformatics. 2021;37(21):3856\u201364.","journal-title":"Bioinformatics"},{"key":"2136_CR81","unstructured":"McCray A. The UMLS semantic network proceedings. In: Symposium on Computer Applications in Medical Care. 1989;503\u2013507. PMCID: PMC2245676."},{"key":"2136_CR82","unstructured":"Peng Y, Halper MH, Perl Y, Geller J. Auditing the UMLS for redundant classifications. In: Proceedings of AMIA Symposium. 2002; 612\u20136. PMID: 12463896; PMCID: PMC2244162."},{"issue":"3","key":"2136_CR83","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1016\/j.jbi.2008.08.006","volume":"42","author":"Y Chen","year":"2009","unstructured":"Chen Y, Gu HH, Perl Y, Geller J. Structural group-based auditing of missing hierarchical relationships in UMLS. J Biomed Inform. 2009;42(3):452\u201367.","journal-title":"J Biomed Inform"},{"key":"2136_CR84","first-page":"294","volume":"2007","author":"HH Gu","year":"2007","unstructured":"Gu HH, Hripcsak G, Chen Y, Morrey CP, Elhanan G, Cimino JJ, Geller J, Perl Y. Evaluation of a UMLS auditing process of semantic type assignments. AMIA Ann Symp Proc. 2007;2007:294.","journal-title":"AMIA Ann Symp Proc"},{"issue":"5","key":"2136_CR85","doi-asserted-by":"publisher","first-page":"746","DOI":"10.1197\/jamia.M2951","volume":"16","author":"Y Chen","year":"2009","unstructured":"Chen Y, Gu H, Perl Y, Halper M, Xu J. Expanding the extent of a UMLS semantic type via group neighborhood auditing. J Am Med Inform Assoc. 2009;16(5):746\u201357.","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2136_CR86","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/j.jbi.2012.09.007","volume":"46","author":"J Geller","year":"2013","unstructured":"Geller J, He Z, Perl Y, Morrey CP, Xu J. Rule-based support system for multiple UMLS semantic type assignments. J Biomed Inform. 2013;46(1):97\u2013110.","journal-title":"J Biomed Inform"},{"issue":"1","key":"2136_CR87","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.artmed.2004.02.002","volume":"31","author":"HH Gu","year":"2004","unstructured":"Gu HH, Perl Y, Elhanan G, Min H, Zhang L, Peng Y. Auditing concept categorizations in the UMLS. Artif Intell Med. 2004;31(1):29\u201344.","journal-title":"Artif Intell Med"},{"key":"2136_CR88","doi-asserted-by":"crossref","unstructured":"He Z, Morrey CP, Perl Y, Elhanan G, Chen L, Chen Y, Geller J. Sculpting the UMLS refined semantic network. Online J Public Health Inf. 2014;6(2).","DOI":"10.5210\/ojphi.v6i2.5412"},{"issue":"1","key":"2136_CR89","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1197\/jamia.M2604","volume":"16","author":"L Chen","year":"2009","unstructured":"Chen L, Morrey CP, Gu H, Halper M, Perl Y. Modeling multi-typed structurally viewed chemicals with the UMLS refined semantic network. J Am Med Inform Assoc. 2009;16(1):116\u201331.","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2136_CR90","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.artmed.2011.05.003","volume":"52","author":"CP Morrey","year":"2011","unstructured":"Morrey CP, Chen L, Halper M, Perl Y. Resolution of redundant semantic type assignments for organic chemicals in the UMLS. Artif Intell Med. 2011;52(3):141\u201351.","journal-title":"Artif Intell Med"},{"issue":"6","key":"2136_CR91","doi-asserted-by":"publisher","first-page":"1042","DOI":"10.1016\/j.jbi.2012.05.006","volume":"45","author":"HH Gu","year":"2012","unstructured":"Gu HH, Elhanan G, Perl Y, Hripcsak G, Cimino JJ, Xu J, Chen Y, Geller J, Morrey CP. A study of terminology auditors\u2019 performance for UMLS semantic type assignments. J Biomed Inform. 2012;45(6):1042\u20138.","journal-title":"J Biomed Inform"},{"key":"2136_CR92","doi-asserted-by":"crossref","unstructured":"Toutanova K, Klein D, Manning CD, Singer Y. Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network. NAACL2003.","DOI":"10.3115\/1073445.1073478"},{"key":"2136_CR93","unstructured":"Part-of-speech tagging. https:\/\/en.wikipedia.org\/wiki\/Part-of-speech_tagging (accessed Oct 15, 2021). 2021."},{"key":"2136_CR94","doi-asserted-by":"crossref","unstructured":"Papagiannopoulou E, Tsoumakas G. A review of keyphrase extraction. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2020;10.","DOI":"10.1002\/widm.1339"},{"key":"2136_CR95","unstructured":"Mihalcea R, Tarau P. TextRank: bringing order into text. EMNLP: Association for Computational Linguistics; 2004: 404\u201311."}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02136-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-023-02136-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02136-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,10]],"date-time":"2024-09-10T08:18:52Z","timestamp":1725956332000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-023-02136-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,24]]},"references-count":95,"journal-issue":{"issue":"S1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2136"],"URL":"https:\/\/doi.org\/10.1186\/s12911-023-02136-0","relation":{},"ISSN":["1472-6947"],"issn-type":[{"type":"electronic","value":"1472-6947"}],"subject":[],"published":{"date-parts":[[2023,2,24]]},"assertion":[{"value":"22 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"40"}}