{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T12:01:52Z","timestamp":1747224112280,"version":"3.40.5"},"reference-count":42,"publisher":"IGI Global","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2010,1,1]]},"abstract":"<p>Ontologies, terminologies and vocabularies are popular repositories for collecting the terms used in a domain.It may be expected that in the future more such ontologies will be created for domain experts. However, there is increasing interest in making the language of experts understandable to casual users. For example, cancer patients often research their cases on the Web. The authors consider the problem of objectively evaluating the quality of ontologies (QoO). This article formalizes the notion of naturalness as a component of QoO and quantitatively measures naturalness for well-known ontologies (UMLS, WordNet, OpenCyc) based on their concepts, IS-A relationships and semantic relationships. To compute numeric values characterizing the naturalness of an ontology, this article defines appropriate metrics. As absolute numbers in such a pursuit are often meaningless, we concentrate on using relative naturalness metrics. That allows us to say that a certain ontology is relatively more natural than another one.<\/p>","DOI":"10.4018\/jcmam.2010072001","type":"journal-article","created":{"date-parts":[[2010,4,16]],"date-time":"2010-04-16T17:53:52Z","timestamp":1271440432000},"page":"1-18","source":"Crossref","is-referenced-by-count":0,"title":["A Formal Approach to Evaluating Medical Ontology Systems using Naturalness"],"prefix":"10.4018","volume":"1","author":[{"given":"Yoo Jung","family":"An","sequence":"first","affiliation":[{"name":"Fairleigh Dickinson University, USA"}]},{"given":"Kuo-Chuan","family":"Huang","sequence":"additional","affiliation":[{"name":"New Jersey Institute of Technology, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9360-4679","authenticated-orcid":true,"given":"Soon Ae","family":"Chun","sequence":"additional","affiliation":[{"name":"College of Staten Island, USA"}]},{"given":"James","family":"Geller","sequence":"additional","affiliation":[{"name":"New Jersey Institute of Technology, USA"}]}],"member":"2432","reference":[{"key":"jcmam.2010072001-0","unstructured":"Agirre, E., Ansa, O., Hovy, E., & Martinez, D. (2000). Enriching very large ontologies using the WWW. In Proceedings of ECAI Workshop on Ontology Learning, Berlin."},{"key":"jcmam.2010072001-1","doi-asserted-by":"crossref","unstructured":"Alani, H., & Brewster, C. (2005). Ontology Ranking based on the Analysis of Concept Structure. In Proceedings of the Third International Conference on Knowledge Capture (K-Cap) (pp. 51-58), Banff, Alberta, Canada.","DOI":"10.1145\/1088622.1088633"},{"key":"jcmam.2010072001-2","unstructured":"An, Y. J. (2008). Ontology Learning for the Semantic Deep Web. Doctoral dissertation, New Jersey Institute of technology. Retrieved from NJIT eTD: The New Jersey Institute of Technology's electronic Theses & Dissertations Web site: http:\/\/library1.njit.edu\/etd\/2000s\/2008\/njit-etd2008-027\/njit-etd2008-027.html"},{"issue":"3","key":"jcmam.2010072001-3","first-page":"63","article-title":"Naturalness of Ontology Concepts for Rating Aspects of the Semantic Web.","volume":"6","author":"Y. J.An","year":"2006","journal-title":"Communications of the International Information Management Association"},{"key":"jcmam.2010072001-4","first-page":"659","article-title":"Building Medical Ontologies Based on Terminology Extraction from Texts: An Experimentation in Pneumology.","volume":"116","author":"B.Baneyx","year":"2005","journal-title":"Studies in Health Technology and Informatics"},{"key":"jcmam.2010072001-5","unstructured":"Brachman, R. J. (1992). Reducing CLASSIC to practice: Knowledge Representation Theory Meets Reality. In Proceedings of the Third International Conference on the Principles of Knowledge Representation and Reasoning (KR-92) (pp. 247-258)."},{"key":"jcmam.2010072001-6","doi-asserted-by":"crossref","unstructured":"Brewster, C., Ciravegna, F., & Wilks, Y. (2002). User-Centred Onlology Learning for Knowledge Management. In Proceedings of the 7th International Workshop on Applications of Natural Language to Information Systems.","DOI":"10.1007\/3-540-36271-1_18"},{"key":"jcmam.2010072001-7","unstructured":"Brewster, C., Ciravegna, F., & Wilks, Y. (2003). Background and Foreground Knowledge in Dynamic Ontology Construction: Viewing Text as Knowledge Maintenance. In Proceedings of the Semantic Web Workshop (SIGIR). Retrieved from AKT EPrint Archiveat Web site: http:\/\/eprints.aktors.org\/307\/"},{"key":"jcmam.2010072001-8","unstructured":"Burgun, A., & Bodenreider, O. (2001). Mapping the UMLS Semantic Network into General Ontologies. In Proceedings of American Medical Informatics Association (AMIA) Annual Symposium (pp. 81-85)."},{"key":"jcmam.2010072001-9","doi-asserted-by":"crossref","unstructured":"Castano, S., & Antonellis, V. D. (1999). A Discovery-Based Approach to Database Ontology Design. Distributed and Parallel Databases - Special Issue on Ontologies and Databases. 7(1), 67-98.","DOI":"10.1023\/A:1008686328306"},{"key":"jcmam.2010072001-10","doi-asserted-by":"crossref","unstructured":"Chun, S. A., & Geller, J. (2008). Evaluating Ontologies based on the Naturalness of their Preferred Terms. In Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008) (p. 238).","DOI":"10.1109\/HICSS.2008.151"},{"key":"jcmam.2010072001-11","unstructured":"Colomb, R. M. (2002, November). Quality of Ontologies in Interoperating Information Systems. Retrieved from the Institute of Cognitive Science and Technology, Laboratory for Applied Ontology Web site: http:\/\/www.loa-cnr.it\/Papers\/ISIB-CNR-TR-18-02.pdf"},{"key":"jcmam.2010072001-12","unstructured":"Cycorp. (2005). OpenCyc Version 0.9. Retrieved from http:\/\/www.opencyc.org\/releases\/"},{"key":"jcmam.2010072001-13","doi-asserted-by":"crossref","unstructured":"Ding, L., Finin, T., Joshi, A., Pan, R., Cost, R., Peng, Y., et al. (2004). Swoogle: a search and metadata engine for the semantic web. In Proceedings of the 13th ACM Conference on Information and Knowledge Management (pp. 652-659).","DOI":"10.1145\/1031171.1031289"},{"key":"jcmam.2010072001-14","unstructured":"Finin, T., Sachs, J., & Parr, C. S. (2007). Finding Data, Knowledge, and Answers on the Semantic Web. In Proceedings of the 20th International FLAIRS Conference (pp. 2-7)."},{"key":"jcmam.2010072001-15","doi-asserted-by":"crossref","unstructured":"Gligorov, R., Aleksovski, Z., Kate, W., & Harmelen, F. (2007). Using Google Distance to Weight Approximate Ontology Matches. In 16th International World Wide Web Conference (pp. 767-775).","DOI":"10.1145\/1242572.1242676"},{"key":"jcmam.2010072001-16","unstructured":"Gruber, T. R. (1993). Toward principles for the design of ontologies used for knowledge sharing. In Proceedings of International Workshop on Formal Ontology, Padova, Italy. Retrieved from Web site: http:\/\/www-ksl.stanford.edu\/knowledge-sharing\/papers\/README.html#onto-design"},{"key":"jcmam.2010072001-17","first-page":"539","article-title":"Automatic Acquisition of Hyponyms form Large Text Corpora.","volume":"COLING-92","author":"M. A.Hearst","year":"1992","journal-title":"Proceedings of"},{"key":"jcmam.2010072001-18","unstructured":"Kalfoglou, Y., & Hu, B. (2006, May). Issues with evaluating and using publicly available ontologies, In Proceedings of the Fourth International Evaluation of Ontologies for the Web Workshop (EON2006), Edinburgh, UK."},{"key":"jcmam.2010072001-19","first-page":"6","article-title":"A note on the use of the non-parametric Wilcoxon-Mann-Whitney test in the analysis of medical studies.","author":"C.Kuhnast","year":"2008","journal-title":"German Medical Science"},{"key":"jcmam.2010072001-20","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2005.08.001"},{"key":"jcmam.2010072001-21","doi-asserted-by":"publisher","DOI":"10.1080\/00048408312341131"},{"key":"jcmam.2010072001-22","doi-asserted-by":"crossref","unstructured":"Maedche, A., & Staab, S. (2000). Mining Ontologies from Text. Proceedings of EKAW-2000, Springer Lecture Notes in Artificial Intelligence (LNAI-1937) (pp. 189-202), Juan-Les-Pins, France.","DOI":"10.1007\/3-540-39967-4_14"},{"key":"jcmam.2010072001-23","first-page":"171","article-title":"Aggregating UMLS Semantic Types for Reducing Conceptual Complexity. In","volume":"2001","author":"A. T.McCray","year":"2001","journal-title":"Proceedings of Medinfo"},{"key":"jcmam.2010072001-24","unstructured":"Mihalcea, R., & Moldovan, D. I. (1999). An Automatic Method for Generating Sense Tagged Corpora. In Proceedings of American Association for Artificial Intelligence (pp. 461-466), Orlando, FL."},{"key":"jcmam.2010072001-25","doi-asserted-by":"crossref","unstructured":"Mizoguchi, R. (2004). Tutorial on Ontological Engineering: Part 3: Advanced Course of Ontological Engineering. New Generation Computing, 22(2).","DOI":"10.1007\/BF03040960"},{"key":"jcmam.2010072001-26","doi-asserted-by":"crossref","unstructured":"Necib, C. B., & Freytag, J. (2003). Ontology Based Query Processing in Database Management Systems. In Proceedings of the 6th International Conference on Ontologies, Databases and Applications of Semantics for Large Scale Information Systems (ODBASE'2003) (pp. 37-99).","DOI":"10.1007\/978-3-540-39964-3_53"},{"issue":"3","key":"jcmam.2010072001-27","first-page":"53","article-title":"The State of the Art in Ontology Design: A Survey and Comparative Review.","volume":"18","author":"N. F.Noy","year":"1997","journal-title":"AI Magazine"},{"key":"jcmam.2010072001-28","unstructured":"Obrst, L., Hughes, T., & Steve Ray, S. (2006, May). Prospects and Possibilities for Ontology Evaluation: The View from NCOR. In Proceedings of the Fourth International Evaluation of Ontologies for the Web Workshop (EON2006), Edinburgh, UK."},{"key":"jcmam.2010072001-29","unstructured":"OpenCyc. (2005). Predicates and Denotational Functions of Cyc [PowerPoint slides]. Retrieved from Web site: http:\/\/www.cyc.com\/doc\/tut\/DnLoad\/TheBasicsOfPredDenoFun.pdf"},{"key":"jcmam.2010072001-30","first-page":"810","article-title":"An Ontological Analysis of the UMLS Metathesaurus.","volume":"5","author":"D. M.Pisanelli","year":"1998","journal-title":"Journal of the American Medical Informatics Association"},{"key":"jcmam.2010072001-31","unstructured":"Princeton University. (2006). WordNet 2.1 Database Statistics. Retrieved from Web site: http:\/\/wordnet.princeton.edu\/man\/wnstats.7WN"},{"key":"jcmam.2010072001-32","doi-asserted-by":"crossref","unstructured":"Seta, K., Ikeda, M., Kakusho, O., & Mizoguchi, R. (1997, June 2-5). Capturing a Conceptual Model for End-User Programming: Task Ontology as a Static User Model. In Proceedings of the Sixth International Conference on User Modeling (pp. 203-214). Chia Laguna, Sardinia, Italy.","DOI":"10.1007\/978-3-7091-2670-7_22"},{"key":"jcmam.2010072001-33","doi-asserted-by":"crossref","unstructured":"Sieg, A., Mobasher, B., & Burke, R. D. (2007). Ontological User Profiles for Representing Context in Web Search. In Proceedings of Web Intelligence\/IAT Workshops (pp. 91-94).","DOI":"10.1109\/WI-IATW.2007.112"},{"key":"jcmam.2010072001-34","unstructured":"Smith, B. C. (1982). Reflection and Semantics in a Procedural Language. PhD thesis. Massachusetts Institute of Technology. MIT-LCS-272:154."},{"key":"jcmam.2010072001-35","unstructured":"Sourceforge.net. (2007). OpenCyc [Computer program]. Retrieved from Web site: http:\/\/sourceforge.net\/project\/showfiles.php?group_id=27274."},{"key":"jcmam.2010072001-36","unstructured":"Sourceforge.net. (2008). JWNL \u2013 Java WordNet Library [Computer program]. Retrieved from Web site: http:\/\/sourceforge.net\/projects\/jwordnet."},{"key":"jcmam.2010072001-37","unstructured":"Staab, S., & Maedche, A. (2000). Axioms are objects too \u2013 Ontology engineering beyond the modeling of concepts and relations. Research report 399, University of Karlsruhe, Institute AIFB."},{"key":"jcmam.2010072001-38","doi-asserted-by":"crossref","unstructured":"Stone, J., Wu, X., & Greenblatt, M. (2004). An Intelligent Digital Library System for Biologists. In Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference (CSB 2004) (pp. 491-492).","DOI":"10.1109\/CSB.2004.1332471"},{"key":"jcmam.2010072001-39","unstructured":"Supekar, K., Patel, C., & Lee, Y. (2004). Characterizing Quality of Knowledge on Semantic Web. In Proceedings of the Seventeenth International Florida Artificial Intelligence Research Symposium Conference (pp. 220-228)."},{"key":"jcmam.2010072001-40","unstructured":"U. S. National Library of Medicine. (2008). UMLS Knowledge Sources. Retrieved from Unified Medical Language System Web site: http:\/\/www.nlm.nih.gov\/research\/umls\/umlsdoc.html."},{"key":"jcmam.2010072001-41","unstructured":"Zeng, Q. T., Tse, T., Crowell, J., Divita, G., Roth, L., & Browne, A. C. (2005) Identifying Consumer-Friendly Display (CFD) Names for Health Concepts. In Proceedings of the AMIA Annual Symposium (pp. 859-863)."}],"container-title":["International Journal of Computational Models and Algorithms in Medicine"],"original-title":[],"language":"ng","link":[{"URL":"https:\/\/www.igi-global.com\/viewtitle.aspx?TitleId=38941","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,1]],"date-time":"2022-06-01T15:42:47Z","timestamp":1654098167000},"score":1,"resource":{"primary":{"URL":"https:\/\/services.igi-global.com\/resolvedoi\/resolve.aspx?doi=10.4018\/jcmam.2010072001"}},"subtitle":[""],"short-title":[],"issued":{"date-parts":[[2010,1,1]]},"references-count":42,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2010,1]]}},"URL":"https:\/\/doi.org\/10.4018\/jcmam.2010072001","relation":{},"ISSN":["1947-3133","1947-3141"],"issn-type":[{"type":"print","value":"1947-3133"},{"type":"electronic","value":"1947-3141"}],"subject":[],"published":{"date-parts":[[2010,1,1]]}}}