{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T10:27:30Z","timestamp":1790677650691,"version":"4.1.0"},"reference-count":37,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T00:00:00Z","timestamp":1627430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Ontologies are widely used nowadays. However, the plethora of ontologies currently available online, makes it really difficult to identify which ontologies are appropriate for a given task and to decide on their quality characteristics. This is further complicated by the fact that multiple quality criteria have been proposed for ontologies, making it even more difficult to decide which ontology to adopt. In this context, in this paper we present Delta, a modular online tool for analyzing and evaluating ontologies. The interested user can upload an ontology to the tool, which then automatically analyzes it and graphically visualizes numerous statistics, metrics, and pitfalls. Those visuals presented include a diverse set of quality dimensions, further guiding users to understand the benefits and the drawbacks of each individual ontology and how to properly develop and extend it.<\/jats:p>","DOI":"10.3390\/info12080301","type":"journal-article","created":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T11:12:52Z","timestamp":1627470772000},"page":"301","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Delta: A Modular Ontology Evaluation System"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9917-4486","authenticated-orcid":false,"given":"Haridimos","family":"Kondylakis","sequence":"first","affiliation":[{"name":"FORTH-ICS, 70013 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Astyrakakis","family":"Nikolaos","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8894-617X","authenticated-orcid":false,"given":"Papatsaroucha","family":"Dimitra","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Koumarelis","family":"Anastasios","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kritikakis","family":"Emmanouel","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kalkanis","family":"Kyriakos","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Skepasianos","family":"Iraklis","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klados","family":"Stylianos","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikos","family":"Papadakis","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Hellenic Mediterranean University, 71500 Heraklion, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1006\/knac.1993.1008","article-title":"A translation approach to portable ontology specifications","volume":"5","author":"Gruber","year":"1993","journal-title":"Knowl. Acquis."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Akbar, Z., Mustika, H.F., Rini, D.S., Manik, L.P., Indrawati, A., Fefirenta, A.D., and Djarwaningsih, T. (2021). An ontology-driven personalized faceted search for exploring knowledge bases of capsicum. Futur. Internet, 13.","DOI":"10.3390\/fi13070172"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pappas, A., Troullinou, G., Roussakis, G., Kondylakis, H., and Plexousakis, D. (2017). Exploring importance measures for summarizing RDF\/S KBs. European Semantic Web Conference, Springer.","DOI":"10.1007\/978-3-319-58068-5_24"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"103342","DOI":"10.1016\/j.jbi.2019.103342","article-title":"Patient empowerment for cancer patients through a novel ICT infrastructure","volume":"101","author":"Kondylakis","year":"2020","journal-title":"J. Biomed. Inform."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1002\/isaf.1424","article-title":"Toward an ontology-driven blockchain design for supply-chain provenance","volume":"25","author":"Kim","year":"2018","journal-title":"Intell. Syst. Account. Financ. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.autcon.2015.12.003","article-title":"EXPRESS to OWL for construction industry: Towards a recommendable and usable ifcowl ontology","volume":"63","author":"Pauwels","year":"2016","journal-title":"Autom. Constr."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Katis, E., Kondylakis, H., Agathangelos, G., and Vassilakis, K. (2018). Developing an ontology for curriculum and syllabus. European Semantic Web Conference, Springer.","DOI":"10.1007\/978-3-319-98192-5_11"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tzagkarakis, E., Kondylakis, H., Vardakis, G., and Papadakis, N. (2021). Ontology based governance for employee services. Algorithms, 14.","DOI":"10.3390\/a14040104"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Spoladore, D., Mahroo, A., Trombetta, A., and Sacco, M. (2021). DOMUS: A domestic ontology managed ubiquitous system. J. Ambient Intell. Humaniz. Comput., 1\u20136.","DOI":"10.1007\/s12652-021-03138-4"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Stratigi, M., Kondylakis, H., and Stefanidis, K. (2017, January 19\u201322). Fairness in group recommendations in the health domain. Proceedings of the IEEE 33rd International Conference on Data Engineering (ICDE), San Diego, CA, USA.","DOI":"10.1109\/ICDE.2017.217"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Stratigi, M., Kondylakis, H., and Stefanidis, K. (2018, January 3\u20136). Fairgrecs: Fair group recommendations by exploiting personal health information. Proceedings of the International Conference on Database and Expert Systems Applications, Regensburg, Germany.","DOI":"10.1007\/978-3-319-98812-2_11"},{"key":"ref_12","unstructured":"Kondylakis, H., Koumakis, L., Psaraki, M., Troullinou, G., Chatzimina, M., Marias, E.K.K., and Tsiknakis, M. (2015, January 11\u201315). Semantically-enabled personal medical information recommender. Proceedings of the International Semantic Web Conference (Posters & Demos), Bethlehem, PA, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kondylakis, H., Flouris, G., Fundulaki, I., Papakonstantinou, V., and Tsiknakis, M. (2015, January 14\u201316). Flexible access to patient data through e-consent. Proceedings of the 5th EAI International Conference on Wireless Mobile Communication and Healthcare, London, UK.","DOI":"10.4108\/eai.14-10-2015.2261673"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Spoladore, D., and Pessot, E. (2021). Collaborative ontology engineering methodologies for the development of decision support systems: Case studies in the healthcare domain. Electronics, 10.","DOI":"10.3390\/electronics10091060"},{"key":"ref_15","unstructured":"(2021, June 16). CIDOC-CRM. Available online: http:\/\/www.CIDOC-CRM.org\/."},{"key":"ref_16","first-page":"3901","article-title":"Automatic ontology construction from text: A review from shallow to deep learning trend","volume":"53","author":"Chan","year":"2019","journal-title":"Artif. Intell. Rev."},{"key":"ref_17","unstructured":"(2021, July 01). Delta Ontology Evaluation System. Available online: http:\/\/83.212.77.24\/."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Goldstein, A., Fink, L., and Ravid, G. (2021). A Framework for evaluating agricultural ontologies. Sustainability, 13.","DOI":"10.3390\/su13116387"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Raad, J., and Cruz, C. (2015, January 12\u201314). A Survey on ontology evaluation methods. Proceedings of the International Conference on Knowledge Engineering and Ontology Development, Lisbon, Portugal.","DOI":"10.5220\/0005591001790186"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"107","DOI":"10.3844\/jcssp.2005.107.113","article-title":"Cohesion metrics for ontology design and application","volume":"1","author":"Yao","year":"2005","journal-title":"J. Comput. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Pathak, J., Johnson, T.M., and Chute, C.G. (2008, January 13\u201315). Modular ontology techniques and their applications in the biomedical domain. Proceedings of the 2008 IEEE International Conference on Information Reuse and Integration, Las Vegas, NV, USA.","DOI":"10.1109\/IRI.2008.4583056"},{"key":"ref_22","unstructured":"Schlicht, A., and Stuckenschmidt, H. (2006, January 5). Towards structural criteria for ontology modularization. Proceedings of the WoMO, Aachen, Germany."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1093\/bib\/bbz009","article-title":"Evaluation of ontology structural metrics based on public repository data","volume":"21","author":"Franco","year":"2020","journal-title":"Brief. Bioinform."},{"key":"ref_24","unstructured":"Brank, J., Grobelnik, M., and Mladenic, D. A Survey of ontology evaluation techniques. Proceedings of the Conference on Data Mining and Data Warehouses (SiKDD), Ljubljana, Slovenia."},{"key":"ref_25","first-page":"1","article-title":"Approaches, methods, metrics, measures, and subjectivity in ontology evaluation: A survey","volume":"1","author":"Hlomani","year":"2014","journal-title":"Semant. Web J."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Aruna, T., Saranya, K., and Bhandari, C. (2011, January 20\u201322). A Survey on ontology evaluation tools. Proceedings of the 2011 International Conference on Process Automation, Control and Computing IEEE, Coimbatore, India.","DOI":"10.1109\/PACC.2011.5978931"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lourdusamy, R., and John, A. (2018, January 19\u201320). A review on metrics for ontology evaluation. Proceedings of the International Conference on Inventive Systems and Control (ICISC), Coimbatore, India.","DOI":"10.1109\/ICISC.2018.8399041"},{"key":"ref_28","unstructured":"Tartir, S., Arpinar, I.B., Moore, M., Sheth, A.P., and Aleman-Meza, B. (2005, January 27). OntoQA: Metric-based ontology quality analysis. Proceedings of the IEEE ICDM Workshop on KADASH, Houston, TX, USA."},{"key":"ref_29","unstructured":"Avila, C.V.S., Maia, G., Franco, W., Rolim, T.V., Franco, A.O.R., and Vidal, V.M.P. (2019, January 4\u20137). OntoVal: A tool for ontology evaluation by domain specialists. Proceedings of the 38th International Conference on Conceptual Modeling, Salvador, Brazil."},{"key":"ref_30","first-page":"186","article-title":"Ontometrics: Putting metrics into use for ontology evaluation","volume":"2","author":"Lantow","year":"2016","journal-title":"IC3K"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Reiz, A., Dibowski, H., Sandkuhl, K., and Lantow, B. (2020, January 2\u20134). Ontology metrics as a service (OMaaS). Proceedings of the KEOD, Budapest, Hungary.","DOI":"10.5220\/0010144002500257"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Amith, M., Manion, F., Liang, C., Harris, M., Wang, D., He, Y., and Tao, C. (2019). OntoKeeper: Semiotic-driven ontology evaluation tool for biomedical ontologists. IEEE BIBM, 1614\u20131617.","DOI":"10.1109\/BIBM.2018.8621458"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.datak.2004.11.010","article-title":"A semiotic metrics suite for assessing the quality of ontologies","volume":"55","author":"Storey","year":"2005","journal-title":"Data Knowl. Eng."},{"key":"ref_34","first-page":"7","article-title":"OOPS! (OntOlogy Pitfall Scanner!): Supporting ontology eval-uation on-line","volume":"10","year":"2014","journal-title":"IJSWIS"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbi.2018.02.010","article-title":"Assessing the practice of biomedical ontology evaluation: Gaps and opportunities","volume":"80","author":"Amith","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_36","unstructured":"Schulz, S., Boeker, M., Ramos, J.A.V., and Jansen, L. (2017, January 21\u201323). Pizza & wine: The need for educational tools for foundational ontologies. Proceedings of the JOWO, Bozen-Bolzano, Italy."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Troullinou, G., Kondylakis, H., Stefanidis, K., and Plexousakis, D. (2018, January 8\u201312). Exploring RDFS kbs using summaries. Proceedings of the International Semantic Web Conference, Monterey, CA, USA.","DOI":"10.1007\/978-3-030-00671-6_16"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/8\/301\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:36:18Z","timestamp":1760164578000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/8\/301"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,28]]},"references-count":37,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["info12080301"],"URL":"https:\/\/doi.org\/10.3390\/info12080301","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,28]]}}}