{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T18:28:44Z","timestamp":1767637724060,"version":"3.48.0"},"reference-count":51,"publisher":"Maximum Academic Press","license":[{"start":{"date-parts":[[2020,7,7]],"date-time":"2020-07-07T00:00:00Z","timestamp":1594080000000},"content-version":"unspecified","delay-in-days":188,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The Knowledge Engineering Review"],"published-print":{"date-parts":[[2020]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Simple ontology alignments, largely studied in the literature, link one single entity of a source ontology to one single entity of a target ontology. One of the limitations of these alignments is, however, their lack of expressiveness, which can be overcome by complex alignments, which are composed of correspondences involving logical constructors or transformation functions. While most work on complex ontology matching has been dedicated to the development of complex matching approaches, there is still a lack of benchmarks on which the complex approaches can be systematically evaluated. The aim of this paper is to present the process of constructing the consensual complex Conference dataset, describing the design choices and the methodology followed for constructing it. We discuss the issues the experts were faced with during the process and discuss the lessons learned and perspectives in the field.<\/jats:p>","DOI":"10.1017\/s0269888920000247","type":"journal-article","created":{"date-parts":[[2020,7,7]],"date-time":"2020-07-07T01:46:50Z","timestamp":1594086410000},"source":"Crossref","is-referenced-by-count":1,"title":["A consensual dataset for complex ontology matching evaluation"],"prefix":"10.48130","volume":"35","author":[{"given":"Elodie","family":"Thi\u00e9blin","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michelle","family":"Cheatham","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2840-005X","authenticated-orcid":false,"given":"Cassia","family":"Trojahn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ondrej","family":"Zamazal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"27968","published-online":{"date-parts":[[2020,7,7]]},"reference":[{"key":"S0269888920000247_ref50","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-00668-6_17"},{"key":"S0269888920000247_ref49","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2017.01.001"},{"key":"S0269888920000247_ref48","doi-asserted-by":"publisher","DOI":"10.4018\/IJSWIS.2016040102"},{"key":"S0269888920000247_ref17","doi-asserted-by":"publisher","DOI":"10.1145\/2247596.2247678"},{"key":"S0269888920000247_ref8","unstructured":"Euzenat, J. , Ferrara, A. , Hollink, L. , Isaac, A. , Joslyn, C. , Malais\u00e9, V. , Meilicke, C. , Nikolov, A. , Pane, J. , Sabou, M. , Scharffe, F. , Shvaiko, P. , Spiliopoulos, V. , Stuckenschmidt, H. , Sv\u00e1b-Zamazal, O. , Sv\u00e1tek, V. , dos Santos, C. T. , Vouros, G. A. & Wang, S. 2009. Results of the ontology alignment evaluation initiative 2009. In Proceedings of the 4th International Workshop on Ontology Matching (OM-2009) collocated with the 8th International Semantic Web Conference (ISWC-2009) Chantilly, USA, October 25, 2009."},{"key":"S0269888920000247_ref39","unstructured":"\u0160v\u00e1b, O. , Sv\u00e1tek, V. , Berka, P. , Rak, D. & Tom\u00e1\u0161ek, P. 2005. Ontofarm: towards an experimental collection of parallel ontologies. In Poster Track of ISWC, 2005."},{"key":"S0269888920000247_ref14","unstructured":"Isaac, A. , Matthezing, H. , van der Meij, L. , Schlobach, S. , Wang, S. & Zinn, C. 2008. Putting ontology alignment in context: usage scenarios, deployment and evaluation in a library case. In 5th European Semantic Web Conference, 402\u2013417."},{"key":"S0269888920000247_ref27","doi-asserted-by":"publisher","DOI":"10.3233\/SW-170268"},{"key":"S0269888920000247_ref13","unstructured":"Hollink, L. , Van Assem, M. , Wang, S. , Isaac, A. & Schreiber, G. 2008. Two variations on ontology alignment evaluation: methodological issues. In 5th European Semantic Web Conference, 388\u2013401."},{"key":"S0269888920000247_ref19","unstructured":"Mathur, S. N. , O\u2019Sullivan, D. & Brennan, R. 2018. Milan: automatic generation of R2RML mappings. In Proceedings of the 26th Irish Conference on Artificial Intelligence and Cognitive Science, Dublin, Ireland, 7th December, 1\u201312."},{"key":"S0269888920000247_ref12","doi-asserted-by":"publisher","DOI":"10.1145\/1014052.1014071"},{"key":"S0269888920000247_ref24","doi-asserted-by":"publisher","DOI":"10.1186\/s13326-017-0171-8"},{"key":"S0269888920000247_ref11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-17746-0_20"},{"key":"S0269888920000247_ref5","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007612"},{"key":"S0269888920000247_ref20","unstructured":"Meilicke, C. 2011. Alignment Incoherence in Ontology Matching. PhD thesis, Universit\u00e4t Mannheim."},{"key":"S0269888920000247_ref23","unstructured":"Nunes, B. P. , Mera, A. , Casanova, M. A. , Breitman, K. K. & Leme, L. A. P. 2011. Complex matching of RDF datatype properties. In 6th ISWC Workshop on Ontology Matching."},{"key":"S0269888920000247_ref22","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02121-3_74"},{"key":"S0269888920000247_ref4","unstructured":"Dekhtyar, A. & Hayes, J. H. 2006. Good benchmarks are hard to find: toward the benchmark for information retrieval applications in software engineering. In Proceedings of the 22th International Conference on Software Maintenance."},{"key":"S0269888920000247_ref6","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36560-5_17"},{"key":"S0269888920000247_ref2","unstructured":"Algergawy, A. , Cheatham, M. , Faria, D. , Ferrara, A. , Fundulaki, I. , Harrow, I. , Hertling, S. , Jim\u00e9nez-Ruiz, E. , Karam, N. , Khiat, A. , Lambrix, P. , Liand, H. , Montanelli, S. , Paulheim, H. , Pesquita, C. , Saveta, T. , Schmidt, D. , Shvaiko, P. , Splendiani, A. , Thi\u00e9blin, E. , Trojahn, C. , Vata\u0161\u010dinov\u00e1, J. , Zamazal, O. & Zhou, L. 2018. Results of the ontology alignment evaluation initiative 2018. In OM-2018: Proceedings of the Twelfth International Workshop on Ontology Matching."},{"key":"S0269888920000247_ref18","unstructured":"Maltese, V. , Giunchiglia, F. & Autayeu, A. 2010. Save up to 99% of your time in mapping validation. In On the Move to Meaningful Internet Systems, OTM 2010 - Confederated International Conferences: CoopIS, IS, DOA and ODBASE, Hersonissos, Crete, Greece, October 25\u201329, 1044\u20131060."},{"key":"S0269888920000247_ref7","unstructured":"Doan, A. (2005). Dataset of complex correspondences. Illinois Semantic Integration Archive. Computer Science Department, University of Illinois. http:\/\/pages.cs.wisc.edu\/~anhai\/wisc-si-archive\/."},{"key":"S0269888920000247_ref1","first-page":"71","article-title":"Means: une approche s\u00e9mantique pour la recherche de r\u00e9ponses aux questions m\u00e9dicales","volume":"55","author":"Abacha","year":"2014","journal-title":"TAL"},{"key":"S0269888920000247_ref46","unstructured":"Van Hage, W. R. , Isaac, A. & Aleksovski, Z. 2007. Sample evaluation of ontology-matching systems. In EON, 2007, 41\u201350."},{"key":"S0269888920000247_ref31","unstructured":"Scharffe, F. 2009. Correspondence Patterns Representation. PhD thesis, Faculty of Mathematics, Computer Science and University of Innsbruck."},{"key":"S0269888920000247_ref15","unstructured":"Jiang, S. , Lowd, D. , Kafle, S. & Dou, D. 2016. Ontology matching with knowledge rules. In Transactions on Large-Scale Data-and Knowledge-Centered Systems XXVIII, Hameurlain, A., K\u00fcng, J., Wagner, R. & Chen, Q. (eds). Springer, 75\u201395."},{"key":"S0269888920000247_ref44","unstructured":"Thi\u00e9blin, E. , Haemmerl\u00e9, O. & Trojahn, C. 2018. Complex matching based on competency questions for alignment: a first sketch. In Ontology Matching Workshop."},{"key":"S0269888920000247_ref9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-38721-0"},{"key":"S0269888920000247_ref3","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-11915-1_3"},{"key":"S0269888920000247_ref36","unstructured":"Solimando, A. , Jim\u00e9nez-Ruiz, E. & Pinkel, C. 2014. Evaluating ontology alignment systems in query answering tasks. In ISWC 2014 International Conference on Posters & Demonstrations, 301\u2013304."},{"key":"#cr-split#-S0269888920000247_ref47.1","unstructured":"Visser, P. R. , Jones, D. M. , Bench-Capon, T. J. & Shave, M. 1997. An analysis of ontology mismatches"},{"key":"#cr-split#-S0269888920000247_ref47.2","unstructured":"heterogeneity versus interoperability. In AAAI 1997 Spring Symposium on Ontological Engineering, Stanford CA., USA, 164-172."},{"key":"S0269888920000247_ref21","unstructured":"Meilicke, C. & Stuckenschmidt, H. 2008. Incoherence as a basis for measuring the quality of ontology mappings. In 3rd International Conference on Ontology Matching, 431, 1\u201312."},{"key":"S0269888920000247_ref10","unstructured":"Faria, D. , Pesquita, C. , Balasubramani, B. S. , Tervo, T. , Carrio, D. , Garrilha, R. , Couto, F. M. & Cruz, I. F. 2018. Results of AML participation in OAEI 2018. In OM-2018: Proceedings of the Twelfth International Workshop on Ontology Matching."},{"key":"S0269888920000247_ref25","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-17746-0_38"},{"key":"S0269888920000247_ref40","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70863-8_3"},{"key":"S0269888920000247_ref26","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35176-1_27"},{"key":"S0269888920000247_ref30","unstructured":"Ritze, D. , V\u00f6lker, J. , Meilicke, C. & \u0160v\u00e1b Zamazal, O. 2010. Linguistic analysis for complex ontology matching. In 5th Workshop on Ontology Matching, 1\u201312."},{"key":"S0269888920000247_ref16","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45810-7_23"},{"key":"S0269888920000247_ref28","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-76848-7_56"},{"key":"S0269888920000247_ref29","unstructured":"Ritze, D. , Meilicke, C. , \u0160v\u00e1b Zamazal, O. & Stuckenschmidt, H. 2009. A pattern-based ontology matching approach for detecting complex correspondences. In 4th ISWC Workshop on Ontology Matching, 25\u201336."},{"key":"S0269888920000247_ref35","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-016-0983-3"},{"key":"S0269888920000247_ref32","doi-asserted-by":"publisher","DOI":"10.4018\/jwp.2011070101"},{"key":"S0269888920000247_ref33","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2003.1201189"},{"key":"S0269888920000247_ref34","unstructured":"Singh, A. , Debruyne, C. , Brennan, R. , Meehan, A. & O\u2019Sullivan, D. 2017. Extension of the M-Gov ontology mapping framework for increased traceability. In Proceedings of the 12th International Workshop on Ontology Matching co-located with the 16th International Semantic Web Conference (ISWC 2017), Vienna, Austria, October 21, 49\u201360."},{"key":"S0269888920000247_ref42","unstructured":"Thi\u00e9blin, \u00c9. , Haemmerl\u00e9, O. , Hernandez, N. & Trojahn, C. 2018b. Task-oriented complex ontology alignment: two alignment evaluation sets. In The Semantic Web - 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3\u20137, 2018, Proceedings, 655\u2013670."},{"key":"S0269888920000247_ref37","unstructured":"Stapleton, G. , Howse, J. , Bonnington, A. & Burton, J. 2014. A vision for diagrammatic ontology engineering. In International Workshop on Visualizations and User Interfaces for Knowledge Engineering and Linked Data Analytics, 1\u201313."},{"key":"S0269888920000247_ref43","unstructured":"Thieblin, E. , Haemmerle, O. , Hernandez, N. & Trojahn, C. to appear. Survey on complex ontology matching. Semantic Web Journal."},{"key":"S0269888920000247_ref38","first-page":"1","article-title":"Measuring expert performance at manually classifying domain entities under upper ontology classes","volume":"57","author":"Stevens","year":"2018","journal-title":"Journal of Web Semantics"},{"key":"S0269888920000247_ref41","unstructured":"Thi\u00e9blin, \u00c9. , Cheatham, M. , dos Santos, C. T., Zamazal, O. & Zhou, L. 2018a. The first version of the OAEI complex alignment benchmark. In Proceedings of the ISWC 2018 Posters & Demonstrations, Industry and Blue Sky Ideas Tracks co-located with 17th International Semantic Web Conference (ISWC 2018), Monterey, USA, October 8\u201312, 2018."},{"key":"S0269888920000247_ref45","doi-asserted-by":"publisher","DOI":"10.1145\/1999676.1999689"}],"container-title":["The Knowledge Engineering Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0269888920000247","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:42:19Z","timestamp":1767624139000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0269888920000247\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020]]},"references-count":51,"alternative-id":["S0269888920000247"],"URL":"https:\/\/doi.org\/10.1017\/s0269888920000247","relation":{},"ISSN":["0269-8889","1469-8005"],"issn-type":[{"type":"print","value":"0269-8889"},{"type":"electronic","value":"1469-8005"}],"subject":[],"published":{"date-parts":[[2020]]},"article-number":"e34"}}