{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T15:59:40Z","timestamp":1781279980436,"version":"3.54.1"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,9,16]],"date-time":"2022-09-16T00:00:00Z","timestamp":1663286400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Union\u00b4s Horizon 2020","award":["951911"],"award-info":[{"award-number":["951911"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Comput. Cult. Herit."],"published-print":{"date-parts":[[2022,9,30]]},"abstract":"<jats:p>This work combines semantic reasoning and machine learning to create tools that allow curators of the visual art collections to identify and correct the annotations of the artwork as well as to improve the relevance of the content-based search results in these collections. The research is based on the Joconde database maintained by the French Ministry of Culture that contains illustrated artwork records from main French public and private museums representing archeological objects, decorative arts, fine arts, historical and scientific documents, etc. The Joconde database includes semantic metadata that describes properties of the artworks and their content. The developed methods create a data pipeline that processes metadata, trains a Convolutional Neural Network image classification model, makes prediction for the entire collection and expands the metadata to be the base for the SPARQL search queries. We developed a set of such queries to identify noise and silence in the human annotations and to search image content with results ranked according to the relevance of the objects quantified by the prediction score provided by the deep learning model. We also developed methods to discover new contextual relationships between the concepts in the metadata by analyzing the contrast between the concepts\u2019 similarities in the Joconde's semantic model and other vocabularies and we tried to improve the model prediction scores based on the semantic relations. Our results show that cross-fertilization between symbolic AI and machine learning can indeed provide the tools to address the challenges of the museum curators work describing the artwork pieces and searching for the relevant images.<\/jats:p>","DOI":"10.1145\/3485844","type":"journal-article","created":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T13:42:44Z","timestamp":1646660564000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":22,"title":["Learning and Reasoning for Cultural Metadata Quality: Coupling Symbolic AI and Machine Learning over a Semantic Web Knowledge Graph to Support Museum Curators in Improving the Quality of Cultural Metadata and Information Retrieval"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0395-2069","authenticated-orcid":false,"given":"Anna","family":"Bobasheva","sequence":"first","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur, CNRS, Inria, I3S, Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0543-1232","authenticated-orcid":false,"given":"Fabien","family":"Gandon","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur, CNRS, Inria, I3S, Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8712-1443","authenticated-orcid":false,"given":"Frederic","family":"Precioso","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur, CNRS, Inria, I3S, Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,9,16]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-bioeng-071516-044442"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2018.8512750"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSPCS.2018.8631731"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2907986"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5244\/c.28.122"},{"key":"e_1_3_2_9_2","first-page":"327","article-title":"Recognizing art style automatically in painting with deep learning","author":"Lecoutre Adrian","year":"2017","unstructured":"Adrian Lecoutre, Benjamin Negrevergne, and Florian Yger. 2017. Recognizing art style automatically in painting with deep learning. In Asian Conference on Machine Learning. PMLR, 327\u2013342.","journal-title":"In Asian Conference on Machine Learning. PMLR"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2018.07.026"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46604-0_52"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2004.03.009"},{"key":"e_1_3_2_13_2","volume-title":"International Conference on Learning Representations","author":"Simonyan Karen","year":"2015","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1038\/scientificamerican0501-34"},{"key":"e_1_3_2_16_2","article-title":"Thesaurus iconographique","author":"Garnier Fran\u00e7ois","year":"1984","unstructured":"Fran\u00e7ois Garnier. 1984. Thesaurus iconographique. Syst\u00e8me Descriptif des Repr\u00e9sentations, Paris: CNRS (1984).","journal-title":"Syst\u00e8me Descriptif des Repr\u00e9sentations"},{"key":"e_1_3_2_17_2","first-page":"63","article-title":"Num\u00e9rique et nouveaux enjeux pour le patrimoine. La Lettre de l'OCIM. Mus\u00e9es","volume":"162","author":"Man\u0153uvre Laurent","year":"2015","unstructured":"Laurent Man\u0153uvre. 2015. Num\u00e9rique et nouveaux enjeux pour le patrimoine. La Lettre de l'OCIM. Mus\u00e9es, Patrimoine et Culture Scientifiques et Techniques 162 (2015), 63\u201365.","journal-title":"Patrimoine et Culture Scientifiques et Techniques"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.3917\/i2d.162.0046"},{"key":"e_1_3_2_19_2","author":"Read Jesse","year":"2010","unstructured":"Jesse Read. 2010. Scalable Multi-label Classification. Ph.D. Dissertation. University of Waikato.","journal-title":"Scalable Multi-label Classification"},{"key":"e_1_3_2_20_2","first-page":"3","volume-title":"International Conference on Dublin Core and Metadata Applications","author":"Miles Alistair","year":"2005","unstructured":"Alistair Miles, Brian Matthews, Michael Wilson, and Dan Brickley. 2005. SKOS core: Simple knowledge organisation for the web. In International Conference on Dublin Core and Metadata Applications. 3\u201310."},{"key":"e_1_3_2_21_2","article-title":"Weaving the pedantic web","author":"Hogan Aidan","year":"2010","unstructured":"Aidan Hogan, Andreas Harth, Alexandre Passant, Stefan Decker, and Axel Polleres. 2010. Weaving the pedantic web. In LDOW.","journal-title":"LDOW"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33290-6_25"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2016.09.031"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.14569\/IJARAI.2015.040401"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/MDM48529.2020.00074"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/FPL.2018.00033"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIT48102.2019.00023"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3230905.3230940"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00432"},{"key":"e_1_3_2_30_2","volume-title":"European Conference on Artificial Intelligence","author":"Corby Olivier","year":"2004","unstructured":"Olivier Corby, Rose Dieng-Kuntz, and Catherine Faron Zucker. 2004. Querying the semantic web with CORESE search engine. In European Conference on Artificial Intelligence."},{"key":"e_1_3_2_31_2","author":"Corby Olivier","year":"2006","unstructured":"Olivier Corby, Rose Dieng-Kuntz, Catherine Faron Zucker, and Fabien Gandon. 2006. Ontology-based Approximate Query Processing for Searching the Semantic Web with CORESE. Ph.D. Dissertation. INRIA.","journal-title":"Ontology-based Approximate Query Processing for Searching the Semantic Web with CORESE"},{"key":"e_1_3_2_32_2","article-title":"Multi-label classification: A comparative study on threshold selection methods","author":"Al-Otaibi Reem","year":"2014","unstructured":"Reem Al-Otaibi, Peter Flach, and Meelis Kull. 2014. Multi-label classification: A comparative study on threshold selection methods. In First International Workshop on Learning over Multiple Contexts (LMCE) at ECML-PKDD. 29","journal-title":"First International Workshop on Learning over Multiple Contexts (LMCE) at ECML-PKDD"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/e-Science.2009.11"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13222-016-0238-1"},{"key":"e_1_3_2_35_2","first-page":"1","article-title":"Sylvie Philipp-Foliguet, and Fr\u00e9d\u00e9ric Precioso. 2007. 3D content-based retrieval in artwork databases","author":"Gorisse David","unstructured":"David Gorisse, Matthieu Cord, Michel Jordan, Sylvie Philipp-Foliguet, and Fr\u00e9d\u00e9ric Precioso. 2007. 3D content-based retrieval in artwork databases. In 2007 3DTV Conference. IEEE, 1\u20134.","journal-title":"2007 3DTV Conference"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2018.8451546"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2008.08.001"},{"key":"e_1_3_2_38_2","first-page":"60730Z","article-title":"Multimedia for art retrieval (m4art)","author":"Broek Egon L. van den","year":"2006","unstructured":"Egon L. van den Broek, Thijs Kok, Theo E. Schouten, and Eduard Hoenkamp. 2006. Multimedia for art retrieval (m4art). In Multimedia Content Analysis, Management, and Retrieval 2006, Vol. 6073. International Society for Optics and Photonics, 60730Z.","journal-title":"Multimedia Content Analysis, Management, and Retrieval 2006, Vol. 6073. International Society for Optics and Photonics"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-020-09995-z"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigMM.2017.72"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2018.01.027"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.323"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"issue":"10","key":"e_1_3_2_44_2","first-page":"1345","article-title":"A survey on transfer learning","volume":"22","author":"Jialin Pan Sinno","year":"2009","unstructured":"Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering 22, 10 (2009), 1345\u20131359.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88690-7_6"},{"key":"e_1_3_2_46_2","first-page":"806","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","author":"Sharif Razavian Ali","year":"2014","unstructured":"Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. CNN features off-the-shelf: An astounding baseline for recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 806\u2013813."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.222"},{"key":"e_1_3_2_48_2","first-page":"3320","article-title":"How transferable are features in deep neural networks","volume":"27","author":"Yosinski Jason","year":"2014","unstructured":"Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks. Advances in Neural Information Processing Systems 27 (2014), 3320\u20133328.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2016.7533051"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00872"},{"key":"e_1_3_2_51_2","first-page":"105721F","volume-title":"13th International Conference on Medical Information Processing and Analysis","author":"Kergosien Yannick L.","year":"2017","unstructured":"Yannick L. Kergosien and Daniel Racoceanu. 2017. Semantic knowledge for histopathological image analysis: From ontologies to processing portals and deep learning. In 13th International Conference on Medical Information Processing and Analysis, Vol. 10572. International Society for Optics and Photonics, 105721F."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.5555\/646130.679965"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2003.1246729"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1142\/S0218001420400029"}],"container-title":["Journal on Computing and Cultural Heritage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3485844","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3485844","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:44Z","timestamp":1750191104000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3485844"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,16]]},"references-count":53,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,9,30]]}},"alternative-id":["10.1145\/3485844"],"URL":"https:\/\/doi.org\/10.1145\/3485844","relation":{},"ISSN":["1556-4673","1556-4711"],"issn-type":[{"value":"1556-4673","type":"print"},{"value":"1556-4711","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,16]]},"assertion":[{"value":"2020-11-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}