{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,23]],"date-time":"2025-10-23T11:18:18Z","timestamp":1761218298386,"version":"build-2065373602"},"reference-count":61,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2020,10,28]],"date-time":"2020-10-28T00:00:00Z","timestamp":1603843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>In recent years the information user needs have been changed due to the heterogeneity of web contents which increasingly involve in multimedia contents. Although modern search engines provide visual queries, it is not easy to find systems that allow searching from a particular domain of interest and that perform such search by combining text and visual queries. Different approaches have been proposed during years and in the semantic research field many authors proposed techniques based on ontologies. On the other hand, in the context of image retrieval systems techniques based on deep learning have obtained excellent results. In this paper we presented novel approaches for image semantic retrieval and a possible combination for multimedia document analysis. Several results have been presented to show the performance of our approach compared with literature baselines.<\/jats:p>","DOI":"10.3390\/fi12110183","type":"journal-article","created":{"date-parts":[[2020,10,28]],"date-time":"2020-10-28T11:43:06Z","timestamp":1603885386000},"page":"183","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["A Knowledge-Driven Multimedia Retrieval System Based on Semantics and Deep Features"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7003-4781","authenticated-orcid":false,"given":"Antonio Maria","family":"Rinaldi","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio, 21, 80125 Napoli, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8732-1733","authenticated-orcid":false,"given":"Cristiano","family":"Russo","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio, 21, 80125 Napoli, Italy"}]},{"given":"Cristian","family":"Tommasino","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Information Technologies, University of Naples Federico II, Via Claudio, 21, 80125 Napoli, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,28]]},"reference":[{"key":"ref_1","unstructured":"Baeza-Yates, R., and Ribeiro-Neto, B. (2011). Modern Information Retrieval: The Concepts and Technology Behind Search, Addison-Wesley Publishing Company. [2nd ed.]."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1552291.1552293","article-title":"An ontology-driven approach for semantic information retrieval on the web","volume":"9","author":"Rinaldi","year":"2009","journal-title":"ACM Trans. Internet Technol. (TOIT)"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1002\/asi.4630260604","article-title":"Relevance: A review of and a framework for the thinking on the notion in information science","volume":"26","author":"Saracevic","year":"1975","journal-title":"J. Am. Soc. Inf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1086\/601800","article-title":"Subjective versus objective relevance in bibliographic retrieval systems","volume":"56","author":"Swanson","year":"1986","journal-title":"Libr. Q."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1002\/(SICI)1097-4571(199210)43:9<602::AID-ASI3>3.0.CO;2-Q","article-title":"Psychological relevance and information science","volume":"43","author":"Harter","year":"1992","journal-title":"J. Am. Soc. Inf. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1293","DOI":"10.1002\/(SICI)1097-4571(1998)49:14<1293::AID-ASI7>3.0.CO;2-E","article-title":"Document representations and clues to document relevance","volume":"49","author":"Barry","year":"1998","journal-title":"J. Am. Soc. Inf. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1086\/602592","article-title":"The nature of relevance in information retrieval: An empirical study","volume":"63","author":"Park","year":"1993","journal-title":"Libr. Q."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1108\/EUM0000000007127","article-title":"Changes in relevance criteria and problem stages in task performance","volume":"56","author":"Vakkari","year":"2000","journal-title":"J. Doc."},{"key":"ref_9","unstructured":"Saracevic, T. (1996, January 13\u201316). Relevance reconsidered. Proceedings of the Second Conference on Conceptions of Library and Information Science (CoLIS 2), Seattle, WA, USA."},{"key":"ref_10","unstructured":"Miller, K. (2005). Communication Theories, Macgraw-Hill."},{"key":"ref_11","unstructured":"Danesi, M., and Perron, P. (1999). Analyzing Cultures: An Introduction and Handbook, Indiana University Press."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Rinaldi, A.M., and Russo, C. (2018, January 10\u201313). User-centered information retrieval using semantic multimedia big data. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.","DOI":"10.1109\/BigData.2018.8622613"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1109\/34.895972","article-title":"Content-based image retrieval at the end of the early years","volume":"22","author":"Smeulders","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, Y., Wang, J.Z., and Krovetz, R. (2003, January 4). An unsupervised learning approach to content-based image retrieval. Proceedings of the Seventh International Symposium on Signal Processing and Its Applications, Paris, France.","DOI":"10.1109\/ISSPA.2003.1224674"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1006\/jvci.1999.0413","article-title":"Image retrieval: Current techniques, promising directions, and open issues","volume":"10","author":"Rui","year":"1999","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.patcog.2006.04.045","article-title":"A survey of content-based image retrieval with high-level semantics","volume":"40","author":"Liu","year":"2007","journal-title":"Pattern Recognit."},{"key":"ref_17","unstructured":"Eakins, J., and Graham, M. (2020, September 02). Content-Based Image Retrieval. Available online: http:\/\/www.leeds.ac.uk\/educol\/documents\/00001240.htm."},{"key":"ref_18","first-page":"1","article-title":"A review of semantic similarity measures in wordnet","volume":"6","author":"Meng","year":"2013","journal-title":"Int. J. Hybrid Inf. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1108\/SR-04-2019-0092","article-title":"Review of image low-level feature extraction methods for content-based image retrieval","volume":"39","author":"Wang","year":"2019","journal-title":"Sens. Rev."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1109\/TPAMI.2017.2709749","article-title":"SIFT meets CNN: A decade survey of instance retrieval","volume":"40","author":"Zheng","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bosch, A., Zisserman, A., and Munoz, X. (2007, January 9\u201311). Representing shape with a spatial pyramid kernel. Proceedings of the 6th ACM International Conference on Image and Video Retrieval, Amsterdam, The Netherlands.","DOI":"10.1145\/1282280.1282340"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, S.Z., and Jain, A.K. (2015). Local Image Features. Encyclopedia of Biometrics, Springer.","DOI":"10.1007\/978-1-4899-7488-4"},{"key":"ref_23","unstructured":"(2020, September 01). Introduction to SIFT (Scale-Invariant Feature Transform). Available online: https:\/\/docs.opencv.org\/master\/da\/df5\/tutorial_py_sift_intro.html."},{"key":"ref_24","unstructured":"(2020, September 01). Introduction to SURF (Speeded-Up Robust Features). Available online: https:\/\/opencv-python-tutroals.readthedocs.io\/en\/latest\/py_tutorials\/py_feature2d\/py_surf_intro\/py_surf_intro.html."},{"key":"ref_25","unstructured":"(2020, September 01). ORB (Oriented FAST and Rotated BRIEF). Available online: https:\/\/docs.opencv.org\/3.4\/d1\/d89\/tutorial_py_orb.html."},{"key":"ref_26","unstructured":"Karami, E., Prasad, S., and Shehata, M. (2017). Image matching using SIFT, SURF, BRIEF and ORB: Performance comparison for distorted images. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Alom, M.Z., Taha, T.M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M.S., Hasan, M., Van Essen, B.C., Awwal, A.A., and Asari, V.K. (2019). A state-of-the-art survey on deep learning theory and architectures. Electronics, 8.","DOI":"10.3390\/electronics8030292"},{"key":"ref_28","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_31","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201322). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wan, J., Wang, D., Hoi, S.C.H., Wu, P., Zhu, J., Zhang, Y., and Li, J. (2014, January 18\u201319). Deep learning for content-based image retrieval: A comprehensive study. Proceedings of the 22nd ACM International Conference on Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/2647868.2654948"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"6424","DOI":"10.1109\/ACCESS.2018.2888856","article-title":"Local Feature Descriptor for Image Matching: A Survey","volume":"7","author":"Leng","year":"2019","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/TCSVT.2002.808079","article-title":"CBSA: Content-based soft annotation for multimodal image retrieval using Bayes point machines","volume":"13","author":"Chang","year":"2003","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1109\/TMM.2002.1017733","article-title":"Narrowing the semantic gap-improved text-based web document retrieval using visual features","volume":"4","author":"Zhao","year":"2002","journal-title":"IEEE Trans. Multimed."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wang, X.J., Ma, W.Y., Xue, G.R., and Li, X. (2004, January 10\u201316). Multi-model similarity propagation and its application for web image retrieval. Proceedings of the 12th Annual ACM International Conference on Multimedia, New York, NY, USA.","DOI":"10.1145\/1027527.1027746"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Clinchant, S., Ah-Pine, J., and Csurka, G. (2011, January 18\u201320). Semantic combination of textual and visual information in multimedia retrieval. Proceedings of the 1st ACM International Conference on Multimedia Retrieval, Trento, Italy.","DOI":"10.1145\/1991996.1992040"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Giordano, D., Kavasidis, I., Pino, C., and Spampinato, C. (2011, January 13\u201315). A semantic-based and adaptive architecture for automatic multimedia retrieval composition. Proceedings of the 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI), Madrid, Spain.","DOI":"10.1109\/CBMI.2011.5972542"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Buscaldi, D., and Zargayouna, H. (2013, January 28). Yasemir: Yet another semantic information retrieval system. Proceedings of the Sixth International Workshop on Exploiting Semantic Annotations in Information Retrieval, San Francisco, CA, USA.","DOI":"10.1145\/2513204.2513211"},{"key":"ref_41","unstructured":"Kannan, P., Bala, P.S., and Aghila, G. (2012, January 30\u201331). A comparative study of multimedia retrieval using ontology for semantic web. Proceedings of the IEEE-International Conference on Advances in Engineering, Science and Management (ICAESM-2012), Nagapattinam, Tamil Nadu, India."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.knosys.2012.07.021","article-title":"Towards a user based recommendation strategy for digital ecosystems","volume":"37","author":"Moscato","year":"2013","journal-title":"Knowl.-Based Syst."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Cao, J., Huang, Z., and Shen, H.T. (2017, January 23\u201327). Local deep descriptors in bag-of-words for image retrieval. Proceedings of the on Thematic Workshops of ACM Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/3126686.3127018"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3131288","article-title":"Semantic reasoning in zero example video event retrieval","volume":"13","author":"Boer","year":"2017","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl. (TOMM)"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Habibian, A., Mensink, T., and Snoek, C.G. (2014, January 18\u201319). Videostory: A new multimedia embedding for few-example recognition and translation of events. Proceedings of the 22nd ACM International Conference on Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/2647868.2654913"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/219717.219748","article-title":"WordNet: A Lexical Database for English","volume":"38","author":"Miller","year":"1995","journal-title":"Commun. ACM"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"27447","DOI":"10.1007\/s11042-018-5931-7","article-title":"Multimedia and geographic data integration for cultural heritage information retrieval","volume":"77","author":"Purificato","year":"2018","journal-title":"Multimed. Tools Appl."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/j.ins.2014.02.017","article-title":"A multimedia ontology model based on linguistic properties and audio-visual features","volume":"277","author":"Rinaldi","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Rinaldi, A.M., and Russo, C. (2018, January 25\u201328). A semantic-based model to represent multimedia big data. Proceedings of the 10th International Conference on Management of Digital EcoSystems, Tokyo, Japan.","DOI":"10.1145\/3281375.3281386"},{"key":"ref_50","unstructured":"(1970, January 01). Web Ontology Language. Available online: https:\/\/www.w3.org\/OWL\/."},{"key":"ref_51","unstructured":"(1970, January 01). ImageNet. Available online: http:\/\/www.image-net.org\/."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Lesk, M. (1986, January 8\u201311). Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. Proceedings of the 5th Annual International Conference on Systems Documentation, Toronto, ON, Canada.","DOI":"10.1145\/318723.318728"},{"key":"ref_53","unstructured":"Vasilescu, F., Langlais, P., and Lapalme, G. (2020, October 27). Evaluating Variants of the Lesk Approach for Disambiguating Words. Available online: http:\/\/www.iro.umontreal.ca\/~felipe\/Papers\/paper-lrec-2004.pdf."},{"key":"ref_54","unstructured":"Tolias, G., Sicre, R., and J\u00e9gou, H. (2015). Particular object retrieval with integral max-pooling of CNN activations. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1007\/BF01238023","article-title":"Combining classifiers: A theoretical framework","volume":"1","author":"Kittler","year":"1998","journal-title":"Pattern Anal. Appl."},{"key":"ref_57","unstructured":"(2020, September 01). 20 Newsgroups Scikit-Lean. Available online: https:\/\/scikit-learn.org\/0.15\/datasets\/twenty_newsgroups.html."},{"key":"ref_58","unstructured":"(2020, September 01). Visual Object Classes Challenge 2012 (VOC2012). Available online: http:\/\/host.robots.ox.ac.uk\/pascal\/VOC\/voc2012\/."},{"key":"ref_59","unstructured":"(2020, September 01). DMOZ Website. Available online: https:\/\/dmoz-odp.org\/."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/978-3-319-56157-8_4","article-title":"A multi-strategy approach for ontology reuse through matching and integration techniques","volume":"561","author":"Caldarola","year":"2018","journal-title":"Adv. Intell. Syst. Comput."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Rinaldi, A.M., and Russo, C. (February, January 31). A matching framework for multimedia data integration using semantics and ontologies. Proceedings of the 2018 IEEE 12th International Conference on Semantic Computing (ICSC), Laguna Hills, CA, USA.","DOI":"10.1109\/ICSC.2018.00074"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/12\/11\/183\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:29:35Z","timestamp":1760178575000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/12\/11\/183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,28]]},"references-count":61,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["fi12110183"],"URL":"https:\/\/doi.org\/10.3390\/fi12110183","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2020,10,28]]}}}