{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T17:14:48Z","timestamp":1778346888948,"version":"3.51.4"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"20","license":[{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100007195","name":"Universit\u00e0 degli Studi di Napoli Federico II","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007195","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Automatic storytelling is a broad challenge in research contexts such as Natural Language Processing and Contend Based Image Analysis. Despite the considerable achievements of machine learning techniques in these research fields, combining different approaches to fill the gap between an automatic generated story and human handwriting is hard. This work proposes a novel storytelling framework in the Cultural Heritage domain. We developed our framework based on a Multimedia Knowledge Graph (MKG), a crucial point of our work. Furthermore, we populated our Multimedia Knowledge Graph with a focused crawler that employs deep learning techniques to recognise a multimedia object from web resources. Furthermore, we used a combined approach of deep learning techniques and Linked Open Data (LOD) to retrieve information about images and depicted figures using Instance Segmentation. The system has a dynamic, user-friendly interface that guides the user during the storytelling process. Finally, we evaluated the system from a qualitative and quantitative point of view.<\/jats:p>","DOI":"10.1007\/s11042-023-14398-x","type":"journal-article","created":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T03:18:39Z","timestamp":1679887119000},"page":"31625-31639","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A storytelling framework based on multimedia knowledge graph using linked open data and deep neural networks"],"prefix":"10.1007","volume":"82","author":[{"given":"Gianluigi","family":"Renzi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7003-4781","authenticated-orcid":false,"given":"Antonio M.","family":"Rinaldi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cristiano","family":"Russo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cristian","family":"Tommasino","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,3,22]]},"reference":[{"key":"14398_CR1","doi-asserted-by":"crossref","unstructured":"Aneja J, Deshpande A, Schwing AG (2018) Convolutional image captioning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5561\u20135570","DOI":"10.1109\/CVPR.2018.00583"},{"key":"14398_CR2","doi-asserted-by":"crossref","unstructured":"Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: a nucleus for a web of open data. In: The semantic web. Springer, pp 722\u2013735","DOI":"10.1007\/978-3-540-76298-0_52"},{"key":"14398_CR3","unstructured":"Bauer F, Kaltenb\u00f6ck M (2011) Linked open data: The essentials. Edition mono\/monochrom, Vienna 710"},{"issue":"11","key":"14398_CR4","doi-asserted-by":"publisher","first-page":"7577","DOI":"10.1007\/s11042-019-08252-2","volume":"79","author":"A Capuano","year":"2020","unstructured":"Capuano A, Rinaldi AM, Russo C (2020) An ontology-driven multimedia focused crawler based on linked open data and deep learning techniques. Multimed Tools Appl 79(11):7577\u20137598","journal-title":"Multimed Tools Appl"},{"key":"14398_CR5","unstructured":"DB A (2022) Arango DB. https:\/\/www.arangodb.com\/ Accessed 01 Mar 2022"},{"key":"14398_CR6","doi-asserted-by":"crossref","unstructured":"Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 319\u2013340","DOI":"10.2307\/249008"},{"key":"14398_CR7","unstructured":"Dictionaries OL (2021) Definition of storytelling. https:\/\/www.oxfordlearnersdictionaries.com\/definition\/english\/storytelling?q=storytelling Accessed 01 Mar 2022"},{"key":"14398_CR8","unstructured":"Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2012) The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http:\/\/www.pascal-network.org\/challenges\/VOC\/voc2012\/workshop\/index.html"},{"key":"14398_CR9","doi-asserted-by":"crossref","unstructured":"Fellbaum C (1998) Wordnet: An electronic lexical database. Bradford Books","DOI":"10.7551\/mitpress\/7287.001.0001"},{"key":"14398_CR10","doi-asserted-by":"crossref","unstructured":"Fensel D, \u015eim\u015fek U., Angele K, Huaman E, K\u00e4rle E., Panasiuk O, Toma I, Umbrich J, Wahler A (2020) Introduction: what is a knowledge graph?. In: Knowledge graphs. Springer, pp 1\u201310","DOI":"10.1007\/978-3-030-37439-6_1"},{"issue":"4","key":"14398_CR11","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1007\/s11023-020-09548-1","volume":"30","author":"L Floridi","year":"2020","unstructured":"Floridi L, Chiriatti M (2020) Gpt-3: Its nature, scope, limits, consequences. Mind Mach 30(4):681\u2013694","journal-title":"Mind Mach"},{"key":"14398_CR12","doi-asserted-by":"crossref","unstructured":"He K, Gkioxari G, Doll\u00e1r P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961\u20132969","DOI":"10.1109\/ICCV.2017.322"},{"issue":"2","key":"14398_CR13","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1109\/TPAMI.2018.2844175","volume":"42","author":"K He","year":"2020","unstructured":"He K, Gkioxari G, Dollar P, Girshick R (2020) Mask r-cnn. IEEE Trans Pattern Anal Mach Intell 42(2):386\u2013397","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"14398_CR14","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"14398_CR15","doi-asserted-by":"crossref","unstructured":"Hou C, Zhou C, Zhou K, Sun J, Xuanyuan S (2019) A survey of deep learning applied to story generation. In: International conference on smart computing and communication. Springer, pp 1\u201310","DOI":"10.1007\/978-3-030-34139-8_1"},{"key":"14398_CR16","doi-asserted-by":"crossref","unstructured":"Huang T-H, Ferraro F, Mostafazadeh N, Misra I, Agrawal A, Devlin J, Girshick R, He X, Kohli P, Batra D et al (2016) Visual storytelling. In: Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies, pp 1233\u20131239","DOI":"10.18653\/v1\/N16-1147"},{"key":"14398_CR17","doi-asserted-by":"crossref","unstructured":"Iacobacci I, Pilehvar MT, Navigli R (2016) Embeddings for word sense disambiguation: an evaluation study. In: Proceedings of the 54th annual meeting of the association for computational linguistics (Volume 1: Long Papers), pp 897\u2013907","DOI":"10.18653\/v1\/P16-1085"},{"issue":"5","key":"14398_CR18","doi-asserted-by":"publisher","first-page":"1012","DOI":"10.1002\/asi.21041","volume":"60","author":"B Kim","year":"2009","unstructured":"Kim B, Han I (2009) The role of trust belief and its antecedents in a community-driven knowledge environment. J Am Soc Inform Sci Technol 60(5):1012\u20131026","journal-title":"J Am Soc Inform Sci Technol"},{"key":"14398_CR19","doi-asserted-by":"crossref","unstructured":"Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Doll\u00e1r P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision. Springer, pp 740\u2013755","DOI":"10.1007\/978-3-319-10602-1_48"},{"issue":"5","key":"14398_CR20","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1016\/j.im.2004.04.003","volume":"42","author":"CS Lin","year":"2005","unstructured":"Lin CS, Wu S, Tsai RJ (2005) Integrating perceived playfulness into expectation-confirmation model for web portal context. Information & Management 42(5):683\u2013693","journal-title":"Information & Management"},{"key":"14398_CR21","unstructured":"Littlefield DF (1992) Evolution of alex posey\u2019s fus fixico persona. Studies in American Indian Literatures 136\u2013144"},{"issue":"4","key":"14398_CR22","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1023\/B:BTTJ.0000047600.45421.6d","volume":"22","author":"H Liu","year":"2004","unstructured":"Liu H, Singh P (2004) Conceptnet\u2014a practical commonsense reasoning tool-kit. BT Technol J 22(4):211\u2013226","journal-title":"BT Technol J"},{"key":"14398_CR23","doi-asserted-by":"crossref","unstructured":"Loganathan K, Kumar RS, Nagaraj V, John TJ (2020) Cnn & lstm using python for automatic image captioning. Materials Today: Proceedings","DOI":"10.1016\/j.matpr.2020.10.624"},{"key":"14398_CR24","doi-asserted-by":"crossref","unstructured":"Lukin SM, Hobbs R, Voss CR (2018) A pipeline for creative visual storytelling. arXiv:1807.08077","DOI":"10.18653\/v1\/W18-1503"},{"key":"14398_CR25","doi-asserted-by":"crossref","unstructured":"Melamud O, Goldberger J, Dagan I (2016) Context2vec: Learning generic context embedding with bidirectional lstm. In: Proceedings of the 20th SIGNLL conference on computational natural language learning, pp 51\u201361","DOI":"10.18653\/v1\/K16-1006"},{"issue":"11","key":"14398_CR26","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1145\/219717.219748","volume":"38","author":"GA Miller","year":"1995","unstructured":"Miller GA (1995) Wordnet: A lexical database for english. Commun ACM 38(11):39\u201341","journal-title":"Commun ACM"},{"key":"14398_CR27","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1162\/tacl_a_00179","volume":"2","author":"A Moro","year":"2014","unstructured":"Moro A, Raganato A, Navigli R (2014) Entity linking meets word sense disambiguation: a unified approach. Transactions of the Association for Computational Linguistics 2:231\u2013244","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"14398_CR28","unstructured":"Parasuraman A, Zeithaml VA, Berry L (1988) Servqual: A multiple-item scale for measuring consumer perceptions of service quality. 1988 64(1): 12\u201340"},{"issue":"8","key":"14398_CR29","first-page":"9","volume":"1","author":"A Radford","year":"2019","unstructured":"Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al. (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9","journal-title":"OpenAI blog"},{"key":"14398_CR30","unstructured":"Ricardo Baeza Y, Berthier Ribeiro N (2011) Modern information retrieval. Pearson"},{"issue":"3","key":"14398_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1552291.1552293","volume":"9","author":"AM Rinaldi","year":"2009","unstructured":"Rinaldi AM (2009) An ontology-driven approach for semantic information retrieval on the web. ACM Transactions on Internet Technology (TOIT) 9(3):1\u201324","journal-title":"ACM Transactions on Internet Technology (TOIT)"},{"issue":"3","key":"14398_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/IJIIT.2019070101","volume":"15","author":"AM Rinaldi","year":"2019","unstructured":"Rinaldi AM (2019) Web summarization and browsing through semantic tag clouds. International Journal of Intelligent Information Technologies (IJIIT) 15 (3):1\u201323","journal-title":"International Journal of Intelligent Information Technologies (IJIIT)"},{"key":"14398_CR33","doi-asserted-by":"crossref","unstructured":"Rinaldi AM, Russo C (2018) A semantic-based model to represent multimedia big data. In: Proceedings of the 10th international conference on management of digital EcoSystems, pp 31\u201338","DOI":"10.1145\/3281375.3281386"},{"issue":"11","key":"14398_CR34","doi-asserted-by":"publisher","first-page":"183","DOI":"10.3390\/fi12110183","volume":"12","author":"AM Rinaldi","year":"2020","unstructured":"Rinaldi AM, Russo C, Tommasino C (2020) A knowledge-driven multimedia retrieval system based on semantics and deep features. Future Internet 12 (11):183","journal-title":"Future Internet"},{"key":"14398_CR35","doi-asserted-by":"publisher","first-page":"114320","DOI":"10.1016\/j.eswa.2020.114320","volume":"169","author":"AM Rinaldi","year":"2021","unstructured":"Rinaldi AM, Russo C, Tommasino C (2021) A semantic approach for document classification using deep neural networks and multimedia knowledge graph. Expert Syst Appl 169:114320","journal-title":"Expert Syst Appl"},{"key":"14398_CR36","doi-asserted-by":"crossref","unstructured":"Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510\u20134520","DOI":"10.1109\/CVPR.2018.00474"},{"key":"14398_CR37","unstructured":"Shiri A (2004) Introduction to modern information retrieval. Library Review"},{"key":"14398_CR38","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556"},{"key":"14398_CR39","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2015) Rethinking the inception architecture for computer vision. 2015. arXiv:1512.00567","DOI":"10.1109\/CVPR.2016.308"},{"issue":"10","key":"14398_CR40","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1145\/2629489","volume":"57","author":"D Vrande\u010di\u0107","year":"2014","unstructured":"Vrande\u010di\u0107 D, Kr\u00f6tzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57(10):78\u201385","journal-title":"Commun ACM"},{"key":"14398_CR41","doi-asserted-by":"crossref","unstructured":"Wang X, Chen W, Wang Y-F, Wang WY (2018) No metrics are perfect: Adversarial reward learning for visual storytelling. arXiv:1804.09160","DOI":"10.18653\/v1\/P18-1083"},{"key":"14398_CR42","doi-asserted-by":"crossref","unstructured":"Yang P, Luo F, Chen P, Li L, Yin Z, He X, Sun X (2019) Knowledgeable storyteller: a commonsense-driven generative model for visual storytelling. In: IJCAI, pp 5356\u20135362","DOI":"10.24963\/ijcai.2019\/744"},{"key":"14398_CR43","unstructured":"Yang X, Tiddi I (2020) Creative storytelling with language models and knowledge graphs. In: CEUR Workshop proceedings, vol 2699"},{"key":"14398_CR44","doi-asserted-by":"crossref","unstructured":"Yang Z, Zhang Y-J, Huang Y et al (2017) Image captioning with object detection and localization. In: International conference on image and graphics. Springer, pp 109\u2013118","DOI":"10.1007\/978-3-319-71589-6_10"},{"key":"14398_CR45","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.patrec.2020.12.020","volume":"143","author":"Y Zhang","year":"2021","unstructured":"Zhang Y, Shi X, Mi S, Yang X (2021) Image captioning with transformer and knowledge graph. Pattern Recogn Lett 143:43\u201349","journal-title":"Pattern Recogn Lett"},{"key":"14398_CR46","unstructured":"Zhong Z, Ng HT (2010) It makes sense: a wide-coverage word sense disambiguation system for free text. In: Proceedings of the ACL 2010 system demonstrations, pp 78\u201383"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-14398-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-023-14398-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-14398-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T13:04:09Z","timestamp":1690376649000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-023-14398-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,22]]},"references-count":46,"journal-issue":{"issue":"20","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["14398"],"URL":"https:\/\/doi.org\/10.1007\/s11042-023-14398-x","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,22]]},"assertion":[{"value":"25 March 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 July 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers\u2019 bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}]}}