{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:59:31Z","timestamp":1760151571527,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,4,8]],"date-time":"2022-04-08T00:00:00Z","timestamp":1649376000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Due to the exponential growth of medical information in the form of, e.g., text, images, Electrocardiograms (ECGs), X-rays, and multimedia, the management of a patient\u2019s data has become a huge challenge. In particular, the extraction of features from various different formats and their representation in a homogeneous way are areas of interest in medical applications. Multimedia Information Retrieval (MMIR) frameworks, like the Generic Multimedia Analysis Framework (GMAF), can contribute to solving this problem, when adapted to special requirements and modalities of medical applications. In this paper, we demonstrate how typical multimedia processing techniques can be extended and adapted to medical applications and how these applications benefit from employing a Multimedia Feature Graph (MMFG) and specialized, efficient indexing structures in the form of Graph Codes. These Graph Codes are transformed to feature relevant Graph Codes by employing a modified Term Frequency Inverse Document Frequency (TFIDF) algorithm, which further supports value ranges and Boolean operations required in the medical context. On this basis, various metrics for the calculation of similarity, recommendations, and automated inferencing and reasoning can be applied supporting the field of diagnostics. Finally, the presentation of these new facilities in the form of explainability is introduced and demonstrated. Thus, in this paper, we show how Graph Codes contribute new querying options for diagnosis and how Explainable Graph Codes can help to readily understand medical multimedia formats.<\/jats:p>","DOI":"10.3390\/jimaging8040104","type":"journal-article","created":{"date-parts":[[2022,4,8]],"date-time":"2022-04-08T12:11:14Z","timestamp":1649419874000},"page":"104","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Explainable Multimedia Feature Fusion for Medical Applications"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2100-7589","authenticated-orcid":false,"given":"Stefan","family":"Wagenpfeil","sequence":"first","affiliation":[{"name":"Faculty of Mathematics and Computer Science, University of Hagen, Universit\u00e4tsstrasse 1, 58097 Hagen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9715-1590","authenticated-orcid":false,"given":"Paul","family":"Mc Kevitt","sequence":"additional","affiliation":[{"name":"Academy for International Science & Research (AISR), Derry BT48 7JL, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4390-411X","authenticated-orcid":false,"given":"Abbas","family":"Cheddad","sequence":"additional","affiliation":[{"name":"Blekinge Institute of Technology, 371 79 Karlskrona, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias","family":"Hemmje","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Computer Science, University of Hagen, Universit\u00e4tsstrasse 1, 58097 Hagen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Wagenpfeil, S., McKevitt, P., and Hemmje, M. (2021). Towards Automated Semantic Explainability of Multimedia Feature Graphs. Information, 12, Available online: https:\/\/www.mdpi.com\/2078-2489\/12\/12\/502.","DOI":"10.3390\/info12120502"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wagenpfeil, S., McKevitt, P., and Hemmje, M. (2021). Fast and Effective Retrieval for Large Multimedia Collections. Big Data Cogn. Comput., 5, Available online: https:\/\/www.mdpi.com\/2504-2289\/5\/3\/33.","DOI":"10.3390\/bdcc5030033"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wagenpfeil, S., MvKevitt, P., and Hemmje, M. (2021). AI-Based Semantic Multimedia Indexing and Retrieval for Social Media on Smartphones. Information, 12, Available online: https:\/\/www.mdpi.com\/2078-2489\/12\/1\/43.","DOI":"10.3390\/info12010043"},{"key":"ref_4","unstructured":"Hemmje, M., Jordan, B., Pfenninger, M., Madsen, A., Murtagh, F., Kramer, M., Bouquett, P., and McIvor, T. (2020). Artificial Intelligence for Hospitals, Healthcare & Humanity (AI4H3), Research Institute for Telecommunication and Cooperation (FTK). R&D White Paper."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1314","DOI":"10.1158\/1055-9965.EPI-13-1240","article-title":"Enhancement of mammographic density measures in breast cancer risk prediction","volume":"23","author":"Cheddad","year":"2014","journal-title":"Cancer Epidemiol. Biomarkers Prev. (CEBP)"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cheddad, A., Czene, K., Eriksson, M., Li, J., Easton, D., Hall, P., and Humphreys, K. (2014). Area and volumetric density estimation in processed full-field digital mammograms for risk assessment of breast cancer. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0110690"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1186\/s13058-016-0761-x","article-title":"Novel mammographic image features differentiate between interval and screen-detected breast cancer: A case-case study","volume":"18","author":"Strand","year":"2016","journal-title":"Breast Cancer Res."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Holtzmann-Kevles, B. (1997). Naked to the Bone: Medical Imaging in the Twentieth Century, Rutgers University Press.","DOI":"10.1063\/1.881857"},{"key":"ref_9","unstructured":"Slichter, P. (2013). Principles of Magnetic Resonance, Springer."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Seeram, E. (2013). Computed Tomography-E-Book: Physical Principles, Clinical Applications, and Quality Control, Elsevier Health Sciences.","DOI":"10.1007\/978-1-4471-4703-9_5"},{"key":"ref_11","unstructured":"Dornheim Segmenter.com (2022, February 02). DICOM Viewer. Available online: https:\/\/www.dornheim-segmenter.com\/en\/products\/dicom-viewer-free\/."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lekamlage, C.D., Afzal, F., Westerberg, E., and Cheddad, A. (2020, January 6\u20139). Mini-DDSM: Mammography-based Automatic Age Estimation. Proceedings of the 3rd International Conference on Digital Medicine and Image Processing (DMIP 2020), Kyoto, Japan.","DOI":"10.1145\/3441369.3441370"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ichalkaranje, N. (2006). Intelligent Paradigms for Assistive and Preventive Healthcare, Springer.","DOI":"10.1007\/11418337"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1080\/07421222.1990.11517898","article-title":"Systems Development in InformationSystems Research","volume":"7","author":"Nunamaker","year":"1991","journal-title":"J. Manag. Inf. Syst."},{"key":"ref_15","unstructured":"Aberer, K., Choi, K., Noy, N., Allemang, D., Lee, K., Nixon, L., Goldbeck, J., Mika, P., Maynard, D., and Mozoguchi, R. The Semantic Web, Springer."},{"key":"ref_16","unstructured":"Subrahmanian, V.S. (1998). Principles of Multimedia Databases, Morgan Kaufman Publishing."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nixon, M. (2020). Feature Extraction and Image Processing for Computer Vision, Academic Press by Elsevier Ltd.","DOI":"10.1016\/B978-0-12-814976-8.00003-8"},{"key":"ref_18","first-page":"35","article-title":"Multimedia Indexing and Retrieval Techniques: A Review","volume":"58","author":"Bhute","year":"2012","journal-title":"Int. J. Comput. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Beyerer, C. (2017). Pattern Recognition\u2014Introduction, Walter de Gruyter GmbH & Co KG.","DOI":"10.1515\/9783110537949"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kurland, O. (2018, January 8\u201312). Fusion in Information Retrieval: SIGIR 2018 Half-Day Tutorial. Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, Ann Arbor, MI, USA.","DOI":"10.1145\/3209978.3210186"},{"key":"ref_21","unstructured":"Leveling, J. (August, January 28). Interpretation of coordinations, compound generation, and result fusion for query variants. Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1126004.1126005","article-title":"Content-Based Multimedia Information Retrieval: State of the Art and Challenges","volume":"2","author":"Lew","year":"2006","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl. (TOMM)"},{"key":"ref_23","unstructured":"Hernandez, C. (2010, January 28\u201330). Data Fusion and Label Weighting for Image Retrieval Based on Spatio-Conceptual Information. Proceedings of the Adaptivity, Personalization and Fusion of Heterogeneous Information, RIAO \u201910, Paris, France."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dufour, R. (2009, January 13\u201317). Local and global models for spontaneous speech segment detection and characterization. Proceedings of the 2009 IEEE Workshop on Automatic Speech Recognition & Understanding, Merano, Italy.","DOI":"10.1109\/ASRU.2009.5372928"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1109\/76.927421","article-title":"Overview of the MPEG-7 standard","volume":"11","author":"Chang","year":"2001","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_26","unstructured":"FFMpeg.org (2020, August 23). FFMPEG Documentation. Available online: http:\/\/ffmpeg.org."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mu, X. (2006, January 11\u201315). Content-Based Video Retrieval: Does Video\u2019s Semantic Visual Feature Matter?. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, WA, USA.","DOI":"10.1145\/1148170.1148314"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Beierle, C. (2019). Methoden Wissensbasierter Systeme\u2014Grundlagen, Springer.","DOI":"10.1007\/978-3-658-27084-1"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bochman, A. (2005). Nonmonotonic Reasoning, World Scientific Publishing Company.","DOI":"10.1142\/9789812567802"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/0004-3702(88)90077-X","article-title":"A logical framework for default reasoning","volume":"36","author":"Poole","year":"1988","journal-title":"Artif. Intell."},{"key":"ref_31","unstructured":"Das, A. (2022, February 02). Knowledge Representation. Available online: https:\/\/www.sciencedirect.com\/science\/article\/pii\/B0122272404001027."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1080\/00048402.2017.1388826","article-title":"Theories of Aboutness","volume":"96","author":"Hawke","year":"2018","journal-title":"Australas. J. Philos."},{"key":"ref_33","first-page":"636","article-title":"Graph Codes-2D Projections of Multimedia Feature Graphs for Fast and Effective Retrieval","volume":"15","author":"Wagenpfeil","year":"2021","journal-title":"Int. J. Comput. Inf. Eng."},{"key":"ref_34","unstructured":"Sciencedirect.com (2020, August 23). Adjacency Matrix. Available online: https:\/\/www.sciencedirect.com\/topics\/mathematics\/adjacency-matrix."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"21662","DOI":"10.1109\/ACCESS.2019.2897849","article-title":"The Use of Ontology in Retrieval: A Study on Textual","volume":"7","author":"Asim","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Domingue, J. (2011). Introduction to the Semantic Web Technologies, Springer.","DOI":"10.1007\/978-3-540-92913-0"},{"key":"ref_37","unstructured":"W3C (2022, February 02). SKOS Simple Knowledge Organisation System. Available online: https:\/\/www.w3.org\/2004\/02\/skos\/."},{"key":"ref_38","unstructured":"Aho, A. (1998). Compilerbau, Oldenbourg Wissenschaftsverlag."},{"key":"ref_39","unstructured":"Hauser, R. (2000). Principles of Computer Linguistics, Springer."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1007\/s10278-013-9657-9","article-title":"Medical Image File Formats","volume":"27","author":"Larobina","year":"2014","journal-title":"J. Digit. Imaging"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Norman, S. (1985). User Centered System Design\u2014New Perspectives on Human-Computer Interaction, Taylor & Francis.","DOI":"10.1201\/b15703"},{"key":"ref_42","unstructured":"Silge, J., and Robinson, D. (2022). Text Mining with R: A Tidy Approach, O\u2019Reilly Media. Available online: https:\/\/www.tidytextmining.com\/tfidf.html."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Eljasik-Swoboda, T. (2020). Explainable and Transferrable Text Categorization, Springer International Publishing.","DOI":"10.1007\/978-3-030-54595-6_1"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Eljasik-Swoboda, T. (2018, January 26\u201328). No Target Function Classifier: Fast Unsupervised Text Categorization Using Semantic Spaces. Proceedings of the 7th International Conference on Data Science, Technology and Applications-DATA, Porto, Portugal.","DOI":"10.5220\/0006847000350046"},{"key":"ref_45","unstructured":"Duttenh\u00f6fer, A., Wagenpfeil, S., and Hemmje, M. (2021, January 11\u201313). Supporting Argument Strength by Integrating Semantic Multimedia Feature Detection with Emerging Argument Extraction. Proceedings of the Argstrength Workshop, Hagen, Germany."},{"key":"ref_46","unstructured":"Apache Software Foundation (2022, February 02). Reasoners and Rule Engines: Jena Inference Support. Available online: https:\/\/jena.apache.org\/documentation\/inference\/."},{"key":"ref_47","first-page":"25","article-title":"Intelligent Information Retrieval: Whose Intelligence?","volume":"96","author":"Belkin","year":"1996","journal-title":"Ing\u00e9nierie Des Syst\u00e8mes D\u2019information-ISI"},{"key":"ref_48","unstructured":"Wagenpfeil, S. (2022, February 02). Github Repository of GMAF and MMFVG. Available online: https:\/\/github.com\/stefanwagenpfeil\/GMAF\/."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Spencer, R. (2000, January 1\u20136). The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI \u201900), The Hague, The Netherlands.","DOI":"10.1145\/332040.332456"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/4\/104\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:50:43Z","timestamp":1760136643000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/4\/104"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,8]]},"references-count":49,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["jimaging8040104"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8040104","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2022,4,8]]}}}