{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T16:50:34Z","timestamp":1774284634166,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"name":"MUSMA project","award":["PRIN 2022, CUP G53D23002930006"],"award-info":[{"award-number":["PRIN 2022, CUP G53D23002930006"]}]},{"name":"Department Strategic Plan (PSD) of the University of Udine \"Interdepartmental Project on Artificial Intelligence\""}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2026,4,30]]},"abstract":"<jats:p>\n                    Metaverse environments offer immersive, multimedia-rich experiences with growing relevance in education, entertainment, and cultural applications. The ability to grasp the contents of these environments, consisting of many rooms or subspaces, is a key building block for implementing Metaverse retrieval systems. However, current methods remain limited, as they are not designed to separate local (e.g., individual multimedia elements, and room-level details) from global, Metaverse-level semantics. Moreover, public datasets on similar topics do not capture the complexity of multi-room environments filled with multimedia contents. Our contributions are twofold. First, we introduce HiCALM, a hierarchical Metaverse retrieval framework built around the structural hierarchy of Metaverse environments. HiCALM models Metaverses in a bottom-up way, progressively grasping local and global semantics. To reduce the gap with textual data and achieve high-performance text-to-Metaverse retrieval, a novel cross-modal hierarchical loss supervises the process by teaching the model to associate the hierarchical visual features with textual information, also extracted hierarchically. Second, to overcome the absence of suitable datasets for the task, we present Museums3k, a large-scale dataset of 3,000 virtual museums annotated with detailed descriptions, each composed of multiple rooms populated with diverse multimedia elements; and GamingMV, a smaller dataset with data coming from 239 gaming-related real-world Metaverses. Through extensive quantitative and qualitative experiments, we show that HiCALM achieves considerable improvements in text-to-Metaverse retrieval, obtaining up to 95.0% R@1 and 60.0%\n                    <jats:inline-formula content-type=\"math\/tex\">\n                      <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\text{nDCG}_{3}@10\\)<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    on Museums3k (more than +40% R@1 and +11% nDCG, compared to existing solutions), and up to 62.0% R@1 and 62.6%\n                    <jats:inline-formula content-type=\"math\/tex\">\n                      <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\text{nDCG}_{3}@10\\)<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    on GamingMV (more than +23% R@1 and +5% nDCG).\n                  <\/jats:p>","DOI":"10.1145\/3799427","type":"journal-article","created":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T14:03:18Z","timestamp":1772114598000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Retrieving Relevant Metaverses Using Hierarchical Features"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4482-0479","authenticated-orcid":false,"given":"Ali","family":"Abdari","sequence":"first","affiliation":[{"name":"University of Udine, Udine, Italy and University of Naples Federico II, Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6325-9066","authenticated-orcid":false,"given":"Alex","family":"Falcon","sequence":"additional","affiliation":[{"name":"University of Udine, Udine, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4269-4501","authenticated-orcid":false,"given":"Giuseppe","family":"Serra","sequence":"additional","affiliation":[{"name":"University of Udine, Udine, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,23]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"4293","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops","author":"Abdari A.","year":"2023","unstructured":"A. Abdari, A. Falcon, and G. Serra. 2023. Farmare: A furniture-aware multi-task methodology for recommending apartments based on the user interests. In Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, 4293\u20134303."},{"key":"e_1_3_2_3_2","first-page":"1","volume-title":"Proceedings of the 1st International Workshop on Deep Multimodal Learning for Information Retrieval","author":"Abdari A.","year":"2023","unstructured":"A. Abdari, A. Falcon, and G. Serra. 2023. Metaverse retrieval: Finding the best metaverse environment via language. In Proceedings of the 1st International Workshop on Deep Multimodal Learning for Information Retrieval, 1\u20139."},{"key":"e_1_3_2_4_2","first-page":"1043","volume-title":"Proceedings of the ACM International Conference on Multimedia Retrieval","author":"Abdari A.","year":"2024","unstructured":"A. Abdari, A. Falcon, and G. Serra. 2024. Adoctera: Adaptive optimization constraints for improved text-guided retrieval of apartments. In Proceedings of the ACM International Conference on Multimedia Retrieval, 1043\u20131050"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1007\/978-3-031-53311-2_35","volume-title":"International Conference on Multimedia Modeling","author":"Abdari A.","year":"2024","unstructured":"A. Abdari, A. Falcon, and G. Serra. 2024. A language-based solution to enable metaverse retrieval. In International Conference on Multimedia Modeling. Springer, 477\u2013488."},{"key":"e_1_3_2_6_2","first-page":"8938","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Achlioptas P.","year":"2019","unstructured":"P. Achlioptas, T. Lamb, V. Hernandez, L. J. Guibas, C. Potts, and O. Litany. 2019. Shapeglot: Learning language for shape discrimination. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), 8938\u20138947."},{"key":"e_1_3_2_7_2","unstructured":"T. Alsop. 2024. Mobile Augmented Reality (ar) Users Worldwide from 2023 to 2028. Retrieved August 9 2025 from https:\/\/www.statista.com\/statistics\/1098630\/global-mobile-augmented-reality-ar-users\/"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"e00259","DOI":"10.1016\/j.daach.2023.e00259","article-title":"Transforming computed tomography scans into a full-immersive virtual museum for the Antikythera mechanism","volume":"28","author":"Anastasovitis E.","year":"2023","unstructured":"E. Anastasovitis and M. Roumeliotis. 2023. Transforming computed tomography scans into a full-immersive virtual museum for the Antikythera mechanism. Digital Applications in Archaeology and Cultural Heritage 28 (2023), e00259.","journal-title":"Digital Applications in Archaeology and Cultural Heritage"},{"key":"e_1_3_2_9_2","unstructured":"H. V. Arts. 2019. Mona Lisa beyond the glass: The Louvre\u2019s first virtual reality experience. Retrieved from https:\/\/www.louvre.fr\/en\/explore\/life-at-the-museum\/mona-lisa-beyond-the-glass-the-louvre-s-first-virtual-reality-experience"},{"issue":"3","key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"e2154","DOI":"10.1002\/cav.2154","article-title":"3D scanning digital models for virtual museums","volume":"34","author":"Barszcz M.","year":"2023","unstructured":"M. Barszcz, K. Dziedzic, M. Skublewska-Paszkowska, and P. Powroznik. 2023. 3D scanning digital models for virtual museums. Computer Animation and Virtual Worlds 34, 3-4 (2023), e2154.","journal-title":"Computer Animation and Virtual Worlds"},{"key":"e_1_3_2_11_2","unstructured":"A. X. Chang T. Funkhouser L. Guibas P. Hanrahan Q. Huang Z. Li S. Savarese M. Savva S. Song H. Su et al. 2015. Shapenet: An information-rich 3d model repository. arXiv:1512.03012. Retrieved from https:\/\/arxiv.org\/abs\/1512.03012"},{"key":"e_1_3_2_12_2","first-page":"100","volume-title":"Proceedings of Asian Conference on Computer Vision (ACCV)","author":"Chen K.","year":"2018","unstructured":"K. Chen, C. B. Choy, M. Savva, A. X. Chang, T. Funkhouser, and S. Savarese. 2018. Text2shape: Generating shapes from natural language by learning joint embeddings. In Proceedings of Asian Conference on Computer Vision (ACCV), 100\u2013116."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TLT.2023.3277952"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00321"},{"key":"e_1_3_2_15_2","unstructured":"M. Consulting. 2022. The Metaverse Reaches 600m Monthly Active Users. Retrieved August 9 2025 from https:\/\/www.metaversed.consulting\/blog\/the-metaverse-reaches-600m-monthly-active-users"},{"issue":"2","key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1111\/1095-9270.12298","article-title":"From boatyard to museum: 3D laser scanning and digital modelling of the Qatar museums watercraft collection, Doha, Qatar","volume":"47","author":"Cooper J. P.","year":"2018","unstructured":"J. P. Cooper, A. Wetherelt, C. Zazzaro, and M. Eyre. 2018. From boatyard to museum: 3D laser scanning and digital modelling of the Qatar museums watercraft collection, Doha, Qatar. International Journal of Nautical Archaeology 47, 2 (2018), 419\u2013442.","journal-title":"International Journal of Nautical Archaeology"},{"key":"e_1_3_2_17_2","first-page":"5828","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Dai A.","year":"2017","unstructured":"A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nie\u00dfner. 2017. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5828\u20135839."},{"key":"e_1_3_2_18_2","volume-title":"International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=YicbFdNTTy","author":"Dosovitskiy A.","year":"2020","unstructured":"A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=YicbFdNTTy"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","unstructured":"A. Ermilov A. Nevolina A. Pospelov and A. Kubaeva A. 2025. Steam Games Dataset 2025. DOI: 10.34740\/KAGGLE\/DSV\/11017460","DOI":"10.34740\/KAGGLE\/DSV\/11017460"},{"key":"e_1_3_2_20_2","first-page":"60","volume-title":"International Conference on Multimedia Modeling","author":"Falcon A.","year":"2024","unstructured":"A. Falcon, A. Abdari, and G. Serra. 2024. Hierartex: Hierarchical representations and art experts supporting the retrieval of museums in the metaverse. In International Conference on Multimedia Modeling. Springer, 60\u201373."},{"key":"e_1_3_2_21_2","first-page":"0","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV) Workshops","author":"Garcia N.","year":"2018","unstructured":"N. Garcia and G. Vogiatzis. 2018. How to read paintings: Semantic art understanding with multi-modal retrieval. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, 0\u20130."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_23_2","first-page":"367","volume-title":"European Conference on Computer Vision","author":"Jin B.","year":"2024","unstructured":"B. Jin, Y. Zheng, P. Li, W. Li, Y. Zheng, S. Hu, X. Liu, J. Zhu, Z. Yan, H. Sun, et al. 2024. Tod3cap: Towards 3d dense captioning in outdoor scenes. In European Conference on Computer Vision. Springer, 367\u2013384."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3631134"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.culher.2016.06.007"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.83"},{"key":"e_1_3_2_27_2","first-page":"12888","volume-title":"International Conference on Machine Learning","author":"Li J.","year":"2022","unstructured":"J. Li, D. Li, C. Xiong, and S. Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning. PMLR, 12888\u201312900."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2022.07.028"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.3390\/heritage6100353"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-022-13365-2"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3071687"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","first-page":"163313","DOI":"10.1109\/ACCESS.2025.3611348","article-title":"M2d-clap: Exploring general-purpose audio-language representations beyond clap","author":"Niizumi D.","year":"2025","unstructured":"D. Niizumi, D. Takeuchi, M. Yasuda, B. T. Nguyen, Y. Ohishi, and N. Harada. 2025. M2d-clap: Exploring general-purpose audio-language representations beyond clap. IEEE Access 13 (2025), 163313\u2013163330.","journal-title":"IEEE Access"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dim.2023.100057"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00937"},{"key":"e_1_3_2_35_2","first-page":"652","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Qi C. R.","year":"2017","unstructured":"C. R. Qi, H. Su, K. Mo, and L. J. Guibas. 2017. PointNet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 652\u2013660."},{"key":"e_1_3_2_36_2","unstructured":"Z. Qi Z. Zhang Y. Fang J. Wang and H. Zhao. 2025. Gpt4scene: Understand 3d scenes from videos with vision-language models. arXiv:2501.01428. Retrieved from https:\/\/arxiv.org\/abs\/2501.01428"},{"key":"e_1_3_2_37_2","first-page":"8748","volume-title":"International Conference on Machine Learning","author":"Radford A.","year":"2021","unstructured":"A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PMLR, 8748\u20138763."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-021-10821-3"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.3390\/s20205851"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3681711"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.114"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2866698"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.01511"},{"key":"e_1_3_2_45_2","first-page":"6000","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani A.","year":"2017","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, \u0141. Kaiser, and I. Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, 6000\u20136010."},{"key":"e_1_3_2_46_2","first-page":"20536","article-title":"Mvp-n: A dataset and benchmark for real-world multi-view object classification","volume":"35","author":"Wang R.","year":"2022","unstructured":"R. Wang, J. Wang, T. S. Kim, J. Kim, and H. J. Lee. 2022. Mvp-n: A dataset and benchmark for real-world multi-view object classification. Advances in Neural Information Processing Systems 35 (2022), 20536\u201320550.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00463"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP49357.2023.10095969"},{"key":"e_1_3_2_49_2","first-page":"1912","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Wu Z.","year":"2015","unstructured":"Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1912\u20131920."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00120"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/978-3-031-53302-0_7","volume-title":"International Conference on Multimedia Modeling","author":"Yu F.","year":"2024","unstructured":"F. Yu, Z. Wang, D. Li, P. Zhu, X. Liang, X. Wang, and M. Okumura. 2024. Towards cross-modal point cloud retrieval for indoor scenes. In International Conference on Multimedia Modeling. Springer, 89\u2013102."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2013.2280560"},{"key":"e_1_3_2_53_2","unstructured":"B. Zhang K. Li Z. Cheng Z. Hu Y. Yuan G. Chen S. Leng Y. Jiang H. Zhang X. Li P. Jin W. Zhang F. Wang L. Bing and D. Zhao. 2025. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv:2501.13106. Retrieved from https:\/\/arxiv.org\/abs\/2501.13106"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2023.3275873"},{"key":"e_1_3_2_55_2","first-page":"188","volume-title":"European Conference on Computer Vision","author":"Zhu Z.","year":"2024","unstructured":"Z. Zhu, Z. Zhang, X. Ma, X. Niu, Y. Chen, B. Jia, Z. Deng, S. Huang, and Q. Li. 2024. Unifying 3d vision-language understanding via promptable queries. In European Conference on Computer Vision. Springer, 188\u2013206."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10030363"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3799427","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T15:51:56Z","timestamp":1774281116000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3799427"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,23]]},"references-count":55,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4,30]]}},"alternative-id":["10.1145\/3799427"],"URL":"https:\/\/doi.org\/10.1145\/3799427","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,23]]},"assertion":[{"value":"2025-08-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}