{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:44:56Z","timestamp":1782841496141,"version":"3.54.5"},"reference-count":0,"publisher":"ECMS","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,6,23]]},"abstract":"<jats:p>Effective emotion recognition is a key component of natural Human\u2013Robot Interaction (HRI), particularly for humanoid robots designed for social engagement. Although multimodal approaches have been studied in literature, their real integration into commercial social robots is still limited. This study presents a multimodal AI-based emotion recognition framework specifically designed for a real implementation on Robot platforms such as NAO developed by SoftBank Robotics.\n\nIn the existing literature, the recognition problem is usually addressed through unimodal approaches, relying either on audio-only or video-only information. This paper highlights that data correlation and fusion, even when implemented through a very simple strategy such as arithmetic mean score averaging, already lead to a measurable improvement in performance compared to unimodal models.\n\nFurthermore, the study demonstrates that adopting a more advanced fusion algorithm, such as CatBoost, results in additional performance gains due to its ability to learn adaptive weights when combining predictions from different models.\n\nFinally, a more sophisticated solution will show the improved performances obtained with an innovative multimodal fusion approach, particularly with respect to cross-entropy loss reduction, thereby suggesting improved probabilistic calibration and increased predictive confidence.<\/jats:p>","DOI":"10.7148\/2026-0419","type":"proceedings-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T16:55:14Z","timestamp":1782838514000},"page":"419-425","source":"Crossref","is-referenced-by-count":0,"title":["A multi-modal perception framework based on ann-catboost data-fusion for emotion detection in human-robot interaction (hri)"],"prefix":"10.7148","author":[{"given":"Giuseppe Lorenzo","family":"Di Prima","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"Collotta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"4144","published-online":{"date-parts":[[2026,6,23]]},"event":{"name":"40th ECMS International Conference on Modelling and Simulation"},"container-title":["ECMS 2026 Proceedings edited by Filippo Sanfilippo, Florenc Demrozi, Fabio Sgarbossa, Mohammad Poursina"],"original-title":[],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T16:55:16Z","timestamp":1782838516000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.scs-europe.net\/dlib\/2026\/ecms2026acceptedpapers\/0419_simai_ecms2026_0079.pdf"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,23]]},"references-count":0,"URL":"https:\/\/doi.org\/10.7148\/2026-0419","relation":{},"subject":[],"published":{"date-parts":[[2026,6,23]]}}}