{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T22:26:57Z","timestamp":1785364017545,"version":"3.55.0"},"reference-count":30,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T00:00:00Z","timestamp":1759881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>With the rise of large language models (LLMs), collaborative storytelling in virtual agents or chatbots has gained popularity. Despite storytelling has long been employed in social robotics as a means to educate, entertain, and persuade audiences, the integration of LLMs into such platforms remains largely unexplored. This paper presents the initial steps for a novel multimodal collaborative storytelling system in which users co-create stories with the social robot Pepper through natural language interaction and by presenting physical objects. The robot employs a YOLO-based vision system to recognize these objects and seamlessly incorporate them into the narrative. Story generation and adaptation are handled autonomously using the Llama model in a zero-shot setting, aiming to assess the usability and maturity of such models in interactive storytelling. To enhance immersion, the robot performs the final story using expressive gestures, emotional cues, and speech modulation. User feedback, collected through questionnaires and semi-structured interviews, indicates a high level of acceptance.<\/jats:p>","DOI":"10.3389\/frobt.2025.1662819","type":"journal-article","created":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T09:01:30Z","timestamp":1759914090000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Exploring multimodal collaborative storytelling with Pepper: a preliminary study with zero-shot LLMs"],"prefix":"10.3389","volume":"12","author":[{"given":"Unai","family":"Zabala","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan","family":"Echevarria","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Igor","family":"Rodriguez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elena","family":"Lazkano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,8]]},"reference":[{"key":"B1","first-page":"223","article-title":"Storytelling robots for training of emotion recognition in children with autism; opinions from experts","volume-title":"International conference on human-computer interaction","author":"Alimardani","year":"2021"},{"key":"B2","first-page":"126","article-title":"Integrate digital storytelling in education","volume":"6","author":"Alismail","year":"2015","journal-title":"J. Educ. Pract."},{"key":"B3","doi-asserted-by":"publisher","first-page":"e13159","DOI":"10.1111\/jar.13159","article-title":"Tangible digital collaborative storytelling in adolescents with intellectual disability and neurodevelopmental disorders","volume":"37","author":"Alonso-Campuzano","year":"2024","journal-title":"J. Appl. Res. Intellect. Disabil."},{"key":"B4","doi-asserted-by":"publisher","first-page":"eaat5954","DOI":"10.1126\/scirobotics.aat5954","article-title":"Social robots for education: a review","volume":"3","author":"Belpaeme","year":"2018","journal-title":"Sci. robotics"},{"key":"B5","doi-asserted-by":"publisher","first-page":"142","DOI":"10.3991\/ijet.v16i13.22187","article-title":"Using digital storytelling as a strategy for enhancing efl writing skills","volume":"16","author":"Castillo-Cuesta","year":"2021","journal-title":"Int. J. Emerg. Technol. Learn."},{"key":"B6","doi-asserted-by":"publisher","first-page":"793","DOI":"10.1007\/s11423-023-10209-0","article-title":"Effects of robot-assisted digital storytelling on hospitalized children\u2019s communication during the covid-19 pandemic","volume":"71","author":"Chang","year":"2023","journal-title":"Educ. Technol. Res. Dev."},{"key":"B7","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1097\/00006223-200309000-00008","article-title":"Storytelling as a teaching technique","volume":"28","author":"Davidhizar","year":"2003","journal-title":"Nurse Educ."},{"key":"B8","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1145\/3536221.3556578","article-title":"Creativebot: a creative storyteller robot to stimulate creativity in children","volume-title":"Proceedings of the 2022 international conference on multimodal interaction","author":"Elgarf","year":"2022"},{"key":"B9","first-page":"027","article-title":"Exploring the world of robot-assisted digital storytelling: trends, models, and educational implications","volume":"23","author":"Essien","year":"2024","journal-title":"J. Inf. Technol. Educ."},{"key":"B10","doi-asserted-by":"publisher","first-page":"e13216","DOI":"10.2196\/13216","article-title":"Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology, and psychotherapy","volume":"21","author":"Fiske","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"B11","doi-asserted-by":"crossref","first-page":"865","DOI":"10.1109\/VRW62533.2024.00229","article-title":"Aisop: exploring immersive vr storytelling leveraging generative ai","volume-title":"2024 IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW)","author":"Gatti","year":"2024"},{"key":"B12","first-page":"208","article-title":"Prompt engineering for narrative choice generation","volume-title":"International conference on interactive digital storytelling","author":"Harmon","year":"2023"},{"key":"B13","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J. Chiropr. Med."},{"key":"B14","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/978-3-540-89350-9_6","article-title":"Construction and evaluation of a user experience questionnaire","volume-title":"HCI and usability for education and work: 4Th symposium of the workgroup human-computer interaction and usability engineering of the Austrian computer Society, USAB 2008, Graz, Austria, November 20-21, 2008. Proceedings 4","author":"Laugwitz","year":"2008"},{"key":"B15","doi-asserted-by":"publisher","first-page":"104827","DOI":"10.1016\/j.compedu.2023.104827","article-title":"A robot-based digital storytelling approach to enhancing efl learners\u2019 multimodal storytelling ability and narrative engagement","volume":"201","author":"Liang","year":"2023","journal-title":"Comput. & Educ."},{"key":"B16","doi-asserted-by":"publisher","first-page":"87","DOI":"10.48550\/arXiv.2306.00978","article-title":"Awq: Activation-aware weight quantization for on-device llm compression and acceleration","volume":"6","author":"Lin","year":"2024","journal-title":"Proc. Mach. Learn. Syst."},{"key":"B17","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/978-3-319-22668-2_19","article-title":"Using affinity diagrams to evaluate interactive prototypes","volume-title":"Human-Computer Interaction\u2013INTERACT 2015: 15th IFIP TC 13 International Conference, Bamberg, Germany, September 14-18, 2015, Proceedings, Part II 15","author":"Lucero","year":"2015"},{"key":"B18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3424636.3426903","article-title":"Collaborative storytelling with large-scale neural language models","volume-title":"Proceedings of the 13th acm siggraph conference on motion, interaction and games","author":"Nichols","year":"2020"},{"key":"B19","first-page":"1903","article-title":"Collaborative storytelling with social robots","volume-title":"2021 IEEE\/RSJ international conference on Intelligent Robots and Systems IROS","author":"Nichols","year":"2021"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2111.09453","article-title":"Robertuito: a pre-trained language model for social media text in spanish","author":"P\u00e9rez","year":"2021","journal-title":"arXiv Prepr. arXiv:2111.09453"},{"key":"B21","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/B978-0-12-558701-3.50007-7","article-title":"A general psychoevolutionary theory of emotion","volume-title":"Theories of emotion","author":"Plutchik","year":"1980"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s12369-021-00851-0","article-title":"Potential applications of social robots in robot-assisted interventions for social anxiety","volume":"14","author":"Rasouli","year":"2022","journal-title":"Int. J. Soc. Robotics"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2406.12665","article-title":"Collabstory: Multi-llm collaborative story generation and authorship analysis","author":"Venkatraman","year":"2024","journal-title":"arXiv Prepr. arXiv:2406"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3613905.3651026","article-title":"Building llm-based ai agents in social virtual reality","author":"Wan","year":"2024","journal-title":"Ext. Abstr. CHI Conf. Hum. Factors Comput. Syst."},{"key":"B25","volume-title":"Storytelling with children","author":"Wright","year":"1995"},{"key":"B26","volume-title":"Mystoryknight: a characterdrawing driven storytelling system using llm hallucinations","author":"Yotam","year":"2024"},{"key":"B27","first-page":"137","article-title":"Can a social robot learn to gesticulate just by observing humans?","volume-title":"Workshop of physical agents","author":"Zabala","year":"2020"},{"key":"B28","doi-asserted-by":"crossref","first-page":"1209","DOI":"10.1109\/RO-MAN53752.2022.9900532","article-title":"Towards an automatic generation of natural gestures for a storyteller robot","volume-title":"2022 31st IEEE international conference on robot and human interactive communication (RO-MAN)","author":"Zabala","year":"2022"},{"key":"B29","first-page":"1","article-title":"Mathemyths: leveraging large language models to teach mathematical language through child-ai co-creative storytelling","volume-title":"Proceedings of the 2024 CHI conference on human factors in computing systems","author":"Zhang","year":""},{"key":"B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3658134","article-title":"Semantic gesticulator: Semantics-aware co-speech gesture synthesis","volume":"43","author":"Zhang","year":"","journal-title":"ACM Trans. Graph. (TOG)"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1662819\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T09:01:34Z","timestamp":1759914094000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1662819\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,8]]},"references-count":30,"alternative-id":["10.3389\/frobt.2025.1662819"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1662819","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,8]]},"article-number":"1662819"}}