{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T11:52:26Z","timestamp":1783425146674,"version":"3.54.6"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T00:00:00Z","timestamp":1755129600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T00:00:00Z","timestamp":1755129600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005855","name":"Universidade Nova de Lisboa","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005855","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Inf Technol Tourism"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>As cities continue to attract a growing number of visitors, the development of a tailored chatbot catering to the tourists\u2019 unique needs becomes increasingly valuable. In this paper, we propose a new methodology for developing generative implementations of touristic chatbots and demonstrate its application through a prototype created specifically for Lisbon. Utilizing a web-scraped knowledge base with over 2000 website pages, the chatbot offers recommendations for tourist routes, events and places to visit, provides general information about Lisbon (including transportation) and engages with visitors. The initial evaluation was conducted using synthetic datasets\u2014one for question-answering and another for recommendations\u2014to guide experiments in data preprocessing, exploration of different ChatGPT models and improvements to the Retrieval-Augmented Generation pipeline. A subsequent evaluation was performed with experts, including professionals from Turismo de Portugal, focusing on three major areas (recommendation, information and engagement), and the chatbot achieved very strong results on all use cases. This paper contributes to the literature on chatbot development, emphasizing the benefits of advanced machine learning models in the tourism industry and highlighting the potential of iterative optimization and evaluation based on both synthetic and human survey data for downstream tasks. Likewise, the proposed approach can significantly enhance the tourism experience by offering personalized, engaging and efficient digital interactions, thereby improving overall visitor satisfaction and supporting sustainable destination management.<\/jats:p>","DOI":"10.1007\/s40558-025-00339-x","type":"journal-article","created":{"date-parts":[[2025,8,14]],"date-time":"2025-08-14T07:54:15Z","timestamp":1755158055000},"page":"1153-1183","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Lisa: a touristic chatbot for Lisbon"],"prefix":"10.1007","volume":"27","author":[{"given":"Miguel","family":"Cruz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bruno","family":"Jardim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miguel de Castro","family":"Neto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,8,14]]},"reference":[{"key":"339_CR1","unstructured":"Abeysinghe B, Circi R (2024) The challenges of evaluating LLM applications: an analysis of automated, human, and LLM-based approaches. http:\/\/arxiv.org\/abs\/2406.03339"},{"key":"339_CR2","doi-asserted-by":"publisher","unstructured":"Alhumoud S, Diab A, Aldukhai D, Alshalhoub A, Alabdullatif R, Alqahtany D, Alalyani M, Bin-Aqeel F (2022) Rahhal: a tourist Arabic Chatbot. Proceedings \u2013\u20092nd International Conference of Smart Systems and Emerging Technologies (SMARTTECH), 66\u201373. https:\/\/doi.org\/10.1109\/SMARTTECH54121.2022.00028","DOI":"10.1109\/SMARTTECH54121.2022.00028"},{"key":"339_CR3","doi-asserted-by":"publisher","unstructured":"Amalia A, Suprayogi MS (2019) Engaging millennials on using chatbot messenger for eco-tourism. Proc Third Int Conf Sustainable Innov 2019 \u2013 Humanity Educ Social Sci (IcoSIHESS 2019) 484\u2013487. https:\/\/doi.org\/10.2991\/icosihess-19.2019.84","DOI":"10.2991\/icosihess-19.2019.84"},{"key":"339_CR4","doi-asserted-by":"publisher","unstructured":"Anand S, Abhishek Sai AM, Karthikeya M (2023) Chatbot Enabled Smart Tourism Service for Indian Cities: an AI Approach. 11th International Conference on Internet of Everything, Microwave Engineering, Communication and Networks, IEMECON 2023. https:\/\/doi.org\/10.1109\/IEMECON56962.2023.10092286","DOI":"10.1109\/IEMECON56962.2023.10092286"},{"key":"339_CR16","doi-asserted-by":"publisher","unstructured":"Arteaga D, Arenas J, Paz F, Tupia M, Bruzza M (2019) Design of information system architecture for the recommendation of tourist sites in the city of Manta, Ecuador through a Chatbot. 2019 14th Iberian Conference on Information Systems and Technologies (CISTI), 1\u20136. https:\/\/doi.org\/10.23919\/CISTI.2019.8760669","DOI":"10.23919\/CISTI.2019.8760669"},{"key":"339_CR5","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1145\/3644815.3644945","volume":"2022\u2013March","author":"S Barnett","year":"2024","unstructured":"Barnett S, Kurniawan S, Thudumu S, Brannelly Z, Abdelrazek M (2024) Seven failure points when engineering a retrieval augmented generation system. 2024 IEEE\/ACM 3rd Int Conf AI Eng \u2013 Softw Eng AI (CAIN) 2022\u2013March:194\u2013199. https:\/\/doi.org\/10.1145\/3644815.3644945","journal-title":"2024 IEEE\/ACM 3rd Int Conf AI Eng \u2013 Softw Eng AI (CAIN)"},{"issue":"24","key":"339_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/app10249082","volume":"10","author":"J Bon\u00e9","year":"2020","unstructured":"Bon\u00e9 J, Ferreira JC, Ribeiro R, Cadete G (2020) Disbot: a Portuguese disaster support dynamic knowledge chatbot. Appl Sci 10(24):1\u201320. https:\/\/doi.org\/10.3390\/app10249082","journal-title":"Appl Sci"},{"issue":"14","key":"339_CR7","doi-asserted-by":"publisher","first-page":"2345","DOI":"10.1080\/13683500.2023.2229480","volume":"27","author":"J Bulchand-Gidumal","year":"2024","unstructured":"Bulchand-Gidumal J, Secin W, O\u2019Connor E, P., Buhalis D (2024) Artificial intelligence\u2019s impact on hospitality and tourism marketing: exploring key themes and addressing challenges. Curr Issues Tourism 27(14):2345\u20132362. https:\/\/doi.org\/10.1080\/13683500.2023.2229480","journal-title":"Curr Issues Tourism"},{"key":"339_CR8","doi-asserted-by":"publisher","unstructured":"Caldarini G, Jaf S, McGarry K (2022) A literature survey of recent advances in chatbots. Information 13(1). https:\/\/doi.org\/10.3390\/info13010041","DOI":"10.3390\/info13010041"},{"key":"339_CR9","doi-asserted-by":"crossref","unstructured":"Calvaresi D, Ibrahim A, Calbimonte J, Schegg R, Fragni\u00e8re E, Schumacher MI (2021) The evolution of Chatbots in tourism: a systematic literature review. Information and Communication Technologies in Tourism 2021","DOI":"10.1007\/978-3-030-65785-7_1"},{"key":"339_CR10","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1016\/j.patrec.2020.01.003","volume":"131","author":"M Casillo","year":"2020","unstructured":"Casillo M, Clarizia F, D\u2019Aniello G, De Santo M, Lombardi M, Santaniello D (2020) CHAT-Bot: a cultural heritage aware teller-bot for supporting touristic experiences. Pattern Recognit Lett 131:234\u2013243. https:\/\/doi.org\/10.1016\/j.patrec.2020.01.003","journal-title":"Pattern Recognit Lett"},{"key":"339_CR11","doi-asserted-by":"publisher","first-page":"366","DOI":"10.1145\/3563359.3596665","volume":"363","author":"F Cena","year":"2023","unstructured":"Cena F, Mauro N, Ardissono L, Ferrero F, Ferrigno S, Rapp A, Mattutino C, Keller R (2023) CARES: an inclusive personalized touristic system for autism. UMAP 2023 - Adjun Proc 31st ACM Conf User Model Adaptation Personalization 363:366. https:\/\/doi.org\/10.1145\/3563359.3596665","journal-title":"UMAP 2023 - Adjun Proc 31st ACM Conf User Model Adaptation Personalization"},{"key":"339_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3641289","volume":"15","author":"Y Chang","year":"2023","unstructured":"Chang Y, Wang X, Wang J, Wu Y, Yang L, Zhu K, Chen H, Yi X, Wang C, Wang Y, Ye W, Zhang Y, Chang Y, Yu PS, Yang Q, Xie X (2023) A survey on evaluation of large Language models. ACM Trans Intell Syst Technol 15:1\u201345. http:\/\/arxiv.org\/abs\/2307.03109","journal-title":"ACM Trans Intell Syst Technol"},{"key":"339_CR13","unstructured":"Chen B, Zhang Z, Langren\u00e9 N, Zhu S (2023) Unleashing the potential of prompt engineering in large language models: a comprehensive review. http:\/\/arxiv.org\/abs\/2310.14735"},{"key":"339_CR14","unstructured":"Chroma. (n.d.). Chroma. Retrieved February 1 (2024) from https:\/\/www.trychroma.com\/"},{"key":"339_CR15","doi-asserted-by":"crossref","unstructured":"Cormack GV, Clarke CL, B\u00fcttcher S (2009) Reciprocal rank fusion outperforms condorcet and individual rank learning methods. Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval","DOI":"10.1145\/1571941.1572114"},{"key":"339_CR17","unstructured":"Do\u011fru\u00f6z AS, Skantze G (2022) How open are the conversations with open-domain chatbots? A proposal for speech event based evaluation. http:\/\/arxiv.org\/abs\/2211.13560"},{"key":"339_CR18","unstructured":"Fraiwan M, Khasawneh N (2023) A review of ChatGPT applications in education, marketing, software engineering, and healthcare. Benefits, Drawbacks, and Research Directions"},{"key":"339_CR19","unstructured":"Gunasekar S, Zhang Y, Aneja J, Mendes CC, Giorno AD, Gopi S, Javaheripi M, Kauffmann P, de Rosa G, Saarikivi O, Salim A, Shah S, Behl HS, Wang X, Bubeck S, Eldan R, Kalai AT, Lee YT, Li Y (2023) Textbooks are all you need."},{"key":"339_CR20","unstructured":"Han Y, Liu C, Wang P (2023) A comprehensive survey on vector database: storage and retrieval technique, Challenge. http:\/\/arxiv.org\/abs\/2310.11703"},{"key":"339_CR21","doi-asserted-by":"publisher","unstructured":"Kenney NM (2023) A brief analysis of the architecture, limitations, and impacts of ChatGPT. https:\/\/doi.org\/10.5281\/zenodo.7762245","DOI":"10.5281\/zenodo.7762245"},{"key":"339_CR22","doi-asserted-by":"publisher","first-page":"92337","DOI":"10.1109\/ACCESS.2022.3201144","volume":"10","author":"S Kusal","year":"2022","unstructured":"Kusal S, Patil S, Choudrie J, Kotecha K, Mishra S, Abraham A (2022) AI-based conversational agents: A scoping review from technologies to future directions. IEEE Access 10:92337\u201392356. https:\/\/doi.org\/10.1109\/ACCESS.2022.3201144","journal-title":"IEEE Access"},{"key":"339_CR23","unstructured":"LangChain. (n.d.). Ensemble retriever. Retrieved February 1 (2024) from https:\/\/python.langchain.com\/v0.1\/docs\/modules\/data_connection\/retrievers\/ensemble\/"},{"key":"339_CR24","unstructured":"Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, K\u00fcttler H, Lewis M, Yih W, Rockt\u00e4schel T, Riedel S, Kiela D (2020) Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. http:\/\/arxiv.org\/abs\/2005.11401"},{"key":"339_CR25","unstructured":"Li X, Zhang Y, Malthouse EC (2023) Exploring fine-tuning ChatGPT for news recommendation. http:\/\/arxiv.org\/abs\/2311.05850"},{"key":"339_CR26","doi-asserted-by":"publisher","unstructured":"Lin CC, Huang AYQ, Yang SJH (2023) A review of AI-Driven conversational chatbots implementation methodologies and challenges (1999\u20132022). Sustainability 15(5). https:\/\/doi.org\/10.3390\/su15054012","DOI":"10.3390\/su15054012"},{"key":"339_CR27","unstructured":"LlamaIndex. (n.d.). QuestionGeneration. Retrieved February 1 (2024) from https:\/\/docs.llamaindex.ai\/en\/stable\/examples\/evaluation\/QuestionGeneration\/"},{"key":"339_CR28","doi-asserted-by":"crossref","unstructured":"Lund BD, Wang T (2023) Chatting about chatgpt: how May AI and GPT impact academia and libraries? https:\/\/ssrn.com\/abstract=4333415","DOI":"10.2139\/ssrn.4333415"},{"issue":"2","key":"339_CR29","doi-asserted-by":"publisher","first-page":"116","DOI":"10.55145\/ajest.2023.02.02.015","volume":"2","author":"M Mijwil","year":"2023","unstructured":"Mijwil M, Hiran M, Doshi KK, Dadhich R, Al-Mistarehi M, A.-H., Bala I (2023) ChatGPT and the future of academic integrity in the artificial intelligence era: a new frontier. Al-Salam J Eng Technol 2(2):116\u2013127. https:\/\/doi.org\/10.55145\/ajest.2023.02.02.015","journal-title":"Al-Salam J Eng Technol"},{"key":"339_CR30","unstructured":"Mnasri M (2019) Recent advances in conversational NLP: towards the standardization of Chatbot building. http:\/\/arxiv.org\/abs\/1903.09025"},{"key":"339_CR31","doi-asserted-by":"crossref","unstructured":"Muennighoff N, Tazi N, Magne L, Reimers N (2022), October 13 MTEB: massive text embedding benchmark. Conference of the European Chapter of the Association for Computational Linguistics. http:\/\/arxiv.org\/abs\/2210.07316","DOI":"10.18653\/v1\/2023.eacl-main.148"},{"key":"339_CR32","doi-asserted-by":"publisher","first-page":"92337","DOI":"10.1109\/ACCESS.2023.3307635","volume":"10","author":"K Nhut Lam","year":"2023","unstructured":"Nhut Lam K, Huu Nguy L, Lam VLE, Kalita J (2023) A Transformer-based educational virtual assistant using diacriticized Latin script. IEEE Access 10:92337\u201392356. https:\/\/doi.org\/10.1109\/ACCESS.2023.3307635","journal-title":"IEEE Access"},{"key":"339_CR34","unstructured":"OpenAI (2023), November 13 A survey of techniques for maximizing LLM performance [Video recording]. https:\/\/www.youtube.com\/watch?v=ahnGLM-RC1Y"},{"key":"339_CR33","unstructured":"OpenAI. (n.d.). API Documentation. Retrieved February 1 (2024) from https:\/\/platform.openai.com\/docs\/introduction"},{"key":"339_CR36","unstructured":"OpenAI, Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, Almeida D, Altenschmidt J, Altman S, Anadkat S, Avila R, Babuschkin I, Balaji S, Balcom V, Baltescu P, Bao H, Bavarian M, Belgum J, Zoph B (2023) GPT-4 Technical Report. http:\/\/arxiv.org\/abs\/2303.08774"},{"key":"339_CR35","unstructured":"OpenAI,:, Hurst A, Lerer A, Goucher AP, Perelman A, Ramesh A, Clark A, Ostrow A, Welihinda A, Hayes A, Radford A, M\u0105dry A, Baker-Whitcomb A, Beutel A, Borzunov A, Carney A, Chow A, Kirillov A, Malkov Y (2024) GPT-4o System Card. http:\/\/arxiv.org\/abs\/2410.21276"},{"key":"339_CR37","doi-asserted-by":"crossref","unstructured":"Risch J, M\u00f6ller T, Gutsch J, Pietsch M (2021) Semantic answer similarity for evaluating question answering models. http:\/\/arxiv.org\/abs\/2108.06130","DOI":"10.18653\/v1\/2021.mrqa-1.15"},{"key":"339_CR38","unstructured":"Selenium. (n.d.). WebDriver Documentation. Retrieved February 1 (2024) from https:\/\/www.selenium.dev\/documentation\/webdriver\/"},{"key":"339_CR39","unstructured":"Silva B, Nunes L, Estev\u00e3o R, Aski V, Chandra R (2023) GPT-4 as an Agronomist Assistant? answering agriculture exams using large language models. http:\/\/arxiv.org\/abs\/2310.06225"},{"key":"339_CR40","doi-asserted-by":"publisher","unstructured":"Sperl\u00ed G (2021) A cultural heritage framework using a deep learning based chatbot for supporting tourist journey. Expert Syst Appl 183. https:\/\/doi.org\/10.1016\/j.eswa.2021.115277","DOI":"10.1016\/j.eswa.2021.115277"},{"key":"339_CR41","unstructured":"Streamlit. (n.d.). Documentation. Retrieved April 15 (2024) from https:\/\/docs.streamlit.io\/"},{"key":"339_CR42","doi-asserted-by":"publisher","unstructured":"Svore KM, Burges CJC (2009) A machine learning approach for improved BM25 retrieval. Int Conf Inform Knowl Manage Proc 1811\u20131814. https:\/\/doi.org\/10.1145\/1645953.1646237","DOI":"10.1145\/1645953.1646237"},{"key":"339_CR43","doi-asserted-by":"crossref","unstructured":"Thompson VU, Panchev C, Oakes MP (2015) Performance evaluation of similarity measures on similar and dissimilar text retrieval. 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K), 577\u2013584","DOI":"10.5220\/0005619105770584"},{"key":"339_CR44","unstructured":"TravelBI by Turismo de Portugal (2024) Turismo em N\u00fameros. https:\/\/travelbi.turismodeportugal.pt\/turismo-em-portugal\/turismo-numeros-2023\/"},{"key":"339_CR45","doi-asserted-by":"publisher","unstructured":"Trotman A, Puurula A, Burgess B (2014) Improvements to BM25 and language models examined. Proceedings of the 19th Australasian Document Computing Symposium., 27-28-November-2014. https:\/\/doi.org\/10.1145\/2682862.2682863","DOI":"10.1145\/2682862.2682863"},{"key":"339_CR46","doi-asserted-by":"crossref","unstructured":"Turing AM (1950) Mind a Quarterly Review Psychology and Philosophy I-computing Machinery and Intelligence","DOI":"10.1093\/mind\/LIX.236.433"},{"key":"339_CR47","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention Is All You Need. Neural Information Processing Systems. http:\/\/arxiv.org\/abs\/1706.03762"},{"key":"339_CR48","unstructured":"Visit Lisboa. (n.d.). Visit Lisboa. Retrieved January 12 (2024) from https:\/\/www.visitlisboa.com\/"},{"issue":"1","key":"339_CR49","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1145\/365153.365168","volume":"9","author":"J Weizenbaum","year":"1966","unstructured":"Weizenbaum J (1966) ELIZA - A computer program for the study of natural Language communication between man and machine. Commun ACM 9(1):36\u201345","journal-title":"Commun ACM"},{"key":"339_CR50","doi-asserted-by":"crossref","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Scao T, Le, Gugger S, Rush AM (2019) HuggingFace\u2019s Transformers: State-of-the-art Natural Language Processing. http:\/\/arxiv.org\/abs\/1910.03771","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"339_CR51","unstructured":"Xi Z, Chen W, Guo X, He W, Ding Y, Hong B, Zhang M, Wang J, Jin S, Zhou E, Zheng R, Fan X, Wang X, Xiong L, Zhou Y, Wang W, Jiang C, Zou Y, Liu X, Gui T (2023) The Rise and Potential of Large Language Model Based Agents: A Survey. http:\/\/arxiv.org\/abs\/2309.07864"}],"container-title":["Information Technology &amp; Tourism"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40558-025-00339-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40558-025-00339-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40558-025-00339-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T21:52:21Z","timestamp":1761774741000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40558-025-00339-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,14]]},"references-count":51,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["339"],"URL":"https:\/\/doi.org\/10.1007\/s40558-025-00339-x","relation":{},"ISSN":["1098-3058","1943-4294"],"issn-type":[{"value":"1098-3058","type":"print"},{"value":"1943-4294","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,14]]},"assertion":[{"value":"21 January 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 July 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 August 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}