{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T11:58:01Z","timestamp":1773403081725,"version":"3.50.1"},"reference-count":68,"publisher":"Cambridge University Press (CUP)","license":[{"start":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T00:00:00Z","timestamp":1710720000000},"content-version":"unspecified","delay-in-days":77,"URL":"http:\/\/creativecommons.org\/licenses\/by-sa\/4.0"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["AIEDAM"],"published-print":{"date-parts":[[2024]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recent advances in machine learning have enabled computers to converse with humans meaningfully. In this study, we propose using this technology to facilitate design conversations in large-scale urban development projects by creating chatbot systems that can automate and streamline information exchange between stakeholders and designers. To this end, we developed and evaluated a proof-of-concept chatbot system that can perform design conversations on a specific construction project and convert those conversations into a list of requirements. Next, in an experiment with 56 participants, we compared the chatbot system to a regular online survey, focusing on user satisfaction and the quality and quantity of collected information. The results revealed that, with regard to user satisfaction, the participants preferred the chatbot experience to a regular survey. However, we found that chatbot conversations produced more data than the survey, with a similar rate of novel ideas but fewer themes. Our findings provide robust evidence that chatbots can be effectively used for design discussions in large-scale design projects and offer a user-friendly experience that can help to engage people in the design process. Based on this evidence, by providing a space for meaningful conversations between stakeholders and expanding the reach of design projects, the use of chatbot systems in interactive design systems can potentially improve design processes and their outcomes.<\/jats:p>","DOI":"10.1017\/s0890060424000027","type":"journal-article","created":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T11:16:12Z","timestamp":1710760572000},"update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":9,"title":["Evaluating large-language-model chatbots to engage communities in large-scale design projects"],"prefix":"10.1017","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7464-8526","authenticated-orcid":false,"given":"Jonathan","family":"Dortheimer","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1824-0243","authenticated-orcid":false,"given":"Nik","family":"Martelaro","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2621-7350","authenticated-orcid":false,"given":"Aaron","family":"Sprecher","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1385-5369","authenticated-orcid":false,"given":"Gerhard","family":"Schubert","sequence":"additional","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2024,3,18]]},"reference":[{"key":"S0890060424000027_r38","doi-asserted-by":"publisher","DOI":"10.1111\/1467-6435.00156"},{"key":"S0890060424000027_r42","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2005.09.001"},{"key":"S0890060424000027_r50","unstructured":"OpenAI (2023) ChatGPT: optimizing language models for dialogue. Available at https:\/\/openai.com\/blog\/chatgpt\/ (accessed 2 February 2023)."},{"key":"S0890060424000027_r52","doi-asserted-by":"publisher","DOI":"10.1016\/0142-694X(95)00000-H"},{"key":"S0890060424000027_r18","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09548-1"},{"key":"S0890060424000027_r61","unstructured":"Wei, J , Kim, S , Jung, H and Kim, Y-H (2023) Leveraging large language models to power chatbots for collecting user self-reported data. Preprint, arXiv:2301.05843."},{"key":"S0890060424000027_r44","doi-asserted-by":"publisher","DOI":"10.1145\/3357236.3395440"},{"key":"S0890060424000027_r65","doi-asserted-by":"publisher","DOI":"10.1145\/3381804"},{"key":"S0890060424000027_r55","doi-asserted-by":"publisher","DOI":"10.1075\/ijcl.10.4.06sha"},{"key":"S0890060424000027_r2","unstructured":"Ahmed, S (2019) An Architecture for Dynamic Conversational Agents for Citizen Participation and Ideation. Master\u2019s thesis, Technische Universit\u00e4t M\u00fcnchen."},{"key":"S0890060424000027_r30","first-page":"1","article-title":"Moderator chatbot for deliberative discussion: effects of discussion structure and discussant facilitation","volume":"5","author":"Kim","year":"2021","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"S0890060424000027_r1","first-page":"72","article-title":"Survey on chatbot design techniques in speech conversation systems","volume":"6","author":"Abdul-Kader","year":"2015","journal-title":"International Journal of Advanced Computer Science and Applications"},{"key":"S0890060424000027_r22","volume-title":"Social Justice and the City","author":"Harvey","year":"1973"},{"key":"S0890060424000027_r59","author":"Giering","year":"2011"},{"key":"S0890060424000027_r5","doi-asserted-by":"publisher","DOI":"10.1145\/344949.345015"},{"key":"S0890060424000027_r28","volume-title":"Death and Life of Great American Cities","author":"Jacobs","year":"1961"},{"key":"S0890060424000027_r21","first-page":"1","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI \u201921)","author":"Han","year":"2021"},{"key":"S0890060424000027_r6","doi-asserted-by":"publisher","DOI":"10.1080\/01944366908977225"},{"key":"S0890060424000027_r24","doi-asserted-by":"publisher","DOI":"10.1007\/s41651-019-0040-3"},{"key":"S0890060424000027_r7","doi-asserted-by":"publisher","DOI":"10.1016\/j.tele.2020.101473"},{"key":"S0890060424000027_r14","doi-asserted-by":"publisher","DOI":"10.3390\/math10040539"},{"key":"S0890060424000027_r3","volume-title":"Notes on the Synthesis of Form","author":"Alexander","year":"1964"},{"key":"S0890060424000027_r64","first-page":"1","volume-title":"CHI Conference on Human Factors in Computing Systems (CHI \u201920)","author":"Xiao","year":"2020a"},{"key":"S0890060424000027_r53","doi-asserted-by":"publisher","DOI":"10.1162\/DESI_a_00157"},{"key":"S0890060424000027_r33","first-page":"103","volume-title":"Computing for a Better Tomorrow \u2013 Proceedings of the 36th eCAADe Conference","volume":"1","author":"Kulcke","year":"2018"},{"key":"S0890060424000027_r43","doi-asserted-by":"publisher","DOI":"10.1109\/EMR.2010.5559142"},{"key":"S0890060424000027_r63","unstructured":"White, J , Fu, Q , Hays, S , Sandborn, M , Olea, C , Gilbert, H , Elnashar, A , Spencer-Smith, J and Schmidt, DC (2023) A prompt pattern catalog to enhance prompt engineering with ChatGPT. Preprint, arXiv:2302.11382."},{"key":"S0890060424000027_r54","first-page":"1","volume-title":"Creativity and Cognition","author":"Schmitt","year":"2021"},{"key":"S0890060424000027_r13","first-page":"1","volume-title":"CUI 2021 \u2013 3rd Conference on Conversational User Interfaces","author":"Cuadra","year":"2021"},{"key":"S0890060424000027_r56","first-page":"1","volume-title":"Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology","author":"Shin","year":"2022"},{"key":"S0890060424000027_r67","doi-asserted-by":"publisher","DOI":"10.4324\/9780203108543"},{"key":"S0890060424000027_r66","doi-asserted-by":"publisher","DOI":"10.1145\/3563657.3596138"},{"key":"S0890060424000027_r11","unstructured":"Brown, TB , Mann, B , Ryder, N , Subbiah, M , Kaplan, J , Dhariwal, P , Neelakantan, A , Shyam, P , Sastry, G , Askell, A , Agarwal, S , Herbert-Voss, A , Krueger, G , Henighan, T , Child, R , Ramesh, A , Ziegler, DM , Wu, J , Winter, C , Hesse, C , Chen, M , Sigler, E , Litwin, M , Gray, S , Chess, B , Clark, J , Berner, C , McCandlish, S , Radford, A , Sutskever, I and Amodei, D (2020) Language models are few-shot learners. Preprint, arXiv:2005.14165."},{"key":"S0890060424000027_r15","doi-asserted-by":"publisher","DOI":"10.1080\/13602365.2020.1758950"},{"key":"S0890060424000027_r10","doi-asserted-by":"publisher","DOI":"10.1177\/1473095209104824"},{"key":"S0890060424000027_r4","volume-title":"A Pattern Language: Towns, Buildings, Construction","author":"Alexander","year":"1977"},{"key":"S0890060424000027_r34","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(97)85459-2"},{"key":"S0890060424000027_r37","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2017.11.040"},{"key":"S0890060424000027_r68","doi-asserted-by":"publisher","DOI":"10.52842\/conf.ecaade.2020.2.155"},{"key":"S0890060424000027_r47","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2017.08.102"},{"key":"S0890060424000027_r16","doi-asserted-by":"publisher","DOI":"10.3390\/buildings13010222"},{"key":"S0890060424000027_r48","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-021-10212-x"},{"key":"S0890060424000027_r57","volume-title":"Proceedings of the International Conference on Information Systems - Bridging the Internet of People, Data, and Things, ICIS 2018","author":"Tavanapour","year":"2018"},{"key":"S0890060424000027_r60","doi-asserted-by":"publisher","DOI":"10.1016\/j.aei.2021.101371"},{"key":"S0890060424000027_r12","doi-asserted-by":"publisher","DOI":"10.1080\/13574809.2019.1677146"},{"key":"S0890060424000027_r62","doi-asserted-by":"publisher","DOI":"10.1145\/357980.357991"},{"key":"S0890060424000027_r29","doi-asserted-by":"publisher","DOI":"10.1080\/15710882.2012.734829"},{"key":"S0890060424000027_r17","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-18557-2_4"},{"key":"S0890060424000027_r58","first-page":"e21982","article-title":"User experience of a chatbot questionnaire versus a regular computer questionnaire","volume":"8","author":"te Pas","year":"2020","journal-title":"Prospective Comparative Study"},{"key":"S0890060424000027_r23","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.02.026"},{"key":"S0890060424000027_r51","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"S0890060424000027_r26","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15035-8_93"},{"key":"S0890060424000027_r8","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2019.029"},{"key":"S0890060424000027_r25","doi-asserted-by":"publisher","DOI":"10.1002\/poi3.90"},{"key":"S0890060424000027_r35","first-page":"1","article-title":"Designing a chatbot as a mediator for promoting deep self-disclosure to a real mental health professional","volume":"4","author":"Lee","year":"2020","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"S0890060424000027_r19","doi-asserted-by":"publisher","DOI":"10.1016\/j.lrp.2011.09.006"},{"key":"S0890060424000027_r41","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2018.10.003"},{"key":"S0890060424000027_r27","first-page":"1","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","author":"Hwang","year":"2021"},{"key":"S0890060424000027_r46","doi-asserted-by":"publisher","DOI":"10.1016\/j.cities.2017.08.018"},{"key":"S0890060424000027_r69","doi-asserted-by":"crossref","unstructured":"Dortheimer, J , Schubert, G , Dalach, A , Brenner, L and Martelaro, N (2023b)Think AI-side the Box!. eCAADe 41 - Digital Design Reconsidered, Graz, Austria, 567\u2013576","DOI":"10.52842\/conf.ecaade.2023.2.567"},{"key":"S0890060424000027_r39","first-page":"297","volume-title":"CAADRIA 2018 \u2013 23rd International Conference on Computer-Aided Architectural Design Research in Asia: Learning, Prototyping and Adapting","volume":"2","author":"Lu","year":"2018"},{"key":"S0890060424000027_r40","doi-asserted-by":"publisher","DOI":"10.1016\/S0142-694X(03)00040-1"},{"key":"S0890060424000027_r20","doi-asserted-by":"publisher","DOI":"10.1145\/3139398"},{"key":"S0890060424000027_r9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-17738-0_25"},{"key":"S0890060424000027_r31","first-page":"1","volume-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI \u201919)","author":"Kim","year":"2019"},{"key":"S0890060424000027_r32","doi-asserted-by":"publisher","DOI":"10.1145\/3319376"},{"key":"S0890060424000027_r45","doi-asserted-by":"publisher","DOI":"10.1080\/15710880802492862"},{"key":"S0890060424000027_r49","doi-asserted-by":"publisher","DOI":"10.1080\/15710880802518054"}],"container-title":["Artificial Intelligence for Engineering Design, Analysis and Manufacturing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0890060424000027","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T14:00:32Z","timestamp":1731592832000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0890060424000027\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":68,"alternative-id":["S0890060424000027"],"URL":"https:\/\/doi.org\/10.1017\/s0890060424000027","relation":{},"ISSN":["0890-0604","1469-1760"],"issn-type":[{"value":"0890-0604","type":"print"},{"value":"1469-1760","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024]]},"assertion":[{"value":"\u00a9 The Author(s), 2024. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This is an Open Access article, distributed under the terms of the Creative Commons Attribution-ShareAlike licence (http:\/\/creativecommons.org\/licenses\/by-sa\/4.0), which permits re-use, distribution, and reproduction in any medium, provided the same Creative Commons licence is used to distribute the re-used or adapted article and the original article is properly cited.","name":"license","label":"License","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}],"article-number":"e4"}}