{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T02:21:32Z","timestamp":1781662892451,"version":"3.54.5"},"reference-count":106,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2025,3,22]],"date-time":"2025-03-22T00:00:00Z","timestamp":1742601600000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007694","name":"KAIA","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100007694","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003565","name":"Ministry of Land, Infrastructure and Transport","doi-asserted-by":"publisher","award":["RS-2021-KA163269"],"award-info":[{"award-number":["RS-2021-KA163269"]}],"id":[{"id":"10.13039\/501100003565","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,3,29]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>This study aimed to evaluate ChatGPT-3.5 and ChatGPT-4\u2019s understanding of the building information modelling (BIM) knowledge domain by testing on the multiple-choice section of Korea's BIM Expertise Exam (KBEE), a professional license exam tailored for BIM. ChatGPT-4 achieved passing scores across all tested years and averaged 85%, 20% higher than its predecessor. Nevertheless, ChatGPT-4 had difficulty in certain areas including the \u2018BIM guidelines\u2019 subcategory, as the problems required access to documents specific to Korea's BIM policies and stipulations. By supplying the required documents using Retrieval Augmented Generation (RAG), GPT-4\u2019s score further improved to 88.6%, an improvement of 25.7%. The results provide evidence that ChatGPT-4, albeit within the context of the KBEE, is overall knowledgeable in its understanding of the BIM domain. The results of RAG demonstrate that partial gaps in its knowledge can be bolstered by providing the appropriate documents. The study verified that this capability was particularly valuable for categories that varied depending on local and regional contexts. The findings suggest that ChatGPT, when integrated with RAG, holds significant potential as a comprehensive and adaptive knowledge model in the BIM domain, and thereby alleviates the barriers for BIM adoption, such as the lack of experts, the cost of training, complexity of the BIM models, and additional workloads required during the BIM delivery process.<\/jats:p>","DOI":"10.1093\/jcde\/qwaf033","type":"journal-article","created":{"date-parts":[[2025,3,20]],"date-time":"2025-03-20T18:31:51Z","timestamp":1742495511000},"page":"94-120","source":"Crossref","is-referenced-by-count":6,"title":["Evaluating ChatGPT on Korea's BIM Expertise Exam and improving its performance through RAG"],"prefix":"10.1093","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3593-0657","authenticated-orcid":false,"given":"Youngsu","family":"Yu","sequence":"first","affiliation":[{"name":"Urban Construction Engineering & Management, Seoul National University of Science and Technology , 232 Gongreung-ro, Nowon-gu, Seoul 01811 ,","place":["Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9775-5770","authenticated-orcid":false,"given":"Sihyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, Seoul National University of Science and Technology , 232 Gongreung-ro, Nowon-gu, Seoul 01811 ,","place":["Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4725-6041","authenticated-orcid":false,"given":"Wonbok","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, Seoul National University of Science and Technology , 232 Gongreung-ro, Nowon-gu, Seoul 01811 ,","place":["Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1029-3375","authenticated-orcid":false,"given":"Bonsang","family":"Koo","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, Seoul National University of Science and Technology , 232 Gongreung-ro, Nowon-gu, Seoul 01811 ,","place":["Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,3,21]]},"reference":[{"key":"2025102112263883700_bib1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.13140\/RG.2.1.4990.4164","article-title":"Barriers to BIM\/4D implementation in Qatar","volume":"533","author":"Ahmed","year":"2014","journal-title":"Smart, Sustainable and Healthy Cities"},{"key":"2025102112263883700_bib2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.02333","article-title":"Beyond extraction: Contextualising tabular data for efficient summarisation by language models","author":"Allu","year":"2024"},{"key":"2025102112263883700_bib3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.autcon.2021.103929","article-title":"Transformer machine learning language model for auto-alignment of long-term and short-term plans in construction","volume":"132","author":"Amer","year":"2021","journal-title":"Automation in Construction"},{"key":"2025102112263883700_bib4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmology Science"},{"key":"2025102112263883700_bib5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2109.10739","article-title":"Predicting efficiency\/effectiveness trade-offs for dense vs. sparse retrieval strategy selection","author":"Arabzadeh","year":"2021"},{"key":"2025102112263883700_bib6","doi-asserted-by":"publisher","first-page":"3781","DOI":"10.1016\/j.procs.2024.09.178","article-title":"A survey on RAG with LLMs","volume":"246","author":"Arslan","year":"2024","journal-title":"Procedia Computer Science"},{"key":"2025102112263883700_bib7","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1109\/ICSC59802.2024.00050","article-title":"Experiments with text-to-SPARQL based on ChatGPT","volume-title":"2024 IEEE 18th International Conference on Semantic Computing (ICSC)","author":"Avila","year":"2024"},{"key":"2025102112263883700_bib8","doi-asserted-by":"publisher","first-page":"52","DOI":"10.61969\/jai.1337500","article-title":"Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning","volume":"7","author":"Baidoo-Anu","year":"2023","journal-title":"Journal of AI"},{"key":"2025102112263883700_bib9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.03195","article-title":"Gpt-4: A review on advancements and opportunities in natural language processing","author":"Baktash","year":"2023"},{"key":"2025102112263883700_bib10","doi-asserted-by":"publisher","first-page":"5232","DOI":"10.1145\/3583780.3615994","article-title":"Comparative analysis of open source and commercial embedding models for question answering","volume-title":"In","author":"Balikas","year":"2023"},{"key":"2025102112263883700_bib11","doi-asserted-by":"publisher","first-page":"269","DOI":"10.5194\/isprs-archives-XLII-5-W1-269-2017","article-title":"Historic BIM: A new repository for structural health monitoring","volume":"42","author":"Banfi","year":"2017","journal-title":"The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences"},{"key":"2025102112263883700_bib12","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1145\/3442188.3445922","article-title":"On the dangers of stochastic parrots: Can language models be too big?","volume-title":"In","author":"Bender","year":"2021"},{"key":"2025102112263883700_bib13","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1038\/s41586-023-06792-0","article-title":"Autonomous chemical research with large language models","volume":"624","author":"Boiko","year":"2023","journal-title":"Nature"},{"key":"2025102112263883700_bib14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.14402","article-title":"GPT takes the bar exam","author":"Bommarito","year":"2022"},{"key":"2025102112263883700_bib15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.08678","article-title":"Can GPT models be financial analysts? An evaluation of ChatGPT and GPT-4 on mock CFA exams","author":"Callanan","year":"2023"},{"key":"2025102112263883700_bib16","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1109\/ICAI63388.2024.10851665","article-title":"The Serbian retrieval-augmented generation system based on hybrid search","volume-title":"In","author":"Ceni\u0107","year":"2024"},{"key":"2025102112263883700_bib17","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/s43681-022-00148-6","article-title":"GPT-3 and InstructGPT: Technological dystopianism, utopianism, and \u201ccontextual\u201d perspectives in AI ethics and industry","volume":"3","author":"Chan","year":"2023","journal-title":"AI and Ethics"},{"key":"2025102112263883700_bib18","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1016\/j.jclepro.2021.126153","article-title":"Multiobjective optimization of building energy consumption based on BIM-DB and LSSVM-NSGA-II","volume":"294","author":"Chen","year":"2021","journal-title":"Journal of Cleaner Production"},{"key":"2025102112263883700_bib20","doi-asserted-by":"publisher","first-page":"21","DOI":"10.2139\/ssrn.4335905","article-title":"ChatGPT goes to law school","volume":"71","author":"Choi","year":"2022","journal-title":"Journal of Legal Education"},{"key":"2025102112263883700_bib19","doi-asserted-by":"publisher","first-page":"21","DOI":"10.13161\/kibim.2023.13.3.021","article-title":"Evaluating ChatGPT's competency in BIM-related knowledge via the Korean BIM expertise exam","volume":"13","author":"Choi","year":"2023","journal-title":"KIBIM Magazine"},{"key":"2025102112263883700_bib21","volume-title":"Chroma Documentation","author":"Chroma","year":"2024"},{"key":"2025102112263883700_bib22","doi-asserted-by":"publisher","first-page":"14","DOI":"10.3758\/s13428-018-1142-4","article-title":"The tool for the automatic analysis of cohesion 2.0: Integrating semantic similarity and text overlap","volume":"51","author":"Crossley","year":"2019","journal-title":"Behavior Research Methods"},{"key":"2025102112263883700_bib23","doi-asserted-by":"publisher","first-page":"2658","DOI":"10.21325\/jotags.2024.1312","article-title":"Cooking with ChatGPT and Bard: A study on competencies of AI tools on recipe correction, adaptation, time management, and presentation","volume":"11","author":"De\u011ferli","year":"2023","journal-title":"Journal of Tourism & Gastronomy Studies"},{"key":"2025102112263883700_bib24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1810.04805","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018"},{"key":"2025102112263883700_bib25","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2408.08054","article-title":"Text2BIM: Generating building models using a large language model-based multi-agent framework","author":"Du","year":"2024"},{"key":"2025102112263883700_bib26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.14387","article-title":"Alpacafarm: A simulation framework for methods that learn from human feedback","author":"Dubois","year":"2023"},{"key":"2025102112263883700_bib27","first-page":"77","article-title":"Building information modelling adoption: An analysis of the barriers of implementation","volume":"2","author":"Eadie","year":"2014","journal-title":"Journal of Engineering and Architecture"},{"key":"2025102112263883700_bib28","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2009.11462","article-title":"Realtoxicityprompts: evaluating neural toxic degeneration in language models","author":"Gehman","year":"2020"},{"key":"2025102112263883700_bib29","unstructured":"George A. S., George A. S. H., Martin A. S. G. (2023). A review of ChatGPT AI's impact on several business sectors. Partners Universal International Innovation Journal, 1, 9\u201323. 10.5281\/zenodo.7644359."},{"key":"2025102112263883700_bib30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1061\/JMENEA.MEENG-6444","article-title":"A survey on large language models: Applications, challenges, limitations, and practical usage","volume":"3","author":"Hadi","year":"2023","journal-title":"TechRxiv"},{"key":"2025102112263883700_bib31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1061\/JMENEA.MEENG-644","article-title":"Enriched construction regulation inquiry responses: A hybrid search approach for large language models","volume":"41","author":"He","year":"2025","journal-title":"Journal of Management in Engineering"},{"key":"2025102112263883700_bib32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/fi16120438","article-title":"Real-time text-to-cypher query generation with large language models for graph databases","volume":"16","author":"Hornsteiner","year":"2024","journal-title":"Future Internet"},{"key":"2025102112263883700_bib33","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1007\/978-981-99-7587-7_14","article-title":"DSQA-LLM: Domain-specific intelligent question answering based on large language models. AI-generated content","volume":"1946","author":"Huang","year":"2023","journal-title":"Communications in Computer and Information Science"},{"key":"2025102112263883700_bib34","doi-asserted-by":"publisher","first-page":"24","DOI":"10.6106\/KJCEM.2024.25.6.024","article-title":"BIM and OpenAI-based model for supporting initial construction planning of bridges and tunnels","volume":"25","author":"Hwang","year":"2024","journal-title":"Korean Journal of Construction Engineering and Management"},{"key":"2025102112263883700_bib35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.aei.2024.102532","article-title":"Automated detailing of exterior walls using NADIA: natural-language-based architectural detailing through interaction with AI","volume":"61","author":"Jang","year":"2024","journal-title":"Advanced Engineering Informatics"},{"key":"2025102112263883700_bib36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.02981","article-title":"Fine-tuning and utilization methods of domain-specific LLMs","author":"Jeong","year":"2024"},{"key":"2025102112263883700_bib37","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1093\/jcde\/qwae017","article-title":"Generative artificial intelligence and building design: Early photorealistic render visualization of fa\u00e7ades using local identity-trained models","volume":"11","author":"Jo","year":"2024","journal-title":"Journal of Computational Design and Engineering"},{"key":"2025102112263883700_bib38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.19794","article-title":"Introducing a new hyper-parameter for RAG: context window utilization","author":"Juvekar","year":"2024"},{"key":"2025102112263883700_bib39","doi-asserted-by":"publisher","first-page":"52","DOI":"10.13161\/kibim.2022.12.4.052","article-title":"Ontology BIM-based knowledge service framework architecture development","volume":"12","author":"Kang","year":"2022","journal-title":"KIBIM Magazine"},{"key":"2025102112263883700_bib40","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2004.04906","article-title":"Dense passage retrieval for open-domain question answering","author":"Karpukhin","year":"2020"},{"key":"2025102112263883700_bib41","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.18027","article-title":"Evaluating GPT-4 and ChatGPT on Japanese medical licensing examinations","author":"Kasai","year":"2023"},{"key":"2025102112263883700_bib42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1098\/rsta.2023.0254","article-title":"GPT-4 passes the bar exam","volume":"382","author":"Katz","year":"2024","journal-title":"Philosophical Transactions of the Royal Society A"},{"key":"2025102112263883700_bib43","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.17866","article-title":"Financial statement analysis with large language models","author":"Kim","year":"2024"},{"key":"2025102112263883700_bib44","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1093\/jcde\/qwaa075","article-title":"A geometric deep learning approach for checking element-to-entity mappings in infrastructure building information models","volume":"8","author":"Koo","year":"2021","journal-title":"Journal of Computational Design and Engineering"},{"key":"2025102112263883700_bib45","volume-title":"Expressway Smart Design Guidelines","author":"Korea Expressway Corporation","year":"2020"},{"key":"2025102112263883700_bib46","author":"Korea Expressway Corporation","year":"2021","journal-title":"Expressway BIM Design Evaluation Criteria"},{"key":"2025102112263883700_bib47","author":"Korean Institute of BIM (KIBIM)","year":"2021","journal-title":"BIM Educational Resources"},{"key":"2025102112263883700_bib48","doi-asserted-by":"publisher","DOI":"10.36227\/techrxiv.22312330.v2","article-title":"GPT-4 vs. GPT-3.5: a concise showdown","author":"Koubaa","year":"2023","journal-title":"TechRxiv"},{"key":"2025102112263883700_bib49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLoS Digital Health"},{"key":"2025102112263883700_bib50","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/j.tplants.2024.04.013","article-title":"Large language models in plant biology","volume":"29","author":"Lam","year":"2024","journal-title":"Trends in Plant Science"},{"key":"2025102112263883700_bib51","author":"LangChain","year":"2024","journal-title":"Build a Retrieval Augmented Generation (RAG) App"},{"key":"2025102112263883700_bib52","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1016\/j.diii.2023.02.003","article-title":"Revolutionizing radiology with GPT-based models: Current applications, future possibilities, and limitations of ChatGPT","volume":"104","author":"Lecler","year":"2023","journal-title":"Diagnostic and Interventional Imaging"},{"key":"2025102112263883700_bib53","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1093\/jcde\/qwae065","article-title":"Generative early architectural visualizations: Incorporating architect's style-trained models","volume":"11","author":"Lee","year":"2024","journal-title":"Journal of Computational Design and Engineering"},{"key":"2025102112263883700_bib54","doi-asserted-by":"publisher","first-page":"45","DOI":"10.6106\/KJCEM.2022.23.3.045","article-title":"Development of graph-based deep learning methods for enhancing the semantic integrity of spaces in BIM models","volume":"23","author":"Lee","year":"2022","journal-title":"Korean Journal of Construction Engineering and Management"},{"key":"2025102112263883700_bib55","first-page":"273","article-title":"Analyzing BIM topics and clusters through ten years of scientific publications","volume":"24","author":"Lemaire","year":"2019","journal-title":"Journal of Information Technology in Construction"},{"key":"2025102112263883700_bib56","first-page":"85","article-title":"Expert systems in the BIM environment","volume":"14","author":"Levchenko","year":"2017","journal-title":"INTERSECTII\/INTERSECTIONS"},{"key":"2025102112263883700_bib57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.11401","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","author":"Lewis","year":"2020"},{"key":"2025102112263883700_bib58","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1061\/(ASCE)ME.1943-5479.0000900","article-title":"Reducing non-value-adding BIM implementation activities for building projects in Singapore: Leading causes","volume":"37","author":"Liao","year":"2021","journal-title":"Journal of Management in Engineering"},{"key":"2025102112263883700_bib59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/fi8030039","article-title":"Ontology-based representation and reasoning in building construction cost estimation in China","volume":"8","author":"Liu","year":"2016","journal-title":"Future Internet"},{"key":"2025102112263883700_bib60","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.metrad.2023.100017","article-title":"Summary of ChatGPT-related research and perspective towards the future of large language models","volume":"1","author":"Liu","year":"2023","journal-title":"Meta-Radiology"},{"key":"2025102112263883700_bib61","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.04055","article-title":"Sparse meets dense: a hybrid approach to enhance scientific document retrieval","author":"Mandikal","year":"2024"},{"key":"2025102112263883700_bib62","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10506-024-09396-9","article-title":"Re-evaluating GPT-4\u2019s bar exam performance","volume":"1","author":"Mart\u00ednez","year":"2024","journal-title":"Artificial Intelligence and Law"},{"key":"2025102112263883700_bib63","doi-asserted-by":"publisher","DOI":"10.21427\/r42p-3\u00d725","article-title":"Establishing the key pillars of innovation required to execute a successful BIM strategy within a construction SME in Ireland","volume-title":"In","author":"McAuley","year":"2017"},{"key":"2025102112263883700_bib64","article-title":"BIM\u2013a knowledge repository","volume-title":"In","author":"Meadati","year":"2010"},{"key":"2025102112263883700_bib65","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2016","journal-title":"Report of Enhancement and Substantiation Plan for Core Technology of Open BIM-based Architectural Design"},{"key":"2025102112263883700_bib66","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2017","journal-title":"Appendix of Open BIM-based Building Design Standards and Infrastructure Construction"},{"key":"2025102112263883700_bib67","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2018","journal-title":"BIM Roadmap and Activation Strategies for Public SOC Projects"},{"key":"2025102112263883700_bib68","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2020","journal-title":"Architect's Scope of Work and Consideration for Public Works Projects"},{"key":"2025102112263883700_bib69","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2020","journal-title":"2030 architecture BIM Activation Roadmap"},{"key":"2025102112263883700_bib70","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2020","journal-title":"Basic Guidelines for BIM in the Construction Industry"},{"key":"2025102112263883700_bib71","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2022","journal-title":"Construction Industry BIM Stipulation Guidelines (owner, contractor, designer)"},{"key":"2025102112263883700_bib72","author":"Ministry of Land, Infrastructure and Transport (MOLIT)","year":"2022","journal-title":"S-Construction 2030"},{"key":"2025102112263883700_bib73","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/ICWR61162.2024.10533345","article-title":"Hybrid retrieval-augmented generation approach for LLMs query response enhancement","volume-title":"In","author":"Omrani","year":"2024"},{"key":"2025102112263883700_bib74","article-title":"New and improved embedding model","author":"OpenAI","year":"2022"},{"key":"2025102112263883700_bib75","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774","article-title":"GPT-4 technical report","author":"OpenAI","year":"2023"},{"key":"2025102112263883700_bib76","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1097\/MD.0000000000034673","article-title":"ChatGPT performance in the medical specialty exam: an observational study","volume":"102","author":"Oztermeli","year":"2023","journal-title":"Medicine"},{"key":"2025102112263883700_bib77","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2203.16714","article-title":"End-to-end table question answering via retrieval-augmented generation","author":"Pan","year":"2022"},{"key":"2025102112263883700_bib78","doi-asserted-by":"publisher","first-page":"46","DOI":"10.2174\/97816810845721170101","article-title":"BIM application and adoption in the UK housing sector","volume-title":"Integrated Building Information Modelling","author":"Park","year":"2017"},{"key":"2025102112263883700_bib79","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/buildings13040857","article-title":"Investigating the use of ChatGPT for the scheduling of construction projects","volume":"13","author":"Prieto","year":"2023","journal-title":"Buildings"},{"key":"2025102112263883700_bib80","author":"Public Procurement Service (PPS)","year":"2022","journal-title":"Guidelines for Applying BIM for Facilities Projects"},{"key":"2025102112263883700_bib81","doi-asserted-by":"publisher","first-page":"1316","DOI":"10.1162\/tacl_a_00605","article-title":"In-context retrieval-augmented language models","volume":"11","author":"Ram","year":"2023","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2025102112263883700_bib82","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4645601","article-title":"Integrating Building Information Modelling (BIM) with ChatGPT, Bard, and similar generative artificial intelligence in the architecture, engineering, and construction industry: applications, a novel framework, challenges, and future scope","author":"Rane","year":"2023","journal-title":"SSRN"},{"key":"2025102112263883700_bib83","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1561\/1500000019","article-title":"The probabilistic relevance framework: BM25 and beyond","volume":"3","author":"Robertson","year":"2009","journal-title":"Foundations and Trends\u00ae in Information Retrieval"},{"key":"2025102112263883700_bib84","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.07927","article-title":"A systematic survey of prompt engineering in large language models: Techniques and applications","author":"Sahoo","year":"2024"},{"key":"2025102112263883700_bib85","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1080\/17452007.2019.1651247","article-title":"Knowledge, skills and functionalities requirements for quantity surveyors in building information modelling (BIM) work environment: an international Delphi study","volume":"16","author":"Saka","year":"2020","journal-title":"Architectural Engineering and Design Management"},{"key":"2025102112263883700_bib86","volume-title":"A Study on the Guideline and Roadmap for Applying BIM to Seoul","author":"Seoul Institute of Technology","year":"2022"},{"key":"2025102112263883700_bib87","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1177\/00472875231214196","article-title":"ChatGPT for trip planning: The effect of narrowing down options","volume":"64","author":"Shin","year":"2025","journal-title":"Journal of Travel Research"},{"key":"2025102112263883700_bib88","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1080\/15578771.2022.2123071","article-title":"Leading UK construction companies\u2019 strategies to tackle BIM training and skills challenges","volume":"19","author":"Shojaei","year":"2023","journal-title":"International Journal of Construction Education and Research"},{"key":"2025102112263883700_bib89","doi-asserted-by":"publisher","first-page":"238","DOI":"10.22260\/ISARC2023\/0034","article-title":"Prospects of integrating BIM and NLP for automatic construction schedule management","volume":"40","author":"Singh","year":"2023","journal-title":"In ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction"},{"key":"2025102112263883700_bib90","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1101\/2023.03.24.23287731","article-title":"Performance of ChatGPT on free-response, clinical reasoning exams","author":"Strong","year":"2023","journal-title":"MedRxiv"},{"key":"2025102112263883700_bib91","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2196\/48002","article-title":"Performance of GPT-3.5 and GPT-4 on the Japanese medical licensing examination: Comparison study","volume":"9","author":"Takagi","year":"2023","journal-title":"JMIR Medical Education"},{"key":"2025102112263883700_bib92","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.18511","article-title":"Do we need domain-specific embedding models? An empirical investigation","author":"Tang","year":"2024"},{"key":"2025102112263883700_bib93","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1108\/S2516-285320190000002052","article-title":"An overview of BIM adoption in the construction industry: Benefits and barriers","volume":"2","author":"Ullah","year":"2019","journal-title":"In"},{"key":"2025102112263883700_bib94","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2408.01107","article-title":"BioRAG: a RAG-LLM framework for biological question reasoning","author":"Wang","year":"2024"},{"key":"2025102112263883700_bib95","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2201.11903","article-title":"Chain-of-thought prompting elicits reasoning in large language models","author":"Wei","year":"2022"},{"key":"2025102112263883700_bib96","doi-asserted-by":"publisher","first-page":"1","DOI":"10.48550\/arXiv.2304.13712","article-title":"Harnessing the power of LLMs in practice: a survey on ChatGPT and beyond","volume":"18","author":"Yang","year":"2023","journal-title":"ACM Transactions on Knowledge Discovery from Data"},{"key":"2025102112263883700_bib97","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2402.05131","article-title":"Financial report chunking for effective retrieval-augmented generation","author":"Yepes","year":"2024"},{"key":"2025102112263883700_bib98","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jobe.2023.107066","article-title":"An ontology-aided, natural language-based approach for multi-constraint BIM model querying","volume":"76","author":"Yin","year":"2023","journal-title":"Journal of Building Engineering"},{"key":"2025102112263883700_bib99","doi-asserted-by":"publisher","first-page":"1449","DOI":"10.1093\/jcde\/qwac064","article-title":"ArchShapesNet: a novel dataset for benchmarking architectural building information modeling element classification algorithms","volume":"9","author":"Yu","year":"2022","journal-title":"Journal of Computational Design and Engineering"},{"key":"2025102112263883700_bib100","doi-asserted-by":"publisher","first-page":"1898","DOI":"10.1007\/s12205-023-2331-y","article-title":"Ensemble-based deep learning approach for performance improvement of BIM element classification","volume":"27","author":"Yu","year":"2023","journal-title":"KSCE Journal of Civil Engineering"},{"key":"2025102112263883700_bib101","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1145\/3604237.3626866","article-title":"Enhancing financial sentiment analysis via retrieval-augmented large language models","volume-title":"Proceedings of the Fourth ACM International Conference on AI in Finance","author":"Zhang","year":"2023"},{"key":"2025102112263883700_bib102","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.01219","article-title":"Siren's song in the AI ocean: a survey on hallucination in large language models","author":"Zhang","year":"2023"},{"key":"2025102112263883700_bib103","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1063\/1.4999209","article-title":"Ontology-based heterogeneous materials database integration and semantic query","volume":"7","author":"Zhao","year":"2017","journal-title":"AIP Advances"},{"key":"2025102112263883700_bib104","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.09333","article-title":"BIM-GPT: a prompt-based virtual assistant framework for BIM information retrieval","author":"Zheng","year":"2023"},{"key":"2025102112263883700_bib105","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2211.01910","article-title":"Large language models are human-level prompt engineers","author":"Zhou","year":"2022"},{"key":"2025102112263883700_bib106","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.16345","article-title":"Cypher4BIM: Releasing the power of graph for building knowledge discovery","author":"Zhu","year":"2024"}],"container-title":["Journal of Computational Design and Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jcde\/advance-article-pdf\/doi\/10.1093\/jcde\/qwaf033\/62521636\/qwaf033.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/12\/4\/94\/62521636\/qwaf033.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/12\/4\/94\/62521636\/qwaf033.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T16:27:07Z","timestamp":1761064027000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcde\/article\/12\/4\/94\/8090188"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,21]]},"references-count":106,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,3,29]]}},"URL":"https:\/\/doi.org\/10.1093\/jcde\/qwaf033","relation":{},"ISSN":["2288-5048"],"issn-type":[{"value":"2288-5048","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,4]]},"published":{"date-parts":[[2025,3,21]]}}}