{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T06:08:58Z","timestamp":1775196538452,"version":"3.50.1"},"reference-count":29,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T00:00:00Z","timestamp":1716508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"institutional resources"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Introduction: Large language models (LLMs), such as ChatGPT, are a topic of major public interest, and their potential benefits and threats are a subject of discussion. The potential contribution of these models to health care is widely discussed. However, few studies to date have examined LLMs. For example, the potential use of LLMs in (individualized) informed consent remains unclear. Methods: We analyzed the performance of the LLMs ChatGPT 3.5, ChatGPT 4.0, and Gemini with regard to their ability to create an information sheet for six basic anesthesiologic procedures in response to corresponding questions. We performed multiple attempts to create forms for anesthesia and analyzed the results checklists based on existing standard sheets. Results: None of the LLMs tested were able to create a legally compliant information sheet for any basic anesthesiologic procedure. Overall, fewer than one-third of the risks, procedural descriptions, and preparations listed were covered by the LLMs. Conclusions: There are clear limitations of current LLMs in terms of practical application. Advantages in the generation of patient-adapted risk stratification within individual informed consent forms are not available at the moment, although the potential for further development is difficult to predict.<\/jats:p>","DOI":"10.3390\/make6020053","type":"journal-article","created":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T08:30:22Z","timestamp":1716539422000},"page":"1145-1153","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Evaluation of AI ChatBots for the Creation of Patient-Informed Consent Sheets"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6597-9585","authenticated-orcid":false,"given":"Florian","family":"Raimann","sequence":"first","affiliation":[{"name":"Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Goethe University Frankfurt, University Hospital Frankfurt, Theodor-Stern Kai 7, 60590 Frankfurt, Germany"}]},{"given":"Vanessa","family":"Neef","sequence":"additional","affiliation":[{"name":"Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Goethe University Frankfurt, University Hospital Frankfurt, Theodor-Stern Kai 7, 60590 Frankfurt, Germany"}]},{"given":"Marie","family":"Hennighausen","sequence":"additional","affiliation":[{"name":"Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Goethe University Frankfurt, University Hospital Frankfurt, Theodor-Stern Kai 7, 60590 Frankfurt, Germany"}]},{"given":"Kai","family":"Zacharowski","sequence":"additional","affiliation":[{"name":"Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Goethe University Frankfurt, University Hospital Frankfurt, Theodor-Stern Kai 7, 60590 Frankfurt, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2886-8947","authenticated-orcid":false,"given":"Armin","family":"Flinspach","sequence":"additional","affiliation":[{"name":"Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, Goethe University Frankfurt, University Hospital Frankfurt, Theodor-Stern Kai 7, 60590 Frankfurt, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2024,5,24]]},"reference":[{"key":"ref_1","unstructured":"Verma, P., and De Vynck, G. (2024, March 04). ChatGPT Took Their Jobs. Now They Walk Dogs and Fix Air Conditioners. The Washington Post. Available online: https:\/\/www.washingtonpost.com\/technology\/2023\/06\/02\/ai-taking-jobs\/."},{"key":"ref_2","unstructured":"Cerullo, M. (2023, June 26). Here\u2019s How Many U.S. Workers ChatGPT Says It Could Replace. Available online: https:\/\/www.cbsnews.com\/news\/chatgpt-artificial-intelligence-jobs\/."},{"key":"ref_3","unstructured":"Tangalakis-Lippert, K. (2023, June 26). IBM Halts Hiring for 7,800 Jobs That Could Be Replaced by AI, Bloomberg Reports. Available online: https:\/\/www.businessinsider.com\/ibm-halts-hiring-for-7800-jobs-that-could-be-replaced-by-ai-report-2023-5."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"e240625","DOI":"10.1001\/jamahealthforum.2024.0625","article-title":"A Novel Machine Learning Algorithm for Creating Risk-Adjusted Payment Formulas","volume":"5","author":"Andriola","year":"2024","journal-title":"JAMA Health Forum"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"e241339","DOI":"10.1001\/jamahealthforum.2024.1339","article-title":"The Role for Policy in AI-Assisted Medical Diagnosis","volume":"5","author":"Sharfstein","year":"2024","journal-title":"JAMA Health Forum"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e246565","DOI":"10.1001\/jamanetworkopen.2024.6565","article-title":"AI-Generated Draft Replies Integrated Into Health Records and Physicians\u2019 Electronic Communication","volume":"7","author":"Baxter","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e231938","DOI":"10.1001\/jamahealthforum.2023.1938","article-title":"ChatGPT and Physicians\u2019 Malpractice Risk","volume":"4","author":"Mello","year":"2023","journal-title":"JAMA Health Forum"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1820","DOI":"10.1001\/jama.2023.19295","article-title":"AI Will\u2014And Should\u2014Change Medical School, Says Harvard\u2019s Dean for Medical Education","volume":"330","author":"Hswen","year":"2023","journal-title":"JAMA"},{"key":"ref_9","unstructured":"Medicine NEJo (2023, November 02). Prescribing Large Language Models for Medicine: What\u2019s The Right Dose? NEJM Group. Available online: https:\/\/events.nejm.org\/events\/617."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.jopan.2023.02.006","article-title":"The Role of ChatGPT in Perianesthesia Nursing","volume":"38","year":"2023","journal-title":"J. PeriAnesthesia Nurs."},{"key":"ref_11","unstructured":"Neff, A.S., and Philipp, S. (2023). KI-Anwendungen: Konkrete Beispiele f\u00fcr den \u00e4rztlichen Alltag. Deutsches \u00c4rzteblatt, 120."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1001\/jama.2023.23003","article-title":"Will Generative AI Tools Improve Access to Reliable Health Information?","volume":"331","author":"Anderer","year":"2024","journal-title":"JAMA"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1001\/jamapsychiatry.2023.5247","article-title":"Managerial and Organizational Challenges in the Age of AI","volume":"81","author":"Obradovich","year":"2024","journal-title":"JAMA Psychiatry"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1007\/s00063-023-01019-6","article-title":"K\u00fcnstliche Intelligenz: Wie werden ChatGPT und andere KI-Anwendungen unseren \u00e4rztlichen Alltag ver\u00e4ndern?","volume":"118","author":"Sonntagbauer","year":"2023","journal-title":"Med. Klin.\u2014Intensivmed. Notfallmedizin"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1001\/jamainternmed.2023.5947","article-title":"Health Disinformation Use Case Highlighting the Urgent Need for Artificial Intelligence Vigilance: Weapons of Mass Disinformation","volume":"184","author":"Menz","year":"2024","journal-title":"JAMA Intern. Med."},{"key":"ref_16","unstructured":"dpa (2023, June 26). Weltgesundheits Organisation Warnt vor Risiken durch K\u00fcnstliche Intelligenz im Gesundheitssektor. Available online: https:\/\/www.aerzteblatt.de\/treffer?mode=s&wo=1041&typ=1&nid=143259&s=ChatGPT."},{"key":"ref_17","first-page":"377","article-title":"The Quality of Informed Consent Forms","volume":"115","author":"Steckelberg","year":"2018","journal-title":"Dtsch. \u00c4rzteblatt Int."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1038\/s41746-024-01039-2","article-title":"Bridging the literacy gap for surgical consents: An AI-human expert collaborative approach","volume":"7","author":"Ali","year":"2024","journal-title":"NPJ Digit. Med."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"AIcs2300145","DOI":"10.1056\/AIcs2300145","article-title":"Using ChatGPT to Facilitate Truly Informed Medical Consent","volume":"1","author":"Mirza","year":"2024","journal-title":"NEJM AI"},{"key":"ref_20","unstructured":"PPR Human Experimentation (1964). Code of ethics of the world medical association. Declaration of Helsinki. Br. Med. J., 2, 177."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/1745-6215-11-32","article-title":"CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials","volume":"11","author":"Schulz","year":"2010","journal-title":"Trials"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1001\/jama.2023.9630","article-title":"Generative AI in Health Care and Liability Risks for Physicians and Safety Concerns for Patients","volume":"330","author":"Duffourc","year":"2023","journal-title":"JAMA"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1001\/jama.2023.9618","article-title":"Health Care Privacy Risks of AI Chatbots","volume":"330","author":"Kanter","year":"2023","journal-title":"JAMA"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1001\/jama.2023.9651","article-title":"The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models","volume":"330","author":"Minssen","year":"2023","journal-title":"JAMA"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2456","DOI":"10.1056\/NEJMra2204787","article-title":"Artificial Intelligence in Molecular Medicine","volume":"388","author":"Gomes","year":"2023","journal-title":"N. Engl. J. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1211","DOI":"10.1056\/NEJMra2212850","article-title":"Where Medical Statistics Meets Artificial Intelligence","volume":"389","author":"Hunter","year":"2023","journal-title":"N. Engl. J. Med."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1001\/jama.2023.25054","article-title":"Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?","volume":"331","author":"Wachter","year":"2024","journal-title":"JAMA"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e244630","DOI":"10.1001\/jamanetworkopen.2024.4630","article-title":"Quality of Large Language Model Responses to Radiation Oncology Patient Care Questions","volume":"7","author":"Yalamanchili","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1075","DOI":"10.1007\/s11036-020-01530-6","article-title":"A Cautionary Tale for Machine Learning Design: Why We Still Need Human-Assisted Big Data Analysis","volume":"25","author":"Roccetti","year":"2020","journal-title":"Mob. Netw. Appl."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/2\/53\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:48:08Z","timestamp":1760107688000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/2\/53"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,24]]},"references-count":29,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["make6020053"],"URL":"https:\/\/doi.org\/10.3390\/make6020053","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,24]]}}}