{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T21:43:03Z","timestamp":1789681383891,"version":"4.0.1"},"reference-count":53,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2025,10,25]],"date-time":"2025-10-25T00:00:00Z","timestamp":1761350400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,10,25]],"date-time":"2025-10-25T00:00:00Z","timestamp":1761350400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100005624","name":"Medizinischen Hochschule Hannover","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100005624","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Br J Clin Pharmacol"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:sec>\n                    <jats:title>Aims<\/jats:title>\n                    <jats:p>To assess the utility of the artificial intelligence (AI) chatbot ChatGPT (openly available version 3.5) in responding to real\u2010world pharmacotherapeutic queries from healthcare professionals.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Three independent and blinded evaluators with different levels of medical expertise and professional experience (\n                      <jats:italic>beginner<\/jats:italic>\n                      ,\n                      <jats:italic>advanced<\/jats:italic>\n                      , and\n                      <jats:italic>expert<\/jats:italic>\n                      ) compared AI chatbot\u2010 and physician\u2010generated responses to 70 real\u2010world pharmacotherapeutic queries submitted to the clinical\u2010pharmacological drug information centre of Hannover Medical School between June and October 2023 with regard to quality of information, answer preference, answer correctness and quality of language. Inter\u2010rater reliability was assessed with Krippendorff's alpha. Two separate investigators not otherwise involved in the conduct or analysis of the study selected the top three clinically relevant errors in chatbot\u2010 and physician\u2010generated responses.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      All three evaluators rated the quality of information of physician\u2010generated responses higher than the quality of information of AI chatbot\u2010generated responses and, accordingly, thought that the physician\u2010generated responses were better than the chatbot\u2010generated responses (answer preference). All evaluators detected factually wrong information more frequently in chatbot\u2010generated responses than in physician\u2010generated responses. Although the\n                      <jats:italic>beginner<\/jats:italic>\n                      and\n                      <jats:italic>expert<\/jats:italic>\n                      evaluators rated the quality of language of physician\u2010generated responses higher than the quality of language of chatbot\u2010generated responses, there was no significant difference according to the\n                      <jats:italic>advanced<\/jats:italic>\n                      evaluator.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>ChatGPT's responses to real\u2010world pharmacotherapeutic queries were substantially inferior compared to conventional physician\u2010generated responses with regard to quality of information and factual correctness. Our study suggests that to date it must be strongly cautioned against the use of ChatGPT in pharmacotherapy counselling.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/bcp.70321","type":"journal-article","created":{"date-parts":[[2025,10,25]],"date-time":"2025-10-25T14:04:15Z","timestamp":1761401055000},"page":"891-902","source":"Crossref","is-referenced-by-count":5,"title":["Human\n                    <i>vs<\/i>\n                    . artificial intelligence: Physicians outperform ChatGPT in real\u2010world pharmacotherapy counselling"],"prefix":"10.1002","volume":"92","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5899-0113","authenticated-orcid":false,"given":"Benjamin","family":"Krichevsky","sequence":"first","affiliation":[{"name":"Hannover Medical School Institute for General Practice and Palliative Care  Hannover Germany"},{"name":"Interdisciplinary Emergency Department University Hospital Schleswig\u2010Holstein (UKSH), Campus Kiel  Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3087-971X","authenticated-orcid":false,"given":"Stefan","family":"Engeli","sequence":"additional","affiliation":[{"name":"Institute for Pharmacology University Medicine Greifswald  Greifswald Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5132-6591","authenticated-orcid":false,"given":"Stefanie M.","family":"Bode\u2010B\u00f6ger","sequence":"additional","affiliation":[{"name":"Institute for Clinical Pharmacology Otto von Guericke University of Magdeburg  Magdeburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7378-7877","authenticated-orcid":false,"given":"Felix","family":"Koop","sequence":"additional","affiliation":[{"name":"Hannover Medical School Institute for Clinical Pharmacology  Hannover Germany"},{"name":"Department of Internal Medicine II, Division of Clinical Toxicology Technical University of Munich  Munich Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3262-8137","authenticated-orcid":false,"given":"Martin","family":"Schulze Westhoff","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Social Psychiatry and Psychotherapy Hannover Medical School  Hannover Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0905-274X","authenticated-orcid":false,"given":"Sebastian","family":"Schr\u00f6der","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Social Psychiatry and Psychotherapy Hannover Medical School  Hannover Germany"},{"name":"PRACTIS Clinician Scientist Program, Dean's Office for Academic Career Development Hannover Medical School  Hannover Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3012-4245","authenticated-orcid":false,"given":"Carsten","family":"Schumacher","sequence":"additional","affiliation":[{"name":"Hannover Medical School Institute for Clinical Pharmacology  Hannover Germany"},{"name":"Center for Clinical Trials Hannover Medical School  Hannover Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1031-1913","authenticated-orcid":false,"given":"Thorben","family":"Pape","sequence":"additional","affiliation":[{"name":"Department of Respiratory Medicine and Infectious Diseases Hannover Medical School  Hannover Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6526-0917","authenticated-orcid":false,"given":"Dirk O.","family":"Stichtenoth","sequence":"additional","affiliation":[{"name":"Hannover Medical School Institute for Clinical Pharmacology  Hannover Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5382-3014","authenticated-orcid":false,"given":"Johannes","family":"Heck","sequence":"additional","affiliation":[{"name":"Hannover Medical School Institute for Clinical Pharmacology  Hannover Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,10,25]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.15446\/rcciquifa.v45n2.59940"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1177\/0018578717724235"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010022\u201024005\u2010y"},{"key":"e_1_2_11_5_1","unstructured":"OpenAI.Introducing ChatGPT.2022. Available at:https:\/\/openai.com\/index\/chatgpt\/. Accessed May 26 2025."},{"key":"e_1_2_11_6_1","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang L","year":"2022","journal-title":"NeurIPS"},{"key":"e_1_2_11_7_1","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare11060887"},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsr2214184"},{"key":"e_1_2_11_9_1","doi-asserted-by":"publisher","DOI":"10.3238\/arztebl.m2023.0124"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.annonc.2019.10.013"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.211590"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12916\u2010021\u201001928\u20103"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.3238\/arztebl.m2023.0113"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12909\u2010023\u201004832\u2010x"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00228\u2010023\u201003547\u20108"},{"key":"e_1_2_11_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.15930"},{"key":"e_1_2_11_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.3083"},{"issue":"7","key":"e_1_2_11_19_1","first-page":"49","article-title":"ChatGPT zum thema PPI\u2010langzeittherapie: Ist k\u00fcnstliche intelligenz eine quelle f\u00fcr unabh\u00e4ngige informationen zu arzneimitteln?","volume":"57","year":"2023","journal-title":"Der Arzneimittelbrief"},{"key":"e_1_2_11_20_1","unstructured":"Arzneimittelbrief.Long\u2010term PPI therapy: assessment.2023. Available at:https:\/\/chat.openai.com\/share\/51012c1a-8ca9-4a4d-a0bb-f731a39c4860. Accessed May 28 2024."},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.3389\/fphar.2021.814858"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.3238\/arztebl.2016.0627"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2023.1835"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.16212"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00108\u2010023\u201001602\u20101"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.3758\/bf03193146"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.3758\/BRM.41.4.1149"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.1136\/ejhpharm-2023-003750"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1080\/19312450709336664"},{"key":"e_1_2_11_31_1","volume-title":"Content Analysis: An Introduction to Its Methodology","author":"Krippendorff K","year":"2004"},{"key":"e_1_2_11_32_1","first-page":"1877","article-title":"Language models are few\u2010shot learners","volume":"33","author":"Brown T","year":"2020","journal-title":"NeurIPS"},{"key":"e_1_2_11_33_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586\u2010023\u201000680\u20103"},{"key":"e_1_2_11_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00115\u2010020\u201000920\u20101"},{"key":"e_1_2_11_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00108\u2010023\u201001600\u20103"},{"key":"e_1_2_11_36_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.36483"},{"key":"e_1_2_11_37_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.3585"},{"key":"e_1_2_11_38_1","doi-asserted-by":"publisher","DOI":"10.61409\/A05240360"},{"key":"e_1_2_11_39_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.15896"},{"key":"e_1_2_11_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sapharm.2023.08.012"},{"key":"e_1_2_11_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pharma.2023.11.001"},{"key":"e_1_2_11_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.japh.2023.11.023"},{"key":"e_1_2_11_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/08971900241256731"},{"key":"e_1_2_11_44_1","article-title":"Assessing accuracy of ChatGPT in response to questions from day to day pharmaceutical care in hospitals","volume":"15","author":"Nuland M","year":"2024","journal-title":"Explor Res Clin Soc Pharm"},{"key":"e_1_2_11_45_1","doi-asserted-by":"publisher","DOI":"10.1002\/jcph.2443"},{"key":"e_1_2_11_46_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMc2305286"},{"key":"e_1_2_11_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00108\u2010023\u201001598\u20108"},{"key":"e_1_2_11_48_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591\u2010024\u201003180\u20107"},{"key":"e_1_2_11_49_1","unstructured":"OpenAI.Introducing GPT\u20105. 2025. Available at:https:\/\/openai.com\/index\/introducing-gpt-5\/. Accessed August 14 2025."},{"key":"e_1_2_11_50_1","unstructured":"CapootA.OpenAI launches new GPT\u20105 model for all ChatGPT users.2025. Available at:https:\/\/www.cnbc.com\/2025\/08\/07\/openai-launches-gpt-5-model-for-all-chatgpt-users.html. Accessed August 14 2025."},{"key":"e_1_2_11_51_1","unstructured":"Google DeepMind.Google Gemini. Available at:https:\/\/gemini.google.com\/app. Accessed August 14 2025."},{"key":"e_1_2_11_52_1","unstructured":"Meta AI.Llama 4. Available at:https:\/\/www.llama.com\/. Accessed August 14 2025."},{"key":"e_1_2_11_53_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMra2302038"},{"key":"e_1_2_11_54_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcpt.13879"}],"container-title":["British Journal of Clinical Pharmacology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/bcp.70321","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/bcp.70321","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/bcp.70321","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T21:29:25Z","timestamp":1773005365000},"score":1,"resource":{"primary":{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/10.1002\/bcp.70321"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,25]]},"references-count":53,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["10.1002\/bcp.70321"],"URL":"https:\/\/doi.org\/10.1002\/bcp.70321","archive":["Portico"],"relation":{},"ISSN":["0306-5251","1365-2125"],"issn-type":[{"value":"0306-5251","type":"print"},{"value":"1365-2125","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,25]]}}}