{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T07:39:16Z","timestamp":1793173156854,"version":"4.5.0"},"reference-count":26,"publisher":"Wiley","issue":"8","license":[{"start":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T00:00:00Z","timestamp":1749513600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T00:00:00Z","timestamp":1749513600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Br J Clin Pharmacol"],"published-print":{"date-parts":[[2026,8]]},"abstract":"<jats:sec>\n                    <jats:title>Aims<\/jats:title>\n                    <jats:p>In medical education, the ability of large language models (LLMs) to match human performance raises questions about their potential as educational tools. This study evaluates LLMs' performance on Clinical Pharmacology and Therapeutics (CPT) exams, comparing their results to medical students and exploring their ability to identify poorly formulated multiple\u2010choice questions (MCQs).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      ChatGPT\u20104 Omni, Gemini Advanced, Le Chat and DeepSeek R1 were tested on local CPT exams (third year of bachelor's degree, first\/second year of master's degree) and the European Prescribing Exam (EuroPE\n                      <jats:sup>+<\/jats:sup>\n                      ). The exams included MCQs and open\u2010ended questions assessing knowledge and prescribing skills. LLM results were analysed using the same scoring system as students. A confusion matrix was used to evaluate the ability of ChatGPT and Gemini to identify ambiguous\/erroneous MCQs.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      LLMs achieved comparable or superior results to medical students across all levels. For local exams, LLMs outperformed M1 students and matched L3 and M2 students. In EuroPE\n                      <jats:sup>+<\/jats:sup>\n                      , LLMs significantly outperformed students in both the knowledge and prescribing skills sections. All LLM errors in EuroPE\n                      <jats:sup>+<\/jats:sup>\n                      were genuine (100%), whereas local exam errors were frequently due to ambiguities or correction flaws (24.3%). When both ChatGPT and Gemini provided the same incorrect answer to an MCQ, the specificity for detecting ambiguous questions was 92.9%, with a negative predictive value of 85.5%.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs demonstrate capabilities comparable to or exceeding medical students in CPT exams. Their ability to flag potentially flawed MCQs highlights their value not only as educational tools but also as quality control instruments in exam preparation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/bcp.70137","type":"journal-article","created":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T23:49:17Z","timestamp":1749599357000},"page":"2574-2583","source":"Crossref","is-referenced-by-count":13,"title":["Evaluating and leveraging large language models in clinical pharmacology and therapeutics assessment: From exam takers to exam shapers"],"prefix":"10.1002","volume":"92","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6591-6966","authenticated-orcid":false,"given":"Alexandre O.","family":"G\u00e9rard","sequence":"first","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7669-2339","authenticated-orcid":false,"given":"Diane","family":"Merino","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5840-8904","authenticated-orcid":false,"given":"Marc","family":"Labriffe","sequence":"additional","affiliation":[{"name":"Pharmacology and Transplantation INSERM U1248, Universit\u00e9 de Limoges  Limoges France"},{"name":"Department of Pharmacology, Toxicology and Pharmacovigilance University Hospital of Limoges  Limoges France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fanny","family":"Rocher","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6140-4664","authenticated-orcid":false,"given":"Delphine","family":"Viard","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laurence","family":"Zemori","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thibaud","family":"Lavrut","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8169-0714","authenticated-orcid":false,"given":"Erik M.","family":"Donker","sequence":"additional","affiliation":[{"name":"Research and Expertise Centre in Pharmacotherapy Education (RECIPE)  Amsterdam The Netherlands"},{"name":"Department of Internal Medicine, Unit Pharmacotherapy Amsterdam UMC, Vrije Universiteit  Amsterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-7380-4876","authenticated-orcid":false,"given":"Joost D.","family":"Pi\u00ebt","sequence":"additional","affiliation":[{"name":"Research and Expertise Centre in Pharmacotherapy Education (RECIPE)  Amsterdam The Netherlands"},{"name":"Department of Internal Medicine, Unit Pharmacotherapy Amsterdam UMC, Vrije Universiteit  Amsterdam The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1363-5107","authenticated-orcid":false,"given":"Jean\u2010Paul","family":"Fournier","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur, Faculty of Medicine of Nice Medical Simulation Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4121-530X","authenticated-orcid":false,"given":"Milou\u2010Daniel","family":"Drici","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6147-9201","authenticated-orcid":false,"given":"Alexandre","family":"Destere","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur Medical Centre Department of Clinical Pharmacology and Pharmacovigilance Center  Nice France"},{"name":"Universit\u00e9 C\u00f4te d'Azur, Inria, CNRS, Laboratoire J.A. Dieudonn\u00e9 Maasai team  Nice France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,6,10]]},"reference":[{"key":"e_1_2_8_2_1","unstructured":"BubeckS ChandrasekaranV EldanR et al.Sparks of artificial general intelligence: early experiments with GPT\u20104. arXiv.org. 2023;abs\/2303.12712. doi:10.48550\/arXiv.2303.12712"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2022.929175"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2023.0254"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare11060887"},{"key":"e_1_2_8_6_1","unstructured":"SchmidgallS. HarrisC. EssienI. OlshvangD. RahmanT. KimJ.W.et al.Addressing cognitive bias in medical language models.2024. doi:10.48550\/arXiv.2402.08113"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591\u2010024\u201003097\u20101"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13000\u2010024\u201001464\u20107"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.64924"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/23821205241238641"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010024\u201068996\u20102"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1093\/ckj\/sfae193"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajoint.2024.100063"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.3390\/clinpract13060130"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12909\u2010024\u201005365\u20107"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.15977"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1080\/0142159X.2023.2294703"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00228\u2010024\u201003649\u2010x"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00228\u2010022\u201003301\u20106"},{"key":"e_1_2_8_21_1","unstructured":"Tichelaar J Brinkman D Donker E vanAgtmaelM. LikicE. B\u00f6ttigerY.et alInternational objectives on medication safety.https:\/\/www.prescribingeducation.eu\/wp-content\/uploads\/2020\/10\/Learning-objectives-European-Prescribing-Exam.pdf"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1111\/bcp.13730"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpt.962"},{"key":"e_1_2_8_24_1","unstructured":"KassambaraA.Rstatix:pipe\u2010friendly framework for basic statistical tests.2023.https:\/\/CRAN.R-project.org\/package=rstatix"},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24277-4"},{"key":"e_1_2_8_26_1","volume-title":"PS\u2010063 \u2010 poster abstracts \u2010 2024 \u2010 fundamental & clinical pharmacology"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1097\/JCMA.0000000000000942"}],"container-title":["British Journal of Clinical Pharmacology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/bcp.70137","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/bcp.70137","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/bcp.70137","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T00:26:19Z","timestamp":1785371179000},"score":1,"resource":{"primary":{"URL":"https:\/\/bpspubs.onlinelibrary.wiley.com\/doi\/10.1002\/bcp.70137"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,10]]},"references-count":26,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2026,8]]}},"alternative-id":["10.1002\/bcp.70137"],"URL":"https:\/\/doi.org\/10.1002\/bcp.70137","archive":["Portico"],"relation":{},"ISSN":["0306-5251","1365-2125"],"issn-type":[{"value":"0306-5251","type":"print"},{"value":"1365-2125","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,10]]}}}