{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T12:00:37Z","timestamp":1784808037522,"version":"3.55.0"},"reference-count":4,"publisher":"Wiley","issue":"7","license":[{"start":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T00:00:00Z","timestamp":1782691200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T00:00:00Z","timestamp":1782691200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/100013607","name":"American College of Clinical Pharmacy","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100013607","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["accpjournals.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Am Coll Clin Pharm"],"published-print":{"date-parts":[[2026,7]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Generative artificial intelligence (AI) tools are increasingly used in health professional education, including for automated writing evaluation (AWE). While AI\u2010based grading may reduce faculty workload and variability, its alignment with human grading remains unclear. This study compared rubric\u2010based grading of pharmacy student drug information papers by faculty and ChatGPT (Open AI, San Francisco, CA).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      We conducted a retrospective comparative analysis of 159 de\u2010identified assignments from a required evidence\u2010based medicine course. Faculty\u2010assigned grades were paired with scores generated by a custom generative pretrained transformer (GPT) configured with the assignment rubric. The primary outcome was the difference in mean total scores; secondary outcomes included rubric section\u2010level scores, variability, and agreement metrics. Paired\n                      <jats:italic>t<\/jats:italic>\n                      \u2010tests, Lin's concordance correlation coefficient, and Cohen's weighted kappa were used for analysis.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Aggregate mean total scores were higher for ChatGPT versus faculty (51.3 vs. 49;\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.0004, mean difference, 2.3; 95% confidence interval, 1.015\u20133.595), with differences in five of seven rubric sections. Faculty scores exhibited greater variability than ChatGPT scores (standard deviation, 7.4 vs. 3.7). Concordance between faculty and AI grading at the individual student level was poor (Lin's coefficient for total score\u2009=\u20090.06; kappa for overall grade\u2009=\u20090.03). ChatGPT also assigned more \u201cB\u201d grades and fewer failing grades compared with faculty.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>AI\u2010based grading produced similar aggregate letter grades and reduced variability but demonstrated poor agreement with faculty scores at the individual level. AI grading may complement, not replace, faculty evaluation for assignments requiring critical appraisal, but further studies are needed.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/jac5.70249","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T03:33:29Z","timestamp":1782790409000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Comparative Study of Artificial Intelligence and Faculty Rubric\u2010Based Grading of Pharmacy Student Writing Assignments in an Evidence\u2010Based Medicine Course"],"prefix":"10.1002","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3251-2823","authenticated-orcid":false,"given":"Jennifer","family":"Phillips","sequence":"first","affiliation":[{"name":"University of Illinois Chicago Retzky College of Pharmacy  Chicago Illinois USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0086-2562","authenticated-orcid":false,"given":"Katherine","family":"Sarna","sequence":"additional","affiliation":[{"name":"University of Illinois Chicago Retzky College of Pharmacy  Chicago Illinois USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6642-498X","authenticated-orcid":false,"given":"Faria","family":"Munir","sequence":"additional","affiliation":[{"name":"University of Illinois Chicago Retzky College of Pharmacy  Chicago Illinois USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0981-1358","authenticated-orcid":false,"given":"Heather","family":"Ipema","sequence":"additional","affiliation":[{"name":"University of Illinois Chicago Retzky College of Pharmacy  Chicago Illinois USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,6,29]]},"reference":[{"key":"e_1_2_11_2_1","volume-title":"Artificial Intelligence and the Future of Teaching and Learning Insights and Recommendations","author":"Cardona M. A.","year":"2023"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.3928\/01484834\u201020231006\u201002"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajpe.2025.101877"},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajpe.2024.100866"}],"container-title":["JACCP:  JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/accpjournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jac5.70249","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/accpjournals.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/jac5.70249","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/accpjournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jac5.70249","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T09:20:35Z","timestamp":1783502435000},"score":1,"resource":{"primary":{"URL":"https:\/\/accpjournals.onlinelibrary.wiley.com\/doi\/10.1002\/jac5.70249"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,29]]},"references-count":4,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2026,7]]}},"alternative-id":["10.1002\/jac5.70249"],"URL":"https:\/\/doi.org\/10.1002\/jac5.70249","archive":["Portico"],"relation":{},"ISSN":["2574-9870","2574-9870"],"issn-type":[{"value":"2574-9870","type":"print"},{"value":"2574-9870","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,29]]},"assertion":[{"value":"2026-01-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70249"}}