{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T13:43:39Z","timestamp":1791639819010,"version":"4.3.4"},"reference-count":39,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Educ"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) have revolutionized natural language processing with their ability to generate human-like text through extensive training on large data sets. These models, including Generative Pre-trained Transformers (GPT)-3.5 (OpenAI), GPT-4 (OpenAI), and Bard (Google LLC), find applications beyond natural language processing, attracting interest from academia and industry. Students are actively leveraging LLMs to enhance learning experiences and prepare for high-stakes exams, such as the National Eligibility cum Entrance Test (NEET) in India.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This comparative analysis aims to evaluate the performance of GPT-3.5, GPT-4, and Bard in answering NEET-2023 questions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this paper, we evaluated the performance of the 3 mainstream LLMs, namely GPT-3.5, GPT-4, and Google Bard, in answering questions related to the NEET-2023 exam. The questions of the NEET were provided to these artificial intelligence models, and the responses were recorded and compared against the correct answers from the official answer key. Consensus was used to evaluate the performance of all 3 models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>It was evident that GPT-4 passed the entrance test with flying colors (300\/700, 42.9%), showcasing exceptional performance. On the other hand, GPT-3.5 managed to meet the qualifying criteria, but with a substantially lower score (145\/700, 20.7%). However, Bard (115\/700, 16.4%) failed to meet the qualifying criteria and did not pass the test. GPT-4 demonstrated consistent superiority over Bard and GPT-3.5 in all 3 subjects. Specifically, GPT-4 achieved accuracy rates of 73% (29\/40) in physics, 44% (16\/36) in chemistry, and 51% (50\/99) in biology. Conversely, GPT-3.5 attained an accuracy rate of 45% (18\/40) in physics, 33% (13\/26) in chemistry, and 34% (34\/99) in biology. The accuracy consensus metric showed that the matching responses between GPT-4 and Bard, as well as GPT-4 and GPT-3.5, had higher incidences of being correct, at 0.56 and 0.57, respectively, compared to the matching responses between Bard and GPT-3.5, which stood at 0.42. When all 3 models were considered together, their matching responses reached the highest accuracy consensus of 0.59.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The study\u2019s findings provide valuable insights into the performance of GPT-3.5, GPT-4, and Bard in answering NEET-2023 questions. GPT-4 emerged as the most accurate model, highlighting its potential for educational applications. Cross-checking responses across models may result in confusion as the compared models (as duos or a trio) tend to agree on only a little over half of the correct responses. Using GPT-4 as one of the compared models will result in higher accuracy consensus. The results underscore the suitability of LLMs for high-stakes exams and their positive impact on education. Additionally, the study establishes a benchmark for evaluating and enhancing LLMs\u2019 performance in educational tasks, promoting responsible and informed use of these models in diverse learning environments.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/51523","type":"journal-article","created":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T10:58:34Z","timestamp":1698663514000},"page":"e51523","source":"Crossref","is-referenced-by-count":60,"title":["Evaluating Large Language Models for the National Premedical Exam in India: Comparative Analysis of GPT-3.5, GPT-4, and Bard"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1310-1586","authenticated-orcid":false,"given":"Faiza","family":"Farhat","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0437-6924","authenticated-orcid":false,"given":"Beenish Moalla","family":"Chaudhry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3664-5014","authenticated-orcid":false,"given":"Mohammad","family":"Nadeem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5944-7371","authenticated-orcid":false,"given":"Shahab Saquib","family":"Sohail","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8735-3332","authenticated-orcid":false,"given":"Dag \u00d8ivind","family":"Madsen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2024,2,21]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1080\/23311916.2023.2222988"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1108\/ijchm-05-2023-0686"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03305-y"},{"key":"ref4","unstructured":"LiuJLZhengJCaiXYinCA descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeonsiScience202308092024-01-27CellPresshttps:\/\/www.cell.com\/iscience\/fulltext\/S2589-0042(23)01667-X"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-023-00621-y"},{"key":"ref6","unstructured":"BeerbaumDGenerative artificial intelligence (GAI) with ChatGPT for accounting: a business caseSocial Science Research Network20232023-12-12https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=4385651"},{"key":"ref7","unstructured":"RaneNRole and challenges of ChatGPT and similar generative artificial intelligence in human resource managementSocial Science Research Network20232024-01-24https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=4603230"},{"key":"ref8","unstructured":"McGeeRWWhat are the top 20 questions in sociology? a ChatGPT replyResearchGate20232023-12-13https:\/\/www.researchgate.net\/profile\/Robert-Mcgee-5\/publication\/369972268_What_Are_the_Top_20_Questions_in_Sociology_A_ChatGPT_Reply\/links\/643727154e83cd0e2fab3dc1\/What-Are-the-Top-20-Questions-in-Sociology-A-ChatGPT-Reply.pdf"},{"key":"ref9","author":"Cao, Y","journal-title":"ArXiv. Preprint posted online on March 7, 2023"},{"key":"ref10","author":"Zhao, WX","journal-title":"ArXiv. Preprint posted online on November 24, 2023"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.52866\/20ijcsm.2023.01.01.0018"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1016\/j.iotcps.2023.04.003"},{"key":"ref13","unstructured":"Google's AI -- BardBard20232024-01-27https:\/\/bard.google.com\/chat"},{"key":"ref14","unstructured":"BorjiAMohammadianMBattle of the Wordsmiths: comparing ChatGPT, GPT-4, Claude, and BardSocial Science Research Network20232023-12-13https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=4476855"},{"key":"ref15","author":"Touvron, H","journal-title":"ArXiv. Preprint posted online on February 27, 2023"},{"key":"ref16","author":"Wu, S","journal-title":"ArXiv. Preprint posted online on May 9, 2023"},{"issue":"240","key":"ref17","first-page":"1","volume":"24","author":"Chowdhery, A","year":"2023","journal-title":"J Mach Learn Res"},{"key":"ref18","unstructured":"OpenAI's ChatGPTOpenAI20222024-01-27https:\/\/openai.com\/ChatGPT"},{"key":"ref19","author":"Thoppilan, R","journal-title":"ArXiv. Preprint posted online on February 10, 2022"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1186\/s40561-023-00237-x"},{"issue":"1","key":"ref21","volume":"5","author":"Hong, WCH","year":"2023","journal-title":"J Educ Technol Inov"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.2196\/48291"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.35542\/osf.io\/sytu3"},{"key":"ref25","unstructured":"TerwieschCWould ChatGPT get a Wharton MBA? a prediction based on its performance in the operations management courseMack Institute for Innovation Management at the Wharton School, University of Pennsylvania20232023-12-13https:\/\/mackinstitute.wharton.upenn.edu\/wp-content\/uploads\/2023\/01\/Would-ChatGPT-get-a-Wharton-MBA.pdf"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.2308\/ISSUES-2023-013"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4335905"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1016\/j.resuscitation.2023.109783"},{"key":"ref29","unstructured":"Bommineni,VLBhagwagarSBalcarcelDDavatzikosCBoyerDPerformance of ChatGPT on the MCAT: the road to personalized and equitable premedical learningMedrxiv202306232024-01-26https:\/\/www.medrxiv.org\/content\/10.1101\/2023.03.05.23286533v3"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.2196\/45312"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"key":"ref32","author":"Teebagy, S","journal-title":"medRxiv. Preprint posted online on April 03, 2023"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1227\/neu.0000000000002551"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1080\/10872981.2023.2220920"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1108\/QAE-07-2018-0080"},{"key":"ref36","author":"Liu, X","journal-title":"medRxiv. Preprint posted online on April 18, 2023"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03335-6"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03326-7"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000205"}],"container-title":["JMIR Medical Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/mededu.jmir.org\/2024\/1\/e51523","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:35:26Z","timestamp":1782840926000},"score":1,"resource":{"primary":{"URL":"https:\/\/mededu.jmir.org\/2024\/1\/e51523"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,21]]},"references-count":39,"URL":"https:\/\/doi.org\/10.2196\/51523","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.51523","asserted-by":"object"}]},"ISSN":["2369-3762"],"issn-type":[{"value":"2369-3762","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,21]]}}}