{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T18:57:13Z","timestamp":1776279433079,"version":"3.50.1"},"reference-count":60,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T00:00:00Z","timestamp":1718323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Intramural Research"},{"DOI":"10.13039\/100000050","name":"NHLBI","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>To evaluate the efficacy of ChatGPT 4 (GPT-4) in delivering genetic information about BRCA1, HFE, and MLH1, building on previous findings with ChatGPT 3.5 (GPT-3.5). To focus on assessing the utility, limitations, and ethical implications of using ChatGPT in medical settings.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>A structured survey was developed to assess GPT-4\u2019s clinical value. An expert panel of genetic counselors and clinical geneticists evaluated GPT-4\u2019s responses to these questions. We also performed comparative analysis with GPT-3.5, utilizing descriptive statistics and using Prism 9 for data analysis.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The findings indicate improved accuracy in GPT-4 over GPT-3.5 (P\u2009&amp;lt;\u2009.0001). However, notable errors in accuracy remained. The relevance of responses varied in GPT-4, but was generally favorable, with a mean in the \u201csomewhat agree\u201d range. There was no difference in performance by disease category. The 7-question subset of the Bot Usability Scale\u00a0(BUS-15) showed no statistically significant difference between the groups but trended lower in the GPT-4 version.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion and Conclusion<\/jats:title>\n                  <jats:p>The study underscores GPT-4\u2019s potential role in genetic education, showing notable progress yet facing challenges like outdated information and the necessity of ongoing refinement. Our results, while showing promise, emphasizes the importance of balancing technological innovation with ethical responsibility in healthcare information delivery.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae128","type":"journal-article","created":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T13:47:34Z","timestamp":1718372854000},"page":"2271-2283","source":"Crossref","is-referenced-by-count":31,"title":["A comparative evaluation of ChatGPT 3.5 and ChatGPT 4 in responses to selected genetics questions"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0065-726X","authenticated-orcid":false,"given":"Scott P","family":"McGrath","sequence":"first","affiliation":[{"name":"CITRIS Health, University of California Berkeley , Berkeley, CA 94720-1764, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9757-7118","authenticated-orcid":false,"given":"Beth A","family":"Kozel","sequence":"additional","affiliation":[{"name":"Laboratory of Vascular and Matrix Genetics, National Heart, Lung, and Blood Institute (NHLBI) , Bethesda, MD 20892, United States"}]},{"given":"Sara","family":"Gracefo","sequence":"additional","affiliation":[{"name":"Intermountain Precision Genomics, Intermountain Healthcare , St George, UT 84790-8723, United States"}]},{"given":"Nykole","family":"Sutherland","sequence":"additional","affiliation":[{"name":"Intermountain Precision Genomics, Intermountain Healthcare , St George, UT 84790-8723, United States"}]},{"given":"Christopher J","family":"Danford","sequence":"additional","affiliation":[{"name":"Transplant Services, Intermountain Medical Center Murray, UT 84107, United States"}]},{"given":"Nephi","family":"Walton","sequence":"additional","affiliation":[{"name":"National Human Genome Research Institute, National Institute of Health , Bethesda, MD 20892-2152, United States"}]}],"member":"286","published-online":{"date-parts":[[2024,6,14]]},"reference":[{"key":"2024092007533282500_ocae128-B1","author":"Radford","year":"2018"},{"key":"2024092007533282500_ocae128-B2","author":"OpenAI","year":"2023"},{"key":"2024092007533282500_ocae128-B3","author":"Ammu","year":"2023"},{"key":"2024092007533282500_ocae128-B4","author":"Loron"},{"key":"2024092007533282500_ocae128-B5","author":"Pichai"},{"key":"2024092007533282500_ocae128-B6","author":"Touvron","year":"2023"},{"key":"2024092007533282500_ocae128-B7","author":"Antropic","year":"2024"},{"key":"2024092007533282500_ocae128-B8","author":"Mittal","year":"2024"},{"key":"2024092007533282500_ocae128-B9","author":"Pierce","year":"2024"},{"key":"2024092007533282500_ocae128-B10","year":"2024"},{"key":"2024092007533282500_ocae128-B11","author":"Perez","year":"2024"},{"key":"2024092007533282500_ocae128-B12","author":"Health A","year":"2024"},{"issue":"1","key":"2024092007533282500_ocae128-B13","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1038\/s41746-023-00873-0","article-title":"The imperative for regulatory oversight of large language models (or generative AI) in healthcare","volume":"6","author":"Mesk\u00f3","year":"2023","journal-title":"NPJ Digit Med"},{"key":"2024092007533282500_ocae128-B14","author":"Leonard","year":"2024"},{"issue":"4","key":"2024092007533282500_ocae128-B15","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1038\/s41431-023-01419-4","article-title":"Can ChatGPT understand genetics?","volume":"32","author":"Emmert-Streib","year":"2023","journal-title":"Eur J Hum Genet"},{"key":"2024092007533282500_ocae128-B16","article-title":"Analysis of large-language model versus human performance for genetics questions","author":"Duong","year":"2023","journal-title":"Eur J Human Genet"},{"key":"2024092007533282500_ocae128-B17","author":"Walton","year":"2023"},{"issue":"3","key":"2024092007533282500_ocae128-B18","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1038\/s41591-023-02289-5","article-title":"Will ChatGPT transform healthcare?","volume":"29","year":"2023","journal-title":"Nat Med"},{"key":"2024092007533282500_ocae128-B19","author":"Nori","year":"2023"},{"issue":"8","key":"2024092007533282500_ocae128-B20","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"2024092007533282500_ocae128-B21","author":"Rebelo","year":"2024"},{"key":"2024092007533282500_ocae128-B22","author":"McKenna","year":"2024"},{"issue":"1","key":"2024092007533282500_ocae128-B23","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1007\/s10897-017-0158-8","article-title":"Projecting the supply and demand for certified genetic counselors: a workforce study","volume":"27","author":"Hoskovec","year":"2018","journal-title":"J Genet Couns"},{"issue":"8","key":"2024092007533282500_ocae128-B24","doi-asserted-by":"crossref","first-page":"1458","DOI":"10.1038\/s41436-021-01162-5","article-title":"The 2019 US medical genetics workforce: a focus on clinical genetics","volume":"23","author":"Jenkins","year":"2021","journal-title":"Genet Med"},{"issue":"1","key":"2024092007533282500_ocae128-B25","doi-asserted-by":"crossref","first-page":"844","DOI":"10.1186\/s12913-019-4679-8","article-title":"Are providers prepared for genomic medicine: interpretation of Direct-to-Consumer genetic testing (DTC-GT) results and genetic self-efficacy by medical professionals","volume":"19","author":"McGrath","year":"2019","journal-title":"BMC Health Serv Res"},{"key":"2024092007533282500_ocae128-B26","author":"National Society of Genetic Counselors"},{"key":"2024092007533282500_ocae128-B27","author":"UGA Office","year":"2024"},{"issue":"5","key":"2024092007533282500_ocae128-B28","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1377\/hlthaff.2017.1427","article-title":"Genetic test availability and spending: where are we now? Where are we going?","volume":"37","author":"Phillips","year":"2018","journal-title":"Health Aff (Millwood)"},{"issue":"4","key":"2024092007533282500_ocae128-B29","doi-asserted-by":"crossref","first-page":"638","DOI":"10.3390\/jpm13040638","article-title":"Trends in availability of genetic tests in the United States, 2012\u20132022","volume":"13","author":"Halbisen","year":"2023","journal-title":"J Pers Med"},{"issue":"6","key":"2024092007533282500_ocae128-B30","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1159\/000489117","article-title":"Assessing genetic literacy awareness and knowledge gaps in the US population: results from the Health Information National Trends Survey","volume":"20","author":"Krakow","year":"2017","journal-title":"Public Health Genom"},{"issue":"4","key":"2024092007533282500_ocae128-B31","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1089\/gtmb.2012.0350","article-title":"Public knowledge of and attitudes toward genetics and genetic testing","volume":"17","author":"Haga","year":"2013","journal-title":"Genet Test Mol Biomarkers"},{"issue":"6","key":"2024092007533282500_ocae128-B32","doi-asserted-by":"crossref","first-page":"1488","DOI":"10.1016\/j.pec.2021.09.027","article-title":"Ask Rosa\u2014the making of a digital genetic conversation tool, a chatbot, about hereditary breast and ovarian cancer","volume":"105","author":"Siglen","year":"2022","journal-title":"Patient Educ Couns"},{"issue":"6","key":"2024092007533282500_ocae128-B33","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1002\/jgc4.1169","article-title":"Patient assessment of chatbots for the scalable delivery of genetic counseling","volume":"28","author":"Schmidlen","year":"2019","journal-title":"J Genet Couns"},{"key":"2024092007533282500_ocae128-B34","author":"Sallam"},{"issue":"13","key":"2024092007533282500_ocae128-B35","doi-asserted-by":"crossref","first-page":"1201","DOI":"10.1056\/NEJMra2302038","article-title":"Artificial intelligence and machine learning in clinical medicine, 2023","volume":"388","author":"Haug","year":"2023","journal-title":"N Engl J Med"},{"issue":"1","key":"2024092007533282500_ocae128-B36","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1038\/s41591-018-0272-7","article-title":"Privacy in the age of medical big data","volume":"25","author":"Price","year":"2019","journal-title":"Nat Med"},{"key":"2024092007533282500_ocae128-B37","author":"Johnson","year":"2024"},{"issue":"6","key":"2024092007533282500_ocae128-B38","doi-asserted-by":"crossref","first-page":"3576","DOI":"10.1007\/s00330-020-06672-5","article-title":"Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations","volume":"30","author":"Recht","year":"2020","journal-title":"Eur Radiol"},{"key":"2024092007533282500_ocae128-B39","doi-asserted-by":"crossref","first-page":"104672","DOI":"10.1016\/j.ebiom.2023.104672","article-title":"Attention is not all you need: the complicated case of ethically using large language models in healthcare and medicine","volume":"90","author":"Harrer","year":"2023","journal-title":"EBioMedicine"},{"issue":"6","key":"2024092007533282500_ocae128-B40","doi-asserted-by":"crossref","first-page":"e15154","DOI":"10.2196\/15154","article-title":"Artificial intelligence and human trust in healthcare: focus on clinicians","volume":"22","author":"Asan","year":"2020","journal-title":"J Med Internet Res"},{"key":"2024092007533282500_ocae128-B41","first-page":"e0000205","volume-title":"ChatGPT Passing USMLE Shines a Spotlight on the Flaws of Medical Education","author":"Mbakwe","year":"2023"},{"issue":"2","key":"2024092007533282500_ocae128-B42","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/s00779-022-01690-0","article-title":"A confirmatory factorial analysis of the Chatbot Usability Scale: a multilanguage validation","volume":"27","author":"Borsci","year":"2022","journal-title":"Pers Ubiquit Comput"},{"issue":"11","key":"2024092007533282500_ocae128-B43","doi-asserted-by":"crossref","first-page":"1867","DOI":"10.3390\/jpm12111867","article-title":"The development of an infrastructure to facilitate the use of whole genome sequencing for population health","volume":"12","author":"Walton","year":"2022","journal-title":"J Pers Med"},{"key":"2024092007533282500_ocae128-B44","doi-asserted-by":"publisher","author":"McGrath","year":"2024","DOI":"10.5061\/dryad.s4mw6m9cv"},{"issue":"6","key":"2024092007533282500_ocae128-B45","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1007\/s10439-023-03171-8","article-title":"Potential use of chat GPT in global warming","volume":"51","author":"Biswas","year":"2023","journal-title":"Ann Biomed Eng"},{"key":"2024092007533282500_ocae128-B46","article-title":"The rise of ChatGPT: Exploring its potential in medical education","author":"Lee","year":"2023","journal-title":"Anat Sci Educ"},{"key":"2024092007533282500_ocae128-B47","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.iotcps.2023.04.003","article-title":"ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope","volume":"3","author":"Ray","year":"2023","journal-title":"Internet of Things Cyber-Phys Syst"},{"issue":"6","key":"2024092007533282500_ocae128-B48","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1038\/s41397-023-00316-9","article-title":"Using ChatGPT to predict the future of personalized medicine","volume":"23","author":"Patrinos","year":"2023","journal-title":"Pharmacogenomics J"},{"issue":"2","key":"2024092007533282500_ocae128-B49","doi-asserted-by":"publisher","first-page":"536","DOI":"10.1093\/jamia\/ocad211","article-title":"Enabling the clinical application of artificial intelligence in genomics: a perspective of the AMIA Genomics and Translational Bioinformatics Workgroup","volume":"31","author":"Walton","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"2024092007533282500_ocae128-B50","doi-asserted-by":"crossref","first-page":"1166120","DOI":"10.3389\/fpubh.2023.1166120","article-title":"ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health","volume":"11","author":"De Angelis","year":"2023","journal-title":"Front Public Health"},{"key":"2024092007533282500_ocae128-B51","article-title":"Generative AI has an intellectual property problem","volume":"7.","author":"Appel","year":"2023","journal-title":"Harvard Business Rev"},{"key":"2024092007533282500_ocae128-B52","author":"Wilkinson","year":"2024"},{"key":"2024092007533282500_ocae128-B53","author":"OpenAI"},{"issue":"5","key":"2024092007533282500_ocae128-B54","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1038\/s42254-023-00581-4","article-title":"Science in the age of large language models","volume":"5","author":"Birhane","year":"2023","journal-title":"Nat Rev Phys"},{"key":"2024092007533282500_ocae128-B55","author":"Genkina","year":"2024"},{"key":"2024092007533282500_ocae128-B56","author":"David","year":"2024"},{"key":"2024092007533282500_ocae128-B57","author":"Lipenkova"},{"issue":"10411","key":"2024092007533282500_ocae128-B58","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1016\/S0140-6736(23)02292-4","article-title":"Machines and empathy in medicine","volume":"402","author":"Topol","year":"2023","journal-title":"Lancet"},{"key":"2024092007533282500_ocae128-B59","author":"Sorin","year":"2023"},{"issue":"6","key":"2024092007533282500_ocae128-B60","doi-asserted-by":"publisher","first-page":"1356","DOI":"10.1093\/jamia\/ocae039","article-title":"Empowering personalized pharmacogenomics with generative AI solutions","volume":"31","author":"Murugan","year":"2024","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/10\/2271\/59206392\/ocae128.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/10\/2271\/59206392\/ocae128.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T07:53:55Z","timestamp":1726818835000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/10\/2271\/7693164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,14]]},"references-count":60,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,6,14]]},"published-print":{"date-parts":[[2024,10,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae128","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,10]]},"published":{"date-parts":[[2024,6,14]]}}}