{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T06:52:19Z","timestamp":1781679139508,"version":"3.54.5"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Use of Large Language Models (LLMs) as a health information source is on the rise. As the amount of people willing to self-diagnose with the tools\u2019 help is increasing, verifying the correctness of generated responses becomes extremely important.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Four LLMs (ChatGPT, Copilot, DeepSeek, Gemini) were chosen for the analysis. All models were asked 90 multiple-choice questions on the various topics surrounding vaccination, sourced from a government information website. Additionally, the LLMs were asked two open-ended questions regarding the vaccine information sources, as well as advantages and disadvantages of universal vaccination. Answers were analysed quantitatively and qualitatively. Data was collected in March 2026.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Biggest proportion of correct answers was achieved by Gemini (0.98), smallest one \u2013 by ChatGPT (0.86). Models\u2019 results were found to be non-homogeneous (\n                      <jats:italic>p<\/jats:italic>\n                      \u202f=\u202f0.0049). While answering open-ended questions, all LLMs were able to cite various, appropriate vaccine information sources. For advantages of vaccination all models pointed to herd immunity. Answers regarding disadvantages to vaccination all included the possibility of adverse reactions. The answers were judged to be factually correct, yet lacking in detail.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Tested LLMs were able to identify correct option when presented with multiple choices and to prepare correct, yet vague information when asked an open-ended question. This points to a limited potential of LLMs serving as a vaccine information source, as the generated content, while factually correct, was not enough to be usable on its own.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1811462","type":"journal-article","created":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T05:55:36Z","timestamp":1781675736000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Assessment of vaccine information accuracy across large language models"],"prefix":"10.3389","volume":"14","author":[{"given":"Julia","family":"Siebielec","sequence":"first","affiliation":[{"name":"Department of Environmental Hazard Prevention, Allergology and Immunology, Warsaw Medical University","place":["Warsaw, Poland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Filip","family":"Raciborski","sequence":"additional","affiliation":[{"name":"Department of Environmental Hazard Prevention, Allergology and Immunology, Warsaw Medical University","place":["Warsaw, Poland"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,6,17]]},"reference":[{"key":"ref1","doi-asserted-by":"crossref","DOI":"10.3386\/w34255","volume-title":"How People Use ChatGPT","author":"Chatterji","year":"2025"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1038\/s41746-024-01157-x","article-title":"The ethics of ChatGPT in medicine and healthcare: a systematic review on large language models (LLMs)","volume":"7","author":"Haltaufderheide","year":"2024","journal-title":"Npj Digit Med"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"210","DOI":"10.5694\/mja2.52598","article-title":"Who is asking ChatGPT health questions? Analysis of a nationally representative Australian community sample","volume":"222","author":"Ayre","year":"2025","journal-title":"Med J Aust"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"e47564","DOI":"10.2196\/47564","article-title":"User intentions to use ChatGPT for self-diagnosis and health-related purposes: cross-sectional survey study","volume":"10","author":"Shahsavar","year":"2023","journal-title":"JMIR Hum Factors"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"20552076251412989","DOI":"10.1177\/20552076251412989","article-title":"Comparative evaluation of large language models for hip fracture-related patient questions: DeepSeek-V3-FW, Gemini 2.0 flash, and ChatGPT-4.5","volume":"12","author":"Zhang","year":"2026","journal-title":"Digit Health"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1186\/s12911-026-03371-x","article-title":"Evaluation of large language models on pediatric asthma: a comparative study of Claude3-opus, Gemini 2.0, ChatGPT-4o, and DeepSeek\u2014a cross-sectional questionnaire study","volume":"26","author":"Hang","year":"2026","journal-title":"BMC Med Inform Decis Mak"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1776697","DOI":"10.3389\/fpubh.2026.1776697","article-title":"The information challenge in public health crises: a study on the reliability and readability of information provided by large language model for thunderstorm asthma","volume":"14","author":"Zhu","year":"2026","journal-title":"Front Public Health"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"1777577","DOI":"10.3389\/fpubh.2026.1777577","article-title":"Decoupled quality and readability in skin cancer education from large language models","volume":"14","author":"Zhang","year":"2026","journal-title":"Front Public Health"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1186\/s12891-026-09731-y","article-title":"Evaluation of ChatGPT-5 responses to patient-centered questions on stromal vascular fraction for knee osteoarthritis: fair to good quality and content","volume":"27","author":"Mercan","year":"2026","journal-title":"BMC Musculoskelet Disord"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"129","DOI":"10.3390\/jpm16030129","article-title":"Evaluating large language models for diagnostic accuracy and health information quality in Oral mucosal diseases","volume":"16","author":"Iacob","year":"2026","journal-title":"J Pers Med"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1111\/jpc.70433","article-title":"Quality of autism information generated by artificial intelligence tools: implications for paediatric care","author":"Balikci","year":"2026","journal-title":"J Paediatr Child Health"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"114354","DOI":"10.1016\/j.dss.2024.114354","article-title":"Know where to go: make LLM a relevant, responsible, and trustworthy searchers","volume":"188","author":"Shi","year":"2024","journal-title":"Decis Support Syst"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1186\/s12909-024-05042-9","article-title":"ChatGPT sits the DFPH exam: large language model performance and potential to support public health learning","volume":"24","author":"Davies","year":"2024","journal-title":"BMC Med Educ"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"3087","DOI":"10.1016\/j.vaccine.2022.04.056","article-title":"Conversational artificial intelligence: a new approach for increasing influenza vaccination rates in children with asthma?","volume":"40","author":"Krupp","year":"2022","journal-title":"Vaccine"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e100075","DOI":"10.1136\/bmjhci-2019-100075","article-title":"Evaluating the quality of voice assistants\u2019 responses to consumer health questions about vaccines: an exploratory comparison of Alexa, Google assistant and Siri","volume":"26","author":"Alagha","year":"2019","journal-title":"BMJ Health Care Inform"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"3020","DOI":"10.21037\/tp-2025-602","article-title":"Evaluating ChatGPT\u2019s responses to vaccine-related questions: the impact of question framing on content and quality","volume":"14","author":"Kusama","year":"2025","journal-title":"Transl Pediatr"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1016\/S0140-6736(25)01037-2","article-title":"Global, regional, and national trends in routine childhood vaccination coverage from 1980 to 2023 with forecasts to 2030: a systematic analysis for the global burden of disease study 2023","volume":"406","year":"2025","journal-title":"Lancet"},{"key":"ref18","year":"2025"},{"key":"ref19","year":"2026"},{"key":"ref20","year":"2025"},{"key":"ref21","year":"2026"},{"key":"ref22","year":"2025"},{"key":"ref23","year":"2025"},{"key":"ref24","year":"2026"},{"key":"ref25","year":"2007"},{"key":"ref26","year":"2007"},{"key":"ref27","year":"2023"},{"key":"ref28","year":"2018"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"914","DOI":"10.3390\/ijerph19020914","article-title":"HPV vaccination: polish-language facebook discourse analysis","volume":"19","author":"Sobeczek","year":"2022","journal-title":"Int J Environ Res Public Health"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"1192","DOI":"10.3390\/vaccines9101192","article-title":"Attitudes of parents with regard to vaccination of children against COVID-19 in Poland. A nationwide online survey","volume":"9","author":"Babicki","year":"2021","journal-title":"Vaccine"},{"key":"ref31","year":""},{"key":"ref32","year":""},{"key":"ref33","doi-asserted-by":"publisher","first-page":"100871","DOI":"10.1016\/j.hlpt.2024.100871","article-title":"COVID-19 in polish-language social media - misinformation vs government information","volume":"13","author":"Waszak","year":"2024","journal-title":"Health Policy Technol"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"e48130","DOI":"10.2196\/48130","article-title":"Medical fake news in polish on the world wide web during the COVID-19 pandemic period: the spread dynamics analysis","volume":"26","author":"Chlabicz","year":"2024","journal-title":"Med Internet Res"},{"key":"ref35","author":"Jackowska","year":"2016"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"e69313","DOI":"10.2196\/69313","article-title":"Role of artificial intelligence in surgical training by assessing GPT-4 and GPT-4o on the Japan surgical board examination with text-only and image-accompanied questions: performance evaluation study","volume":"11","author":"Maruyama","year":"2025","journal-title":"JMIR Med Educ"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"2592430","DOI":"10.1080\/10872981.2025.2592430","article-title":"When AI models take the exam: large language models vs medical students on multiple-choice course exams","volume":"30","author":"Ros-Arlanz\u00f3n","year":"2025","journal-title":"Med Educ Online"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"1637","DOI":"10.3390\/healthcare12161637","article-title":"Assessment study of ChatGPT-3.5\u2019s performance on the final polish medical examination: accuracy in answering 980 questions","volume":"12","author":"Siebielec","year":"2024","journal-title":"Healthcare"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"e64284","DOI":"10.2196\/64284","article-title":"Comparative study of large language models in radiology board exams: performance evaluation and implications","volume":"11","author":"Wei","year":"2024","journal-title":"JMIR Med Educ"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"e0317423","DOI":"10.1371\/journal.pone.0317423","article-title":"Accuracy of latest large language models in answering multiple choice questions in dentistry: a comparative study","volume":"20","author":"Nguyen","year":"2025","journal-title":"PLoS One"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"e81399","DOI":"10.2196\/81399","article-title":"Performance evaluation of large language models in multilingual medical multiple-choice questions: a mixed method approach","volume":"12","author":"Strasser","year":"2025","journal-title":"JMIR Med Educ."},{"key":"ref42","doi-asserted-by":"publisher","first-page":"430","DOI":"10.5114\/pjr.2023.131215","article-title":"Will ChatGPT pass the polish specialty exam in radiology and diagnostic imaging? Insights into strengths and limitations","volume":"88","author":"Kufel","year":"2023","journal-title":"Pol J Radiol"},{"key":"ref43","doi-asserted-by":"publisher","first-page":"sfae193","DOI":"10.1093\/ckj\/sfae193","article-title":"The potential of ChatGPT in medicine: an example analysis of nephrology specialty exams in Poland","volume":"17","author":"Nicikowski","year":"2024","journal-title":"Clin Kidney J"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"1360597","DOI":"10.3389\/fpubh.2024.1360597","article-title":"Medical knowledge of ChatGPT in public health, infectious diseases, COVID-19 pandemic, and vaccines: multiple choice questions examination based performance","volume":"12","author":"Meo","year":"2024","journal-title":"Front Public Health"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"42096","DOI":"10.1038\/s41598-025-26036-7","article-title":"The pitfalls of multiple-choice questions in generative AI and medical education","volume":"15","author":"Singh","year":"2025","journal-title":"Sci Rep"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.3390\/vaccines11071217","article-title":"Artificial intelligence and public health: evaluating ChatGPT responses to vaccination myths and misconceptions","volume":"11","author":"Deiana","year":"2023","journal-title":"Vaccine"},{"key":"ref47","doi-asserted-by":"publisher","first-page":"63","DOI":"10.59213\/TP.2026.322","article-title":"Are AI chatbots reliable sources for parents to ask about their concerns about childhood vaccination?","volume":"7","author":"Erg\u00fcn \u00d6zdel","year":"2026","journal-title":"Trends in Pediatrics"},{"key":"ref48","doi-asserted-by":"publisher","first-page":"e79720","DOI":"10.2196\/79720","article-title":"Examining artificial intelligence chatbots\u2019 responses in providing human papillomavirus vaccine information for young adults: qualitative content analysis","volume":"12","author":"Laily","year":"2026","journal-title":"JMIR Public Health Surveill"},{"key":"ref49","doi-asserted-by":"publisher","first-page":"e60939","DOI":"10.2196\/60939","article-title":"Ensuring accuracy and equity in vaccination information from ChatGPT and CDC: mixed-methods cross-language evaluation","volume":"8","author":"Joshi","year":"2024","journal-title":"JMIR Form Res"},{"key":"ref50","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1177\/09636625241268910","article-title":"How generative artificial intelligence portrays science: interviewing ChatGPT from the perspective of different audience segments","volume":"34","author":"Volk","year":"2024","journal-title":"Public Underst Sci"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"3615","DOI":"10.1038\/s41467-025-58551-6","article-title":"An automated framework for assessing how well LLMs cite relevant medical references","volume":"16","author":"Wu","year":"2025","journal-title":"Nat Commun"},{"key":"ref52","year":""},{"key":"ref53","year":""}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1811462\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T05:55:37Z","timestamp":1781675737000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1811462\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":53,"alternative-id":["10.3389\/fpubh.2026.1811462"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1811462","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,17]]},"article-number":"1811462"}}