{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,11]],"date-time":"2026-10-11T13:55:07Z","timestamp":1791726907004,"version":"4.3.4"},"reference-count":24,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,3,22]],"date-time":"2024-03-22T00:00:00Z","timestamp":1711065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Since its introduction in November 2022, the artificial intelligence large language model ChatGPT has taken the world by storm. Among other applications it can be used by patients as a source of information on diseases and their treatments. However, little is known about the quality of the sarcoma-related information ChatGPT provides. We therefore aimed at analyzing how sarcoma experts evaluate the quality of ChatGPT\u2019s responses on sarcoma-related inquiries and assess the bot\u2019s answers in specific evaluation metrics.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The ChatGPT responses to a sample of 25 sarcoma-related questions (5 definitions, 9 general questions, and 11 treatment-related inquiries) were evaluated by 3 independent sarcoma experts. Each response was compared with authoritative resources and international guidelines and graded on 5 different metrics using a 5-point Likert scale: completeness, misleadingness, accuracy, being up-to-date, and appropriateness. This resulted in maximum 25 and minimum 5 points per answer, with higher scores indicating a higher response quality. Scores \u226521 points were rated as very good, between 16 and 20 as good, while scores \u226415 points were classified as poor (11\u201315) and very poor (\u226410).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The median score that ChatGPT\u2019s answers achieved was 18.3 points (IQR, i.e., Inter-Quartile Range, 12.3\u201320.3 points). Six answers were classified as very good, 9 as good, while 5 answers each were rated as poor and very poor. The best scores were documented in the evaluation of how appropriate the response was for patients (median, 3.7 points; IQR, 2.5\u20134.2 points), which were significantly higher compared to the accuracy scores (median, 3.3 points; IQR, 2.0\u20134.2 points;\n                      <jats:italic>p<\/jats:italic>\n                      =\u20090.035). ChatGPT fared considerably worse with treatment-related questions, with only 45% of its responses classified as good or very good, compared to general questions (78% of responses good\/very good) and definitions (60% of responses good\/very good).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>The answers ChatGPT provided on a rare disease, such as sarcoma, were found to be of very inconsistent quality, with some answers being classified as very good and others as very poor. Sarcoma physicians should be aware of the risks of misinformation that ChatGPT poses and advise their patients accordingly.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2024.1303319","type":"journal-article","created":{"date-parts":[[2024,3,22]],"date-time":"2024-03-22T00:53:03Z","timestamp":1711068783000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["Artificial intelligence large language model ChatGPT: is it a trustworthy and reliable source of information for sarcoma patients?"],"prefix":"10.3389","volume":"12","author":[{"given":"Marisa","family":"Valentini","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joanna","family":"Szkandera","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria Anna","family":"Smolle","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susanne","family":"Scheipl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Leithner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dimosthenis","family":"Andreou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,3,22]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1016\/j.ejca.2012.09.011","article-title":"Descriptive epidemiology of sarcomas in Europe: report from the RARECARE project","volume":"49","author":"Stiller","year":"2013","journal-title":"Eur J Cancer"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"2462","DOI":"10.18632\/oncotarget.26809","article-title":"Sarcomas in the United States: recent trends and a call for improved staging","volume":"10","author":"Gage","year":"2019","journal-title":"Oncotarget"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1348","DOI":"10.1016\/j.annonc.2021.07.006","article-title":"ESMO guidelines committee, EURACAN and GENTURIS. Soft tissue and visceral sarcomas: ESMO-EURACAN-GENTURIS clinical practice guidelines for diagnosis, treatment and follow-up\u2606","volume":"32","author":"Gronchi","year":"2021","journal-title":"Ann Oncol"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"893","DOI":"10.1080\/14737140.2020.1814150","article-title":"A multidisciplinary approach to soft-tissue sarcoma of the extremities","volume":"20","author":"Nakayama","year":"2020","journal-title":"Expert Rev Anticancer Ther"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1067\/j.cpsurg.2018.10.006","article-title":"Multidisciplinary sarcoma care","volume":"55","author":"Pollock","year":"2018","journal-title":"Curr Probl Surg"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"1520","DOI":"10.1016\/j.annonc.2021.08.1995","article-title":"Bone sarcomas: ESMO-EURACAN-GENTURIS-ERN Paed can clinical practice guideline for diagnosis, treatment and follow-up","volume":"32","author":"Strauss","year":"2021","journal-title":"Ann Oncol"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"979","DOI":"10.3390\/life13040979","article-title":"Centralized and interdisciplinary therapy Management in the Treatment of sarcomas","volume":"13","author":"Str\u00f6nisch","year":"2023","journal-title":"Life (Basel)"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"868","DOI":"10.1007\/s10439-023-03172-7","article-title":"Role of chat GPT in public health","volume":"51","author":"Biswas","year":"2023","journal-title":"Ann Biomed Eng"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1101\/2023.02.19.23286155","article-title":"The utility of chatGPT as an example of large language models in healthcare education, research and practice: systematic review on the future perspectives and potential limitations","volume":"2023","author":"Sallam","year":"2023","journal-title":"med Rxiv"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"e45312","DOI":"10.2196\/45312","article-title":"How does chat GPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment","volume":"9","author":"Gilson","year":"2023","journal-title":"JMIR Med Educ"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of chat GPT on USMLE: potential for AI-assisted medical education using large language models. PLOS digital","volume":"2","author":"Kung","year":"2023","journal-title":"Health"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1016\/j.bbi.2023.02.022","article-title":"A chat (GPT) about the future of scientific publishing","volume":"110","author":"Hill-Yardin","year":"2023","journal-title":"Brain Behav Immun"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"1199350","DOI":"10.3389\/frai.2023.1199350","article-title":"Human-like problem-solving abilities in large language models using chat GPT","volume":"6","author":"Orr\u00f9","year":"2023","journal-title":"Front Artif Intell"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence Chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-2566942\/v1","article-title":"Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the chat-GPT model","author":"Johnson","year":"2023","journal-title":"Res Sq [Preprint]"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"186","DOI":"10.5582\/bst.2023.01138","article-title":"New possibilities for medical support systems utilizing artificial intelligence (AI) and data platforms","volume":"17","author":"Karako","year":"2023","journal-title":"Biosci Trends"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"1166014","DOI":"10.3389\/frai.2023.1166014","article-title":"Impact of chat GPT on medical chatbots as a disruptive technology","volume":"6","author":"Chow","year":"2023","journal-title":"Front Artif Intell"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"2281","DOI":"10.1093\/humrep\/dead207","article-title":"AI language models in human reproduction research: exploring chat GPT's potential to assist academic writing","volume":"38","author":"Semrl","year":"2023","journal-title":"Hum Reprod"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"1343","DOI":"10.1111\/1756-185X.14749","article-title":"\"Dr chat GPT\": is it a reliable and useful source for common rheumatic diseases?","volume":"26","author":"Uz","year":"2023","journal-title":"Int J Rheum Dis"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"4271","DOI":"10.1007\/s00405-023-08051-4","article-title":"Chat GPT's quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions","volume":"280","author":"Hoch","year":"2023","journal-title":"Eur Arch Otorrinolaringol"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"373","DOI":"10.3238\/arztebl.m2023.0113","article-title":"Chat GPT passes German state examination in medicine with picture questions omitted","volume":"120","author":"Jung","year":"2023","journal-title":"Dtsch Arztebl Int"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1136\/jamia.2010.004507","article-title":"Wikipedia and osteosarcoma: a trustworthy patients' information?","volume":"17","author":"Leithner","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1007\/s10354-014-0304-y","article-title":"Osteosarcoma: reliability and quality of the information in the internet","volume":"164","author":"Schippinger","year":"2014","journal-title":"Wien Med Wochenschr"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"e19.00181","DOI":"10.5435\/JAAOSGlobal-D-19-00181","article-title":"The quality of online Orthopaedic oncology information","volume":"4","author":"Zade","year":"2020","journal-title":"J Am Acad Orthop Surg Glob Res Rev"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2024.1303319\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T07:04:48Z","timestamp":1713855888000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2024.1303319\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,22]]},"references-count":24,"alternative-id":["10.3389\/fpubh.2024.1303319"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2024.1303319","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,22]]},"article-number":"1303319"}}