{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T17:20:06Z","timestamp":1788888006468,"version":"build-2803163510"},"reference-count":62,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T00:00:00Z","timestamp":1720656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Medical researchers are increasingly utilizing advanced LLMs like ChatGPT-4 and Gemini to enhance diagnostic processes in the medical field. This research focuses on their ability to comprehend and apply complex medical classification systems for breast conditions, which can significantly aid plastic surgeons in making informed decisions for diagnosis and treatment, ultimately leading to improved patient outcomes. Fifty clinical scenarios were created to evaluate the classification accuracy of each LLM across five established breast-related classification systems. Scores from 0 to 2 were assigned to LLM responses to denote incorrect, partially correct, or completely correct classifications. Descriptive statistics were employed to compare the performances of ChatGPT-4 and Gemini. Gemini exhibited superior overall performance, achieving 98% accuracy compared to ChatGPT-4\u2019s 71%. While both models performed well in the Baker classification for capsular contracture and UTSW classification for gynecomastia, Gemini consistently outperformed ChatGPT-4 in other systems, such as the Fischer Grade Classification for gender-affirming mastectomy, Kajava Classification for ectopic breast tissue, and Regnault Classification for breast ptosis. With further development, integrating LLMs into plastic surgery practice will likely enhance diagnostic support and decision making.<\/jats:p>","DOI":"10.3390\/diagnostics14141491","type":"journal-article","created":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T11:30:55Z","timestamp":1720697455000},"page":"1491","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Evaluating Large Language Model (LLM) Performance on Established Breast Classification Systems"],"prefix":"10.3390","volume":"14","author":[{"given":"Syed Ali","family":"Haider","sequence":"first","affiliation":[{"name":"Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sophia M.","family":"Pressman","sequence":"additional","affiliation":[{"name":"Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sahar","family":"Borna","sequence":"additional","affiliation":[{"name":"Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0603-3192","authenticated-orcid":false,"given":"Cesar A.","family":"Gomez-Cabello","sequence":"additional","affiliation":[{"name":"Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7562-8758","authenticated-orcid":false,"given":"Ajai","family":"Sehgal","sequence":"additional","affiliation":[{"name":"Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bradley C.","family":"Leibovich","sequence":"additional","affiliation":[{"name":"Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA"},{"name":"Department of Urology, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2004-7538","authenticated-orcid":false,"given":"Antonio Jorge","family":"Forte","sequence":"additional","affiliation":[{"name":"Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA"},{"name":"Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1055\/s-0043-1770963","article-title":"A New Clinical Classification for Gynecomastia Management and Predictive Outcome","volume":"56","author":"Punia","year":"2023","journal-title":"Indian J. Plast. Surg."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/j.bjps.2023.04.003","article-title":"Classification and assessment techniques of breast ptosis: A systematic review","volume":"83","author":"See","year":"2023","journal-title":"J. Plast. Reconstr. Aesthet. Surg."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2177","DOI":"10.1007\/s00266-023-03451-7","article-title":"Kajava Classification: The Person and the Research","volume":"47","author":"Surcel","year":"2023","journal-title":"Aesthetic Plast. Surg."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1016\/j.pop.2006.02.002","article-title":"Breast disorders in children and adolescents","volume":"33","author":"Greydanus","year":"2006","journal-title":"Prim. Care"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"104431","DOI":"10.1016\/j.ijmedinf.2021.104431","article-title":"Use of standardized terminologies in clinical practice: A scoping review","volume":"149","author":"Fennelly","year":"2021","journal-title":"Int. J. Med. Inf."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Secinaro, S., Calandra, D.M., Secinaro, A., Muthurangu, V., and Biancone, P.P. (2021). The role of artificial intelligence in healthcare: A structured literature review. BMC Med. Inform. Decis. Mak., 21.","DOI":"10.1186\/s12911-021-01488-9"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Al-Antari, M.A. (2023). Artificial Intelligence for Medical Diagnostics-Existing and Future AI Technology!. Diagnostics, 13.","DOI":"10.3390\/diagnostics13040688"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1177\/08465371231193716","article-title":"Comparative Performance of ChatGPT and Bard in a Text-Based Radiology Knowledge Assessment","volume":"75","author":"Patil","year":"2024","journal-title":"Can. Assoc. Radiol. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jor.2023.11.063","article-title":"The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard","volume":"50","author":"Agharia","year":"2024","journal-title":"J. Orthop."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"210","DOI":"10.7326\/M23-2772","article-title":"Large Language Models in Medicine: The Potentials and Pitfalls: A Narrative Review","volume":"177","author":"Omiye","year":"2024","journal-title":"Ann. Intern. Med."},{"key":"ref_11","unstructured":"Wang, S., Zhao, Z., Ouyang, X., Wang, Q., and Shen, D. (2023). ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Akilesh, S., Abinaya, R., Dhanushkodi, S., and Sekar, R. (2023, January 1\u20132). A Novel AI-based chatbot Application for Personalized Medical Diagnosis and review using Large Language Models. Proceedings of the 2023 International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE), Chennai, India.","DOI":"10.1109\/RMKMATE59243.2023.10368616"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yuan, M., Bao, P., Yuan, J., Shen, Y., Chen, Z., Xie, Y., Zhao, J., Chen, Y., Zhang, L., and Shen, L. (2023). Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review. arXiv.","DOI":"10.1016\/j.medp.2024.100030"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1306","DOI":"10.1177\/21925682221090891","article-title":"Telemedicine: Could it represent a new problem for spine surgeons to solve?","volume":"12","author":"Montemurro","year":"2022","journal-title":"Glob. Spine J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"19","DOI":"10.21037\/mhealth-23-28","article-title":"A narrative review of telemedicine and its adoption across specialties","volume":"10","author":"Garcia","year":"2024","journal-title":"Mhealth"},{"key":"ref_16","unstructured":"Wu, C.-K., Chen, W.-L., and Chen, H.-H. (2023). Large Language Models Perform Diagnostic Reasoning. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2022","journal-title":"Nature"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1001\/jama.2023.1044","article-title":"Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence Model","volume":"329","author":"Sarraju","year":"2023","journal-title":"JAMA"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e230424","DOI":"10.1148\/radiol.230424","article-title":"Appropriateness of Breast Cancer Prevention and Screening Recommendations Provided by ChatGPT","volume":"307","author":"Haver","year":"2023","journal-title":"Radiology"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e230922","DOI":"10.1148\/radiol.230922","article-title":"How AI Responds to Common Lung Cancer Questions: ChatGPT vs Google Bard","volume":"307","author":"Rahsepar","year":"2023","journal-title":"Radiology"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kung, T.H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepa\u00f1o, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., and Maningo, J. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit. Health, 2.","DOI":"10.1371\/journal.pdig.0000198"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e230582","DOI":"10.1148\/radiol.230582","article-title":"Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations","volume":"307","author":"Bhayana","year":"2023","journal-title":"Radiology"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e230987","DOI":"10.1148\/radiol.230987","article-title":"GPT-4 in Radiology: Improvements in Advanced Reasoning","volume":"307","author":"Bhayana","year":"2023","journal-title":"Radiology"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"963","DOI":"10.1097\/PRS.0000000000007312","article-title":"Discussion: The Baker Classification for Capsular Contracture in Breast Implant Surgery Is Unreliable as a Diagnostic Tool","volume":"146","year":"2020","journal-title":"Plast. Reconstr. Surg."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1097\/SAP.0000000000003575","article-title":"Addressing the Gray Zone in Affirmative Mastectomy: An Analysis of Fischer 2 Patients","volume":"91","author":"Krakauer","year":"2023","journal-title":"Ann. Plast. Surg."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/S0094-1298(20)30220-0","article-title":"Breast ptosis. Definition and treatment","volume":"3","author":"Regnault","year":"1976","journal-title":"Clin. Plast. Surg."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1097\/01.PRS.0000042146.40379.25","article-title":"Classification and Management of Gynecomastia: Defining the Role of Ultrasound-Assisted Liposuction","volume":"111","author":"Rohrich","year":"2003","journal-title":"Plast. Reconstr. Surg."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e5034","DOI":"10.1097\/GOX.0000000000005034","article-title":"Advances on Capsular Contracture-Prevention and Management Strategies: A Narrative Review of the Literature","volume":"11","author":"Susini","year":"2023","journal-title":"Plast. Reconstr. Surg. Glob. Open"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1097\/PRS.0000000000010573","article-title":"Toward a Consensus Aproach for Assessing Capsular Contracture Severity and Progression: A Systematic Review","volume":"153","author":"Mohan","year":"2024","journal-title":"Plast. Reconstr. Surg."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijso.2016.04.001","article-title":"A literature review and summary of capsular contracture: An ongoing challenge to breast surgeons and their patients","volume":"3","author":"Malahias","year":"2016","journal-title":"Int. J. Surg. Open"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"bcr2014204990","DOI":"10.1136\/bcr-2014-204990","article-title":"Accessory breast tissue mimicking pedunculated lipoma","volume":"2014","author":"Husain","year":"2014","journal-title":"BMJ Case Rep."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"149","DOI":"10.5455\/medarh.2022.76.149-151","article-title":"Primary Ectopic Breast Carcinoma in Man Mimicking Keloid","volume":"76","author":"Alqahtani","year":"2022","journal-title":"Med. Arch."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9137","DOI":"10.1038\/s41598-018-27505-y","article-title":"Intra-individual Gene Expression Variability of Histologically Normal Breast Tissue","volume":"8","author":"Sun","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"67","DOI":"10.7863\/jum.2005.24.1.67","article-title":"Hormonal Variations in the Vascularity of Breast Tissue","volume":"24","author":"Weinstein","year":"2005","journal-title":"J. Ultrasound Med."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Atashgaran, V., Wrin, J., Barry, S.C., Dasari, P., and Ingman, W.V. (2016). Dissecting the Biology of Menstrual Cycle-Associated Breast Cancer Risk. Front. Oncol., 6.","DOI":"10.3389\/fonc.2016.00267"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1016\/S0039-6109(16)43479-1","article-title":"Imaging of the breast. A perspective","volume":"64","author":"Strax","year":"1984","journal-title":"Surg. Clin. N. Am."},{"key":"ref_37","unstructured":"Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., and Neal, D. (2023). Towards Expert-Level Medical Question Answering with Large Language Models. arXiv."},{"key":"ref_38","unstructured":"Saab, K., Tu, T., Weng, W.-H., Tanno, R., Stutz, D., Wulczyn, E., Zhang, F., Strother, T., Park, C., and Vedadi, E. (2024). Capabilities of Gemini Models in Medicine. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Markus, A.F., Kors, J.A., and Rijnbeek, P.R. (2021). The role of explainability in creating trustworthy artificial intelligence for health care: A comprehensive survey of the terminology, design choices, and evaluation strategies. J. Biomed. Inf., 113.","DOI":"10.1016\/j.jbi.2020.103655"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","year":"2023","journal-title":"Commun. Med."},{"key":"ref_41","unstructured":"Tonmoy, S.M.T.I., Zaman, S.M.M., Jain, V., Rani, A., Rawte, V., Chadha, A., and Das, A. (2024). A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1038\/s41746-022-00689-4","article-title":"Integrated multimodal artificial intelligence framework for healthcare applications","volume":"5","author":"Soenksen","year":"2022","journal-title":"NPJ Digit. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e52865","DOI":"10.2196\/52865","article-title":"The Impact of Multimodal Large Language Models on Health Care\u2019s Future","volume":"25","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_44","unstructured":"Zertuche, O. (2023). Google Introduces the Multimodal Gemini Ultra, Pro, & Nano Models. Cody\u2014The AI Trained on Your Business, meetcody.ai."},{"key":"ref_45","unstructured":"Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., and Hauth, A. (2023). Gemini: A Family of Highly Capable Multimodal Models. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"e5057","DOI":"10.1097\/GOX.0000000000005057","article-title":"Artificial Intelligence for Plastic Surgeons","volume":"11","author":"Sue","year":"2023","journal-title":"Plast. Reconstr. Surg. Glob. Open"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lim, B., Seth, I., Kah, S., Sofiadellis, F., Ross, R.J., Rozen, W.M., and Cuomo, R. (2023). Using Generative Artificial Intelligence Tools in Cosmetic Surgery: A Study on Rhinoplasty, Facelifts, and Blepharoplasty Procedures. J. Clin. Med., 12.","DOI":"10.3390\/jcm12206524"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1038\/s41746-023-00846-3","article-title":"The digital transformation of surgery","volume":"6","author":"Marwaha","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"43","DOI":"10.3928\/01477447-20161013-05","article-title":"Patient Satisfaction Is Associated With Time With Provider But Not Clinic Wait Time Among Orthopedic Patients","volume":"40","author":"Patterson","year":"2017","journal-title":"Orthopedics"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1097\/01.GOX.0000799800.31313.2b","article-title":"Essential Elements of Surgeon Communication Impacting Patient Satisfaction: A Systematic Review","volume":"9","author":"Mauch","year":"2021","journal-title":"Plast. Reconstr. Surg. Glob. Open"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"e50945","DOI":"10.2196\/50945","article-title":"The Role of Large Language Models in Medical Education: Applications and Implications","volume":"9","author":"Safranek","year":"2023","journal-title":"JMIR Med. Educ."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1001\/jama.2023.9458","article-title":"AI Chatbots, Health Privacy, and Challenges to HIPAA Compliance","volume":"330","author":"Marks","year":"2023","journal-title":"Jama"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Plant, R., Giuffrida, V., and Gkatzia, D. (2022). You Are What You Write: Preserving Privacy in the Era of Large Language Models. arXiv.","DOI":"10.2139\/ssrn.4417900"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Kiseleva, A., Kotzinos, D., and De Hert, P. (2022). Transparency of AI in Healthcare as a Multilayered System of Accountabilities: Between Legal Requirements and Technical Limitations. Front. Artif. Intell., 5.","DOI":"10.3389\/frai.2022.879603"},{"key":"ref_55","first-page":"e30168","article-title":"Health Records Database and Inherent Security Concerns: A Review of the Literature","volume":"14","author":"Basil","year":"2022","journal-title":"Cureus"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Murdoch, B. (2021). Privacy and artificial intelligence: Challenges for protecting health information in a new era. BMC Med. Ethics, 22.","DOI":"10.1186\/s12910-021-00687-3"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Almalawi, A., Khan, A.I., Alsolami, F., Abushark, Y.B., and Alfakeeh, A.S. (2023). Managing Security of Healthcare Data for a Modern Healthcare System. Sensors, 23.","DOI":"10.3390\/s23073612"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.1007\/s00146-021-01262-5","article-title":"AI, big data, and the future of consent","volume":"37","author":"Andreotta","year":"2022","journal-title":"AI Soc."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1093\/jamia\/ocaa088","article-title":"MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care","volume":"27","author":"Bozkurt","year":"2020","journal-title":"J. Am. Med. Inf. Assoc."},{"key":"ref_60","unstructured":"Microsoft (2024, April 01). Microsoft Copilot in Bing. Available online: https:\/\/www.bing.com\/chat."},{"key":"ref_61","unstructured":"Anthropic (2024, April 01). Claude. Available online: https:\/\/claude.ai\/login?returnTo=%2F."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"e23","DOI":"10.2106\/JBJS.RVW.23.00232","article-title":"Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review","volume":"12","author":"Huffman","year":"2024","journal-title":"JBJS Rev."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/14\/14\/1491\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:15:07Z","timestamp":1760109307000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/14\/14\/1491"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,11]]},"references-count":62,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["diagnostics14141491"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics14141491","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,11]]}}}