{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:59:10Z","timestamp":1782842350858,"version":"3.54.5"},"reference-count":38,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"\n Background<\/jats:title>\n Refractive surgery research aims to optimally precategorize patients by their suitability for various types of surgery. Recent advances have led to the development of artificial intelligence\u2013powered algorithms, including machine learning approaches, to assess risks and enhance workflow. Large language models (LLMs) like ChatGPT-4 (OpenAI LP) have emerged as potential general artificial intelligence tools that can assist across various disciplines, possibly including refractive surgery decision-making. However, their actual capabilities in precategorizing refractive surgery patients based on real-world parameters remain unexplored.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This exploratory study aimed to validate ChatGPT-4\u2019s capabilities in precategorizing refractive surgery patients based on commonly used clinical parameters. The goal was to assess whether ChatGPT-4\u2019s performance when categorizing batch inputs is comparable to those made by a refractive surgeon. A simple binary set of categories (patient suitable for laser refractive surgery or not) as well as a more detailed set were compared.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Data from 100 consecutive patients from a refractive clinic were anonymized and analyzed. Parameters included age, sex, manifest refraction, visual acuity, and various corneal measurements and indices from Scheimpflug imaging. This study compared ChatGPT-4\u2019s performance with a clinician\u2019s categorizations using Cohen \u03ba coefficient, a chi-square test, a confusion matrix, accuracy, precision, recall, F1-score, and receiver operating characteristic area under the curve.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n A statistically significant noncoincidental accordance was found between ChatGPT-4 and the clinician\u2019s categorizations with a Cohen \u03ba coefficient of 0.399 for 6 categories (95% CI 0.256-0.537) and 0.610 for binary categorization (95% CI 0.372-0.792). The model showed temporal instability and response variability, however. The chi-square test on 6 categories indicated an association between the 2 raters\u2019 distributions (\u03c7\u00b25=94.7, P<.001). Here, the accuracy was 0.68, precision 0.75, recall 0.68, and F1-score 0.70. For 2 categories, the accuracy was 0.88, precision 0.88, recall 0.88, F1-score 0.88, and area under the curve 0.79.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n This study revealed that ChatGPT-4 exhibits potential as a precategorization tool in refractive surgery, showing promising agreement with clinician categorizations. However, its main limitations include, among others, dependency on solely one human rater, small sample size, the instability and variability of ChatGPT\u2019s (OpenAI LP) output between iterations and nontransparency of the underlying models. The results encourage further exploration into the application of LLMs like ChatGPT-4 in health care, particularly in decision-making processes that require understanding vast clinical data. Future research should focus on defining the model\u2019s accuracy with prompt and vignette standardization, detecting confounding factors, and comparing to other versions of ChatGPT-4 and other LLMs to pave the way for larger-scale validation and real-world implementation.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/51798","type":"journal-article","created":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T04:03:36Z","timestamp":1701662616000},"page":"e51798","source":"Crossref","is-referenced-by-count":15,"title":["Exploring the Potential of ChatGPT-4 in Predicting Refractive Surgery Categorizations: Comparative Study"],"prefix":"10.2196","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2122-5175","authenticated-orcid":false,"given":"Aleksandar","family":"\u0106irkovi\u0107","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3160-0231","authenticated-orcid":false,"given":"Toam","family":"Katz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2023,12,28]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1016\/j.joco.2016.01.009"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajo.2022.12.016"},{"key":"ref3","author":"Radford, A","journal-title":"OpenAI"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.5860\/choice.189890"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.12712"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1038\/s41433-023-02563-3"},{"issue":"8","key":"ref7","first-page":"9","volume":"1","author":"Radford, A","year":"2019","journal-title":"OpenAI blog"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.2196\/46885"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1038\/s41433-023-02619-4"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare11060887"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1101\/2023.02.02.23285399"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230424"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.42972"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajp.2023.103770"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.06488"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.08072"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1101\/2023.04.16.23288645"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.2196\/49995"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.2196\/48659"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1101\/2023.02.25.23286451"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.2196\/48808"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03272-4"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.13793"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.2967\/jnumed.123.265687"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2023.1161098"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.4103\/tjem.tjem_79_23"},{"key":"ref27","unstructured":"ChatGPT2023-11-27https:\/\/chat.openai.com"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.g7594"},{"issue":"3","key":"ref29","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"McHugh, ML","year":"2012","journal-title":"Biochem Med (Zagreb)"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1038\/s41523-023-00557-8"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.08152"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4502081"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.16416"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.2147\/OPTH.S66598"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1186\/s12886-017-0584-2"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09009"},{"key":"ref37","unstructured":"How to build your own custom ChatGPT with OpenAI's GPT builderZapier2023-11-28https:\/\/zapier.com\/blog\/custom-chatgpt\/"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02656-2"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2023\/1\/e51798","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:35:59Z","timestamp":1782840959000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2023\/1\/e51798"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,28]]},"references-count":38,"URL":"https:\/\/doi.org\/10.2196\/51798","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.51798","asserted-by":"object"}]},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,28]]}}}