{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T21:44:01Z","timestamp":1793742241965,"version":"4.6.1"},"posted":{"date-parts":[[2026,6,19]]},"group-title":"Ophthalmology","reference-count":31,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T00:00:00Z","timestamp":1781827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.medrxiv.org\/about\/FAQ#license"}],"funder":[{"award":["EY036593"],"award-info":[{"award-number":["EY036593"]}],"id":[{"id":"https:\/\/ror.org\/03wkg3b53","id-type":"ROR","asserted-by":"publisher"}]},{"award":["EY033831"],"award-info":[{"award-number":["EY033831"]}],"id":[{"id":"https:\/\/ror.org\/03wkg3b53","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,6,19]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                <jats:sec>\n                  <jats:title>Purpose<\/jats:title>\n                  <jats:p>To evaluate the performance of secure cloud-based large language models (LLMs) in extracting glaucoma diagnosis, type, and severity from free-text clinical notes in the electronic health record (EHR).<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Design<\/jats:title>\n                  <jats:p>Retrospective chart review analysis.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Participants<\/jats:title>\n                  <jats:p>1,250 subjects from the Bascom Palmer Ophthalmic Repository.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Clinical notes of glaucoma-related encounters between 2014 and 2024 were extracted from the Bascom Palmer Ophthalmic Repository. Two fellowship-trained glaucoma specialists annotated clinical notes for glaucoma presence, type, and severity at the eye level. The dataset was split into development (10%), validation (10%), and test (80%) sets. Development and validation sets were used for prompt engineering and refinement, and the held-out test set was used for evaluation. Five LLMs (Claude Opus 4.6, DeepSeek-V3.2, GPT-5.2, Grok 4.1, and Qwen3.6-35B-A3B) were accessed via Azure AI Foundry within HIPAA-compliant containers. Model performance was assessed using standard metrics. Clinician-entered ICD-10 codes were also compared with adjudicated labels.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Main Outcome Measures<\/jats:title>\n                  <jats:p>Gwet AC1, accuracy, sensitivity, specificity, and F1-score.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Inter-grader agreement was high for glaucoma detection (Gwet AC1= 0.930 (95% CI: 0.917\u20130.945), type classification (Gwet AC1= 0.917 (95% CI: 0.904\u20130.930), and severity staging (Gwet AC1= 0.901 (95% CI: 0.884\u20130.916). For glaucoma diagnosis, LLMs demonstrated high overall accuracy, with Claude achieving 97.5%, DeepSeek 96.0%, GPT 96.2%, Grok 94.4%, and Qwen 95.5%. F1 scores for glaucoma detection ranged from 95.4% to 98.9% across models. For glaucoma type classification, accuracies were 97.1%, 94.2%, 94.2%, 94.0%, and 94.4% for Claude, DeepSeek, GPT, Grok, and Qwen, respectively. F1 scores for the most prevalent type (POAG) ranged from 96.3% to 98.9%. For severity staging, accuracies were 95.0%, 94.8%, 94.5%, 94.0%, and 95.2%, respectively, with F1 scores ranging from 89.7% to 96.3% across severity categories and models. ICD-10 codes demonstrated substantially lower performance for type and severity staging, with overall accuracies of 89.2% and 58.5%, respectively.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Secure cloud-based LLMs accurately extracted glaucoma diagnosis, type, and severity information from free-text ophthalmology notes, achieving performance approaching expert clinician adjudication while substantially outperforming ICD-based phenotyping approaches, particularly for disease severity classification. These findings demonstrate the potential of LLMs to transform unstructured clinical documentation into scalable, research-ready phenotypic data for large-scale glaucoma cohort development and EHR-based ophthalmic research.<\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.06.18.26355532","type":"posted-content","created":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T21:20:18Z","timestamp":1781904018000},"source":"Crossref","is-referenced-by-count":1,"title":["Extraction of Glaucoma Diagnosis, Type, and Severity from Clinical Notes using Secure Cloud-based Large Language Models"],"prefix":"10.64898","author":[{"given":"Gustavo A.","family":"Samico","sequence":"first","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"},{"name":"Department of Ophthalmology and Visual Sciences, Escola Paulista de Medicina Universidade Federal de S\u00e3o Paulo, S\u00e3o Paulo, SP, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Solages","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rafael","family":"Scherer","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rohit","family":"Muralidhar","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naomi E.","family":"Gutkind","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vitoria","family":"Palazoni","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felipe A.","family":"Medeiros","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7198-6602","authenticated-orcid":false,"given":"Swarup S.","family":"Swaminathan","sequence":"additional","affiliation":[{"name":"Bascom Palmer Eye Institute, University of Miami, Miami, FL, USA;"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"issue":"12","key":"2026062205000689000_2026.06.18.26355532v1.1","doi-asserted-by":"crossref","first-page":"e1221","DOI":"10.1016\/S2214-109X(17)30393-5","article-title":"Global causes of blindness and distance vision impairment 1990\u20132020: a systematic review and meta-analysis","volume":"5","year":"2017","journal-title":"The Lancet Global Health"},{"key":"2026062205000689000_2026.06.18.26355532v1.2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2014.05.013"},{"key":"2026062205000689000_2026.06.18.26355532v1.3","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2014.3192"},{"issue":"5","key":"2026062205000689000_2026.06.18.26355532v1.4","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1001\/jamaophthalmol.2018.7051","article-title":"Evaluation of an Algorithm for Identifying Ocular Conditions in Electronic Health Record Data","volume":"137","year":"2019","journal-title":"JAMA Ophthalmol"},{"issue":"5","key":"2026062205000689000_2026.06.18.26355532v1.5","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1016\/j.ogla.2024.04.002","article-title":"Quality of Reporting Electronic Health Record Data in Glaucoma: A Systematic Literature Review","volume":"7","year":"2024","journal-title":"Ophthalmol Glaucoma"},{"issue":"1","key":"2026062205000689000_2026.06.18.26355532v1.6","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1097\/IJG.0000000000002480","article-title":"Assessment of Missing Data on Glaucoma Severity Among Participants in the NIH All of Us Research Program of the United States","volume":"34","year":"2025","journal-title":"J Glaucoma"},{"issue":"6","key":"2026062205000689000_2026.06.18.26355532v1.7","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1016\/j.ogla.2025.06.012","article-title":"Accuracy of ICD-10 Glaucoma Codes in a Large Academic Practice","volume":"8","year":"2025","journal-title":"Ophthalmol Glaucoma"},{"issue":"1","key":"2026062205000689000_2026.06.18.26355532v1.8","doi-asserted-by":"crossref","first-page":"100091","DOI":"10.1016\/j.xops.2021.100091","article-title":"Discrepancies in Ophthalmic Medication Documentation for Glaucoma Patients","volume":"2","year":"2022","journal-title":"Ophthalmol Sci"},{"issue":"8","key":"2026062205000689000_2026.06.18.26355532v1.9","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1001\/jamaophthalmol.2019.1444","article-title":"Medication Accuracy in Electronic Health Records for Microbial Keratitis","volume":"137","year":"2019","journal-title":"JAMA Ophthalmology"},{"key":"2026062205000689000_2026.06.18.26355532v1.10","first-page":"773","article-title":"Extraction of Active Medications and Adherence Using Natural Language Processing for Glaucoma Patients","volume":"2021","year":"2021","journal-title":"AMIA Annu Symp Proc"},{"key":"2026062205000689000_2026.06.18.26355532v1.11","doi-asserted-by":"crossref","unstructured":"Shah R , Moradi M , Eslami S , et al. OphthaBERT: Automated Glaucoma Diagnosis from Clinical Notes. medRxiv 2025.","DOI":"10.1101\/2025.06.08.25329151"},{"key":"2026062205000689000_2026.06.18.26355532v1.12","doi-asserted-by":"crossref","unstructured":"Stein JD , An HS , Andrews CA , et al. Enhanced Phenotype Identification of Common Ocular Diseases in Real-World Datasets. Ophthalmology Science 2025;5(4).","DOI":"10.1016\/j.xops.2025.100717"},{"key":"2026062205000689000_2026.06.18.26355532v1.13","unstructured":"Vaswani A , Shazeer N , Parmar N , et al. Attention is all you need. Advances in neural information processing systems 2017;30."},{"issue":"14","key":"2026062205000689000_2026.06.18.26355532v1.14","doi-asserted-by":"crossref","first-page":"2711","DOI":"10.1038\/s41433-024-03103-3","article-title":"Racial-ethnic disparities in concurrent rates of peripapillary & macular OCT parameters among a large glaucomatous clinical population","volume":"38","year":"2024","journal-title":"Eye (Lond"},{"issue":"3","key":"2026062205000689000_2026.06.18.26355532v1.15","doi-asserted-by":"crossref","first-page":"100454","DOI":"10.1016\/j.xops.2023.100454","article-title":"Performance of Linear Mixed Models in Estimating Structural Rates of Glaucoma Progression Using Varied Random Effect Distributions","volume":"4","year":"2024","journal-title":"Ophthalmol Sci"},{"issue":"5","key":"2026062205000689000_2026.06.18.26355532v1.16","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1167\/tvst.15.5.13","article-title":"Identification of Risk Factors for Glaucoma Progression in Free-Text Clinical Notes Using a Local Large Language Model","volume":"15","year":"2026","journal-title":"Transl Vis Sci Technol"},{"key":"2026062205000689000_2026.06.18.26355532v1.17","unstructured":"Hodapp E , Parrish RK , Anderson DR . Clinical decisions in glaucoma. St. Louis: Mosby, 1993."},{"key":"2026062205000689000_2026.06.18.26355532v1.18","unstructured":"Agrawal M , Hegselmann S , Lang H , et al. Large language models are few-shot clinical information extractors. arXiv: 2205.126892022."},{"key":"2026062205000689000_2026.06.18.26355532v1.19","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocad259"},{"key":"2026062205000689000_2026.06.18.26355532v1.20","doi-asserted-by":"crossref","unstructured":"Shah K , Xu AY , Sharma Y , et al. Large Language Model Prompting Techniques for Advancement in Clinical Medicine. J Clin Med 2024;13(17).","DOI":"10.3390\/jcm13175101"},{"issue":"2","key":"2026062205000689000_2026.06.18.26355532v1.21","doi-asserted-by":"crossref","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models","volume":"2","year":"2023","journal-title":"PLOS Digit Health"},{"key":"2026062205000689000_2026.06.18.26355532v1.22","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01079-8"},{"key":"2026062205000689000_2026.06.18.26355532v1.23","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.ajo.2024.01.030","article-title":"Using Natural Language Processing to Identify Different Lens Pathology in Electronic Health Records","volume":"262","year":"2024","journal-title":"Am J Ophthalmol"},{"issue":"3","key":"2026062205000689000_2026.06.18.26355532v1.24","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1167\/tvst.14.3.27","article-title":"Multimodal Artificial Intelligence Models Predicting Glaucoma Progression Using Electronic Health Records and Retinal Nerve Fiber Layer Scans","volume":"14","year":"2025","journal-title":"Transl Vis Sci Technol"},{"key":"2026062205000689000_2026.06.18.26355532v1.25","doi-asserted-by":"crossref","unstructured":"Mungle T , Naderalvojoud B , Andrews CA , et al. Ensemble learning to enhance accurate identification of patients with glaucoma using electronic health records. JAMIA Open 2025;8(4):ooaf080.","DOI":"10.1093\/jamiaopen\/ooaf080"},{"key":"2026062205000689000_2026.06.18.26355532v1.26","doi-asserted-by":"crossref","unstructured":"Liebman DL , Johnson GE , Marte A , et al. Accuracy of ICD-10 Glaucoma Codes in a Large Academic Practice. Ophthalmol Glaucoma 2025.","DOI":"10.1016\/j.ogla.2025.06.012"},{"key":"2026062205000689000_2026.06.18.26355532v1.27","doi-asserted-by":"publisher","DOI":"10.2147\/CLEP.S443872"},{"issue":"2","key":"2026062205000689000_2026.06.18.26355532v1.28","doi-asserted-by":"crossref","first-page":"100127","DOI":"10.1016\/j.xops.2022.100127","article-title":"Deep Learning Approaches for Predicting Glaucoma Progression Using Electronic Health Records and Natural Language Processing","volume":"2","year":"2022","journal-title":"Ophthalmol Sci"},{"issue":"7","key":"2026062205000689000_2026.06.18.26355532v1.29","doi-asserted-by":"crossref","first-page":"807","DOI":"10.3390\/diagnostics15070807","article-title":"Deep Learning-Based Glaucoma Detection Using Clinical Notes: A Comparative Study of Long Short-Term Memory and Convolutional Neural Network Models","volume":"15","year":"2025","journal-title":"Diagnostics"},{"issue":"2","key":"2026062205000689000_2026.06.18.26355532v1.30","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1093\/jamia\/ocad213","article-title":"Prediction of multiclass surgical outcomes in glaucoma using multimodal deep learning based on free-text operative notes and structured EHR data","volume":"31","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2026062205000689000_2026.06.18.26355532v1.31","first-page":"1157016","article-title":"Predicting near-term glaucoma progression: An artificial intelligence approach using clinical free-text notes and data from electronic health records","volume":"10","year":"2023","journal-title":"Front Med (Lausanne"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.06.18.26355532","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T12:00:28Z","timestamp":1782129628000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.06.18.26355532"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,19]]},"references-count":31,"URL":"https:\/\/doi.org\/10.64898\/2026.06.18.26355532","relation":{},"subject":[],"published":{"date-parts":[[2026,6,19]]},"subtype":"preprint"}}