{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:45:52Z","timestamp":1760060752202,"version":"build-2065373602"},"reference-count":53,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T00:00:00Z","timestamp":1758499200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: In clinical informatics, the term \u2018information overload\u2019 is increasingly used to describe the operational impediments of excessive documentation. While electronic health records (EHRs) are growing in abundance, many medical records (MRs) remain in legacy formats that impede efficient, systematic processing, contributing to the extenuating challenges of care fragmentation. Thus, there is a growing interest in using generative AI (genAI) for automated MR summarization and characterization. Methods: MRs for a set of 78 individuals were digitized. Some were known systemic lupus erythematosus (SLE) cases, while others were under evaluation for possible SLE classification. A two-pass genAI assessment strategy was implemented using the Claude 3.5 large language model (LLM) to mine MRs for information relevant to classifying SLE vs. undifferentiated connective tissue disorder (UCTD) vs. neither via the 22-criteria EULAR 2019 model. Results: Compared to clinical determination, the antinuclear antibody (ANA) criterion (whose results are crucial for classifying SLE-negative cases) exhibited favorable sensitivity 0.78 \u00b1 0.09 (95% confidence interval) and a positive predictive value 0.85 \u00b1 0.08 but a marginal performance for specificity 0.60 \u00b1 0.11 and uncertain predictivity for the negative predictive value 0.48 \u00b1 0.11. Averaged over the remaining 21 criteria, these four performance metrics were 0.69 \u00b1 0.11, 0.87 \u00b1 0.04, 0.54 \u00b1 0.10, and 0.93 \u00b1 0.03. Conclusions: ANA performance statistics imply that genAI yields confident assessments of SLE negativity (per high sensitivity) but weaker positivity. The remaining genAI criterial determinations support (per specificity) confident assertions of SLE-positivity but tend to misclassify a significant fraction of clinical positives as UCTD.<\/jats:p>","DOI":"10.3390\/diagnostics15182409","type":"journal-article","created":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T11:18:16Z","timestamp":1758539896000},"page":"2409","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Criteria and Protocol: Assessing Generative AI Efficacy in Perceiving EULAR 2019 Lupus Classification"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5821-4253","authenticated-orcid":false,"given":"Gerald H.","family":"Lushington","sequence":"first","affiliation":[{"name":"Progentec Diagnostics Inc., Oklahoma City, OK 73104, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sandeep","family":"Nair","sequence":"additional","affiliation":[{"name":"Progentec Diagnostics Inc., Oklahoma City, OK 73104, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6868-4742","authenticated-orcid":false,"given":"Eldon R.","family":"Jupe","sequence":"additional","affiliation":[{"name":"Progentec Diagnostics Inc., Oklahoma City, OK 73104, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9250-3865","authenticated-orcid":false,"given":"Bernard","family":"Rubin","sequence":"additional","affiliation":[{"name":"Progentec Diagnostics Inc., Oklahoma City, OK 73104, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohan","family":"Purushothaman","sequence":"additional","affiliation":[{"name":"Progentec Diagnostics Inc., Oklahoma City, OK 73104, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3460","DOI":"10.1002\/nop2.1607","article-title":"Fragmented care and chronic illness patient outcomes: A systematic review","volume":"10","author":"Joo","year":"2023","journal-title":"Nurs. Open"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41746-025-01442-3","article-title":"A pathway from fragmentation to interoperability through standards-based enterprise architecture to enhance patient safety","volume":"8","author":"Wong","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1369","DOI":"10.1002\/acr.23161","article-title":"Disease Outcomes and Care Fragmentation Among Patients with Systemic Lupus Erythematosus","volume":"69","author":"Walunas","year":"2017","journal-title":"Arthritis Care Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Khairat, S., Morelli, J., Boynton, M.H., Bice, T., A Gold, J., and Carson, S.S. (2025). Investigation of Information Overload in Electronic Health Records: Protocol for Usability Study. JMIR Res. Protoc., 14.","DOI":"10.2196\/66127"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Asgari, E., Kaur, J., Nuredini, G., Balloch, J., Taylor, A.M., Sebire, N., Robinson, R., Peters, C., Sridharan, S., and Pimenta, D. (2024). Impact of Electronic Health Record Use on Cognitive Load and Burnout Among Clinicians: Narrative Review. JMIR Med. Inform., 12.","DOI":"10.2196\/55499"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cahill, M., Cleary, B.J., and Cullinan, S. (2025). The influence of electronic health record design on usability and medication safety: Systematic review. BMC Health Serv. Res., 25.","DOI":"10.1186\/s12913-024-12060-2"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e999","DOI":"10.1097\/PTS.0000000000001002","article-title":"Patient Safety Issues from Information Overload in Electronic Medical Records","volume":"18","author":"Nijor","year":"2022","journal-title":"J. Patient Saf."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1186\/s13012-024-01357-9","article-title":"Generative AI in healthcare: An implementation science informed translational path on application, integration and governance","volume":"19","author":"Reddy","year":"2024","journal-title":"Implement. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sequ\u00ed-Sabater, J.M., and Benavent, D. (2025). Artificial intelligence in rheumatology research: What is it good for?. RMD Open, 11.","DOI":"10.1136\/rmdopen-2024-004309"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Templin, T., Perez, M.W., Sylvia, S., Leek, J., Sinnott-Armstrong, N., and Silva, J.N.A. (2024). Addressing 6 challenges in generative AI for digital health: A scoping review. PLoS Digit. Health, 3.","DOI":"10.1371\/journal.pdig.0000503"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chustecki, M. (2024). Benefits and Risks of AI in Health Care: Narrative Review. Interact. J. Med. Res., 13.","DOI":"10.2196\/53616"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hassan, R., Faruqui, H., Alquraa, R., Eissa, A., Alshaiki, F., and Cheikh, M. (2021). Classification Criteria and Clinical Practice Guidelines for Rheumatic Diseases. Skills in Rheumatology, Springer.","DOI":"10.1007\/978-981-15-8323-0_25"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1016\/j.berh.2015.04.004","article-title":"The use and abuse of diagnostic\/classification criteria","volume":"28","author":"June","year":"2014","journal-title":"Best Pract. Res. Clin. Rheumatol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1580","DOI":"10.1136\/ard.2010.138461","article-title":"2010 Rheumatoid arthritis classification criteria: An American College of Rheumatology\/European League Against Rheumatism collaborative initiative","volume":"69","author":"Aletaha","year":"2010","journal-title":"Ann. Rheum. Dis."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1747","DOI":"10.1136\/annrheumdis-2013-204424","article-title":"2013 classification criteria for systemic sclerosis: An American college of rheumatology\/European league against rheumatism collaborative initiative","volume":"72","author":"Khanna","year":"2013","journal-title":"Ann. Rheum. Dis."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2557","DOI":"10.1002\/art.39254","article-title":"2015 Gout Classification Criteria: An American College of Rheumatology\/European League Against Rheumatism collaborative initiative","volume":"67","author":"Neogi","year":"2015","journal-title":"Arthritis Rheumatol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1002\/acr.20140","article-title":"The American College of rheumatology preliminary diagnostic criteria for fibromyalgia and measurement of symptom severity","volume":"62","author":"Wolfe","year":"2010","journal-title":"Arthritis Care Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2665","DOI":"10.1002\/art.21972","article-title":"Classification criteria for psoriatic arthritis: Development of new criteria from a large international study","volume":"54","author":"Taylor","year":"2006","journal-title":"Arthritis Care Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2271","DOI":"10.1002\/art.40320","article-title":"2017 European League Against Rheumatism\/American College of Rheumatology Classification Criteria for Adult and Juvenile Idiopathic Inflammatory Myopathies and Their Major Subgroups","volume":"69","author":"Lundberg","year":"2017","journal-title":"Arthritis Rheumatol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1400","DOI":"10.1002\/art.40930","article-title":"2019 European League Against Rheumatism\/American College of Rheumatology Classification Criteria for Systemic Lupus Erythematosus","volume":"71","author":"Aringer","year":"2019","journal-title":"Arthritis Rheumatol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Nair, S., Lushington, G.H., Purushothaman, M., Rubin, B., Jupe, E., and Gattam, S. (2025). Prediction of Lupus Classification Criteria via Generative AI Medical Record Profiling. BioTech, 14.","DOI":"10.3390\/biotech14010015"},{"key":"ref_22","unstructured":"(Claude, 2024). Claude, version 3.0."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1725","DOI":"10.1002\/art.1780400928","article-title":"Updating the American College of Rheumatology revised criteria for the classification of systemic lupus ery-thematosus","volume":"40","author":"Hochberg","year":"1997","journal-title":"Arthritis Rheum."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"214","DOI":"10.3899\/jrheum.2025-0390.PV217","article-title":"Expediting lupus classification of at-risk individuals using novel technology: Outcomes of a pilot study","volume":"52","author":"Jupe","year":"2025","journal-title":"J. Rheumatol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"101471","DOI":"10.1016\/j.conctc.2025.101471","article-title":"Impact of a digital platform and flare risk blood biomarker index on lupus: A study protocol design for evaluating self efficacy and disease management","volume":"45","author":"Jupe","year":"2025","journal-title":"Contemp. Clin. Trials Commun."},{"key":"ref_26","unstructured":"Belval, E., Delteil, T., Schade, M., and Radhakrishna, S. (Amazon Textract, 2025). Amazon Textract, version 1.9.2."},{"key":"ref_27","first-page":"9459","article-title":"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"101260","DOI":"10.1016\/j.patter.2025.101260","article-title":"Unleashing the potential of prompt engineering for large language models","volume":"6","author":"Chen","year":"2025","journal-title":"Patterns"},{"key":"ref_29","unstructured":"(Claude, 2025). Claude, version 3.5."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1090\/conm\/026\/737400","article-title":"Extensions of Lipschitz mappings into a Hilbert space","volume":"26","author":"Johnson","year":"1984","journal-title":"Contemp. Math."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sauro, J., and Lewis, J.R. (2005, January 26\u201330). Comparison of Wald, Adj-Wald, exact, and Wilson intervals calculator. Proceedings of the Human Factors and Ergonomics Society, 49th Annual Meeting (HFES 2005), Orlando, FL, USA.","DOI":"10.1177\/154193120504902407"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"VBommineni, L., Bhagwagar, S., Balcarcel, D., Davatzikos, C., and Boyer, D. (2023). Performance of ChatGPT on the MCAT: The Road to Personalized and Equitable Premedical Learning. MedRxiv.","DOI":"10.1101\/2023.03.05.23286533"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1038\/s41746-025-01543-z","article-title":"A systematic review and meta-analysis of diagnostic performance comparison between generative AI and physicians","volume":"8","author":"Takita","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_34","first-page":"164","article-title":"Rheumatologic Tests: A Primer for Family Physicians","volume":"98","author":"Ali","year":"2018","journal-title":"Am. Fam. Physician"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1002\/art.10561","article-title":"Evidence-based guidelines for the use of immunologic tests: Antinuclear antibody testing","volume":"47","author":"Solomon","year":"2002","journal-title":"Arthritis Care Res."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1136\/annrheumdis-2014-eular.4028","article-title":"AB1027 Diagnostic Accuracy of Automated Determination of Antinuclear Antibodies (ANA) by Indirect REAction of Immunofluorescence on Human Hep-2 Cells (IIF-HEP-2) and Enzyme-Linked Immunosorbent Assay (ELISA) for Diagnosis of Systemic Lupus Erythematosus (SLE)","volume":"73","author":"Verizhnikova","year":"2014","journal-title":"Ann. Rheum. Dis."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e53008","DOI":"10.2196\/53008","article-title":"Generative AI in Medical Practice: In-Depth Exploration of Privacy and Security Challenges","volume":"26","author":"Chen","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e32364","DOI":"10.1016\/j.heliyon.2024.e32364","article-title":"Generative artificial intelligence in healthcare from the perspective of digital media: Applications, opportunities and challenges","volume":"10","author":"Xu","year":"2024","journal-title":"Heliyon"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1007\/s10916-024-02136-1","article-title":"Generative Artificial Intelligence Use in Healthcare: Opportunities for Clinical Excellence and Administrative Efficiency","volume":"49","author":"Bhuyan","year":"2025","journal-title":"J. Med. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1038\/s41746-024-01409-w","article-title":"A review on generative AI models for synthetic medical text, time series, and longitudinal data","volume":"8","author":"Loni","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s13750-025-00362-9","article-title":"Evaluating generative AI for qualitative data extraction in community-based fisheries management literature","volume":"14","author":"Spillias","year":"2025","journal-title":"Environ. Evid."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"074501","DOI":"10.1115\/1.4068275","article-title":"Human\u2013Artificial Intelligence Teaming for Scientific Information Extraction from Data-Driven Additive Manufacturing Literature Using Large Language Models","volume":"25","author":"Safdar","year":"2025","journal-title":"J. Comput. Inf. Sci. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1093\/jamia\/ocaf030","article-title":"Enhancing systematic literature reviews with generative artificial intelligence: Development, applications, and performance evaluation","volume":"32","author":"Li","year":"2025","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1007\/s10916-025-02167-2","article-title":"Using Generative AI to Extract Structured Information from Free Text Pathology Reports","volume":"49","author":"Shahid","year":"2025","journal-title":"J. Med. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1016\/j.arthro.2024.12.011","article-title":"Ethical Application of Generative Artificial Intelligence in Medicine","volume":"41","author":"Hasan","year":"2024","journal-title":"Arthrosc. J. Arthrosc. Relat. Surg."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1038\/s41746-025-01721-z","article-title":"Situating governance and regulatory concerns for generative artificial intelligence and large language models in medical education","volume":"8","author":"Tran","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"e848","DOI":"10.1016\/S2589-7500(24)00143-2","article-title":"Generative artificial intelligence and ethical considerations in health care: A scoping review and ethics checklist","volume":"6","author":"Ning","year":"2024","journal-title":"Lancet Digit. Health"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"e52073","DOI":"10.2196\/52073","article-title":"Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review","volume":"12","author":"Yim","year":"2024","journal-title":"JMIR Med. Inform."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Sj\u00f6wall, C., and Parodis, I. (2022). Clinical Heterogeneity, Unmet Needs and Long-Term Outcomes in Patients with Systemic Lupus Erythematosus. J. Clin. Med., 11.","DOI":"10.3390\/jcm11226869"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1038\/s41392-025-02168-0","article-title":"Systemic lupus erythematosus: Updated insights on the pathogenesis, diagnosis, prevention and therapeutics","volume":"10","author":"Dai","year":"2025","journal-title":"Signal Transduct. Target. Ther."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1016\/1047-2797(94)00096-C","article-title":"A connective tissue disease screening questionnaire for population studies","volume":"5","author":"Karlson","year":"1995","journal-title":"Ann. Epidemiol."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1191\/0961203305lu2227oa","article-title":"High sensitivity, specificity and predictive value of the Connective Tissue Disease Screening Questionnaire among urban African-American women","volume":"14","author":"Karlson","year":"2005","journal-title":"Lupus"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2677","DOI":"10.1002\/art.34473","article-title":"Derivation and validation of the Systemic Lupus International Collaborating Clinics classification criteria for systemic lupus erythematosus","volume":"64","author":"Petri","year":"2012","journal-title":"Arthritis Rheum."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/18\/2409\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:47:02Z","timestamp":1760035622000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/18\/2409"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,22]]},"references-count":53,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["diagnostics15182409"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics15182409","relation":{},"ISSN":["2075-4418"],"issn-type":[{"type":"electronic","value":"2075-4418"}],"subject":[],"published":{"date-parts":[[2025,9,22]]}}}