{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:43:39Z","timestamp":1785545019200,"version":"3.56.0"},"reference-count":25,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Manual chart abstraction from electronic health records is a critical step in clinical outcomes research but is time-intensive and prone to human error. Advances in artificial intelligence (AI), particularly large language models, offer the potential to automate the extraction of structured data from unstructured clinical documentation with improved efficiency and consistency.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate the accuracy and efficiency of an AI-assisted approach for extracting patient-reported outcomes from clinical notes compared with traditional human abstraction.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We conducted a retrospective study of 26 patients treated with low-dose radiation therapy for osteoarthritis. Human reviewers abstracted numeric rating scale (NRS; 0\u201010) pain scores at baseline, the end of treatment, and the first follow-up, and von Pannewitz score (VPS; 0\u20104) improvement scores at posttreatment time points. A HIPAA (Health Insurance Portability and Accountability Act)\u2013compliant generative pretrained transformer\u2013based AI system was prompted to extract the same end points from clinical notes. Concordance was assessed using exact match rates, the intraclass correlation coefficient for the NRS, and weighted Cohen \u03ba for the VPS. The time required for AI vs manual abstraction was recorded. The AI system was not trained or fine-tuned on study data, and performance was evaluated directly against human abstraction to reflect real-world deployment.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The AI system demonstrated high concordance with human abstraction, achieving an exact match rate of 92% for the NRS (95% CI 84\u201096; intraclass correlation coefficient=0.96) and 94% for the VPS (95% CI 84\u201098; \u03ba=0.91). All discrepancies were minor, and no spurious values were generated. The AI system identified 1 clinically relevant data point missed during manual review. Average abstraction time per patient decreased from approximately 30 minutes to 2 minutes, representing time savings of >90%. The system also captured trends in analgesic use, but these results were not statistically significant, including reductions without escalation.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n AI-assisted data abstraction demonstrated high concordance with human review in this single-institution cohort while substantially reducing the time requirements. These findings support the feasibility of AI-assisted abstraction workflows, although further validation across larger and more diverse datasets is needed.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/96755","type":"journal-article","created":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:21:12Z","timestamp":1783437672000},"page":"e96755-e96755","source":"Crossref","is-referenced-by-count":0,"title":["AI-Assisted Clinical Data Abstraction From Electronic Health Records: Retrospective Concordance Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-2968-2255","authenticated-orcid":false,"given":"Camille Sarah","family":"Schwartz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1786-3871","authenticated-orcid":false,"given":"Michael John","family":"Anderson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8857-8860","authenticated-orcid":false,"given":"Kelsey Nicole","family":"Moakler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6486-4428","authenticated-orcid":false,"given":"Bradley Adam","family":"Newby","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4120-3667","authenticated-orcid":false,"given":"David Alan","family":"Davenport","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2285-3115","authenticated-orcid":false,"given":"Matthew Wilson","family":"Schwartz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,7]]},"reference":[{"issue":"6","key":"R1","doi-asserted-by":"publisher","first-page":"438","DOI":"10.1089\/pop.2023.0227","article-title":"Chart review is dead; long live chart review: how artificial intelligence will make human review of medical records obsolete, one day","volume":"26","author":"Agatstein","journal-title":"Popul Health Manag"},{"issue":"41","key":"R2","doi-asserted-by":"publisher","first-page":"1101","DOI":"10.15585\/mmwr.mm7241a1","article-title":"Prevalence of diagnosed arthritis - United States, 2019-2021","volume":"72","author":"Fallon","journal-title":"MMWR Morb Mortal Wkly Rep"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00066-014-0757-3","article-title":"DEGRO guidelines for the radiotherapy of non-malignant disorders. Part II: painful degenerative skeletal disorders","volume":"191","author":"Ott","journal-title":"Strahlenther Onkol"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1016\/j.ijrobp.2025.04.040","article-title":"Low-dose radiation therapy for osteoarthritis: a retrospective single-institution analysis of 69 patients and 168 joints","volume":"123","author":"Koneru","journal-title":"Int J Radiat Oncol Biol Phys"},{"issue":"4","key":"R5","doi-asserted-by":"publisher","first-page":"1198","DOI":"10.1016\/j.ijrobp.2025.08.030","article-title":"Clinical effectiveness of single course low-dose radiation therapy in knee osteoarthritis: short-term results from the randomized, sham-controlled trial","volume":"123","author":"Kim","journal-title":"Int J Radiat Oncol Biol Phys"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1093\/jamia\/ocz063","article-title":"A study of deep learning approaches for medication and adverse drug event extraction from clinical text","volume":"27","author":"Wei","journal-title":"J Am Med Inform Assoc"},{"issue":"7","key":"R7","doi-asserted-by":"publisher","first-page":"696","DOI":"10.1111\/acem.14937","article-title":"Using natural language processing in emergency medicine health service research: a systematic review and meta-analysis","volume":"31","author":"Wang","journal-title":"Acad Emerg Med"},{"issue":"11","key":"R8","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.73994","article-title":"Evaluating the impact of artificial intelligence (AI) on clinical documentation efficiency and accuracy across clinical settings: a scoping review","volume":"16","author":"Lee","journal-title":"Cureus"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-77535-y","article-title":"Development and validation of a novel AI framework using NLP with LLM integration for relevant clinical data extraction through automated chart review","volume":"14","author":"Dagli","journal-title":"Sci Rep"},{"issue":"2","key":"R10","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooae044","article-title":"Natural language processing systems for extracting information from electronic health records about activities of daily living. A systematic review","volume":"7","author":"Wieland-Jorna","journal-title":"JAMIA Open"},{"issue":"4","key":"R11","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1080\/14737167.2024.2322664","article-title":"Using natural language processing to analyze unstructured patient-reported outcomes data derived from electronic health records for cancer populations: a systematic review","volume":"24","author":"Sim","journal-title":"Expert Rev Pharmacoecon Outcomes Res"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1136\/bmjhci-2024-101139","article-title":"Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation","volume":"32","author":"Ntinopoulos","journal-title":"BMJ Health Care Inform"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01645-8","article-title":"Social determinants of health extraction from clinical notes across institutions using large language models","volume":"8","author":"Keloth","journal-title":"NPJ Digit Med"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00742-2","article-title":"A large language model for electronic health records","volume":"5","author":"Yang","journal-title":"NPJ Digit Med"},{"issue":"6","key":"R15","doi-asserted-by":"publisher","first-page":"797","DOI":"10.1001\/jamaoncol.2016.0213","article-title":"Natural language processing in oncology: a review","volume":"2","author":"Yim","journal-title":"JAMA Oncol"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.2196\/65984","article-title":"Large language model applications for health information extraction in oncology: scoping review","volume":"11","author":"Chen","journal-title":"JMIR Cancer"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.2196\/73605","article-title":"Large language model versus manual review for clinical data curation in breast cancer: retrospective comparative study","volume":"13","author":"Kang","journal-title":"JMIR Med Inform"},{"issue":"5","key":"R18","doi-asserted-by":"publisher","first-page":"e374","DOI":"10.1016\/j.jpainsymman.2025.01.019","article-title":"NLP for analyzing electronic health records and clinical notes in cancer research: a review","volume":"69","author":"Bilal","journal-title":"J Pain Symptom Manage"},{"key":"R19","doi-asserted-by":"publisher","DOI":"10.2196\/68707","article-title":"Performance of natural language processing for information extraction from electronic health records within cancer: systematic review","volume":"13","author":"Dahl","journal-title":"JMIR Med Inform"},{"issue":"3","key":"R20","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1302\/2633-1462.63.BJO-2024-0081.R1","article-title":"A systematic review of natural language processing applications in Trauma & Orthopaedics","volume":"6","author":"Farrow","journal-title":"Bone Jt Open"},{"issue":"3","key":"R21","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1016\/j.ijrobp.2021.01.044","article-title":"Clinical natural language processing for radiation oncology: a review and practical primer","volume":"110","author":"Bitterman","journal-title":"Int J Radiat Oncol Biol Phys"},{"key":"R22","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","journal-title":"BMJ"},{"issue":"10","key":"R23","doi-asserted-by":"publisher","first-page":"e537","DOI":"10.1016\/S2589-7500(20)30218-1","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension","volume":"2","author":"Liu","journal-title":"Lancet Digit Health"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m3210","article-title":"Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension","volume":"370","author":"Rivera","journal-title":"BMJ"},{"key":"R25","unstructured":"Ethics and governance of artificial intelligence for health: guidance on large multi-modal models. World Health Organization. 2024. URL: https:\/\/iris.who.int\/server\/api\/core\/bitstreams\/e9e62c65-6045-481e-bd04-20e206bc5039\/content [Accessed 23-10-2025]"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e96755","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:23:15Z","timestamp":1785536595000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e96755"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,7]]},"references-count":25,"URL":"https:\/\/doi.org\/10.2196\/96755","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,7]]},"article-number":"v10i6e96755"}}