{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T07:24:00Z","timestamp":1792653840869,"version":"4.5.0"},"reference-count":18,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Accurate preanesthetic assessment is essential for perioperative risk stratification, but conventional tools such as the American Society of Anesthesiologists (ASA) physical status classification and postoperative nausea and vomiting (PONV) risk scores may be affected by subjective judgment, incomplete documentation, and fragmented clinical data. Large language models may support preanesthetic assessment by integrating structured and unstructured clinical information.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to develop and retrospectively validate a GPT-4o (OpenAI)\u2013powered AI system for preanesthetic assessment. The primary validation focus was the agreement between AI-generated and clinician-assigned ASA physical status classifications. PONV risk stratification was evaluated as an additional clinically relevant performance outcome. A secondary objective was to assess the incremental contribution of National Health Insurance (NHI) cloud data to model performance.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n This single-center retrospective validation study included 600 adult surgical patients randomly sampled from 4404 eligible inpatient surgical patients between January and May 2025. The system processed structured and unstructured electronic health record data, with and without NHI cloud data, to generate preanesthetic assessment outputs. ASA agreement was assessed using Cohen \u03ba with 95% CIs. PONV prediction was evaluated using sensitivity, specificity, overall accuracy, positive predictive value, negative predictive value, confusion matrix analysis, the 2-sided Fisher-Freeman-Halton exact test, and Cram\u00e9r V.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Among the 600 patients, clinician-assigned ASA classifications were ASA I in 39 patients, ASA II in 498 patients, ASA III in 57 patients, and ASA IV in 6 patients. Agreement between AI-generated and clinician-assigned ASA classifications was high when NHI data were incorporated (\u03ba=0.883, 95% CI 0.841\u20100.946), whereas agreement was lower without NHI data (\u03ba=0.518, 95% CI 0.412\u20100.691). A total of 49 (8.2%) patients experienced documented PONV within 24 hours after surgery. Using the high-risk category as the primary test-positive threshold, the model achieved a sensitivity of 34.7% (95% CI 22.9\u201048.7), a specificity of 99.1% (95% CI 97.9\u201099.6), a positive predictive value of 77.3%, and a negative predictive value of 94.5%. The association between the model-assigned PONV risk categories and observed PONV outcomes was statistically significant (\n P<\/jats:italic>\n <.001, Cram\u00e9r V=0.531).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The GPT-4o\u2013powered preanesthetic assessment system demonstrated feasibility and high agreement with clinician-assigned ASA classifications when NHI cloud data were incorporated. For PONV risk stratification, the system showed high specificity and a high negative predictive value but modest sensitivity, indicating that it identified a small high-risk subgroup with a high observed PONV incidence but did not capture all patients who developed PONV. These findings support the use of large language model\u2013based decision-support tools as adjuncts to, rather than replacements for, anesthesiologist judgment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/83893","type":"journal-article","created":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T14:45:07Z","timestamp":1785854707000},"page":"e83893-e83893","source":"Crossref","is-referenced-by-count":2,"title":["GPT-4o\u2013Powered Preanesthetic AI: Development and Validation Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-3315-5380","authenticated-orcid":false,"given":"Bo-Han","family":"Lin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2217-3231","authenticated-orcid":false,"given":"Cheng-Wei","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0520-8946","authenticated-orcid":false,"given":"I-Ching","family":"Hou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1374-5189","authenticated-orcid":false,"given":"Tzu-Yu","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8378-7119","authenticated-orcid":false,"given":"Chin-E","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1040-9857","authenticated-orcid":false,"given":"Chong-Sun","family":"Khoi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,25]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinane.2023.111103","article-title":"Development and validation of a machine learning ASA-score to identify candidates for comprehensive preoperative screening and risk stratification","volume":"87","author":"Wongtangman","journal-title":"J Clin Anesth"},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinane.2024.111475","article-title":"Evaluating the accuracy of ChatGPT-4 in predicting ASA scores: a prospective multicentric study ChatGPT-4 in ASA score prediction","volume":"96","author":"Turan","journal-title":"J Clin Anesth"},{"issue":"7","key":"R3","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.1177\/15589447251383047","article-title":"Artificial intelligence can inform prediction of American Society of Anesthesiologists physical status classification in hand surgery","volume":"21","author":"Zhuang","journal-title":"Hand (N Y)"},{"issue":"3","key":"R4","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1097\/00000542-199909000-00022","article-title":"A simplified risk score for predicting postoperative nausea and vomiting: conclusions from cross-validations between two centers","volume":"91","author":"Apfel","journal-title":"Anesthesiology"},{"issue":"5","key":"R5","doi-asserted-by":"publisher","DOI":"10.2196\/38241","article-title":"Predicting postoperative mortality with deep neural networks and natural language processing: model development and validation","volume":"10","author":"Chen","journal-title":"JMIR Med Inform"},{"issue":"5","key":"R6","doi-asserted-by":"publisher","DOI":"10.3390\/jpm13050717","article-title":"Development and validation of artificial-intelligence-based radiomics model using computed tomography features for preoperative risk stratification of gastrointestinal stromal tumors","volume":"13","author":"Rengo","journal-title":"J Pers Med"},{"issue":"11","key":"R7","doi-asserted-by":"publisher","first-page":"6245","DOI":"10.3390\/app15116245","article-title":"Evaluating the performance of large language models on the CONACEM anesthesiology certification exam: a comparison with human participants","volume":"15","author":"Altermatt","journal-title":"Appl Sci"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-025-08084-9","article-title":"Evaluating GPT-4o in high-stakes medical assessments: performance and error analysis on a Chilean anesthesiology exam","volume":"25","author":"Altermatt","journal-title":"BMC Med Educ"},{"issue":"11","key":"R9","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1007\/s11604-024-01619-y","article-title":"Diagnostic performances of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro in \u201cDiagnosis Please\u201d cases","volume":"42","author":"Sonoda","journal-title":"Jpn J Radiol"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01519-z","article-title":"Retrieval augmented generation for 10 large language models and its generalizability in assessing medical fitness","volume":"8","author":"Ke","journal-title":"NPJ Digit Med"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","DOI":"10.1186\/s12871-025-02987-2","article-title":"Predicting postoperative nausea and vomiting using machine learning: a model development and validation study","volume":"25","author":"Glebov","journal-title":"BMC Anesthesiol"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-019-0189-7","article-title":"Human-machine partnership with artificial intelligence for chest radiograph diagnosis","volume":"2","author":"Patel","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R13","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0001278","article-title":"Real world human-LLM interactions - prospective blinded versus unblinded expert physician assessments of LLM responses to complex medical dilemmas","volume":"5","author":"Ben Shitrit","journal-title":"PLOS Digit Health"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","article-title":"The TRIPOD-LLM reporting guideline for studies using large language models","volume":"31","author":"Gallifant","journal-title":"Nat Med"},{"key":"R15","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","journal-title":"BMJ"},{"issue":"5","key":"R16","doi-asserted-by":"publisher","first-page":"614","DOI":"10.1177\/0310057X1404200511","article-title":"Inter-rater reliability of the ASA physical status classification in a sample of anaesthetists in Western Australia","volume":"42","author":"Riley","journal-title":"Anaesth Intensive Care"},{"issue":"3","key":"R17","doi-asserted-by":"publisher","first-page":"424","DOI":"10.1093\/bja\/aeu100","article-title":"Reliability of the American Society of Anesthesiologists physical status scale in clinical practice","volume":"113","author":"Sankar","journal-title":"Br J Anaesth"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-10736-5","article-title":"Discordant American Society of Anesthesiologists physical status classification between anesthesiologists and surgeons and its correlation with adverse patient outcomes","volume":"12","author":"Kwa","journal-title":"Sci Rep"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T19:12:01Z","timestamp":1792609921000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e83893"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,25]]},"references-count":18,"URL":"https:\/\/doi.org\/10.2196\/83893","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,25]]},"article-number":"v14i7e83893"}}