{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:46:49Z","timestamp":1760060809291,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T00:00:00Z","timestamp":1758499200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background and Objectives: Interpreting acid\u2013base disorders is challenging, particularly in complex or mixed cases. Given the growing potential of large language models (LLMs) to assist in cognitively demanding tasks, this study evaluated their performance in interpreting arterial blood gas (ABG) results. Materials and Methods: In this single-center retrospective study, 200 ABG datasets were curated to include 40 cases in each of five diagnostic categories: metabolic acidosis, respiratory acidosis, metabolic alkalosis, respiratory alkalosis, and no acid\u2013base disorder. Three medical students, each assigned to one LLM (ChatGPT GPT-4o, Copilot GPT-4, or Gemini 1.5-flash\/2.5-flash), perform ABG interpretation using two evaluation methods: interpretation (LLM-I) and interpretation with supervision model (LLM-S). Two clinical pathologists independently performed the conventional evaluation to serve as the reference standard. Results: Agreement for identifying the primary acid\u2013base (APD) disorder was strong across all approaches (Cohen\u2019s \u03ba \u2265 0.88). For identifying both primary and secondary disorders regardless of order (APSD), LLM-I showed moderate agreement (ChatGPT \u03ba = 0.65, Copilot \u03ba = 0.61, Gemini \u03ba = 0.62), whereas LLM-S achieved strong agreement (ChatGPT \u03ba = 0.91, Copilot \u03ba = 0.81, Gemini \u03ba = 0.81). Conclusions: LLM-assisted ABG interpretation demonstrates strong concordance with expert interpretation in detecting primary acid\u2013base disorders. These tools may enhance the understanding of acid\u2013base disorders while reducing calculation-related errors among medical students.<\/jats:p>","DOI":"10.3390\/jcm14186676","type":"journal-article","created":{"date-parts":[[2025,9,22]],"date-time":"2025-09-22T13:04:44Z","timestamp":1758546284000},"page":"6676","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Human-in-the-Loop Performance of LLM-Assisted Arterial Blood Gas Interpretation: A Single-Center Retrospective Study"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5184-3532","authenticated-orcid":false,"given":"Sergio","family":"Ayala-De la Cruz","sequence":"first","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Departamento de Patolog\u00eda Cl\u00ednica, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-2563-3287","authenticated-orcid":false,"given":"Paola Elizabeth","family":"Arenas-Hern\u00e1ndez","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-4007-608X","authenticated-orcid":false,"given":"Mar\u00eda Fernanda","family":"Fern\u00e1ndez-Herrera","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rebeca Alejandrina","family":"Qui\u00f1ones-D\u00edaz","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6875-2530","authenticated-orcid":false,"given":"Jorge Mart\u00edn","family":"Llaca-D\u00edaz","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Departamento de Patolog\u00eda Cl\u00ednica, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Erik Alejandro","family":"D\u00edaz-Chuc","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Departamento de Patolog\u00eda Cl\u00ednica, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2802-5643","authenticated-orcid":false,"given":"Diana Guadalupe","family":"Robles-Espino","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Departamento de Patolog\u00eda Cl\u00ednica, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Erik Alejandro","family":"San Miguel-Garay","sequence":"additional","affiliation":[{"name":"Facultad de Medicina y Hospital Universitario \u201cDr. Jos\u00e9 Eleuterio Gonz\u00e1lez\u201d, Departamento de Patolog\u00eda Cl\u00ednica, Universidad Aut\u00f3noma de Nuevo Le\u00f3n, Monterrey 64460, Nuevo Le\u00f3n, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1549","DOI":"10.1007\/s00259-023-06172-w","article-title":"Large language models (LLM) and ChatGPT: What will the impact on nuclear medicine be?","volume":"50","author":"Alberts","year":"2023","journal-title":"Eur. J. Nucl. Med. Mol. Imaging"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2792","DOI":"10.7150\/ijms.111780","article-title":"Large Language Models in Medicine: Applications, Challenges, and Future Directions","volume":"22","author":"Yu","year":"2025","journal-title":"Int. J. Med. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","article-title":"Evaluation and mitigation of the limitations of large language models in clinical decision-making","volume":"30","author":"Hager","year":"2024","journal-title":"Nat. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1093\/clinchem\/hvad106","article-title":"AI Chatbots in Clinical Laboratory Medicine: Foundations and Trends","volume":"69","author":"Yang","year":"2023","journal-title":"Clin. Chem."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Vrdoljak, J., Boban, Z., Vilovi\u0107, M., Kumri\u0107, M., and Bo\u017ei\u0107, J. (2025). A Review of Large Language Models in Medical Education, Clinical Decision Support, and Healthcare Administration. Healthcare, 13.","DOI":"10.3390\/healthcare13060603"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Doumat, G., Daher, D., Ghanem, N.-N., and Khater, B. (2022). Knowledge and attitudes of medical students in Lebanon toward artificial intelligence: A national survey study. Front. Artif. Intell., 5.","DOI":"10.3389\/frai.2022.1015418"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"205520762210890","DOI":"10.1177\/20552076221089099","article-title":"Artificial intelligence and medical education: A global mixed-methods study of medical students\u2019 perspectives","volume":"8","author":"Ejaz","year":"2022","journal-title":"Digit. Health"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1434","DOI":"10.1056\/NEJMra1003327","article-title":"Physiological Approach to Assessment of Acid\u2013Base Disturbances","volume":"371","author":"Berend","year":"2014","journal-title":"N. Engl. J. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1056\/NEJMra1711860","article-title":"Diagnostic Use of Base Excess in Acid\u2013Base Disorders","volume":"378","author":"Berend","year":"2018","journal-title":"N. Engl. J. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1681\/ASN.2009121211","article-title":"Secondary Responses to Altered Acid-Base Status","volume":"21","author":"Madias","year":"2010","journal-title":"J. Am. Soc. Nephrol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Donner, A., and Rotondi, M.A. (2010). Sample Size Requirements for Interval Estimation of the Kappa Statistic for Interobserver Agreement Studies with a Binary Outcome and Multiple Raters. Int. J. Biostat., 6.","DOI":"10.2202\/1557-4679.1275"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: The kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.redar.2019.04.001","article-title":"El algoritmo de la gasometr\u00eda arterial: Propuesta de un enfoque sistem\u00e1tico para el an\u00e1lisis de los trastornos del equilibrio \u00e1cido-base","volume":"67","author":"Fletcher","year":"2020","journal-title":"Rev. Esp. Anestesiol. Reanim."},{"key":"ref_14","unstructured":"Kaufman, D. (2025, July 14). Interpretation of Arterial Blood Gases (ABGs). Available online: https:\/\/www.thoracic.org\/professionals\/clinical-resources\/critical-care\/clinical-education\/abgs.php."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1053\/j.ajkd.2021.02.341","article-title":"Approach to Patients with High Anion Gap Metabolic Acidosis: Core Curriculum 2021","volume":"78","author":"Fenves","year":"2021","journal-title":"Am. J. Kidney Dis."},{"key":"ref_16","unstructured":"Emmett, M., and Palmer, B. (2025, July 16). The Delta Anion Gap\/Delta HCO3 Ratio in Patients with a High Anion Gap Metabolic Acidosis. Available online: https:\/\/www.uptodate.com\/contents\/the-delta-anion-gap-delta-hco3-ratio-in-patients-with-a-high-anion-gap-metabolic-acidosis."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Delgado, R., and Tibau, X.-A. (2019). Why Cohen\u2019s Kappa should be avoided as performance measure in classification. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0222916"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.compbiolchem.2004.09.006","article-title":"Comparing two K-category assignments by a K-category correlation coefficient","volume":"28","author":"Gorodkin","year":"2004","journal-title":"Comput. Biol. Chem."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.patcog.2017.03.008","article-title":"Generic performance measure for multiclass-classifiers","volume":"68","author":"Kautz","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_20","unstructured":"Brinkman, J.E., and Sharma, S. (2025, July 16). Physiology, Metabolic Alkalosis, StatPearls [Internet], Available online: https:\/\/pubmed.ncbi.nlm.nih.gov\/29493916\/."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"gfae069-1768-1772","DOI":"10.1093\/ndt\/gfae069.1768","article-title":"#1772 Comparison of interpretation of acid-base disorder in patients with critical illness: Nephrologist versus ChatGPT","volume":"39","author":"Lee","year":"2024","journal-title":"Nephrol. Dial. Transplant."},{"key":"ref_22","unstructured":"(2025, July 15). OpenAI Model Release Notes. Available online: https:\/\/help.openai.com\/en\/articles\/9624314-model-release-notes?utm_source=chatgpt.com#h_8e49be5daa."},{"key":"ref_23","unstructured":"(2025, July 15). Google Gemini 2.5 Flash. Available online: https:\/\/cloud.google.com\/vertex-ai\/generative-ai\/docs\/models\/gemini\/2-5-flash."},{"key":"ref_24","unstructured":"Spataro, J. (2025, July 15). Bringing the Latest Capabilities to Copilot for Microsoft 365 Customers. Available online: https:\/\/www.microsoft.com\/en-us\/microsoft-365\/blog\/2024\/04\/02\/bringing-the-latest-capabilities-to-copilot-for-microsoft-365-customers\/."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ajem.2025.04.028","article-title":"AI-Assisted Blood Gas Interpretation: A Comparative Study with an Emergency Physician","volume":"94","year":"2025","journal-title":"Am. J. Emerg. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2429","DOI":"10.1681\/ASN.2006121408","article-title":"Use of the \u0394AG\/\u0394HCO3\u2212 Ratio in the Diagnosis of Mixed Acid-Base Disorders","volume":"18","author":"Rastegar","year":"2007","journal-title":"J. Am. Soc. Nephrol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"20","DOI":"10.34067\/KID.0000842019","article-title":"The \u0394 Anion Gap\/\u0394 Bicarbonate Ratio in Early Lactic Acidosis: Time for Another Delta?","volume":"2","author":"Rudkin","year":"2021","journal-title":"Kidney360"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Khasawneh, E., Gosling, C., and Williams, B. (2021). What impact does maths anxiety have on university students?. BMC Psychol., 9.","DOI":"10.1186\/s40359-021-00537-2"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"111787","DOI":"10.1016\/j.jclinane.2025.111787","article-title":"Assessing the accuracy of ChatGPT in interpreting blood gas analysis results ChatGPT-4 in blood gas analysis","volume":"102","author":"Turan","year":"2025","journal-title":"J. Clin. Anesth."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"209","DOI":"10.2147\/IJNRD.S62126","article-title":"Facing acid\u2013base disorders in the third millennium\u2014The Stewart approach revisited","volume":"7","author":"Honore","year":"2014","journal-title":"Int. J. Nephrol. Renov. Dis."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s11845-024-03767-6","article-title":"Interpretation of acid\u2013base metabolism on arterial blood gas samples via machine learning algorithms","volume":"194","author":"Ozdemir","year":"2025","journal-title":"Ir. J. Med. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Urda-C\u00eempean, A.E., Leucu\u021ba, D.-C., Drugan, C., Du\u021bu, A.-G., C\u0103linici, T., and Drugan, T. (2025). Assessing the Accuracy of Diagnostic Capabilities of Large Language Models. Diagnostics, 15.","DOI":"10.3390\/diagnostics15131657"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1093\/ajcp\/aqad158","article-title":"Evaluation of ChatGPT pathology knowledge using board-style questions","volume":"161","author":"Geetha","year":"2024","journal-title":"Am. J. Clin. Pathol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"B\u00e9lisle-Pipon, J.-C. (2024). Why we need to be careful with LLMs in medicine. Front. Med., 11.","DOI":"10.3389\/fmed.2024.1495582"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"229","DOI":"10.7180\/kmj.24.140","article-title":"The ethics of using artificial intelligence in medical research","volume":"39","author":"Yu","year":"2024","journal-title":"Kosin Med. J."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1177\/15533506241248239","article-title":"The Role of Artificial Intelligence in Medical Education: A Systematic Review","volume":"31","author":"Tozsin","year":"2024","journal-title":"Surg. Innov."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"62","DOI":"10.5492\/wjccm.v4.i1.62","article-title":"Has Stewart approach improved our ability to diagnose acid-base disorders in critically ill patients?","volume":"4","author":"Masevicius","year":"2015","journal-title":"World J. Crit. Care Med."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1026","DOI":"10.1001\/jamapediatrics.2025.1729","article-title":"Evaluating a Large Language Model in Translating Patient Instructions to Spanish Using a Standardized Framework","volume":"179","author":"Ray","year":"2025","journal-title":"JAMA Pediatr."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Delaunay, J., and Cusido, J. (2024). Evaluating the Performance of Large Language Models in Predicting Diagnostics for Spanish Clinical Cases in Cardiology. Appl. Sci., 15.","DOI":"10.3390\/app15010061"},{"key":"ref_40","first-page":"28186","article-title":"Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages","volume":"39","author":"Li","year":"2024","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Liu, Y., Huang, W., Mao, J., Wang, R., and Hu, H. (2024). MELA: Multilingual Evaluation of Linguistic Acceptability. arXiv.","DOI":"10.18653\/v1\/2024.acl-long.146"}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/18\/6676\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:47:07Z","timestamp":1760035627000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/18\/6676"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,22]]},"references-count":41,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["jcm14186676"],"URL":"https:\/\/doi.org\/10.3390\/jcm14186676","relation":{},"ISSN":["2077-0383"],"issn-type":[{"type":"electronic","value":"2077-0383"}],"subject":[],"published":{"date-parts":[[2025,9,22]]}}}