{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T17:46:48Z","timestamp":1790876808975,"version":"4.1.0"},"reference-count":30,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Large language models (LLMs) demonstrate potential in the laboratory, yet rigorous clinical evaluation remains limited. The opacity of LLM decision-making constrains their safe application in interpreting complete blood count (CBC) reports for hematologic diseases.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to conduct an exploratory evaluation of GPT-5, Grok 4, and DeepSeek R1 in interpreting real-world CBC reports, particularly their reasoning capabilities and clinical safety.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n This single-center retrospective study analyzed 100 CBC reports from initial-visit patients with hematologic conditions. After responses were generated by the 3 LLMs using standardized Chinese prompts, four trained laboratory physicians blindly evaluated them across 6 quality and 5 task dimensions. Interrater reliability was assessed using intraclass correlation coefficients (ICCs), and performance differences were assessed based on 4-rater consensus scores and Friedman and Wilcoxon tests. For task 4 (ablation analysis), the McNemar test was used to compare top-1 diagnostic concordance with the gold-standard diagnosis within each model, with and without initial clinical suspicion in the prompt. Error types and distributions were documented during the task evaluation.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n DeepSeek R1 demonstrated excellent interrater reliability across most quality dimensions (ICC \u22650.75). In the quality dimension, DeepSeek R1 significantly outperformed the other models in comprehensiveness, accuracy, clarity, relevance, and practicality. In the task 4 evaluation, GPT-5 demonstrated the highest concordance (93\/100, 93%) with gold-standard diagnoses, followed by DeepSeek R1 (92\/100, 92%) and Grok 4 (89\/100, 89%). After removing the initial clinical suspicion, these rates decreased to 79% (79\/100), 77% (77\/100), and 72% (72\/100), representing statistically significant within-model reductions for all models (\n P<\/jats:italic>\n <.001). Post hoc error analysis revealed distinct patterns across task dimensions. GPT-5 exhibited 12 hallucinations in the analyzer alert processing task; DeepSeek R1 demonstrated 1 hallucination in the abnormal item identification task, whereas Grok 4 displayed none. All models exhibited reasoning errors and varying degrees of deficiencies in the correlation analysis and preliminary diagnosis tasks, characterized by unwarranted inferences of disease status from isolated results without clinical integration. Grok 4 generated 9 reasoning errors in the clinical management task by providing generic recommendations not tailored to case-specific CBC data, potentially compromising individualized treatment decisions.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n While current LLMs demonstrate potential for interpreting CBC reports in hematologic diseases, they show performance heterogeneity across models. The ablation study findings underscore the necessity of integrating clinical context for accurate laboratory test interpretation. Low scores, hallucinations, and reasoning errors in model outputs indicate that current clinical deployment requires human oversight and quality control. As this single-center, Chinese-language exploratory assessment provides only preliminary, possibly context-dependent evidence, multicenter, cross-lingual prospective validation is needed to delineate the practical boundaries and safety standards for clinical deployment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/87802","type":"journal-article","created":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T17:41:54Z","timestamp":1780681314000},"page":"e87802-e87802","source":"Crossref","is-referenced-by-count":1,"title":["Performance Evaluation of GPT-5, Grok 4, and DeepSeek R1 in Interpreting Complete Blood Count Reports for Hematologic Diseases: Retrospective Comparative Study"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5988-8304","authenticated-orcid":false,"given":"Xianfei","family":"Ye","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8263-6638","authenticated-orcid":false,"given":"Xinglun","family":"Qi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3642-2848","authenticated-orcid":false,"given":"Lina","family":"Fan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0550-5861","authenticated-orcid":false,"given":"Qian","family":"Yu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9290-6090","authenticated-orcid":false,"given":"Suming","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4204-2407","authenticated-orcid":false,"given":"Chunyun","family":"Ren","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1118-6536","authenticated-orcid":false,"given":"Dagan","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,6,5]]},"reference":[{"issue":"8","key":"R1","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"issue":"11","key":"R2","doi-asserted-by":"publisher","first-page":"2186","DOI":"10.1515\/cclm-2025-0647","article-title":"Evaluating large language models as clinical laboratory test recommenders in primary and emergency care: a crucial step in clinical decision making","volume":"63","author":"Zayed","journal-title":"Clin Chem Lab Med"},{"issue":"7","key":"R3","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1515\/cclm-2023-0355","article-title":"Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. An assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI)","volume":"61","author":"Cadamuro","journal-title":"Clin Chem Lab Med"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-22315-5","article-title":"Pre-trained ChatGPT for report generation in automated microbial identification and antibiotic susceptibility testing systems","volume":"15","author":"Hu","journal-title":"Sci Rep"},{"issue":"9","key":"R5","doi-asserted-by":"publisher","first-page":"1122","DOI":"10.1093\/clinchem\/hvae093","article-title":"ChatGPT vs medical professional: analyzing responses to laboratory medicine questions on social media","volume":"70","author":"Girton","journal-title":"Clin Chem"},{"issue":"3","key":"R6","doi-asserted-by":"publisher","first-page":"327","DOI":"10.3343\/alm.2025.0200","article-title":"Evaluation of the performance of advanced large language models in laboratory medicine using residency examinations","volume":"46","author":"Jung","journal-title":"Ann Lab Med"},{"issue":"7","key":"R7","doi-asserted-by":"publisher","first-page":"1362","DOI":"10.1515\/cclm-2023-1058","article-title":"Comparison of three chatbots as an assistant for problem-solving in clinical laboratory","volume":"62","author":"Abusoglu","journal-title":"Clin Chem Lab Med"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.2147\/JMDH.S538253","article-title":"Assessing DeepSeek-R1 for clinical decision support in multidisciplinary laboratory medicine","volume":"18","author":"Li","journal-title":"J Multidiscip Healthc"},{"issue":"11","key":"R9","doi-asserted-by":"publisher","first-page":"2792","DOI":"10.7150\/ijms.111780","article-title":"Large language models in medicine: applications, challenges, and future directions","volume":"22","author":"Yu","journal-title":"Int J Med Sci"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","journal-title":"NPJ Digit Med"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","article-title":"The TRIPOD-LLM reporting guideline for studies using large language models","volume":"31","author":"Gallifant","journal-title":"Nat Med"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2024-083305","article-title":"Reporting guidelines for chatbot health advice studies: explanation and elaboration for the Chatbot Assessment Reporting Tool (CHART)","volume":"390","author":"CHART Collaborative","journal-title":"BMJ"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01766-0","article-title":"Development and validation of an interpretable risk prediction model for the early classification of thalassemia","volume":"8","author":"Lai","journal-title":"NPJ Digit Med"},{"issue":"5","key":"R14","doi-asserted-by":"publisher","first-page":"e323","DOI":"10.1016\/S2589-7500(24)00044-X","article-title":"Evaluation of a machine-learning model based on laboratory parameters for the prediction of acute leukaemia subtypes: a multicentre model development and validation study in France","volume":"6","author":"Alcazer","journal-title":"Lancet Digit Health"},{"issue":"6","key":"R15","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1111\/ijlh.14538","article-title":"Acute leukemia warning model combined CBC and CPD data based on machine learning","volume":"47","author":"Gao","journal-title":"Int J Lab Hematol"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics12010138","article-title":"Beyond the in-practice CBC: the research CBC parameters-driven machine learning predictive modeling for early differentiation among leukemias","volume":"12","author":"Haider","journal-title":"Diagnostics (Basel)"},{"issue":"5","key":"R17","doi-asserted-by":"publisher","first-page":"793","DOI":"10.1515\/cclm-2023-1037","article-title":"Machine learning-based clinical decision support using laboratory data","volume":"62","author":"\u00c7ubuk\u00e7u","journal-title":"Clin Chem Lab Med"},{"issue":"5","key":"R18","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1080\/10408363.2025.2488842","article-title":"Rigorous validation of machine learning in laboratory medicine: guidance toward quality improvement","volume":"62","author":"Miller","journal-title":"Crit Rev Clin Lab Sci"},{"issue":"12","key":"R19","doi-asserted-by":"publisher","first-page":"2425","DOI":"10.1515\/cclm-2024-0246","article-title":"Comparison of ChatGPT, Gemini, and Le Chat with physician interpretations of medical laboratory questions from an online health forum","volume":"62","author":"Meyer","journal-title":"Clin Chem Lab Med"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.2196\/56655","article-title":"Quality of answers of generative large language models versus peer users for interpreting laboratory test results for lay patients: evaluation study","volume":"26","author":"He","journal-title":"J Med Internet Res"},{"issue":"21","key":"R21","doi-asserted-by":"publisher","first-page":"1866","DOI":"10.1001\/jama.2025.6571","article-title":"DeepSeek\u2019s \u201clow-cost\u201d adoption across China\u2019s hospital systems: too fast, too soon?","volume":"333","author":"Zeng","journal-title":"JAMA"},{"issue":"7","key":"R22","doi-asserted-by":"publisher","first-page":"1703","DOI":"10.1038\/s41375-022-01613-1","article-title":"The 5th edition of the World Health Organization Classification of Haematolymphoid Tumours: myeloid and histiocytic\/dendritic neoplasms","volume":"36","author":"Khoury","journal-title":"Leukemia"},{"issue":"4","key":"R23","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1037\/1040-3590.6.4.284","article-title":"Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology","volume":"6","author":"Cicchetti","journal-title":"Psychol Assess"},{"issue":"8","key":"R24","doi-asserted-by":"publisher","first-page":"2550","DOI":"10.1038\/s41591-025-03726-3","article-title":"Comparative benchmarking of the DeepSeek large language model on medical tasks and clinical reasoning","volume":"31","author":"Tordjman","journal-title":"Nat Med"},{"issue":"8","key":"R25","doi-asserted-by":"publisher","first-page":"2546","DOI":"10.1038\/s41591-025-03727-2","article-title":"Benchmark evaluation of DeepSeek large language models in clinical decision-making","volume":"31","author":"Sandmann","journal-title":"Nat Med"},{"issue":"8","key":"R26","doi-asserted-by":"publisher","first-page":"e199","DOI":"10.1515\/cclm-2025-0055","article-title":"Performance evaluation of large language models with chain-of-thought reasoning ability in clinical laboratory case interpretation","volume":"63","author":"Yang","journal-title":"Clin Chem Lab Med"},{"issue":"7","key":"R27","doi-asserted-by":"publisher","first-page":"1131","DOI":"10.1515\/cclm-2023-0387","article-title":"ChatGPT: Angel or demond? Critical thinking is still needed","volume":"61","author":"Plebani","journal-title":"Clin Chem Lab Med"},{"issue":"8","key":"R28","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.43861","article-title":"Large language models in hematology case solving: a comparative study of ChatGPT-3.5, Google Bard, and Microsoft Bing","volume":"15","author":"Kumari","journal-title":"Cureus"},{"issue":"9","key":"R29","doi-asserted-by":"publisher","first-page":"1698","DOI":"10.1515\/cclm-2025-0089","article-title":"Evaluation of error detection and treatment recommendations in nucleic acid test reports using ChatGPT models","volume":"63","author":"Han","journal-title":"Clin Chem Lab Med"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpi.2025.100520","article-title":"Retrieval-augmented generation for interpreting clinical laboratory regulations using large language models","volume":"19","author":"Nanua","journal-title":"J Pathol Inform"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e87802","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:25:08Z","timestamp":1785536708000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e87802"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,5]]},"references-count":30,"URL":"https:\/\/doi.org\/10.2196\/87802","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,5]]},"article-number":"v28i8e87802"}}