{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T21:51:39Z","timestamp":1757627499656,"version":"3.44.0"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T00:00:00Z","timestamp":1755561600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T00:00:00Z","timestamp":1755561600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Stanford Center for Artificial Intelligence in Medicine & Imaging (AIMI) and the Stanford Human-Centered Artificial Intelligence (HAI) Joint Partnership Grant"},{"name":"Stanford Center for Artificial Intelligence in Medicine Imaging & Center for Human Centered AI Joint Grant: AIMI-HAI-21-05"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Next-generation sequencing-based tests have advanced the field of medical diagnostics, but their novelty and cost can lead to uncertainty in clinical deployment. The Heme-STAMP is one such assay that tracks mutations in genes implicated in hematolymphoid neoplasms. Rather than limiting its clinical usage or imposing rule-based criteria, we propose leveraging machine learning to guide clinical decision-making on whether this test should be ordered. We trained a machine learning model to predict the outcome of Heme-STAMP testing using 3472 orders placed between May 2018 and September 2021 from an academic medical center and demonstrated how to integrate a custom machine learning model into a live clinical environment to obtain real-time model and physician estimates. The model predicted the results of a complex next-generation sequencing test with discriminatory power comparable to expert hematologists (AUC score: 0.77 [0.66, 0.87], 0.78 [0.68, 0.86] respectively) and with the capacity to improve the calibration of human estimates.<\/jats:p>","DOI":"10.1038\/s41746-025-01816-7","type":"journal-article","created":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T12:16:35Z","timestamp":1755605795000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Real time machine learning prediction of next generation sequencing test results in live clinical settings"],"prefix":"10.1038","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2674-6682","authenticated-orcid":false,"given":"Grace Y. E.","family":"Kim","sequence":"first","affiliation":[]},{"given":"Matthew","family":"Schwede","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1293-9053","authenticated-orcid":false,"given":"Conor K.","family":"Corbin","sequence":"additional","affiliation":[]},{"given":"Sajjad","family":"Fouladvand","sequence":"additional","affiliation":[]},{"given":"Rondeep","family":"Brar","sequence":"additional","affiliation":[]},{"given":"David","family":"Iberri","sequence":"additional","affiliation":[]},{"given":"William","family":"Shomali","sequence":"additional","affiliation":[]},{"given":"Jean S.","family":"Oak","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9182-8123","authenticated-orcid":false,"given":"Dita","family":"Gratzinger","sequence":"additional","affiliation":[]},{"given":"Henning","family":"Stehr","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4387-8740","authenticated-orcid":false,"given":"Jonathan H.","family":"Chen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,19]]},"reference":[{"key":"1816_CR1","doi-asserted-by":"publisher","first-page":"868","DOI":"10.3324\/haematol.2018.205955","volume":"104","author":"RL Levine","year":"2019","unstructured":"Levine, R. L. & Valk, P. J. M. Next-generation sequencing in the diagnosis and minimal residual disease assessment of acute myeloid leukemia. Haematologica 104, 868\u2013871 (2019).","journal-title":"Haematologica"},{"key":"1816_CR2","doi-asserted-by":"publisher","first-page":"2386","DOI":"10.1056\/NEJMoa1716984","volume":"378","author":"CD DiNardo","year":"2018","unstructured":"DiNardo, C. D. et al. Durable remissions with Ivosidenib in IDH1-mutated relapsed or refractory AML. N. Engl. J. Med. 378, 2386\u20132398 (2018).","journal-title":"N. Engl. J. Med."},{"key":"1816_CR3","doi-asserted-by":"publisher","first-page":"1200","DOI":"10.1182\/blood.2022015850","volume":"140","author":"DA Arber","year":"2022","unstructured":"Arber, D. A. et al. International Consensus Classification of Myeloid Neoplasms and Acute Leukemias: integrating morphologic, clinical, and genomic data. Blood 140, 1200\u20131228 (2022).","journal-title":"Blood"},{"key":"1816_CR4","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1093\/ajcp\/aqac045","volume":"158","author":"F Fei","year":"2022","unstructured":"Fei, F., Natkunam, Y., Zehnder, J. L., Stehr, H. & Gratzinger, D. Diagnostic impact of next-generation sequencing panels for lymphoproliferative neoplasms on small-volume biopsies. Am. J. Clin. Pathol. 158, 345\u2013361 (2022).","journal-title":"Am. J. Clin. Pathol."},{"key":"1816_CR5","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1182\/blood.2022016867","volume":"140","author":"H D\u00f6hner","year":"2022","unstructured":"D\u00f6hner, H. et al. Diagnosis and management of AML in adults: 2022 recommendations from an international expert panel on behalf of the ELN. Blood 140, 1345\u20131377 (2022).","journal-title":"Blood"},{"key":"1816_CR6","doi-asserted-by":"publisher","first-page":"2107","DOI":"10.1056\/NEJMp2210638","volume":"387","author":"RL Schilsky","year":"2022","unstructured":"Schilsky, R. L. & Longo, D. L. Closing the gap in cancer genomic testing. N. Engl. J. Med. 387, 2107\u20132110 (2022).","journal-title":"N. Engl. J. Med."},{"key":"1816_CR7","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1056\/NEJMoa1701719","volume":"377","author":"S Jaiswal","year":"2017","unstructured":"Jaiswal, S. et al. Clonal hematopoiesis and risk of atherosclerotic cardiovascular disease. N. Engl. J. Med. 377, 111\u2013121 (2017).","journal-title":"N. Engl. J. Med."},{"key":"1816_CR8","doi-asserted-by":"publisher","first-page":"1109","DOI":"10.1056\/NEJM198005153022003","volume":"302","author":"SG Pauker","year":"1980","unstructured":"Pauker, S. G. & Kassirer, J. P. The threshold approach to clinical decision making. N. Engl. J. Med. 302, 1109\u20131117 (1980).","journal-title":"N. Engl. J. Med."},{"key":"1816_CR9","first-page":"641","volume":"2021","author":"GYE Kim","year":"2021","unstructured":"Kim, G. Y. E. et al. Machine learning predictability of clinical next generation sequencing for hematologic malignancies to guide high-value precision medicine. AMIA Annu. Symp. Proc. 2021, 641\u2013650 (2021).","journal-title":"AMIA Annu. Symp. Proc."},{"key":"1816_CR10","doi-asserted-by":"publisher","first-page":"e1910967","DOI":"10.1001\/jamanetworkopen.2019.10967","volume":"2","author":"S Xu","year":"2019","unstructured":"Xu, S. et al. Prevalence and predictability of low-yield inpatient laboratory diagnostic tests. JAMA Netw. Open 2, e1910967 (2019).","journal-title":"JAMA Netw. Open"},{"key":"1816_CR11","doi-asserted-by":"publisher","first-page":"100706","DOI":"10.1016\/j.blre.2020.100706","volume":"42","author":"RM Shallis","year":"2020","unstructured":"Shallis, R. M. et al. Epidemiology of the classical myeloproliferative neoplasms: The four corners of an expansive and complex map. Blood Rev. 42, 100706 (2020).","journal-title":"Blood Rev."},{"key":"1816_CR12","doi-asserted-by":"crossref","unstructured":"Jansko-Gadermeir, B. et al. Myeloid NGS analyses of paired samples from bone marrow and peripheral blood yield concordant results: a prospective cohort analysis of the AGMT Study Group. Cancers 15, 2305 (2023).","DOI":"10.3390\/cancers15082305"},{"key":"1816_CR13","doi-asserted-by":"publisher","first-page":"2488","DOI":"10.1056\/NEJMoa1408617","volume":"371","author":"S Jaiswal","year":"2014","unstructured":"Jaiswal, S. et al. Age-related clonal hematopoiesis associated with adverse outcomes. N. Engl. J. Med. 371, 2488\u20132498 (2014).","journal-title":"N. Engl. J. Med."},{"key":"1816_CR14","doi-asserted-by":"crossref","unstructured":"Corbin, C. K. et al. DEPLOYR: A technical framework for deploying custom real-time machine learning models into the electronic medical record. J Am Med Inform Assoc. 30, 1532\u20131542 (2023).","DOI":"10.1093\/jamia\/ocad114"},{"key":"1816_CR15","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/j.clinbiochem.2023.01.002","volume":"113","author":"N Rabbani","year":"2023","unstructured":"Rabbani, N. et al. Targeting repetitive laboratory testing with electronic health records-embedded predictive decision support: A pre-implementation study. Clin. Biochem. 113, 70\u201377 (2023).","journal-title":"Clin. Biochem."},{"key":"1816_CR16","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1038\/s41746-021-00426-3","volume":"4","author":"BK Beaulieu-Jones","year":"2021","unstructured":"Beaulieu-Jones, B. K. et al. Machine learning for patient risk stratification: standing on, or looking over, the shoulders of clinicians?. NPJ Digit Med 4, 62 (2021).","journal-title":"NPJ Digit Med"},{"key":"1816_CR17","doi-asserted-by":"publisher","first-page":"866","DOI":"10.7326\/M18-1990","volume":"169","author":"A Rajkomar","year":"2018","unstructured":"Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G. & Chin, M. H. Ensuring Fairness in Machine Learning to Advance Health Equity. Ann. Intern. Med. 169, 866\u2013872 (2018).","journal-title":"Ann. Intern. Med."},{"key":"1816_CR18","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1182\/bloodadvances.2022007679","volume":"7","author":"LE Merz","year":"2023","unstructured":"Merz, L. E. et al. Absolute neutrophil count by Duffy status among healthy Black and African American adults. Blood Adv. 7, 317\u2013320 (2023).","journal-title":"Blood Adv."},{"key":"1816_CR19","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1038\/s41375-020-0935-5","volume":"35","author":"A De-Morgan","year":"2021","unstructured":"De-Morgan, A., Meggendorfer, M., Haferlach, C. & Shlush, L. Male predominance in AML is associated with specific preleukemic mutations. Leukemia 35, 867\u2013870 (2021).","journal-title":"Leukemia"},{"key":"1816_CR20","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1002\/ajh.25486","volume":"94","author":"AE Quesada","year":"2019","unstructured":"Quesada, A. E. et al. DDX41 mutations in myeloid neoplasms are associated with male gender, TP53 mutations and high-risk disease. Am. J. Hematol. 94, 757\u2013766 (2019).","journal-title":"Am. J. Hematol."},{"key":"1816_CR21","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-12-77","volume":"12","author":"X Robin","year":"2011","unstructured":"Robin, X. et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinforma. 12, 77 (2011).","journal-title":"BMC Bioinforma."},{"key":"1816_CR22","first-page":"391","volume":"2009","author":"HJ Lowe","year":"2009","unstructured":"Lowe, H. J., Ferris, T. A., Hernandez, P. M. & Weber, S. C. STRIDE-An integrated standards-based translational research informatics platform. AMIA Annu. Symp. Proc. 2009, 391\u2013395 (2009).","journal-title":"AMIA Annu. Symp. Proc."},{"key":"1816_CR23","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Random Forests. Mach. Learn. 45, 5\u201332 (2001).","journal-title":"Mach. Learn."},{"key":"1816_CR24","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","volume":"1","author":"A Rajkomar","year":"2018","unstructured":"Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med 1, 18 (2018).","journal-title":"NPJ Digit Med"},{"key":"1816_CR25","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1016\/j.jbi.2015.10.006","volume":"58","author":"K Jung","year":"2015","unstructured":"Jung, K. & Shah, N. H. Implications of non-stationarity on predictive modeling using EHRs. J. Biomed. Inform. 58, 168\u2013174 (2015).","journal-title":"J. Biomed. Inform."},{"key":"1816_CR26","unstructured":"Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst. 30, (2017)."},{"key":"1816_CR27","doi-asserted-by":"crossref","unstructured":"Kendall, M. G. A new measure of rank correlation. Biometrika. 30, 81\u201393 (1948).","DOI":"10.1093\/biomet\/30.1-2.81"},{"key":"1816_CR28","unstructured":"Summary - FHIR v5.0.0. https:\/\/www.hl7.org\/fhir\/summary.html."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01816-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01816-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01816-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T03:59:25Z","timestamp":1757476765000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01816-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,19]]},"references-count":28,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1816"],"URL":"https:\/\/doi.org\/10.1038\/s41746-025-01816-7","relation":{},"ISSN":["2398-6352"],"issn-type":[{"type":"electronic","value":"2398-6352"}],"subject":[],"published":{"date-parts":[[2025,8,19]]},"assertion":[{"value":"12 February 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare the following competing interests. JHC co-founded Reaction Explorer LLC, which develops and licenses organic chemistry education software, and receives consultation payments from the National Institute of Drug Abuse Clinical Trials Network, Tuolc Inc., Roche Inc., and Younker Hyde MacFarlane PLLC. CKC is employed by SmarterDx, a health AI company. This employment does not directly pertain to the findings in this manuscript. All other authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"533"}}