{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T11:08:43Z","timestamp":1793444923592,"version":"4.5.0"},"reference-count":27,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T00:00:00Z","timestamp":1635724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"Roger Moe Fellowship"},{"name":"GPU"},{"DOI":"10.13039\/100007065","name":"NVIDIA","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007065","id-type":"DOI","asserted-by":"crossref"}]},{"name":"University of Washington Department of Laboratory Medicine and Pathology","id":[{"id":"https:\/\/ror.org\/00cvxb145","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,3,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>We desired an automated approach to expedite ordering additional antibody panels in our clinical flow cytometry lab. This addition could improve turnaround times, decrease time spent revisiting cases, and improve consistency.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We trained a machine learning classifier to use our screening B-cell panel to predict whether we should order an additional panel to distinguish chronic lymphocytic lymphoma from mantle cell lymphoma. We used data from 2016 to 2018 for training and validation, and cases were restricted to the first case per patient (9,635 cases, 887 with the additional panel). We applied the model in real time over approximately 2.5 months in 2020 to 376 sequential cases, with automated email notifications for positive predictions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Using 80% of the data from 2016 to 2018 to train and 20% for validation, we achieved 95% area under the receiving operating characteristic curve (AUROC) and 94% accuracy in the validation set. Applying the classifier in real time achieved 89% AUROC and 94% real-time prediction accuracy (precision [positive predictive value] = 51%, recall [sensitivity] = 78%, and F1 score = 0.62). Fourteen of the 17 false positives had prior diagnoses to which the algorithm was not privy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>As an observational, not interventional study, our system performed well on testing within our laboratory for identifying cases to be flagged but cannot be used without laboratory-specific modifications.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/ajcp\/aqab166","type":"journal-article","created":{"date-parts":[[2021,9,8]],"date-time":"2021-09-08T07:21:22Z","timestamp":1631085682000},"page":"443-450","source":"Crossref","is-referenced-by-count":15,"title":["Potential for Process Improvement of Clinical Flow Cytometry by Incorporating Real-Time Automated Screening of Data to Expedite Addition of Antibody Panels"],"prefix":"10.1093","volume":"157","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6237-6861","authenticated-orcid":false,"given":"Paul D","family":"Simonson","sequence":"first","affiliation":[{"name":"Department of Laboratory Medicine and Pathology, University of Washington , Seattle , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron Y","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of Washington , Seattle , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Laboratory Medicine and Pathology, University of Washington , Seattle , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,11,1]]},"reference":[{"key":"2023020707510580500_CIT0001","doi-asserted-by":"crossref","first-page":"S14","DOI":"10.1002\/cyto.b.20363","article-title":"2006 Bethesda International Consensus recommendations on the immunophenotypic analysis of hematolymphoid neoplasia by flow cytometry: optimal reagents and reporting for the flow cytometric diagnosis of hematopoietic neoplasia","volume":"72(suppl 1)","author":"Wood","year":"2007","journal-title":"Cytometry B Clin Cytom."},{"key":"2023020707510580500_CIT0002","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1038\/s41379-021-00826-6","article-title":"Artificial intelligence for advance requesting of immunohistochemistry in diagnostically uncertain prostate biopsies","volume":"34","author":"Chatrian","year":"2021","journal-title":"Mod Pathol."},{"key":"2023020707510580500_CIT0003","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1097\/CCO.0000000000000607","article-title":"Computational analysis of flow cytometry data in hematological malignancies: future clinical practice?","volume":"32","author":"Duetz","year":"2020","journal-title":"Curr Opin Oncol."},{"key":"2023020707510580500_CIT0004","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.ebiom.2018.10.042","article-title":"Clinically validated machine learning algorithm for detecting residual diseases with multicolor flow cytometry analysis in acute myeloid leukemia and myelodysplastic syndrome","volume":"37","author":"Ko","year":"2018","journal-title":"EBioMedicine."},{"key":"2023020707510580500_CIT0005","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1093\/ajcp\/aqaa166","article-title":"Augmented human intelligence and automated diagnosis in flow cytometry for hematologic malignancies","volume":"155","author":"Ng","year":"2020","journal-title":"Am J Clin Pathol."},{"key":"2023020707510580500_CIT0006","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1309\/AJCPY8E2LYHCGUFP","article-title":"Computer-aided detection of rare tumor populations in flow cytometry: an example with classic Hodgkin lymphoma","volume":"144","author":"Ng","year":"2015","journal-title":"Am J Clin Pathol."},{"key":"2023020707510580500_CIT0007","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1038\/nmeth.2365","article-title":"Critical assessment of automated flow cytometry data analysis techniques","volume":"10","author":"Aghaeepour","year":"2013","journal-title":"Nat Methods."},{"key":"2023020707510580500_CIT0008","article-title":"De novo identification and visualization of important cell populations for classic Hodgkin lymphoma using flow cytometry and machine learning [published online ahead of print June 27, 2021]","author":"Simonson","journal-title":"Am J Clin Pathol"},{"key":"2023020707510580500_CIT0009","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1002\/cyto.a.24159","article-title":"Hematologist-level classification of mature B-cell neoplasm using deep learning on multiparameter flow cytometry data","volume":"97","author":"Zhao","year":"2020","journal-title":"Cytometry A."},{"key":"2023020707510580500_CIT0010","doi-asserted-by":"crossref","first-page":"861","DOI":"10.21105\/joss.00861","article-title":"UMAP: Uniform Manifold Approximation and Projection","volume":"3","author":"McInnes","year":"2018","journal-title":"J Open Source Softw."},{"key":"2023020707510580500_CIT0011","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1038\/nbt.4314","article-title":"Dimensionality reduction for visualizing single-cell data using UMAP","volume":"37","author":"Becht","year":"2019","journal-title":"Nat Biotechnol."},{"key":"2023020707510580500_CIT0012","article-title":"UMAP: Uniform Manifold Approximation and Projection for dimension reduction","author":"McInnes"},{"key":"2023020707510580500_CIT0013","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MCSE.2007.58","article-title":"Python for scientific computing","volume":"9","author":"Oliphant","year":"2007","journal-title":"Comput Sci Eng."},{"key":"2023020707510580500_CIT0014","author":"Yurtsev"},{"key":"2023020707510580500_CIT0015","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res."},{"key":"2023020707510580500_CIT0016","first-page":"265","article-title":"TensorFlow: a system for large-scale machine learning","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation","author":"Abadi","year":"2016"},{"key":"2023020707510580500_CIT0017","article-title":"Deep Learning With Python.","author":"Chollet","journal-title":"Manning Publications; 2017"},{"key":"2023020707510580500_CIT0018","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: a 2D graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Comput Sci Eng."},{"key":"2023020707510580500_CIT0019","author":"Waskom"},{"key":"2023020707510580500_CIT0020","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1109\/MCSE.2011.37","article-title":"The NumPy array: a structure for efficient numerical computation","volume":"13","author":"van der Walt","year":"2011","journal-title":"Comput Sci Eng."},{"key":"2023020707510580500_CIT0021","author":"Reback"},{"key":"2023020707510580500_CIT0022","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat Biomed Eng."},{"key":"2023020707510580500_CIT0023","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1038\/s42256-019-0138-9","article-title":"From local explanations to global understanding with explainable AI for trees","volume":"2","author":"Lundberg","year":"2020","journal-title":"Nat Mach Intell."},{"key":"2023020707510580500_CIT0024","article-title":"A unified approach to interpreting model predictions","author":"Lundberg"},{"key":"2023020707510580500_CIT0025","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1002\/cyto.a.20825","article-title":"Data file standard for flow cytometry, version FCS 3.1","volume":"77","author":"Spidlen","year":"2010","journal-title":"Cytometry A."},{"key":"2023020707510580500_CIT0026"},{"key":"2023020707510580500_CIT0027","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1093\/bioinformatics\/btw570","article-title":"Precrec: fast and accurate precision-recall and ROC curve calculations in R","volume":"33","author":"Saito","year":"2017","journal-title":"Bioinformatics."}],"container-title":["American Journal of Clinical Pathology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ajcp\/article-pdf\/157\/3\/443\/49113528\/aqab166.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ajcp\/article-pdf\/157\/3\/443\/49113528\/aqab166.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,8]],"date-time":"2023-11-08T09:23:43Z","timestamp":1699435423000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ajcp\/article\/157\/3\/443\/6415167"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,1]]},"references-count":27,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,11,1]]},"published-print":{"date-parts":[[2022,3,3]]}},"URL":"https:\/\/doi.org\/10.1093\/ajcp\/aqab166","relation":{},"ISSN":["0002-9173","1943-7722"],"issn-type":[{"value":"0002-9173","type":"print"},{"value":"1943-7722","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,3,1]]},"published":{"date-parts":[[2021,11,1]]}}}