{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T11:08:44Z","timestamp":1793444924826,"version":"4.5.0"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2021,6,27]],"date-time":"2021-06-27T00:00:00Z","timestamp":1624752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000053","name":"National Eye Institute","doi-asserted-by":"publisher","award":["K23EY029246"],"award-info":[{"award-number":["K23EY029246"]}],"id":[{"id":"10.13039\/100000053","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100001818","name":"Research to Prevent Blindness","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100001818","id-type":"DOI","asserted-by":"publisher"}]},{"name":"University of Washington Department of Laboratory Medicine and Pathology"},{"name":"Roger Moe Fellowship"},{"name":"NVIDIA Corporation"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Automated classification of flow cytometry data has the potential to reduce errors and accelerate flow cytometry interpretation. We desired a machine learning approach that is accurate, is intuitively easy to understand, and highlights the cells that are most important in the algorithm\u2019s prediction for a given case.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We developed an ensemble of convolutional neural networks for classification and visualization of impactful cell populations in detecting classic Hodgkin lymphoma using two-dimensional (2D) histograms. Data from 977 and 245 clinical flow cytometry cases were used for training and testing, respectively. Seventy-eight nongated 2D histograms were created per flow cytometry file. Shapley additive explanation (SHAP) values were calculated to determine the most impactful 2D histograms and regions within histograms. SHAP values from all 78 histograms were then projected back to the original cell data for gating and visualization using standard flow cytometry software.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The algorithm achieved 67.7% recall (sensitivity), 82.4% precision, and 0.92 area under the receiver operating characteristic. Visualization of the important cell populations for individual predictions demonstrated correlations with known biology.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The method presented enables model explainability while highlighting important cell populations in individual flow cytometry specimens, with potential applications in both diagnosis and discovery of previously overlooked key cell populations.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/ajcp\/aqab076","type":"journal-article","created":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T15:16:30Z","timestamp":1617808590000},"page":"1092-1102","source":"Crossref","is-referenced-by-count":22,"title":["De Novo Identification and Visualization of Important Cell Populations for Classic Hodgkin Lymphoma Using Flow Cytometry and Machine Learning"],"prefix":"10.1093","volume":"156","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6237-6861","authenticated-orcid":false,"given":"Paul D","family":"Simonson","sequence":"first","affiliation":[{"name":"Department of Pathology and Laboratory Medicine, Weill Cornell Medicine , New York, NY , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of Washington , Seattle, WA , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Laboratory Medicine and Pathology, University of Washington , Seattle, WA , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan R","family":"Fromm","sequence":"additional","affiliation":[{"name":"Department of Laboratory Medicine and Pathology, University of Washington , Seattle, WA , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron Y","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, University of Washington , Seattle, WA , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,6,27]]},"reference":[{"key":"2023020912513763900_CIT0001","volume-title":"WHO Classification of Tumours of Haematopoietic and Lymphoid Tissues.","author":"Campo","year":"2017"},{"key":"2023020912513763900_CIT0002","doi-asserted-by":"crossref","first-page":"7","DOI":"10.3322\/caac.21387","article-title":"Cancer statistics, 2017","volume":"67","author":"Siegel","year":"2017","journal-title":"CA Cancer J Clin."},{"key":"2023020912513763900_CIT0003","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1002\/1097-0142(20010225)93:1<52::AID-CNCR9007>3.0.CO;2-3","article-title":"Fine-needle aspiration cytology of Hodgkin disease: a study of 89 cases with emphasis on false-negative cases","volume":"93","author":"Chhieng","year":"2001","journal-title":"Cancer."},{"key":"2023020912513763900_CIT0004","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.ypat.2011.11.026","article-title":"Combined core needle biopsy and fine-needle aspiration with ancillary studies correlate highly with traditional techniques in the diagnosis of nodal-based lymphoma","volume":"2012","author":"Raab","year":"2012","journal-title":"Yearbook Pathol Lab Med"},{"key":"2023020912513763900_CIT0005","doi-asserted-by":"crossref","first-page":"764","DOI":"10.1309\/7371XK6F6P7474XX","article-title":"Identification and purification of classical Hodgkin cells from lymph nodes by flow cytometry and flow cytometric cell sorting","volume":"126","author":"Fromm","year":"2006","journal-title":"Am J Clin Pathol."},{"key":"2023020912513763900_CIT0006","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1309\/AJCPW3UN9DYLDSPB","article-title":"Flow cytometry can diagnose classical Hodgkin lymphoma in lymph nodes with high sensitivity and specificity","volume":"131","author":"Fromm","year":"2009","journal-title":"Am J Clin Pathol."},{"key":"2023020912513763900_CIT0007","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1309\/AJCP0Q1SVOXBHMAM","article-title":"A six-color flow cytometry assay for immunophenotyping classical Hodgkin lymphoma in lymph nodes","volume":"141","author":"Fromm","year":"2014","journal-title":"Am J Clin Pathol."},{"key":"2023020912513763900_CIT0008","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1309\/AJCPY8E2LYHCGUFP","article-title":"Computer-aided detection of rare tumor populations in flow cytometry: an example with classic Hodgkin lymphoma","volume":"144","author":"Ng","year":"2015","journal-title":"Am J Clin Pathol."},{"key":"2023020912513763900_CIT0009","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1038\/nmeth.2365","article-title":"Critical assessment of automated flow cytometry data analysis techniques","volume":"10","author":"Aghaeepour","year":"2013","journal-title":"Nat Methods."},{"key":"2023020912513763900_CIT0010","doi-asserted-by":"crossref","article-title":"\u201cWhy should I trust you?\u201d: explaining the predictions of any classifier","author":"Ribeiro","DOI":"10.1145\/2939672.2939778"},{"key":"2023020912513763900_CIT0011","author":"Lundberg","year":"2017"},{"key":"2023020912513763900_CIT0012","article-title":"Explainable AI for trees: from local explanations to global understanding","author":"Lundberg","year":"2019"},{"key":"2023020912513763900_CIT0013","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat Biomed Eng."},{"key":"2023020912513763900_CIT0014","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1007\/s10115-013-0679-x","article-title":"Explaining prediction models and individual predictions with feature contributions","volume":"41","author":"\u0160trumbelj","year":"2014","journal-title":"Knowl Inf Syst"},{"key":"2023020912513763900_CIT0015","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MCSE.2007.58","article-title":"Python for scientific computing","volume":"9","author":"Oliphant","year":"2007","journal-title":"Comput Sci Eng"},{"key":"2023020912513763900_CIT0016","author":"Yurtsev"},{"key":"2023020912513763900_CIT0017","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1145\/2786984.2786995","article-title":"Scikit-learn","volume":"19","author":"Varoquaux","year":"2015","journal-title":"GetMobile."},{"key":"2023020912513763900_CIT0018","first-page":"265","article-title":"TensorFlow: a system for large-scale machine learning.","author":"Abadi","year":"2016"},{"key":"2023020912513763900_CIT0019","volume-title":"Deep Learning With Python.","author":"Chollet","year":"2018"},{"key":"2023020912513763900_CIT0020","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: a 2D graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Comput Sci Eng"},{"key":"2023020912513763900_CIT0021","author":"Waskom","year":"2020"},{"key":"2023020912513763900_CIT0022","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1109\/MCSE.2011.37","article-title":"The NumPy array: a structure for efficient numerical computation","volume":"13","author":"van der Walt","year":"2011","journal-title":"Comput Sci Eng."},{"key":"2023020912513763900_CIT0023","author":"Reback","year":"2020"},{"key":"2023020912513763900_CIT0024","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res."},{"key":"2023020912513763900_CIT0025","doi-asserted-by":"crossref","first-page":"23","DOI":"10.11120\/msor.2001.01010023","article-title":"The R project in statistical computing","volume":"1","author":"Ripley","year":"2001","journal-title":"MSOR Connect."},{"key":"2023020912513763900_CIT0026"},{"key":"2023020912513763900_CIT0027","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1093\/bioinformatics\/btw570","article-title":"Precrec: fast and accurate precision-recall and ROC curve calculations in R","volume":"33","author":"Saito","year":"2017","journal-title":"Bioinformatics."},{"key":"2023020912513763900_CIT0028","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1002\/cyto.a.20825","article-title":"Data file standard for flow cytometry, version FCS 3.1","volume":"77","author":"Spidlen","year":"2010","journal-title":"Cytometry A."},{"key":"2023020912513763900_CIT0029","doi-asserted-by":"crossref","DOI":"10.1093\/ajcp\/aqaa166","article-title":"Augmented human intelligence and automated diagnosis in flow cytometry for hematologic malignancies","author":"Ng","year":"2021;155:597-605","journal-title":"Am J Clin Pathol."},{"key":"2023020912513763900_CIT0030","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.ebiom.2018.10.042","article-title":"Clinically validated machine learning algorithm for detecting residual diseases with multicolor flow cytometry analysis in acute myeloid leukemia and myelodysplastic syndrome","volume":"37","author":"Ko","year":"2018","journal-title":"Ebiomedicine."},{"key":"2023020912513763900_CIT0031","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1002\/cyto.a.24159","article-title":"Hematologist-level classification of mature B-cell neoplasm using deep learning on multiparameter flow cytometry data","volume":"97","author":"Zhao","year":"2020","journal-title":". Cytometry A."},{"key":"2023020912513763900_CIT0032","doi-asserted-by":"crossref","DOI":"10.1097\/CCO.0000000000000607","article-title":"Computational analysis of flow cytometry data in hematological malignancies: future clinical practice?","author":"Duetz","year":"2020;32:162-169","journal-title":"Curr Opin Oncol."},{"key":"2023020912513763900_CIT0033","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1002\/cyto.b.20459","article-title":"Overexpression of CD7 in classical Hodgkin lymphoma-infiltrating T lymphocytes","volume":"76","author":"Seegmiller","year":"2009","journal-title":"Cytometry B Clin Cytom."},{"key":"2023020912513763900_CIT0034","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/cyto.b.20376","article-title":"Comparative flow immunophenotypic features of the inflammatory infiltrates of Hodgkin lymphoma and lymphoid hyperplasia","volume":"74","author":"Hudnall","year":"2008","journal-title":"Cytometry B Clin Cytom."},{"key":"2023020912513763900_CIT0035","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1002\/cyto.b.20407","article-title":"Detection of T-regulatory cells has a potential role in the diagnosis of classical Hodgkin lymphoma","volume":"74","author":"Bosler","year":"2008","journal-title":"Cytometry B Clin Cytom."},{"key":"2023020912513763900_CIT0036","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1002\/cyto.b.20535","article-title":"Increased expression of T cell antigens on T cells in classical Hodgkin lymphoma","volume":"78","author":"Fromm","year":"2010","journal-title":"Cytometry B Clin Cytom."}],"container-title":["American Journal of Clinical Pathology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ajcp\/article-pdf\/156\/6\/1092\/49137438\/aqab076.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ajcp\/article-pdf\/156\/6\/1092\/49137438\/aqab076.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,1]],"date-time":"2023-11-01T16:10:44Z","timestamp":1698855044000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ajcp\/article\/156\/6\/1092\/6310263"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,27]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6,27]]},"published-print":{"date-parts":[[2021,11,8]]}},"URL":"https:\/\/doi.org\/10.1093\/ajcp\/aqab076","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.12.18.20248526","asserted-by":"object"}]},"ISSN":["0002-9173","1943-7722"],"issn-type":[{"value":"0002-9173","type":"print"},{"value":"1943-7722","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,12,1]]},"published":{"date-parts":[[2021,6,27]]}}}