{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T19:36:54Z","timestamp":1790797014311,"version":"4.1.0"},"reference-count":40,"publisher":"Wiley","issue":"9","license":[{"start":{"date-parts":[[2019,7,7]],"date-time":"2019-07-07T00:00:00Z","timestamp":1562457600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100007311","name":"Deutsche Kinderkrebsstiftung","doi-asserted-by":"publisher","award":["DKS2013.12"],"award-info":[{"award-number":["DKS2013.12"]}],"id":[{"id":"10.13039\/501100007311","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Cytometry Pt A"],"published-print":{"date-parts":[[2019,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Minimal residual disease (MRD) as measured by multiparameter flow cytometry (FCM) is an independent and strong prognostic factor in B\u2010cell acute lymphoblastic leukemia (B\u2010ALL). However, reliable flow cytometric detection of MRD strongly depends on operator skills and expert knowledge. Hence, an objective, automated tool for reliable FCM\u2010MRD quantification, able to overcome the technical diversity and analytical subjectivity, would be most helpful. We developed a supervised machine learning approach using a combination of multiple Gaussian Mixture Models (GMM) as a parametric density model. The approach was used for finding the weights of a linear combination of multiple GMMs to represent new, \u201cunseen\u201d samples by an interpolation of stored samples. The experimental data set contained FCM\u2010MRD data of 337 bone marrow samples collected at day 15 of induction therapy in three different laboratories from pediatric patients with B\u2010ALL for which accurate, expert\u2010set gates existed. We compared MRD quantification by our proposed GMM approach to operator assessments, its performance on data from different laboratories, as well as to other state\u2010of\u2010the\u2010art automated read\u2010out methods. Our proposed GMM\u2010combination approach proved superior over support vector machines, deep neural networks, and a single GMM approach in terms of precision and average <jats:italic>F<\/jats:italic><jats:sub>\n<jats:italic>1<\/jats:italic><\/jats:sub>\u2010scores. A high correlation of expert operator\u2010based and automated MRD assessment was achieved with reliable automated MRD quantification (<jats:italic>F<\/jats:italic><jats:sub>\n<jats:italic>1<\/jats:italic><\/jats:sub>\u2010scores &gt;0.5 in more than 95% of samples) in the clinically relevant range. Although best performance was found, if test and training samples were from the same system (i.e., flow cytometer and staining panel; lowest median <jats:italic>F<\/jats:italic><jats:sub>\n<jats:italic>1<\/jats:italic><\/jats:sub>\u2010score 0.92), cross\u2010system performance remained high with a median <jats:italic>F<\/jats:italic><jats:sub>\n<jats:italic>1<\/jats:italic><\/jats:sub>\u2010score above 0.85 in all settings. In conclusion, our proposed automated approach could potentially be used to assess FCM\u2010MRD in B\u2010ALL in an objective and standardized manner across different laboratories. \u00a9 2019 International Society for Advancement of Cytometry<\/jats:p>","DOI":"10.1002\/cyto.a.23852","type":"journal-article","created":{"date-parts":[[2019,7,8]],"date-time":"2019-07-08T06:44:30Z","timestamp":1562568270000},"page":"966-975","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["Automated Flow Cytometric MRD Assessment in Childhood Acute B\u2010 Lymphoblastic Leukemia Using Supervised Machine Learning"],"prefix":"10.1002","volume":"95","author":[{"given":"Michael","family":"Reiter","sequence":"first","affiliation":[{"name":"Immunological Diagnostics, Children's Cancer Research Institute  Vienna Austria"},{"name":"Computer Vision Lab, Faculty of Informatics Technical University of Vienna  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Diem","sequence":"additional","affiliation":[{"name":"Immunological Diagnostics, Children's Cancer Research Institute  Vienna Austria"},{"name":"Computer Vision Lab, Faculty of Informatics Technical University of Vienna  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angela","family":"Schumich","sequence":"additional","affiliation":[{"name":"Immunological Diagnostics, Children's Cancer Research Institute  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Margarita","family":"Maurer\u2010Granofszky","sequence":"additional","affiliation":[{"name":"Immunological Diagnostics, Children's Cancer Research Institute  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonid","family":"Karawajew","sequence":"additional","affiliation":[{"name":"Department of Pediatric Oncology\/Hematology Charit\u00e9 Universit\u00e4tsmedizin Berlin  Berlin Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jorge G.","family":"Rossi","sequence":"additional","affiliation":[{"name":"Cellular Immunology Laboratory Hospital de Pediatria \u201cDr. Juan P. Garrahan\u201d  Buenos Aires Argentina"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Ratei","sequence":"additional","affiliation":[{"name":"Department of Hematology, Oncology and Tumor Immunology HELIOS Klinikum Berlin\u2010Buch  Berlin Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefanie","family":"Groeneveld\u2010Krentz","sequence":"additional","affiliation":[{"name":"Department of Pediatric Oncology\/Hematology Charit\u00e9 Universit\u00e4tsmedizin Berlin  Berlin Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elisa O.","family":"Sajaroff","sequence":"additional","affiliation":[{"name":"Cellular Immunology Laboratory Hospital de Pediatria \u201cDr. Juan P. Garrahan\u201d  Buenos Aires Argentina"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susanne","family":"Suhendra","sequence":"additional","affiliation":[{"name":"Labdia Labordiagnostik GmbH  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Kampel","sequence":"additional","affiliation":[{"name":"Computer Vision Lab, Faculty of Informatics Technical University of Vienna  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael N.","family":"Dworzak","sequence":"additional","affiliation":[{"name":"Immunological Diagnostics, Children's Cancer Research Institute  Vienna Austria"},{"name":"Labdia Labordiagnostik GmbH  Vienna Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"name":"for the International Berlin\u2010Frankfurt\u2010M\u00fcnster (iBFM)\u2010FLOW\u2010network and the AutoFLOW consortium","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2019,7,7]]},"reference":[{"key":"e_1_2_7_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(08)60457-2"},{"key":"e_1_2_7_3_1","doi-asserted-by":"publisher","DOI":"10.1053\/j.seminhematol.2008.09.007"},{"key":"e_1_2_7_4_1","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2012.48.5680"},{"key":"e_1_2_7_5_1","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2008.20.8934"},{"key":"e_1_2_7_6_1","doi-asserted-by":"publisher","DOI":"10.1182\/blood-2008-01-132837"},{"key":"e_1_2_7_7_1","doi-asserted-by":"publisher","DOI":"10.1182\/asheducation-2010.1.7"},{"key":"e_1_2_7_8_1","doi-asserted-by":"publisher","DOI":"10.3324\/haematol.2011.060426"},{"key":"e_1_2_7_9_1","doi-asserted-by":"publisher","DOI":"10.1182\/blood-2016-12-725804"},{"key":"e_1_2_7_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.21101"},{"key":"e_1_2_7_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/bjh.13834"},{"key":"e_1_2_7_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.21195"},{"key":"e_1_2_7_13_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa1716863"},{"key":"e_1_2_7_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/sj.leu.2403001"},{"key":"e_1_2_7_15_1","doi-asserted-by":"publisher","DOI":"10.1182\/blood.V99.6.1952"},{"key":"e_1_2_7_16_1","doi-asserted-by":"publisher","DOI":"10.1182\/blood-2016-07-726307"},{"key":"e_1_2_7_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.20430"},{"key":"e_1_2_7_18_1","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2009.268"},{"key":"e_1_2_7_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2016.44"},{"key":"e_1_2_7_20_1","doi-asserted-by":"publisher","DOI":"10.3324\/haematol.2008.000414"},{"key":"e_1_2_7_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.21528"},{"key":"e_1_2_7_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-21444-4"},{"key":"e_1_2_7_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.20435"},{"key":"e_1_2_7_24_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.a.22626"},{"key":"e_1_2_7_25_1","doi-asserted-by":"publisher","DOI":"10.1155\/2009\/686759"},{"key":"e_1_2_7_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.a.21007"},{"key":"e_1_2_7_27_1","doi-asserted-by":"publisher","DOI":"10.1155\/2009\/247646"},{"key":"e_1_2_7_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.20554"},{"key":"e_1_2_7_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/0471722731"},{"key":"e_1_2_7_30_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.a.22446"},{"key":"e_1_2_7_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.clim.2014.12.009"},{"key":"e_1_2_7_32_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1003130"},{"key":"e_1_2_7_33_1","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2010.160"},{"key":"e_1_2_7_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2016.04.004"},{"key":"e_1_2_7_35_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-7-282"},{"key":"e_1_2_7_36_1","doi-asserted-by":"publisher","DOI":"10.1002\/cyto.b.21518"},{"key":"e_1_2_7_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-45528-0"},{"key":"e_1_2_7_38_1","doi-asserted-by":"crossref","unstructured":"KimHJ AdluruN BanerjeeM VemuriBC SinghV.Interpolation on the Manifold of K Component GMMs. Proceedings of IEEE International Conference on Computer Vision2015; p 2884\u20132892.","DOI":"10.1109\/ICCV.2015.330"},{"key":"e_1_2_7_39_1","doi-asserted-by":"publisher","DOI":"10.1182\/blood-2009-10-248146"},{"key":"e_1_2_7_40_1","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2013.354"},{"key":"e_1_2_7_41_1","doi-asserted-by":"publisher","DOI":"10.1172\/jci.insight.121867"}],"container-title":["Cytometry Part A"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cyto.a.23852","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/cyto.a.23852","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cyto.a.23852","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T17:37:05Z","timestamp":1694281025000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cyto.a.23852"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,7]]},"references-count":40,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2019,9]]}},"alternative-id":["10.1002\/cyto.a.23852"],"URL":"https:\/\/doi.org\/10.1002\/cyto.a.23852","archive":["Portico"],"relation":{},"ISSN":["1552-4922","1552-4930"],"issn-type":[{"value":"1552-4922","type":"print"},{"value":"1552-4930","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,7]]},"assertion":[{"value":"2019-03-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-05-28","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}