{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T01:05:52Z","timestamp":1784768752994,"version":"3.55.0"},"reference-count":53,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"9","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>Recent advances in machine learning have led to the development of classifiers that predict molecular subtypes of acute lymphoblastic leukemia (ALL) using RNA-sequencing (RNA-seq) data. Although these models have shown promising results, they often lack robust performance guarantees. The aim of this study was three-fold: to quantify the uncertainty of these classifiers, to provide prediction sets that control the false-negative rate (FNR), and to perform implicit error reduction by transforming incorrect predictions into uncertain predictions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS<\/jats:title>\n                    <jats:p>Conformal prediction (CP) is a distribution-agnostic framework for generating statistically calibrated prediction sets whose size reflects model uncertainty. In this study, we applied an extension called conformal risk control to three RNA-seq ALL subtype classifiers. Leveraging RNA-seq data from 1,227 patient samples taken at diagnosis, we developed a multiclass conformal predictor ALLCoP, which generates statistically guaranteed FNR-controlled prediction sets.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>\n                      ALLCoP was able to create prediction sets with specified FNR tolerances ranging from 7.5% to 30%. In a validation cohort, ALLCoP successfully reduced the FNR of the ALLIUM RNA-seq ALL subtype classifier from 8.95% to 3.5%. For patients whose subtype was not previously known, the use of ALLCoP was able to reduce the occurrence of empty predictions from 37% to 17%. Notably, up to 34% of the multiple-class prediction sets included the\n                      <jats:italic toggle=\"yes\">PAX5<\/jats:italic>\n                      alt subtype, suggesting that increased prediction set size may reflect secondary aberrations and biological complexity, contributing to classifier uncertainty. Finally, ALLCoP was validated on two additional RNA-seq ALL subtype classifiers, ALLSorts and ALLCatchR.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>Our results highlight the potential of CP in enhancing the use of oncologic RNA-seq subtyping classifiers and also in uncovering additional molecular aberrations of potential clinical importance.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-24-00324","type":"journal-article","created":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T17:09:20Z","timestamp":1748452160000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":2,"title":["Error Reduction in Leukemia Machine Learning Classification With Conformal Prediction"],"prefix":"10.1200","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0012-2310","authenticated-orcid":true,"given":"Mariya","family":"Lysenkova Wiklander","sequence":"first","affiliation":[{"name":"Department of Medical Sciences, Uppsala University, Uppsala, Sweden"},{"name":"SciLifeLab, Uppsala University, Uppsala, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6698-0166","authenticated-orcid":true,"given":"Dave","family":"Zachariah","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Uppsala University, Uppsala, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6436-9531","authenticated-orcid":true,"given":"Olga","family":"Krali","sequence":"additional","affiliation":[{"name":"Department of Medical Sciences, Uppsala University, Uppsala, Sweden"},{"name":"SciLifeLab, Uppsala University, Uppsala, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8699-9959","authenticated-orcid":true,"given":"Jessica","family":"Nordlund","sequence":"additional","affiliation":[{"name":"Department of Medical Sciences, Uppsala University, Uppsala, Sweden"},{"name":"SciLifeLab, Uppsala University, Uppsala, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_8_2_2","doi-asserted-by":"publisher","DOI":"10.1158\/2159-8290.CD-21-0090"},{"key":"e_1_3_8_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2023.01.035"},{"key":"e_1_3_8_4_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41698-023-00479-5"},{"key":"e_1_3_8_5_2","doi-asserted-by":"publisher","DOI":"10.1182\/bloodadvances.2021005894"},{"key":"e_1_3_8_6_2","doi-asserted-by":"publisher","DOI":"10.1097\/HS9.0000000000000939"},{"key":"e_1_3_8_7_2","first-page":"1741","article-title":"MD-ALL: An integrative platform for molecular diagnosis of B-acute lymphoblastic leukemia","volume":"109","author":"Hu Z","year":"2024","unstructured":"Hu Z, Jia Z, Liu J, et al: MD-ALL: An integrative platform for molecular diagnosis of B-acute lymphoblastic leukemia. Haematologica 109:1741-1754, 2024","journal-title":"Haematologica"},{"key":"e_1_3_8_8_2","doi-asserted-by":"publisher","DOI":"10.1182\/bloodadvances.2023010385"},{"key":"e_1_3_8_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00428-022-03448-8"},{"key":"e_1_3_8_10_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2016.70.7836"},{"key":"e_1_3_8_11_2","doi-asserted-by":"publisher","unstructured":"Wadensten E Wessman S Abel F et al: Diagnostic yield from a nationwide implementation of precision medicine for all children with cancer. JCO Precis Oncol 10.1200\/PO.23.00039","DOI":"10.1200\/PO.23.00039"},{"key":"e_1_3_8_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmoldx.2023.09.013"},{"key":"e_1_3_8_13_2","doi-asserted-by":"publisher","DOI":"10.1017\/pcm.2023.3"},{"key":"e_1_3_8_14_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-019-1426-2"},{"key":"e_1_3_8_15_2","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-018-03067-x"},{"key":"e_1_3_8_16_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02562-7"},{"key":"e_1_3_8_17_2","volume-title":"Uncertainty in artificial intelligence: proceedings of the fourteenth conference (1998), July 24-26, 1998, University of Wisconsin, Madison, WI","author":"Gammerman A","year":"1998","unstructured":"Gammerman A, Vovk V, Vapnik V: Learning by transduction, in Cooper GF (ed): Uncertainty in artificial intelligence: proceedings of the fourteenth conference (1998), July 24-26, 1998, University of Wisconsin, Madison, WI. San Francisco, CA, Morgan Kaufmann, 1998"},{"key":"e_1_3_8_18_2","volume-title":"Algorithmic Learning in a Random World","author":"Vovk V","year":"2005","unstructured":"Vovk V, Gammerman A, Shafer G: Algorithmic Learning in a Random World. New York, NY, Springer-Verlag, 2005. http:\/\/link.springer.com\/10.1007\/b106715"},{"key":"e_1_3_8_19_2","unstructured":"Shafer G Vovk V: A tutorial on conformal prediction 2007. https:\/\/arxiv.org\/abs\/0706.3188"},{"key":"e_1_3_8_20_2","unstructured":"Angelopoulos AN Bates S: A gentle introduction to conformal prediction and distribution-free uncertainty quantification. 2021. https:\/\/arxiv.org\/abs\/2107.07511"},{"key":"e_1_3_8_21_2","unstructured":"Cresswell JC Sui Y Kumar B et al: Conformal prediction sets improve human decision making 2024. https:\/\/arxiv.org\/abs\/2401.13744"},{"key":"e_1_3_8_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s41666-021-00113-8"},{"key":"e_1_3_8_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.xphs.2020.09.055"},{"key":"e_1_3_8_24_2","doi-asserted-by":"crossref","unstructured":"Sreenivasan AP Vaivade A Noui Y et al: Conformal prediction enables disease course prediction and allows individualized diagnostic uncertainty in multiple sclerosis. npj Digit Med 8:224 2025","DOI":"10.1038\/s41746-025-01616-z"},{"key":"e_1_3_8_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2020.2996300"},{"key":"e_1_3_8_26_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-34945-8"},{"key":"e_1_3_8_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITAB.2009.5394447"},{"key":"e_1_3_8_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/GRC.2006.1635774"},{"key":"e_1_3_8_29_2","doi-asserted-by":"publisher","DOI":"10.1158\/2159-8290.CD-20-1230"},{"key":"e_1_3_8_30_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41375-021-01209-1"},{"key":"e_1_3_8_31_2","doi-asserted-by":"publisher","DOI":"10.1182\/bloodadvances.2021005634"},{"key":"e_1_3_8_32_2","volume-title":"Conformal Risk Control","author":"Angelopoulos AN","year":"2022","unstructured":"Angelopoulos AN, Bates S, Fisch A, et al: Conformal Risk Control, 2022. https:\/\/arxiv.org\/abs\/2208.02814"},{"key":"e_1_3_8_33_2","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2009.135"},{"key":"e_1_3_8_34_2","volume-title":"Artificial Intelligence Act","year":"2024","unstructured":"Artificial Intelligence Act. 2024. http:\/\/data.europa.eu\/eli\/reg\/2024\/1689\/oj"},{"key":"e_1_3_8_35_2","volume-title":"Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?","author":"Gyevnar B","year":"2023","unstructured":"Gyevnar B, Ferguson N, Schafer B: Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act? 2023. https:\/\/arxiv.org\/abs\/2302.10766"},{"key":"e_1_3_8_36_2","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2023.1282800"},{"key":"e_1_3_8_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11517-024-03145-5"},{"key":"e_1_3_8_38_2","unstructured":"Filho TS Song H Perello-Nieto M et al: Classifier calibration: A survey on how to assess and improve predicted class probabilities. 2021. https:\/\/arxiv.org\/abs\/2112.10327"},{"key":"e_1_3_8_39_2","volume-title":"On Calibration of Modern Neural Networks","author":"Guo C","year":"2017","unstructured":"Guo C, Pleiss G, Sun Y, et al: On Calibration of Modern Neural Networks, 2017. https:\/\/arxiv.org\/abs\/1706.04599"},{"key":"e_1_3_8_40_2","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2017.265"},{"key":"e_1_3_8_41_2","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.23.02238"},{"key":"e_1_3_8_42_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-024-07807-0"},{"key":"e_1_3_8_43_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-025-56229-7"},{"key":"e_1_3_8_44_2","doi-asserted-by":"publisher","DOI":"10.1158\/2643-3230.BCD-20-0229"},{"key":"e_1_3_8_45_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature05690"},{"key":"e_1_3_8_46_2","doi-asserted-by":"publisher","DOI":"10.3150\/21-BEJ1447"},{"key":"e_1_3_8_47_2","volume-title":"Conformal Prediction with Large Language Models for Multi-Choice Question Answering","author":"Kumar B","year":"2023","unstructured":"Kumar B, Lu C, Gupta G, et al: Conformal Prediction with Large Language Models for Multi-Choice Question Answering, 2023. https:\/\/arxiv.org\/abs\/2305.18404"},{"key":"e_1_3_8_48_2","volume-title":"Sample-Efficient Safety Assurances Using Conformal Prediction","author":"Luo R","year":"2021","unstructured":"Luo R, Zhao S, Kuck J, et al: Sample-Efficient Safety Assurances Using Conformal Prediction, 2021. https:\/\/arxiv.org\/abs\/2109.14082"},{"key":"e_1_3_8_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPCE61193.2024.10541258"},{"key":"e_1_3_8_50_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-021-01312-x"},{"key":"e_1_3_8_51_2","unstructured":"Lysenkova Wiklander M: Molmed\/allium_prepro: Initial release v1.1.0. 2024. https:\/\/zenodo.org\/doi\/10.5281\/zenodo.14329215"},{"key":"e_1_3_8_52_2","unstructured":"Lysenkova Wiklander M Krali O: Molmed\/allium: v2.4.0. 2024. https:\/\/zenodo.org\/doi\/10.5281\/zenodo.14329233"},{"key":"e_1_3_8_53_2","unstructured":"Lysenkova Wiklander M: Molmed\/conformist: v1.1.1 2024. https:\/\/zenodo.org\/doi\/10.5281\/zenodo.14329248"},{"key":"e_1_3_8_54_2","unstructured":"Lysenkova Wiklander M: Molmed\/ALLCoP: v1.0.2 2024. https:\/\/zenodo.org\/doi\/10.5281\/zenodo.14333709"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-24-00324","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,4]],"date-time":"2025-06-04T12:10:51Z","timestamp":1749039051000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-24-00324"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6]]},"references-count":53,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["10.1200\/CCI-24-00324"],"URL":"https:\/\/doi.org\/10.1200\/cci-24-00324","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.12.11.627902","asserted-by":"object"}]},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6]]},"assertion":[{"value":"2024-12-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-07","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2400324"}}