{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T07:00:51Z","timestamp":1781679651683,"version":"3.54.5"},"reference-count":59,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T00:00:00Z","timestamp":1654646400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T00:00:00Z","timestamp":1654646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Precision medicine for cancer treatment relies on an accurate pathological diagnosis. The number of known tumor classes has increased rapidly, and reliance on traditional methods of histopathologic classification alone has become unfeasible. To help reduce variability, validation costs, and standardize the histopathological diagnostic process, supervised machine learning models using DNA-methylation data have been developed for tumor classification. These methods require large labeled training data sets to obtain clinically acceptable classification accuracy. While there is abundant unlabeled epigenetic data across multiple databases, labeling pathology data for machine learning models is time-consuming and resource-intensive, especially for rare tumor types. Semi-supervised learning (SSL) approaches have been used to maximize the utility of labeled and unlabeled data for classification tasks and are effectively applied in genomics. SSL methods have not yet been explored with epigenetic data nor demonstrated beneficial to central nervous system (CNS) tumor classification.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>This paper explores the application of semi-supervised machine learning on methylation data to improve the accuracy of supervised learning models in classifying CNS tumors. We comprehensively evaluated 11 SSL methods and developed a novel combination approach that included a self-training with editing using support vector machine (SETRED-SVM) model and an L2-penalized, multinomial logistic regression model to obtain high confidence labels from a few labeled instances. Results across eight random forest and neural net models show that the pseudo-labels derived from our SSL method can significantly increase prediction accuracy for 82 CNS tumors and 9 normal controls.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The proposed combination of semi-supervised technique and multinomial logistic regression holds the potential to leverage the abundant publicly available unlabeled methylation data effectively. Such an approach is highly beneficial in providing additional training examples, especially for scarce tumor types, to boost the prediction accuracy of supervised models.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12859-022-04764-1","type":"journal-article","created":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T16:03:46Z","timestamp":1654704226000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Comprehensive study of semi-supervised learning for DNA methylation-based supervised classification of central nervous system tumors"],"prefix":"10.1186","volume":"23","author":[{"given":"Quynh T.","family":"Tran","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Md Zahangir","family":"Alom","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brent A.","family":"Orr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,6,8]]},"reference":[{"issue":"3","key":"4764_CR1","first-page":"264","volume":"8","author":"L Balyen","year":"2019","unstructured":"Balyen L, Peto T. Promising artificial intelligence-machine learning-deep learning algorithms in ophthalmology. Asia Pac J Ophthalmol (Phila). 2019;8(3):264\u201372.","journal-title":"Asia Pac J Ophthalmol (Phila)"},{"issue":"4","key":"4764_CR2","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1016\/j.jmir.2019.09.005","volume":"50","author":"G Currie","year":"2019","unstructured":"Currie G, Hawk KE, Rohren E, Vial A, Klein R. Machine learning and deep learning in medical imaging: intelligent imaging. J Med Imaging Radiat Sci. 2019;50(4):477\u201387.","journal-title":"J Med Imaging Radiat Sci"},{"issue":"8","key":"4764_CR3","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","volume":"18","author":"A Hosny","year":"2018","unstructured":"Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500\u201310.","journal-title":"Nat Rev Cancer"},{"issue":"5","key":"4764_CR4","doi-asserted-by":"publisher","first-page":"584","DOI":"10.1016\/j.cmi.2019.09.009","volume":"26","author":"N Peiffer-Smadja","year":"2020","unstructured":"Peiffer-Smadja N, Rawson TM, Ahmad R, Buchard A, Georgiou P, Lescure FX, Birgand G, Holmes AH. Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect. 2020;26(5):584\u201395.","journal-title":"Clin Microbiol Infect"},{"issue":"5","key":"4764_CR5","doi-asserted-by":"publisher","first-page":"1452","DOI":"10.1111\/cas.14377","volume":"111","author":"H Shimizu","year":"2020","unstructured":"Shimizu H, Nakayama KI. Artificial intelligence in oncology. Cancer Sci. 2020;111(5):1452\u201360.","journal-title":"Cancer Sci"},{"issue":"1","key":"4764_CR6","doi-asserted-by":"publisher","first-page":"4627","DOI":"10.1038\/s41467-018-06715-y","volume":"9","author":"JIJ Orozco","year":"2018","unstructured":"Orozco JIJ, Knijnenburg TA, Manughian-Peter AO, Salomon MP, Barkhoudarian G, Jalas JR, Wilmott JS, Hothi P, Wang X, Takasumi Y, et al. Epigenetic profiling for the molecular classification of metastatic brain tumors. Nat Commun. 2018;9(1):4627.","journal-title":"Nat Commun"},{"issue":"1","key":"4764_CR7","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1186\/s13148-020-00842-4","volume":"12","author":"S Rauschert","year":"2020","unstructured":"Rauschert S, Raubenheimer K, Melton PE, Huang RC. Machine learning and clinical epigenetics: a review of challenges for diagnosis and classification. Clin Epigenetics. 2020;12(1):51.","journal-title":"Clin Epigenetics"},{"issue":"7697","key":"4764_CR8","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1038\/nature26000","volume":"555","author":"D Capper","year":"2018","unstructured":"Capper D, Jones DTW, Sill M, Hovestadt V, Schrimpf D, Sturm D, Koelsche C, Sahm F, Chavez L, Reuss DE, et al. DNA methylation-based classification of central nervous system tumours. Nature. 2018;555(7697):469\u201374.","journal-title":"Nature"},{"issue":"1","key":"4764_CR9","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1038\/s41467-020-20603-4","volume":"12","author":"C Koelsche","year":"2021","unstructured":"Koelsche C, Schrimpf D, Stichel D, Sill M, Sahm F, Reuss DE, Blattner M, Worst B, Heilig CE, Beck K, et al. Sarcoma classification by DNA methylation profiling. Nat Commun. 2021;12(1):498.","journal-title":"Nat Commun"},{"issue":"8","key":"4764_CR10","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0237428","volume":"15","author":"G Camargo","year":"2020","unstructured":"Camargo G, Bugatti PH, Saito PTM. Active semi-supervised learning for biological data classification. PLoS ONE. 2020;15(8): e0237428.","journal-title":"PLoS ONE"},{"key":"4764_CR11","doi-asserted-by":"publisher","first-page":"1529","DOI":"10.1109\/TKDE.2005.186","volume":"17","author":"Z Zhou","year":"2005","unstructured":"Zhou Z, Li M. Tri-training: exploiting unlabeled data using three classifiers. IEE Trans Knowl Data Eng. 2005;17:1529\u201341.","journal-title":"IEE Trans Knowl Data Eng"},{"issue":"3","key":"4764_CR12","first-page":"542","volume":"20","author":"O Chapelle","year":"2009","unstructured":"Chapelle O, Scholkopf B, Zien AE. Semi-supervise learning. J Intell Fuzzy Syst. 2009;20(3):542.","journal-title":"J Intell Fuzzy Syst"},{"key":"4764_CR13","doi-asserted-by":"publisher","first-page":"1483","DOI":"10.3233\/JIFS-169689","volume":"35","author":"G Kostopoulos","year":"2018","unstructured":"Kostopoulos G, Karlos S, Kotsiantis S, Ragos O. Semi-supervised regression: a recent review. J Intell Fuzzy Syst. 2018;35:1483\u2013500.","journal-title":"J Intell Fuzzy Syst"},{"key":"4764_CR14","first-page":"9","volume":"1","author":"N Grira","year":"2004","unstructured":"Grira N, Crucianu M, Boujemaa N. Unsupervised and semi-supervised clustering: a brief survey. A review of machine learning techniques for processing multimedia content. J Intell Fuzzy Syst. 2004;1:9\u201316.","journal-title":"J Intell Fuzzy Syst"},{"key":"4764_CR15","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1016\/j.ins.2021.01.075","volume":"560","author":"S Fu","year":"2021","unstructured":"Fu S, Liu W, Zhang K, Zhou Y, Tao D. Semi-supervised classification by graph p-Lapacian convolutional networks. Inf Sci. 2021;560:92\u2013106.","journal-title":"Inf Sci"},{"issue":"10","key":"4764_CR16","doi-asserted-by":"publisher","first-page":"1474","DOI":"10.1093\/neuonc\/noaa077","volume":"22","author":"K Fukuoka","year":"2020","unstructured":"Fukuoka K, Mamatjan Y, Tatevossian R, Zapotocky M, Ryall S, Stucklin AG, Bennett J, Nobre LF, Arnoldo A, Luu B, et al. Clinical impact of combined epigenetic and molecular analysis of pediatric low-grade gliomas. Neuro Oncol. 2020;22(10):1474\u201383.","journal-title":"Neuro Oncol"},{"key":"4764_CR17","doi-asserted-by":"publisher","first-page":"107307","DOI":"10.1016\/j.knosys.2021.107307","volume":"229","author":"X Hu","year":"2021","unstructured":"Hu X, Zeng Y, Xu X, Zhou S, Liu L. Robust semi-supervised classification based on data augmented online ELMs with deep features. Knowl-Based Syst. 2021;229:107307.","journal-title":"Knowl-Based Syst"},{"key":"4764_CR18","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1016\/j.neunet.2021.05.019","volume":"143","author":"Z Ji","year":"2021","unstructured":"Ji Z, Wang Q, Cui B, Pang Y, Cao X, Li X. A semi-supervised zero-shot image classification method based on soft-target. Neural Netw. 2021;143:88\u201396.","journal-title":"Neural Netw"},{"key":"4764_CR19","doi-asserted-by":"publisher","first-page":"107627","DOI":"10.1016\/j.patcog.2020.107627","volume":"110","author":"Z Kang","year":"2021","unstructured":"Kang Z, Peng C, Cheng Q, Liu X, Peng X, Xu Z, Tian L. Structured graph learning for clustering and semi-supervised classification. Pattern Recogn. 2021;110:107627.","journal-title":"Pattern Recogn"},{"issue":"1","key":"4764_CR20","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1109\/TCYB.2018.2883472","volume":"51","author":"Z Feng","year":"2021","unstructured":"Feng Z, Yang S, Wang M, Jiao L. Learning dual geometric low-rank structure for semisupervised hyperspectral image classification. IEEE Trans Cybern. 2021;51(1):346\u201358.","journal-title":"IEEE Trans Cybern"},{"key":"4764_CR21","doi-asserted-by":"publisher","first-page":"105782","DOI":"10.1016\/j.cmpb.2020.105782","volume":"198","author":"A In\u00e9s","year":"2021","unstructured":"In\u00e9s A, Dom\u00ednguez C, Heras J, Mata E, Pascual V. Biomedical image classification made easier thanks to transfer and semi-supervised learning. Comput Methods Programs Biomed. 2021;198:105782.","journal-title":"Comput Methods Programs Biomed"},{"key":"4764_CR22","doi-asserted-by":"publisher","first-page":"3781951","DOI":"10.1155\/2017\/3781951","volume":"2017","author":"AA Nahid","year":"2017","unstructured":"Nahid AA, Kong Y. Involvement of machine learning for breast cancer image classification: a survey. Comput Math Methods Med. 2017;2017:3781951.","journal-title":"Comput Math Methods Med"},{"key":"4764_CR23","first-page":"2806","volume":"2021","author":"S Roychowdhury","year":"2021","unstructured":"Roychowdhury S, Tang KS, Ashok M, Sanka A. SISE-PC: semi-supervised image subsampling for explainable pathology classification. Annu Int Conf IEEE Eng Med Biol Soc. 2021;2021:2806\u20139.","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"key":"4764_CR24","doi-asserted-by":"publisher","first-page":"104788","DOI":"10.1016\/j.compbiomed.2021.104788","volume":"137","author":"L Su","year":"2021","unstructured":"Su L, Liu Y, Wang M, Li A. Semi-HIC: a novel semi-supervised deep learning method for histopathological image classification. Comput Biol Med. 2021;137:104788.","journal-title":"Comput Biol Med"},{"issue":"4","key":"4764_CR25","doi-asserted-by":"publisher","first-page":"528","DOI":"10.1093\/jamiaopen\/ooz040","volume":"2","author":"AY Ling","year":"2019","unstructured":"Ling AY, Kurian AW, Caswell-Jin JL, Sledge GW Jr, Shah NH, Tamang SR. Using natural language processing to construct a metastatic breast cancer cohort from linked cancer registry and electronic medical records data. JAMIA Open. 2019;2(4):528\u201337.","journal-title":"JAMIA Open"},{"key":"4764_CR26","doi-asserted-by":"publisher","first-page":"103268","DOI":"10.1016\/j.jbi.2019.103268","volume":"98","author":"A Sarker","year":"2019","unstructured":"Sarker A, Klein AZ, Mee J, Harik P, Gonzalez-Hernandez G. An interpretable natural language processing system for written medical examination assessment. J Biomed Inform. 2019;98:103268.","journal-title":"J Biomed Inform"},{"issue":"1","key":"4764_CR27","doi-asserted-by":"publisher","first-page":"e210085","DOI":"10.1148\/ryai.210085","volume":"4","author":"S Zaman","year":"2022","unstructured":"Zaman S, Petri C, Vimalesvaran K, Howard J, Bharath A, Francis D, Peters N, Cole GD, Linton N. Automatic diagnosis labeling of cardiovascular MRI by using semisupervised natural language processing of text reports. Radiol Artif Intell. 2022;4(1):e210085.","journal-title":"Radiol Artif Intell"},{"key":"4764_CR28","first-page":"121650","volume":"2014","author":"D Zhou","year":"2014","unstructured":"Zhou D, He Y. Semi-supervised learning of statistical models for natural language understanding. ScientificWorldJournal. 2014;2014:121650.","journal-title":"ScientificWorldJournal"},{"key":"4764_CR29","doi-asserted-by":"crossref","unstructured":"Blum A, Mitchell T: Combining labeled and unlabeled data with co-training. In: Proceedings of the eleventh annual conference on Computational learning theory. COLT\u2019 98 edn. New York, NY, USA: Association for Computing Machinery; 1998: 92\u2013100.","DOI":"10.1145\/279943.279962"},{"issue":"4","key":"4764_CR30","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1136\/amiajnl-2012-001570","volume":"20","author":"J Kim","year":"2013","unstructured":"Kim J, Shin H. Breast cancer survivability prediction using labeled, unlabeled, and pseudo-labeled patient data. J Am Med Inform Assoc. 2013;20(4):613\u20138.","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"4764_CR31","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1038\/nrg3920","volume":"16","author":"MW Libbrecht","year":"2015","unstructured":"Libbrecht MW, Noble WS. Machine learning applications in genetics and genomics. Nat Rev Genet. 2015;16(6):321\u201332.","journal-title":"Nat Rev Genet"},{"issue":"Suppl 1","key":"4764_CR32","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1186\/s40246-019-0221-7","volume":"13","author":"M Sheikh Hassani","year":"2019","unstructured":"Sheikh Hassani M, Green JR. A semi-supervised machine learning framework for microRNA classification. Hum Genomics. 2019;13(Suppl 1):43.","journal-title":"Hum Genomics"},{"issue":"1","key":"4764_CR33","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1186\/s12880-020-00485-0","volume":"20","author":"C Ge","year":"2020","unstructured":"Ge C, Gu IY, Jakola AS, Yang J. Deep semi-supervised learning for brain tumor classification. BMC Med Imaging. 2020;20(1):87.","journal-title":"BMC Med Imaging"},{"key":"4764_CR34","doi-asserted-by":"crossref","unstructured":"Li M, Zhou Z: Setred: Self-training with editing. In: Advances in Knowledge Discovery and Data Mining. vol. 3518: Springer, Heidelberg; 2005: 611\u2013621.","DOI":"10.1007\/11430919_71"},{"issue":"6","key":"4764_CR35","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1016\/j.knosys.2010.03.012","volume":"23","author":"Y Wang","year":"2010","unstructured":"Wang Y, Xu X, Zhao H, Hua Z. Semi-supervised learning based on nearest neighbor rule and cut edges. Knowl-Based Syst. 2010;23(6):547\u201354.","journal-title":"Knowl-Based Syst"},{"key":"4764_CR36","doi-asserted-by":"crossref","unstructured":"Yarowsky D: Unsupervised word sense disambiguation rivaling supervised methods. In: Proceedings of the 33rd annual meeting on Association for Computational Linguistics; Cambridge, Massachusetts. Association for Computational Linguistics 1995: 189\u2013196.","DOI":"10.3115\/981658.981684"},{"key":"4764_CR37","unstructured":"Zhou Y, Goldman S: Democratic co-learning. IEEE 2004 (IEEE 16th International Conference on Tools with Artificial Intelligence (ICTAI)):8."},{"issue":"3","key":"4764_CR38","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1002\/bimj.200710415","volume":"50","author":"MD Ruopp","year":"2008","unstructured":"Ruopp MD, Perkins NJ, Whitcomb BW, Schisterman EF. Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection. Biom J. 2008;50(3):419\u201330.","journal-title":"Biom J"},{"key":"4764_CR39","doi-asserted-by":"publisher","first-page":"462","DOI":"10.1016\/j.neucom.2013.03.011","volume":"119","author":"Z Wei","year":"2013","unstructured":"Wei Z, Wang H, Zhao R. Semi-supervised multi-label image classification based on nearest neighbor editing. Neurocomput. 2013;119:462\u20138.","journal-title":"Neurocomput"},{"key":"4764_CR40","unstructured":"Cohen N, Abutbul R, Hoshen Y: Out-of-Distribution Detection without Class Labels. In. CoRR: arXiv; 2021."},{"key":"4764_CR41","doi-asserted-by":"crossref","unstructured":"Wan X: The Influence of Polynomial Order in Logistic Regression on Decision Boundary. In: IOP Conference Series: Earth and Environmental Science. 2019.","DOI":"10.1088\/1755-1315\/267\/4\/042077"},{"issue":"2","key":"4764_CR42","doi-asserted-by":"publisher","first-page":"e0229345","DOI":"10.1371\/journal.pone.0229345","volume":"15","author":"DG Jenkins","year":"2020","unstructured":"Jenkins DG, Quintana-Ascencio PF. A solution to minimum sample size for regressions. PLoS ONE. 2020;15(2):e0229345.","journal-title":"PLoS ONE"},{"key":"4764_CR43","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.ins.2017.08.093","volume":"422","author":"P Qian","year":"2018","unstructured":"Qian P, Xi C, Xu M, Jiang Y, Su KH, Wang S, Muzic RF Jr. SSC-EKE: semi-supervised classification with extensive knowledge exploitation. Inf Sci (N Y). 2018;422:51\u201376.","journal-title":"Inf Sci (N Y)"},{"key":"4764_CR44","unstructured":"Yoon J, Zhang Y, Jordon J, van der Schaar M: VIME: Extending the Success of Self- and Semi-supervised Learning to Tabular Data. In: Advance in Neural Information Processing Systems: 2020. Curran Associates, Inc.: 11033\u201311043."},{"issue":"2","key":"4764_CR45","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","volume":"67","author":"H Zou","year":"2005","unstructured":"Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Statistical Methodology). 2005;67(2):301\u201320.","journal-title":"J R Stat Soc Ser B (Statistical Methodology)"},{"key":"4764_CR46","doi-asserted-by":"crossref","unstructured":"Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efro A: Context Encoders: Feature Learning by Inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition (CVRP): 2016; Las Vegas, NV, USA. 2016: 2536\u20132544.","DOI":"10.1109\/CVPR.2016.278"},{"key":"4764_CR47","unstructured":"Zhang H, Cisse M, Dauphin YN, Lopez-Paz D: mixup: Beyond Empirical Risk Minimization. In: International Conference on Learning Representation: 2018; Vancouver, BC, Canada."},{"issue":"10","key":"4764_CR48","doi-asserted-by":"publisher","first-page":"1363","DOI":"10.1093\/bioinformatics\/btu049","volume":"30","author":"MJ Aryee","year":"2014","unstructured":"Aryee MJ, Jaffe AE, Corrada-Bravo H, Ladd-Acosta C, Feinberg AP, Hansen KD, Irizarry RA. Minfi: a flexible and comprehensive Bioconductor package for the analysis of Infinium DNA methylation microarrays. Bioinformatics. 2014;30(10):1363\u20139.","journal-title":"Bioinformatics"},{"issue":"7","key":"4764_CR49","doi-asserted-by":"publisher","first-page":"e90","DOI":"10.1093\/nar\/gkt090","volume":"41","author":"TJ Triche Jr","year":"2013","unstructured":"Triche TJ Jr, Weisenberger DJ, Van Den Berg D, Laird PW, Siegmund KD. Low-level processing of Illumina Infinium DNA Methylation BeadArrays. Nucleic Acids Res. 2013;41(7):e90.","journal-title":"Nucleic Acids Res"},{"key":"4764_CR50","unstructured":"Gonzalez M, Rosado-Falcon O, Rodriguez JD: ssc: An R Package for Semi-Supervised Classification. R package version 21\u20130 2021."},{"key":"4764_CR51","doi-asserted-by":"crossref","unstructured":"Brodersen KH, Ong CS, Stephan KE, Buhmann JM: The Balanced Accuracy and Its Posterior Distribution. In: 20th International Conference on Pattern Recognition: 23\u201326 Aug 2019. 2010: 3121\u20133124.","DOI":"10.1109\/ICPR.2010.764"},{"key":"4764_CR52","unstructured":"Grandini M, Bagli E, Visani G: Metrics for multi-class classification: an overview. arXiv preprint arXiv:200805756 2020."},{"issue":"1","key":"4764_CR53","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v033.i01","volume":"33","author":"J Friedman","year":"2010","unstructured":"Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):1\u201322.","journal-title":"J Stat Softw"},{"issue":"3","key":"4764_CR54","first-page":"307","volume":"29","author":"JA Hanley","year":"1989","unstructured":"Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Crit Rev Diagn Imaging. 1989;29(3):307\u201335.","journal-title":"Crit Rev Diagn Imaging"},{"issue":"1","key":"4764_CR55","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","volume":"143","author":"JA Hanley","year":"1982","unstructured":"Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29\u201336.","journal-title":"Radiology"},{"key":"4764_CR56","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1186\/1471-2105-12-77","volume":"12","author":"X Robin","year":"2011","unstructured":"Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.","journal-title":"BMC Bioinformatics"},{"issue":"20","key":"4764_CR57","doi-asserted-by":"publisher","first-page":"3940","DOI":"10.1093\/bioinformatics\/bti623","volume":"21","author":"T Sing","year":"2005","unstructured":"Sing T, Sander O, Beerenwinkel N, Lengauer T. ROCR: visualizing classifier performance in R. Bioinformatics. 2005;21(20):3940\u20131.","journal-title":"Bioinformatics"},{"key":"4764_CR58","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Ge Varoquaux, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Duborg V, et al. Scikit-learn: machine learning in python. J Mach Learn Res. 2011;12:2825\u201330.","journal-title":"J Mach Learn Res"},{"key":"4764_CR59","volume-title":"Deep learning with Keras","author":"A Gulli","year":"2017","unstructured":"Gulli A, Pal S. Deep learning with Keras. London: Packt Publishing Ltd; 2017."}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04764-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-022-04764-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04764-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,8]],"date-time":"2022-06-08T17:03:57Z","timestamp":1654707837000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-022-04764-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,8]]},"references-count":59,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["4764"],"URL":"https:\/\/doi.org\/10.1186\/s12859-022-04764-1","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,8]]},"assertion":[{"value":"21 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 May 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 June 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"223"}}