{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:23:04Z","timestamp":1760170984197},"reference-count":60,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2016,12,5]],"date-time":"2016-12-05T00:00:00Z","timestamp":1480896000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Min Knowl Disc"],"published-print":{"date-parts":[[2017,5]]},"DOI":"10.1007\/s10618-016-0484-8","type":"journal-article","created":{"date-parts":[[2016,12,5]],"date-time":"2016-12-05T05:58:58Z","timestamp":1480917538000},"page":"661-701","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["On classifier behavior in the presence of mislabeling noise"],"prefix":"10.1007","volume":"31","author":[{"given":"Katsiaryna","family":"Mirylenka","sequence":"first","affiliation":[]},{"given":"George","family":"Giannakopoulos","sequence":"additional","affiliation":[]},{"given":"Le Minh","family":"Do","sequence":"additional","affiliation":[]},{"given":"Themis","family":"Palpanas","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2016,12,5]]},"reference":[{"key":"484_CR1","unstructured":"Abdulrahman SM, Brazdil P, van Rijn JN, Vanschoren J (2015) Algorithm selection via meta-learning and sample-based active testing. In: Proceedings of the 2015 international workshop on meta-learning and algorithm selection (MetaSel) co-located with European conference on machine learning and principles and practice of knowledge discovery in databases (ECMLPKDD), pp 55\u201366"},{"issue":"6","key":"484_CR2","doi-asserted-by":"publisher","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","volume":"19","author":"H Akaike","year":"1974","unstructured":"Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716\u2013723. doi:\n                        10.1109\/TAC.1974.1100705","journal-title":"IEEE Trans Autom Control"},{"issue":"2","key":"484_CR3","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.asoc.2004.12.002","volume":"6","author":"S Ali","year":"2006","unstructured":"Ali S, Smith K (2006) On learning algorithm selection for classification. Appl Soft Comput 6(2):119\u2013138","journal-title":"Appl Soft Comput"},{"key":"484_CR4","unstructured":"Bod\u00e9n M (2002) A guide to recurrent neural networks and backpropagation. The Dallas project, SICS technical report (2), pp 1\u201310. \n                        http:\/\/130.102.79.1\/~mikael\/papers\/rn_dallas"},{"key":"484_CR5","volume-title":"Time series analysis: forecasting and control","author":"GE Box","year":"2015","unstructured":"Box GE, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: forecasting and control. Wiley, New York"},{"issue":"7","key":"484_CR6","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","volume":"30","author":"A Bradley","year":"1997","unstructured":"Bradley A (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit 30(7):1145\u20131159","journal-title":"Pattern Recognit"},{"key":"484_CR7","unstructured":"Brazdil P, Giraud Carrier C, Soares C, Vilalta R (2009) Development of metalearning systems for algorithm recommendation. Metalearning, 31\u201359"},{"issue":"3","key":"484_CR8","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1023\/A:1021713901879","volume":"50","author":"PB Brazdil","year":"2003","unstructured":"Brazdil PB, Soares C, Pinto Da Costa J (2003) Ranking learning algorithms: using IBL and meta-learning on accuracy and time results. Mach Learn 50(3):251\u2013277","journal-title":"Mach Learn"},{"issue":"10","key":"484_CR9","doi-asserted-by":"crossref","first-page":"1404","DOI":"10.1109\/TPAMI.2002.1039212","volume":"24","author":"F Camastra","year":"2002","unstructured":"Camastra F, Vinciarelli A (2002) Estimating the intrinsic dimension of data with a fractal-based method. IEEE Trans Pattern Anal Mach Intell 24(10):1404\u20131407","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"484_CR10","unstructured":"Chevaleyre Y, Zucker JD (2000) Noise-tolerant rule induction from multi-instance data. In: Proceedings of the workshop on attribute-value and relational learning: crossing the boundaries, co-located with international conference on machine learning (ICML), pp 47\u201352"},{"key":"484_CR11","doi-asserted-by":"crossref","unstructured":"Cohen WW (1995) Fast effective rule induction. In: Proceedings of the twelfth international conference on machine learning, pp 115\u2013123","DOI":"10.1016\/B978-1-55860-377-6.50023-2"},{"key":"484_CR12","doi-asserted-by":"crossref","DOI":"10.1002\/9781118165881","volume-title":"Nonparametric statistics for non-statisticians: a step-by-step approach","author":"GW Corder","year":"2009","unstructured":"Corder GW, Foreman DI (2009) Nonparametric statistics for non-statisticians: a step-by-step approach. Wiley, Hoboken"},{"issue":"5","key":"484_CR13","doi-asserted-by":"crossref","first-page":"1925","DOI":"10.1016\/j.patcog.2014.12.003","volume":"48","author":"RM Cruz","year":"2015","unstructured":"Cruz RM, Sabourin R, Cavalcanti GD, Ren TI (2015) Meta-des: a dynamic ensemble selection framework using meta-learning. Pattern Recognit 48(5):1925\u20131935","journal-title":"Pattern Recognit"},{"key":"484_CR14","doi-asserted-by":"crossref","unstructured":"de\u00a0Sousa E, Traina A, Traina\u00a0Jr. C, Faloutsos C (2006) Evaluating the intrinsic dimension of evolving data streams. In: Proceedings of the 2006 ACM symposium on applied computing, pp 643\u2013648","DOI":"10.1145\/1141277.1141426"},{"key":"484_CR15","unstructured":"Dupont P (2006) Noisy sequence classification with smoothed Markov chains. In: Proceedings of the 8th French conference on machine learning (CAP 2006), pp 187\u2013201"},{"key":"484_CR16","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","volume":"211","author":"J Elman","year":"1990","unstructured":"Elman J (1990) Finding structure in time. Cognit Sci 211:179\u2013211","journal-title":"Cognit Sci"},{"key":"484_CR17","doi-asserted-by":"crossref","unstructured":"Eom SB, Ketcherside MA, Lee HH, Rodgers ML, Starrett D (2004) The determinants of web-based instructional systems\u2019 outcome and satisfaction: an empirical investigation. Cognitive aspects of online programs. Instr Technol, pp 96\u2013139","DOI":"10.4018\/978-1-59140-565-8.ch004"},{"key":"484_CR18","unstructured":"Fran\u00e7ois JM (2013) Jahmm-hidden Markov model (HMM): an implementation in Java. \n                        https:\/\/code.google.com\/p\/jahmm\/"},{"key":"484_CR19","unstructured":"Lichman M (2013) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences. \n                        http:\/\/archive.ics.uci.edu\/ml"},{"key":"484_CR20","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.neucom.2014.12.100","volume":"176","author":"LPF Garcia","year":"2016","unstructured":"Garcia LPF, de Carvalho ACPLF, Lorena AC (2016) Noise detection in the meta-learning level. Neurocomputing 176:14\u201325","journal-title":"Neurocomputing"},{"key":"484_CR21","doi-asserted-by":"publisher","unstructured":"Giannakopoulos G, Palpanas T (2010) The effect of history on modeling systems\u2019 performance: the problem of the demanding lord. In: IEEE 10th international conference on data mining (ICDM). doi:\n                        10.1109\/ICDM.2010.90","DOI":"10.1109\/ICDM.2010.90"},{"issue":"3","key":"484_CR22","doi-asserted-by":"publisher","first-page":"653","DOI":"10.1007\/s10115-012-0568-8","volume":"36","author":"G Giannakopoulos","year":"2013","unstructured":"Giannakopoulos G, Palpanas T (2013) Revisiting the effect of history on learning performance: the problem of the demanding lord. Knowl Inf Syst 36(3):653\u2013691. doi:\n                        10.1007\/s10115-012-0568-8","journal-title":"Knowl Inf Syst"},{"issue":"3","key":"484_CR23","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1023\/B:MACH.0000015878.60765.42","volume":"54","author":"C Giraud-Carrier","year":"2004","unstructured":"Giraud-Carrier C, Vilalta R, Brazdil P (2004) Introduction to the special issue on meta-learning. Mach Learn 54(3):187\u2013193","journal-title":"Mach Learn"},{"issue":"1","key":"484_CR24","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","volume":"11","author":"M Hall","year":"2009","unstructured":"Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I (2009) The weka data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10\u201318","journal-title":"ACM SIGKDD Explor Newsl"},{"key":"484_CR25","volume-title":"Data mining: concepts and techniques","author":"J Han","year":"2006","unstructured":"Han J, Kamber M (2006) Data mining: concepts and techniques. Morgan Kaufmann, San Francisco"},{"key":"484_CR26","unstructured":"Haussler D (1990) Probably approximately correct learning. University of California, Santa Cruz, Computer Research Laboratory"},{"issue":"3","key":"484_CR27","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/s10710-014-9236-y","volume":"16","author":"MI Heywood","year":"2015","unstructured":"Heywood MI (2015) Evolutionary model building under streaming data for classification tasks: opportunities and challenges. Genet Program Evolvable Mach 16(3):283\u2013326","journal-title":"Genet Program Evolvable Mach"},{"key":"484_CR28","unstructured":"Kalapanidas E, Avouris N, Craciun M, Neagu D (2003) Machine learning algorithms: a study on noise sensitivity. In: Proceedings 1st Balcan conference in informatics, pp 356\u2013365"},{"issue":"3","key":"484_CR29","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1162\/089976601300014493","volume":"13","author":"S Keerthi","year":"2001","unstructured":"Keerthi S, Shevade S, Bhattacharyya C, Murthy K (2001) Improvements to platt\u2019s SMO algorithm for SVM classifier design. Neural Comput 13(3):637\u2013649","journal-title":"Neural Comput"},{"key":"484_CR30","unstructured":"Klinkenberg R (2005) Meta-learning, model selection, and example selection in machine learning domains with concept drift. In: Lernen, Wissensentdeckung und Adaptivit\u00e4t (LWA) 2005, GI Workshops, Saarbr\u00fccken, October 10th\u201312th, pp 164\u2013171"},{"key":"484_CR31","unstructured":"Kuh A, Petsche T, Rivest RL (1990) Learning time-varying concepts. In: Conference on neural information processing systems (NIPS), pp 183\u2013189"},{"key":"484_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied predictive modeling","author":"M Kuhn","year":"2013","unstructured":"Kuhn M, Johnson K (2013) Applied predictive modeling. Springer, New York. doi:\n                        10.1007\/978-1-4614-6849-3"},{"key":"484_CR33","unstructured":"Lafferty J, McCallum A, Pereira F (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning (ICML), pp 282\u2013289"},{"key":"484_CR34","unstructured":"Li Q, Li T, Zhu S, Kambhamettu C (2002) Improving medical\/biological data classification performance by wavelet preprocessing. In: Proceedings of the IEEE international conference on data mining (ICDM), pp 657\u2013660"},{"issue":"1","key":"484_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v008.i18","volume":"8","author":"G Marsaglia","year":"2003","unstructured":"Marsaglia G, Tsang WW, Wang J (2003) Evaluating Kolmogorov\u015b distribution. J Stat Softw 8(1):1\u20134. doi:\n                        10.18637\/jss.v008.i18","journal-title":"J Stat Softw"},{"issue":"253","key":"484_CR36","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1080\/01621459.1951.10500769","volume":"46","author":"FJ Massey Jr","year":"1951","unstructured":"Massey FJ Jr (1951) The Kolmogorov-Smirnov test for goodness of fit. J Am Stat Assoc 46(253):68\u201378","journal-title":"J Am Stat Assoc"},{"key":"484_CR37","unstructured":"McCallum AK (2002) Mallet: a machine learning for language toolkit. \n                        http:\/\/mallet.cs.umass.edu"},{"issue":"3","key":"484_CR38","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1007\/s00778-015-0382-5","volume":"24","author":"K Mirylenka","year":"2015","unstructured":"Mirylenka K, Cormode G, Palpanas T, Srivastava D (2015) Conditional heavy hitters: detecting interesting correlations in data streams. Int J Very Large Data Bases (VLDB) 24(3):395\u2013414","journal-title":"Int J Very Large Data Bases (VLDB)"},{"key":"484_CR39","doi-asserted-by":"crossref","unstructured":"Mirylenka K, Giannakopoulos G, Palpanas T (2012) SRF: a framework for the study of classifier behavior under training set mislabeling noise. In: Advances in knowledge discovery and data mining, lecture notes in computer science, vol 7301, pp 109\u2013121","DOI":"10.1007\/978-3-642-30217-6_10"},{"key":"484_CR40","doi-asserted-by":"crossref","unstructured":"Mirylenka K, Palpanas T, Cormode G, Srivastava D (2013) Finding interesting correlations with conditional heavy hitters. In: IEEE 29th international conference on data engineering (ICDE), pp 1069\u20131080","DOI":"10.1109\/ICDE.2013.6544898"},{"key":"484_CR41","unstructured":"Mantovani RG, Rossi ALD, Vanschoren J, Carvalho ACPLF (2015) Meta-learning recommendation of default hyper-parameter values for SVMs in classification tasks. In: Proceedings of the 2015 international workshop on meta-learning and algorithm selection (MetaSel), European conference on machine learning and principles and practice of knowledge discovery in databases (ECMLPKDD), pp 80\u201392"},{"issue":"4","key":"484_CR42","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1007\/s10462-010-9156-z","volume":"33","author":"DF Nettleton","year":"2010","unstructured":"Nettleton DF, Orriols-Puig A, Fornells A (2010) A study of the effect of different types of noise on the precision of supervised learning techniques. Artif Intell Rev 33(4):275\u2013306. doi:\n                        10.1007\/s10462-010-9156-z","journal-title":"Artif Intell Rev"},{"key":"484_CR43","doi-asserted-by":"crossref","unstructured":"Pechenizkiy M (2015) Predictive analytics on evolving data streams anticipating and adapting to changes in known and unknown contexts. In: IEEE international conference on high performance computing & simulation (HPCS), pp 658\u2013659","DOI":"10.1109\/HPCSim.2015.7237112"},{"key":"484_CR44","unstructured":"Pendrith M, Sammut C (1994) On reinforcement learning of control actions in noisy and non-markovian domains. Technical report, School of Computer Science and Engineering, The University of New South Wales, Sydney"},{"issue":"1","key":"484_CR45","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MASSP.1986.1165342","volume":"3","author":"L Rabiner","year":"1986","unstructured":"Rabiner L, Juang B (1986) An introduction to hidden Markov models. IEEE ASSP Mag 3(1):4\u201316","journal-title":"IEEE ASSP Mag"},{"key":"484_CR46","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.neucom.2013.05.048","volume":"127","author":"ALD Rossi","year":"2014","unstructured":"Rossi ALD, de Leon Ponce, Ferreira de Carvalho AC, Soares C, Feres de Souza B (2014) MetaStream: a meta-learning based method for periodic algorithm selection in time-changing data. Neurocomputing 127:52\u201364","journal-title":"Neurocomputing"},{"key":"484_CR47","unstructured":"Smith MR, Mitchell L, Giraud-Carrier C, Martinez T (2014) Recommending learning algorithms and their associated hyperparameters. \n                        arXiv:1407.1890"},{"key":"484_CR48","unstructured":"Sutton C, McCallum A (2010) An introduction to conditional random fields. \n                        arXiv:1011.4088"},{"issue":"1","key":"484_CR49","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1177\/875647939000600106","volume":"6","author":"R Taylor","year":"1990","unstructured":"Taylor R (1990) Interpretation of the correlation coefficient: a basic review. J Diagn Med Sonogr 6(1):35\u201339","journal-title":"J Diagn Med Sonogr"},{"key":"484_CR50","unstructured":"Teytaud O (2001) Learning with noise. Extension to regression. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN\u201901) vol\u00a03, pp 1787\u20131792"},{"key":"484_CR51","volume-title":"Pattern recognition","author":"S Theodoridis","year":"2003","unstructured":"Theodoridis S, Koutroumbas K (2003) Pattern recognition. Academic Press, San Diego"},{"issue":"11","key":"484_CR52","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1145\/1968.1972","volume":"27","author":"L Valiant","year":"1984","unstructured":"Valiant L (1984) A theory of the learnable. Commun ACM 27(11):1134\u20131142","journal-title":"Commun ACM"},{"key":"484_CR53","unstructured":"Vapnik VN (1998) Statistical learning theory, vol 1. Wiley, New York"},{"key":"484_CR54","unstructured":"Waluyan L, Sasipan S, Noguera S, Asai T (2009) Analysis of potential problems in people management concerning information security in cross-cultural environment -in the case of Malaysia. In: Proceedings of the third international symposium on human aspects of information security & assurance (HAISA), pp 13\u201324"},{"issue":"3","key":"484_CR55","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1023\/A:1007365809034","volume":"27","author":"G Widmer","year":"1997","unstructured":"Widmer G (1997) Tracking context changes through meta-learning. Mach Learn 27(3):259\u2013286","journal-title":"Mach Learn"},{"key":"484_CR56","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1162\/neco.1996.8.7.1391","volume":"8","author":"D Wolpert","year":"1996","unstructured":"Wolpert D (1996) The existence of a priori distinctions between learning algorithms. Neural Comput 8:1391\u20131421","journal-title":"Neural Comput"},{"key":"484_CR57","unstructured":"Wolpert D (2001) The supervised learning no-free-lunch theorems. In: Proceedings of the 6th online world conference on soft computing in industrial applications. Springer, London, pp 25\u201342"},{"key":"484_CR58","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1162\/neco.1996.8.7.1341","volume":"8","author":"DH Wolpert","year":"1996","unstructured":"Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8:1341\u20131390","journal-title":"Neural Comput"},{"key":"484_CR59","unstructured":"won Lee J, Giraud-Carrier C (2008) New insights into learning algorithms and datasets. In: IEEE seventh international conference on machine learning and applications (ICMLA\u201908), pp 135\u2013140"},{"issue":"1","key":"484_CR60","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1145\/1882471.1882478","volume":"12","author":"Z Xing","year":"2010","unstructured":"Xing Z, Pei J, Keogh E (2010) A brief survey on sequence classification. ACM SIGKDD Explor Newsl 12(1):40. doi:\n                        10.1145\/1882471.1882478","journal-title":"ACM SIGKDD Explor Newsl"}],"container-title":["Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10618-016-0484-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-016-0484-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-016-0484-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2018,12,4]],"date-time":"2018-12-04T03:46:41Z","timestamp":1543895201000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10618-016-0484-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,12,5]]},"references-count":60,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,5]]}},"alternative-id":["484"],"URL":"https:\/\/doi.org\/10.1007\/s10618-016-0484-8","relation":{},"ISSN":["1384-5810","1573-756X"],"issn-type":[{"value":"1384-5810","type":"print"},{"value":"1573-756X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,12,5]]}}}