{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T16:44:17Z","timestamp":1755794657291,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":28,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,1,2]],"date-time":"2021-01-02T00:00:00Z","timestamp":1609545600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Claritrics Inc","award":["RB1920CS200BUDD008156"],"award-info":[{"award-number":["RB1920CS200BUDD008156"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,1,2]]},"DOI":"10.1145\/3430984.3430997","type":"proceedings-article","created":{"date-parts":[[2020,12,28]],"date-time":"2020-12-28T05:34:44Z","timestamp":1609133684000},"page":"299-306","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Is it hard to learn a classifier on this dataset?"],"prefix":"10.1145","author":[{"given":"Sudarsun","family":"Santhiappan","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering IIT Madras, India"}]},{"given":"Nitin","family":"Shravan","sequence":"additional","affiliation":[{"name":"Research and Development Division BUDDI.AI New York, USA"}]},{"given":"Balaraman","family":"Ravindran","sequence":"additional","affiliation":[{"name":"Robert Bosch Centre for Data Science and AI (RBC-DSAI) IIT Madras, India"}]}],"member":"320","published-online":{"date-parts":[[2021,1,2]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Experimenting multiresolution analysis for identifying regions of different classification complexity. Pattern Analysis and Applications 19 (01","author":"Armano Giuliano","year":"2015","unstructured":"Giuliano Armano and Emanuele Tamponi . 2015. Experimenting multiresolution analysis for identifying regions of different classification complexity. Pattern Analysis and Applications 19 (01 2015 ). https:\/\/doi.org\/10.1007\/s10044-014-0446-y 10.1007\/s10044-014-0446-y Giuliano Armano and Emanuele Tamponi. 2015. Experimenting multiresolution analysis for identifying regions of different classification complexity. Pattern Analysis and Applications 19 (01 2015). https:\/\/doi.org\/10.1007\/s10044-014-0446-y"},{"key":"e_1_3_2_1_2_1","volume-title":"An Introduction to Bootstrap Methods with Applications to R","author":"Chernick R.","unstructured":"Michael\u00a0 R. Chernick and Robert\u00a0 A. LaBudde . 2011. An Introduction to Bootstrap Methods with Applications to R ( 1 st ed.). Wiley Publishing , USA. Michael\u00a0R. Chernick and Robert\u00a0A. LaBudde. 2011. An Introduction to Bootstrap Methods with Applications to R (1st ed.). Wiley Publishing, USA.","edition":"1"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.1979.4766909"},{"key":"e_1_3_2_1_4_1","unstructured":"Bernard Desgraupes. 2013. Clustering Indices. https:\/\/cran.r-project.org\/web\/packages\/clusterCrit\/vignettes\/clusterCrit.pdf. https:\/\/cran.r-project.org\/web\/packages\/clusterCrit\/vignettes\/clusterCrit.pdf  Bernard Desgraupes. 2013. Clustering Indices. https:\/\/cran.r-project.org\/web\/packages\/clusterCrit\/vignettes\/clusterCrit.pdf. https:\/\/cran.r-project.org\/web\/packages\/clusterCrit\/vignettes\/clusterCrit.pdf"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1080\/01969727408546059"},{"key":"e_1_3_2_1_6_1","volume-title":"A Working Guide to Boosted Regression Trees. The Journal of animal ecology 77 (08","author":"Elith Jane","year":"2008","unstructured":"Jane Elith , J Leathwick , and Trevor Hastie . 2008. A Working Guide to Boosted Regression Trees. The Journal of animal ecology 77 (08 2008 ), 802\u201313. https:\/\/doi.org\/10.1111\/j.1365-2656.2008.01390.x 10.1111\/j.1365-2656.2008.01390.x Jane Elith, J Leathwick, and Trevor Hastie. 2008. A Working Guide to Boosted Regression Trees. The Journal of animal ecology 77 (08 2008), 802\u201313. https:\/\/doi.org\/10.1111\/j.1365-2656.2008.01390.x"},{"key":"e_1_3_2_1_7_1","volume-title":"Auto-sklearn: Efficient and Robust Automated Machine Learning","author":"Feurer Matthias","year":"2019","unstructured":"Matthias Feurer , Aaron Klein , Katharina Eggensperger , Jost\u00a0Tobias Springenberg , Manuel Blum , and Frank Hutter . 2019 . Auto-sklearn: Efficient and Robust Automated Machine Learning . Springer International Publishing , Cham , 113\u2013134. https:\/\/doi.org\/10.1007\/978-3-030-05318-5_6 10.1007\/978-3-030-05318-5_6 Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost\u00a0Tobias Springenberg, Manuel Blum, and Frank Hutter. 2019. Auto-sklearn: Efficient and Robust Automated Machine Learning. Springer International Publishing, Cham, 113\u2013134. https:\/\/doi.org\/10.1007\/978-3-030-05318-5_6"},{"key":"e_1_3_2_1_8_1","volume-title":"Effect of label noise in the complexity of classification problems. Neurocomputing 160 (02","author":"Garcia Lu\u00eds\u00a0Paulo","year":"2015","unstructured":"Lu\u00eds\u00a0Paulo Garcia , Andre de Carvalho , and Ana Lorena . 2015. Effect of label noise in the complexity of classification problems. Neurocomputing 160 (02 2015 ). https:\/\/doi.org\/10.1016\/j.neucom.2014.10.085 10.1016\/j.neucom.2014.10.085 Lu\u00eds\u00a0Paulo Garcia, Andre de Carvalho, and Ana Lorena. 2015. Effect of label noise in the complexity of classification problems. Neurocomputing 160 (02 2015). https:\/\/doi.org\/10.1016\/j.neucom.2014.10.085"},{"key":"e_1_3_2_1_9_1","volume-title":"Complexity Measures of Supervised Classification Problems","author":"Ho Tin","year":"2002","unstructured":"Tin Ho and Mitra Basu . 2002. Complexity Measures of Supervised Classification Problems . IEEE Trans. Pattern Anal. Mach. Intell . 24 (03 2002 ), 289\u2013300. https:\/\/doi.org\/10.1109\/34.990132 10.1109\/34.990132 Tin Ho and Mitra Basu. 2002. Complexity Measures of Supervised Classification Problems. IEEE Trans. Pattern Anal. Mach. Intell. 24 (03 2002), 289\u2013300. https:\/\/doi.org\/10.1109\/34.990132"},{"volume-title":"Measures of Geometrical Complexity in Classification Problems","author":"Ho Tin\u00a0Kam","key":"e_1_3_2_1_10_1","unstructured":"Tin\u00a0Kam Ho , Mitra Basu , and Martin Hiu\u00a0Chung Law . 2006. Measures of Geometrical Complexity in Classification Problems . Springer London , London , 1\u201323. https:\/\/doi.org\/10.1007\/978-1-84628-172-3_1 10.1007\/978-1-84628-172-3_1 Tin\u00a0Kam Ho, Mitra Basu, and Martin Hiu\u00a0Chung Law. 2006. Measures of Geometrical Complexity in Classification Problems. Springer London, London, 1\u201323. https:\/\/doi.org\/10.1007\/978-1-84628-172-3_1"},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of 13th International Conference on Pattern Recognition 4","volume":"4","author":"Hoekstra Aarnoud","year":"1996","unstructured":"Aarnoud Hoekstra and Robert P . \u00a0W. Duin. 1996. On the nonlinearity of pattern classifiers . Proceedings of 13th International Conference on Pattern Recognition 4 ( 1996 ), 271\u2013275 vol. 4 . Aarnoud Hoekstra and Robert P.\u00a0W. Duin. 1996. On the nonlinearity of pattern classifiers. Proceedings of 13th International Conference on Pattern Recognition 4 (1996), 271\u2013275 vol.4."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2044-8317.1976.tb00714.x"},{"key":"e_1_3_2_1_13_1","volume-title":"ExploreKit: Automatic Feature Generation and Selection. 2016 IEEE 16th International Conference on Data Mining (ICDM) 1","author":"Katz Gilad","year":"2016","unstructured":"Gilad Katz , Eui Chul\u00a0Richard Shin , and Dawn\u00a0Xiaodong Song . 2016 . ExploreKit: Automatic Feature Generation and Selection. 2016 IEEE 16th International Conference on Data Mining (ICDM) 1 (2016), 979\u2013984. Gilad Katz, Eui Chul\u00a0Richard Shin, and Dawn\u00a0Xiaodong Song. 2016. ExploreKit: Automatic Feature Generation and Selection. 2016 IEEE 16th International Conference on Data Mining (ICDM) 1 (2016), 979\u2013984."},{"key":"e_1_3_2_1_15_1","volume-title":"Analysis of complexity indices for classification problems: Cancer gene expression data. Neurocomputing 75 (01","author":"Lorena Ana","year":"2012","unstructured":"Ana Lorena , Ivan Costa , Newton Spola\u00f4r , and Marcilio de Souto . 2012. Analysis of complexity indices for classification problems: Cancer gene expression data. Neurocomputing 75 (01 2012 ), 33\u201342. https:\/\/doi.org\/10.1016\/j.neucom.2011.03.054 10.1016\/j.neucom.2011.03.054 Ana Lorena, Ivan Costa, Newton Spola\u00f4r, and Marcilio de Souto. 2012. Analysis of complexity indices for classification problems: Cancer gene expression data. Neurocomputing 75 (01 2012), 33\u201342. https:\/\/doi.org\/10.1016\/j.neucom.2011.03.054"},{"key":"e_1_3_2_1_16_1","volume-title":"Article 107 (Sept.","author":"Lorena C.","year":"2019","unstructured":"Ana\u00a0 C. Lorena , Lu\u00eds P.\u00a0F. Garcia , Jens Lehmann , Marcilio C.\u00a0P. Souto , and Tin\u00a0Kam Ho. 2019. How Complex Is Your Classification Problem? A Survey on Measuring Classification Complexity. ACM Comput. Surv. 52, 5 , Article 107 (Sept. 2019 ), 34\u00a0pages. https:\/\/doi.org\/10.1145\/3347711 10.1145\/3347711 Ana\u00a0C. Lorena, Lu\u00eds P.\u00a0F. Garcia, Jens Lehmann, Marcilio C.\u00a0P. Souto, and Tin\u00a0Kam Ho. 2019. How Complex Is Your Classification Problem? A Survey on Measuring Classification Complexity. ACM Comput. Surv. 52, 5, Article 107 (Sept. 2019), 34\u00a0pages. https:\/\/doi.org\/10.1145\/3347711"},{"key":"e_1_3_2_1_17_1","unstructured":"Martin Maechler Peter Rousseeuw Anja Struyf Mia Hubert and Kurt Hornik. 2019. cluster: Cluster Analysis Basics and Extensions. CRAN 1(2019) 82\u00a0pages. R package version 2.0.8.  Martin Maechler Peter Rousseeuw Anja Struyf Mia Hubert and Kurt Hornik. 2019. cluster: Cluster Analysis Basics and Extensions. CRAN 1(2019) 82\u00a0pages. R package version 2.0.8."},{"volume-title":"Introduction to Information Retrieval","author":"Manning D.","key":"e_1_3_2_1_18_1","unstructured":"Christopher\u00a0 D. Manning , Prabhakar Raghavan , and Hinrich Sch\u00fctze . 2008. Introduction to Information Retrieval . Cambridge University Press , Cambridge, UK . http:\/\/nlp.stanford.edu\/IR-book\/information-retrieval-book.html Christopher\u00a0D. Manning, Prabhakar Raghavan, and Hinrich Sch\u00fctze. 2008. Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK. http:\/\/nlp.stanford.edu\/IR-book\/information-retrieval-book.html"},{"volume-title":"Pattern Recognition and Image Analysis, Jorge\u00a0S. Marques, Nicol\u00e1s P\u00e9rez de\u00a0la Blanca","author":"Mollineda A.","key":"e_1_3_2_1_19_1","unstructured":"Ram\u00f3n\u00a0 A. Mollineda , J.\u00a0 Salvador S\u00e1nchez , and Jos\u00e9\u00a0 M. Sotoca . 2005. Data Characterization for Effective Prototype Selection . In Pattern Recognition and Image Analysis, Jorge\u00a0S. Marques, Nicol\u00e1s P\u00e9rez de\u00a0la Blanca , and Pedro Pina (Eds.). Springer Berlin Heidelberg , Berlin, Heidelberg , 27\u201334. Ram\u00f3n\u00a0A. Mollineda, J.\u00a0Salvador S\u00e1nchez, and Jos\u00e9\u00a0M. Sotoca. 2005. Data Characterization for Effective Prototype Selection. In Pattern Recognition and Image Analysis, Jorge\u00a0S. Marques, Nicol\u00e1s P\u00e9rez de\u00a0la Blanca, and Pedro Pina (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 27\u201334."},{"key":"e_1_3_2_1_20_1","volume-title":"Proceedings - 2013 Brazilian Conference on Intelligent Systems, BRACIS 2013 1, 12\u201318","author":"Morais Gleison","year":"2013","unstructured":"Gleison Morais and Ronaldo Prati . 2013 . Complex Network Measures for Data Set Characterization, In 2013 Brazilian Conference on Intelligent Systems . Proceedings - 2013 Brazilian Conference on Intelligent Systems, BRACIS 2013 1, 12\u201318 . https:\/\/doi.org\/10.1109\/BRACIS.2013.11 10.1109\/BRACIS.2013.11 Gleison Morais and Ronaldo Prati. 2013. Complex Network Measures for Data Set Characterization, In 2013 Brazilian Conference on Intelligent Systems. Proceedings - 2013 Brazilian Conference on Intelligent Systems, BRACIS 2013 1, 12\u201318. https:\/\/doi.org\/10.1109\/BRACIS.2013.11"},{"key":"e_1_3_2_1_21_1","volume-title":"Azure: Using Azure Machine Learning to Quickly Build AI Solutions. O\u2019Reilly Media, Incorporated, USA. https:\/\/books.google.co.in\/books?id=CgB4xgEACAAJ","author":"Mukunthu D.","year":"2019","unstructured":"D. Mukunthu , P. Shah , and W.H. Tok . 2019 . Practical Automated Machine Learning on Azure: Using Azure Machine Learning to Quickly Build AI Solutions. O\u2019Reilly Media, Incorporated, USA. https:\/\/books.google.co.in\/books?id=CgB4xgEACAAJ D. Mukunthu, P. Shah, and W.H. Tok. 2019. Practical Automated Machine Learning on Azure: Using Azure Machine Learning to Quickly Build AI Solutions. O\u2019Reilly Media, Incorporated, USA. https:\/\/books.google.co.in\/books?id=CgB4xgEACAAJ"},{"key":"e_1_3_2_1_22_1","volume-title":"DCoL: Data Complexity Library in C++ (Documentation). SourceForge 1 (01","author":"Orriols-Puig A","year":"2010","unstructured":"A Orriols-Puig , N\u00faria Maci\u00e0 , and Tin Ho. 2010. DCoL: Data Complexity Library in C++ (Documentation). SourceForge 1 (01 2010 ). A Orriols-Puig, N\u00faria Maci\u00e0, and Tin Ho. 2010. DCoL: Data Complexity Library in C++ (Documentation). SourceForge 1 (01 2010)."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1948.tb00008.x"},{"volume-title":"Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)","author":"Rosenberg Andrew","key":"e_1_3_2_1_24_1","unstructured":"Andrew Rosenberg and Julia Hirschberg . 2007. V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure . In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL) . Association for Computational Linguistics , Prague, Czech Republic, 410\u2013420. https:\/\/www.aclweb.org\/anthology\/D07-1043 Andrew Rosenberg and Julia Hirschberg. 2007. V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). Association for Computational Linguistics, Prague, Czech Republic, 410\u2013420. https:\/\/www.aclweb.org\/anthology\/D07-1043"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/0377-0427(87)90125-7"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2018.02.007"},{"volume-title":"Classification Potential vs. Classification Accuracy: A Comprehensive Study of Evolutionary Algorithms with Biomedical Datasets","author":"Tanwani Ajay\u00a0Kumar","key":"e_1_3_2_1_27_1","unstructured":"Ajay\u00a0Kumar Tanwani and Muddassar Farooq . 2010. Classification Potential vs. Classification Accuracy: A Comprehensive Study of Evolutionary Algorithms with Biomedical Datasets . In Learning Classifier Systems, Jaume Bacardit, Will Browne, Jan Drugowitsch, Ester Bernad\u00f3-Mansilla, and Martin\u00a0V. Butz (Eds.). Springer Berlin Heidelberg , Berlin, Heidelberg , 127\u2013144. Ajay\u00a0Kumar Tanwani and Muddassar Farooq. 2010. Classification Potential vs. Classification Accuracy: A Comprehensive Study of Evolutionary Algorithms with Biomedical Datasets. In Learning Classifier Systems, Jaume Bacardit, Will Browne, Jan Drugowitsch, Ester Bernad\u00f3-Mansilla, and Martin\u00a0V. Butz (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 127\u2013144."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487629"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3269299"}],"event":{"name":"CODS COMAD 2021: 8th ACM IKDD CODS and 26th COMAD","acronym":"CODS COMAD 2021","location":"Bangalore India"},"container-title":["Proceedings of the 3rd ACM India Joint International Conference on Data Science & Management of Data (8th ACM IKDD CODS & 26th COMAD)"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3430984.3430997","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3430984.3430997","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:43Z","timestamp":1750195483000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3430984.3430997"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,2]]},"references-count":28,"alternative-id":["10.1145\/3430984.3430997","10.1145\/3430984"],"URL":"https:\/\/doi.org\/10.1145\/3430984.3430997","relation":{},"subject":[],"published":{"date-parts":[[2021,1,2]]},"assertion":[{"value":"2021-01-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}