{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T06:34:50Z","timestamp":1774679690897,"version":"3.50.1"},"reference-count":31,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,3,27]],"date-time":"2021-03-27T00:00:00Z","timestamp":1616803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Australian Research Council, the Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico"},{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"crossref","award":["FL140100012, 305291\/2017-3, 2012\/22608-8 and 2019\/20328-7"],"award-info":[{"award-number":["FL140100012, 305291\/2017-3, 2012\/22608-8 and 2019\/20328-7"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2021,4,30]]},"abstract":"<jats:p>The quest for greater insights into algorithm strengths and weaknesses, as revealed when studying algorithm performance on large collections of test problems, is supported by interactive visual analytics tools. A recent advance is Instance Space Analysis, which presents a visualization of the space occupied by the test datasets, and the performance of algorithms across the instance space. The strengths and weaknesses of algorithms can be visually assessed, and the adequacy of the test datasets can be scrutinized through visual analytics. This article presents the first Instance Space Analysis of regression problems in Machine Learning, considering the performance of 14 popular algorithms on 4,855 test datasets from a variety of sources. The two-dimensional instance space is defined by measurable characteristics of regression problems, selected from over 26 candidate features. It enables the similarities and differences between test instances to be visualized, along with the predictive performance of regression algorithms across the entire instance space. The purpose of creating this framework for visual analysis of an instance space is twofold: one may assess the capability and suitability of various regression techniques; meanwhile the bias, diversity, and level of difficulty of the regression problems popularly used by the community can be visually revealed. This article shows the applicability of the created regression instance space to provide insights into the strengths and weaknesses of regression algorithms, and the opportunities to diversify the benchmark test instances to support greater insights.<\/jats:p>","DOI":"10.1145\/3436893","type":"journal-article","created":{"date-parts":[[2021,3,27]],"date-time":"2021-03-27T16:11:16Z","timestamp":1616861476000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["An Instance Space Analysis of Regression Problems"],"prefix":"10.1145","volume":"15","author":[{"given":"Mario Andr\u00e9s","family":"Mu\u00f1oz","sequence":"first","affiliation":[{"name":"The University of Melbourne, Parkville, Victoria, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Yan","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Parkville, Victoria, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matheus R.","family":"Leal","sequence":"additional","affiliation":[{"name":"Universidade Federal de Minas Gerais, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kate","family":"Smith-Miles","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Parkville, Victoria, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ana Carolina","family":"Lorena","sequence":"additional","affiliation":[{"name":"Instituto Tecnol\u00f3gico de Aeron\u00e1utica, SP - Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gisele L.","family":"Pappa","sequence":"additional","affiliation":[{"name":"Universidade Federal de Minas Gerais, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R\u00f4mulo Madureira","family":"Rodrigues","sequence":"additional","affiliation":[{"name":"Instituto Tecnol\u00f3gico de Aeron\u00e1utica"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,3,27]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"2","article-title":"KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework","volume":"17","author":"Alcal\u00e1-Fdez Jes\u00fas","year":"2011","unstructured":"Jes\u00fas Alcal\u00e1-Fdez , Alberto Fern\u00e1ndez , Juli\u00e1n Luengo , Joaqu\u00edn Derrac , Salvador Garc\u00eda , Luciano S\u00e1nchez , and Francisco Herrera . 2011 . KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework . Journal of Multiple-Valued Logic & Soft Computing 17 , 2 -- 3 (2011), 255--287. Jes\u00fas Alcal\u00e1-Fdez, Alberto Fern\u00e1ndez, Juli\u00e1n Luengo, Joaqu\u00edn Derrac, Salvador Garc\u00eda, Luciano S\u00e1nchez, and Francisco Herrera. 2011. KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. Journal of Multiple-Valued Logic & Soft Computing 17, 2--3 (2011), 255--287.","journal-title":"Journal of Multiple-Valued Logic & Soft Computing"},{"key":"e_1_2_1_2_1","volume-title":"Ersoy","author":"Fatih Amasyali M.","year":"2009","unstructured":"M. Fatih Amasyali and Okan K . Ersoy . 2009 . A Study of Meta Learning for Regression. Technical Report. ECE Technical Reports, Purdue University . M. Fatih Amasyali and Okan K. Ersoy. 2009. A Study of Meta Learning for Regression. Technical Report. ECE Technical Reports, Purdue University."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics","volume":"89","author":"Chen Yu","year":"2019","unstructured":"Yu Chen , Telmo Silva Filho , Ricardo B. C. Prud\u00eancio , Tom Diethe , and Peter Flach . 2019 . IRT: A new item response model and its applications . In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics , Vol. 89 . Yu Chen, Telmo Silva Filho, Ricardo B. C. Prud\u00eancio, Tom Diethe, and Peter Flach. 2019. IRT: A new item response model and its applications. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, Vol. 89."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7439(01)00111-3"},{"key":"e_1_2_1_5_1","unstructured":"Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. Retrieved from http:\/\/archive.ics.uci.edu\/ml.  Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. Retrieved from http:\/\/archive.ics.uci.edu\/ml."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2933261"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2011.07.005"},{"key":"e_1_2_1_9_1","volume-title":"Automated Machine Learning","author":"Hutter Frank","unstructured":"Frank Hutter , Lars Kotthoff , and Joaquin Vanschoren . 2019. Automated Machine Learning . Springer . Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. 2019. Automated Machine Learning. Springer."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2006.03.001"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2016.09.004"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the IBERAMIA.","volume":"2002","author":"Kuba Petr","year":"2002","unstructured":"Petr Kuba , Pavel Brazdil , Carlos Soares , and Adam Woznica . 2002 . Exploiting sampling and meta-learning for parameter setting support vector machines . In Proceedings of the IBERAMIA. Vol. 2002 . 217--225. Petr Kuba, Pavel Brazdil, Carlos Soares, and Adam Woznica. 2002. Exploiting sampling and meta-learning for parameter setting support vector machines. In Proceedings of the IBERAMIA. Vol. 2002. 217--225."},{"key":"e_1_2_1_13_1","unstructured":"MultiMedia LLC. 2019. International Institution of Forecasters. Retrieved from https:\/\/forecasters.org\/resources\/time-series-data\/m3-competition\/.  MultiMedia LLC. 2019. International Institution of Forecasters. Retrieved from https:\/\/forecasters.org\/resources\/time-series-data\/m3-competition\/."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-017-5681-1"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-013-0700-4"},{"key":"e_1_2_1_16_1","volume-title":"Item response theory in AI: Analysing machine learning classifiers at the instance level. Artificial Intelligence","author":"Mart\u00ednez-Plumed Fernando","year":"2019","unstructured":"Fernando Mart\u00ednez-Plumed , Ricardo B. C. Prud\u00eancio , Adolfo Mart\u00ednez-Us\u00f3 , and Jos\u00e9 Hern\u00e1ndez-Orallo . 2019. Item response theory in AI: Analysing machine learning classifiers at the instance level. Artificial Intelligence 271, June 2019 (2019), 18--42. Fernando Mart\u00ednez-Plumed, Ricardo B. C. Prud\u00eancio, Adolfo Mart\u00ednez-Us\u00f3, and Jos\u00e9 Hern\u00e1ndez-Orallo. 2019. Item response theory in AI: Analysing machine learning classifiers at the instance level. Artificial Intelligence 271, June 2019 (2019), 18--42."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco_a_00262"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco_a_00194"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-017-5629-5"},{"key":"e_1_2_1_20_1","volume-title":"Mu\u00f1oz and Kate Smith-Miles","author":"Mario","year":"2020","unstructured":"Mario A. Mu\u00f1oz and Kate Smith-Miles . 2020 . Instance Space Analysis: A Toolkit for the Assessment of Algorithmic Power. Retrieved from https:\/\/github.com\/andremun\/InstanceSpace. Mario A. Mu\u00f1oz and Kate Smith-Miles. 2020. Instance Space Analysis: A Toolkit for the Assessment of Algorithmic Power. Retrieved from https:\/\/github.com\/andremun\/InstanceSpace."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_2_1_22_1","volume-title":"Advances in Computers.","author":"Rice John R.","unstructured":"John R. Rice . 1976. The algorithm selection problem . In Advances in Computers. Vol. 15 . Elsevier , 65--118. John R. Rice. 1976. The algorithm selection problem. In Advances in Computers. Vol. 15. Elsevier, 65--118."},{"key":"e_1_2_1_23_1","volume-title":"Modern Regression Methods","author":"Ryan Thomas P.","unstructured":"Thomas P. Ryan . 2008. Modern Regression Methods . Vol. 655 . John Wiley & Sons . Thomas P. Ryan. 2008. Modern Regression Methods. Vol. 655. John Wiley & Sons."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3068335"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-013-5422-z"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2013.11.015"},{"key":"e_1_2_1_27_1","volume-title":"MATILDA: Melbourne Algorithm\u00a0Test Instance Library with Data Analytics.","author":"Smith-Miles Kate","year":"2019","unstructured":"Kate Smith-Miles , Mario A. Mu\u00f1oz and Neelofar. 2019 . MATILDA: Melbourne Algorithm\u00a0Test Instance Library with Data Analytics. Retrieved from https:\/\/matilda.unimelb.edu.au. Kate Smith-Miles, Mario A. Mu\u00f1oz and Neelofar. 2019. MATILDA: Melbourne Algorithm\u00a0Test Instance Library with Data Analytics. Retrieved from https:\/\/matilda.unimelb.edu.au."},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 2012 IEEE Congress on Evolutionary Computation. IEEE, 3446--3453","author":"Smith-Miles Kate","unstructured":"Kate Smith-Miles and Thomas T. Tan . 2012. Measuring algorithm footprints in instance space . In Proceedings of the 2012 IEEE Congress on Evolutionary Computation. IEEE, 3446--3453 . Kate Smith-Miles and Thomas T. Tan. 2012. Measuring algorithm footprints in instance space. In Proceedings of the 2012 IEEE Congress on Evolutionary Computation. IEEE, 3446--3453."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1456650.1456656"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:MACH.0000015879.28004.9b"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641190.2641198"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1019956318069"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436893","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3436893","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:20Z","timestamp":1750197740000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436893"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,27]]},"references-count":31,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4,30]]}},"alternative-id":["10.1145\/3436893"],"URL":"https:\/\/doi.org\/10.1145\/3436893","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,27]]},"assertion":[{"value":"2020-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}