{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T17:10:27Z","timestamp":1773249027495,"version":"3.50.1"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2003,3,1]],"date-time":"2003-03-01T00:00:00Z","timestamp":1046476800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2003,3,1]],"date-time":"2003-03-01T00:00:00Z","timestamp":1046476800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Machine Learning"],"published-print":{"date-parts":[[2003,3]]},"DOI":"10.1023\/a:1021713901879","type":"journal-article","created":{"date-parts":[[2003,3,20]],"date-time":"2003-03-20T21:15:11Z","timestamp":1048194911000},"page":"251-277","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":288,"title":["Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results"],"prefix":"10.1007","volume":"50","author":[{"given":"Pavel B.","family":"Brazdil","sequence":"first","affiliation":[]},{"given":"Carlos","family":"Soares","sequence":"additional","affiliation":[]},{"given":"Joaquim Pinto","family":"da Costa","sequence":"additional","affiliation":[]}],"member":"297","reference":[{"key":"5110565_CR1","first-page":"1","volume-title":"Proceedings of the Ninth International Workshop on Machine Learning (ML92)","author":"D. Aha","year":"1992","unstructured":"Aha, D. (1992). Generalizing from case studies: A case study. In D. Sleeman & P. Edwards (Eds.), Proceedings of the Ninth International Workshop on Machine Learning (ML92) (pp. 1\u201310). San Mateo, CA: Morgan Kaufmann."},{"key":"5110565_CR2","first-page":"11","volume-title":"Locally Weighted Learning","author":"C. G. Atkeson","year":"1997","unstructured":"Atkeson, C. G., Moore, A. W., & Schaal, S. (1997). Locally Weighted Learning (Vol. 11) (pp. 11\u201374). Boston: Kluwer."},{"key":"5110565_CR3","unstructured":"Bensusan, H., & Giraud-Carrier, C. (2000). If you see la sagrada familia, you know where you are: Landmarking the learner space. Technical report, Department of Computer Science, University of Bristol."},{"key":"5110565_CR4","first-page":"25","volume-title":"Proceedings of the 12th European Conference on Machine Learning","author":"H. Bensusan","year":"2001","unstructured":"Bensusan, H., & Kalousis, A. (2001). Estimating the predictive accuracy of a classifier. In P. Flach & L. de Raedt (Eds.), Proceedings of the 12th European Conference on Machine Learning (pp. 25\u201336). New York: Springer."},{"key":"5110565_CR5","unstructured":"Bernstein, A., & Provost, F. (2001). An intelligent assistant for the knowledge discovery process. In W. Hsu, H. Kargupta, H. Liu, & N. Street (Eds.), Proceedings of the IJCAI-01 Workshop on Wrappers for Performance Enhancement in KDD."},{"key":"5110565_CR6","unstructured":"Berrer, H., Paterson, I., & Keller, J. (2000). Evaluation of machine-learning algorithm ranking advisors. In P. Brazdil & A. Jorge (Eds.), Proceedings of the PKDD2000 Workshop on Data Mining, Decision Support, Meta-Learning and ILP: Forum for Practical Problem Presentation and Prospective Solutions (pp. 1\u201313)."},{"key":"5110565_CR7","unstructured":"Blake, C., Keogh, E., & Merz, C. (1998). Repository of machine learning databases. Available at http:\/www. ics.uci.edu\/~mlearn\/MLRepository.html"},{"key":"5110565_CR8","unstructured":"Brachman, R., & Anand, T. (1996). The process of knowledge discovery in databases. In U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, & R. Uthurusamy (Eds.), Advances in Knowledge Discovery and Data Mining, ch. 2 (pp. 37\u201357). AAAI Press\/The MIT Press."},{"key":"5110565_CR9","first-page":"83","volume-title":"Proceedings of the European Conference on Machine Learning (ECML-94)","author":"P. Brazdil","year":"1994","unstructured":"Brazdil, P., Gama, J., & Henery, B. (1994). Characterizing the applicability of classification algorithms using metalevel learning. In F. Bergadano & L. de Raedt (Eds.), Proceedings of the European Conference on Machine Learning (ECML-94) (pp. 83\u2013102). Berlin: Springer-Verlag."},{"key":"5110565_CR10","first-page":"63","volume-title":"Machine Learning: Proceedings of the 11th European Conference on Machine Learning ECML2000","author":"P. Brazdil","year":"2000","unstructured":"Brazdil, P., & Soares, C. (2000). A comparison of ranking methods for classification algorithm selection. In R. de M\u00e1ntaras & E. Plaza (Eds.), Machine Learning: Proceedings of the 11th European Conference on Machine Learning ECML2000 (pp. 63\u201374). Berlin: Springer."},{"key":"5110565_CR11","volume-title":"Proceedings of the 10th Portuguese Conference on Artificial Intelligence (EPIA 2001)","author":"P. Brazdil","year":"2001","unstructured":"Brazdil, P., Soares, C., & Pereira, R. (2001). Reducing rankings of classifiers by eliminating redundant cases. In P. Brazdil & A. Jorge (Eds.), Proceedings of the 10th Portuguese Conference on Artificial Intelligence (EPIA 2001). New York: Springer."},{"key":"5110565_CR12","first-page":"17","volume-title":"Proceedings of the Tenth International Conference on Machine Learning","author":"C. Brodley","year":"1993","unstructured":"Brodley, C. (1993). Addressing the selective superiority problem: Automatic algorithm\/model class selection. In P. Utgoff (Ed.), Proceedings of the Tenth International Conference on Machine Learning (pp. 17\u201324). San Mateo, CA: Morgan Kaufmann."},{"key":"5110565_CR13","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1016\/0377-2217(78)90138-8","volume":"2","author":"A. Charnes","year":"1978","unstructured":"Charnes, A., Cooper, W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2, 429\u2013444.","journal-title":"European Journal of Operational Research"},{"key":"5110565_CR14","first-page":"115","volume-title":"Proceedings of the 11th International Conference on Machine Learning","author":"W. Cohen","year":"1995","unstructured":"Cohen, W. (1995). Fast effective rule induction. In A. Prieditis & S. Russell (Eds.), Proceedings of the 11th International Conference on Machine Learning (pp. 115\u2013123). San Mateo, CA: Morgan Kaufmann."},{"key":"5110565_CR15","unstructured":"F\u00fcrnkranz, J., & Petrak, J. (2001). An evaluation of landmarking variants. In C. Giraud-Carrier, N. Lavrac, & S. Moyle (Eds.), Working Notes of the ECML\/PKDD 2000 Workshop on Integrating Aspects of Data Mining, Decision Support and Meta-Learning (pp. 57\u201368)."},{"key":"5110565_CR16","first-page":"134","volume-title":"Proceedings of the 14th International Machine Learning Conference (ICML97)","author":"J. Gama","year":"1997","unstructured":"Gama, J. (1997). Probabilistic linear tree. In D. Fisher (Ed.), Proceedings of the 14th International Machine Learning Conference (ICML97) (pp. 134\u2013142). San Mateo, CA: Morgan Kaufmann."},{"key":"5110565_CR17","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/3-540-60428-6_16","volume-title":"Progress in Artificial Intelligence","author":"J. Gama","year":"1995","unstructured":"Gama, J., & Brazdil, P. (1995). Characterization of classification algorithms. In C. Pinto-Ferreira & N. Mamede (Eds.), Progress in Artificial Intelligence (pp. 189\u2013200). Berlin: Springer-Verlag."},{"key":"5110565_CR18","unstructured":"Henery, R. (1994). Methods for comparison. In D. Michie, D. Spiegelhalter, & C. Taylor (Eds.), Machine Learning, Neural and Statistical Classification, ch. 7 (pp. 107\u2013124). Ellis Horwood."},{"key":"5110565_CR19","volume-title":"Proceedings of the IEEE SMC'99 International Conference on Systems, Man and Cybernetics","author":"M. Hilario","year":"1999","unstructured":"Hilario, M., & Kalousis, A. (1999). Building algorithm profiles for prior model selection in knowledge discovery systems. In Proceedings of the IEEE SMC'99 International Conference on Systems, Man and Cybernetics. New York: IEEE Press."},{"key":"5110565_CR20","volume-title":"Proceedings of the Fifth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD01)","author":"M. Hilario","year":"2001","unstructured":"Hilario, M., & Kalousis, A. (2001). Fusion of meta-knowledge and meta-data for case-based model selection. In A. Siebes & L. de Raedt (Eds.), Proceedings of the Fifth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD01). New York: Springer."},{"key":"5110565_CR21","first-page":"87","volume-title":"Lecture Notes on Comp. Sci. 2130, Proc. Intl. Conf. On Artificial Neural Networks (ICANN-2001)","author":"S. Hochreiter","year":"2001","unstructured":"Hochreiter, S., Younger, A., & Conwell, P. (2001). Learning to learn using gradient descent. In G. Dorffner, H. Bischof, & K. Hornik (Eds.), Lecture Notes on Comp. Sci. 2130, Proc. Intl. Conf. On Artificial Neural Networks (ICANN-2001) (pp. 87\u201394). New York: Springer."},{"key":"5110565_CR22","unstructured":"Kalousis, A., & Hilario, M. (2000). A comparison of inducer selection via instance-based and boosted decision tree meta-learning. In R. Michalski & P. Brazdil (Eds.), Proceedings of the Fifth International Workshop on Multistrategy Learning (pp. 233\u2013247)."},{"issue":"5","key":"5110565_CR23","first-page":"319","volume":"3","author":"A. Kalousis","year":"1999","unstructured":"Kalousis, A., & Theoharis, T. (1999).NOEMON:Design, implementation and performance results of an intelligent assistant for classifier selection. Intelligent Data Analysis, 3:5, 319\u2013337.","journal-title":"Intelligent Data Analysis"},{"key":"5110565_CR24","unstructured":"Keller, J., Paterson, I., & Berrer, H. (2000). An integrated concept for multi-criteria ranking of data-mining algorithms. In J. Keller & C. Giraud-Carrier (Eds.), Meta-Learning: Building Automatic Advice Strategies for Model Selection and Method Combination."},{"key":"5110565_CR25","doi-asserted-by":"crossref","unstructured":"Kohavi, R., John, G., Long, R., Mangley, D., & Pfleger, K. (1994). MLC++: A machine learning library in c++. Technical report, Stanford University.","DOI":"10.1109\/TAI.1994.346412"},{"key":"5110565_CR26","doi-asserted-by":"crossref","unstructured":"Lindner, G., & Studer, R. (1999). AST: Support for algorithm selection with a CBR approach. In C. Giraud-Carrier & B. Pfahringer (Eds.), Recent Advances in Meta-Learning and Future Work (pp. 38\u201347). J. Stefan Institute. Available at http:\/\/ftp.cs.bris.ac.uk\/cgc\/ICML99\/lindner.ps.Z","DOI":"10.1007\/978-3-540-48247-5_52"},{"key":"5110565_CR27","first-page":"59","volume-title":"Advances in Neural Information Processing Systems","author":"O. Maron","year":"1994","unstructured":"Maron, O., & Moore, A. (1994). Hoeffding races: Accelerating model selection search for classification and function approximation. In J. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in Neural Information Processing Systems (pp. 59\u201366). San Mateo, CA: Morgan Kaufmann."},{"key":"5110565_CR28","unstructured":"METAL Consortium (2002). Esprit project METAL (#26.357). Available at www.metal-kdd.org."},{"key":"5110565_CR29","unstructured":"Michie, D., Spiegelhalter, D., & Taylor, C. (1994). Machine Learning, Neural and Statistical Classification. Ellis Horwood."},{"key":"5110565_CR30","volume-title":"Machine Learning","author":"T. Mitchell","year":"1997","unstructured":"Mitchell, T. (1997). Machine Learning. New York: McGraw-Hill."},{"key":"5110565_CR31","unstructured":"Nakhaeizadeh, G., & Schnabl, A. (1997). Towards the personalization of algorithms evaluation in data mining. In R. Agrawal & P. Stolorz (Eds.), Proceedings of the Third International Conference on Knowledge Discovery & Data Mining (pp. 289\u2013293). AAAI Press."},{"key":"5110565_CR32","unstructured":"Nakhaeizadeh, G., & Schnabl, A. (1998). Development of multi-criteria metrics for evaluation of data mining algorithms. In D. Heckerman, H. Mannila, D. Pregibon, & R. Uthurusamy (Eds.), Proceedings of the Fourth International Conference on Knowledge Discovery in Databases & Data Mining (pp. 37\u201342). AAAI Press."},{"key":"5110565_CR33","volume-title":"Distribution-Free Tests","author":"H. Neave","year":"1992","unstructured":"Neave, H., & Worthington, P. (1992). Distribution-Free Tests. London: Routledge."},{"key":"5110565_CR34","first-page":"743","volume-title":"Proceedings of the Seventeenth International Conference on Machine Learning (ICML2000)","author":"B. Pfahringer","year":"2000","unstructured":"Pfahringer, B., Bensusan, H., & Giraud-Carrier, C. (2000). Tell me who can learn you and i can tell you who you are: Landmarking various learning algorithms. In P. Langley (Ed.), Proceedings of the Seventeenth International Conference on Machine Learning (ICML2000) (pp. 743\u2013750). San Mateo, CA: Morgan Kaufmann."},{"key":"5110565_CR35","unstructured":"Quinlan, R. (1998). C5.0: An Informal Tutorial. RuleQuest. Available at http:\/\/www.rulequest.com\/see5-unix.html."},{"key":"5110565_CR36","doi-asserted-by":"crossref","unstructured":"Ripley, B. (1996). Pattern Recognition and Neural Networks. Cambridge.","DOI":"10.1017\/CBO9780511812651"},{"issue":"1","key":"5110565_CR37","first-page":"135","volume":"13","author":"C. Schaffer","year":"1993","unstructured":"Schaffer, C. (1993). Selecting a classification method by cross-validation. Machine Learning, 13:1, 135\u2013143.","journal-title":"Machine Learning"},{"key":"5110565_CR38","first-page":"293","volume-title":"Reinforcement Learning With Self-Modifying Policies","author":"J. Schmidhuber","year":"1997","unstructured":"Schmidhuber, J., Zhao, J., & Schraudolph, N. (1997). Reinforcement Learning With Self-Modifying Policies (pp. 293\u2013309). Boston: Kluwer."},{"key":"5110565_CR39","first-page":"126","volume-title":"Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD2000)","author":"C. Soares","year":"2000","unstructured":"Soares, C., & Brazdil, P. (2000). Zoomed ranking: Selection of classification algorithms based on relevant performance information. In D. Zighed, J. Komorowski, & J. Zytkow (Eds.), Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD2000) (pp. 126\u2013135). New York: Springer."},{"key":"5110565_CR40","first-page":"119","volume-title":"Data Analysis, Classification and Related Methods, Proceedings of the Seventh Conference of the International Federation of Classification Societies IFCS","author":"C. Soares","year":"2000","unstructured":"Soares, C., Brazdil, P., & Costa, J. (2000). Measures to compare rankings of classification algorithms. In H. Kiers, J.-P. Rasson, P. Groenen, & M. Schader (Eds.), Data Analysis, Classification and Related Methods, Proceedings of the Seventh Conference of the International Federation of Classification Societies IFCS (pp. 119\u2013124). New York: Springer."},{"key":"5110565_CR41","unstructured":"Soares, C., Costa, J., & Brazdil, P. (2001a). Improved statistical support for matchmaking: Rank correlation taking rank importance into account. In JOCLAD 2001: VII Jornadas de Classifica\u00e7\u00e3o e An\u00e1lise de Dados (pp. 72\u201375)."},{"key":"5110565_CR42","first-page":"88","volume-title":"Proceedings of the 10th Portuguese Conference on Artificial Intelligence (EPIA 2001)","author":"C. Soares","year":"2001","unstructured":"Soares, C., Petrak, J., & Brazdil, P. (2001b). Sampling-based relative landmarks: Systematically test-driving algorithms before choosing. In P. Brazdil & A. Jorge (Eds.), Proceedings of the 10th Portuguese Conference on Artificial Intelligence (EPIA 2001) (pp. 88\u201394). New York: Springer."},{"issue":"11","key":"5110565_CR43","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/34.809107","volume":"21","author":"S. Sohn","year":"1999","unstructured":"Sohn, S. (1999). Meta analysis of classification algorithms for pattern recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21:11, 1137\u20131144.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"5110565_CR44","unstructured":"Suyama, A., Negishi, N., & Yamaguchi, T. (1999). CAMLET: A platform for automatic composition of inductive applications using ontologies. In C. Giraud-Carrier & B. Pfahringer (Eds.), Proceedings of the ICML-99 Workshop on Recent Advances in Meta-Learning and Future Work (pp. 59\u201365)."},{"key":"5110565_CR45","unstructured":"Todorovski, L., Brazdil, P., & Soares, C. (2000). Report on the experiments with feature selection in meta-level learning. In P. Brazdil & A. Jorge (Eds.), Proceedings of the Data Mining, Decision Support, Meta-Learning and ILP Workshop at PKDD2000 (pp. 27\u201339)."},{"key":"5110565_CR46","first-page":"98","volume-title":"Proceedings of the Third European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD99)","author":"L. Todorovski","year":"1999","unstructured":"Todorovski, L., & D\u017eeroski, S. (1999). Experiments in meta-level learning with ILP. In J. Rauch & J. Zytkow (Eds.), Proceedings of the Third European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD99) (pp. 98\u2013106). New York: Springer."},{"key":"5110565_CR47","first-page":"54","volume-title":"Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD00)","author":"L. Todorovski","year":"2000","unstructured":"Todorovski, L., & D\u017eeroski, S. (2000). Combining multiple models with meta decision trees. In D. Zighed, J. Komorowski, & J. Zytkow (Eds.), Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD00) (pp. 54\u201364). New York: Springer."},{"issue":"3","key":"5110565_CR48","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1023\/A:1021709817809","volume":"50","author":"L. Todorovski","year":"2003","unstructured":"Todorovski, L., & D\u017eeroski, S. (2003). Combining classifiers with meta-decision trees. Machine Learning Journal. 50:3 pp. 223\u2013249.","journal-title":"Machine Learning Journal"},{"key":"5110565_CR49","unstructured":"Vilalta, R. (1999). Understanding accuracy performance through concept characterization and algorithm analysis. In C. Giraud-Carrier & B. Pfahringer (Eds.), Recent Advances in Meta-Learning and Future Work (pp. 3\u20139). J. Stefan Institute."},{"key":"5110565_CR50","unstructured":"Wolpert, D., & Macready, W. (1996). No free lunch theorems for search. Technical Report SFI-TR-95-02-010, The Santa Fe Institute. Available at http:\/\/lucy.ipk.fhg.de:80\/~stephan\/nfl\/nfl.ps"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1023\/A:1021713901879.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1023\/A:1021713901879\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1023\/A:1021713901879.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,10]],"date-time":"2025-07-10T11:40:25Z","timestamp":1752147625000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1023\/A:1021713901879"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2003,3]]},"references-count":50,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2003,3]]}},"alternative-id":["5110565"],"URL":"https:\/\/doi.org\/10.1023\/a:1021713901879","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2003,3]]},"assertion":[{"value":"This content has been made available to all.","name":"free","label":"Free to read"}]}}