{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T04:18:02Z","timestamp":1778559482017,"version":"3.51.4"},"reference-count":46,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,11,13]],"date-time":"2020-11-13T00:00:00Z","timestamp":1605225600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["315550"],"award-info":[{"award-number":["315550"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["311877"],"award-info":[{"award-number":["311877"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Minimal Learning Machine (MLM) is a recently popularized supervised learning method, which is composed of distance-regression and multilateration steps. The computational complexity of MLM is dominated by the solution of an ordinary least-squares problem. Several different solvers can be applied to the resulting linear problem. In this paper, a thorough comparison of possible and recently proposed, especially randomized, algorithms is carried out for this problem with a representative set of regression datasets. In addition, we compare MLM with shallow and deep feedforward neural network models and study the effects of the number of observations and the number of features with a special dataset. To our knowledge, this is the first time that both scalability and accuracy of such a distance-regression model are being compared to this extent. We expect our results to be useful on shedding light on the capabilities of MLM and in assessing what solution algorithms can improve the efficiency of MLM. We conclude that (i) randomized solvers are an attractive option when the computing time or resources are limited and (ii) MLM can be used as an out-of-the-box tool especially for high-dimensional problems.<\/jats:p>","DOI":"10.3390\/make2040029","type":"journal-article","created":{"date-parts":[[2020,11,13]],"date-time":"2020-11-13T08:44:02Z","timestamp":1605257042000},"page":"533-557","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2573-1240","authenticated-orcid":false,"given":"Joakim","family":"Linja","sequence":"first","affiliation":[{"name":"Faculty of Information Technology, University of Jyv\u00e4skyl\u00e4, P.O. Box 35, University of Jyv\u00e4skyl\u00e4, FI-40014 Jyv\u00e4skyl\u00e4, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8466-9232","authenticated-orcid":false,"given":"Joonas","family":"H\u00e4m\u00e4l\u00e4inen","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, University of Jyv\u00e4skyl\u00e4, P.O. Box 35, University of Jyv\u00e4skyl\u00e4, FI-40014 Jyv\u00e4skyl\u00e4, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3581-3736","authenticated-orcid":false,"given":"Paavo","family":"Nieminen","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, University of Jyv\u00e4skyl\u00e4, P.O. Box 35, University of Jyv\u00e4skyl\u00e4, FI-40014 Jyv\u00e4skyl\u00e4, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0327-1167","authenticated-orcid":false,"given":"Tommi","family":"K\u00e4rkk\u00e4inen","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, University of Jyv\u00e4skyl\u00e4, P.O. Box 35, University of Jyv\u00e4skyl\u00e4, FI-40014 Jyv\u00e4skyl\u00e4, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"De Souza Junior, A.H., Corona, F., Miche, Y., Lendasse, A., Barreto, G.A., and Simula, O. (2013). Minimal Learning Machine: A New Distance-Based Method for supervised Learning. International Work-Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-642-38679-4_40"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.neucom.2014.11.073","article-title":"Minimal Learning Machine: A novel supervised distance-based approach for regression and classification","volume":"164","author":"Corona","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_3","first-page":"23","article-title":"Monte Carlo Simulations of Au38(SCH3)24 Nanocluster Using Distance-Based Machine Learning Methods","volume":"124","author":"Linja","year":"2020","journal-title":"J. Phys. Chem."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.eswa.2016.12.007","article-title":"A novel mobile robot localization approach based on topological maps using classification with reject option in omnidirectional images","volume":"72","author":"Marinho","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Coelho, D.N., Barreto, G.A., Medeiros, C.M.S., and Santos, J.D.A. (2014, January 9\u201312). Performance comparison of classifiers in the detection of short circuit incipient fault in a three-phase induction motor. Proceedings of the 2014 IEEE Symposium on Computational Intelligence for Engineering Solutions (CIES), Orlando, FL, USA.","DOI":"10.1109\/CIES.2014.7011829"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1049\/el:20010121","article-title":"Automatic pattern recognition by similarity representations","volume":"37","author":"Pekalska","year":"2001","journal-title":"Electron. Lett."},{"key":"ref_7","first-page":"175","article-title":"A generalized kernel approach to dissimilarity-based classification","volume":"2","author":"Pekalska","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/s10994-008-5059-5","article-title":"A theory of learning with similarity functions","volume":"72","author":"Balcan","year":"2008","journal-title":"Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.chemolab.2011.11.005","article-title":"Dissimilarity partial least squares applied to non-linear modeling problems","volume":"110","author":"Zerzucha","year":"2012","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sanchez, J.D., R\u00eago, L.C., and Ospina, R. (2019). Prediction by Empirical Similarity via Categorical Regressors. Mach. Learn. Knowl. Extr., 1.","DOI":"10.3390\/make1020038"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.neucom.2018.12.078","article-title":"Extreme minimal learning machine: Ridge regression with distance-based basis","volume":"342","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_12","unstructured":"H\u00e4m\u00e4l\u00e4inen, J., Alencar, A.S., K\u00e4rkk\u00e4inen, T., Mattos, C.L., J\u00fanior, A.H.S., and Gomes, J.P. (2019). Minimal Learning Machine: Theoretical Results and Clustering-Based Reference Point Selection. arXiv, To appear."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1016\/j.neucom.2020.03.088","article-title":"A new perspective for Minimal Learning Machines: A lightweight approach","volume":"401","author":"Oliveira","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"H\u00e4m\u00e4l\u00e4inen, J., and K\u00e4rkk\u00e4inen, T. (2020). Newton\u2019s Method for Minimal Learning Machine. Computational Sciences and Artificial Intelligence in Industry\u2014New Digital Technologies for Solving Future Societal and Economical Challenges, Springer Nature.","DOI":"10.1007\/978-3-030-70787-3_7"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_16","unstructured":"Dias, M.L.D., de Souza, L.S., da Rocha Neto, A.R., and de Souza Junior, A.H. (2018, January 25\u201327). Opposite neighborhood: A new method to select reference points of minimal learning machines. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning\u2014ESANN, Bruges, Belgium."},{"key":"ref_17","unstructured":"Barreto, G.A., and Coelho, R. (2018). A Fuzzy C-Means-based Approach for Selecting Reference Points in Minimal Learning Machines, Springer."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1007\/s11063-017-9587-5","article-title":"Ensemble of Efficient Minimal Learning Machines for Classification and Regression","volume":"46","author":"Mesquita","year":"2017","journal-title":"Neural Process. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.neucom.2015.03.107","article-title":"Singular value decomposition update and its application to (Inc)-OP-ELM","volume":"174","author":"Grigorievskiy","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.acha.2010.02.003","article-title":"A randomized algorithm for the decomposition of matrices","volume":"30","author":"Martinsson","year":"2011","journal-title":"Appl. Comput. Harmon. Anal."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.acha.2016.04.006","article-title":"Randomized LU decomposition","volume":"44","author":"Shabat","year":"2018","journal-title":"Appl. Comput. Harmon. Anal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2700","DOI":"10.1109\/TPDS.2018.2842785","article-title":"Analysis and Design Techniques towards High-Performance and Energy-Efficient Dense Linear Solvers on GPUs","volume":"29","author":"Abdelfattah","year":"2018","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/0304-3975(85)90224-5","article-title":"Clustering to minimize the maximum intercluster distance","volume":"38","author":"Gonzalez","year":"1985","journal-title":"Theor. Comput. Sci."},{"key":"ref_24","unstructured":"Huang, G.B., Zhu, Q.Y., and Siew, C.K. (2004, January 25\u201329). Extreme learning machine: A new learning scheme of feedforward neural networks. Proceedings of the 2004 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary."},{"key":"ref_25","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_26","unstructured":"Oliphant, T.E. (2006). A Guide to NumPy, Trelgol Publishing."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., and Bright, J. (2019). SciPy 1.0\u2013Fundamental Algorithms for Scientific Computing in Python. arXiv.","DOI":"10.1038\/s41592-020-0772-5"},{"key":"ref_28","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1137\/1035134","article-title":"On the Early History of the Singular Value Decomposition","volume":"35","author":"Stewart","year":"1993","journal-title":"SIAM Rev."},{"key":"ref_30","unstructured":"Tikhonov, A.N., and Arsenin, V.J. (1977). Solution of Ill-Posed Problems, Winston&Sons."},{"key":"ref_31","unstructured":"Halko, N., Martinsson, P.G., and Tropp, J.A. (2009). Finding Structure with Randomness: Stochastic Algorithms for Constructing Approximate matrix Decompositions. ACM Tech. Rep."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1080\/01621459.1952.10483441","article-title":"Use of ranks in one-criterion variance analysis","volume":"47","author":"Kruskal","year":"1952","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Juarez-Mosqueda, R., Sami, M., and Hannu, H. (2019). Ab initio molecular dynamics studies of Au38(SR)24 isomers under heating. Eur. Phys. J., 73.","DOI":"10.1140\/epjd\/e2019-90441-5"},{"key":"ref_34","unstructured":"Dua, D., and Graff, C. (2020, November 13). UCI Machine Learning Repository. Available online: http:\/\/archive.ics.uci.edu\/ml."},{"key":"ref_35","unstructured":"Torgo, L. (1991). Airplane Companies Stocks, Faculdade de Ci\u00eancias da Universidade do Porto."},{"key":"ref_36","unstructured":"University of Toronto (1996). Delve Datasets, University of Toronto."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e12400","DOI":"10.1111\/exsy.12400","article-title":"A systematic review on deep learning architectures and applications","volume":"36","author":"Khamparia","year":"2019","journal-title":"Expert Syst."},{"key":"ref_39","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Emmert-Streib, F., and Dehmer, M. (2019). Understanding Statistical Hypothesis Testing: The Logic of Statistical Inference. Mach. Learn. Knowl. Extr., 1.","DOI":"10.3390\/make1030054"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1093\/nsr\/nwy108","article-title":"Deep forest","volume":"6","author":"Zhou","year":"2018","journal-title":"Natl. Sci. Rev."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1016\/j.amc.2011.05.036","article-title":"Chebyshev-type methods and preconditioning techniques","volume":"218","author":"Li","year":"2011","journal-title":"Appl. Math. Comput."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1090\/S0025-5718-1978-0483340-6","article-title":"Accelerated overrelaxation method","volume":"32","author":"Hadjidimos","year":"1978","journal-title":"Math. Comput."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"8280","DOI":"10.1021\/ja103592z","article-title":"Total Structure Determination of Thiolate-Protected Au38 Nanoparticles","volume":"132","author":"Qian","year":"2010","journal-title":"J. Am. Chem. Soc."},{"key":"ref_45","unstructured":"Huo, H., and Rupp, M. (2017). Unified Representation of Molecules and Crystals for Machine Learning. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"106949","DOI":"10.1016\/j.cpc.2019.106949","article-title":"DScribe: Library of descriptors for machine learning in materials science","volume":"247","author":"Himanen","year":"2020","journal-title":"Comput. Phys. Commun."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/2\/4\/29\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:32:59Z","timestamp":1760178779000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/2\/4\/29"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,13]]},"references-count":46,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["make2040029"],"URL":"https:\/\/doi.org\/10.3390\/make2040029","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,13]]}}}