{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T16:40:29Z","timestamp":1715964029896},"reference-count":57,"publisher":"MIT Press","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2013,11]]},"abstract":"<jats:p>We present a supervised learning algorithm for estimation of generic input-output relations in a real-time, online fashion. The proposed method is based on a generalized expectation-maximization approach to fit an infinite mixture of linear experts (IMLE) to an online stream of data samples. This probabilistic model, while not fully Bayesian, can efficiently choose the number of experts that are allocated to the mixture, this way effectively controlling the complexity of the resulting model. The result is an incremental, online, and localized learning algorithm that performs nonlinear, multivariate regression on multivariate outputs by approximating the target function by a linear relation within each expert input domain and that can allocate new experts as needed. A distinctive feature of the proposed method is the ability to learn multivalued functions: one-to-many mappings that naturally arise in some robotic and computer vision learning domains, using an approach based on a Bayesian generative model for the predictions provided by each of the mixture experts. As a consequence, it is able to directly provide forward and inverse relations from the same learned mixture model. We conduct an extensive set of experiments to evaluate the proposed algorithm performance, and the results show that it can outperform state-of-the-art online function approximation algorithms in single-valued regression, while demonstrating good estimation capabilities in a multivalued function approximation context.<\/jats:p>","DOI":"10.1162\/neco_a_00510","type":"journal-article","created":{"date-parts":[[2013,9,3]],"date-time":"2013-09-03T15:41:03Z","timestamp":1378222863000},"page":"3044-3091","source":"Crossref","is-referenced-by-count":8,"title":["Online Learning of Single- and Multivalued Functions with an Infinite Mixture of Linear Experts"],"prefix":"10.1162","volume":"25","author":[{"given":"Bruno","family":"Damas","sequence":"first","affiliation":[{"name":"Instituto de Sistemas e Rob\u00f3tica IST, 1049-001 Lisboa, Portugal, and Escola Superior de Tecnologia IPS, 2910-761 Set\u00fabal, Portugal"}]},{"given":"Jos\u00e9","family":"Santos-Victor","sequence":"additional","affiliation":[{"name":"Instituto de Sistemas e Rob\u00f3tica IST, 1049-001 Lisboa, Portugal"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176342871"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1006559212014"},{"key":"B3","volume-title":"Mixture density networks","author":"Bishop C.","year":"1994"},{"key":"B4","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198538493.001.0001","volume-title":"Neural networks for pattern recognition","author":"Bishop C.","year":"1995"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2003.10.012"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2009.00698.x"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1109\/34.888716"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2008.4650995"},{"key":"B9","volume-title":"Introduction to robotics: Mechanics and control","author":"Craig J.","year":"1989"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1162\/089976602317250933"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1977.tb01600.x"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2001.973374"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2003.1227989"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/34.990138"},{"key":"B15","volume-title":"Bayesian data analysis","author":"Gelman A.","year":"2004"},{"key":"B16","volume-title":"The EM algorithm for mixtures of factor analyzers","author":"Ghahramani Z.","year":"1997"},{"key":"B17","volume-title":"Advances in neural information processing systems, 6","author":"Ghahramani Z.","year":"1994"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2008.4587370"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2010.5650500"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1177011002"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1970.10488634"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.1.79"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1994.6.2.181"},{"key":"B24","volume-title":"Kendall's advanced theory of statistics, vol. 2. Classical inference and the linear model","author":"Kendall M. G.","year":"1998"},{"key":"B25","first-page":"93","volume-title":"Proceedings of the International Joint Conference on Neural Networks, 2001","volume":"1","author":"Lee K.","year":"2001"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2007.4399566"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1162\/089976603322385117"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1002\/0471721182"},{"key":"B29","first-page":"883","volume-title":"Advances in neural information processing systems, 18","author":"Meeds E.","year":"2006"},{"key":"B30","volume-title":"Parallel robots","author":"Merlet J.","year":"2006"},{"key":"B31","first-page":"355","volume-title":"Learning in graphical models","author":"Neal R.","year":"1999"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2008.4650850"},{"key":"B33","volume-title":"Kendall's advanced theory of statistics: Vol. 2B; Bayesian inference","author":"O'Hagan A.","year":"1994"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2006.II.033"},{"key":"B35","doi-asserted-by":"crossref","first-page":"1979","DOI":"10.1109\/ROBOT.2008.4543497","volume-title":"IEEE International Conference on Robotics and Automation, 2008","author":"Qin C.","year":"2008"},{"key":"B36","first-page":"1939","volume":"6","author":"Qui\u00f1onero-Candela J.","year":"2005","journal-title":"Journal of Machine Learning Research"},{"key":"B37","first-page":"554","volume-title":"Advances in neural information processing systems, 12","author":"Rasmussen C.","year":"2000"},{"key":"B39","volume-title":"Gaussian processes for machine learning","author":"Rasmussen C.","year":"2006"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1214\/009053605000000417"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/ICHR.2004.1442111"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1162\/089976601750265045"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015853"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1162\/089976698300016963"},{"key":"B45","first-page":"633","volume-title":"Advances in neural information processing systems, 10","author":"Schaal S.","year":"1998"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1109\/ICNN.1994.374152"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1002\/ecjc.4430790910"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1023\/B:STCO.0000035301.49549.88"},{"key":"B49","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"Tibshirani R.","year":"1996","journal-title":"Journal of the Royal Statistical Society (B)"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2009.02-08-702"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102465"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1162\/089976605774320557"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013258808932"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013844811137"},{"key":"B56","first-page":"422","volume-title":"Advances in neural information processing systems, 25","author":"Wang C.","year":"2012"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1007\/BF00058926"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1023\/A:1016535925043"},{"key":"B59","first-page":"633","volume-title":"Advances in neural information processing systems, 7","author":"Xu L.","year":"1995"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00510","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T16:02:37Z","timestamp":1715961757000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/25\/11\/3044-3091\/7931"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,11]]},"references-count":57,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2013,11]]}},"alternative-id":["10.1162\/NECO_a_00510"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00510","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,11]]}}}