{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T03:44:13Z","timestamp":1783741453365,"version":"3.55.0"},"reference-count":124,"publisher":"Annual Reviews","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Annu. Rev. Control Robot. Auton. Syst."],"published-print":{"date-parts":[[2019,5,3]]},"abstract":"<jats:p>Estimation of functions from sparse and noisy data is a central theme in machine learning. In the last few years, many algorithms have been developed that exploit Tikhonov regularization theory and reproducing kernel Hilbert spaces. These are the so-called kernel-based methods, which include powerful approaches like regularization networks, support vector machines, and Gaussian regression. Recently, these techniques have also gained popularity in the system identification community. In both linear and nonlinear settings, kernels that incorporate information on dynamic systems, such as the smoothness and stability of the input\u2013output map, can challenge consolidated approaches based on parametric model structures. In the classical parametric setting, the complexity of the model (the model order) needs to be chosen, typically from a finite family of alternatives, by trading bias and variance. This (discrete) model order selection step may be critical, especially when the true model does not belong to the model class. In regularization-based approaches, model complexity is controlled by tuning (continuous) regularization parameters, making the model selection step more robust. In this article, we review these new kernel-based system identification approaches and discuss extensions based on nuclear and [Formula: see text] norms.<\/jats:p>","DOI":"10.1146\/annurev-control-053018-023744","type":"journal-article","created":{"date-parts":[[2018,11,28]],"date-time":"2018-11-28T22:55:01Z","timestamp":1543445701000},"page":"281-304","source":"Crossref","is-referenced-by-count":121,"title":["System Identification: A Machine Learning Perspective"],"prefix":"10.1146","volume":"2","author":[{"given":"A.","family":"Chiuso","sequence":"first","affiliation":[{"name":"Department of Information Engineering, University of Padova, 35131 Padova, Italy;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G.","family":"Pillonetto","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Padova, 35131 Padova, Italy;"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"22","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.1974.1100705"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176344136"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1214\/09-SS054"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-21606-5"},{"key":"B5","volume-title":"Neural Networks for Pattern Recognition","author":"Bishops C","year":"1996"},{"key":"B6","volume-title":"Neural Networks and Learning Machines","author":"Haykin S","year":"2009"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"B8","volume-title":"Solutions of Ill-Posed Problems","author":"Tikhonov A","year":"1977"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1016\/S0065-2539(08)60946-4"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9947-1950-0051437-7"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1090\/surv\/005"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/5.5962"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611970128"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/5.58326"},{"key":"B15","unstructured":"15.\u2002 Girosi F. 1997.An equivalence between sparse approximation and support vector machines. AI Memo 1606, CBCL Pap. 147, Mass. Inst. Technol., Cambridge, MA"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177697089"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9947-01-02852-5"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/20\/3\/001"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2014.2337939"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018946025316"},{"key":"B21","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/4175.001.0001","volume-title":"Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond","author":"Sch\u00f6lkopf B","year":"2001"},{"key":"B22","first-page":"155","volume-title":"Advances in Neural Information Processing Systems 9","author":"Drucker H","year":"1997"},{"key":"B23","volume-title":"Statistical Learning Theory","author":"Vapnik V","year":"1998"},{"key":"B24","volume-title":"Gaussian Processes for Machine Learning","author":"Rasmussen C","year":"2006"},{"key":"B25","first-page":"143","volume":"1","author":"Collobert R","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"B26","volume-title":"Empirical Bayes Methods","author":"Maritz JS","year":"1989"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.3182\/20120711-3-BE-2027.00353"},{"key":"B28","first-page":"217","volume":"15","author":"Aravkin A","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1137\/0714044"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1007\/s00365-006-0659-y"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1214\/09-AOS772"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-004-0155-9"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1007\/s10444-004-7634-z"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1038\/nature02341"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1145\/263867.263927"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-46769-6_9"},{"key":"B37","first-page":"499","volume":"2","author":"Bousquet O","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1016\/0165-1765(79)90025-9"},{"key":"B39","unstructured":"39.\u2002 Akaike H. 1979.Smoothness priors and the distributed lag estimator. Tech. Rep. 40, Dep. Stat., Stanford Univ., Stanford, CA"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0761-0"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2016.04.013"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/9.148344"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2014.7039906"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-012-9135-7"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1137\/090755436"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2009.5400711"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1109\/ACC.2010.5531594"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2014.2351711"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2016.02.012"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.12.3097"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1109\/72.548162"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2002.804313"},{"key":"B53","volume-title":"Local Polynomial Modelling and Its Applications","author":"Fan J","year":"1996"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2005.849842"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.23919\/ECC.2003.7085352"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2011.2131830"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2014.10.069"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2004.11.010"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2007.900826"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2010.2042343"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1142\/5089"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1214\/09-SS052"},{"key":"B63","first-page":"3156","volume-title":"Advances in Neural Information Processing Systems 26","author":"Frigola R","year":"2013"},{"key":"B64","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2013.6760734"},{"key":"B65","volume-title":"System Identification: Theory for the User","author":"Ljung L","year":"1999"},{"key":"B66","volume-title":"Proceedings of the 2014 International Conference of the System Dynamics Society","author":"Pruyt E","year":"2014"},{"key":"B67","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2009.10.031"},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2010.11.004"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2012.05.026"},{"key":"B70","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-006-9352-0"},{"key":"B71","doi-asserted-by":"publisher","DOI":"10.1142\/S0219530506000838"},{"key":"B72","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2014.01.001"},{"key":"B73","doi-asserted-by":"publisher","DOI":"10.1137\/130920319"},{"key":"B74","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2018.03.065"},{"key":"B75","doi-asserted-by":"publisher","DOI":"10.1090\/S0273-0979-01-00923-5"},{"key":"B76","first-page":"748","volume-title":"Proceedings of the 31st International Conference on Machine Learning","author":"Argyriou A","year":"2014"},{"key":"B77","doi-asserted-by":"publisher","DOI":"10.1109\/ACC.2010.5530862"},{"key":"B78","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2018.04.015"},{"key":"B79","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1974.10489157"},{"key":"B80","doi-asserted-by":"publisher","DOI":"10.1016\/0005-1098(96)00003-9"},{"key":"B81","doi-asserted-by":"publisher","DOI":"10.1007\/BF01404567"},{"key":"B82","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1979.10489751"},{"key":"B83","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2014.2337939"},{"key":"B84","doi-asserted-by":"publisher","DOI":"10.1119\/1.1990764"},{"key":"B85","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.3.415"},{"key":"B86","doi-asserted-by":"publisher","DOI":"10.1016\/S0005-1098(96)00254-3"},{"key":"B87","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2015.05.012"},{"key":"B88","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-4485-6"},{"key":"B89","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2009.10.015"},{"key":"B90","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2009.00736.x"},{"key":"B91","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2016.01.036"},{"key":"B92","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2007.19.6.1633"},{"key":"B93","first-page":"2467","volume":"8","author":"Dinuzzo F","year":"2007","journal-title":"J. Mach. Learn. Res."},{"key":"B94","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-008-5095-1"},{"key":"B95","first-page":"19","volume":"11","author":"Mairal J","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"B96","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2006.871582"},{"key":"B97","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","volume":"58","author":"Tibshirani R","year":"1996","journal-title":"J. R. Stat. Soc. B"},{"key":"B98","doi-asserted-by":"publisher","DOI":"10.1214\/009053604000000067"},{"key":"B99","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2005.00532.x"},{"key":"B100","doi-asserted-by":"publisher","DOI":"10.1198\/016214506000000735"},{"key":"B101","first-page":"3704","volume":"100","author":"Mackay D","year":"1994","journal-title":"ASHRAE Trans."},{"key":"B102","first-page":"1625","volume-title":"Advances in Neural Information Processing Systems 20","author":"Wipf D","year":"2007"},{"key":"B103","first-page":"211","volume":"1","author":"Tipping M","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"B104","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1988.10478694"},{"key":"B105","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2010.2042413"},{"key":"B106","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2010.2042347"},{"key":"B107","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2012.05.054"},{"key":"B108","doi-asserted-by":"publisher","DOI":"10.1016\/j.ifacol.2018.09.118"},{"key":"B109","doi-asserted-by":"publisher","DOI":"10.1016\/S0019-9958(63)90092-5"},{"key":"B110","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2007.06.010"},{"key":"B111","doi-asserted-by":"publisher","DOI":"10.1080\/00207720802083018"},{"key":"B112","doi-asserted-by":"publisher","DOI":"10.1137\/110853996"},{"key":"B113","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2017.01.020"},{"key":"B114","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2015.2491678"},{"key":"B115","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2016.08.014"},{"key":"B116","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2016.03.011"},{"key":"B117","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2005.02.002"},{"key":"B118","doi-asserted-by":"publisher","DOI":"10.3182\/20090706-3-FR-2004.00136"},{"key":"B119","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84996-513-2_15"},{"key":"B120","doi-asserted-by":"publisher","DOI":"10.1016\/j.conengprac.2012.05.006"},{"key":"B121","doi-asserted-by":"publisher","DOI":"10.3182\/20120711-3-BE-2027.00274"},{"key":"B122","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.1970.1099514"},{"key":"B123","doi-asserted-by":"publisher","DOI":"10.1109\/ACC.2001.945730"},{"key":"B124","volume-title":"Real and Complex Analysis","author":"Rudin W","year":"1987"}],"container-title":["Annual Review of Control, Robotics, and Autonomous Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.annualreviews.org\/doi\/pdf\/10.1146\/annurev-control-053018-023744","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T21:33:53Z","timestamp":1720820033000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.annualreviews.org\/doi\/10.1146\/annurev-control-053018-023744"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,3]]},"references-count":124,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,5,3]]}},"alternative-id":["10.1146\/annurev-control-053018-023744"],"URL":"https:\/\/doi.org\/10.1146\/annurev-control-053018-023744","relation":{},"ISSN":["2573-5144","2573-5144"],"issn-type":[{"value":"2573-5144","type":"print"},{"value":"2573-5144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,3]]}}}