{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T23:08:08Z","timestamp":1777417688640,"version":"3.51.4"},"reference-count":82,"publisher":"MIT Press - Journals","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2015,2]]},"abstract":"<jats:p> The mathematical foundations of a new theory for the design of intelligent agents are presented. The proposed learning paradigm is centered around the concept of constraint, representing the interactions with the environment, and the parsimony principle. The classical regularization framework of kernel machines is naturally extended to the case in which the agents interact with a richer environment, where abstract granules of knowledge, compactly described by different linguistic formalisms, can be translated into the unified notion of constraint for defining the hypothesis set. Constrained variational calculus is exploited to derive general representation theorems that provide a description of the optimal body of the agent (i.e., the functional structure of the optimal solution to the learning problem), which is the basis for devising new learning algorithms. We show that regardless of the kind of constraints, the optimal body of the agent is a support constraint machine (SCM) based on representer theorems that extend classical results for kernel machines and provide new representations. In a sense, the expressiveness of constraints yields a semantic-based regularization theory, which strongly restricts the hypothesis set of classical regularization. Some guidelines to unify continuous and discrete computational mechanisms are given so as to accommodate in the same framework various kinds of stimuli, for example, supervised examples and logic predicates. The proposed view of learning from constraints incorporates classical learning from examples and extends naturally to the case in which the examples are subsets of the input space, which is related to learning propositional logic clauses. <\/jats:p>","DOI":"10.1162\/neco_a_00686","type":"journal-article","created":{"date-parts":[[2014,11,7]],"date-time":"2014-11-07T19:42:44Z","timestamp":1415389364000},"page":"388-480","source":"Crossref","is-referenced-by-count":37,"title":["Foundations of Support Constraint Machines"],"prefix":"10.1162","volume":"27","author":[{"given":"Giorgio","family":"Gnecco","sequence":"first","affiliation":[{"name":"Institute for Advanced Studies, 55100 Lucca, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marco","family":"Gori","sequence":"additional","affiliation":[{"name":"Department of Information Engineering and Mathematics, University of Siena, 53100 Siena, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefano","family":"Melacci","sequence":"additional","affiliation":[{"name":"Department of Information Engineering and Mathematics, University of Siena, 53100 Siena, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcello","family":"Sanguineti","sequence":"additional","affiliation":[{"name":"Department of Informatics, Bioengineering, Robotics, and System Engineering, University of Genoa, 16145 Genova, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","volume-title":"Sobolev spaces","author":"Adams R. A.","year":"2003","edition":"2"},{"key":"B2","first-page":"25","volume-title":"Advances in neural information processing systems","author":"Argyriou A.","year":"2008"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718782"},{"key":"B5","volume-title":"Variational principles in physics","author":"Basdevant J.-L.","year":"2006"},{"key":"B6","first-page":"2399","volume":"7","author":"Belkin M.","year":"2006","journal-title":"J. Machine Learning Research"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-9096-9"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1887\/0750304359"},{"key":"B9","volume-title":"Nonlinear programming","author":"Bertsekas D.","year":"1999","edition":"2"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-45528-0"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511804441"},{"key":"B12","volume-title":"Products and convolutions of gaussian distributions","author":"Bromiley P.","year":"2003"},{"key":"B13","first-page":"1615","volume":"9","author":"Caponnetto A.","year":"2008","journal-title":"J. Machine Learning Research"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007379606734"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1145\/321356.321363"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760805296"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-78652-8"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23954-0_7"},{"key":"B19","unstructured":"Diligenti, M., Gori, M., Maggini, M. & Rigutini, L. (2010). Multitask kernel-based learning with logic constraints. In Proc. 19th European Conf. on Artificial Intelligence (pp. 433\u2013438). Berlin: Springer."},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-011-5243-x"},{"key":"B21","first-page":"189","volume-title":"Advances in neural information processing systems, 25","author":"Dinuzzo F.","year":"2012"},{"key":"B22","volume-title":"Sobolev spaces of infinite order and differential equations","author":"Dubinskij J. A.","year":"1986"},{"key":"B23","volume-title":"Differential equations and calculus of variations.","author":"Ernestovic E. L.","year":"1970"},{"key":"B24","volume-title":"Partial differential equations.","author":"Evans L. C.","year":"2000"},{"key":"B25","first-page":"615","volume":"6","author":"Evgeniou T.","year":"2005","journal-title":"J. Machine Learning Research"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018946025316"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1007\/s00211-011-0391-2"},{"key":"B28","volume-title":"Proc. NeSy Workshop\u2014IJCAI2013.","author":"Frandina S.","year":"2013"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40728-4_11"},{"key":"B30","first-page":"23","volume-title":"Proc. CoCoMile Workshop, 20th European Conf. on Artificial Intelligence","author":"Frandina S.","year":"2012"},{"key":"B31","volume-title":"Calculus of variations.","author":"Gelfand I. M.","year":"1963"},{"key":"B32","volume-title":"Calculus of variations I","author":"Giaquinta M.","year":"1996"},{"key":"B33","first-page":"97","volume-title":"Artificial neural networks for speech and vision","author":"Girosi F.","year":"1993"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1995.7.2.219"},{"key":"B35","volume-title":"Genius: The life and science of Richard Feynman.","author":"Gleick J.","year":"1992"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40728-4_19"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2012.06.065"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00417"},{"key":"B39","first-page":"153","volume":"2","author":"Gnecco G.","year":"2008","journal-title":"Applied Mathematical Sciences"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2009.06.048"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15825-4_41"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2241787"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_62"},{"key":"B45","volume-title":"Tables of integrals, series and products: Corrected and enlarged edition prepared by A. Jeffrey.","author":"Gradshteyn I. S.","year":"1980"},{"key":"B46","first-page":"20","volume-title":"Computing and Phylosophy: AISB 2008 Proc.","volume":"12","author":"Guerin F.","year":"2008"},{"key":"B47","first-page":"49","volume":"13","author":"Hadamard J.","year":"1902","journal-title":"Princeton University Bulletin"},{"key":"B48","first-page":"175","volume":"3","author":"Herbrich R.","year":"2002","journal-title":"J. Machine Learning Research"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(81)90024-2"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1109\/34.291440"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1037\/10034-000"},{"key":"B52","first-page":"201","volume":"5","author":"Ivaz K.","year":"2006","journal-title":"Applied and Computational Mathematics"},{"key":"B53","volume-title":"Statistical and computational inverse problems.","author":"Kaipio J.","year":"1994"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-015-9540-7"},{"key":"B55","first-page":"1","volume":"1","author":"Kolmogorov A.","year":"1965","journal-title":"Problems of Information Transmission"},{"key":"B56","volume-title":"Introductory functional analysis with applications.","author":"Kreyszig E.","year":"1989"},{"key":"B57","first-page":"959","volume":"10","author":"Lishan L.","year":"1996","journal-title":"Indian J. Pure and Applied Mathematics"},{"key":"B58","volume-title":"Optimization by vector space methods.","author":"Luenberger D. G.","year":"1969"},{"key":"B59","first-page":"1149","volume":"12","author":"Melacci S.","year":"2011","journal-title":"J. Machine Learning Research"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23954-0_5"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.73"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04277-5_66"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36434-X_1"},{"key":"B64","doi-asserted-by":"publisher","DOI":"10.1162\/0899766052530802"},{"key":"B65","volume-title":"La psychologie de l\u2019intelligence.","author":"Piaget J.","year":"1961"},{"key":"B66","volume-title":"A theory of networks for approximation and learning","author":"Poggio T.","year":"1989"},{"key":"B67","volume-title":"Proc. 9th Workshop on Mining and Learning with Graphs.","author":"Sacc\u00e0 C.","year":"2011"},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2011.156"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-15-103"},{"key":"B70","volume-title":"Advances in neural information processing systems","volume":"10","author":"Sch\u00f6lkopf B.","year":"1998"},{"key":"B71","volume-title":"Learning with kernels.","author":"Sch\u00f6lkopf B.","year":"2002"},{"key":"B72","volume-title":"Th\u00e9orie des distributions.","author":"Schwartz L.","year":"1978"},{"key":"B73","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-00616-6_12"},{"key":"B74","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(98)00032-X"},{"key":"B75","first-page":"1","volume-title":"Information and control","author":"Solomonoff R.","year":"1964"},{"key":"B76","volume-title":"Singular integrals and differentiability properties of functions.","author":"Stein E. M.","year":"1970"},{"key":"B77","volume-title":"Solution of ill-posed problems.","author":"Tikhonov A. N.","year":"1977"},{"key":"B78","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2004.02.007"},{"key":"B79","volume-title":"The calculus of variations.","author":"van Brunt B.","year":"2003"},{"key":"B80","volume-title":"Statistical learning theory.","author":"Vapnik V. N.","year":"1998"},{"key":"B81","first-page":"183","volume":"23","author":"Wahba G.","year":"1975","journal-title":"Numerische Mathematik"},{"key":"B82","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611970128"},{"key":"B83","doi-asserted-by":"publisher","DOI":"10.1109\/CCV.1988.590011"},{"key":"B84","doi-asserted-by":"publisher","DOI":"10.1007\/BF00126430"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00686","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:40:38Z","timestamp":1615585238000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/27\/2\/388-480\/8052"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,2]]},"references-count":82,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2015,2]]}},"alternative-id":["10.1162\/NECO_a_00686"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00686","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,2]]}}}