{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,26]],"date-time":"2026-07-26T02:52:04Z","timestamp":1785034324792,"version":"3.55.0"},"reference-count":59,"publisher":"MIT Press","issue":"1","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce, for example, monotonicity of the function over selected inputs. The aim is to find models which conform to expected behavior and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shape-constrained symbolic regression: (i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and (ii) a two-population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.<\/jats:p>","DOI":"10.1162\/evco_a_00294","type":"journal-article","created":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T00:51:02Z","timestamp":1618447862000},"page":"75-98","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":65,"title":["Shape-Constrained Symbolic Regression\u2014Improving Extrapolation with Prior Knowledge"],"prefix":"10.1162","volume":"30","author":[{"given":"G.","family":"Kronberger","sequence":"first","affiliation":[{"name":"Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria gabriel.kronberger@fh-ooe.at"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"F. O.","family":"de Franca","sequence":"additional","affiliation":[{"name":"Center for Mathematics, Computation and Cognition (CMCC), Heuristics, Analysis and Learning Laboratory (HAL), Federal University of ABC, Santo Andre, Brazil folivetti@ufabc.edu.br"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"B.","family":"Burlacu","sequence":"additional","affiliation":[{"name":"Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria bogdan.burlacu@fh-hagenberg.at"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C.","family":"Haider","sequence":"additional","affiliation":[{"name":"Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria christian.haider@fh-hagenberg.at"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M.","family":"Kommenda","sequence":"additional","affiliation":[{"name":"Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria michael.kommenda@fh-hagenberg.at"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2022,3,1]]},"reference":[{"issue":"2","key":"2022030107595088500_B1","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1137\/18M118935X","article-title":"DSOS and SDSOS optimization: More tractable alternatives to sum of squares and semidefinite optimization","volume":"3","author":"Ahmadi","year":"2019","journal-title":"SIAM Journal on Applied Algebra and Geometry"},{"key":"2022030107595088500_B2","volume-title":"Modern compressible flow: With historical perspective","author":"Anderson","year":"1982"},{"key":"2022030107595088500_B3","author":"Anderson","year":"2010","journal-title":"Fundamentals of aerodynamics"},{"key":"2022030107595088500_B4","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1145\/2576768.2598291","volume-title":"Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (GECCO)","author":"Arnaldo","year":"2014"},{"key":"2022030107595088500_B5","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1145\/3321707.3321743","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Bladek","year":"2019"},{"key":"2022030107595088500_B6","first-page":"1975","volume-title":"Genetic and Evolutionary Computation Conference","author":"Castillo","year":"2003"},{"key":"2022030107595088500_B7","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/978-1-4614-6846-2_10","volume-title":"Genetic Programming Theory and Practice X","author":"Castillo","year":"2013"},{"key":"2022030107595088500_B8","first-page":"109:25","article-title":"A multilevel block building algorithm for fast modeling generalized separable systems.","author":"Chen","year":"2018","journal-title":"Expert Systems with Applications"},{"key":"2022030107595088500_B9","first-page":"1","volume-title":"2018 IEEE Congress on Evolutionary Computation","author":"Cov\u00f5es","year":"2018"},{"key":"2022030107595088500_B10","article-title":"Shape-constrained regression using sum of squares polynomials.","author":"Curmei","year":"2020"},{"key":"2022030107595088500_B11","first-page":"442:18","article-title":"A greedy search tree heuristic for symbolic regression.","author":"de Fran\u00e7a","year":"2018","journal-title":"Information Sciences"},{"key":"2022030107595088500_B12","doi-asserted-by":"publisher","DOI":"10.1162\/evco_a_00285","article-title":"Interaction-transformation evolutionary algorithm for symbolic regression.","author":"de Franca","year":"2021","journal-title":"Evolutionary Computation"},{"key":"2022030107595088500_B13","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1145\/3067695.3076107","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference Companion","author":"Dick","year":"2017"},{"issue":"2","key":"2022030107595088500_B14","first-page":"87","article-title":"Gene expression programming: A new adaptive algorithm for solving problems","volume":"13","author":"Ferreira","year":"2001","journal-title":"Complex Systems"},{"key":"2022030107595088500_B15","first-page":"2962","volume-title":"Advances in neural information processing systems","author":"Feurer","year":"2015"},{"issue":"Jul","key":"2022030107595088500_B16","first-page":"2171","article-title":"Deap: Evolutionary algorithms made easy","volume":"13","author":"Fortin","year":"2012","journal-title":"Journal of Machine Learning Research"},{"issue":"4","key":"2022030107595088500_B17","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1214\/18-STS665","article-title":"Nonparametric shape-restricted regression","volume":"33","author":"Guntuboyina","year":"2018","journal-title":"Statistical Science"},{"issue":"109","key":"2022030107595088500_B18","first-page":"1","article-title":"Monotonic calibrated interpolated look-up tables","volume":"17","author":"Gupta","year":"2016","journal-title":"Journal of Machine Learning Research"},{"key":"2022030107595088500_B19","article-title":"Optimization over nonnegative and convex polynomials with and without semidefinite programming.","author":"Hall","year":"2018"},{"issue":"5","key":"2022030107595088500_B20","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1145\/502102.502106","article-title":"Interval arithmetic: From principles to implementation","volume":"48","author":"Hickey","year":"2001","journal-title":"Journal of the ACM (JACM)"},{"issue":"1","key":"2022030107595088500_B21","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1162\/evco.1997.5.1.31","article-title":"Empirical investigation of the benefits of partial Lamarckianism","volume":"5","author":"Houck","year":"1997","journal-title":"Evolutionary Computation"},{"issue":"1","key":"2022030107595088500_B22","doi-asserted-by":"crossref","DOI":"10.1063\/1.5112747","article-title":"Prediction of stress-strain curves for aluminium alloys using symbolic regression","volume":"2113","author":"Kabliman","year":"2019","journal-title":"AIP Conference Proceedings"},{"key":"2022030107595088500_B23","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1007\/3-540-36599-0_7","volume-title":"European Conference on Genetic Programming","author":"Keijzer","year":"2003"},{"key":"2022030107595088500_B24","article-title":"Integration of neural network-based symbolic regression in deep learning for scientific discovery.","author":"Kim","year":"2019"},{"issue":"2","key":"2022030107595088500_B25","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1016\/j.ejor.2007.06.028","article-title":"On a feasible\u2013infeasible two-population (FI-2Pop) genetic algorithm for constrained optimization: Distance tracing and no free lunch","volume":"190","author":"Kimbrough","year":"2008","journal-title":"European Journal of Operational Research"},{"issue":"3","key":"2022030107595088500_B26","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1007\/s10710-019-09371-3","article-title":"Parameter identification for symbolic regression using nonlinear least squares","volume":"21","author":"Kommenda","year":"2020","journal-title":"Genetic Programming and Evolvable Machines"},{"key":"2022030107595088500_B27","author":"Koza","year":"1992","journal-title":"Genetic programming: On the programming of computers by means of natural selection"},{"issue":"6","key":"2022030107595088500_B28","doi-asserted-by":"crossref","first-page":"1572","DOI":"10.1109\/TBME.2012.2189771","article-title":"Extrapolatable analytical functions for tendon excursions and moment arms from sparse datasets","volume":"59","author":"Kurse","year":"2012","journal-title":"IEEE Transactions on Biomedical Engineering"},{"key":"2022030107595088500_B29","article-title":"Neural-guided symbolic regression with semantic prior.","author":"Li","year":"2019"},{"key":"2022030107595088500_B30","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/978-3-319-16549-3_27","volume-title":"European Conference on the Applications of Evolutionary Computation","author":"Liapis","year":"2015"},{"issue":"1","key":"2022030107595088500_B31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.arcontrol.2009.12.001","article-title":"Perspectives on system identification","volume":"34","author":"Ljung","year":"2010","journal-title":"Annual Reviews in Control"},{"key":"2022030107595088500_B32","first-page":"4:274","article-title":"Two fast tree-creation algorithms for genetic programming.","author":"Luke","year":"2000","journal-title":"IEEE Transactions on Evolutionary Computation"},{"issue":"3","key":"2022030107595088500_B33","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1214\/ss\/1009213727","article-title":"A general projection framework for constrained smoothing","volume":"16","author":"Mammen","year":"2001","journal-title":"Statistical Science"},{"key":"2022030107595088500_B34","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1007\/978-1-4614-1770-5_13","volume-title":"Genetic Programming Theory and Practice IX","author":"McConaghy","year":"2011"},{"key":"2022030107595088500_B35","doi-asserted-by":"crossref","first-page":"2701","DOI":"10.1145\/1388969.1389075","volume-title":"Proceedings of the 10th Annual Conference Companion on Genetic and Evolutionary Computation","author":"Miller","year":"2008"},{"key":"2022030107595088500_B36","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1007\/978-3-642-32937-1_3","volume-title":"International Conference on Parallel Problem Solving from Nature","author":"Moraglio","year":"2012"},{"issue":"4","key":"2022030107595088500_B37","first-page":"285","article-title":"A comparison of several linear genetic programming techniques","volume":"14","author":"Oltean","year":"2003","journal-title":"Complex Systems"},{"issue":"4","key":"2022030107595088500_B38","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1109\/4235.942529","article-title":"Grammatical evolution","volume":"5","author":"O'Neill","year":"2001","journal-title":"IEEE Transactions on Evolutionary Computation"},{"key":"2022030107595088500_B39","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1145\/3205455.3205539","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Orzechowski","year":"2018"},{"issue":"1","key":"2022030107595088500_B40","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1080\/10618600.2012.707343","article-title":"Shape-constrained estimation using nonnegative splines","volume":"23","author":"Papp","year":"2014","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"2","key":"2022030107595088500_B41","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1162\/evco_a_00205","article-title":"Competent geometric semantic genetic programming for symbolic regression and Boolean function synthesis","volume":"26","author":"Pawlak","year":"2018","journal-title":"Evolutionary Computation"},{"key":"2022030107595088500_B42","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1145\/1830483.1830648","volume-title":"Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation","author":"Pennachin","year":"2010"},{"key":"2022030107595088500_B43","article-title":"Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients","author":"Petersen","year":"2021","journal-title":"Proceedings of the International Conference on Learning Representations"},{"key":"2022030107595088500_B44","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1007\/978-3-030-44094-7_11","volume-title":"Genetic programming","author":"Ruberto","year":"2020"},{"issue":"5","key":"2022030107595088500_B45","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nature Machine Intelligence"},{"key":"2022030107595088500_B46","article-title":"Learning equations for extrapolation and control.","author":"Sahoo","year":"2018"},{"key":"2022030107595088500_B47","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1145\/1569901.1570048","volume-title":"Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation","author":"Schmidt","year":"2009"},{"key":"2022030107595088500_B48","first-page":"202","volume-title":"International Conference on Computational Intelligence in Music, Sound, Art and Design","author":"Scirea","year":"2016"},{"key":"2022030107595088500_B49","first-page":"64","article-title":"Soft monotonic constraint support vector regression.","author":"Shah","year":"2016","journal-title":"COMAD"},{"key":"2022030107595088500_B50","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/0-387-23254-0_17","volume-title":"Genetic Programming Theory and Practice II","author":"Smits","year":"2005"},{"key":"2022030107595088500_B51","first-page":"2576","volume-title":"Thirty-First AAAI Conference on Artificial Intelligence","author":"Stewart","year":"2017"},{"key":"2022030107595088500_B52","doi-asserted-by":"crossref","first-page":"2167","DOI":"10.1109\/CEC.2019.8789889","volume-title":"2019 IEEE Congress on Evolutionary Computation","author":"Sun","year":"2019"},{"issue":"1","key":"2022030107595088500_B53","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1198\/TECH.2010.10111","article-title":"Nearly-isotonic regression","volume":"53","author":"Tibshirani","year":"2011","journal-title":"Technometrics"},{"key":"2022030107595088500_B54","first-page":"155","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)","author":"Topchy","year":"2001"},{"issue":"16","key":"2022030107595088500_B55","doi-asserted-by":"crossref","DOI":"10.1126\/sciadv.aay2631","article-title":"AI Feynman: A physics-inspired method for symbolic regression","volume":"6","author":"Udrescu","year":"2020","journal-title":"Science Advances"},{"key":"2022030107595088500_B56","first-page":"318:981","author":"Versino","year":"2017","journal-title":"Data driven modeling of plastic deformation."},{"key":"2022030107595088500_B57","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1145\/2463372.2463486","volume-title":"Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation","author":"Worm","year":"2013"},{"issue":"5","key":"2022030107595088500_B58","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1214\/aos\/1176345140","article-title":"Isotonic, convex and related splines","volume":"8","author":"Wright","year":"1980","journal-title":"The Annals of Statistics"},{"key":"2022030107595088500_B59","first-page":"394:56","article-title":"Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data.","author":"Zhu","year":"2019","journal-title":"Journal of Computational Physics"}],"container-title":["Evolutionary Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/evco\/article-pdf\/30\/1\/75\/1995582\/evco_a_00294.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/evco\/article-pdf\/30\/1\/75\/1995582\/evco_a_00294.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T03:34:16Z","timestamp":1698896056000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/evco\/article\/30\/1\/75\/99840\/Shape-Constrained-Symbolic-Regression-Improving"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":59,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,3,1]]},"published-print":{"date-parts":[[2022,3,1]]}},"URL":"https:\/\/doi.org\/10.1162\/evco_a_00294","relation":{},"ISSN":["1530-9304"],"issn-type":[{"value":"1530-9304","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}