{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T23:20:46Z","timestamp":1776122446994,"version":"3.50.1"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2021,6,24]],"date-time":"2021-06-24T00:00:00Z","timestamp":1624492800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,24]],"date-time":"2021-06-24T00:00:00Z","timestamp":1624492800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["200020_159557"],"award-info":[{"award-number":["200020_159557"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RGPIN-2014-04634"],"award-info":[{"award-number":["RGPIN-2014-04634"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Connaught New Researcher Award"},{"DOI":"10.13039\/501100003006","name":"ETH Zurich","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003006","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2023,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Selecting the right tuning parameters for algorithms is a pravelent problem in machine learning that can significantly affect the performance of algorithms. Data-efficient optimization algorithms, such as Bayesian optimization, have been used to automate this process. During experiments on real-world systems such as robotic platforms these methods can evaluate unsafe parameters that lead to safety-critical system failures and can destroy the system. Recently, a safe Bayesian optimization algorithm, called\u00a0<jats:sc>SafeOpt<\/jats:sc>, has been developed, which guarantees that the performance of the system never falls below a critical value; that is, safety is defined based on the performance function. However, coupling performance and safety is often not desirable in practice, since they are often opposing objectives. In this paper, we present a generalized algorithm that allows for multiple safety constraints separate from the objective. Given an initial set of safe parameters, the algorithm maximizes performance but only evaluates parameters that satisfy safety for all constraints with high probability. To this end, it carefully explores the parameter space by exploiting regularity assumptions in terms of a Gaussian process prior. Moreover, we show how context variables can be used to safely transfer knowledge to new situations and tasks. We provide a theoretical analysis and demonstrate that the proposed algorithm enables fast, automatic, and safe optimization of tuning parameters in experiments on a quadrotor vehicle.<\/jats:p>","DOI":"10.1007\/s10994-021-06019-1","type":"journal-article","created":{"date-parts":[[2021,6,24]],"date-time":"2021-06-24T20:02:16Z","timestamp":1624564936000},"page":"3713-3747","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":172,"title":["Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics"],"prefix":"10.1007","volume":"112","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5179-6606","authenticated-orcid":false,"given":"Felix","family":"Berkenkamp","sequence":"first","affiliation":[]},{"given":"Andreas","family":"Krause","sequence":"additional","affiliation":[]},{"given":"Angela P.","family":"Schoellig","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,6,24]]},"reference":[{"key":"6019_CR1","unstructured":"Achiam, J., Held, D., Tamar, A., & Abbeel, P. (2017). Constrained policy optimization. In Proceedings of the international conference on machine learning (ICML), 1705.10528."},{"key":"6019_CR2","doi-asserted-by":"crossref","unstructured":"Akametalu, A. K., Kaynama, S., Fisac, J. F., Zeilinger, M. N., Gillula, J. H., & Tomlin, C. J. (2014). Reachability-based safe learning with Gaussian processes. In Proceedings of the IEEE conference on decision and control (CDC) (pp. 1424\u20131431).","DOI":"10.1109\/CDC.2014.7039601"},{"issue":"3","key":"6019_CR3","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1561\/2200000036","volume":"4","author":"MA \u00c1lvarez","year":"2012","unstructured":"\u00c1lvarez, M. A., Rosasco, L., & Lawrence, N. D. (2012). Kernels for vector-valued functions: A review. Foundations and Trends in Machine Learning, 4(3), 195\u2013266.","journal-title":"Foundations and Trends in Machine Learning"},{"issue":"5","key":"6019_CR4","doi-asserted-by":"publisher","first-page":"1216","DOI":"10.1016\/j.automatica.2013.02.003","volume":"49","author":"A Aswani","year":"2013","unstructured":"Aswani, A., Gonzalez, H., Sastry, S. S., & Tomlin, C. (2013). Provably safe and robust learning-based model predictive control. Automatica, 49(5), 1216\u20131226.","journal-title":"Automatica"},{"key":"6019_CR6","unstructured":"Berkenkamp, F., Krause, A., Angela, P., & Schoellig (2016a). Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics. arXiv:160204450 [csRO]"},{"key":"6019_CR5","doi-asserted-by":"crossref","unstructured":"Berkenkamp, F., & Schoellig, A. P. (2015). Safe and robust learning control with Gaussian processes. In Proceedings of the European control conference (ECC) (pp. 2501\u20132506).","DOI":"10.1109\/ECC.2015.7330913"},{"key":"6019_CR7","doi-asserted-by":"crossref","unstructured":"Berkenkamp, F., Schoellig, A. P., & Krause, A. (2016b). Safe controller optimization for quadrotors with Gaussian processes. In IEEE international conference on robotics and automation (ICRA) (pp. 493\u2013496).","DOI":"10.1109\/ICRA.2016.7487170"},{"key":"6019_CR8","unstructured":"Berkenkamp, F., Turchetta, M., Schoellig, A. P., & Krause, A. (2017). Safe model-based reinforcement learning with stability guarantees. In Neural information processing systems (NeurIPS) (pp. 908\u2013918)."},{"key":"6019_CR9","doi-asserted-by":"crossref","unstructured":"Boyd, S., Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"6019_CR10","first-page":"2879","volume":"12","author":"AD Bull","year":"2011","unstructured":"Bull, A. D. (2011). Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, 12, 2879\u20132904.","journal-title":"Journal of Machine Learning Research"},{"key":"6019_CR11","doi-asserted-by":"crossref","unstructured":"Calandra, R., Gopalan, N., Seyfarth, A., Peters, J., & Deisenroth, M. P. (2014a). Bayesian gait optimization for bipedal locomotion. In Learning and intelligent optimization (pp 274\u2013290). Springer.","DOI":"10.1007\/978-3-319-09584-4_25"},{"key":"6019_CR12","doi-asserted-by":"crossref","unstructured":"Calandra, R., Seyfarth, A., Peters, J., Deisenroth, M. P. (2014b). An experimental comparison of Bayesian optimization for bipedal locomotion. In 2014 IEEE international conference on robotics and automation (ICRA) (pp. 1951\u20131958).","DOI":"10.1109\/ICRA.2014.6907117"},{"key":"6019_CR13","unstructured":"Chowdhury, S. R., & Gopalan, A. (2017). On kernelized multi-armed bandits. In Proceedings of the 34th international conference on machine learning, PMLR, proceedings of machine learning research (Vol. 70, pp. 844\u2013853)."},{"key":"6019_CR14","unstructured":"Christmann, A., & Steinwart, I. (2008). Support vector machines. Information science and statistics. Springer."},{"key":"6019_CR15","unstructured":"Dalal, G., Dvijotham, K., Vecerik, M., Hester, T., Paduraru, C., & Tassa, Y. (2018). Safe exploration in continuous action spaces. arXiv preprint arXiv:180108757"},{"key":"6019_CR16","first-page":"1025","volume":"26","author":"J Djolonga","year":"2013","unstructured":"Djolonga, J., Krause, A., & Cevher, V. (2013). High-dimensional Gaussian process bandits. Advances in Neural Information Processing Systems, 26, 1025\u20131033.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6019_CR17","doi-asserted-by":"crossref","unstructured":"Duivenvoorden, R. R., Berkenkamp, F., Carion, N., Krause, A., & Schoellig, A. P. (2017). Constrained Bayesian optimization with particle swarms for adaptive controller tuning. In Proceedings of the IFAC (international federation of automatic control) world congress (pp. 12306\u201312313).","DOI":"10.1016\/j.ifacol.2017.08.1991"},{"key":"6019_CR18","first-page":"226","volume":"24","author":"DK Duvenaud","year":"2011","unstructured":"Duvenaud, D. K., Nickisch, H., & Rasmussen, C. E. (2011). Additive Gaussian processes. Advances in Neural Information Processing Systems, 24, 226\u2013234.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6019_CR19","doi-asserted-by":"crossref","unstructured":"Eldar, Y. C., & Kutyniok, G. (2012). Compressed sensing: Theory and applications. Cambridge University Press.","DOI":"10.1017\/CBO9780511794308"},{"key":"6019_CR20","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1613\/jair.3761","volume":"45","author":"J Garcia","year":"2012","unstructured":"Garcia, J., & Fern\u00e1ndez, F. (2012). Safe exploration of state and action spaces in reinforcement learning. Journal of Artificial Intelligence Research, 45, 515\u2013564.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"6019_CR21","unstructured":"Gelbart, M.A., Snoek, J., Adams, R.P. (2014). Bayesian optimization with unknown constraints. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) (pp. 250\u2013259)."},{"issue":"5","key":"6019_CR22","doi-asserted-by":"publisher","first-page":"2413","DOI":"10.1214\/009053606000000795","volume":"34","author":"S Ghosal","year":"2006","unstructured":"Ghosal, S., & Roy, A. (2006). Posterior consistency of Gaussian process prior for nonparametric binary regression. The Annals of Statistics, 34(5), 2413\u20132429.","journal-title":"The Annals of Statistics"},{"key":"6019_CR23","unstructured":"Hans, A., Schneega\u00df, D., Sch\u00e4fer, A. M., & Udluft, S. (2008). Safe exploration for reinforcement learning. In ESANN (pp. 143\u2013148). Citeseer."},{"issue":"4","key":"6019_CR24","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1023\/A:1012771025575","volume":"21","author":"DR Jones","year":"2001","unstructured":"Jones, D. R. (2001). A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization, 21(4), 345\u2013383.","journal-title":"Journal of Global Optimization"},{"issue":"11","key":"6019_CR25","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1177\/0278364913495721","volume":"32","author":"J Kober","year":"2013","unstructured":"Kober, J., & Peters, J. (2013). Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32(11), 1238\u20131274.","journal-title":"The International Journal of Robotics Research"},{"key":"6019_CR26","unstructured":"Krause, A., & Ong, C. S. (2011). Contextual Gaussian process bandit optimization. In Proceedings of neural information processing systems (NIPS) (pp. 2447\u20132455)."},{"key":"6019_CR27","unstructured":"Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv:150902971 [cs, stat] 1509.02971"},{"key":"6019_CR28","unstructured":"Lizotte, D. J., Wang, T., Bowling, M. H., & Schuurmans, D. (2007). Automatic gait optimization with Gaussian process regression. In Proceedings of the twentieth international joint conference on artificial intelligence (IJCAI) (Vol. 7, pp. 944\u2013949)."},{"issue":"1","key":"6019_CR29","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.mechatronics.2013.11.006","volume":"24","author":"S Lupashin","year":"2014","unstructured":"Lupashin, S., Hehn, M., Mueller, M. W., Schoellig, A. P., Sherback, M., & D\u2019Andrea, R. (2014). A platform for aerial robotics research and demonstration: The Flying Machine Arena. Mechatronics, 24(1), 41\u201354.","journal-title":"Mechatronics"},{"key":"6019_CR30","doi-asserted-by":"crossref","unstructured":"Marco, A., Berkenkamp, F., Hennig, P., Schoellig, A. P., Krause, A., Schaal, S., & Trimpe, S. (2017). Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization. In Proceedings of the IEEE international conference on robotics and automation (ICRA) (pp. 1557\u20131563).","DOI":"10.1109\/ICRA.2017.7989186"},{"key":"6019_CR31","unstructured":"Mockus, J. (2012). Bayesian approach to global optimization: Theory and applications. Springer"},{"key":"6019_CR32","unstructured":"Moldovan, T. M., & Abbeel, P. (2012). Safe exploration in Markov decision processes. In Proceedings of the international conference on machine learning (ICML) (pp. 1711\u20131718)."},{"issue":"13","key":"6019_CR33","doi-asserted-by":"publisher","first-page":"1547","DOI":"10.1177\/0278364916645661","volume":"35","author":"CJ Ostafew","year":"2016","unstructured":"Ostafew, C. J., Schoellig, A. P., & Barfoot, T. D. (2016). Robust constrained learning-based NMPC enabling reliable mobile robot path tracking. The International Journal of Robotics Research (IJRR), 35(13), 1547\u20131536.","journal-title":"The International Journal of Robotics Research (IJRR)"},{"key":"6019_CR34","doi-asserted-by":"crossref","unstructured":"Peters, J., & Schaal, S. (2006). Policy gradient methods for robotics. In Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems (pp. 2219\u20132225).","DOI":"10.1109\/IROS.2006.282564"},{"issue":"4","key":"6019_CR35","doi-asserted-by":"publisher","first-page":"682","DOI":"10.1016\/j.neunet.2008.02.003","volume":"21","author":"J Peters","year":"2008","unstructured":"Peters, J., & Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients. Neural Networks, 21(4), 682\u2013697.","journal-title":"Neural Networks"},{"key":"6019_CR36","volume-title":"Gaussian processes for machine learning","author":"CE Rasmussen","year":"2006","unstructured":"Rasmussen, C. E., & Williams, C. K. (2006). Gaussian processes for machine learning. Cambridge MA: MIT Press."},{"issue":"2","key":"6019_CR37","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1109\/MRA.2010.936957","volume":"17","author":"S Schaal","year":"2010","unstructured":"Schaal, S., & Atkeson, C. G. (2010). Learning control in robotics. IEEE Robotics & Automation Magazine, 17(2), 20\u201329.","journal-title":"IEEE Robotics & Automation Magazine"},{"key":"6019_CR38","doi-asserted-by":"crossref","unstructured":"Schoellig, A., Wiltsche, C., & D\u2019Andrea, R. (2012). Feed-forward parameter identification for precise periodic quadrocopter motions. In Proceedings of the American control conference (ACC) (pp. 4313\u20134318).","DOI":"10.1109\/ACC.2012.6315248"},{"key":"6019_CR39","doi-asserted-by":"crossref","unstructured":"Schoellig, A. P., Hehn, M., Lupashin, S., & D\u2019Andrea, R. (2011). Feasiblity of motion primitives for choreographed quadrocopter flight. In Proceedings of the American control conference (ACC) (pp. 3843\u20133849).","DOI":"10.1109\/ACC.2011.5991482"},{"key":"6019_CR40","doi-asserted-by":"crossref","unstructured":"Schreiter, J., Nguyen-Tuong, D., Eberts, M., Bischoff, B., Markert, H., & Toussaint, M. (2015). Safe exploration for active learning with Gaussian processes. In Machine learning and knowledge discovery in databases (Vol. 9286, pp. 133\u2013149). Springer International Publishing.","DOI":"10.1007\/978-3-319-23461-8_9"},{"key":"6019_CR41","unstructured":"Solak, E., Murray-Smith, R., Leithead, W. E., Leith, D. J., & Rasmussen, C. E. (2003). Derivative observations in Gaussian process models of dynamic systems. In S. Becker, S. Thrun , K. Obermayer (eds) Proceedings of neural information processing systems (NIPS) (pp. 1057\u20131064). MIT Press."},{"issue":"5","key":"6019_CR42","doi-asserted-by":"publisher","first-page":"3250","DOI":"10.1109\/TIT.2011.2182033","volume":"58","author":"N Srinivas","year":"2012","unstructured":"Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. (2012). Gaussian process optimization in the bandit setting: No regret and experimental design. IEEE Transactions on Information Theory, 58(5), 3250\u20133265 ((0912.3995)).","journal-title":"IEEE Transactions on Information Theory"},{"key":"6019_CR43","unstructured":"Sui, Y., Gotovos, A., Burdick, J. W., & Krause, A. (2015). Safe exploration for optimization with Gaussian processes. In Proceedings of the international conference on machine learning (ICML) (pp. 997\u20131005)."},{"key":"6019_CR44","unstructured":"Sui, Y., Zhuang, B. J., & Yue, Y. (2018). Stagewise safe Bayesian optimization with Gaussian processes. In International conference on machine learning (pp. 4781\u20134789)."},{"key":"6019_CR45","doi-asserted-by":"crossref","unstructured":"Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. MIT press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"6019_CR46","doi-asserted-by":"crossref","unstructured":"Tesch, M., Schneider, J., & Choset, H. (2011). Using response surfaces and expected improvement to optimize snake robot gait parameters. In Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems (IROS) (pp. 1069\u20131074).","DOI":"10.1109\/IROS.2011.6095076"},{"key":"6019_CR47","unstructured":"The GPy authors (2012) GPy: A Gaussian process framework in Python. https:\/\/github.com\/SheffieldML\/GPy"},{"key":"6019_CR48","unstructured":"Turchetta, M., Berkenkamp, F., & Krause, A. (2016). Safe exploration in finite markov decision processes with Gaussian processes. In Neural information processing systems (NeurIPS) (pp. 4305\u20134313)."},{"key":"6019_CR49","unstructured":"Turchetta, M., Berkenkamp, F., & Krause, A. (2019). Safe exploration for interactive machine learning. In Neural information processing systems (NeurIPS)."},{"key":"6019_CR50","unstructured":"Wang, Z., Zoghi, M., Hutter, F., Matheson, D., & De\u00a0Freitas, N. (2013). Bayesian optimization in high dimensions via random embeddings. In Proceedings of the international joint conference on artificial intelligence (IJCAI) (pp. 1778\u20131784). AAAI Press."},{"key":"6019_CR51","unstructured":"Zhou, K., & Doyle, J. C. (1998). Essentials of robust control, (Vol. 104). Prentice Hall."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06019-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-021-06019-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06019-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T22:03:47Z","timestamp":1694124227000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-021-06019-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,24]]},"references-count":51,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2023,10]]}},"alternative-id":["6019"],"URL":"https:\/\/doi.org\/10.1007\/s10994-021-06019-1","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,24]]},"assertion":[{"value":"2 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 May 2020","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 June 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 June 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}