{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T22:04:41Z","timestamp":1778537081236,"version":"3.51.4"},"reference-count":54,"publisher":"Springer Science and Business Media LLC","issue":"11-12","license":[{"start":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T00:00:00Z","timestamp":1631491200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T00:00:00Z","timestamp":1631491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"University of Dublin, Trinity College"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Skew-Gaussian Processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution, we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization. These two tasks are fundamental for design of experiments and in Data Science.<\/jats:p>","DOI":"10.1007\/s10994-021-06039-x","type":"journal-article","created":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T22:02:24Z","timestamp":1631570544000},"page":"3095-3133","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes"],"prefix":"10.1007","volume":"110","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2522-7178","authenticated-orcid":false,"given":"Alessio","family":"Benavoli","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dario","family":"Azzimonti","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dario","family":"Piga","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,9,13]]},"reference":[{"issue":"23","key":"6039_CR1","doi-asserted-by":"publisher","first-page":"4936","DOI":"10.1080\/03610926.2012.737498","volume":"43","author":"M Alodat","year":"2014","unstructured":"Alodat, M., & Al-Momani, E. (2014). Skew Gaussian process for nonlinear regression. Communications in Statistics-Theory and Methods, 43(23), 4936\u20134961.","journal-title":"Communications in Statistics-Theory and Methods"},{"issue":"3","key":"6039_CR2","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1007\/s40300-014-0046-z","volume":"72","author":"M Alodat","year":"2014","unstructured":"Alodat, M., & Al-Rawwash, M. (2014). The extended skew Gaussian process for regression. Metron, 72(3), 317\u2013330.","journal-title":"Metron"},{"key":"6039_CR3","doi-asserted-by":"publisher","first-page":"112665","DOI":"10.1016\/j.cam.2019.112665","volume":"370","author":"M Alodat","year":"2020","unstructured":"Alodat, M., & Shakhatreh, M. K. (2020). Gaussian process regression with skewed errors. Journal of Computational and Applied Mathematics, 370, 112665.","journal-title":"Journal of Computational and Applied Mathematics"},{"issue":"3","key":"6039_CR4","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1111\/j.1467-9469.2006.00503.x","volume":"33","author":"RB Arellano","year":"2006","unstructured":"Arellano, R. B., & Azzalini, A. (2006). On the unification of families of skew-normal distributions. Scandinavian Journal of Statistics, 33(3), 561\u2013574.","journal-title":"Scandinavian Journal of Statistics"},{"key":"6039_CR5","doi-asserted-by":"crossref","unstructured":"Azzalini, A. (2013). The skew-normal and related families (Vol. 3). Cambridge University Press.","DOI":"10.1017\/CBO9781139248891"},{"issue":"2","key":"6039_CR6","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1080\/10618600.2017.1360781","volume":"27","author":"D Azzimonti","year":"2018","unstructured":"Azzimonti, D., & Ginsbourger, D. (2018). Estimating orthant probabilities of high-dimensional gaussian vectors with an application to set estimation. Journal of Computational and Graphical Statistics, 27(2), 255\u2013267.","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"6039_CR7","unstructured":"Bauer, M., van der Wilk, M., & Rasmussen, C. E. (2016). Understanding probabilistic sparse Gaussian process approximations. Advances in neural information processing systems (pp. 1533\u20131541)."},{"issue":"2","key":"6039_CR8","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1007\/s10994-020-05935-y","volume":"110","author":"A Bemporad","year":"2021","unstructured":"Bemporad, A., & Piga, D. (2021). Global optimization based on active preference learning with radial basis functions. Machine Learning, 110(2), 417\u2013448.","journal-title":"Machine Learning"},{"key":"6039_CR9","doi-asserted-by":"publisher","first-page":"1877","DOI":"10.1007\/s10994-020-05906-3","volume":"109","author":"A Benavoli","year":"2020","unstructured":"Benavoli, A., Azzimonti, D., & Piga, D. (2020). Skew Gaussian processes for classification. Machine Learning, 109, 1877\u20131902.","journal-title":"Machine Learning"},{"key":"6039_CR10","doi-asserted-by":"crossref","unstructured":"Benavoli, A., Azzimonti, D., Piga, D., & (2021). Preferential Bayesian optimisation with skew Gaussian Processes. In Genetic and evolutionary computation conference companion (GECCO\u201921 companion), July 10\u201314, 2021, Lille, France, New York, NY. ACM.","DOI":"10.1145\/3449726.3463128"},{"key":"6039_CR11","unstructured":"Berkenkamp, F., Krause, A., & Schoellig, A. P. (2016a). Bayesian optimization with safety constraints: Safe and automatic parameter tuning in robotics. arXiv preprint arXiv:1602.04450."},{"key":"6039_CR12","doi-asserted-by":"crossref","unstructured":"Berkenkamp, F., Schoellig, A.\u00a0P., & Krause, A. (2016b). Safe controller optimization for quadrotors with gaussian processes. In 2016 IEEE international conference on robotics and automation (ICRA) (pp. 491\u2013496). IEEE.","DOI":"10.1109\/ICRA.2016.7487170"},{"key":"6039_CR13","unstructured":"Bishop, C. M. (2006). Pattern recognition and machine learning. Springer."},{"issue":"1","key":"6039_CR14","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1111\/rssb.12162","volume":"79","author":"ZI Botev","year":"2017","unstructured":"Botev, Z. I. (2017). The normal law under linear restrictions: Simulation and estimation via minimax tilting. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(1), 125\u2013148.","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"6039_CR15","unstructured":"Brochu, E., de Freitas, N., & Ghosh, A. (2008). Active preference learning with discrete choice data. Advances in neural information processing systems (pp. 409\u2013416)."},{"key":"6039_CR16","unstructured":"Cao, J., Durante, D., & Genton, M. G. (2020). Scalable computation of predictive probabilities in probit models with gaussian process priors. arXiv:2009.01471."},{"key":"6039_CR17","first-page":"1019","volume":"6","author":"W Chu","year":"2005","unstructured":"Chu, W., & Ghahramani, Z. (2005a). Gaussian processes for ordinal regression. Journal of Machine Learning Research, 6, 1019\u20131041.","journal-title":"Journal of Machine Learning Research"},{"key":"6039_CR18","doi-asserted-by":"crossref","unstructured":"Chu, W., & Ghahramani, Z. (2005b). Preference learning with gaussian processes. Association for Computing Machinery.","DOI":"10.1145\/1102351.1102369"},{"key":"6039_CR19","unstructured":"Cunningham, J. P., Hennig, P., & Lacoste-Julien, S. (2013). Gaussian probabilities and expectation propagation."},{"issue":"4","key":"6039_CR20","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1093\/biomet\/asz034","volume":"106","author":"D Durante","year":"2019","unstructured":"Durante, D. (2019). Conjugate Bayes for probit regression via unified skew-normal distributions. Biometrika, 106(4), 765\u2013779.","journal-title":"Biometrika"},{"issue":"4","key":"6039_CR21","first-page":"457","volume":"7","author":"A Gelman","year":"1992","unstructured":"Gelman, A., Rubin, D. B., et al. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7(4), 457\u2013472.","journal-title":"Statistical Science"},{"issue":"2","key":"6039_CR22","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1080\/10618600.2017.1375936","volume":"27","author":"MG Genton","year":"2018","unstructured":"Genton, M. G., Keyes, D. E., & Turkiyyah, G. (2018). Hierarchical decompositions for the computation of high-dimensional multivariate normal probabilities. Journal of Computational and Graphical Statistics, 27(2), 268\u2013277.","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"2","key":"6039_CR23","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1080\/10618600.1992.10477010","volume":"1","author":"A Genz","year":"1992","unstructured":"Genz, A. (1992). Numerical computation of multivariate normal probabilities. Journal of Computational and Graphical Statistics, 1(2), 141\u2013149.","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"6039_CR24","doi-asserted-by":"crossref","unstructured":"Genz, A., & Bretz, F. (2009). Computation of multivariate normal and t probabilities (Vol. 195). Springer Science & Business Media.","DOI":"10.1007\/978-3-642-01689-9"},{"key":"6039_CR25","unstructured":"Gessner, A., Kanjilal, O., & Hennig, P. (2020). Integrals over Gaussians under linear domain constraints. In S. Chiappa, & R. Calandra (Eds.), Proceedings of the twenty third international conference on artificial intelligence and statistics (Vol. 108, pp. 2764\u20132774). PMLR."},{"issue":"6","key":"6039_CR26","doi-asserted-by":"publisher","first-page":"1458","DOI":"10.1109\/72.883477","volume":"11","author":"MN Gibbs","year":"2000","unstructured":"Gibbs, M. N., & MacKay, D. J. (2000). Variational gaussian process classifiers. IEEE Transactions on Neural Networks, 11(6), 1458\u20131464.","journal-title":"IEEE Transactions on Neural Networks"},{"key":"6039_CR27","unstructured":"Gonz\u00e1lez, J., Dai, Z., Damianou, A., & Lawrence, N. D. (2017). Preferential Bayesian optimization. In Proceedings of the 34th international conference on machine learning (Vol. 70, pp. 1282\u20131291). JMLR. org."},{"key":"6039_CR28","unstructured":"GPy (2012). GPy: A gaussian process framework in python. http:\/\/github.com\/SheffieldML\/GPy."},{"issue":"3","key":"6039_CR29","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1080\/15598608.2013.790233","volume":"7","author":"AK Gupta","year":"2013","unstructured":"Gupta, A. K., Aziz, M. A., & Ning, W. (2013). On some properties of the unified skew normal distribution. Journal of Statistical Theory and Practice, 7(3), 480\u2013495.","journal-title":"Journal of Statistical Theory and Practice"},{"key":"6039_CR30","unstructured":"Hensman, J., Fusi, N., & Lawrence, N. D. (2013). Gaussian processes for big data. In Proceedings of the twenty-ninth conference on uncertainty in artificial intelligence, UAI\u201913, Arlington, Virginia, USA (pp. 282\u2013290). AUAI Press."},{"key":"6039_CR31","unstructured":"Hern\u00e1ndez-Lobato, D., & Hern\u00e1ndez-Lobato, J. M. (2016). Scalable gaussian process classification via expectation propagation. Artificial Intelligence and Statistics (pp. 168\u2013176)."},{"key":"6039_CR32","unstructured":"Houlsby, N., Husz\u00e1r, F., Ghahramani, Z., & Lengyel, M. (2011). Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745."},{"key":"6039_CR33","first-page":"1679","volume":"6","author":"M Kuss","year":"2005","unstructured":"Kuss, M., & Rasmussen, C. E. (2005). Assessing approximate inference for binary Gaussian process classification. Journal of Machine Learning Research, 6, 1679\u20131704.","journal-title":"Journal of Machine Learning Research"},{"key":"6039_CR34","doi-asserted-by":"crossref","unstructured":"MacKay, D. J. (1996). In Models of neural networks III. Bayesian methods for backpropagation networks (pp. 211\u2013254). Springer.","DOI":"10.1007\/978-1-4612-0723-8_6"},{"key":"6039_CR35","unstructured":"Minka, T. P. (2001). A family of algorithms for approximate bayesian inference. In UAI (pp. 362\u2013369). Morgan Kaufmann."},{"key":"6039_CR36","unstructured":"Murray, I., Adams, R., & MacKay, D. (2010). Elliptical slice sampling. In Proceedings of the 13th international conference on artificial intelligence and statistics, Chia, Italy (pp. 541\u2013548). PMLR."},{"key":"6039_CR37","first-page":"2035","volume":"9","author":"H Nickisch","year":"2008","unstructured":"Nickisch, H., & Rasmussen, C. E. (2008). Approximations for binary gaussian process classification. Journal of Machine Learning Research, 9, 2035\u20132078.","journal-title":"Journal of Machine Learning Research"},{"issue":"1","key":"6039_CR38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1978.tb01643.x","volume":"40","author":"A O\u2019Hagan","year":"1978","unstructured":"O\u2019Hagan, A. (1978). Curve fitting and optimal design for prediction. Journal of the Royal Statistical Society: Series B (Methodological), 40(1), 1\u201324.","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"issue":"1","key":"6039_CR39","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1093\/biomet\/63.1.201","volume":"63","author":"A O\u2019Hagan","year":"1976","unstructured":"O\u2019Hagan, A., & Leonard, T. (1976). Bayes estimation subject to uncertainty about parameter constraints. Biometrika, 63(1), 201\u2013203.","journal-title":"Biometrika"},{"issue":"3","key":"6039_CR40","doi-asserted-by":"publisher","first-page":"786","DOI":"10.1162\/neco.2008.08-07-592","volume":"21","author":"M Opper","year":"2009","unstructured":"Opper, M., & Archambeau, C. (2009). The variational gaussian approximation revisited. Neural Computation, 21(3), 786\u2013792.","journal-title":"Neural Computation"},{"issue":"4","key":"6039_CR41","doi-asserted-by":"publisher","first-page":"1521","DOI":"10.1016\/j.csda.2010.10.005","volume":"55","author":"I Phinikettos","year":"2011","unstructured":"Phinikettos, I., & Gandy, A. (2011). Fast computation of high-dimensional multivariate normal probabilities. Computational Statistics & Data Analysis, 55(4), 1521\u20131529.","journal-title":"Computational Statistics & Data Analysis"},{"key":"6039_CR42","first-page":"1939","volume":"6","author":"J Qui\u00f1onero-Candela","year":"2005","unstructured":"Qui\u00f1onero-Candela, J., & Rasmussen, C. E. (2005). A unifying view of sparse approximate Gaussian process regression. Journal of Machine Learning Research, 6, 1939\u20131959.","journal-title":"Journal of Machine Learning Research"},{"key":"6039_CR43","doi-asserted-by":"crossref","unstructured":"Rasmussen, C. E., & Williams, C. K. (2006). Gaussian processes for machine learning. MIT Press.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"6039_CR44","doi-asserted-by":"crossref","unstructured":"Sadigh, D., Dragan, A. D., Sastry, S., & Seshia, S. A. (2017). Active preference-based learning of reward functions. In Robotics: Science and systems.","DOI":"10.15607\/RSS.2017.XIII.053"},{"key":"6039_CR45","doi-asserted-by":"publisher","first-page":"e55","DOI":"10.7717\/peerj-cs.55","volume":"2","author":"J Salvatier","year":"2016","unstructured":"Salvatier, J., Wiecki, T. V., & Fonnesbeck, C. (2016). Probabilistic programming in python using pymc3. PeerJ Computer Science, 2, e55.","journal-title":"PeerJ Computer Science"},{"key":"6039_CR46","first-page":"109","volume":"120","author":"M Schuerch","year":"2020","unstructured":"Schuerch, M., Azzimonti, D., Benavoli, A., & Zaffalon, M. (2020). Recursive estimation for sparse Gaussian process regression. Automatica, 120, 109\u2013127.","journal-title":"Automatica"},{"issue":"1","key":"6039_CR47","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1109\/JPROC.2015.2494218","volume":"104","author":"B Shahriari","year":"2015","unstructured":"Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & De Freitas, N. (2015). Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1), 148\u2013175.","journal-title":"Proceedings of the IEEE"},{"key":"6039_CR48","unstructured":"Snelson, E., & Ghahramani, Z. (2006). Sparse Gaussian processes using pseudo-inputs. Advances in neural information processing systems (pp. 1257\u20131264)."},{"key":"6039_CR49","unstructured":"Sui, Y., Gotovos, A., Burdick, J., & Krause, A. (2015). Safe exploration for optimization with gaussian processes. In International conference on machine learning (pp. 997\u20131005). PMLR."},{"key":"6039_CR50","unstructured":"Titsias, M. (2009). Variational learning of inducing variables in sparse Gaussian processes. In D. van Dyk, & M. Welling (Eds.), Proceedings of the twelfth international conference on artificial intelligence and statistics, volume\u00a05 of proceedings of machine learning research. Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA (pp. 567\u2013574). PMLR."},{"issue":"5","key":"6039_CR51","doi-asserted-by":"publisher","first-page":"989","DOI":"10.1007\/s11222-014-9468-y","volume":"25","author":"G Trinh","year":"2015","unstructured":"Trinh, G., & Genz, A. (2015). Bivariate conditioning approximations for multivariate normal probabilities. Statistics and Computing, 25(5), 989\u2013996.","journal-title":"Statistics and Computing"},{"key":"6039_CR52","doi-asserted-by":"crossref","unstructured":"Villacampa-Calvo, C., Zaldivar, B., Garrido-Merch\u00e1n, E.\u00a0C., & Hern\u00e1ndez-Lobato, D. (2020). Multi-class gaussian process classification with noisy inputs. arXiv preprint arXiv:2001.10523.","DOI":"10.1016\/j.neucom.2019.09.090"},{"issue":"12","key":"6039_CR53","doi-asserted-by":"publisher","first-page":"1342","DOI":"10.1109\/34.735807","volume":"20","author":"CK Williams","year":"1998","unstructured":"Williams, C. K., & Barber, D. (1998). Bayesian classification with gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12), 1342\u20131351.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"6039_CR54","unstructured":"Zoghi, M., Whiteson, S., Munos, R., & Rijke, M. (2014). Relative upper confidence bound for the k-armed dueling bandit problem. In Proceedings of the 31st international conference on machine learning, Beijing, China (pp. 10\u201318)."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06039-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-021-06039-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06039-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,8]],"date-time":"2024-09-08T08:39:59Z","timestamp":1725784799000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-021-06039-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,13]]},"references-count":54,"journal-issue":{"issue":"11-12","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["6039"],"URL":"https:\/\/doi.org\/10.1007\/s10994-021-06039-x","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,13]]},"assertion":[{"value":"27 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 June 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 September 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}