{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T03:44:19Z","timestamp":1777520659143,"version":"3.51.4"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T00:00:00Z","timestamp":1720483200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T00:00:00Z","timestamp":1720483200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Consiglio Nazionale Delle Ricerche"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Conformal predictions make it possible to define reliable and robust learning algorithms. But they are essentially a method for evaluating whether an algorithm is good enough to be used in practice. To define a reliable learning framework for classification from the very beginning of its design, the concept of scalable classifier was introduced to generalize the concept of classical classifier by linking it to statistical order theory and probabilistic learning theory. In this paper, we analyze the similarities between scalable classifiers and conformal predictions by introducing a new definition of a score function and defining a special set of input variables, the conformal safety set, which can identify patterns in the input space that satisfy the error coverage guarantee, i.e., that the probability of observing the wrong (possibly unsafe) label for points belonging to this set is bounded by a predefined <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\varepsilon$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>\u03b5<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> error level. We demonstrate the practical implications of this framework through an application in cybersecurity for identifying DNS tunneling attacks. Our work contributes to the development of probabilistically robust and reliable machine learning models.<\/jats:p>","DOI":"10.1007\/s10994-024-06571-6","type":"journal-article","created":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T18:09:16Z","timestamp":1720548556000},"page":"6645-6661","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Conformal predictions for probabilistically robust scalable machine learning classification"],"prefix":"10.1007","volume":"113","author":[{"given":"Alberto","family":"Carlevaro","sequence":"first","affiliation":[]},{"given":"Teodoro","family":"Alamo","sequence":"additional","affiliation":[]},{"given":"Fabrizio","family":"Dabbene","sequence":"additional","affiliation":[]},{"given":"Maurizio","family":"Mongelli","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,7,9]]},"reference":[{"issue":"1","key":"6571_CR21","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1080\/03081070500473490","volume":"35","author":"J Abell\u00e1n","year":"2006","unstructured":"Abell\u00e1n, J., Klir, G. J., & Moral, S. (2006). Disaggregated total uncertainty measure for Credal sets. International Journal of General Systems, 35(1), 29\u201344.","journal-title":"International Journal of General Systems"},{"key":"6571_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s43681-023-00400-7","volume":"4","author":"L And\u00e9ol","year":"2024","unstructured":"And\u00e9ol, L., Fel, T., De Grancey, F., & Mossina, L. (2024). Conformal prediction for trustworthy detection of railway signals. AI and Ethics, 4, 1\u20135.","journal-title":"AI and Ethics"},{"issue":"4","key":"6571_CR5","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1561\/2200000101","volume":"16","author":"AN Angelopoulos","year":"2023","unstructured":"Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and trends\u00aein machine learning, 16(4), 494\u2013591. https:\/\/doi.org\/10.1561\/2200000101","journal-title":"Foundations and trends\u00aein machine learning"},{"key":"6571_CR11","unstructured":"Angelopoulos, A.N., Bates, S., Jordan, M. & Malik, J. (2020) Uncertainty sets for image classifiers using conformal prediction. In: International conference on learning representations"},{"issue":"14","key":"6571_CR26","doi-asserted-by":"publisher","first-page":"1987","DOI":"10.1002\/dac.2836","volume":"28","author":"M Aiello","year":"2015","unstructured":"Aiello, M., Mongelli, M., & Papaleo, G. (2015). DNS tunneling detection through statistical fingerprints of protocol messages and machine learning. International Journal of Communication Systems, 28(14), 1987\u20132002.","journal-title":"International Journal of Communication Systems"},{"key":"6571_CR9","unstructured":"Balasubramanian, V.N., Gouripeddi, R., Panchanathan, S., Vermillion, J., Bhaskaran, A. & Siegel, R. (2009) Support vector machine based conformal predictors for risk of complications following a coronary drug eluting stent procedure. In: 2009 36th annual computers in cardiology conference (CinC), (pp. 5\u20138) . IEEE"},{"key":"6571_CR14","unstructured":"Carlevaro, A., Alamo, T., Dabbene, F. & Mongelli, M. (2023) Probabilistic safety regions via finite families of scalable classifiers arXiv:2309.04627 [stat.ML]"},{"key":"6571_CR17","doi-asserted-by":"publisher","first-page":"60849","DOI":"10.1109\/ACCESS.2022.3180026","volume":"10","author":"A Carlevaro","year":"2022","unstructured":"Carlevaro, A., Lenatti, M., Paglialonga, A., & Mongelli, M. (2022). Counterfactual building and evaluation via explainable support vector data description. IEEE Access, 10, 60849\u201360861. https:\/\/doi.org\/10.1109\/ACCESS.2022.3180026","journal-title":"IEEE Access"},{"key":"6571_CR27","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2021.3123669","author":"A Carlevaro","year":"2021","unstructured":"Carlevaro, A., & Mongelli, M. (2021). A new SVDD approach to reliable and eXplainable AI. IEEE Intell Syst. https:\/\/doi.org\/10.1109\/MIS.2021.3123669","journal-title":"IEEE Intell Syst"},{"key":"6571_CR15","unstructured":"Chzhen, E., Denis, C., Hebiri, M. & Lorieul, T. (2021) Set-valued classification\u2013overview via a unified framework. arXiv:2102.12318 [stat.ML]"},{"issue":"1","key":"6571_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3150\/21-BEJ1447","volume":"29","author":"M Fontana","year":"2023","unstructured":"Fontana, M., Zeni, G., & Vantini, S. (2023). Conformal prediction: A unified review of theory and new challenges. Bernoulli, 29(1), 1\u201323. https:\/\/doi.org\/10.3150\/21-BEJ1447","journal-title":"Bernoulli"},{"key":"6571_CR8","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.tiv.2018.01.021","volume":"48","author":"A Forreryd","year":"2018","unstructured":"Forreryd, A., Norinder, U., Lindberg, T., & Lindstedt, M. (2018). Predicting skin sensitizers with confidence\u2013Using conformal prediction to determine applicability domain of gard. Toxicology in Vitro, 48, 179\u2013187. https:\/\/doi.org\/10.1016\/j.tiv.2018.01.021","journal-title":"Toxicology in Vitro"},{"key":"6571_CR19","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","volume":"110","author":"E H\u00fcllermeier","year":"2021","unstructured":"H\u00fcllermeier, E., & Waegeman, W. (2021). Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110, 457\u2013506. https:\/\/doi.org\/10.1007\/s10994-021-05946-3","journal-title":"Machine Learning"},{"issue":"11","key":"6571_CR16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0272825","volume":"17","author":"M Lenatti","year":"2022","unstructured":"Lenatti, M., Carlevaro, A., Guergachi, A., Keshavjee, K., Mongelli, M., & Paglialonga, A. (2022). A novel method to derive personalized minimum viable recommendations for type 2 diabetes prevention based on counterfactual explanations. PLOS ONE, 17(11), 1\u201324. https:\/\/doi.org\/10.1371\/journal.pone.0272825","journal-title":"PLOS ONE"},{"key":"6571_CR10","unstructured":"Narteni, S., Carlevaro, A., Dabbene, F., Muselli, M. & Mongelli, M. (2023) Confiderai: Conformal interpretable-by-design score function for explainable and reliable artificial intelligence. In: Conformal and probabilistic prediction with applications, (pp. 485\u2013487)."},{"key":"6571_CR12","unstructured":"Park, S., Bastani, O., Matni, N. & Lee, I. (2019) Pac confidence sets for deep neural networks via calibrated prediction. In: International conference on learning representations."},{"key":"6571_CR24","first-page":"37920","volume":"35","author":"S Park","year":"2022","unstructured":"Park, S., Dobriban, E., Lee, I., & Bastani, O. (2022). PAC prediction sets for meta-learning. Advances in Neural Information Processing Systems, 35, 37920\u201337931.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6571_CR22","unstructured":"Sale, Y., Bengs, V., Caprio, M. & H\u00fcllermeier, E. (2023) Second-order uncertainty quantification: A distance-based approach arXiv:2312.00995 [cs.LG]"},{"key":"6571_CR20","unstructured":"Sale, Y., Caprio, M. & H\u00fcllermeier, E. (2023) Is the volume of a credal set a good measure for epistemic uncertainty? arXiv:2306.09586 [cs.LG]"},{"key":"6571_CR1","unstructured":"Shafer, G., & Vovk, V. (2008). A tutorial on conformal prediction, 9, 371\u2013421. http:\/\/jmlr.org\/papers\/v9\/shafer08a.html."},{"key":"6571_CR7","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108507","volume":"124","author":"P Toccaceli","year":"2022","unstructured":"Toccaceli, P. (2022). Introduction to conformal predictors. Pattern Recognition, 124, 108507. https:\/\/doi.org\/10.1016\/j.patcog.2021.108507","journal-title":"Pattern Recognition"},{"key":"6571_CR28","doi-asserted-by":"publisher","first-page":"83949","DOI":"10.1109\/ACCESS.2022.3197299","volume":"10","author":"I Vaccari","year":"2022","unstructured":"Vaccari, I., Carlevaro, A., Narteni, S., Cambiaso, E., & Mongelli, M. (2022). eXplainable and reliable against adversarial machine learning in data analytics. IEEE Access, 10, 83949\u201383970. https:\/\/doi.org\/10.1109\/ACCESS.2022.3197299","journal-title":"IEEE Access"},{"key":"6571_CR23","volume-title":"Probably approximately correct: Nature\u2019s algorithms for learning and prospering in a complex world","author":"L Valiant","year":"2013","unstructured":"Valiant, L. (2013). Probably approximately correct: Nature\u2019s algorithms for learning and prospering in a complex world. USA: Basic Books Inc."},{"key":"6571_CR25","unstructured":"Vovk, V. (2012) Conditional validity of inductive conformal predictors. In: Hoi, S.C.H., Buntine, W. (eds.) Proceedings of the Asian conference on machine learning. Proceedings of machine learning research, vol. 25, pp. 475\u2013490. PMLR, Singapore Management University, Singapore . https:\/\/proceedings.mlr.press\/v25\/vovk12.html"},{"key":"6571_CR18","unstructured":"Vovk, V., Gammerman, A. & Saunders, C. (1999) Machine-learning applications of algorithmic randomness. In: Proceedings of the sixteenth international conference on machine learning. ICML \u201999, pp. 444\u2013453. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA."},{"key":"6571_CR2","volume-title":"Algorithmic learning in a random world","author":"V Vovk","year":"2005","unstructured":"Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic learning in a random world. Berlin, Heidelberg: Springer."},{"key":"6571_CR4","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1007\/978-3-031-06649-8_6","volume-title":"Probabilistic classification: Venn predictors","author":"V Vovk","year":"2022","unstructured":"Vovk, V., Gammerman, A., & Shafer, G. (2022). Probabilistic classification: Venn predictors (pp. 157\u2013179). Cham: Springer. https:\/\/doi.org\/10.1007\/978-3-031-06649-8_6"},{"key":"6571_CR3","unstructured":"Vovk, V., Shen, J., Manokhin, V. & Xie, M. (2017) Nonparametric predictive distributions based on conformal prediction. In: Conformal and probabilistic prediction and applications, pp. 82\u2013102"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06571-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06571-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06571-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T17:31:53Z","timestamp":1723051913000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06571-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,9]]},"references-count":28,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["6571"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06571-6","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,9]]},"assertion":[{"value":"30 December 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 February 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 May 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 July 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"There are no either Conflict of interest or Conflict of interest to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not Applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Not Applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not Applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}