{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T00:24:27Z","timestamp":1767140667354,"version":"build-2238731810"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,11,25]],"date-time":"2023-11-25T00:00:00Z","timestamp":1700870400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,25]],"date-time":"2023-11-25T00:00:00Z","timestamp":1700870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100009244","name":"Stockholm University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100009244","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Classif"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    In many applications there is ambiguity about which (if any) of a finite number\n                    <jats:italic>N<\/jats:italic>\n                    of hypotheses that best fits an observation. It is of interest then to possibly output a whole set of categories, that is, a scenario where the size of the classified set of categories ranges from 0 to\n                    <jats:italic>N<\/jats:italic>\n                    . Empty sets correspond to an outlier, sets of size 1 represent a firm decision that singles out one hypothesis, sets of size\n                    <jats:italic>N<\/jats:italic>\n                    correspond to a rejection to classify, whereas sets of sizes\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$2,\\ldots ,N-1$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mrow>\n                            <mml:mn>2<\/mml:mn>\n                            <mml:mo>,<\/mml:mo>\n                            <mml:mo>\u2026<\/mml:mo>\n                            <mml:mo>,<\/mml:mo>\n                            <mml:mi>N<\/mml:mi>\n                            <mml:mo>-<\/mml:mo>\n                            <mml:mn>1<\/mml:mn>\n                          <\/mml:mrow>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    represent a partial rejection to classify, where some hypotheses are excluded from further analysis. In this paper, we review and unify several proposed methods of Bayesian set-valued classification, where the objective is to find the optimal Bayesian classifier that maximizes the expected reward. We study a large class of reward functions with rewards for sets that include the true category, whereas additive or multiplicative penalties are incurred for sets depending on their size. For models with one homogeneous block of hypotheses, we provide general expressions for the accompanying Bayesian classifier, several of which extend previous results in the literature. Then, we derive novel results for the more general setting when hypotheses are partitioned into blocks, where ambiguity within and between blocks are of different severity. We also discuss how well-known methods of classification, such as conformal prediction, indifference zones, and hierarchical classification, fit into our framework. Finally, set-valued classification is illustrated using an ornithological data set, with taxa partitioned into blocks and parameters estimated using MCMC. The associated reward function\u2019s tuning parameters are chosen through cross-validation.\n                  <\/jats:p>","DOI":"10.1007\/s00357-023-09455-x","type":"journal-article","created":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T23:21:55Z","timestamp":1700868115000},"page":"2-37","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Classification Under Partial Reject Options"],"prefix":"10.1007","volume":"41","author":[{"given":"M\u00e5ns","family":"Karlsson","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2767-8818","authenticated-orcid":false,"given":"Ola","family":"H\u00f6ssjer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,11,25]]},"reference":[{"key":"9455_CR1","doi-asserted-by":"crossref","unstructured":"Angelopoulos, A.\u00a0N. & Bates, S. (2022). A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv:2107:07511v6.","DOI":"10.1561\/9781638281597"},{"key":"9455_CR2","doi-asserted-by":"crossref","unstructured":"Arrow, K.\u00a0J., Blackwell, D., & Girshick, M.A. (1949). Bayes and minimax solutions of sequential decision problems. Econometrica, Journal of the Econometric Society, 213\u2013244.","DOI":"10.2307\/1905525"},{"key":"9455_CR3","doi-asserted-by":"crossref","unstructured":"Bechhofer, R.\u00a0E. (1954). A single-sample multiple decision procedure for ranking means of normal populations with known variances. The Annals of Mathematical Statistics, 16\u201339.","DOI":"10.1214\/aoms\/1177728845"},{"key":"9455_CR4","unstructured":"Berger, J.\u00a0O. (2013). Statistical decision theory and Bayesian analysis. Springer Science & Business Media."},{"key":"9455_CR5","volume-title":"Pattern recognition and machine learning","author":"CM Bishop","year":"2006","unstructured":"Bishop, C. M. (2006). Pattern recognition and machine learning. Springer."},{"key":"9455_CR6","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1002\/sim.3781","volume":"29","author":"M Bottai","year":"2010","unstructured":"Bottai, M., Cai, B., & McKeown, R. E. (2010). Logistic quantile regression for bounded outcomes. Statistical Medicine, 29, 309\u2013317.","journal-title":"Statistical Medicine"},{"key":"9455_CR7","doi-asserted-by":"crossref","unstructured":"Carlin, B.P., Kadane, J.B., & Gelfand, A.E. (1998). Approaches for optimal sequential decision analysis in clinical trials. Biometrics, 964\u2013975.","DOI":"10.2307\/2533849"},{"issue":"1","key":"9455_CR8","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1109\/TIT.1970.1054406","volume":"16","author":"C Chow","year":"1970","unstructured":"Chow, C. (1970). On optimum recognition error and reject tradeoff. IEEE Transactions on information theory, 16(1), 41\u201346.","journal-title":"IEEE Transactions on information theory"},{"key":"9455_CR9","unstructured":"Chzhen, E., Denis, C., Hebiri, M., & Lorieul, T. (2021). Set-valued classification-overview via a unified framework. arXiv:2102:12318v1."},{"key":"9455_CR10","first-page":"2273","volume":"10","author":"JJ del Coz","year":"2009","unstructured":"del Coz, J. J., D\u00edez, J., & Bahamonde, A. (2009). Learning nondeterministic classifiers. Journal of Machine Learning Research, 10, 2273\u20132293.","journal-title":"Journal of Machine Learning Research"},{"key":"9455_CR11","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s10994-012-5285-8","volume":"88","author":"K Dembczy\u0144ski","year":"2012","unstructured":"Dembczy\u0144ski, K., Waegeman, W., Cheng, W., & Hullermeier, E. (2012). On label dependence and loss minimization in multi-label classification. Machine Learning, 88, 5\u201345.","journal-title":"Machine Learning"},{"key":"9455_CR12","first-page":"1","volume":"18","author":"C Denis","year":"2017","unstructured":"Denis, C., & Hebiri, M. (2017). Confidence sets with expected sizes for multiclass classification. Journal of Machine Learning Research, 18, 1\u201328.","journal-title":"Journal of Machine Learning Research"},{"key":"9455_CR13","doi-asserted-by":"crossref","unstructured":"Freund, Y., Mansour, Y., & Schapire, R. E. (2004). Generalization bounds for averaged classifiers. The Annals of Statistics, 32(4), 1698\u20131722.","DOI":"10.1214\/009053604000000058"},{"key":"9455_CR14","unstructured":"Genz, A., Bretz, F., Miwa, T., Mi, X., Leisch, F., Scheipl, F., & Hothorn T. (2021). mvtnorm: Multivariate Normal and t Distributions. R package version 1.1-2."},{"key":"9455_CR15","doi-asserted-by":"crossref","unstructured":"Goldsman, D. (1986). Tutorial on indifference-zone normal means ranking and selection procedures. In Proceedings of the 18th conference on Winter simulation, pp. 370\u2013375.","DOI":"10.1145\/318242.318461"},{"key":"9455_CR16","first-page":"218","volume-title":"Information and classification, studies in classification, data analysis and knowledge organization","author":"E Grycko","year":"1993","unstructured":"Grycko, E. (1993). Classification with set-valued decision functions. In O. Opitz, B. Lausen, & R. Klar (Eds.), Information and classification, studies in classification, data analysis and knowledge organization (pp. 218\u2013224). Berlin, Heidelberg: Springer."},{"key":"9455_CR17","unstructured":"Ha, T.\u00a0M. (1996). An optimum class-selective rejection fule for pattern recognition. In Proceedings of the 13th International Conference on Pattern Recognition, Volume\u00a02, pp. 75\u201380. IEEE."},{"issue":"6","key":"9455_CR18","doi-asserted-by":"publisher","first-page":"608","DOI":"10.1109\/34.601248","volume":"19","author":"TM Ha","year":"1997","unstructured":"Ha, T. M. (1997). The optimum class-selective rejection rule. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(6), 608\u2013615.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"9455_CR19","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., & Friedman, J., et\u00a0al. (2009). The elements of statistical learning (Second ed.). Number\u00a010. Springer series in statistics New York.","DOI":"10.1007\/978-0-387-84858-7"},{"issue":"3","key":"9455_CR20","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1109\/TSSC.1970.300339","volume":"6","author":"ME Hellman","year":"1970","unstructured":"Hellman, M. E. (1970). The nearest neighbor classification rule with a reject option. IEEE Transactions on Systems Science and Cybernetics, 6(3), 179\u2013185.","journal-title":"IEEE Transactions on Systems Science and Cybernetics"},{"key":"9455_CR21","doi-asserted-by":"crossref","unstructured":"Herbei, R. & Wegkamp M.H. (2006). Classification with reject option. The Canadian Journal of Statistics\/La Revue Canadienne de Statistique, 709\u2013721.","DOI":"10.1002\/cjs.5550340410"},{"issue":"4","key":"9455_CR22","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1093\/jrsssc\/qlad036","volume":"72","author":"M Karlsson","year":"2023","unstructured":"Karlsson, M., & H\u00f6ssjer, O. (2023). Identification of taxon through classification with partial reject options. Journal of the Royal Statistical Society, Series C, 72(4), 937\u2013975.","journal-title":"Journal of the Royal Statistical Society, Series C"},{"key":"9455_CR23","doi-asserted-by":"crossref","unstructured":"Koenker (2005). Quantile Regression (Econometric Society monographs; no. 38). Cambridge University Press.","DOI":"10.1017\/CBO9780511754098"},{"key":"9455_CR24","doi-asserted-by":"publisher","first-page":"843","DOI":"10.1016\/j.patcog.2013.07.020","volume":"47","author":"H Le Capitaine","year":"2014","unstructured":"Le Capitaine, H. (2014). A unified view of class-selection with probabilistic classifiers. Pattern Recognition, 47, 843\u2013853.","journal-title":"Pattern Recognition"},{"key":"9455_CR25","volume-title":"The enterprise of knowledge: An assay on knowledge, credal probability, and chance","author":"I Levi","year":"1983","unstructured":"Levi, I. (1983). The enterprise of knowledge: An assay on knowledge, credal probability, and chance. MIT Press."},{"key":"9455_CR26","doi-asserted-by":"crossref","unstructured":"Lewis, D.\u00a0D. (1995). Evaluating and optimizing autonomous text classification systems. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR), pp. 246\u2013254. ACM.","DOI":"10.1145\/215206.215366"},{"key":"9455_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.106742","volume":"114","author":"L Ma","year":"2021","unstructured":"Ma, L., & Denoeux, T. (2021). Partial classification in the belief function framework. Knowledge-Based Systems, 114, 106742.","journal-title":"Knowledge-Based Systems"},{"issue":"July","key":"9455_CR28","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1080\/03078698.2013.811190","volume":"28","author":"B Malmhagen","year":"2013","unstructured":"Malmhagen, B., Karlsson, M., & Menzie, S. (2013). Using wing morphology to separate four species of Acrocephalus warblers in Scandinavia. Ringing & Migration, 28(July), 63\u201368.","journal-title":"Ringing & Migration"},{"key":"9455_CR29","unstructured":"Mortier, T., Hullermeyer, E. Dembczy\u0144ski, K., & Waegeman W. (2022). Set-valued prediction in hierarchical classification with constrained representation complexity. In Proceedings of the 38th Conference on Uncertainty in Artificial Intelligence (UAI 2022). PLMR, Vol. 180, pp 1392\u20131401."},{"issue":"4","key":"9455_CR30","doi-asserted-by":"publisher","first-page":"1435","DOI":"10.1007\/s10618-021-00751-x","volume":"35","author":"T Mortier","year":"2021","unstructured":"Mortier, T., Wydmuch, M., Dembczy\u0144ski, K., Hullermeier, E., & Waegeman, W. (2021). Efficient set-valued prediction in multi-class classification. Data Mining and Knowledge Discovery, 35(4), 1435\u20131469.","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"4","key":"9455_CR31","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1093\/comjnl\/7.4.308","volume":"7","author":"JA Nelder","year":"1965","unstructured":"Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7(4), 308\u2013313.","journal-title":"The Computer Journal"},{"key":"9455_CR32","doi-asserted-by":"crossref","unstructured":"Nguyen, V.-L. & E.\u00a0Hullermeier (2020). Reliable multilabel classification: Prediction with partial abstention. In Proceedings of the 34th AAAI Conference on Artificial Intelligene (AAAI-20), vol. 34, pp. 5264\u20135271.","DOI":"10.1609\/aaai.v34i04.5972"},{"key":"9455_CR33","volume-title":"R: A language and environment for statistical computing","author":"R Core Team","year":"2021","unstructured":"R Core Team. (2021). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing."},{"key":"9455_CR34","volume-title":"Pattern recognition and neural networks","author":"BD Ripley","year":"2007","unstructured":"Ripley, B. D. (2007). Pattern recognition and neural networks. Cambridge University Press."},{"key":"9455_CR35","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015). Set-valued classification - overview via a unified framework. International Journal of Computer Vision, 115, 211\u2013252.","journal-title":"International Journal of Computer Vision"},{"issue":"525","key":"9455_CR36","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1080\/01621459.2017.1395341","volume":"114","author":"M Sadinle","year":"2019","unstructured":"Sadinle, M., Lei, J., & Wasserman, L. (2019). Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association, 114(525), 223\u2013234.","journal-title":"Journal of the American Statistical Association"},{"key":"9455_CR37","unstructured":"Shafer, G. & Vovk, V. (2008). A tutorial on conformal prediction. Journal of Machine Learning Research\u00a09(3)."},{"key":"9455_CR38","unstructured":"Svensson, L. (1992). Identification guide to European passerines."},{"issue":"3","key":"9455_CR39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/jdwm.2007070101","volume":"3","author":"G Tsoumakas","year":"2007","unstructured":"Tsoumakas, G., & Katakis, I. (2007). Multi-label classification: An overview. International Journal of Data Warehousing and Mining, 3(3), 1\u201313.","journal-title":"International Journal of Data Warehousing and Mining"},{"key":"9455_CR40","unstructured":"Vovk, V., A.\u00a0Gammerman, & Shafer, G. (2005). Algorithmic learning in a random world. Springer Science & Business Media."},{"key":"9455_CR41","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s10472-017-9540-3","volume":"81","author":"V Vovk","year":"2017","unstructured":"Vovk, V., Nouretdinov, I., Federova, V., Petej, I., & Gammerman, A. (2017). Criteria of efficiency for set-valued classification. Annals of Mathematics and Artificial Intelligence, 81, 21\u201347.","journal-title":"Annals of Mathematics and Artificial Intelligence"},{"key":"9455_CR42","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-3472-7","volume-title":"Statistical reasoning with imprecise probabilities","author":"P Walley","year":"1991","unstructured":"Walley, P. (1991). Statistical reasoning with imprecise probabilities. Chapman and Hall."},{"issue":"12","key":"9455_CR43","doi-asserted-by":"publisher","first-page":"4316","DOI":"10.1109\/TCYB.2016.2607237","volume":"47","author":"G Yang","year":"2017","unstructured":"Yang, G., Destercke, S., & Marie-H\u00e9l\u00e8ne, M. (2017). The cost of indeterminacy: How to determine them? IEEE Transactions on Cybernetics, 47(12), 4316\u20134327.","journal-title":"IEEE Transactions on Cybernetics"},{"issue":"1","key":"9455_CR44","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/S0378-3758(01)00201-4","volume":"105","author":"M Zaffalon","year":"2002","unstructured":"Zaffalon, M. (2002). The naive credal classifier. Journal of Statistiical Planning and Inference, 105(1), 5\u201321.","journal-title":"Journal of Statistiical Planning and Inference"},{"key":"9455_CR45","doi-asserted-by":"publisher","first-page":"1282","DOI":"10.1016\/j.ijar.2012.06.022","volume":"53","author":"M Zaffalon","year":"2012","unstructured":"Zaffalon, M., Corani, G., & Mau\u00e1, D. (2012). Evaluating credal classifiers by utility-discounted predictive accuracy. International Journal of Approximate Reasoning, 53, 1282\u20131301.","journal-title":"International Journal of Approximate Reasoning"},{"key":"9455_CR46","doi-asserted-by":"publisher","first-page":"1819","DOI":"10.1109\/TKDE.2013.39","volume":"26","author":"M-L Zhang","year":"2013","unstructured":"Zhang, M.-L., & Zhou, Z.-H. (2013). A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 26, 1819\u20131837.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"}],"updated-by":[{"DOI":"10.1007\/s00357-023-09459-7","type":"erratum","label":"Erratum","source":"publisher","updated":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000}}],"container-title":["Journal of Classification"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00357-023-09455-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00357-023-09455-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00357-023-09455-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,13]],"date-time":"2024-04-13T05:02:52Z","timestamp":1712984572000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00357-023-09455-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,25]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["9455"],"URL":"https:\/\/doi.org\/10.1007\/s00357-023-09455-x","relation":{},"ISSN":["0176-4268","1432-1343"],"issn-type":[{"value":"0176-4268","type":"print"},{"value":"1432-1343","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,25]]},"assertion":[{"value":"23 October 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 November 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2023","order":3,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Erratum","order":4,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"An Erratum to this paper has been published:","order":5,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1007\/s00357-023-09459-7","URL":"https:\/\/doi.org\/10.1007\/s00357-023-09459-7","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}}]}}