{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T17:48:33Z","timestamp":1769881713252,"version":"3.49.0"},"reference-count":61,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T00:00:00Z","timestamp":1736985600000},"content-version":"vor","delay-in-days":15,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Junior Professor Chair in Trustworthy AI","award":["Ref. ANR-R311CHD"],"award-info":[{"award-number":["Ref. ANR-R311CHD"]}]},{"DOI":"10.13039\/100012945","name":"Universit\u00e9 de Technologie de Compi\u00e8gne","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100012945","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>In this paper, we present a formal framework to (1) aggregate probabilistic ensemble members into either a representative classifier or a credal classifier, and (2) perform various decision tasks based on this uncertainty quantification. We first elaborate on the aggregation problem under a class of distances between distributions. We then propose generic methods to robustify uncertainty quantification and decisions, based on the obtained ensemble and representative probability. To facilitate the scalability of the proposed framework, for all the problems and applications covered, we elaborate on their computational complexities from the theoretical aspects and leverage theoretical results to derive efficient algorithmic solutions. Finally, relevant sets of experiments are conducted to assess the usefulness of the proposed framework in uncertainty sampling, classification with a reject option, and set-valued prediction-making.<\/jats:p>","DOI":"10.1007\/s10994-024-06703-y","type":"journal-article","created":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T16:36:39Z","timestamp":1737045399000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Credal ensembling in multi-class classification"],"prefix":"10.1007","volume":"114","author":[{"given":"Vu-Linh","family":"Nguyen","sequence":"first","affiliation":[]},{"given":"Haifei","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"S\u00e9bastien","family":"Destercke","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,1,16]]},"reference":[{"key":"6703_CR1","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1016\/j.knosys.2017.07.019","volume":"134","author":"J Abell\u00e1n","year":"2017","unstructured":"Abell\u00e1n, J., Mantas, C. J., & Castellano, J. G. (2017). A random forest approach using imprecise probabilities. Knowledge-Based Systems, 134, 72\u201384.","journal-title":"Knowledge-Based Systems"},{"issue":"05","key":"6703_CR2","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1142\/S0218488512500353","volume":"20","author":"J Abellan","year":"2012","unstructured":"Abellan, J., & Masegosa, A. R. (2012). Imprecise classification with credal decision trees. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 20(05), 763\u2013787.","journal-title":"International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems"},{"key":"6703_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107739","volume":"112","author":"YCC Alarcon","year":"2021","unstructured":"Alarcon, Y. C. C., & Destercke, S. (2021). Imprecise gaussian discriminant classification. Pattern Recognition, 112, 107739.","journal-title":"Pattern Recognition"},{"key":"6703_CR4","doi-asserted-by":"crossref","unstructured":"Antonucci, A., & De\u00a0Campos, C. P. (2011). Decision making by credal nets. In 2011 Third international conference on intelligent human-machine systems and cybernetics (pp. 201\u2013204). IEEE.","DOI":"10.1109\/IHMSC.2011.55"},{"key":"6703_CR5","doi-asserted-by":"publisher","DOI":"10.1002\/9781118763117","volume-title":"Introduction to imprecise probabilities","author":"T Augustin","year":"2014","unstructured":"Augustin, T., Coolen, F. P., De Cooman, G., et al. (2014). Introduction to imprecise probabilities. John Wiley & Sons."},{"key":"6703_CR6","unstructured":"Bock, J. D., Campos, C. P. d., & Antonucci, A. (2014). Global sensitivity analysis for map inference in graphical models. In Proceedings of the 27th international conference on neural information processing systems (NIPS) (pp. 2690\u20132698)."},{"key":"6703_CR7","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex optimization","author":"S Boyd","year":"2004","unstructured":"Boyd, S., Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press."},{"key":"6703_CR8","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/BF00058655","volume":"24","author":"L Breiman","year":"1996","unstructured":"Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123\u2013140.","journal-title":"Machine Learning"},{"issue":"4","key":"6703_CR9","first-page":"300","volume":"1","author":"SH Cha","year":"2007","unstructured":"Cha, S. H. (2007). Comprehensive survey on distance\/similarity measures between probability density functions. International Journal of Mathematical models and Methods in Applied Sciences, 1(4), 300\u2013307.","journal-title":"International Journal of Mathematical models and Methods in Applied Sciences"},{"issue":"1","key":"6703_CR10","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1109\/TIT.1970.1054406","volume":"16","author":"C Chow","year":"1970","unstructured":"Chow, C. (1970). On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory, 16(1), 41\u201346.","journal-title":"IEEE Transactions on Information Theory"},{"key":"6703_CR11","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1016\/j.patcog.2016.10.011","volume":"63","author":"F Condessa","year":"2017","unstructured":"Condessa, F., Bioucas-Dias, J., & Kova\u010devi\u0107, J. (2017). Performance measures for classification systems with rejection. Pattern Recognition, 63, 437\u2013450.","journal-title":"Pattern Recognition"},{"key":"6703_CR12","doi-asserted-by":"publisher","first-page":"818","DOI":"10.1016\/j.csda.2012.11.010","volume":"71","author":"G Corani","year":"2014","unstructured":"Corani, G., & Antonucci, A. (2014). Credal ensembles of classifiers. Computational Statistics & Data Analysis, 71, 818\u2013831.","journal-title":"Computational Statistics & Data Analysis"},{"issue":"9","key":"6703_CR13","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1016\/j.ijar.2010.08.007","volume":"51","author":"G Corani","year":"2010","unstructured":"Corani, G., & De Campos, C. P. (2010). A tree augmented classifier based on extreme imprecise dirichlet model. International Journal of Approximate Reasoning, 51(9), 1053\u20131068.","journal-title":"International Journal of Approximate Reasoning"},{"issue":"4","key":"6703_CR14","first-page":"581","volume":"9","author":"G Corani","year":"2008","unstructured":"Corani, G., & Zaffalon, M. (2008). Learning reliable classifiers from small or incomplete data sets: The naive credal classifier 2. Journal of Machine Learning Research, 9(4), 581.","journal-title":"Journal of Machine Learning Research"},{"key":"6703_CR15","doi-asserted-by":"crossref","unstructured":"Datta, B. N. (2010). Numerical linear algebra and applications, vol 116. Siam","DOI":"10.1137\/1.9780898717655"},{"key":"6703_CR16","doi-asserted-by":"crossref","unstructured":"Decadt, A., Erreygers, A., & De\u00a0Bock, J., et\u00a0al. (2022). Decision-making with e-admissibility given a finite assessment of choices. In International conference on soft methods in probability and statistics (pp. 96\u2013103). Springer.","DOI":"10.1007\/978-3-031-15509-3_13"},{"issue":"10","key":"6703_CR17","first-page":"2273","volume":"10","author":"JJ Del Coz","year":"2009","unstructured":"Del Coz, J. J., D\u00edez, J., & Bahamonde, A. (2009). Learning nondeterministic classifiers. Journal of Machine Learning Research, 10(10), 2273.","journal-title":"Journal of Machine Learning Research"},{"key":"6703_CR18","doi-asserted-by":"crossref","unstructured":"Dietterich, T. G. (2000). Ensemble methods in machine learning. In Proceedings of the First International workshop on multiple classifier systems (MCS) (pp. 1\u201315). Springer.","DOI":"10.1007\/3-540-45014-9_1"},{"key":"6703_CR19","unstructured":"Elkan, C. (2001). The foundations of cost-sensitive learning. In Proceedings of the 17th international conference on artificial intelligence (IJCAI) (pp. 973\u2013978)."},{"issue":"3","key":"6703_CR20","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1111\/j.1751-5823.2002.tb00178.x","volume":"70","author":"AL Gibbs","year":"2002","unstructured":"Gibbs, A. L., & Su, F. E. (2002). On choosing and bounding probability metrics. International Statistical Review, 70(3), 419\u2013435.","journal-title":"International Statistical Review"},{"issue":"2","key":"6703_CR21","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1007\/s10994-023-06397-8","volume":"113","author":"C Gilet","year":"2024","unstructured":"Gilet, C., Guyomard, M., Destercke, S., et al. (2024). Softmin discrete minimax classifier for imbalanced classes and prior probability shifts. Machine Learning, 113(2), 605\u2013645.","journal-title":"Machine Learning"},{"key":"6703_CR22","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1016\/j.eswa.2016.12.035","volume":"73","author":"G Haixiang","year":"2017","unstructured":"Haixiang, G., Yijing, L., Shang, J., et al. (2017). Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 73, 220\u2013239.","journal-title":"Expert Systems with Applications"},{"key":"6703_CR23","unstructured":"Ho, T. K. (1995). Random decision forests. In textitProceedings of 3rd international conference on document analysis and recognition (ICDAR) (pp. 278\u2013282). IEEE."},{"key":"6703_CR24","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","volume":"110","author":"E H\u00fcllermeier","year":"2021","unstructured":"H\u00fcllermeier, E., & Waegeman, W. (2021). Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110, 457\u2013506.","journal-title":"Machine Learning"},{"key":"6703_CR25","doi-asserted-by":"crossref","unstructured":"Jansen, C., Schollmeyer, G., & Augustin, T. (2022). Quantifying degrees of e-admissibility in decision making with imprecise probabilities. In Reflections on the Foundations of Probability and Statistics: Essays in Honor of Teddy Seidenfeld (pp. 319\u2013346). Springer.","DOI":"10.1007\/978-3-031-15436-2_13"},{"issue":"2","key":"6703_CR26","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1109\/MCI.2022.3155327","volume":"17","author":"LV Jospin","year":"2022","unstructured":"Jospin, L. V., Laga, H., Boussaid, F., et al. (2022). Hands-on bayesian neural networks\u2013A tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2), 29\u201348.","journal-title":"IEEE Computational Intelligence Magazine"},{"issue":"3","key":"6703_CR27","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1109\/TSMCA.2010.2084081","volume":"41","author":"TM Khoshgoftaar","year":"2010","unstructured":"Khoshgoftaar, T. M., Van Hulse, J., & Napolitano, A. (2010). Comparing boosting and bagging techniques with noisy and imbalanced data. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 41(3), 552\u2013568.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans"},{"issue":"4","key":"6703_CR28","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1007\/s13748-016-0094-0","volume":"5","author":"B Krawczyk","year":"2016","unstructured":"Krawczyk, B. (2016). Learning from imbalanced data: Open challenges and future directions. Progress in Artificial Intelligence, 5(4), 221\u2013232.","journal-title":"Progress in Artificial Intelligence"},{"key":"6703_CR29","unstructured":"Lachiche, N., & Flach, P. A. (2003). Improving accuracy and cost of two-class and multi-class probabilistic classifiers using roc curves. In Proceedings of the 20th international conference on machine learning (ICML) (pp. 416\u2013423)."},{"key":"6703_CR30","doi-asserted-by":"crossref","unstructured":"Lee, L. (1999). Measures of distributional similarity. In Proceedings of the 37th annual meeting of the association for computational linguistics on computational linguistics (ACL) (pp. 25\u201332).","DOI":"10.3115\/1034678.1034693"},{"issue":"1","key":"6703_CR31","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1038\/s41746-022-00709-3","volume":"5","author":"A Lemay","year":"2022","unstructured":"Lemay, A., Hoebel, K., Bridge, C. P., et al. (2022). Improving the repeatability of deep learning models with monte carlo dropout. npj Digital Medicine, 5(1), 174.","journal-title":"npj Digital Medicine"},{"key":"6703_CR32","unstructured":"Levi, I. (1983). The enterprise of knowledge: An essay on knowledge, credal probability, and chance. MIT press"},{"key":"6703_CR33","unstructured":"Mindermann, S., Brauner, J. M., & Razzak, M. T., et\u00a0al. (2022). Prioritized training on points that are learnable, worth learning, and not yet learnt. In Proceedings of the 39th international conference on machine learning (ICML) (pp. 15630\u201315649). PMLR."},{"issue":"6","key":"6703_CR34","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1080\/03081079.2020.1778682","volume":"49","author":"I Montes","year":"2020","unstructured":"Montes, I., Miranda, E., & Destercke, S. (2020). Unifying neighbourhood and distortion models: Part i-new results on old models. International Journal of General Systems, 49(6), 602\u2013635.","journal-title":"International Journal of General Systems"},{"issue":"4","key":"6703_CR35","doi-asserted-by":"publisher","first-page":"1435","DOI":"10.1007\/s10618-021-00751-x","volume":"35","author":"T Mortier","year":"2021","unstructured":"Mortier, T., Wydmuch, M., Dembczy\u0144ski, K., et al. (2021). Efficient set-valued prediction in multi-class classification. Data Mining and Knowledge Discovery, 35(4), 1435\u20131469.","journal-title":"Data Mining and Knowledge Discovery"},{"key":"6703_CR36","volume-title":"Machine learning: A probabilistic perspective","author":"KP Murphy","year":"2012","unstructured":"Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT press."},{"key":"6703_CR37","volume-title":"Linear programming","author":"KG Murty","year":"1983","unstructured":"Murty, K. G. (1983). Linear programming. Springer."},{"key":"6703_CR38","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/j.ijar.2019.06.008","volume":"113","author":"N Nakharutai","year":"2019","unstructured":"Nakharutai, N., Troffaes, M. C., & Caiado, C. C. (2019). Improving and benchmarking of algorithms for decision making with lower previsions. International Journal of Approximate Reasoning, 113, 91\u2013105.","journal-title":"International Journal of Approximate Reasoning"},{"key":"6703_CR39","doi-asserted-by":"crossref","unstructured":"Nguyen, V. L., Destercke, S., Masson, M. H., et\u00a0al. (2018). Reliable multi-class classification based on pairwise epistemic and aleatoric uncertainty. In Proceedings of the 27th international joint conference on artificial intelligence (IJCAI) (pp. 5089\u20135095).","DOI":"10.24963\/ijcai.2018\/706"},{"key":"6703_CR40","doi-asserted-by":"crossref","unstructured":"Nguyen, V. L., H\u00fcllermeier, E., Rapp, M., et\u00a0al. (2020). On aggregation in ensembles of multilabel classifiers. In Proceedings of the 23rd international conference on discovery science (DS) (pp. 533\u2013547). Springer.","DOI":"10.1007\/978-3-030-61527-7_35"},{"key":"6703_CR41","unstructured":"Nguyen, V. L., Yang, Y., de\u00a0Campos, C. P. (2023a). Probabilistic multi-dimensional classification. In Proceedings of the 39th conference on uncertainty in artificial intelligence (UAI)."},{"key":"6703_CR42","doi-asserted-by":"crossref","unstructured":"Nguyen, V. L., Zhang, H., & Destercke, S. (2023b). Learning sets of probabilities through ensemble methods. In Proceedings of the 17th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU)","DOI":"10.1007\/978-3-031-45608-4_21"},{"key":"6703_CR43","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1613\/jair.1.12610","volume":"72","author":"VL Nguyen","year":"2021","unstructured":"Nguyen, V. L., & H\u00fcllermeier, E. (2021). Multilabel classification with partial abstention: Bayes-optimal prediction under label independence. Journal of Artificial Intelligence Research, 72, 613\u2013665.","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"1","key":"6703_CR44","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1007\/s10994-021-06003-9","volume":"111","author":"VL Nguyen","year":"2022","unstructured":"Nguyen, V. L., Shaker, M. H., & H\u00fcllermeier, E. (2022). How to measure uncertainty in uncertainty sampling for active learning. Machine Learning, 111(1), 89\u2013122.","journal-title":"Machine Learning"},{"key":"6703_CR45","doi-asserted-by":"crossref","unstructured":"O\u2019brien, D. B., Gupta, M. R., & Gray, R. M. (2008). Cost-sensitive multi-class classification from probability estimates. In Proceedings of the 25th international conference on machine learning (ICML) (pp. 712\u2013719).","DOI":"10.1145\/1390156.1390246"},{"key":"6703_CR46","doi-asserted-by":"crossref","unstructured":"Pugh, C. C. (2015). Real mathematical analysis. Undergraduate Texts in Mathematics","DOI":"10.1007\/978-3-319-17771-7"},{"key":"6703_CR47","doi-asserted-by":"publisher","first-page":"412","DOI":"10.1016\/j.patcog.2017.10.019","volume":"77","author":"B Quost","year":"2018","unstructured":"Quost, B., & Destercke, S. (2018). Classification by pairwise coupling of imprecise probabilities. Pattern Recognition, 77, 412\u2013425.","journal-title":"Pattern Recognition"},{"key":"6703_CR48","unstructured":"Rahimian, H., & Mehrotra, S. (2019). Distributionally robust optimization: A review. arXiv preprint arXiv:1908.05659"},{"issue":"2","key":"6703_CR49","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1137\/1035044","volume":"35","author":"RT Rockafellar","year":"1993","unstructured":"Rockafellar, R. T. (1993). Lagrange multipliers and optimality. SIAM Review, 35(2), 183.","journal-title":"SIAM Review"},{"key":"6703_CR50","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1016\/j.patcog.2016.03.012","volume":"57","author":"JA S\u00e1ez","year":"2016","unstructured":"S\u00e1ez, J. A., Krawczyk, B., & Wo\u017aniak, M. (2016). Analyzing the oversampling of different classes and types of examples in multi-class imbalanced datasets. Pattern Recognition, 57, 164\u2013178.","journal-title":"Pattern Recognition"},{"issue":"4","key":"6703_CR51","volume":"8","author":"O Sagi","year":"2018","unstructured":"Sagi, O., & Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1249.","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"issue":"6","key":"6703_CR52","doi-asserted-by":"publisher","first-page":"2168","DOI":"10.1214\/aos\/1034713653","volume":"23","author":"T Seidenfeld","year":"1995","unstructured":"Seidenfeld, T., Schervish, M. J., & Kadane, J. B. (1995). A representation of partially ordered preferences. The Annals of Statistics, 23(6), 2168\u20132217.","journal-title":"The Annals of Statistics"},{"key":"6703_CR53","doi-asserted-by":"crossref","unstructured":"Shaker, M. H., & H\u00fcllermeier, E. (2020). Aleatoric and epistemic uncertainty with random forests. In Proceedings of the eighteenth international symposium on intelligent data analysis (IDA) (pp. 444\u2013456).","DOI":"10.1007\/978-3-030-44584-3_35"},{"key":"6703_CR54","first-page":"1517","volume":"11","author":"BK Sriperumbudur","year":"2010","unstructured":"Sriperumbudur, B. K., Gretton, A., Fukumizu, K., et al. (2010). Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research, 11, 1517\u20131561.","journal-title":"The Journal of Machine Learning Research"},{"issue":"1","key":"6703_CR55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1348\/000711005X48266","volume":"59","author":"D Steinley","year":"2006","unstructured":"Steinley, D. (2006). K-means clustering: A half-century synthesis. British Journal of Mathematical and Statistical Psychology, 59(1), 1\u201334.","journal-title":"British Journal of Mathematical and Statistical Psychology"},{"issue":"1","key":"6703_CR56","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.ijar.2006.06.001","volume":"45","author":"MC Troffaes","year":"2007","unstructured":"Troffaes, M. C. (2007). Decision making under uncertainty using imprecise probabilities. International Journal of Approximate Reasoning, 45(1), 17\u201329.","journal-title":"International Journal of Approximate Reasoning"},{"issue":"5","key":"6703_CR57","doi-asserted-by":"publisher","first-page":"988","DOI":"10.1109\/72.788640","volume":"10","author":"VN Vapnik","year":"1999","unstructured":"Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5), 988\u2013999.","journal-title":"IEEE Transactions on Neural Networks"},{"key":"6703_CR58","unstructured":"Yang, G., Destercke, S., & Masson, M. H. (2014). Nested dichotomies with probability sets for multi-class classification. In Proceedings of the Twenty-first European conference on artificial intelligence (ECAI) (pp. 363\u2013368)."},{"issue":"8","key":"6703_CR59","doi-asserted-by":"publisher","first-page":"1282","DOI":"10.1016\/j.ijar.2012.06.022","volume":"53","author":"M Zaffalon","year":"2012","unstructured":"Zaffalon, M., Corani, G., & Mau\u00e1, D. (2012). Evaluating credal classifiers by utility-discounted predictive accuracy. International Journal of Approximate Reasoning, 53(8), 1282.","journal-title":"International Journal of Approximate Reasoning"},{"key":"6703_CR60","doi-asserted-by":"crossref","unstructured":"Zhang, H., Quost, B., & Masson, M. H. (2023). Cautious decision-making for tree ensembles. In Proceedings of the 17th European conference on symbolic and quantitative approaches to reasoning with uncertainty (ECSQARU).","DOI":"10.1007\/978-3-031-45608-4_1"},{"issue":"3","key":"6703_CR61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1753783.1753784","volume":"6","author":"J Zhu","year":"2010","unstructured":"Zhu, J., Wang, H., Hovy, E., et al. (2010). Confidence-based stopping criteria for active learning for data annotation. ACM Transactions on Speech and Language Processing, 6(3), 1\u201324.","journal-title":"ACM Transactions on Speech and Language Processing"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06703-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06703-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06703-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T16:20:51Z","timestamp":1738945251000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06703-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":61,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["6703"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06703-y","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"4 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 November 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 November 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We declare that we have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"We declare that this research did not require Ethical approval.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"All the authors of this manuscript consent to its publication.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"19"}}