{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T16:04:25Z","timestamp":1783699465001,"version":"3.55.0"},"reference-count":20,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,6,1]],"date-time":"2021-06-01T00:00:00Z","timestamp":1622505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a novel approach to the assessment of decision confidence when multi-class recognition is concerned. When many classification problems are considered, while eliminating human interaction with the system might be one goal, it is not the only possible option\u2014lessening the workload of human experts can also bring huge improvement to the production process. The presented approach focuses on providing a tool that will significantly decrease the amount of work that the human expert needs to conduct while evaluating different samples. Instead of hard classification, which assigns a single label to each class, the described solution focuses on evaluating each case in terms of decision confidence\u2014checking how sure the classifier is in the case of the currently processed example, and deciding if the final classification should be performed, or if the sample should instead be manually evaluated by a human expert. The method can be easily adjusted to any number of classes. It can also focus either on the classification accuracy or coverage of the used dataset, depending on user preferences. Different confidence functions are evaluated in that aspect. The results obtained during experiments meet the initial criteria, providing an acceptable quality for the final solution.<\/jats:p>","DOI":"10.3390\/s21113834","type":"journal-article","created":{"date-parts":[[2021,6,1]],"date-time":"2021-06-01T23:07:03Z","timestamp":1622588823000},"page":"3834","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Decision Confidence Assessment in Multi-Class Classification"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1567-879X","authenticated-orcid":false,"given":"Micha\u0142","family":"Bukowski","sequence":"first","affiliation":[{"name":"Institute of Information Technology, Warsaw University of Life Sciences, Nowoursynowska 159, 02-776 Warsaw, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2789-4732","authenticated-orcid":false,"given":"Jaros\u0142aw","family":"Kurek","sequence":"additional","affiliation":[{"name":"Institute of Information Technology, Warsaw University of Life Sciences, Nowoursynowska 159, 02-776 Warsaw, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3157-3381","authenticated-orcid":false,"given":"Izabella","family":"Antoniuk","sequence":"additional","affiliation":[{"name":"Institute of Information Technology, Warsaw University of Life Sciences, Nowoursynowska 159, 02-776 Warsaw, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8935-845X","authenticated-orcid":false,"given":"Albina","family":"Jegorowa","sequence":"additional","affiliation":[{"name":"Institute of Wood Sciences and Furniture, Warsaw University of Life Sciences, Nowoursynowska 159, 02-776 Warsaw, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3379","DOI":"10.15376\/biores.14.2.3379-3388","article-title":"Use of cutting force and vibro-acoustic signals in tool wear monitoring based on multiple regression technique for compreg milling","volume":"14","author":"Szymanowski","year":"2019","journal-title":"BioResources"},{"key":"ref_2","first-page":"633","article-title":"Developing automatic recognition system ofdrill wear in standard laminated chipboard drilling process","volume":"64","author":"Kurek","year":"2016","journal-title":"Bull. Pol. Acad. Sci. Technol. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/s00170-011-3504-2","article-title":"Tool condition monitoring based on numerous signal features","volume":"59","author":"Jemielniak","year":"2012","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/j.jmatprotec.2005.10.021","article-title":"Drill wear monitoring using back propagation neural network","volume":"172","author":"Panda","year":"2006","journal-title":"J. Mater. Process. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/S0952-1976(00)00008-7","article-title":"Multi-sensor integration for on-line tool wear estimation through artificial neural networks and fuzzy neural network","volume":"13","author":"Kuo","year":"2000","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3","DOI":"10.22630\/MGV.2019.28.1.1","article-title":"Data Augmentation Techniques for Transfer Learning Improvement in Drill Wear Classification Using Convolutional Neural Network","volume":"28","author":"Kurek","year":"2019","journal-title":"Mach. Graph. Vis."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"13","DOI":"10.22630\/MGV.2019.28.1.2","article-title":"Classifiers ensemble of transfer learning for improved drill wear classification using convolutional neural network","volume":"28","author":"Kurek","year":"2019","journal-title":"Mach. Graph. Vis."},{"key":"ref_8","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00226-020-01245-7","article-title":"Deep learning methods for drill wear classification based on images of holes drilled in melamine faced chipboard","volume":"55","author":"Jegorowa","year":"2021","journal-title":"Wood Sci. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kurek, J., Antoniuk, I., \u015awiderski, B., Jegorowa, A., and Bukowski, M. (2020). Application of Siamese Networks to the Recognition of the Drill Wear State Based on Images of Drilled Holes. Sensors, 20.","DOI":"10.3390\/s20236978"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"9611","DOI":"10.15376\/biores.15.4.9611-9624","article-title":"Time-efficient Approach to Drill Condition Monitoring Based on Images of Holes Drilled in Melamine Faced Chipboard","volume":"15","author":"Jegorowa","year":"2020","journal-title":"BioResources"},{"key":"ref_12","first-page":"3146","article-title":"LightGBM: A highly efficient gradient boosting decision tree","volume":"30","author":"Ke","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_13","unstructured":"Nwankpa, C., Ijomah, W., Gachagan, A., and Marshall, S. (2018). Activation Functions: Comparison of trends in Practice and Research for Deep Learning. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2160","DOI":"10.1111\/risa.13313","article-title":"Shannon Entropy for Quantifying Uncertainty and Risk in Economic Disparity","volume":"39","author":"Mishra","year":"2019","journal-title":"Risk Anal."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1007\/BF00379115","article-title":"Comparison of skewness coefficient, coefficient of variation, and Gini coefficient as inequality measures within populations","volume":"78","author":"Bendel","year":"1989","journal-title":"Oecologia"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1186\/cc3000","article-title":"Statistics review 13: Receiver operating characteristic curves","volume":"8","author":"Bewick","year":"2004","journal-title":"Crit. Care"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Dosilovic, F.K., Brcic, M., and Hlupic, N. (2018, January 21\u201325). Explainable artificial intelligence: A survey. Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia.","DOI":"10.23919\/MIPRO.2018.8400040"},{"key":"ref_18","unstructured":"Chopra, P. (2021, February 09). Making Your Neural Network Say \u201cI Don\u2019t Know\u201d\u2014Bayesian NNs Using Pyro and PyTorch. 27 November 2018. Available online: https:\/\/towardsdatascience.com\/making-your-neural-network-say-i-dont-know-bayesian-nns-using-pyro-and-pytorch-b1c24e6ab8cd."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kuncheva, L.I. (2000). Fuzzy Classifier Design, Springer.","DOI":"10.1007\/978-3-7908-1850-5"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nakashima, T., and Ghosh, A. (2011, January 7\u201310). Classification Confidence of Fuzzy Rule-Based Classifiers. Proceedings of the 25th European Conference on Modelling and Simulation (ECMS), Krakow, Poland.","DOI":"10.7148\/2011-0466-0471"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/11\/3834\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:09:53Z","timestamp":1760162993000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/11\/3834"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,1]]},"references-count":20,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["s21113834"],"URL":"https:\/\/doi.org\/10.3390\/s21113834","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,1]]}}}