{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T10:27:57Z","timestamp":1771064877174,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,1,10]],"date-time":"2023-01-10T00:00:00Z","timestamp":1673308800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004271","name":"Sapienza University","doi-asserted-by":"publisher","award":["RM1161550376E40E"],"award-info":[{"award-number":["RM1161550376E40E"]}],"id":[{"id":"10.13039\/501100004271","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004271","name":"Sapienza University","doi-asserted-by":"publisher","award":["RM120172B870E2E2"],"award-info":[{"award-number":["RM120172B870E2E2"]}],"id":[{"id":"10.13039\/501100004271","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>User requests to a customer service, also known as tickets, are essentially short texts in natural language. They should be grouped by topic to be answered efficiently. The effectiveness increases if this semantic categorization becomes automatic. We pursue this goal by using text mining to extract the features from the tickets, and classification to perform the categorization. This is however a difficult multi-class problem, and the classification algorithm needs a suitable hyperparameter configuration to produce a practically useful categorization. As recently highlighted by several researchers, the selection of these hyperparameters is often the crucial aspect. Therefore, we propose to view the hyperparameter choice as a higher-level optimization problem where the hyperparameters are the decision variables and the objective is the predictive performance of the classifier. However, an explicit analytical model of this problem cannot be defined. Therefore, we propose to solve it as a black-box model by means of derivative-free optimization techniques. We conduct experiments on a relevant application: the categorization of the requests received by the Contact Center of the Italian National Statistics Institute (Istat). Results show that the proposed approach is able to effectively categorize the requests, and that its performance is increased by the proposed hyperparameter optimization.<\/jats:p>","DOI":"10.3390\/a16010046","type":"journal-article","created":{"date-parts":[[2023,1,10]],"date-time":"2023-01-10T01:57:48Z","timestamp":1673315868000},"page":"46","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Hyperparameter Black-Box Optimization to Improve the Automatic Classification of Support Tickets"],"prefix":"10.3390","volume":"16","author":[{"given":"Renato","family":"Bruni","sequence":"first","affiliation":[{"name":"Department of Computer Control and Management Engineering, \u201cSapienza\u201d University of Rome, 00100 Rome, Italy"}]},{"given":"Gianpiero","family":"Bianchi","sequence":"additional","affiliation":[{"name":"Directorate for Data Collection, Italian National Institute of Statistics \u201cIstat\u201d, 00100 Rome, Italy"}]},{"given":"Pasquale","family":"Papa","sequence":"additional","affiliation":[{"name":"Directorate for Data Collection, Italian National Institute of Statistics \u201cIstat\u201d, 00100 Rome, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Aggarwal, C.C. (2018). Machine Learning for Text, Springer.","DOI":"10.1007\/978-3-319-73531-3"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1109\/TNSM.2017.2668363","article-title":"Knowledge Guided Hierarchical Multi-Label Classification Over Ticket Data","volume":"14","author":"Zeng","year":"2017","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.knosys.2018.03.003","article-title":"An automated text categorization framework based on hyperparameter optimization","volume":"149","author":"Tellez","year":"2018","journal-title":"Knowl.-Based Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Han, J., and Akbari, M. (2018, January 2\u20137). Vertical Domain Text Classification: Towards Understanding IT Tickets Using Deep Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence 2018, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11375"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ane, B., Cakravastia, A., and Diawati, L. (2019). A Knowledge Extraction Framework for Call Center Analytics. Proceedings of the 18th Online World Conference on Soft Computing in Industrial Applications (WSC18). Advances in Intelligent Systems and Computing, Springer.","DOI":"10.1007\/978-3-030-00612-9"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"129469","DOI":"10.1109\/ACCESS.2020.3007528","article-title":"Classification of Financial Tickets Using Weakly Supervised Fine-Grained Networks","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"193380","DOI":"10.1109\/ACCESS.2020.3032840","article-title":"IT Ticket Classification: The Simpler, the Better","volume":"8","author":"Revina","year":"2020","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"174","DOI":"10.25046\/aj050123","article-title":"Classification Model of Contact Center Customers Emails Using Machine Learning","volume":"5","author":"Putong","year":"2020","journal-title":"Adv. Sci. Technol. Eng. Syst. J."},{"key":"ref_9","first-page":"4263","article-title":"The automated machine learning classification approach on telco trouble ticket dataset","volume":"16","author":"Yayah","year":"2021","journal-title":"J. Eng. Sci. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"29","DOI":"10.24138\/jcomss.v17i1.1024","article-title":"Analysis of patterns and similarities in service tickets using natural language processing","volume":"17","author":"Tolciu","year":"2021","journal-title":"J. Commun. Softw. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"106622","DOI":"10.1016\/j.knosys.2020.106622","article-title":"AutoML: A survey of the state-of-the-art","volume":"212","author":"He","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_12","first-page":"1","article-title":"Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA","volume":"18","author":"Kotthoff","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_13","unstructured":"Bergstra, J., Bardenet, R., Bengio, Y., and K\u00e9gl, B. (2011). Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems (NIPS) 2011, NeurIPS."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.ins.2019.06.005","article-title":"A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves SVM classifiers","volume":"501","author":"Mantovani","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.knosys.2019.04.019","article-title":"Hyperparameter optimization of deep neural network using univariate dynamic encoding algorithm for searches","volume":"178","author":"Yoo","year":"2019","journal-title":"Knowl.-Based Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"106247","DOI":"10.1016\/j.knosys.2020.106247","article-title":"Fast hyperparameter tuning using Bayesian optimization with directional derivatives","volume":"205","author":"Joy","year":"2020","journal-title":"Knowl.-Based Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1089\/big.2019.0123","article-title":"MonkeyKing: Adaptive Parameter Tuning on Big Data Platforms with Deep Reinforcement Learning","volume":"8","author":"Du","year":"2020","journal-title":"Big Data"},{"key":"ref_18","first-page":"1","article-title":"Hyperband: A novel bandit-based approach to hyperparameter optimization","volume":"18","author":"Li","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1016\/j.ins.2022.01.040","article-title":"Auto-CASH: A meta-learning embedding approach for autonomous classification algorithm selection","volume":"591","author":"Mu","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_20","unstructured":"Mikolov, T., Le, Q.V., and Sutskever, I. (2013). Exploiting Similarities among Languages for Machine Translation. arXiv."},{"key":"ref_21","first-page":"1090","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Proc. Adv. Neural Inf. Process. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2119","DOI":"10.1162\/089976601750399335","article-title":"Training v-support vector classifiers: Theory and algorithms","volume":"13","author":"Chang","year":"2001","journal-title":"Neural Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1016\/j.ejor.2015.12.018","article-title":"Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO","volume":"252","author":"Boukouvala","year":"2016","journal-title":"Eur. J. Oper. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1007\/s12532-020-00182-7","article-title":"An algorithmic framework based on primitive directions and nonmonotone line searches for black-box optimization problems with integer variables","volume":"12","author":"Liuzzi","year":"2020","journal-title":"Math. Program. Comput."},{"key":"ref_25","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_26","unstructured":"Yoon, K. (2014, January 25\u201329). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar."},{"key":"ref_27","unstructured":"Nwankpa, C., Ijomah, W., Gachagan, A., and Marshall, S. (2018). Activation functions: Comparison of trends in practice and research for deep learning. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Vapnik, V. (1995). The Nature of Statistical Learning Theory, Springer. [2nd ed.].","DOI":"10.1007\/978-1-4757-2440-0"},{"key":"ref_29","unstructured":"Oswal, B.V. (2022, December 01). CNN-Text-Classification-Keras, GitHub Repository. Available online: https:\/\/github.com\/bhaveshoswal\/CNN-text-classification-keras."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2349","DOI":"10.1109\/TKDE.2015.2416727","article-title":"Effective Classification using Binarization and Statistical Analysis","volume":"27","author":"Bruni","year":"2015","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.cor.2018.04.014","article-title":"Logical Analysis of Data as a Tool for the Analysis of Probabilistic Discrete Choice Behavior","volume":"106","author":"Bruni","year":"2019","journal-title":"Comput. Oper. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"113001","DOI":"10.1016\/j.eswa.2019.113001","article-title":"Website categorization: A formal approach and robustness analysis in the case of e-commerce detection","volume":"142","author":"Bruni","year":"2019","journal-title":"Expert Syst. Appl."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/1\/46\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:05:08Z","timestamp":1760119508000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/1\/46"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,10]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["a16010046"],"URL":"https:\/\/doi.org\/10.3390\/a16010046","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,10]]}}}