{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T10:58:05Z","timestamp":1788173885678,"version":"build-2803163510"},"reference-count":54,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T00:00:00Z","timestamp":1651708800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>A very important task of Natural Language Processing is text categorization (or text classification), which aims to automatically classify a document into categories. This kind of task includes numerous applications, such as sentiment analysis, language or intent detection, heavily used by social-\/brand-monitoring tools, customer service, and the voice of customer, among others. Since the introduction of Fuzzy Set theory, its application has been tested in many fields, from bioinformatics to industrial and commercial use, as well as in cases with vague, incomplete, or imprecise data, highlighting its importance and usefulness in the fields. The most important aspect of the application of Fuzzy Set theory is the measures employed to calculate how similar or dissimilar two samples in a dataset are. In this study, we evaluate the performance of 43 similarity and 19 distance measures in the task of text document classification, using the widely used BBC News and BBC Sports benchmark datasets. Their performance is optimized through hyperparameter optimization techniques and evaluated via a leave-one-out cross-validation technique, presenting their performance using the accuracy, precision, recall, and F1-score metrics.<\/jats:p>","DOI":"10.3390\/info13050235","type":"journal-article","created":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T13:10:26Z","timestamp":1651756226000},"page":"235","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Text Classification Using Intuitionistic Fuzzy Set Measures\u2014An Evaluation Study"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3722-0934","authenticated-orcid":false,"given":"George K.","family":"Sidiropoulos","sequence":"first","affiliation":[{"name":"MLV Research Group, Department of Computer Science, International Hellenic University, 65404 Kavala, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikolaos","family":"Diamianos","sequence":"additional","affiliation":[{"name":"MLV Research Group, Department of Computer Science, International Hellenic University, 65404 Kavala, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0394-7454","authenticated-orcid":false,"given":"Kyriakos D.","family":"Apostolidis","sequence":"additional","affiliation":[{"name":"MLV Research Group, Department of Computer Science, International Hellenic University, 65404 Kavala, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5545-1499","authenticated-orcid":false,"given":"George A.","family":"Papakostas","sequence":"additional","affiliation":[{"name":"MLV Research Group, Department of Computer Science, International Hellenic University, 65404 Kavala, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/1500000011","article-title":"Opinion Mining and Sentiment Analysis","volume":"2","author":"Pang","year":"2008","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_2","unstructured":"Liu, B. (2022, March 14). Sentiment Analysis and Opinion Mining. Available online: https:\/\/www.cs.uic.edu\/liub\/FBS\/SentimentAnalysis-and-OpinionMining.pdf."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"\u0158eh\u016f\u0159ek, R., and Kolkus, M. (2009). Language Identification on the Web: Extending the Dictionary Method, Springer.","DOI":"10.1007\/978-3-642-00382-0_29"},{"key":"ref_4","unstructured":"Cai, W., Cai, D., Huang, S., and Li, M. (2022, March 14). Utterance-Level End-to-End Language Identification Using Attention-Based CNN-BLSTM. Available online: https:\/\/arxiv.org\/abs\/1902.07374."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Brenes, D.J., Gayo-Avello, D., and P\u00e9rez-Gonz\u00e1lez, K. (2009, January 9\u201311). Survey and Evaluation of Query Intent Detection Methods. Proceedings of the 2009 Workshop on Web Search Click Data WSCD \u201909, Barcelona, Spain.","DOI":"10.1145\/1507509.1507510"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, C., and Blei, D.M. (2011). Collaborative Topic Modeling for Recommending Scientific Articles, ACM Press.","DOI":"10.1145\/2020408.2020480"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1016\/S0019-9958(65)90241-X","article-title":"Fuzzy sets","volume":"8","author":"Zadeh","year":"1965","journal-title":"Inf. Control."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.mbs.2018.08.004","article-title":"Uncertainty, imprecision, and many-valued logics in protein bioinformatics","volume":"309","year":"2019","journal-title":"Math. Biosci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/S0165-0114(86)80034-3","article-title":"Intuitionistic Fuzzy Sets","volume":"20","author":"Atanassov","year":"1986","journal-title":"Fuzzy Sets Syst."},{"key":"ref_10","unstructured":"Intarapaiboon, P. (2019). A Framework for Text Classification Using Intuitionistic Fuzzy Sets, Springer."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Abbasi, B.Z., Hussain, S., and Faisal, M.I. (2019). An Automated Text Classification Method: Using Improved Fuzzy Set Approach for Feature Selection, IEEE.","DOI":"10.1109\/IBCAST.2019.8667159"},{"key":"ref_12","unstructured":"Widyantoro, D.H., and Yen, J. (2022, March 14). A Fuzzy Similarity Approach in Text Classification Task. Available online: https:\/\/www.researchgate.net\/publication\/3845652_A_fuzzy_similarity_approach_in_text_classification_task."},{"key":"ref_13","unstructured":"Puri, S. (2022, March 14). A Fuzzy Similarity Based Concept Mining Model for Text Classification. Available online: https:\/\/arxiv.org\/ftp\/arxiv\/papers\/1204\/1204.2061.pdf."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1109\/TKDE.2010.122","article-title":"A fuzzy self-constructing feature clustering algorithm for text classification","volume":"23","author":"Jiang","year":"2011","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2813","DOI":"10.1016\/j.eswa.2011.08.141","article-title":"Expert Systems with Applications FSKNN: Multi-label text categorization based on fuzzy similarity and k nearest neighbors q","volume":"39","author":"Jiang","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Szmidt, E., and Kacprzyk, J. (2008). Using Intuitionistic Fuzzy Sets in Text Categorization. Artificial Intelligence and Soft Computing\u2013ICAISC 2008, Springer.","DOI":"10.1007\/978-3-540-69731-2_35"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bounabi, M., Moutaouakil, K.E., and Satori, K. (2019). Text Classification Using Fuzzy TF-IDF and Machine Learning Models, ACM.","DOI":"10.1145\/3372938.3372956"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"9915","DOI":"10.1007\/s00500-020-05410-9","article-title":"Text Document Classification Using Fuzzy Rough Set Based on Robust Nearest Neighbor (FRS-RNN)","volume":"25","author":"Behera","year":"2021","journal-title":"Soft Comput."},{"key":"ref_19","unstructured":"Sharma, H., Saraswat, M., Yadav, A., Kim, J.H., and Bansal, J.C. (2020, January 5\u20136). Efficient Fuzzy Similarity-Based Text Classification with SVM and Feature Reduction. Proceedings of the Congress on Intelligent Systems, New Delhi, India."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2063","DOI":"10.1016\/j.patrec.2005.03.018","article-title":"Distance measure between Intuitionistic Fuzzy Sets","volume":"26","author":"Wang","year":"2005","journal-title":"Pattern Recognit. Lett."},{"key":"ref_21","unstructured":"Kacprzyk, J. (1997). Multistage Fuzzy Control: A Prescriptive Approach, John Wiley & Sons, Inc.. [1st ed.]."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Atanassov, K.T. (1999). Intuitionistic Fuzzy Sets. Studies in Fuzziness and Soft Computing, Physica-Verlag HD Fuzzy Sets and Systems.","DOI":"10.1007\/978-3-7908-1870-3"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1016\/S0165-0114(98)00244-9","article-title":"Distances between Intuitionistic Fuzzy Sets","volume":"114","author":"Szmidt","year":"2000","journal-title":"Fuzzy Sets Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/S0167-8655(01)00110-6","article-title":"New Similarity Measures of Intuitionistic Fuzzy Sets and Application to Pattern Recognitions","volume":"23","author":"Li","year":"2002","journal-title":"Pattern Recognit. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3101","DOI":"10.1016\/S0167-8655(03)00169-7","article-title":"On the Dengfeng\u2013Chuntian Similarity Measure and Its Application to Pattern Recognition","volume":"24","author":"Mitchell","year":"2003","journal-title":"Pattern Recognit. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1603","DOI":"10.1016\/j.patrec.2004.06.006","article-title":"Similarity Measures of Intuitionistic Fuzzy Sets Based on Hausdorff Distance","volume":"25","author":"Hung","year":"2004","journal-title":"Pattern Recognit. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/0165-0114(94)00339-9","article-title":"Measures of similarity between vague sets","volume":"74","author":"Chen","year":"1995","journal-title":"Fuzzy Sets Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sidiropoulos, G.K., Apostolidis, K.D., Damianos, N., and Papakostas, G.A. (2022). Fsmpy: A Fuzzy Set Measures Python Library. Information, 13.","DOI":"10.3390\/info13020064"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1016\/j.fss.2003.08.005","article-title":"Distances between Intuitionistic Fuzzy Sets and\/or Interval-Valued Fuzzy Sets Based on the Hausdorff Metric","volume":"148","author":"Grzegorzewski","year":"2004","journal-title":"Fuzzy Sets Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1109\/3477.552198","article-title":"Similarity Measures between Vague Sets and between Elements","volume":"27","author":"Chen","year":"1997","journal-title":"IEEE Trans. Syst. Man Cybern. Part Cybern."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/S0020-0255(98)10083-X","article-title":"A Note on Similarity Measures between Vague Sets and between Elements","volume":"115","author":"Hong","year":"1999","journal-title":"Inf. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2687","DOI":"10.1016\/S0167-8655(03)00111-9","article-title":"Similarity Measures on Intuitionistic Fuzzy Sets","volume":"24","author":"Liang","year":"2003","journal-title":"Pattern Recognit. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.mcm.2005.04.002","article-title":"New Similarity Measures between Intuitionistic Fuzzy Sets and between Elements","volume":"42","author":"Liu","year":"2005","journal-title":"Math. Comput. Model."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1016\/j.patrec.2005.11.020","article-title":"Similarity Measures on Three Kinds of Fuzzy Sets","volume":"27","author":"Zhang","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.ijar.2006.10.002","article-title":"Similarity Measures of Intuitionistic Fuzzy Sets Based on Lp Metric","volume":"46","author":"Hung","year":"2007","journal-title":"Int. J. Approx. Reason."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1007\/978-3-540-71441-5_3","article-title":"New Similarity Measures on Intuitionistic Fuzzy Sets","volume":"Volume 40","author":"Cao","year":"2007","journal-title":"Fuzzy Information and Engineering"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1002\/int.20271","article-title":"On Similarity Measures between Intuitionistic Fuzzy Sets","volume":"23","author":"Hung","year":"2008","journal-title":"Int. J. Intell. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1641","DOI":"10.1016\/j.ins.2007.11.006","article-title":"On the J-divergence of Intuitionistic Fuzzy Sets with Its Application to Pattern Recognition","volume":"178","author":"Hung","year":"2008","journal-title":"Inf. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.mcm.2010.07.022","article-title":"Cosine Similarity Measures for Intuitionistic Fuzzy Sets and Their Applications","volume":"53","author":"Ye","year":"2011","journal-title":"Math. Comput. Model."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/978-3-642-33478-8_36","article-title":"Modified Cosine Similarity Measure between Intuitionistic Fuzzy Sets","volume":"Volume 7530","author":"Lei","year":"2012","journal-title":"Artificial Intelligence and Computational Intelligence"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1219","DOI":"10.1016\/j.patrec.2012.01.008","article-title":"On the Mitchell Similarity Measure and Its Application to Pattern Recognition","volume":"33","author":"Julian","year":"2012","journal-title":"Pattern Recognit. Lett."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.patrec.2014.02.010","article-title":"Intuitionistic Fuzzy Similarity Measures Based on Frank T-Norms Family","volume":"42","author":"Iancu","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1016\/j.ins.2015.04.028","article-title":"Monotonic Similarity Measures between Intuitionistic Fuzzy Sets and Their Relationship with Entropy and Inclusion Measure","volume":"316","author":"Deng","year":"2015","journal-title":"Inf. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1007\/s10489-014-0596-z","article-title":"A Novel Similarity Measure on Intuitionistic Fuzzy Sets with Its Applications","volume":"42","author":"Song","year":"2015","journal-title":"Appl. Intell."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ins.2016.01.040","article-title":"A Novel Similarity Measure between Intuitionistic Fuzzy Sets Based on the Centroid Points of Transformed Fuzzy Numbers with Applications to Pattern Recognition","volume":"343\u2013344","author":"Chen","year":"2016","journal-title":"Inf. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.asoc.2015.12.002","article-title":"A Similarity Measure of Intuitionistic Fuzzy Soft Sets and Its Application in Medical Diagnosis","volume":"41","author":"Muthukumar","year":"2016","journal-title":"Appl. Soft Comput."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"8665","DOI":"10.1016\/j.eswa.2012.01.199","article-title":"Consistency of 2D and 3D Distances of Intuitionistic Fuzzy Sets","volume":"39","author":"Yang","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.patrec.2006.07.004","article-title":"Intuitionistic Fuzzy Information\u2013Applications to Pattern Recognition","volume":"28","author":"Vlachos","year":"2007","journal-title":"Pattern Recognit. Lett."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.eswa.2015.09.045","article-title":"A Novel Similarity\/Dissimilarity Measure for Intuitionistic Fuzzy Sets and Its Application in Pattern Recognition","volume":"45","author":"Nguyen","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Greene, D., and Cunningham, P. (2006, January 25\u201329). Practical Solutions to the Problem of Diagonal Dominance in Kernel Document Clustering. Proceedings of the 23rd International Conference on Machine Learning (ICML\u201906), Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143892"},{"key":"ref_51","unstructured":"Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., M\u00fcller, A., Nothman, J., and Louppe, G. (2021, August 11). Scikit-Learn: Machine Learning in Python. Available online: https:\/\/arxiv.org\/abs\/1201.0490."},{"key":"ref_52","unstructured":"Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python, O\u2019Reilly. [1st ed.]."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1002\/int.21529","article-title":"A novel distance measure of Intuitionistic Fuzzy Sets and its application to pattern recognition problems","volume":"27","author":"Hatzimichailidis","year":"2012","journal-title":"Int. J. Intell. Syst."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1007\/s10700-007-9004-z","article-title":"Some Similarity Measures of Intuitionistic Fuzzy Sets and Their Applications to Multiple Attribute Decision Making","volume":"6","author":"Xu","year":"2007","journal-title":"Fuzzy Optim. Decis. Mak."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/5\/235\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:06:33Z","timestamp":1760137593000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/5\/235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,5]]},"references-count":54,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["info13050235"],"URL":"https:\/\/doi.org\/10.3390\/info13050235","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,5]]}}}