{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T15:48:01Z","timestamp":1781884081773,"version":"3.54.5"},"reference-count":32,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,5,19]],"date-time":"2025-05-19T00:00:00Z","timestamp":1747612800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science Committee of the Ministry of Science and Higher Education of the Republic of Kazakhstan","award":["AP19677451"],"award-info":[{"award-number":["AP19677451"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This article presents a comprehensive review of short text clustering using state-of-the-art methods: Bidirectional Encoder Representations from Transformers (BERT), Term Frequency-Inverse Document Frequency (TF-IDF), and the novel hybrid method Latent Dirichlet Allocation + BERT + Autoencoder (LDA + BERT + AE). The article begins by outlining the theoretical foundation of each technique and its merits and limitations. BERT is critiqued for its ability to understand word dependence in text, while TF-IDF is lauded for its applicability in terms of importance assessment. The experimental section compares the efficacy of these methods in clustering short texts, with a specific focus on the hybrid LDA + BERT + AE approach. A detailed examination of the LDA-BERT model\u2019s training and validation loss over 200 epochs shows that the loss values start above 1.2 and quickly decrease to around 0.8 within the first 25 epochs, eventually stabilizing at approximately 0.4. The close alignment of these curves suggests the model\u2019s practical learning and generalization capabilities, with minimal overfitting. The study demonstrates that the hybrid LDA + BERT + AE method significantly enhances text clustering quality compared to individual methods. Based on the findings, the study recommends the optimum choice and use of clustering methods for different short texts and natural language processing operations. The applications of these methods in industrial and educational settings, where successful text handling and categorization are critical, are also addressed. The study ends by emphasizing the importance of the holistic handling of short texts for deeper semantic comprehension and effective information retrieval.<\/jats:p>","DOI":"10.3390\/a18050289","type":"journal-article","created":{"date-parts":[[2025,5,19]],"date-time":"2025-05-19T05:37:13Z","timestamp":1747633033000},"page":"289","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Analysis of Short Texts Using Intelligent Clustering Methods"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9179-0428","authenticated-orcid":false,"given":"Jamalbek","family":"Tussupov","sequence":"first","affiliation":[{"name":"Department of Information Systems, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4614-4021","authenticated-orcid":false,"given":"Akmaral","family":"Kassymova","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Zhangir Khan University, Uralsk 090000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ayagoz","family":"Mukhanova","sequence":"additional","affiliation":[{"name":"Department of Information Systems, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Assyl","family":"Bissengaliyeva","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Zhangir Khan University, Uralsk 090000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanar","family":"Azhibekova","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Technologies, Non-Profit Joint Stock Company S. Asfendiyarov Kazakh National Medical University, Almaty 050000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Moldir","family":"Yessenova","sequence":"additional","affiliation":[{"name":"Department of Information Systems, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanargul","family":"Abuova","sequence":"additional","affiliation":[{"name":"Department of Information Technology, Zhangir Khan University, Uralsk 090000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"21415","DOI":"10.1007\/s00521-023-08629-3","article-title":"Multilingual Text Categorization and Sentiment Analysis: A Comparative Analysis of Multilingual Approaches for Classifying Twitter Data","volume":"35","author":"Manias","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.patrec.2023.02.027","article-title":"Re-Ranking and TOPSIS-Based Ensemble Feature Selection with Multi-Stage Aggregation for Text Categorization","volume":"168","author":"Fu","year":"2023","journal-title":"Pattern Recognit. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5309","DOI":"10.1007\/s12652-019-01399-8","article-title":"Sentiment Analysis and Text Categorization of Cancer Medical Records with LSTM","volume":"14","author":"Edara","year":"2023","journal-title":"J. Ambient Intell. Humaniz. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100395","DOI":"10.1016\/j.cosrev.2021.100395","article-title":"Machine Learning Algorithms for Social Media Analysis: A Survey","volume":"40","author":"Balaji","year":"2021","journal-title":"Comput. Sci. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2016556","DOI":"10.1080\/23311975.2021.2016556","article-title":"Bibliometrix Analysis of Information Sharing in Social Media","volume":"9","author":"Abbas","year":"2022","journal-title":"Cogent Bus. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1007\/s10708-022-10584-w","article-title":"Collecting, Analyzing, and Visualizing Location-Based Social Media Data: Review of Methods in GIS-Social Media Analysis","volume":"88","author":"McKitrick","year":"2023","journal-title":"GeoJournal"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1450","DOI":"10.1080\/09669582.2020.1851699","article-title":"Climate Crisis and Flying: Social Media Analysis Traces the Rise of \u201cFlightshame\u201d","volume":"29","author":"Becken","year":"2021","journal-title":"J. Sustain. Tour."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/TCSS.2019.2956957","article-title":"A Survey of Sentiment Analysis from Social Media Data","volume":"7","author":"Chakraborty","year":"2020","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Horta Ribeiro, M., Cheng, J., and West, R. (May, January 30). Automated Content Moderation Increases Adherence to Community Guidelines. Proceedings of the ACM Web Conference 2023, New York, NY, USA.","DOI":"10.1145\/3543507.3583275"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, Q., Hong, Y., and Raghu, T.S. (2021). The Effects of Machine-Powered Platform Governance: An Empirical Study of Content Moderation. SSRN Electron. J., Available online: http:\/\/hdl.handle.net\/10125\/80064.","DOI":"10.2139\/ssrn.3767680"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1002\/poi3.391","article-title":"Can Facebook\u2019s Community Standards Keep Up with Legal Certainty? Content Moderation Governance under the Pressure of the Digital Services Act","volume":"16","author":"Fasel","year":"2024","journal-title":"Policy Internet"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"339","DOI":"10.32604\/iasc.2023.031987","article-title":"A Machine Learning-Based Technique with Intelligent WordNet Lemmatize for Twitter Sentiment Analysis","volume":"36","author":"Saranya","year":"2023","journal-title":"Intell. Autom. Soft Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1038\/s42256-023-00729-y","article-title":"A Taxonomy and Review of Generalization Research in NLP","volume":"5","author":"Hupkes","year":"2023","journal-title":"Nat. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"105020","DOI":"10.1016\/j.autcon.2023.105020","article-title":"Comparing Natural Language Processing (NLP) Applications in Construction and Computer Science Using Preferred Reporting Items for Systematic Reviews (PRISMA)","volume":"154","author":"Chung","year":"2023","journal-title":"Autom. Constr."},{"key":"ref_15","first-page":"30","article-title":"The Implementation of an AI-Driven Advertising Push System Based on a NLP Algorithm","volume":"1","author":"Xin","year":"2023","journal-title":"Int. J. Comput. Sci. Inf. Technol."},{"key":"ref_16","first-page":"1140","article-title":"NLP Transformers: Analysis of LLMs and Traditional Approaches for Enhanced Text Summarization","volume":"32","year":"2024","journal-title":"Eski\u015fehir Osman. Univ. J. Eng. Archit. Fac."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1177\/0049124118769114","article-title":"The Future of Coding: A Comparison of Hand-Coding and Three Types of Computer-Assisted Text Analysis Methods","volume":"50","author":"Nelson","year":"2021","journal-title":"Sociol. Methods Res."},{"key":"ref_18","unstructured":"Zhang, X., Ju, T., Liang, H., Fu, Y., and Zhang, Q. (2024). LLMs Instruct LLMs: An Extraction and Editing Method. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., and Sun, L. (2023). A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT. arXiv.","DOI":"10.1007\/s13042-024-02443-6"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1177\/0165551521990616","article-title":"Unsupervised Extractive Multi-Document Summarization Method Based on Transfer Learning from BERT Multi-Task Fine-Tuning","volume":"49","author":"Lamsiyah","year":"2023","journal-title":"J. Inf. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"120114","DOI":"10.1016\/j.eswa.2023.120114","article-title":"Discovering Topics and Trends in the Field of Artificial Intelligence: Using LDA Topic Modeling","volume":"225","author":"Yu","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1007\/s42979-022-01634-8","article-title":"Aspect Oriented Sentiment Analysis on Customer Reviews on Restaurant Using the LDA and BERT Method","volume":"4","author":"Lohith","year":"2023","journal-title":"SN Comput. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"110176","DOI":"10.1016\/j.asoc.2023.110176","article-title":"A Comprehensive Survey on Design and Application of Autoencoder in Deep Learning","volume":"138","author":"Li","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, S., and Guo, W. (2023). Auto-Encoders in Deep Learning\u2014A Review with New Perspectives. Mathematics, 11.","DOI":"10.3390\/math11081777"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"69812","DOI":"10.1109\/ACCESS.2024.3397775","article-title":"Advancements in Generative AI: A Comprehensive Review of GANs, GPT, Autoencoders, Diffusion Model, and Transformers","volume":"12","author":"Bengesi","year":"2024","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ahmed, M.H., Tiun, S., Omar, N., and Sani, N.S. (2023). Short Text Clustering Algorithms, Application and Challenges: A Survey. Appl. Sci., 13.","DOI":"10.3390\/app13010342"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1016\/j.ins.2022.11.139","article-title":"K-Means Clustering Algorithms: A Comprehensive Review, Variants Analysis, and Advances in the Era of Big Data","volume":"622","author":"Ikotun","year":"2023","journal-title":"Inf. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e252965","DOI":"10.48048\/asi.2023.252965","article-title":"Business Intelligent Framework Using Sentiment Analysis for Smart Digital Marketing in the E-Commerce Era","volume":"16","author":"Kyaw","year":"2023","journal-title":"Asia Soc. Issues"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5133","DOI":"10.1007\/s10462-022-10254-w","article-title":"Short Text Topic Modelling Approaches in the Context of Big Data: Taxonomy, Survey, and Analysis","volume":"56","author":"Murshed","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Habbak, H., Mahmoud, M., Metwally, K., Fouda, M.M., and Ibrahem, M.I. (2023). Load Forecasting Techniques and Their Applications in Smart Grids. Energies, 16.","DOI":"10.3390\/en16031480"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"92037","DOI":"10.1109\/ACCESS.2019.2927345","article-title":"Discovering Topic Representative Terms for Short Text Clustering","volume":"7","author":"Yang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_32","unstructured":"Tussupov, J. (2025, May 05). Text Clustering with BERT and LDA. Available online: https:\/\/github.com\/JamalbekTussupov01\/Text-clustering\/tree\/main."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/5\/289\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:34:55Z","timestamp":1760031295000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/5\/289"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,19]]},"references-count":32,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["a18050289"],"URL":"https:\/\/doi.org\/10.3390\/a18050289","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,19]]}}}