{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T15:19:09Z","timestamp":1775229549105,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,8,26]],"date-time":"2022-08-26T00:00:00Z","timestamp":1661472000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"<jats:p>The multi-label customer reviews classification task aims to identify the different thoughts of customers about the product they are purchasing. Due to the impact of the COVID-19 pandemic, customers have become more prone to shopping online. As a consequence, the amount of text data on e-commerce is continuously increasing, which enables new studies to be carried out and important findings to be obtained with more detailed analysis. Nowadays, e-commerce customer reviews are analyzed by both researchers and sector experts, and are subject to many sentiment analysis studies. Herein, an analysis of customer reviews is carried out in order to obtain more in-depth thoughts about the product, rather than engaging in emotion-based analysis. Initially, we form a new customer reviews dataset made up of reviews by Turkish consumers in order to perform the proposed analysis. The created dataset contains more than 50,000 reviews in three different categories, and each review has multiple labels according to the comments made by the customers. Later, we applied machine learning methods employed for multi-label classification to the dataset. Finally, we compared and analyzed the results we obtained using a diverse set of statistical metrics. As a result of our experimental studies, we found the Micro Precision 0.9157, Micro Recall 0.8837, Micro F1 Score 0.8925, and Hamming Loss 0.0278 to be the most successful approaches.<\/jats:p>","DOI":"10.3390\/axioms11090436","type":"journal-article","created":{"date-parts":[[2022,8,28]],"date-time":"2022-08-28T21:22:56Z","timestamp":1661721776000},"page":"436","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":49,"title":["Multi-Label Classification of E-Commerce Customer Reviews via Machine Learning"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1563-9256","authenticated-orcid":false,"given":"Emre","family":"Deniz","sequence":"first","affiliation":[{"name":"Computer Engineering Department, Hitit University, Corum 19030, T\u00fcrkiye"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7555-541X","authenticated-orcid":false,"given":"Hasan","family":"Erbay","sequence":"additional","affiliation":[{"name":"Computer Engineering Department, University of Turkish Aeronautical Association, Ankara 06790, T\u00fcrkiye"}]},{"given":"Mustafa","family":"Co\u015far","sequence":"additional","affiliation":[{"name":"Computer Engineering Department, Hitit University, Corum 19030, T\u00fcrkiye"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., and Brown, D. (2019). Text classification algorithms: A survey. Information, 10.","DOI":"10.3390\/info10040150"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Rusli, A., Young, J.C., and Iswari, N.M.S. (2020, January 7\u20138). Identifying fake news in Indonesian via supervised binary text classification. Proceedings of the 2020 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), Bali, Indonesia.","DOI":"10.1109\/IAICT50021.2020.9172020"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"143721","DOI":"10.1109\/ACCESS.2019.2944089","article-title":"Hybrid water cycle optimization algorithm with simulated annealing for spam e-mail detection","volume":"7","author":"Mamat","year":"2019","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1585","DOI":"10.1109\/TIFS.2017.2675361","article-title":"NetSpam: A network-based spam detection framework for reviews in online social media","volume":"12","author":"Shehnepoor","year":"2017","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"13575","DOI":"10.1007\/s11042-020-10361-2","article-title":"Novel authorship verification model for social media accounts compromised by a human","volume":"80","author":"Erbay","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_6","first-page":"8869681","article-title":"Design and Analysis of a Novel Authorship Verification Framework for Hijacked Social Media Accounts Compromised by a Human","volume":"2021","author":"Erbay","year":"2021","journal-title":"Secur. Commun. Netw."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1696","DOI":"10.1109\/TKDE.2014.2382600","article-title":"TASC: Topic-adaptive sentiment classification on dynamic tweets","volume":"27","author":"Liu","year":"2014","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Esposito, F., Corazza, A., and Cutugno, F. (2016, January 5\u20137). Topic Modelling with Word Embeddings. Proceedings of the Third Italian Conference on Computational Linguistics (CLiC-it 2016) & Fifth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2016), Napoli, Italy.","DOI":"10.4000\/books.aaccademia.1767"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1515\/pralin-2016-0006","article-title":"A minimally supervised approach for synonym extraction with word embeddings","volume":"105","author":"Leeuwenberg","year":"2016","journal-title":"Prague Bull. Math. Linguist."},{"key":"ref_10","unstructured":"WPForms (2022, August 04). 68 Useful eCommerce Statistics You Must Know in 2022. Available online: https:\/\/wpforms.com\/ecommerce-statistics\/."},{"key":"ref_11","unstructured":"Nasdaq (2022, August 04). UK Online Shopping and E-Commerce Statistics for 2017. Available online: https:\/\/www.nasdaq.com\/articles\/uk-online-shopping-and-e-commerce-statistics-2017-2017-03-14."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"189513","DOI":"10.1109\/ACCESS.2020.3031588","article-title":"A multiclassification model of sentiment for E-commerce reviews","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1016\/j.asej.2014.04.011","article-title":"Sentiment analysis algorithms and applications: A survey","volume":"5","author":"Medhat","year":"2014","journal-title":"Ain Shams Eng. J."},{"key":"ref_14","first-page":"8","article-title":"Support vector machine (svm) optimization using grid search and unigram to improve e-commerce review accuracy","volume":"1","author":"Muslim","year":"2020","journal-title":"J. Soft Comput. Explor."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102221","DOI":"10.1016\/j.ipm.2020.102221","article-title":"E-commerce product review sentiment classification based on a na\u00efve Bayes continuous learning framework","volume":"57","author":"Xu","year":"2020","journal-title":"Inf. Process. Manag."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Vanaja, S., and Belwal, M. (2018, January 11). Aspect-level sentiment analysis on e-commerce data. Proceedings of the 2018 International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India.","DOI":"10.1109\/ICIRCA.2018.8597286"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jabbar, J., Urooj, I., JunSheng, W., and Azeem, N. (2019, January 9\u201311). Real-time sentiment analysis on E-commerce application. Proceedings of the 2019 IEEE 16th International Conference on Networking, Sensing and Control (ICNSC), Banff, AB, Canada.","DOI":"10.1109\/ICNSC.2019.8743331"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Parveen, N., Santhi, M., Burra, L.R., Pellakuri, V., and Pellakuri, H. Women\u2019s e-commerce clothing sentiment analysis by probabilistic model LDA using R-SPARK. Mater. Today Proc., 2021. in press.","DOI":"10.1016\/j.matpr.2020.10.064"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4520","DOI":"10.1016\/j.matpr.2020.08.012","article-title":"Analysis and prediction of extent of helpfulness of reviews on E-commerce websites","volume":"33","author":"Tripathi","year":"2020","journal-title":"Mater. Today Proc."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kumar, K.S., Desai, J., and Majumdar, J. (2016, January 15\u201317). Opinion mining and sentiment analysis on online customer review. Proceedings of the 2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), Chennai, India.","DOI":"10.1109\/ICCIC.2016.7919584"},{"key":"ref_21","unstructured":"Miyoshi, T., and Nakagami, Y. (2017, January 5\u20138). Sentiment classification of customer reviews on electric products. Proceedings of the 2007 IEEE International Conference on Systems, Man and Cybernetics, Banff, AB, Canada."},{"key":"ref_22","unstructured":"Guan, Z., Chen, L., Zhao, W., Zheng, Y., Tan, S., and Cai, D. (2016, January 9\u201315). Weakly-Supervised Deep Learning for Customer Review Sentiment Classification. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), New York, NY, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"119121","DOI":"10.1109\/ACCESS.2019.2937518","article-title":"Customer reviews analysis with deep neural networks for e-commerce recommender systems","volume":"7","author":"Shoja","year":"2019","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"121014","DOI":"10.1109\/ACCESS.2020.3006569","article-title":"Sentiment analysis via deep multichannel neural networks with variational information bottleneck","volume":"8","author":"Gu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"B\u0130len, B., and Horasan, F. (2021). LSTM network based sentiment analysis for customer reviews. Politek. Derg.","DOI":"10.2339\/politeknik.844019"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Vural, A.G., Cambazoglu, B.B., Senkul, P., and Tokgoz, Z.O. (2013). A framework for sentiment analysis in turkish: Application to polarity detection of movie reviews in turkish. Computer and Information Sciences III, Springer.","DOI":"10.1007\/978-1-4471-4594-3_45"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Acikalin, U.U., Bardak, B., and Kutlu, M. (2020, January 5\u20137). Turkish sentiment analysis using bert. Proceedings of the 2020 28th Signal Processing and Communications Applications Conference (SIU), Gaziantep, Turkey.","DOI":"10.1109\/SIU49456.2020.9302492"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Santur, Y. (2019, January 21\u201322). Sentiment analysis based on gated recurrent unit. Proceedings of the 2019 International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey.","DOI":"10.1109\/IDAP.2019.8875985"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"114231","DOI":"10.1016\/j.eswa.2020.114231","article-title":"A new topic modeling based approach for aspect extraction in aspect based sentiment analysis: SS-LDA","volume":"168","author":"Ozyurt","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kadhim, A.I. (2019, January 2\u20134). Term weighting for feature extraction on Twitter: A comparison between BM25 and TF-IDF. Proceedings of the 2019 International Conference on Advanced Science and Engineering (ICOASE), Zakho, Iraq.","DOI":"10.1109\/ICOASE.2019.8723825"},{"key":"ref_31","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_32","first-page":"3111","article-title":"Distributed representations of words and phrases and their compositionality","volume":"26","author":"Mikolov","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C.D. (2014, January 25\u201329). GloVe: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_34","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30, Long Beach, CA, USA."},{"key":"ref_35","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_36","first-page":"3770924","article-title":"Berturk-Bert Models for Turkish","volume":"2020","author":"Schweter","year":"2020","journal-title":"Zenodo"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1819","DOI":"10.1109\/TKDE.2013.39","article-title":"A review on multi-label learning algorithms","volume":"26","author":"Zhang","year":"2013","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1007\/s11704-017-7031-7","article-title":"Binary relevance for multi-label learning: An overview","volume":"12","author":"Zhang","year":"2018","journal-title":"Front. Comput. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/jdwm.2007070101","article-title":"Multi-label classification: An overview","volume":"3","author":"Tsoumakas","year":"2007","journal-title":"Int. J. Data Warehous. Min. (IJDWM)"}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/11\/9\/436\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:16:14Z","timestamp":1760141774000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/11\/9\/436"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,26]]},"references-count":39,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["axioms11090436"],"URL":"https:\/\/doi.org\/10.3390\/axioms11090436","relation":{},"ISSN":["2075-1680"],"issn-type":[{"value":"2075-1680","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,26]]}}}