{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:35:55Z","timestamp":1776443755601,"version":"3.51.2"},"reference-count":51,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,2,29]],"date-time":"2024-02-29T00:00:00Z","timestamp":1709164800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Software"],"abstract":"<jats:p>This paper presents a pioneering methodology for refining product recommender systems, introducing a synergistic integration of unsupervised models\u2014K-means clustering, content-based filtering (CBF), and hierarchical clustering\u2014with the cutting-edge GPT-4 large language model (LLM). Its innovation lies in utilizing GPT-4 for model evaluation, harnessing its advanced natural language understanding capabilities to enhance the precision and relevance of product recommendations. A flask-based API simplifies its implementation for e-commerce owners, allowing for the seamless training and evaluation of the models using CSV-formatted product data. The unique aspect of this approach lies in its ability to empower e-commerce with sophisticated unsupervised recommender system algorithms, while the GPT model significantly contributes to refining the semantic context of product features, resulting in a more personalized and effective product recommendation system. The experimental results underscore the superiority of this integrated framework, marking a significant advancement in the field of recommender systems and providing businesses with an efficient and scalable solution to optimize their product recommendations.<\/jats:p>","DOI":"10.3390\/software3010004","type":"journal-article","created":{"date-parts":[[2024,2,29]],"date-time":"2024-02-29T08:13:44Z","timestamp":1709194424000},"page":"62-80","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Precision-Driven Product Recommendation Software: Unsupervised Models, Evaluated by GPT-4 LLM for Enhanced Recommender Systems"],"prefix":"10.3390","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8098-1616","authenticated-orcid":false,"given":"Konstantinos I.","family":"Roumeliotis","sequence":"first","affiliation":[{"name":"Department of Informatics and Telecommunications, University of Peloponnese, Akadimaikou G. K. Vla-chou Street, 22131 Tripoli, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5799-3558","authenticated-orcid":false,"given":"Nikolaos D.","family":"Tselikas","sequence":"additional","affiliation":[{"name":"Department of Informatics and Telecommunications, University of Peloponnese, Akadimaikou G. K. Vla-chou Street, 22131 Tripoli, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7386-1029","authenticated-orcid":false,"given":"Dimitrios K.","family":"Nasiopoulos","sequence":"additional","affiliation":[{"name":"Department of Agribusiness and Supply Chain Management, School of Applied Economics and Social Sciences, Agricultural University of Athens, 11855 Athens, Greece"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.eswa.2017.03.069","article-title":"Does Order Matter? Effect of Order in Group Recommendation","volume":"82","author":"Agarwal","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100048","DOI":"10.1016\/j.nlp.2023.100048","article-title":"A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4","volume":"6","author":"Kalyan","year":"2023","journal-title":"Nat. Lang. Process. J."},{"key":"ref_3","unstructured":"(2024, February 18). Products Suggestions for WooCommerce\u2014WordPress Plugin|WordPress.Org. Available online: https:\/\/wordpress.org\/plugins\/cart-products-suggestions-for-woocommerce\/."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lai, C.H., and Tseng, K.C. (2022). Applying Deep Learning Models to Analyze Users\u2019 Aspects, Sentiment, and Semantic Features for Product Recommendation. Appl. Sci., 12.","DOI":"10.3390\/app12042118"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, B., Xu, H., Zhao, Q., Su, P., Chabbi, M., Jiao, S., and Liu, X. (2022, January 21\u201329). OJXPERF: Featherlight Object Replica Detection for Java Programs. Proceedings of the 44th International Conference on Software Engineering 2022, Pittsburgh, PA, USA.","DOI":"10.1145\/3510003.3510083"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, B., Zhao, Q., Jiao, S., and Liu, X. (2023, January 2\u20136). DroidPerf: Profiling Memory Objects on Android Devices. Proceedings of the Annual International Conference on Mobile Computing and Networking, Madrid, Spain.","DOI":"10.1145\/3570361.3592503"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hell, F., Taha, Y., Hinz, G., Heibei, S., M\u00fcller, H., and Knoll, A. (2020). Graph Convolutional Neural Network for a Pharmacy Cross-Selling Recommender System. Information, 11.","DOI":"10.3390\/info11110525"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1080\/07421222.2021.1912930","article-title":"Recommendations and Cross-Selling: Pricing Strategies When Personalizing Firms Cross-Sell","volume":"38","author":"Ghoshal","year":"2021","journal-title":"J. Manag. Inf. Syst."},{"key":"ref_9","unstructured":"Vatavwala, S., Kumar, B., and Sharma, A. (2024). Customer Centric Support Services in the Digital Age, Palgrave Macmillan."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1007\/s13385-020-00236-z","article-title":"A Recommendation System for Car Insurance","volume":"10","author":"Lesage","year":"2020","journal-title":"Eur. Actuar. J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1016\/j.jretai.2022.03.005","article-title":"The Dark Side of Up-Selling Promotions: Evidence from an Analysis of Cross-Brand Purchase Behavior","volume":"98","author":"Park","year":"2022","journal-title":"J. Retail."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhu, T., Harrington, P., Li, J., and Tang, L. (2014, January 6\u201311). Bundle Recommendation in ECommerce. Proceedings of the SIGIR 2014\u201437th International ACM SIGIR Conference on Research and Development in Information Retrieval, Gold Coast, Australia.","DOI":"10.1145\/2600428.2609603"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhou, H., Xiong, F., Chen, H.A., Zhou, H., Xiong, F., and Chen, H. (2023). A Comprehensive Survey of Recommender Systems Based on Deep Learning. Appl. Sci., 13.","DOI":"10.3390\/app132011378"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1007\/s11257-021-09318-3","article-title":"EvoRecSys: Evolutionary Framework for Health and Well-Being Recommender Systems","volume":"32","author":"Cartlidge","year":"2022","journal-title":"User Model. User-Adapt. Interact."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1007\/s41019-020-00138-w","article-title":"Co-Purchaser Recommendation for Online Group Buying","volume":"5","author":"Chen","year":"2020","journal-title":"Data Sci. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"113475","DOI":"10.1016\/j.dss.2020.113475","article-title":"Recommendation Systems and Convergence of Online Reviews: The Type of Product Network Matters!","volume":"142","author":"Khobzi","year":"2021","journal-title":"Decis. Support Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1016\/j.procs.2021.12.194","article-title":"Mudjahidin Development of Conceptual Model to Increase Customer Interest Using Recommendation System in E-Commerce","volume":"197","author":"Wijaya","year":"2022","journal-title":"Procedia Comput. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Aldino, A.A., Pratiwi, E.D., Sintaro, S., and Putra, A.D. (2021, January 27\u201328). Comparison of Market Basket Analysis to Determine Consumer Purchasing Patterns Using Fp-Growth and Apriori Algorithm. Proceedings of the 2021 International Conference on Computer Science, Information Technology, and Electrical Engineering (ICOMITEE), Banyuwangi, Indonesia.","DOI":"10.1109\/ICOMITEE53461.2021.9650317"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1934","DOI":"10.1016\/j.procs.2020.03.215","article-title":"Generating Items Recommendations by Fusing Content and User-Item Based Collaborative Filtering","volume":"167","author":"Tewari","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Rahmatillah, I., Astuty, E., and Sudirman, I.D. (2023, January 25\u201326). An Improved Decision Tree Model for Forecasting Consumer Decision in a Medium Groceries Store. Proceedings of the 2023 IEEE 17th International Conference on Industrial and Information Systems, ICIIS, Peradeniya, Sri Lanka.","DOI":"10.1109\/ICIIS58898.2023.10253592"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jbene, M., Tigani, S., Rachid, S., and Chehri, A. (2021). Deep Neural Network and Boosting Based Hybrid Quality Ranking for E-Commerce Product Search. Big Data Cogn. Comput., 5.","DOI":"10.3390\/bdcc5030035"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1016\/j.ins.2020.02.073","article-title":"A Survey of Evolutionary Computation for Association Rule Mining","volume":"524","author":"Telikani","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"102735","DOI":"10.1016\/j.jvcir.2019.102735","article-title":"E-Commerce Personalized Recommendation Analysis by Deeply-Learned Clustering","volume":"71","author":"Wang","year":"2020","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"274","DOI":"10.3991\/ijet.v16i03.18851","article-title":"A Review of Content-Based and Context-Based Recommendation Systems","volume":"16","author":"Javed","year":"2021","journal-title":"Int. J. Emerg. Technol. Learn. iJET"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"6087","DOI":"10.1080\/03772063.2021.1997357","article-title":"Matrix Factorization in Recommender Systems: Algorithms, Applications, and Peculiar Challenges","volume":"69","author":"Isinkaye","year":"2023","journal-title":"IETE J. Res."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wu, G., Sanner, S., Luo, K., and Soh, H. (2019, January 16\u201320). Deep Language-Based Critiquing for Recommender Systems. Proceedings of the RecSys 2019\u201413th ACM Conference on Recommender Systems, Copenhagen, Denmark.","DOI":"10.1145\/3298689.3347009"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bhaskaran, S., Marappan, R., and Santhi, B. (2021). Design and Analysis of a Cluster-Based Intelligent Hybrid Recommendation System for E-Learning Applications. Mathematics, 9.","DOI":"10.3390\/math9020197"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ko, H., Lee, S., Park, Y., and Choi, A. (2022). A Survey of Recommendation Systems: Recommendation Models, Techniques, and Application Fields. Electronics, 11.","DOI":"10.3390\/electronics11010141"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"100056","DOI":"10.1016\/j.nlp.2024.100056","article-title":"LLMs in E-Commerce: A Comparative Analysis of GPT and LLaMA Models in Product Review Evaluation","volume":"6","author":"Roumeliotis","year":"2024","journal-title":"Nat. Lang. Process. J."},{"key":"ref_30","unstructured":"Rothman, D. (2021). Transformers for Natural Language Processing: Build Innovative Deep Neural Network Architectures for NLP with Python, PyTorch, TensorFlow, BERT, Packt Publishing."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.aiopen.2022.03.001","article-title":"Data Augmentation Approaches in Natural Language Processing: A Survey","volume":"3","author":"Li","year":"2022","journal-title":"AI Open"},{"key":"ref_32","unstructured":"Fanni, S.C., Febi, M., Aghakhanyan, G., and Neri, E. (2023). Introduction to Artificial Intelligence, Springer."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"de Curt\u00f2, J., de Zarz\u00e0, I., and Calafate, C.T. (2023). Semantic Scene Understanding with Large Language Models on Unmanned Aerial Vehicles. Drones, 7.","DOI":"10.3390\/drones7020114"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Roumeliotis, K.I., Tselikas, N.D., and Nasiopoulos, D.K. (2023). Unveiling Sustainability in Ecommerce: GPT-Powered Software for Identifying Sustainable Product Features. Sustainability, 15.","DOI":"10.3390\/su151512015"},{"key":"ref_35","unstructured":"Liu, Z., Yu, X., Zhang, L., Wu, Z., Cao, C., Dai, H., Zhao, L., Liu, W., Shen, D., and Li, Q. (2023). DeID-GPT: Zero-Shot Medical Text De-Identification by GPT-4. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1016\/j.fmre.2021.11.011","article-title":"A Commentary of GPT-3 in MIT Technology Review 2021","volume":"1","author":"Zhang","year":"2021","journal-title":"Fundam. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s11334-020-00372-5","article-title":"Product Recommendation for E-Commerce Business by Applying Principal Component Analysis (PCA) and K-Means Clustering: Benefit for the Society","volume":"17","author":"Bandyopadhyay","year":"2021","journal-title":"Innov. Syst. Softw. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"80716","DOI":"10.1109\/ACCESS.2020.2988796","article-title":"Unsupervised K-Means Clustering Algorithm","volume":"8","author":"Sinaga","year":"2020","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Nguyen, L.V., Nguyen, T.H., and Jung, J.J. (2020, January 13\u201316). Content-Based Collaborative Filtering Using Word Embedding: A Case Study on Movie Recommendation. Proceedings of the RACS\u201920: International Conference on Research in Adaptive and Convergent Systems, Gwangju, Republic of Korea.","DOI":"10.1145\/3400286.3418253"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"15724","DOI":"10.1109\/TITS.2022.3222789","article-title":"Hierarchical Clustering Based on Dendrogram in Sustainable Transportation Systems","volume":"24","author":"Sangaiah","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lamb, D.S., Downs, J., and Reader, S. (2020). Space-Time Hierarchical Clustering for Identifying Clusters in Spatiotemporal Point Data. ISPRS Int. J. Geo-Inf., 9.","DOI":"10.3390\/ijgi9020085"},{"key":"ref_42","first-page":"3828","article-title":"An Ensemble Agglomerative Hierarchical Clustering Algorithm Based on Clusters Clustering Technique and the Novel Similarity Measurement","volume":"34","author":"Li","year":"2022","journal-title":"J. King Saud Univ.\u2014Comput. Inf. Sci."},{"key":"ref_43","unstructured":"(2024, February 03). Amazon UK Products Dataset 2023 (2.2M Products). Available online: https:\/\/www.kaggle.com\/datasets\/asaniczka\/amazon-uk-products-dataset-2023."},{"key":"ref_44","unstructured":"(2024, February 03). Sklearn.Model_Selection.Train_Test_Split\u2014Scikit-Learn 1.4.0 Documentation. Available online: https:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.model_selection.train_test_split.html."},{"key":"ref_45","unstructured":"(2024, February 03). GitHub\u2014Kroumeliotis\/Product-Recommendations-Software-Unsupervised-Models-Evaluated-by-GPT-4-LLM: Precision-Driven Product Recommendations Software: Un-Leashing the Power of Unsupervised Models, Evaluated by GPT-4 LLM for Enhanced Recommender Systems. Available online: https:\/\/github.com\/kroumeliotis\/Product-Recommendations-Software-Unsupervised-Models-Evaluated-by-GPT-4-LLM."},{"key":"ref_46","unstructured":"(2024, February 03). Colab.Google. Available online: https:\/\/colab.google\/."},{"key":"ref_47","unstructured":"Walkowiak, T. (July, January 28). Subject Classification of Texts in Polish\u2014From TF-IDF to Transformers. Proceedings of the Sixteenth International Conference on Dependability of Computer Systems DepCoS-RELCOMEX, Wroc\u0142aw, Poland."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Kumar, V., and Subba, B. (2020, January 21\u201323). A Tfidfvectorizer and SVM Based Sentiment Analysis Framework for Text Data Corpus. Proceedings of the 26th National Conference on Communications (NCC), Kharagpur, India.","DOI":"10.1109\/NCC48643.2020.9056085"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Hong, Y., Tantithamthavorn, C., Thongtanunam, P., and Aleti, A. (2022, January 14\u201318). CommentFinder: A Simpler, Faster, More Accurate Code Review Comments Recommendation. Proceedings of the ESEC\/FSE 2022\u201430th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Singapore.","DOI":"10.1145\/3540250.3549119"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Hasani, A.M., Singh, S., Zahergivar, A., Ryan, B., Nethala, D., Bravomontenegro, G., Mendhiratta, N., Ball, M., Farhadi, F., and Malayeri, A. (2023). Evaluating the Performance of Generative Pre-Trained Transformer-4 (GPT-4) in Standardizing Radiology Reports. Eur. Radiol., 1\u20139.","DOI":"10.1007\/s00330-023-10384-x"},{"key":"ref_51","unstructured":"(2023, November 14). OpenAI API. Available online: https:\/\/openai.com\/blog\/openai-api."}],"container-title":["Software"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2674-113X\/3\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:07:02Z","timestamp":1760105222000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2674-113X\/3\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,29]]},"references-count":51,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["software3010004"],"URL":"https:\/\/doi.org\/10.3390\/software3010004","relation":{},"ISSN":["2674-113X"],"issn-type":[{"value":"2674-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,29]]}}}