{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:24:58Z","timestamp":1772252698171,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T00:00:00Z","timestamp":1753228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006378","name":"The Directorate of Research and Development, Universitas Indonesia","doi-asserted-by":"publisher","award":["NKB-476\/UN2.RST\/HKP.05.00\/2023"],"award-info":[{"award-number":["NKB-476\/UN2.RST\/HKP.05.00\/2023"]}],"id":[{"id":"10.13039\/501100006378","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Customer sentiment analysis plays a pivotal role in the digital economy by offering comprehensive insights that inform strategic business decisions, optimize digital marketing initiatives, and improve overall customer satisfaction. We propose a large language model-based topic-level sentiment analysis framework. We employ a BERT-based model to generate contextualized vector representations of the documents, and then clustering algorithms are automatically applied to group documents into topics. Once the topics are formed, a GPT model is used to perform sentiment classification on the content related to each topic. The simulations show the effectiveness of this approach, where selecting appropriate clustering techniques yields more semantically coherent topics. Furthermore, topic-level sentiment polarization shows that 31.7% of all negative sentiment concentrates on the shopping experience, despite an overall positive sentiment trend.<\/jats:p>","DOI":"10.3390\/bdcc9080194","type":"journal-article","created":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T14:22:44Z","timestamp":1753280564000},"page":"194","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Large Language Model-Based Topic-Level Sentiment Analysis for E-Grocery Consumer Reviews"],"prefix":"10.3390","volume":"9","author":[{"given":"Julizar Isya Pandu","family":"Wangsa","sequence":"first","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"given":"Yudhistira Jinawi","family":"Agung","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"given":"Safira Raissa","family":"Rahmi","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2722-5763","authenticated-orcid":false,"given":"Hendri","family":"Murfi","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1384-0781","authenticated-orcid":false,"given":"Nora","family":"Hariadi","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"given":"Siti","family":"Nurrohmah","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"given":"Yudi","family":"Satria","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Universitas Indonesia, Depok 16424, Indonesia"}]},{"given":"Choiru","family":"Za\u2019in","sequence":"additional","affiliation":[{"name":"Computer Science and Information Technology, La Trobe University, Melbourne 32935, Australia"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Grande-Ram\u00edrez, J.R., Rold\u00e1n-Reyes, E., Aguilar-Lasserre, A.A., and Ju\u00e1rez-Mart\u00ednez, U. (2022). Integration of Sentiment Analysis of Social Media in the Strategic Planning Process to Generate the Balanced Scorecard. Appl. Sci., 12.","DOI":"10.3390\/app122312307"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102868","DOI":"10.1016\/j.mex.2024.102868","article-title":"Shopping marketplace analysis based on customer insights using social media analytics","volume":"13","author":"Nasrabadi","year":"2024","journal-title":"MethodsX"},{"key":"ref_3","unstructured":"(2025, March 24). Indonesia.go.id., Konsumsi Masyarakat Masih Tinggi, Produk Lokal Paling Laku di Harbolnas 2024. Available online: https:\/\/indonesia.go.id\/kategori\/editorial\/8889\/konsumsi-masyarakat-masih-tinggi-produk-lokal-paling-laku-di-harbolnas."},{"key":"ref_4","unstructured":"Titipku Research Team (2022). Indonesia E-Grocery Report 2022, Titipku Research Team."},{"key":"ref_5","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_7","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_8","unstructured":"Alaparthi, S., and Mishra, M. (2020). Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey. arXiv."},{"key":"ref_9","unstructured":"Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2025, March 31). Improving Language Understanding by Generative Pretraining. Available online: https:\/\/gluebenchmark.com\/leaderboard."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s40547-024-00143-4","article-title":"Sentiment Analysis in the Age of Generative AI","volume":"11","author":"Krugmann","year":"2024","journal-title":"Cust. Needs Solut."},{"key":"ref_11","first-page":"200308","article-title":"ChatGPT and fine-tuned BERT: A comparative study for developing intelligent design support systems","volume":"21","author":"Qiu","year":"2024","journal-title":"Intell. Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"7764","DOI":"10.1016\/j.eswa.2014.05.045","article-title":"A novel deterministic approach for aspect-based opinion mining in tourism products reviews","volume":"41","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1177\/0165551510388123","article-title":"Aspect-based sentiment analysis of movie reviews on discussion boards","volume":"36","author":"Thet","year":"2010","journal-title":"J. Inf. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/IJISSS.311861","article-title":"Topic Modeling and Sentiment Analysis of Yelp Restaurant Reviews","volume":"14","author":"Zhang","year":"2022","journal-title":"Int. J. Inf. Syst. Serv. Sect."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"107057","DOI":"10.1016\/j.asoc.2020.107057","article-title":"Topic detection and sentiment analysis in Twitter content related to COVID-19 from Brazil and the USA","volume":"101","author":"Garcia","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"908","DOI":"10.1016\/j.procs.2023.01.071","article-title":"Topic Modelling and Opinion Analysis On Climate Change Twitter Data Using LDA and BERT Model","volume":"218","author":"Uthirapathy","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2733","DOI":"10.1109\/JBHI.2020.3001216","article-title":"Deep Sentiment Classification and Topic Discovery on Novel Coronavirus or COVID-19 Online Discussions: NLP Using LSTM Recurrent Neural Network Approach","volume":"24","author":"Jelodar","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kwon, H.-J., Ban, H.-J., Jun, J.-K., and Kim, H.-S. (2021). Topic Modeling and Sentiment Analysis of Online Review for Airlines. Information, 12.","DOI":"10.3390\/info12020078"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1177\/14673584221085470","article-title":"Topic and sentiment analysis of crisis communications about the COVID-19 pandemic in Twitter\u2019s tourism hashtags","volume":"23","author":"Iturralde","year":"2023","journal-title":"Tour. Hosp. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1186\/s43067-023-00070-9","article-title":"Sentiment analysis of COVID-19 tweets from selected hashtags in Nigeria using VADER and Text Blob analyser","volume":"10","author":"Abiola","year":"2023","journal-title":"J. Electr. Syst. Inf. Technol."},{"key":"ref_21","first-page":"993","article-title":"Latent Dirichlet Allocation","volume":"3","author":"Blei","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"18","DOI":"10.4018\/JOEUC.294901","article-title":"Topic Modelling and Sentiment Analysis of Global Warming Tweets","volume":"34","author":"Qiao","year":"2021","journal-title":"J. Organ. End User Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107440","DOI":"10.1016\/j.asoc.2021.107440","article-title":"Topic-level sentiment analysis of social media data using deep learning","volume":"108","author":"Pathak","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1915","DOI":"10.1109\/TKDE.2020.2999489","article-title":"Multi Task Mutual Learning for Joint Sentiment Classification and Topic Detection","volume":"34","author":"Gui","year":"2020","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_25","unstructured":"Nur, K., Najahaty, I., Hidayati, L., Murfi, H., and Nurrohmah, S. (2015, January 10\u201311). Combination of singular value decomposition and K-means clustering methods for topic detection on Twitter. Proceedings of the 2015 International Conference on Advanced Computer Science and Information Systems (ICACSIS), Depok, Indonesia."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tounsi, A., Elkefi, S., and Bhar, S.L. (May, January 29). Exploring the Reactions of Early Users of ChatGPT to the Tool using Twitter Data: Sentiment and Topic Analyses. Proceedings of the 2023 IEEE International Conference on Advanced Systems and Emergent Technologies (IC_ASET), Hammamet, Tunisia.","DOI":"10.1109\/IC_ASET58101.2023.10150870"},{"key":"ref_27","unstructured":"Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv."},{"key":"ref_28","first-page":"527","article-title":"A scalable eigenspace-based fuzzy c-means for topic detection","volume":"55","author":"Murfi","year":"2021","journal-title":"Data Technol. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"030140","DOI":"10.1063\/1.4991244","article-title":"Eigenspace-based fuzzy c-means for sensing trending topics in Twitter","volume":"1862","author":"Muliawati","year":"2017","journal-title":"AIP Conf. Proc."},{"key":"ref_30","unstructured":"Qiu, M. (2018). The Accuracy of Fuzzy C-Means in Lower-Dimensional Space for Topic Detection. Smart Computing and Communication, Springer International Publishing."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kheiri, K., and Karimi, H. (2023). SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning. arXiv.","DOI":"10.1109\/BigData62323.2024.10825350"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Koto, F., Lau, J.H., and Baldwin, T. (2021). IndoBERTweet: A Pretrained Language Model for Indonesian Twitter with Effective Domain-Specific Vocabulary Initialization. arXiv.","DOI":"10.18653\/v1\/2021.emnlp-main.833"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1002\/aris.1440380105","article-title":"Latent semantic analysis","volume":"38","author":"Dumais","year":"2004","journal-title":"Annu. Rev. Inf. Sci. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the Dimensionality of Data with Neural Networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"McInnes, L., Healy, J., and Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv.","DOI":"10.21105\/joss.00861"},{"key":"ref_36","unstructured":"McQueen, J.B. (July, January 21). Some methods of classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, SC, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/0098-3004(84)90020-7","article-title":"FCM: The fuzzy c-means clustering algorithm","volume":"10","author":"Bezdek","year":"1984","journal-title":"Comput. Geosci."},{"key":"ref_38","unstructured":"Ester, M., Kriegel, H.-P., Sander, J., and Xu, X. (1996). A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise, AAAI Press."},{"key":"ref_39","first-page":"160","article-title":"Density-Based Clustering Based on Hierarchical Density Estimates","volume":"Volume 7819","author":"Campello","year":"2019","journal-title":"Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining"},{"key":"ref_40","unstructured":"Joachims, T. (1997, January 8\u201412). A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization. Proceedings of the Fourteenth International Conference on Machine Learning (ICML\u201997), Nashville, TN, USA."},{"key":"ref_41","unstructured":"Zhang, X., Zhao, J., and Lecun, Y. (2015). Text Understanding from Scratch. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation; MIT Press."},{"key":"ref_42","first-page":"1","article-title":"Fuzzy C-means in high dimensional spaces","volume":"1","author":"Winkler","year":"2011","journal-title":"Int. J. Fuzzy Syst. Appl."},{"key":"ref_43","first-page":"1","article-title":"Fuzziness parameter selection in fuzzy c-means: The perspective of cluster validation","volume":"57","author":"Zhou","year":"2014","journal-title":"Sci. China Inf. Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s40537-022-00564-9","article-title":"The performance of BERT as data representation of text clustering","volume":"9","author":"Subakti","year":"2022","journal-title":"J. Big Data"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"5645","DOI":"10.1016\/j.eswa.2015.02.055","article-title":"An analysis of the coherence of descriptors in topic modeling","volume":"42","author":"Greene","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Dieng, A.B., Ruiz, F.J.R., and Blei, D.M. (2019). Topic Modeling in Embedding Spaces. arXiv.","DOI":"10.1162\/tacl_a_00325"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.aej.2020.06.051","article-title":"Design of sustainable urban electronic grocery distribution network","volume":"60","author":"Liu","year":"2021","journal-title":"Alex. Eng. J."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"90367","DOI":"10.1109\/ACCESS.2023.3307308","article-title":"Sentiment Analysis in E-Commerce Platforms: A Review of Current Techniques and Future Directions","volume":"11","author":"Huang","year":"2023","journal-title":"IEEE Access"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/8\/194\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:14:48Z","timestamp":1760033688000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/8\/194"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,23]]},"references-count":48,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["bdcc9080194"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9080194","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3637575\/v2","asserted-by":"object"},{"id-type":"doi","id":"10.21203\/rs.3.rs-3637575\/v1","asserted-by":"object"},{"id-type":"doi","id":"10.21203\/rs.3.rs-3637575\/v3","asserted-by":"object"}]},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,23]]}}}