{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T02:34:12Z","timestamp":1775010852622,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,4,23]],"date-time":"2023-04-23T00:00:00Z","timestamp":1682208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>With sentiment prediction technology, businesses can quickly look at user reviews to find ways to improve their products and services. We present the BertBilstm Multiple Emotion Judgment (BBMEJ) model for small-sample emotion prediction tasks to solve the difficulties of short emotion identification datasets and the high dataset annotation costs encountered by small businesses. The BBMEJ model is suitable for many datasets. When an insufficient quantity of relevant datasets prevents the model from achieving the desired training results, the prediction accuracy of the model can be enhanced by fine-tuning it with additional datasets prior to training. Due to the number of parameters in the Bert model, fine-tuning requires a lot of data, which drives up the cost of fine-tuning. We present the Bert Tail Attention Fine-Tuning (BTAFT) method to make fine-tuning work better. Our experimental findings demonstrate that the BTAFT fine-tuning approach performs better in terms of the prediction effect than fine-tuning all parameters. Our model obtains a small sample prediction accuracy of 0.636, which is better than the ideal baseline of 0.064. The Macro-F1 (F1) evaluation metrics significantly exceed other models.<\/jats:p>","DOI":"10.3390\/fi15050158","type":"journal-article","created":{"date-parts":[[2023,4,24]],"date-time":"2023-04-24T01:35:44Z","timestamp":1682300144000},"page":"158","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Chinese Short-Text Sentiment Prediction: A Study of Progressive Prediction Techniques and Attentional Fine-Tuning"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3641-1096","authenticated-orcid":false,"given":"Jinlong","family":"Wang","sequence":"first","affiliation":[{"name":"School of Information and Electrical Engineering, Hebei University of Engineering, Handan 056038, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dong","family":"Cui","sequence":"additional","affiliation":[{"name":"School of Information and Electrical Engineering, Hebei University of Engineering, Handan 056038, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information and Electrical Engineering, Hebei University of Engineering, Handan 056038, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhou, L., Yang, X., Jia, H., Li, W., and Zhang, J. (2023). User Sentiment Analysis of COVID-19 via Adversarial Training Based on the BERT-FGM-BiGRU Model. Systems, 11.","DOI":"10.3390\/systems11030129"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yan, S., Wang, J., and Song, Z. (2022). Microblog Sentiment Analysis Based on Dynamic Character-Level and Word-Level Features and Multi-Head Self-Attention Pooling. Future Internet, 14.","DOI":"10.3390\/fi14080234"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e12306","DOI":"10.1016\/j.heliyon.2022.e12306","article-title":"Weibo users and Academia\u2019s foci on tourism safety: Implications from institutional differences and digital divide","volume":"9","author":"Zeng","year":"2023","journal-title":"Heliyon"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.ijhm.2018.05.007","article-title":"Learning from peers: The effect of sales history disclosure on peer-to-peer short-term rental purchases","volume":"76","author":"Xie","year":"2019","journal-title":"Int. J. Hosp. Manag."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Jamal, N., Xianqiao, C., and Aldabbas, H. (2019). Deep Learning-Based Sentimental Analysis for Large-Scale Imbalanced Twitter Data. Future Internet, 11.","DOI":"10.3390\/fi11090190"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bibi, R., Qamar, U., Ansar, M., and Shaheen, A. (2019, January 29\u201331). Sentiment Analysis for Urdu News Tweets Using Decision Tree. Proceedings of the 2019 IEEE 17th International Conference on Software Engineering Research, Management and Applications (SERA), Honolulu, HI, USA.","DOI":"10.1109\/SERA.2019.8886788"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Petrolini, M., Cagnoni, S., and Mordonini, M. (2022). Automatic Detection of Sensitive Data Using Transformer- Based Classifiers. Future Internet, 14.","DOI":"10.3390\/fi14080228"},{"key":"ref_8","unstructured":"Kan, D. (2023, April 01). Rule-based approach to sentiment analysis at ROMIP 2011.In Komp\u2019iuternaia Lingvistika i Intellektual\u2019nye Tekhnologii: Trudy Mezhdunarodnoi Konferentsii Dialog. 2012; Volume 24. Available online: https:\/\/www.scimagojr.com\/journalsearch.php?q=21100325444&tip=sid&clean=0."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"92757","DOI":"10.1109\/ACCESS.2020.3009292","article-title":"A Novel Emotion Lexicon for Chinese Emotional Expression Analysis on Weibo: Using Grounded Theory and Semi-Automatic Methods","volume":"9","author":"Xu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1142\/S0218488520500154","article-title":"A hybrid multilingual fuzzy-based approach to the sentiment analysis problem using SentiWordNet","volume":"28","author":"Madani","year":"2020","journal-title":"Int. J. Uncertain. Fuzziness-Knowl.-Based Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.engappai.2016.01.012","article-title":"Recognizing emotions in text using ensemble of classifiers","volume":"51","author":"Perikos","year":"2016","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Naz, S., Sharan, A., and Malik, N. (2018, January 3\u20136). Sentiment classification on twitter data using support vector machine. Proceedings of the 2018 IEEE\/WIC\/ACM International Conference on Web Intelligence (WI), Santiago, Chile.","DOI":"10.1109\/WI.2018.00-13"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1016\/j.eswa.2017.07.044","article-title":"Detecting Variation of Emotions in Online Activities","volume":"89","author":"Chatzakou","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1016\/j.future.2021.01.024","article-title":"Scalable multi-channel dilated CNN-BiLSTM model with attention mechanism for Chinese textual sentiment analysis","volume":"118","author":"Gan","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yan, W., Wang, X., and Tan, S. (2022). YOLO-DFAN: Effective High-Altitude Safety Belt Detection Network. Future Internet, 14.","DOI":"10.3390\/fi14120349"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shin, J., Kim, Y., Yoon, S., and Jung, K. (2018, January 15\u201317). Contextual-CNN: A Novel Architecture Capturing Unified Meaning for Sentence Classification. Proceedings of the 2018 IEEE International Conference on Big Data and Smart Computing (BigComp), Shanghai, China.","DOI":"10.1109\/BigComp.2018.00079"},{"key":"ref_17","unstructured":"Gordeev, D. (2016, January 23\u201327). Detecting state of aggression in sentences using CNN. Proceedings of the Speech and Computer: 18th International Conference, SPECOM 2016, Budapest, Hungary. Proceedings 18."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.dss.2018.09.002","article-title":"Deep learning for affective computing: Text-based emotion recognition in decision support","volume":"115","author":"Kratzwald","year":"2018","journal-title":"Decis. Support Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1016\/j.procs.2017.06.037","article-title":"CNN for situations understanding based on sentiment analysis of twitter data","volume":"111","author":"Liao","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_20","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2020.06.002","article-title":"Deep learning based emotion analysis of microblog texts","volume":"64","author":"Xu","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_22","first-page":"36","article-title":"The optimally designed dynamic memory networks for targeted sentiment classification","volume":"390","author":"Zhang","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gao, M., Xiao, Q., Wu, S., and Deng, K. (2019). An Improved Method for Named Entity Recognition and Its Application to CEMR. Future Internet, 11.","DOI":"10.3390\/fi11090185"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"51522","DOI":"10.1109\/ACCESS.2019.2909919","article-title":"Sentiment Analysis of Comment Texts Based on BiLSTM","volume":"7","author":"Xu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Felbo, B., Mislove, A., S\u00f8gaard, A., Rahwan, I., and Lehmann, S. (2017). Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. arXiv.","DOI":"10.18653\/v1\/D17-1169"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/j.chb.2018.12.029","article-title":"Understanding emotions in text using deep learning and big data","volume":"93","author":"Chatterjee","year":"2019","journal-title":"Comput. Hum. Behav."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.eswa.2016.10.065","article-title":"Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN","volume":"72","author":"Chen","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_28","unstructured":"Huang, Y.H., Lee, S.R., Ma, M.Y., Chen, Y.H., Yu, Y.W., and Chen, Y.S. (2019). EmotionX-IDEA: Emotion BERT\u2013an Affectional Model for Conversation. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1016\/j.neunet.2022.03.017","article-title":"A BERT based dual-channel explainable text emotion recognition system","volume":"150","author":"Kumar","year":"2022","journal-title":"Neural Netw."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"104156","DOI":"10.1109\/ACCESS.2022.3210119","article-title":"Chinese Named Entity Recognition of Epidemiological Investigation of Information on COVID-19 Based on BERT","volume":"10","author":"Yang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_31","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141, and Polosukhin, I. (2017). Attention Is All You Need. arXiv."},{"key":"ref_32","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_33","unstructured":"Zaken, E.B., Ravfogel, S., and Goldberg, Y. (2021). Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Emnlp 2014, Doha, Qatar, 25\u201329 October 2014, a Meeting of Sigdat, a Special Interest Group of the Acl, Association for Computational Linguistics.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., and Xu, B. (2016, January 7\u201312). Attention-based bidirectional long short-term memory networks for relation classification. Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers), Berlin, Germany.","DOI":"10.18653\/v1\/P16-2034"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lai, S., Xu, L., Liu, K., and Zhao, J. (2015, January 25\u201330). Recurrent convolutional neural networks for text classification. Proceedings of the AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9513"},{"key":"ref_38","unstructured":"Peng, S., Zeng, R., Liu, H., Chen, G., Wu, R., Yang, A., and Yu, S. (2021, January 23\u201325). Emotion classification of text based on BERT and broad learning system. Proceedings of the Web and Big Data: 5th International Joint Conference, APWeb-WAIM 2021, Guangzhou, China. Proceedings, Part I 5."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Song, G., and Huang, D. (2021). A sentiment-aware contextual model for real-time disaster prediction using Twitter data. Future Internet, 13.","DOI":"10.3390\/fi13070163"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/15\/5\/158\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:22:02Z","timestamp":1760124122000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/15\/5\/158"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,23]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["fi15050158"],"URL":"https:\/\/doi.org\/10.3390\/fi15050158","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,23]]}}}