{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T16:40:47Z","timestamp":1773247247729,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,6,22]],"date-time":"2022-06-22T00:00:00Z","timestamp":1655856000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>In recent years, deep learning-based sentiment analysis has received attention mainly because of the rise of social media and e-commerce. In this paper, we showcase the fact that the polarity detection and subjectivity detection subtasks of sentiment analysis are inter-related. To this end, we propose a knowledge-sharing-based multitask learning framework. To ensure high-quality knowledge sharing between the tasks, we use the Neural Tensor Network, which consists of a bilinear tensor layer that links the two entity vectors. We show that BERT-based embedding with our MTL framework outperforms the baselines and achieves a new state-of-the-art status in multitask learning. Our framework shows that the information across datasets for related tasks can be helpful for understanding task-specific features.<\/jats:p>","DOI":"10.3390\/fi14070191","type":"journal-article","created":{"date-parts":[[2022,6,22]],"date-time":"2022-06-22T04:12:01Z","timestamp":1655871121000},"page":"191","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Polarity and Subjectivity Detection with Multitask Learning and BERT Embedding"],"prefix":"10.3390","volume":"14","author":[{"given":"Ranjan","family":"Satapathy","sequence":"first","affiliation":[{"name":"Graphene AI, 28 Genting Ln, Singapore 349585, Singapore"}]},{"given":"Shweta Rajesh","family":"Pardeshi","sequence":"additional","affiliation":[{"name":"Granular AI, Mumbai 410206, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3030-1280","authenticated-orcid":false,"given":"Erik","family":"Cambria","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798, Singapore"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Pang, B., and Lee, L. (2005). Seeing Stars: Exploiting Class Relationships For Sentiment Categorization With Respect To Rating Scales. arXiv.","DOI":"10.3115\/1219840.1219855"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1007\/s12559-020-09723-7","article-title":"A review of shorthand systems: From brachygraphy to microtext and beyond","volume":"12","author":"Satapathy","year":"2020","journal-title":"Cogn. Comput."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"ref_4","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Pang, B., Lee, L., and Vaithyanathan, S. (2002). Thumbs Up? Sentiment Classification Using Machine Learning Techniques. arXiv.","DOI":"10.3115\/1118693.1118704"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pang, B., and Lee, L. (2004). A Sentimental Education: Sentiment Analysis Using Subjectivity. arXiv.","DOI":"10.3115\/1218955.1218990"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Balikas, G., Moura, S., and Amini, M.R. (2017, January 7\u201311). Multitask learning for fine-grained twitter sentiment analysis. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan.","DOI":"10.1145\/3077136.3080702"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/MIS.2019.2904691","article-title":"Sentiment and sarcasm classification with multitask learning","volume":"34","author":"Majumder","year":"2019","journal-title":"IEEE Intell. Syst."},{"key":"ref_9","unstructured":"Liu, P., Qiu, X., and Huang, X.J. (August, January 30). Adversarial Multi-task Learning for Text Classification. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BA, Canada."},{"key":"ref_10","unstructured":"Kochkina, E., Liakata, M., and Zubiaga, A. (2018, January 20\u201326). All-in-one: Multi-task Learning for Rumour Verification. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mishra, A., Tamilselvam, S., Dasgupta, R., Nagar, S., and Dey, K. (2018, January 2\u20137). Cognition-Cognizant Sentiment Analysis With Multitask Subjectivity Summarization Based on Annotators\u2019 Gaze Behavior. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12068"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1016\/j.jfranklin.2017.06.007","article-title":"Bayesian network based extreme learning machine for subjectivity detection","volume":"355","author":"Chaturvedi","year":"2018","journal-title":"J. Frankl. Inst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C.D. (2014, January 25\u201329). GloVe: Global Vectors for Word Representation. Proceedings of the Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_14","unstructured":"Rashkin, H., Smith, E.M., Li, M., and Boureau, Y.L. (August, January 28). Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_15","unstructured":"Alonso, H.M., and Plank, B. (2017, January 3). When is multitask learning effective? Semantic sequence prediction under varying data conditions. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, Valencia, Spain."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Stein, C. (1956, January 1). Inadmissibility of the Usual Estimator for the Mean of a Multivariate Normal Distribution. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, Berkeley, CA, USA.","DOI":"10.1525\/9780520313880-018"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1007\/s11222-008-9111-x","article-title":"Joint covariate selection and joint subspace selection for multiple classification problems","volume":"20","author":"Obozinski","year":"2010","journal-title":"Stat. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Collobert, R., and Weston, J. (2008, January 5\u20139). A unified architecture for natural language processing: Deep neural networks with multitask learning. Proceedings of the 25th international conference on Machine learning, Helsinki, Finland.","DOI":"10.1145\/1390156.1390177"},{"key":"ref_19","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, X., Gao, J., He, X., Deng, L., Duh, K., and Wang, Y.Y. (2015, January 4). Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, CO, USA.","DOI":"10.3115\/v1\/N15-1092"},{"key":"ref_21","unstructured":"Bansal, T., Belanger, D., and McCallum, A. (2016, January 15). Ask the gru: Multi-task learning for deep text recommendations. Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA."},{"key":"ref_22","unstructured":"Yim, J., Jung, H., Yoo, B., Choi, C., Park, D., and Kim, J. (2015, January 7\u201312). Rotating your face using multi-task deep neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"854","DOI":"10.1109\/TPAMI.2007.1055","article-title":"Sharing visual features for multiclass and multiview object detection","volume":"29","author":"Torralba","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Misra, I., Shrivastava, A., Gupta, A., and Hebert, M. (2016, January 27\u201330). Cross-stitch networks for multi-task learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA.","DOI":"10.1109\/CVPR.2016.433"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wiebe, J., Bruce, R., and O\u2019Hara, T.P. (1999, January 20\u201326). Development and use of a gold-standard data set for subjectivity classifications. Proceedings of the 37th annual meeting of the Association for Computational Linguistics, College Park, MD, USA.","DOI":"10.3115\/1034678.1034721"},{"key":"ref_26","unstructured":"Crawshaw, M. (2020). Multi-task learning with deep neural networks: A survey. arXiv."},{"key":"ref_27","first-page":"926","article-title":"Reasoning with neural tensor networks for knowledge base completion","volume":"26","author":"Socher","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2019). HuggingFace\u2019s Transformers: State-of-the-art Natural Language Processing. arXiv.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"ref_29","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 5998\u20136008."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Cambria, E., Li, Y., Xing, F., Poria, S., and Kwok, K. (2020, January 19\u201323). SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis. Proceedings of the CIKM, Virtual Event, Ireland.","DOI":"10.1145\/3340531.3412003"},{"key":"ref_31","unstructured":"Zhao, H., Lu, Z., and Poupart, P. (2015, January 25\u201331). Self-adaptive hierarchical sentence model. Proceedings of the 24th International Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Amplayo, R.K., Lee, K., Yeo, J., and Hwang, S.W. (2018, January 13\u201319). Translations as additional contexts for sentence classification. Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/550"},{"key":"ref_33","unstructured":"Liu, P., Qiu, X., and Huang, X. (2016, January 9\u201315). Recurrent neural network for text classification with multi-task learning. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_34","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/7\/191\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:37:12Z","timestamp":1760139432000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/7\/191"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,22]]},"references-count":34,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["fi14070191"],"URL":"https:\/\/doi.org\/10.3390\/fi14070191","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,22]]}}}