{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T07:03:28Z","timestamp":1763535808797},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,26]],"date-time":"2021-02-26T00:00:00Z","timestamp":1614297600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,26]],"date-time":"2021-02-26T00:00:00Z","timestamp":1614297600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Soc. Netw. Anal. Min."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Nowadays, social media have become one of the most important methods of communication that provide a real-time and rich source of information, including sentiments. Understanding the population sentiment is a key goal for organisations and governments. In recent years, quite a lot of research has been done on sentiment analysis from social media. However, all the work in the state of the art is focused on a specific pre-defined subset of tweets, e.g. sentiment analysis via keywords search from tweets for relevant brands, products, services, events and so forth. Monitoring the general sentiment at national level through the whole social media stream is not done due to the challenges of filtering sentiment-irrelevant information, diversity of vocabulary usage in general tweets across topics causing low accuracy and the need for bilingual or multilingual models. This paper proposes a system for general population sentiment monitoring from a social media stream (Twitter), through comprehensive multi-level filters, and our proposed improved latent Dirichlet allocation (LDA) (Wang et al. in ACM Trans Internet Technol 18(1):1\u201323, 2017; Wang and Al-Rubaie in Appl Soft Comput 33:250\u2013262, 2015; https:\/\/patents.google.com\/patent\/US20170293597A1\/en) method for sentiment classification. Experiments show that our proposed improved LDA for sentiment analysis yields the best results, and also validate our proposed system for national sentiment monitoring in Abu Dhabi using twitter.<\/jats:p>","DOI":"10.1007\/s13278-021-00728-0","type":"journal-article","created":{"date-parts":[[2021,2,26]],"date-time":"2021-02-26T07:04:00Z","timestamp":1614323040000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["National happiness index monitoring using Twitter for bilanguages"],"prefix":"10.1007","volume":"11","author":[{"given":"Di","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmad","family":"Al-Rubaie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"Hirsch","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gregory Cameron","family":"Pole","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,2,26]]},"reference":[{"key":"728_CR1","doi-asserted-by":"crossref","unstructured":"Abdelwahab O, Bahgat M, Lowrance CJ, Elmaghraby A (2015) Effect of training set size on SVM and Na\u00efve Bayes for Twitter sentiment analysis. In: 2015 IEEE international symposium on signal processing and information technology (ISSPIT), pp 46\u201351, 2015.","DOI":"10.1109\/ISSPIT.2015.7394379"},{"issue":"2003","key":"728_CR2","first-page":"993","volume":"3","author":"DM Blei","year":"2003","unstructured":"Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3(2003):993\u20131022","journal-title":"J Mach Learn Res"},{"key":"728_CR3","doi-asserted-by":"crossref","unstructured":"Bravo-Marquez F, Frank E, Pfahringer B (2016) From opinion lexicons to sentiment classification of tweets and vice versa: a transfer learning approach. In: 2016 IEEE\/WIC\/ACM international conference on web intelligence, pp 145\u2013152","DOI":"10.1109\/WI.2016.0030"},{"key":"728_CR4","doi-asserted-by":"publisher","first-page":"6266","DOI":"10.1016\/j.eswa.2013.05.057","volume":"40","author":"M Chiassi","year":"2013","unstructured":"Chiassi M, Skinner J, Zimbra D (2013) Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network. Expert Syst Appl 40:6266\u20136282","journal-title":"Expert Syst Appl"},{"key":"728_CR5","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1016\/j.eswa.2016.11.022","volume":"71","author":"M Daniel","year":"2017","unstructured":"Daniel M, Neves RF, Horta N (2017) Company event popularity for financial markets using Twitter and sentiment analysis. Expert Syst Appl 71:111\u2013124","journal-title":"Expert Syst Appl"},{"key":"728_CR6","doi-asserted-by":"crossref","unstructured":"Deshwal A, Sharma SK (2016) Twitter sentiment analysis using various classification algorithms. In: 2016 5th international conference on reliability, infocom technologies and optimization (ICRITO) (trends and future directions), Sept. 7\u20139, 2016, AIIT, Noida, India, pp 251\u2013257","DOI":"10.1109\/ICRITO.2016.7784960"},{"key":"728_CR7","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1, 2019"},{"key":"728_CR8","doi-asserted-by":"crossref","unstructured":"Furini M, Montangero M (2016) TSentiment: on Gamifying Twitter sentiment analysis. In: ISCC 2016 Workshop: DENVECT, 2016.","DOI":"10.1109\/ISCC.2016.7543720"},{"key":"728_CR9","unstructured":"Gupta VS, Kohli S (2016) Twitter sentiment analysis in healthcare using Hadoop and R. In: 2016 international conference on computing for sustainable global development (INDIACom), pp 3766\u20133772, 2016."},{"key":"728_CR10","doi-asserted-by":"crossref","unstructured":"Hassan A, Abbasi A, Zeng D (2013) Twitter sentiment analysis: a bootstrap ensemble framework, pp 357\u2013364. SocialCom\/PASSAT\/BigData\/EconCom\/BioMedCom 2013","DOI":"10.1109\/SocialCom.2013.56"},{"key":"728_CR11","doi-asserted-by":"crossref","unstructured":"Kanakaraj M, Guddeti RMR (2015) Performance analysis of ensemble methods on Twitter sentiment analysis using NLP technique. In: Proceedings of the 2015 IEEE 9th international conference on semantic computing (IEEE ICSC 2015), pp 169\u2013170, 2015","DOI":"10.1109\/ICOSC.2015.7050801"},{"key":"728_CR12","doi-asserted-by":"publisher","first-page":"4065","DOI":"10.1016\/j.eswa.2013.01.001","volume":"40","author":"E Kontopoulos","year":"2013","unstructured":"Kontopoulos E, Berberidis C, Dergiades T, Bassiliades N (2013) Ontology-based sentiment analysis of twitter posets. Expert Syst Appl 40:4065\u20134074","journal-title":"Expert Syst Appl"},{"issue":"8","key":"728_CR13","first-page":"755","volume":"23","author":"A Krouska","year":"2017","unstructured":"Krouska A, Troussas C, Virvou M (2017) Comparative evaluation of algorithms for sentiment analysis over social networking services. J Univ Comput Sci 23(8):755\u2013768","journal-title":"J Univ Comput Sci"},{"key":"728_CR14","doi-asserted-by":"crossref","unstructured":"Krouska A, Troussas C, Virvou M (2016) The effect of preprocessing techniques on Twitter sentiment analysis. In: 2016 7th international conference on information, intelligence, systems & applications (IISA), 2016","DOI":"10.1109\/IISA.2016.7785373"},{"key":"728_CR15","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/s10579-015-9328-1","volume":"50","author":"P Nakov","year":"2016","unstructured":"Nakov P, Rosenthal S, Kiritchenko S, Mohammad SM, Kozareva Z, Ritter A, Stoyanov V, Zhu X (2016) Developing a successful SemEval task in sentiment analysis of Twitter and other social media texts. Lang Resour Eval 50:35\u201365","journal-title":"Lang Resour Eval"},{"key":"728_CR16","doi-asserted-by":"crossref","unstructured":"Nakov P, Ritter A, Rosenthal S, Sebastiani F, Stoyanov V (2016) SemEval-2016Task4: sentiment analysis in Twitter. In: Proceedings of SemEval-2016, pp 1\u201318, 2016","DOI":"10.18653\/v1\/S16-1001"},{"key":"728_CR17","doi-asserted-by":"publisher","first-page":"8901","DOI":"10.1007\/s11042-016-3525-9","volume":"76","author":"HL Nguyen","year":"2017","unstructured":"Nguyen HL, Jung JE (2017) Statistical approach for figurative sentiment analysis on social networking services: a case study on Twitter. Multimed Tools Appl 76:8901\u20138914","journal-title":"Multimed Tools Appl"},{"key":"728_CR18","doi-asserted-by":"publisher","first-page":"764","DOI":"10.1016\/j.ipm.2017.02.004","volume":"53","author":"AC Pandey","year":"2017","unstructured":"Pandey AC, Rajpoot DS, Saraswat M (2017) Twitter sentiment analysis using hybrid cuckoo search method. Inf Process Manag 53:764\u2013779","journal-title":"Inf Process Manag"},{"key":"728_CR19","doi-asserted-by":"crossref","unstructured":"Peng Y, Moh M, Moh T-S (2016) Efficient adverse drug event extraction using Twitter sentiment analysis. In: 2016 IEEE\/ACM international conference on advances in social networks analysis and mining (ASONAM), pp 1011\u20131018","DOI":"10.1109\/ASONAM.2016.7752365"},{"key":"728_CR20","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.ijhm.2016.02.001","volume":"55","author":"K Philander","year":"2016","unstructured":"Philander K, Zhong YY (2016) Twitter sentiment analysis: capturing sentiment form integrated resort tweets. Int J Hosp Manag 55:16\u201324","journal-title":"Int J Hosp Manag"},{"key":"728_CR21","doi-asserted-by":"crossref","unstructured":"Porshnev A, Redkin I, Shevchenko A (2013) Machine learning in prediction of stock market indicators based on historical data and data from Twitter sentiment analysis. In: 2013 IEEE 13th international conference on data mining workshops, pp 440\u2013444","DOI":"10.1109\/ICDMW.2013.111"},{"key":"728_CR22","doi-asserted-by":"crossref","unstructured":"Qaisi LM, Aljarah I (2016) A Twitter sentiment analysis for cloud providers: a case study of Azure vs. A WS. In: 2016 7th international conference on computer science and information technology (CSTT), 2016.","DOI":"10.1109\/CSIT.2016.7549473"},{"key":"728_CR23","doi-asserted-by":"crossref","unstructured":"Ramteke J, Godhia D, Shah S, Shaikh A (2016) Election result prediction using Twitter sentiment analysis. In: International conference on inventive computation technologies (ICICT), 2016.","DOI":"10.1109\/INVENTIVE.2016.7823280"},{"key":"728_CR24","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1016\/j.ins.2016.06.040","volume":"369","author":"Y Ren","year":"2016","unstructured":"Ren Y, Wang R, Ji D (2016) A topic-enhanced word embedding for Twitter sentiment classification. Inf Sci 369:188\u2013198","journal-title":"Inf Sci"},{"key":"728_CR25","doi-asserted-by":"crossref","unstructured":"Sahu S, Rout SK, Mohanty D (2015) Twitter sentiment analysis a more enhanced way of classification and scoring. In: 2015 IEEE international symposium on nanoelectronic and information systems, pp 67\u201372","DOI":"10.1109\/iNIS.2015.40"},{"key":"728_CR26","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.ipm.2015.01.005","volume":"52","author":"H Saifa","year":"2016","unstructured":"Saifa H, He Y, Fernandez M, Alani H (2016) Contextual semantics for sentiment analysis of Twitter. Inf Process Manage 52:5\u201319","journal-title":"Inf Process Manage"},{"key":"728_CR27","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1016\/j.dss.2016.05.010","volume":"88","author":"RP Schumaker","year":"2016","unstructured":"Schumaker RP, Jarmoszko AT, Labedz CS Jr (2016) Predicting wins and spread in the Premier League using a sentiment analysis of twitter. Decis Support Syst 88:76\u201384","journal-title":"Decis Support Syst"},{"key":"728_CR28","doi-asserted-by":"crossref","unstructured":"Shyamasundar LB, Jhansi RP (2016) Twitter sentiment analysis with different feature extractors and dimensionality reduction using supervised learning algorithms. In: India conference (INDICON), 2016 IEEE Annual","DOI":"10.1109\/INDICON.2016.7839075"},{"key":"728_CR29","doi-asserted-by":"crossref","unstructured":"Siddiqua UA, Ahsan T, Chy AN (2016) Combining a rule-based classifier with weakly supervised learning for Twitter sentiment analysis. In: 2016 international conference on innovations in science, engineering and technology (ICISET)","DOI":"10.1109\/ICISET.2016.7856499"},{"key":"728_CR30","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1016\/j.procs.2016.06.095","volume":"89","author":"T Singh","year":"2016","unstructured":"Singh T, Kumari M (2016) Role of text pre-processing in Twitter sentiment analysis. Procedia Comput Sci 89:549\u2013554","journal-title":"Procedia Comput Sci"},{"key":"728_CR31","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.ins.2014.04.034","volume":"285","author":"J Smailovic","year":"2014","unstructured":"Smailovic J, Grcar M, Lavrac N, Znidarsic M (2014) Stream-based active learning for sentiment analysis in the financial domain. Inf Sci 285:181\u2013203","journal-title":"Inf Sci"},{"key":"728_CR32","doi-asserted-by":"crossref","unstructured":"Troussas C, Krouska A, Virvou M (2016) Evaluation of ensemble-based sentiment classifiers for Twitter Data. In: 2016 7th international conference on information, intelligence, systems & applications (IISA), 2016","DOI":"10.1109\/IISA.2016.7785380"},{"key":"728_CR33","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1016\/j.asoc.2015.04.044","volume":"33","author":"D Wang","year":"2015","unstructured":"Wang D, Al-Rubaie A (2015) Incremental learning with partial-supervision based on hierarchical dirichlet process and the application for document classification. Appl Soft Comput 33:250\u2013262","journal-title":"Appl Soft Comput"},{"issue":"1","key":"728_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3122982","volume":"18","author":"D Wang","year":"2017","unstructured":"Wang D, Al-Rubaie A, Clarke SS, Davies J (2017) Real-time traffic event detection from social media. ACM Trans Internet Technol 18(1):1\u201323","journal-title":"ACM Trans Internet Technol"},{"key":"728_CR35","doi-asserted-by":"crossref","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Le Scao T, Gugger S, Drame M, Lhoest Q, Rush AM (2020) Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 EMNLP (systems demonstrations), pp 38\u201345, Nov. 16\u201320, 2020.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"728_CR36","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1016\/j.ins.2016.01.094","volume":"373","author":"F Wu","year":"2016","unstructured":"Wu F, Song Y, Huang Y (2016) Microblog sentiment classification with heterogeneous sentiment knowledge. Inf Sci 373:146\u2013164","journal-title":"Inf Sci"},{"key":"728_CR37","doi-asserted-by":"publisher","first-page":"392","DOI":"10.1016\/j.chb.2015.01.075","volume":"49","author":"Y Yu","year":"2015","unstructured":"Yu Y, Wang X (2015) World Cup 2014 in the Twitter World: a big data analysis of sentiments in U.S. sports fans tweets. Comput Hum Behav 49:392\u2013400","journal-title":"Comput Hum Behav"},{"key":"728_CR38","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1007\/s10115-018-1236-4","volume":"60","author":"L Yue","year":"2018","unstructured":"Yue L, Chen W, Li X, Zuo W, Yin M (2018) A survey of sentiment analysis in social media. Knowl Inf Syst 60:617\u2013663","journal-title":"Knowl Inf Syst"},{"key":"728_CR39","first-page":"2870","volume":"5","author":"J Zhao","year":"2017","unstructured":"Zhao J, Gui X (2017) Comparison research on text pre-processing methods on Twitter sentiment analysis. IEEE Transl Content Min 5:2870\u20132879","journal-title":"IEEE Transl Content Min"},{"key":"728_CR40","unstructured":"Zhao J (2016) Combing semantic and prior polarity features for boosting Twitter sentiment analysis using ensemble learning. In: 2016 IEEE first international conference on data science in cyberspace, pp 709\u2013714, 2016."},{"key":"728_CR41","unstructured":"Zhao J, Cao X (2015) Combining semantic and prior polarity for boosting twitter sentiment analysis. In: The 2015 IEEE international conference on smart City\/SocialCom\/SustainCom together with DataCom 2015 and SC2 2015, pp 832\u2013837, 2015"},{"key":"728_CR42","doi-asserted-by":"crossref","unstructured":"Zimbra D, Ghiassi M, Lee S (2016) Brand-related twitter sentiment analysis using feature engineering and the dynamic architecture for artificial neural networks. In: 2016 49th Hawaii international conference on system sciences, pp 1930\u20131938, 2016.","DOI":"10.1109\/HICSS.2016.244"}],"container-title":["Social Network Analysis and Mining"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13278-021-00728-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13278-021-00728-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13278-021-00728-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,20]],"date-time":"2021-11-20T14:11:57Z","timestamp":1637417517000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13278-021-00728-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,26]]},"references-count":42,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["728"],"URL":"https:\/\/doi.org\/10.1007\/s13278-021-00728-0","relation":{},"ISSN":["1869-5450","1869-5469"],"issn-type":[{"value":"1869-5450","type":"print"},{"value":"1869-5469","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,26]]},"assertion":[{"value":"21 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 December 2020","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 January 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 February 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"24"}}