{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T00:28:40Z","timestamp":1777854520048,"version":"3.51.4"},"reference-count":49,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T00:00:00Z","timestamp":1618185600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100002383","name":"King Saud University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002383","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Information Science"],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:p>Sentiment analysis (SA) aims to extract users\u2019 opinions automatically from their posts and comments. Almost all prior works have used machine learning algorithms. Recently, SA research has shown promising performance in using the deep learning approach. However, deep learning is greedy and requires large datasets to learn, so it takes more time for data annotation. In this research, we proposed a semiautomatic approach using Na\u00efve Bayes (NB) to annotate a new dataset in order to reduce the human effort and time spent on the annotation process. We created a dataset for the purpose of training and testing the classifier by collecting Saudi dialect tweets. The dataset produced from the semiautomatic model was then used to train and test deep learning classifiers to perform Saudi dialect SA. The accuracy achieved by the NB classifier was 83%. The trained semiautomatic model was used to annotate the new dataset before it was fed into the deep learning classifiers. The three deep learning classifiers tested in this research were convolutional neural network (CNN), long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). Support vector machine (SVM) was used as the baseline for comparison. Overall, the performance of the deep learning classifiers exceeded that of SVM. The results showed that CNN reported the highest performance. On one hand, the performance of Bi-LSTM was higher than that of LSTM and SVM, and, on the other hand, the performance of LSTM was higher than that of SVM. The proposed semiautomatic annotation approach is usable and promising to increase speed and save time and effort in the annotation process.<\/jats:p>","DOI":"10.1177\/01655515211006594","type":"journal-article","created":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T00:24:00Z","timestamp":1618273440000},"page":"398-410","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":16,"title":["A semiautomatic annotation approach for sentiment analysis"],"prefix":"10.1177","volume":"49","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7966-5985","authenticated-orcid":false,"given":"Rahma","family":"Alahmary","sequence":"first","affiliation":[{"name":"Information Systems Department, King Saud University, Saudi Arabia; Information Systems Department, Al Imam Mohammad Ibn Saud Islamic University, Saudi Arabia"}]},{"given":"Hmood","family":"Al-Dossari","sequence":"additional","affiliation":[{"name":"Information Systems Department, King Saud University, Saudi Arabia"}]}],"member":"179","published-online":{"date-parts":[[2021,4,12]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Internet World Stats. Top ten languages used in the web 2020 https:\/\/www.internetworldstats.com\/stats7.htm (accessed 10 November 2018)."},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Al Shboul B Al-Ayyouby M Jararwehy Y. Multi-way sentiment classification of Arabic reviews. In: Proceedings of the 2015 6th international conference on information and communication systems (ICICS) 2015 pp. 206\u2013211 https:\/\/ieeexplore.ieee.org\/document\/7103228","DOI":"10.1109\/IACS.2015.7103228"},{"issue":"6","key":"e_1_3_2_4_2","first-page":"283","article-title":"Sentiment analysis and opinion mining: a survey","volume":"2","author":"Vinodhini G","year":"2012","unstructured":"Vinodhini G, Chandrasekaran R. Sentiment analysis and opinion mining: a survey. Int J Adv Res Comput Sci Softw Eng 2012; 2(6): 283\u2013292.","journal-title":"Int J Adv Res Comput Sci Softw Eng"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/1644879.1644881"},{"key":"e_1_3_2_6_2","unstructured":"Saad MK Ashour W. Arabic morphological tools for text mining. In: Proceedings of the 6th international conference on electrical and computer systems (EECS\u201910) 2010 pp. 1\u20136 http:\/\/site.iugaza.edu.ps\/wp-content\/uploads\/mksaad-ArabicMorphologicalToolsforTextMining-EECS10-rev9.pdf"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551516688143"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","unstructured":"Duwairi RM Qarqaz I. Arabic sentiment analysis using supervised classification. In: Proceedings of 2014 international conference on future internet of things and cloud (FICLOUD) 2014 pp. 579\u2013583. IEEE https:\/\/ieeexplore.ieee.org\/document\/6984256","DOI":"10.1109\/FiCloud.2014.100"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Sadat F Kazemi F Farzindar A. Automatic identification of Arabic language varieties and dialects in social media. In: Proceedings of second workshop on natural language processing for social media (SocialNLP) 2014 pp. 22\u201327 http:\/\/rali.iro.umontreal.ca\/rali\/?q=en\/node\/1381","DOI":"10.3115\/v1\/W14-5904"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2005.01.004"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Al-Kabi MN Abdulla N Al-Ayyoub M. An analytical study of Arabic sentiments: Maktoob case study. In: Proceedings of the 8th international conference for internet technology and secured transactions (ICITST) 2013 pp. 89\u201394 https:\/\/ieeexplore.ieee.org\/document\/6750168\/","DOI":"10.1109\/ICITST.2013.6750168"},{"key":"e_1_3_2_12_2","unstructured":"Kim Y. Convolutional neural networks for sentence classification. arXiv1408.5882 2014 https:\/\/arxiv.org\/abs\/1408.5882"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","unstructured":"Vateekul P Koomsubha T. A study of sentiment analysis using deep learning techniques on Thai Twitter data. In: Proceedings of 13th international joint conference on computer science and software engineering (JCSSE) 2016 pp. 1\u20136 https:\/\/ieeexplore.ieee.org\/document\/7748849","DOI":"10.1109\/JCSSE.2016.7748849"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Roshanfekr B Khadivi S Rahmati M. Sentiment analysis using deep learning on Persian texts. In: Proceedings of 25th Iranian conference on electrical engineering (ICEE 2017) 2017 pp. 1503\u20131508 https:\/\/ieeexplore.ieee.org\/document\/7985281","DOI":"10.1109\/IranianCEE.2017.7985281"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"Padmamala R Prema V. Sentiment analysis of online Tamil contents using recursive neural network models approach for Tamil language. In: Proceedings of the IEEE international conference on smart technologies and management for computing communication controls energy and materials (ICSTM) 2017 pp. 28\u201331 https:\/\/ieeexplore.ieee.org\/document\/8089122\/","DOI":"10.1109\/ICSTM.2017.8089122"},{"key":"e_1_3_2_16_2","volume-title":"Social media and the Internet of things: towards data-driven policymaking in the Arab World","author":"Salem F","year":"2017","unstructured":"Salem F. Social media and the Internet of things: towards data-driven policymaking in the Arab World, vol. 7. Dubai, United Arab Emirates: MBR School of Government, 2017."},{"key":"e_1_3_2_17_2","first-page":"91","volume-title":"Proceedings of the LREC workshop for language resources (LRS) and human language technologies (HLT) for semitic languages: status, updates, and prospects","author":"Benajiba Y","unstructured":"Benajiba Y, Diab M. A web application for dialectal Arabic text annotation. In: Proceedings of the LREC workshop for language resources (LRS) and human language technologies (HLT) for semitic languages: status, updates, and prospects, Mediterranean Conference Centre, Valetta, Malta, 17 May 2010, pp. 91\u201398."},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","unstructured":"Alshutayri A Atwell E. Arabic dialects annotation using an online game. In: Proceedings of the 2nd international conference on natural language and speech processing: ICNLSP 2018 2018 pp. 1\u20135 http:\/\/eprints.whiterose.ac.uk\/131819\/1\/Alshutayri%20ICNLSP2018.pdf","DOI":"10.1109\/ICNLSP.2018.8374371"},{"key":"e_1_3_2_19_2","unstructured":"Alosaimy A Atwell E. Web-based annotation tool for inflectional language resources. In: Proceedings of LREC 2018 \u2013 11th international conference on language resources and evaluation 2018 pp. 3933\u20133939 https:\/\/www.aclweb.org\/anthology\/L18-1621.pdf"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"Samih Y Maier W Kallmeyer L. SAWT: sequence annotation web tool. In: Proceedings of the second workshop on computational approaches to code switching 2016 pp. 65\u201370 https:\/\/www.aclweb.org\/anthology\/W16-5808.pdf","DOI":"10.18653\/v1\/W16-5808"},{"issue":"2","key":"e_1_3_2_21_2","first-page":"125","article-title":"Arabic text classification","volume":"4","author":"Duwairi R","year":"2007","unstructured":"Duwairi R. Arabic text classification. Int Arab J Inf Technol 2007; 4(2): 125\u2013131.","journal-title":"Int Arab J Inf Technol"},{"issue":"2","key":"e_1_3_2_22_2","first-page":"13","article-title":"A comparative study of the efficiency of different measures to classify Arabic text","volume":"4","author":"Al-Kabi MN","year":"2007","unstructured":"Al-Kabi MN, Al-Sinjilawi SI. A comparative study of the efficiency of different measures to classify Arabic text. Univ Sharjah J Pure Appl Sci 2007; 4(2): 13\u201326.","journal-title":"Univ Sharjah J Pure Appl Sci"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551514524675"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551514534143"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21598"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Joachims T. Text categorization with support vector machines: learning with many relevant features. In: Proceedings of European conference on machine learning 1998 pp. 137\u2013142 https:\/\/dl.acm.org\/doi\/10.1007\/BFb0026683","DOI":"10.1007\/BFb0026683"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Shoukry A Rafea A. Sentence-level Arabic sentiment analysis. In: Proceedings of the 2012 international conference on collaboration technologies and systems (CTS) 2012 2012 pp. 546\u2013550 https:\/\/ieeexplore.ieee.org\/document\/6261103","DOI":"10.1109\/CTS.2012.6261103"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Abdulla NA Ahmed NA Shehab MA et al. Arabic sentiment analysis: Lexicon-based and corpus-based. In: Proceedings of 2013 IEEE Jordan conference on applied electrical engineering and computing technologies (AEECT) 2013 https:\/\/ieeexplore.ieee.org\/document\/6716448","DOI":"10.1109\/AEECT.2013.6716448"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551520930917."},{"key":"e_1_3_2_30_2","unstructured":"Dahou A Xiong S Zhou J et al. Word embeddings and convolutional neural network for Arabic sentiment classification. In: Proceedings of the COLING 26th international conference on computational linguistics: technical papers 2016 pp. 2418\u20132427 https:\/\/www.aclweb.org\/anthology\/C16-1228\/"},{"key":"e_1_3_2_31_2","unstructured":"Aly M Atiya A. LABR: a large scale Arabic book reviews dataset. In: Proceedings of the 51st annual meeting of the association for computational linguistics 2013 pp. 494\u2013498 https:\/\/www.aclweb.org\/anthology\/P13-2088\/"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Nabil M Aly M Atiya AF. ASTD: Arabic sentiment tweets dataset. In: Proceedings of the 2015 conference on empirical methods in natural language processing (EMNLP) 2015 pp. 2515\u20132519 https:\/\/www.aclweb.org\/anthology\/D15-1299\/","DOI":"10.18653\/v1\/D15-1299"},{"key":"e_1_3_2_33_2","unstructured":"Refaee E Rieser V. An Arabic twitter corpus for subjectivity and sentiment analysis. In: Proceedings of the 9th international conference on language resources and evaluation (LREC\u201914) 2014 pp. 2268\u20132273 https:\/\/www.aclweb.org\/anthology\/L14-1280\/"},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","unstructured":"ElSahar H El-Beltagy SR. Building large Arabic multi-domain resources for sentiment analysis. In: Proceedings of the international conference on intelligent text processing and computational linguistics vol. 9042 2015 pp. 23\u201334 https:\/\/link.springer.com\/chapter\/10.1007\/978-3-319-18117-2_2","DOI":"10.1007\/978-3-319-18117-2_2"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Al-Azani S El-Alfy E-SM. Hybrid deep learning for sentiment polarity determination of Arabic microblogs. In: Proceedings of the international conference on neural information processing. Springer 2017 pp. 491\u2013500 https:\/\/link.springer.com\/chapter\/10.1007\/978-3-319-70096-0_51","DOI":"10.1007\/978-3-319-70096-0_51"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551519849516"},{"key":"e_1_3_2_37_2","first-page":"32","article-title":"Combining Na\u00efve Bayes and modified maximum entropy classifiers for text classification","volume":"9","author":"Kashyap H","year":"2016","unstructured":"Kashyap H, Buksh DB. Combining Na\u00efve Bayes and modified maximum entropy classifiers for text classification. IJ Inf Technol Comput Sci 2016; 9: 32\u201338.","journal-title":"IJ Inf Technol Comput Sci"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","unstructured":"Alahmary RM Al-Dossari HZ Emam AZ. Sentiment analysis of Saudi dialect using deep learning techniques. In: Proceedings of ICEIC 2019 \u2013 international conference on electronics information and communication 2019 pp. 1\u20136 https:\/\/ieeexplore.ieee.org\/document\/8706408","DOI":"10.23919\/ELINFOCOM.2019.8706408"},{"key":"e_1_3_2_39_2","unstructured":"Mikolov T Chen K Corrado G et al. Efficient estimation of word representations in vector space. arXiv:1301.3781v3 2013 https:\/\/arxiv.org\/abs\/1301.3781"},{"key":"e_1_3_2_40_2","first-page":"3111","article-title":"Distributed representations of words and phrases and their compositionality","volume":"26","author":"Mikolov T","year":"2013","unstructured":"Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. Adv Neural Inf Process Syst 2013; 26: 3111\u20133119.","journal-title":"Adv Neural Inf Process Syst"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1171"},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","unstructured":"Ouyang X Zhou P Li CH et al. Sentiment analysis using convolutional neural network. In: Proceedings of 2015 IEEE international conference on computer and information technology; ubiquitous computing and communications; dependable autonomic and secure computing; pervasive intelligence and computing 2015 pp. 2359\u20132364 https:\/\/ieeexplore.ieee.org\/document\/7363395","DOI":"10.1109\/CIT\/IUCC\/DASC\/PICOM.2015.349"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog1402_1"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/78.650093"},{"key":"e_1_3_2_46_2","first-page":"115","article-title":"Learning precise timing with LSTM recurrent networks","volume":"3","author":"Gers FA","year":"2002","unstructured":"Gers FA, Schraudolph NN, Schmidhuber J. Learning precise timing with LSTM recurrent networks. J Mach Learn Res 2002; 3: 115\u2013143.","journal-title":"J Mach Learn Res"},{"key":"e_1_3_2_47_2","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky A","year":"2012","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 2012; 25: 1097\u20131105.","journal-title":"Adv Neural Inf Process Syst"},{"key":"e_1_3_2_48_2","doi-asserted-by":"crossref","unstructured":"Simard PY Steinkraus D Platt JC. Best practices for convolutional neural networks applied to visual document analysis. In: Proceedings of the 7th international conference on document analysis and recognition 2003 p. 958 https:\/\/ieeexplore.ieee.org\/document\/1227801","DOI":"10.1109\/ICDAR.2003.1227801"},{"key":"e_1_3_2_49_2","first-page":"643","article-title":"Semantic parsing for single-relation question answering","volume":"2","author":"Yih W","year":"2014","unstructured":"Yih W, He X, Meek C. Semantic parsing for single-relation question answering. J Obstet Gynecol Neonatal Nurs 2014; 2: 643\u2013648.","journal-title":"J Obstet Gynecol Neonatal Nurs"},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","unstructured":"Shen Y He X Gao J et al. Learning semantic representations using convolutional neural networks for web search. In: Proceedings of the 23rd international conference on World Wide Web 2014 pp. 373\u2013374 https:\/\/dl.acm.org\/doi\/10.1145\/2567948.2577348","DOI":"10.1145\/2567948.2577348"}],"container-title":["Journal of Information Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/01655515211006594","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/01655515211006594","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/01655515211006594","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T23:09:17Z","timestamp":1777504157000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/01655515211006594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,12]]},"references-count":49,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["10.1177\/01655515211006594"],"URL":"https:\/\/doi.org\/10.1177\/01655515211006594","relation":{},"ISSN":["0165-5515","1741-6485"],"issn-type":[{"value":"0165-5515","type":"print"},{"value":"1741-6485","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,12]]}}}