{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T06:53:40Z","timestamp":1780642420702,"version":"3.54.1"},"reference-count":75,"publisher":"Emerald","issue":"1","license":[{"start":{"date-parts":[[2020,2,3]],"date-time":"2020-02-03T00:00:00Z","timestamp":1580688000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["EL"],"published-print":{"date-parts":[[2020,3,19]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>Student recruitment and retention are important issues for all higher education institutions. Constant monitoring of student satisfaction levels is therefore crucial. Traditionally, students voice their opinions through official surveys organized by the universities. In addition to that, nowadays, social media and review websites such as \u201cRate my professors\u201d are rich sources of opinions that should not be ignored. Automated mining of students\u2019 opinions can be realized via aspect-based sentiment analysis (ABSA). ABSA s is a sub-discipline of natural language processing (NLP) that focusses on the identification of sentiments (negative, neutral, positive) and aspects (sentiment targets) in a sentence. The purpose of this paper is to introduce a system for ABSA of free text reviews expressed in student opinion surveys in the Serbian language. Sentiment analysis was carried out at the finest level of text granularity \u2013 the level of sentence segment (phrase and clause).<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>The presented system relies on NLP techniques, machine learning models, rules and dictionaries. The corpora collected and annotated for system development and evaluation comprise students\u2019 reviews of teaching staff at the Faculty of Technical Sciences, University of Novi Sad, Serbia, and a corpus of publicly available reviews from the Serbian equivalent of the \u201cRate my professors\u201d website.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>The research results indicate that positive sentiment can successfully be identified with the F-measure of 0.83, while negative sentiment can be detected with the F-measure of 0.94. While the F-measure for the aspect\u2019s range is between 0.49 and 0.89, depending on their frequency in the corpus. Furthermore, the authors have concluded that the quality of ABSA depends on the source of the reviews (official students\u2019 surveys vs review websites).<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Practical implications<\/jats:title>\n<jats:p>The system for ABSA presented in this paper could improve the quality of service provided by the Serbian higher education institutions through a more effective search and summary of students\u2019 opinions. For example, a particular educational institution could very easily find out which aspects of their service the students are not satisfied with and to which aspects of their service more attention should be directed.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>To the best of the authors\u2019 knowledge, this is the first study of ABSA carried out at the level of sentence segment for the Serbian language. The methodology and findings presented in this paper provide a much-needed bases for further work on sentiment analysis for the Serbian language that is well under-resourced and under-researched in this area.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/el-06-2019-0140","type":"journal-article","created":{"date-parts":[[2020,2,3]],"date-time":"2020-02-03T10:04:56Z","timestamp":1580724296000},"page":"44-64","source":"Crossref","is-referenced-by-count":47,"title":["Aspect-based sentiment analysis of reviews in the domain of higher education"],"prefix":"10.1108","volume":"38","author":[{"given":"Nikola","family":"Nikoli\u0107","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olivera","family":"Grljevi\u0107","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aleksandar","family":"Kova\u010devi\u0107","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","published-online":{"date-parts":[[2020,2,3]]},"reference":[{"key":"key2021102914450207700_ref001","article-title":"A vector space approach for aspect based sentiment analysis","year":"2015"},{"key":"key2021102914450207700_ref002","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1016\/j.jocs.2017.11.006","article-title":"Deep recurrent neural network vs. support vector machine for aspect-based sentiment analysis of Arabic hotels\u2019 reviews","volume":"27","year":"2018","journal-title":"Journal of Computational Science"},{"key":"key2021102914450207700_ref003","article-title":"GTI at SemEval-2016 task 5: SVM and CRF for aspect detection and unsupervised aspect-based sentiment analysis","year":"2016"},{"key":"key2021102914450207700_ref004","first-page":"7181","article-title":"Aspect-based opinion mining framework using heuristic patterns","volume":"2","year":"2017","journal-title":"Cluster Computing"},{"issue":"2","key":"key2021102914450207700_ref005","doi-asserted-by":"crossref","first-page":"119","DOI":"10.4018\/IJRSDA.2018040106","article-title":"A novel aspect based framework for tourism sector with improvised aspect and opinion mining algorithm","volume":"5","year":"2018","journal-title":"International Journal of Rough Sets and Data Analysis"},{"issue":"1","key":"key2021102914450207700_ref006","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0022-5371(69)80002-2","article-title":"The Pollyanna hypothesis","volume":"8","year":"1969","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"key":"key2021102914450207700_ref007","first-page":"259","article-title":"Aspect-based sentiment analysis of students\u2019 feedback to improve teaching-learning process","volume":"107","year":"2018","journal-title":"Information and Communication Technology for Intelligent Systems"},{"key":"key2021102914450207700_ref008","article-title":"Clause sentiment identification based on convolutional neural network with context embedding","volume-title":"12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD \u201816)","year":"2016"},{"key":"key2021102914450207700_ref009","article-title":"A syntactic approach for aspect based opinion mining","volume-title":"Proceedings of the 9th International Conference on Semantic Computing (ICSC \u201815)","year":"2015"},{"issue":"1","key":"key2021102914450207700_ref010","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","year":"1960","journal-title":"Educational and Psychological Measurement"},{"issue":"8","key":"key2021102914450207700_ref011","doi-asserted-by":"crossref","first-page":"3174","DOI":"10.1016\/j.eswa.2012.12.031","article-title":"Long autonomy or long delay?\u2019 the importance of domain in opinion mining","volume":"40","year":"2013","journal-title":"Expert Systems with Applications"},{"key":"key2021102914450207700_ref012","article-title":"Mining the peanut gallery","volume-title":"Proceedings of the Twelfth International Conference on World Wide Web (WWW \u201803)","year":"2003"},{"key":"key2021102914450207700_ref013","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.eswa.2018.10.003","article-title":"Deep learning for aspect-based sentiment analysis: a comparative review","volume":"118","year":"2019","journal-title":"Expert Systems with Applications"},{"issue":"5","key":"key2021102914450207700_ref014","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","year":"1971","journal-title":"Psychological Bulletin"},{"key":"key2021102914450207700_ref015","article-title":"Statistical methods for rates and proportions","volume-title":"Wiley Series in Probability and Statistics","year":"2003"},{"key":"key2021102914450207700_ref016","article-title":"The linguistic construction of sentiment expressions in student opinionated content: a corpus-based study","volume-title":"Poznan Studies in Contemporary Linguistics","year":"2019"},{"issue":"2","key":"key2021102914450207700_ref017","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1007\/s11063-017-9605-7","article-title":"Cascaded convolutional neural networks for aspect-based opinion summary","volume":"46","year":"2017","journal-title":"Neural Processing Letters"},{"issue":"5","key":"key2021102914450207700_ref018","doi-asserted-by":"crossref","first-page":"4721","DOI":"10.3233\/JIFS-179021","article-title":"Aspect-based sentiment analysis of mobile reviews","volume":"36","year":"2019","journal-title":"Journal of Intelligent and Fuzzy Systems"},{"key":"key2021102914450207700_ref019","article-title":"UWB at SemEval-2016 task 5: aspect based sentiment analysis","year":"2016"},{"key":"key2021102914450207700_ref020","article-title":"Exploiting emoticons in sentiment analysis","volume-title":"Proceedings of the 28th Annual ACM Symposium on Applied Computing (SAC \u201813)","year":"2013"},{"key":"key2021102914450207700_ref021","article-title":"Mining and summarizing customer reviews","volume-title":"Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201804)","year":"2004"},{"key":"key2021102914450207700_ref022","article-title":"Fine-grained product features extraction and categorization in reviews opinion mining","volume-title":"12th International Conference on Data Mining Workshops","year":"2012"},{"key":"key2021102914450207700_ref023","article-title":"OpinionMiner: a novel machine learning system for web opinion mining and extraction","volume-title":"Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201809)","year":"2009"},{"issue":"2","key":"key2021102914450207700_ref024","first-page":"23a","article-title":"A suffix subsumption-based approach to building stemmers and lemmatizers for highly inflectional languages with sparse resources","volume":"9","year":"2008","journal-title":"INFOtheca \u2013 Journal of Informatics and Librarianship"},{"key":"key2021102914450207700_ref025","first-page":"1056","article-title":"Crystal: analyzing predictive opinions on the web","year":"2007","journal-title":"ACL Anthology"},{"key":"key2021102914450207700_ref026","article-title":"NRC-Canada-2014: detecting aspects and sentiment in customer reviews","year":"2014"},{"issue":"12","key":"key2021102914450207700_ref027","doi-asserted-by":"crossref","first-page":"e0144296","DOI":"10.1371\/journal.pone.0144296","article-title":"Sentiment of emojis","volume":"10","year":"2015","journal-title":"PLOS One"},{"key":"key2021102914450207700_ref028","volume-title":"Content Analysis","year":"2019"},{"key":"key2021102914450207700_ref029","unstructured":"Levitz, R.N. (2018), \u201c2018 cost of recruiting an undergraduate student report\u201d, available at: http:\/\/learn.ruffalonl.com\/rs\/395-EOG-977\/images\/RNL_2018_Cost_of_Recruiting_Report_EM-005.pdf (accessed 23 October 2019)."},{"key":"key2021102914450207700_ref030","first-page":"653","article-title":"Structure-aware review mining and summarization","volume-title":"Proceedings of the 23rd International Conference on Computational Linguistics","year":"2010"},{"issue":"4","key":"key2021102914450207700_ref031","first-page":"375","article-title":"Exploring the relationships among service quality, customer loyalty and word-of-mouth for private higher education in Taiwan","volume":"18","year":"2013","journal-title":"Asia Pacific Management Review"},{"key":"key2021102914450207700_ref032","article-title":"Sentiment analysis and opinion mining","volume-title":"Synthesis Lectures on Human Language Technologies Series","year":"2012"},{"key":"key2021102914450207700_ref033","volume-title":"Sentiment Analysis: Mining Opinions, Sentiments, and Emotions","year":"2017"},{"key":"key2021102914450207700_ref034","first-page":"313","article-title":"Retrieving information in Croatian: building a simple and efficient rule-based stemmer","year":"2007","journal-title":"1. Me\u0111unarodna Znanstvena Konferencija the Future of Information Sciences: INFuture2007 \u2013 Digital Information and Heritage"},{"key":"key2021102914450207700_ref035","article-title":"Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM","volume-title":"Thirty-Second AAAI Conference on Artificial Intelligence","year":"2018"},{"issue":"17","key":"key2021102914450207700_ref036","doi-asserted-by":"crossref","first-page":"7764","DOI":"10.1016\/j.eswa.2014.05.045","article-title":"A novel deterministic approach for aspect-based opinion mining in tourism products reviews","volume":"41","year":"2014","journal-title":"Expert Systems with Applications"},{"key":"key2021102914450207700_ref037","unstructured":"Milo\u0161evi\u0107, N. (2012), \u201cStemmer for Serbian language\u201d, ArXiv.Org, available at: https:\/\/arxiv.org\/abs\/1209.4471 (accessed 23 October 2019)."},{"key":"key2021102914450207700_ref038","first-page":"416","article-title":"Improving aspect identification with reviews segmentation","volume-title":"Natural Language Processing and Chinese Computing","year":"2018"},{"key":"key2021102914450207700_ref039","article-title":"Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales","year":"2005"},{"issue":"1\/2","key":"key2021102914450207700_ref040","first-page":"1","article-title":"Opinion mining and sentiment analysis","volume":"2","year":"2008","journal-title":"Foundations and Trends\u00ae in Information Retrieval"},{"key":"key2021102914450207700_ref041","article-title":"Thumbs up? Sentiment classification using machine learning techniques","volume-title":"Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing (EMNLP \u201802)","year":"2002"},{"key":"key2021102914450207700_ref042","unstructured":"Pedregosa, F. Varoquaux, G. Gramfort, A. Michel, V. Thirion, B. Grisel, O. and Blondel, M. (2012), \u201cScikit-learn: machine learning in python\u201d, ArXiv.Org, available at: https:\/\/arxiv.org\/abs\/1201.0490 (accessed 23 October 2019)."},{"key":"key2021102914450207700_ref043","article-title":"SemEval-2016 task 5: aspect based sentiment analysis","year":"2016"},{"key":"key2021102914450207700_ref044","article-title":"SemEval-2015 task 12: aspect based sentiment analysis","year":"2015"},{"key":"key2021102914450207700_ref045","article-title":"SemEval-2014 task 4: aspect based sentiment analysis","year":"2014"},{"key":"key2021102914450207700_ref046","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/978-1-84628-754-1_2","article-title":"Extracting product features and opinions from reviews","volume-title":"Natural Language Processing and Text Mining","year":"2007"},{"issue":"8","key":"key2021102914450207700_ref047","doi-asserted-by":"crossref","first-page":"3077","DOI":"10.1016\/j.jbusres.2016.01.023","article-title":"Brand management in higher education: the university brand personality scale","volume":"69","year":"2016","journal-title":"Journal of Business Research"},{"key":"key2021102914450207700_ref048","doi-asserted-by":"crossref","unstructured":"Ruder, S. Ghaffari, P. and Breslin, J.G. (2016), \u201cINSIGHT-1 at SemEval-2016 task 5: deep learning for multilingual aspect-based sentiment analysis\u201d, ArXiv.Org, available at: https:\/\/arxiv.org\/abs\/1609.02748 (accessed 23 October 2019).","DOI":"10.18653\/v1\/S16-1053"},{"key":"key2021102914450207700_ref049","first-page":"1546","article-title":"SentiHood: targeted aspect based sentiment analysis dataset for urban neighbourhoods","year":"2016","journal-title":"ACL Anthology"},{"key":"key2021102914450207700_ref050","first-page":"767","volume-title":"Sentiue: Target and Aspect Based Sentiment Analysis in SemEval-2015 Task 12","year":"2015"},{"key":"key2021102914450207700_ref051","volume-title":"Introduction to Modern Information Retrieval","year":"1986"},{"key":"key2021102914450207700_ref052","doi-asserted-by":"crossref","unstructured":"Samha, A.K. Li, Y. and Zhang, J. (2014), \u201cAspect-based opinion extraction from customer reviews\u201d, ArXiv.Org, available at: https:\/\/arxiv.org\/abs\/1404.1982 (accessed 23 October 2019).","DOI":"10.5121\/csit.2014.4413"},{"issue":"1","key":"key2021102914450207700_ref053","doi-asserted-by":"crossref","first-page":"34","DOI":"10.30537\/sjcms.v3i1.375","article-title":"Aspects based opinion mining for teacher and course evaluation","volume":"3","year":"2019","journal-title":"Sukkur IBA Journal of Computing and Mathematical Sciences"},{"issue":"3","key":"key2021102914450207700_ref054","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1093\/ptj\/85.3.257","article-title":"The kappa statistic in reliability studies: use, interpretation, and sample size requirements","volume":"85","year":"2005","journal-title":"Physical Therapy"},{"key":"key2021102914450207700_ref055","doi-asserted-by":"crossref","first-page":"108729","DOI":"10.1109\/ACCESS.2019.2928872","article-title":"Aspect-based opinion mining on student\u2019s feedback for faculty teaching performance evaluation","volume":"7","year":"2019","journal-title":"IEEE Access"},{"key":"key2021102914450207700_ref056","first-page":"712","article-title":"Sentiment analysis of movie reviews: a new feature-based heuristic for aspect-level sentiment classification","volume-title":"International Mutli-Conference on Automation, Computing, Communication, Control and Compressed Sensing (IMac4s \u201813)","year":"2013"},{"issue":"1","key":"key2021102914450207700_ref057","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1146\/annurev-linguistics-011415-040518","article-title":"Sentiment analysis: an overview from linguistics","volume":"2","year":"2016","journal-title":"Annual Review of Linguistics"},{"key":"key2021102914450207700_ref058","first-page":"447","article-title":"Sentiment-specific representation learning for document-level sentiment analysis","volume-title":"Proceedings of the Eighth ACM International Conference","year":"2015"},{"key":"key2021102914450207700_ref059","first-page":"1014","article-title":"Learning semantic representations of users and products for document level sentiment classification","year":"2015"},{"issue":"6","key":"key2021102914450207700_ref060","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1177\/0165551510388123","article-title":"Aspect-based sentiment analysis of movie reviews on discussion boards","volume":"36","year":"2010","journal-title":"Journal of Information Science"},{"issue":"4","key":"key2021102914450207700_ref061","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1108\/EL-04-2017-0075","article-title":"Capturing user sentiments for online Indian movie reviews: a comparative analysis of different machine-learning models","volume":"36","year":"2018","journal-title":"The Electronic Library"},{"key":"key2021102914450207700_ref062","article-title":"Thumbs up or thumbs down?","year":"2002"},{"key":"key2021102914450207700_ref063","article-title":"Multi-aspect and multi-class based document sentiment analysis of educational data catering accreditation process","volume-title":"International Conference on Cloud and Ubiquitous Computing and Emerging Technologies","year":"2013"},{"key":"key2021102914450207700_ref064","unstructured":"Wang, B. and Liu, M. (2015), \u201cDeep learning for aspect-based sentiment analysis\u201d, Stanford University report, available at: https:\/\/cs224d.stanford.edu\/reports\/WangBo.pdf (accessed 23 October 2019)."},{"key":"key2021102914450207700_ref065","article-title":"Sentiment expression via emoticons on social media","volume-title":"IEEE International Conference on Big Data (Big Data \u201815)","year":"2015"},{"key":"key2021102914450207700_ref066","article-title":"Aspect-level sentiment analysis using as-capsules","volume-title":"The World Wide Web Conference (WWW \u201819)","year":"2019"},{"key":"key2021102914450207700_ref067","article-title":"Aspect-based opinion summarization with convolutional neural networks","volume-title":"International Joint Conference on Neural Networks (IJCNN \u201816)","year":"2016"},{"key":"key2021102914450207700_ref068","doi-asserted-by":"crossref","unstructured":"Xu, J. Chen, D. Qiu, X. and Huang, X. (2016), \u201cCached long short-term memory neural networks for document-level sentiment classification\u201d, ArXiv.Org, available at: https:\/\/arxiv.org\/abs\/1610.04989 (accessed 23 October 2019).","DOI":"10.18653\/v1\/D16-1172"},{"key":"key2021102914450207700_ref069","article-title":"Aspect based sentiment analysis with gated convolutional networks","year":"2018"},{"key":"key2021102914450207700_ref070","first-page":"1","article-title":"The impact of electronic words of mouth (eWOM) to the brand determination of higher education in Malaysia: from the perspective of Middle East\u2019s student","volume":"4","year":"2014","journal-title":"Journal of Mass Communication Journalism"},{"key":"key2021102914450207700_ref071","unstructured":"Yang, B. and Cardie, C. (2013), \u201cJoint inference for fine-grained opinion extraction\u201d, ACL Anthology, pp. 1640-1649, available at: www.aclweb.org\/anthology\/P13-1161\/ (accessed 23 October 2019)."},{"key":"key2021102914450207700_ref072","article-title":"Sentiment analyzer: extracting sentiments about a given topic using natural language processing techniques","volume-title":"Third IEEE International Conference on Data Mining","year":"2003"},{"issue":"5","key":"key2021102914450207700_ref073","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1108\/EL-05-2017-0120","article-title":"Analysing the features of negative sentiment tweets","volume":"36","year":"2018","journal-title":"The Electronic Library"},{"issue":"4","key":"key2021102914450207700_ref074","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1108\/EL-09-2016-0184","article-title":"Emotion evolutions of sub-topics about popular events on microblogs","volume":"35","year":"2017","journal-title":"The Electronic Library"},{"key":"key2021102914450207700_ref075","unstructured":"Zirn, C. Niepert, M. Stuckenschmidt, H. and Strube, M. (2011), \u201cFine-grained sentiment analysis with structural features\u201d, ACL Anthology, pp. 336-344, available at: www.aclweb.org\/anthology\/I11-1038\/ (accessed 23 October 2019)."}],"container-title":["The Electronic Library"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/EL-06-2019-0140\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/EL-06-2019-0140\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T01:07:26Z","timestamp":1753405646000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/el\/article\/38\/1\/44-64\/85750"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,3]]},"references-count":75,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,2,3]]},"published-print":{"date-parts":[[2020,3,19]]}},"alternative-id":["10.1108\/EL-06-2019-0140"],"URL":"https:\/\/doi.org\/10.1108\/el-06-2019-0140","relation":{},"ISSN":["0264-0473","0264-0473"],"issn-type":[{"value":"0264-0473","type":"print"},{"value":"0264-0473","type":"print"}],"subject":[],"published":{"date-parts":[[2020,2,3]]}}}