{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T23:05:13Z","timestamp":1778886313403,"version":"3.51.4"},"reference-count":47,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2015,12,25]],"date-time":"2015-12-25T00:00:00Z","timestamp":1451001600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The European Commission under the 7th Framework Programme","award":["316097"],"award-info":[{"award-number":["316097"]}]},{"name":"Polish National Science Centre","award":["DEC-2013\/09\/B\/ST6\/02317"],"award-info":[{"award-number":["DEC-2013\/09\/B\/ST6\/02317"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We propose a novel method for counting sentiment orientation that outperforms supervised learning approaches in time and memory complexity and is not statistically significantly different from them in accuracy. Our method consists of a novel approach to generating unigram, bigram and trigram lexicons. The proposed method, called frequentiment, is based on calculating the frequency of features (words) in the document and averaging their impact on the sentiment score as opposed to documents that do not contain these features. Afterwards, we use ensemble classification to improve the overall accuracy of the method. What is important is that the frequentiment-based lexicons with sentiment threshold selection outperform other popular lexicons and some supervised learners, while being 3\u20135 times faster than the supervised approach. We compare 37 methods (lexicons, ensembles with lexicon\u2019s predictions as input and supervised learners) applied to 10 Amazon review data sets and provide the first statistical comparison of the sentiment annotation methods that include ensemble approaches. It is one of the most comprehensive comparisons of domain sentiment analysis in the literature.<\/jats:p>","DOI":"10.3390\/e18010004","type":"journal-article","created":{"date-parts":[[2015,12,28]],"date-time":"2015-12-28T04:23:49Z","timestamp":1451276629000},"page":"4","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Comprehensive Study on Lexicon-based Ensemble Classification Sentiment Analysis"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4090-4480","authenticated-orcid":false,"given":"\u0141ukasz","family":"Augustyniak","sequence":"first","affiliation":[{"name":"Department of Computational Intelligence, Wroc\u0142aw University of Technology, Wybrze\u017ce Stanis\u0142awa Wyspia\u0144skiego 27, Wroc\u0142aw 50-370, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7733-3239","authenticated-orcid":false,"given":"Piotr","family":"Szyma\u0144ski","sequence":"additional","affiliation":[{"name":"Department of Computational Intelligence, Wroc\u0142aw University of Technology, Wybrze\u017ce Stanis\u0142awa Wyspia\u0144skiego 27, Wroc\u0142aw 50-370, Poland"},{"name":"Illimites Foundation, Gajowicka 64 lok. 1, Wroc\u0142aw 53-422, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tomasz","family":"Kajdanowicz","sequence":"additional","affiliation":[{"name":"Department of Computational Intelligence, Wroc\u0142aw University of Technology, Wybrze\u017ce Stanis\u0142awa Wyspia\u0144skiego 27, Wroc\u0142aw 50-370, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"W\u0142odzimierz","family":"Tulig\u0142owicz","sequence":"additional","affiliation":[{"name":"Department of Computational Intelligence, Wroc\u0142aw University of Technology, Wybrze\u017ce Stanis\u0142awa Wyspia\u0144skiego 27, Wroc\u0142aw 50-370, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,12,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1609\/icwsm.v4i1.14009","article-title":"Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment","volume":"10","author":"Tumasjan","year":"2010","journal-title":"ICWSM"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"6266","DOI":"10.1016\/j.eswa.2013.05.057","article-title":"Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network","volume":"40","author":"Ghiassi","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_3","unstructured":"Brody, S., and Elhadad, N. (2010, January 1\u20136). An Unsupervised Aspect-sentiment Model for Online Reviews. Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, Los Angeles, CA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jocs.2010.12.007","article-title":"Twitter mood predicts the stock market","volume":"2","author":"Bollen","year":"2011","journal-title":"J. Comput. Sci."},{"key":"ref_5","unstructured":"Liu, B., and Zhang, L. (2012). Mining Text Data, Springer-Verlag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/1500000011","article-title":"Opinion mining and sentiment analysis","volume":"2","author":"Pang","year":"2008","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1016\/j.asej.2014.04.011","article-title":"Sentiment analysis algorithms and applications: A survey","volume":"5","author":"Medhat","year":"2014","journal-title":"Ain Shams Eng. J."},{"key":"ref_8","unstructured":"Liu, B. (2011, January 7\u201311). Sentiment Analysis and Opinion Mining. Proceedings of the Twenty-Fifth Conference on Artificial Intelligence (AAAI-11), San Francisco, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hu, M., and Liu, B. (2004, January 22\u201325). Mining and Summarizing Customer Reviews. Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA.","DOI":"10.1145\/1014052.1014073"},{"key":"ref_10","unstructured":"Qiu, G., Liu, B., Bu, J., and Chen, C. (2009, January 11\u201317). Expanding Domain Sentiment Lexicon Through Double Propagation. Proceedings of the 21st International Jont Conference on Artifical Intelligence (IJCAI\u201909), Pasadena, CA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hatzivassiloglou, V., and McKeown, K.R. (1997, January 7\u201312). Predicting the semantic orientation of adjectives. Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics, Madrid, Spain.","DOI":"10.3115\/976909.979640"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yu, H., and Hatzivassiloglou, V. (2003, January 11\u201312). Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences. Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201903), Sapporo, Japan.","DOI":"10.3115\/1119355.1119372"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Read, J., and Carroll, J. (2009, January 6). Weakly supervised techniques for domain-independent sentiment classification. Proceedings of the 1st International CIKM Workshop on Topic-Sentiment Analysis for Mass Opinion, Hong Kong, China.","DOI":"10.1145\/1651461.1651470"},{"key":"ref_14","unstructured":"Whitehead, M., and Yaeger, L. (2010). Innovations and Advances in Computer Sciences and Engineering, Springer Netherlands."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lu, Y., Castellanos, M., Dayal, U., and Zhai, C. (2011, January 30). Automatic Construction of a Context-aware Sentiment Lexicon: An Optimization Approach. Proceedings of the 20th International Conference on World Wide Web (WWW\u201911), Hyderabad, India.","DOI":"10.1145\/1963405.1963456"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ding, X., Liu, B., and Yu, P.S. (2008, January 11\u201312). A Holistic Lexicon-based Approach to Opinion Mining. Proceedings of the 2008 International Conference on Web Search and Data Mining (WSDM\u201908), Palo Alto, CA, USA.","DOI":"10.1145\/1341531.1341561"},{"key":"ref_17","unstructured":"Mohammad, S.M., Kiritchenko, S., and Zhu, X. (2013, January 14\u201315). NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets. Proceedings of the Seventh International Workshop on Semantic Evaluation Exercises (SemEval-2013), Atlanta, GA, USA."},{"key":"ref_18","unstructured":"Weiss, S.M., and Kulikowski, C.A. (1991). Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems, Morgan Kaufmann Publishers Inc."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"79","DOI":"10.3115\/1118693.1118704","article-title":"Thumbs up?: Sentiment classification using machine learning techniques","volume":"Volume 10","author":"Bo","year":"2002","journal-title":"Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6527","DOI":"10.1016\/j.eswa.2008.07.035","article-title":"Sentiment classification of online reviews to travel destinations by supervised machine learning approaches","volume":"36","author":"Ye","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref_21","unstructured":"Schler, J. (2005, January 26\u201327). The Importance of Neutral Examples for Learning Sentiment. Proceedings of the Workshop on the Analysis of Informal and Formal Information Exchange during Negotiations (FINEXIN05), Ottawa, ON, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"732","DOI":"10.1016\/j.dss.2010.08.024","article-title":"Predicting consumer sentiments from online text","volume":"50","author":"Bai","year":"2011","journal-title":"Decis. Support Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., and Potts, C. (2013, January 18\u201321). Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), Seattle, WA, USA.","DOI":"10.18653\/v1\/D13-1170"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Narayanan, V., Arora, I., and Bhatia, A. (2013, January 20\u201323). Fast and Accurate Sentiment Classification Using an Enhanced Naive Bayes Model. Proceedings of the 14th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL 2013), Hefei, China.","DOI":"10.1007\/978-3-642-41278-3_24"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gamon, M. (2004, January 23\u201327). Sentiment Classification on Customer Feedback Data: Noisy Data, Large Feature Vectors, and the Role of Linguistic Analysis. Proceedings of the 20th International Conference on Computational Linguistics (COLING\u201904), Geneva, Switzerland,.","DOI":"10.3115\/1220355.1220476"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Augustyniak, L., Kajdanowicz, T., Kazienko, P., Kulisiewicz, M., and Tuliglowicz, W. (2014, January 11\u201313). An Approach to Sentiment Analysis of Movie Reviews: Lexicon Based vs.. Classification. Proceedings of the 9th International Conference on Hybrid Artificial Intelligence Systems (HAIS 2014), Salamanca, Spain.","DOI":"10.1007\/978-3-319-07617-1_15"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Augustyniak, L., Kajdanowicz, T., Szymanski, P., Tuliglowicz, W., Kazienko, P., Alhajj, R., and Szymanski, B.K. (2014, January 17\u201320). Simpler is better? Lexicon-based ensemble sentiment classification beats supervised methods. Proceedings of the 2014 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014), Beijing, China.","DOI":"10.1109\/ASONAM.2014.6921696"},{"key":"ref_28","unstructured":"Whitehead, M., and Yaeger, L. (2010). Innovations and Advances in Computer Sciences and Engineering, Springer Netherlands."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging Predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_30","unstructured":"Schapire, R.E. (August, January 31). A Brief Introduction to Boosting. Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 99), Stockholm, Sweden."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MCAS.2006.1688199","article-title":"Ensemble Based Systems in Decision Making","volume":"6","author":"Polikar","year":"2006","journal-title":"IEEE Circ. Syst. Mag."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"McAuley, J., and Leskovec, J. (2013, January 12\u201316). Hidden factors and hidden topics: understanding rating dimensions with review text. Proceedings of the 7th ACM Conference on Recommender Systems, Hong Kong, China.","DOI":"10.1145\/2507157.2507163"},{"key":"ref_33","unstructured":"Python Library Beautiful Soup. Available online: https:\/\/pypi.python.org\/pypi\/beautifulsoup4."},{"key":"ref_34","unstructured":"Python Library Unidecode. Available online: https:\/\/pypi.python.org\/pypi\/Unidecode."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1145\/944012.944013","article-title":"Measuring Praise and Criticism: Inference of Semantic Orientation from Association","volume":"21","author":"Turney","year":"2003","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_36","unstructured":"Nielsen, F.\u00c5. AFINN Informatics and Mathematical Modelling, Technical University of Denmark, 2011. Available online: http:\/\/www2.imm.dtu.dk\/pubdb\/p.php?6010."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wilson, T., Wiebe, J., and Hoffmann, P. (2005, January 6\u20138). Recognizing Contextual Polarity in Phrase-level Sentiment Analysis. Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Vancouver, BC, Canada.","DOI":"10.3115\/1220575.1220619"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1111\/j.1467-8640.2012.00460.x","article-title":"Crowdsourcing a Word-Emotion Association Lexicon","volume":"29","author":"Mohammad","year":"2013","journal-title":"Comput. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kiritchenko, S., Zhu, X., Cherry, C., and Mohammad, S. (2014, January 23\u201324). NRC-Canada-2014: Detecting Aspects and Sentiment in Customer Reviews. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland.","DOI":"10.3115\/v1\/S14-2076"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1145\/944012.944013","article-title":"Measuring praise and criticism: Inference of semantic orientation from association","volume":"21","author":"Turney","year":"2003","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_41","unstructured":"SemEval Contest Website. Available online: https:\/\/www.cs.york.ac.uk\/semeval-2013\/task2."},{"key":"ref_42","unstructured":"Cieliebak, M., D\u00fcrr, O., and Uzdilli, F. (2014, January 26\u201331). Meta-Classifiers Easily Improve Commercial Sentiment Detection Tools. Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC\u201914), Reykjavik, Iceland."},{"key":"ref_43","unstructured":"NRC Canada Website. Available online: http:\/\/saifmohammad.com\/WebPages\/lexicons.html."},{"key":"ref_44","unstructured":"Macquarie Semantic Orientation Lexicon (MSOL) Link. Available online: http:\/\/www.saifmohammad.com\/Release\/MSOL-June15-09.txt."},{"key":"ref_45","unstructured":"Higher-level threading interface in Python. Available online: https:\/\/docs.python.org\/2\/library\/threading.html."},{"key":"ref_46","unstructured":"Python Library Memory Profiler. Available online: https:\/\/pypi.python.org\/pypi\/memory\/profiler."},{"key":"ref_47","unstructured":"Tang, D., Wei, F., Qin, B., Zhou, M., and Liu, T. (2014, January 23\u201329). Building Large-Scale Twitter-Specific Sentiment Lexicon:A Representation Learning Approach. Proceedings of the 25th International Conference on Computational Linguistics (COLING 2014), Dublin, Ireland."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:54:47Z","timestamp":1760216087000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,12,25]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2016,1]]}},"alternative-id":["e18010004"],"URL":"https:\/\/doi.org\/10.3390\/e18010004","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,12,25]]}}}