{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T20:25:25Z","timestamp":1767903925302,"version":"3.49.0"},"reference-count":55,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,4,19]],"date-time":"2023-04-19T00:00:00Z","timestamp":1681862400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Social media have been a valuable data source for studying people\u2019s opinions, intentions, and behaviours. Such a data source incorporating advanced big data analysis methods, such as machine-operated emotion and sentiment analysis, will open unprecedented opportunities for innovative data-driven destination monitoring and management. However, a big challenge any machine-operated text analysis method faces is the ambiguity of the natural languages, which may cause an expression to have different meanings in different contexts. In this work, we address the ambiguity challenge by proposing a context-aware dictionary-based target-oriented emotion and sentiment analysis method that incorporates inputs from both humans and machines to introduce an alternative approach to measuring emotions and sentiment in limited tourism-related data. The study makes a methodological contribution by creating a target dictionary specifically for tourism sentiment analysis. To demonstrate the performance of the proposed method, a case of target-oriented emotion and sentiment analysis of posts from Twitter for the Gold Coast of Australia as a tourist destination was considered. The results suggest that Twitter data cover a broad range of destination attributes and can be a valuable source for comprehensive monitoring of tourist experiences at a destination.<\/jats:p>","DOI":"10.3390\/fi15040150","type":"journal-article","created":{"date-parts":[[2023,4,19]],"date-time":"2023-04-19T01:09:22Z","timestamp":1681866562000},"page":"150","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Target-Oriented Data Annotation for Emotion and Sentiment Analysis in Tourism Related Social Media Data"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1669-2606","authenticated-orcid":false,"given":"Alireza","family":"Alaei","sequence":"first","affiliation":[{"name":"Faculty of Science and Engineering, Gold Coast Campus, Southern Cross University, Gold Coast, QLD 4225, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3020-5317","authenticated-orcid":false,"given":"Ying","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Hotel and Tourism Management, Hong Kong Polytechnic University, Hong Kong"}]},{"given":"Vinh","family":"Bui","sequence":"additional","affiliation":[{"name":"Faculty of Science and Engineering, Gold Coast Campus, Southern Cross University, Gold Coast, QLD 4225, Australia"}]},{"given":"Bela","family":"Stantic","sequence":"additional","affiliation":[{"name":"School of Information and Communication Technology, Griffith University, Gold Coast, QLD 4222, Australia"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,19]]},"reference":[{"key":"ref_1","unstructured":"Wang, Y. (2023, April 12). More Important than ever: Measuring Tourist Satisfaction. Griffith Institute for Tourism Research Report No 10. Available online: https:\/\/www.griffith.edu.au\/__data\/assets\/pdf_file\/0029\/18884\/Measuring-Tourist-Satisfaction.pdf."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.ijhm.2015.01.010","article-title":"eWOM effects on hotel booking intentions, attitudes, trust, and website perceptions","volume":"46","author":"Ladhari","year":"2015","journal-title":"Int. J. Hosp. Manag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.ijhm.2016.12.006","article-title":"Tourists\u2019 willingness to pay for an accommodation: The effect of eWOM and internal reference price","volume":"62","year":"2017","journal-title":"Int. J. Hosp. Manag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1080\/13683500.2013.841656","article-title":"Opinion mining of hotel customer-generated contents in Chinese weblogs","volume":"18","author":"Chiu","year":"2015","journal-title":"Curr. Issues Tour."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1002\/jtr.2029","article-title":"Twenty-five years of word-of-mouth studies: A critical review of tourism research","volume":"17","author":"Confente","year":"2015","journal-title":"Int. J. Tour. Res."},{"key":"ref_6","first-page":"19","article-title":"Benefits and pitfalls of using tweets to assess destination sentiment","volume":"11","author":"Becken","year":"2019","journal-title":"J. Hosp. Tour. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1177\/0047287517747753","article-title":"Sentiment Analysis in Tourism: Capitalizing on Big Data","volume":"58","author":"Alaei","year":"2019","journal-title":"J. Travel Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e12189","DOI":"10.1002\/eng2.12189","article-title":"Text-based emotion detection: Advances, challenges, and opportunities","volume":"2","author":"Acheampong","year":"2020","journal-title":"Eng. Rep."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1080\/02699939208411068","article-title":"An argument for basic emotions","volume":"6","author":"Ekman","year":"1992","journal-title":"Cogn. Emot."},{"key":"ref_10","unstructured":"Mohammad, S.M. (2021). Emotion Measurement, Woodhead Publishing."},{"key":"ref_11","unstructured":"Wolny, W. (2016, January 24\u201326). Emotion Analysis of Twitter Data That Use Emoticons and Emoji Ideograms. Proceedings of the 25th International Conference on Information Systems Development (ISD2016 POLAND), Katowice, Poland. Available online: https:\/\/aisel.aisnet.org\/isd2014\/proceedings2016\/CreativitySupport\/5\/."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.ijhm.2014.10.013","article-title":"What can big data and text analytics tell us about hotel guest experience and satisfaction?","volume":"44","author":"Xiang","year":"2015","journal-title":"Int. J. Hosp. Manag."},{"key":"ref_13","unstructured":"Liu, C.-C., Yang, T.-H., Hsieh, C.-T., and Soo, V.-W. (2009, January 3\u20135). Towards Text-based Emotion Detection: A Survey and Possible Improvements. Proceedings of the International Conference on Information Management and Engineering, Kuala Lumpur, Malaysia."},{"key":"ref_14","unstructured":"Toprak CJakob, N., and Gurevych, I. (2010, January 13). Sentence and expression level annotation of opinions in user-generated discourse. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL \u201910, Uppsala, Sweden."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ribeiro, F.N., Araujo, M., Goncalves, P., Goncalves, M.A., and Benevenuto, F. (2016). A Benchmark Comparison of State-of-the-Practice Sentiment Analysis Methods. EPJ Data Sci., 23. Available online: https:\/\/epjdatascience.springeropen.com\/articles\/10.1140\/epjds\/s13688-016-0085-1.","DOI":"10.1140\/epjds\/s13688-016-0085-1"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s40558-015-0035-y","article-title":"Analyzing user reviews in tourism with topic models","volume":"16","author":"Rossetti","year":"2016","journal-title":"Inf. Technol. Tour."},{"key":"ref_17","unstructured":"Kessler, J.S., Eckert, M., Clark, L., and Nicolov, N. (2010, January 23\u201326). The 2010 ICWSM JDPA sentiment corpus for the automotive domain. Proceedings of the 4th International AAAI Conference on Weblogs and Social Media Data Workshop Challenge, Washington, DC, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/j.ipm.2018.03.008","article-title":"Implicit aspect extraction in sentiment analysis: Review, taxonomy, oppportunities, and open challenges","volume":"54","author":"Tubishat","year":"2018","journal-title":"Inf. Process. Manag."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pang, B., and Lee, L. (2004, January 21\u201326). A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, Barcelona, Spain.","DOI":"10.3115\/1218955.1218990"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/1500000011","article-title":"Opinion Mining and Sentiment Analysis","volume":"2","author":"Pang","year":"2008","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_21","unstructured":"Brob, J. (2013). Aspect-Oriented Sentiment Analysis of Customer Reviews Using Distant Supervision Techniques. [Ph.D. Thesis, University of Berlin]."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bjorkelund, E., Burnett, T.H., and Norvag, K. (2012, January 3\u20135). A study of opinion mining and visualization of hotel reviews. Proceedings of the 14th International Conference on Information Integration and Web-based Applications & Services, Bali, Indonesia.","DOI":"10.1145\/2428736.2428773"},{"key":"ref_23","unstructured":"Gr\u00e4bner, D., Zanker Fliedl, M.G., and Fuchs, M. (2012). Information and Communication Technologies in Tourism, Springer."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Hutto, C., and Gilbert, E. (2014, January 1\u20134). Vader: A parsimonious rule-based model for sentiment analysis of social media text. Proceedings of the 8th International AAAI Conference on Weblogs and Social Media, Ann Arbor, MI, USA.","DOI":"10.1609\/icwsm.v8i1.14550"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1108\/MD-03-2012-0235","article-title":"Uncovering customer service experiences with Twitter: The case of airline industry","volume":"52","author":"Misopoulos","year":"2014","journal-title":"Manag. Decis."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1504\/MEJM.2013.054071","article-title":"Tourism, travel and tweets: Algorithmic text analysis methodologies in tourism","volume":"1","author":"Claster","year":"2013","journal-title":"Middle East J. Manag."},{"key":"ref_27","unstructured":"Kasper, W., and Vela, M. (2011, January 17\u201319). Sentiment analysis for hotel reviews. Proceedings of the Computational Linguistics-Applications Conference, Jachranka, Poland."},{"key":"ref_28","first-page":"15","article-title":"Airline Service Quality Evaluation Based on Customer Review Using Machine Learning Approach and Sentiment Analysis","volume":"26","author":"Jeon","year":"2021","journal-title":"J. Soc. e-Bus. Stud."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Duan, W., Cao, Q., Yu, Y., and Levy, S. (2013, January 7\u201310). Mining online user-generated content: Using sentiment analysis technique to study hotel service quality. Proceedings of the 46th System Sciences (HICSS), Hawaii International Conference, Maui, HI, USA. Available online: http:\/\/ieeexplore.ieee.org\/xpls\/abs_all.jsp?arnumber=6480220.","DOI":"10.1109\/HICSS.2013.400"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1080\/13683500.2018.1549025","article-title":"Understanding customers\u2019 hotel revisiting behaviour: A sentiment analysis of online feedback reviews","volume":"23","author":"Park","year":"2020","journal-title":"Curr. Issues Tour."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1016\/j.chb.2015.01.075","article-title":"World Cup 2014 in the Twitter World: A big data analysis of sentiments in US sports fans\u2019 tweets","volume":"48","author":"Yu","year":"2015","journal-title":"Comput. Hum. Behav."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Strapparava, C., and Mihalcea, R. (2007, January 23\u201324). SemEval-2007 task 14: Affective text. Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval \u201807), Association for Computational Linguistics, Prague, Czech Republic.","DOI":"10.3115\/1621474.1621487"},{"key":"ref_33","unstructured":"Holzman, L.E., and Pottenger, W.M. (2023, April 12). Classification of Emotions in Internet Chat: An Application of Machine Learning Using Speech Phonemes. Available online: https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=420ddd737833e808965e051a8af20d8bcd76f423."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Alm, C.O., Roth, D., and Sproat, R. (2005, January 6\u20138). Emotions from Text: Machine Learning for Text-based Emotion Prediction. Proceedings of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Vancouver, BC, Canada.","DOI":"10.3115\/1220575.1220648"},{"key":"ref_35","unstructured":"Aman, S., and Szpakowicz, S. (2023, April 12). Identifying Expressions of Emotion in Text. Available online: http:\/\/saimacs.github.io\/pubs\/2007-TSD-paper.pdf."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ding, X., Liu, B., and Yu, P.S. (2008, January 11\u201312). A holistic lexicon-based approach to opinion mining. Proceedings of the International Conference on Web Search and Web Data Mining, Palo Alto, CA, USA.","DOI":"10.1145\/1341531.1341561"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/S0261-5177(99)00095-3","article-title":"Marketing the competitive destination of the future","volume":"21","author":"Buhalis","year":"2000","journal-title":"Tour. Manag."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"103371","DOI":"10.1016\/j.annals.2022.103371","article-title":"Assessing destination satisfaction by social media: An innovative approach using Importance-Performance Analysis","volume":"93","author":"Chen","year":"2022","journal-title":"Ann. Tour. Res."},{"key":"ref_39","unstructured":"Tourism Research Australia (2023, April 12). Chinese Visitor Satisfaction. Available online: https:\/\/www.tourism.australia.com."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1109\/5254.920602","article-title":"Ontology Learning for the Semantic Web","volume":"16","author":"Maedche","year":"2001","journal-title":"IEEE Intell. Syst."},{"key":"ref_41","unstructured":"Ganu, G., Elhadad, N., and Marian, A. (2009, January 28). Beyond the Stars: Improving Rating Predictions Using Review Text Content. Proceedings of the 12th International Workshop on the Web and Databases (WebDB 2009), Providence, RI, USA. Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.150.140&rep=rep1&type=pdf."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wan, H., Yang, Y., Du, J., Liu, Y., Qi, K., and Pan, J.Z. (2020, January 7\u201312). Target-Aspect-Sentiment Joint Detection for Aspect-Based Sentiment Analysis. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6447"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Pontiki, M., Galanis, D., Papageorgiou, H., Manandhar, S., and Androutsopoulos, I. (2015, January 4\u20135). Semeval-2015 task 12: Aspect based sentiment analysis. Proceedings of the 9th International Workshop on Semantic Evaluation, Denver, CO, USA.","DOI":"10.18653\/v1\/S15-2082"},{"key":"ref_44","unstructured":"Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Al-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., and Clercq, O.D. (2016). ProWorkshop on Semantic Evaluation (SemEval-2016), Association for Computational Linguistics."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1080\/13683500.2014.975679","article-title":"Current and potential methods for measuring emotion in tourism experiences: A review","volume":"18","author":"Li","year":"2015","journal-title":"Curr. Issues Tour."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1109\/MIS.2007.79","article-title":"Toward an Affect-Sensitive AutoTutor","volume":"22","author":"Picard","year":"2007","journal-title":"IEEE Intell. Syst."},{"key":"ref_47","unstructured":"McLaren, K. (2023, April 12). Your Emotional Vocabulary List. Dynamic Emotional Integration. Available online: https:\/\/karlamclaren.com\/wp-content\/uploads\/2016\/05\/Emotional-Vocabulary-List-Color.pdf."},{"key":"ref_48","unstructured":"Plutchik, R. (1980). Emotion: A Psychoevolutionary Synthesis, Harper and Row."},{"key":"ref_49","unstructured":"Elbagir, S., and Yang, J. (2020). IAENG Transactions on Engineering Sciences."},{"key":"ref_50","unstructured":"Fleiss, J.L. (1981). Statistical Methods for Rates and Proportions, John Wiley. [2nd ed.]."},{"key":"ref_51","unstructured":"Altman, D.G. (1999). Practical Statistics for Medical Research, Chapman & Hall\/CRC Press."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1177\/1356766710380884","article-title":"Bloggers\u2019 reported tourist experiences: Their utility as a tourism data source and their effect on prospective tourists","volume":"16","author":"Volo","year":"2010","journal-title":"J. Vacat. Mark."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2389","DOI":"10.1073\/pnas.1411678112","article-title":"Human language reveals a universal positivity bias","volume":"112","author":"Dodds","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_54","unstructured":"Vishnubhotla, K., and Mohammad, S.M. (2022, January 20\u201325). Tweet Emotion Dynamics: Emotion Word Usage in Tweets from US and Canada. Proceedings of the 13th Language Resources and Evaluation Conference (LREC-2022), Marseille, France."},{"key":"ref_55","unstructured":"Pappas, N., and Popescu-Belis, A. (August, January 28). Sentiment Analysis of User Comments for OneClass Collaborative Filtering over TED Talks. Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/15\/4\/150\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:18:36Z","timestamp":1760123916000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/15\/4\/150"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,19]]},"references-count":55,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["fi15040150"],"URL":"https:\/\/doi.org\/10.3390\/fi15040150","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,19]]}}}