{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T18:23:48Z","timestamp":1767205428624,"version":"build-2238731810"},"reference-count":51,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,4,15]],"date-time":"2019-04-15T00:00:00Z","timestamp":1555286400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.mdpi.com"],"crossmark-restriction":true},"short-container-title":["Symmetry"],"abstract":"<jats:p>It is the age of the social web, where people express themselves by giving their opinions about various issues, from their personal life to the world\u2019s political issues. This process generates a lot of opinion data on the web that can be processed for valuable information, and therefore, semantic annotation of opinions becomes an important task. Unfortunately, existing opinion annotation schemes have failed to satisfy annotation challenges and cannot even adhere to the basic definition of opinion. Opinion holders, topical features and temporal expressions are major components of an opinion that remain ignored in existing annotation schemes. In this work, we propose OpinionML, a new Markup Language, that aims to compensate for the issues that existing typical opinion markup languages fail to resolve. We present a detailed discussion about existing annotation schemes and their associated problems. We argue that OpinionML is more robust, flexible and easier for annotating opinion data. Its modular approach while implementing a logical model provides us with a flexible and easier model of annotation. OpinionML can be considered a step towards \u201cinformation symmetry\u201d. It is an effort for consistent sentiment annotations across the research community. We perform experiments to prove robustness of the proposed OpinionML and the results demonstrate its capability of retrieving significant components of opinion segments. We also propose OpinionML ontology in an effort to make OpinionML more inter-operable. The ontology proposed is more complete than existing opinion ontologies like Marl and Onyx. A comprehensive comparison of the proposed ontology with existing sentiment ontologies Marl and Onyx proves its worth.<\/jats:p>","DOI":"10.3390\/sym11040545","type":"journal-article","created":{"date-parts":[[2019,4,15]],"date-time":"2019-04-15T11:15:58Z","timestamp":1555326958000},"page":"545","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["OpinionML\u2014Opinion Markup Language for Sentiment Representation"],"prefix":"10.3390","volume":"11","author":[{"given":"Mohammed","family":"Attik","sequence":"first","affiliation":[{"name":"Opinaka Lab, 97 rue Freyr, 34000 Montpellier, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9903-0274","authenticated-orcid":false,"given":"Malik","family":"Missen","sequence":"additional","affiliation":[{"name":"Department of Computer Science & IT, The Islamia University, Bahawalpur 63100, Pakistan"}]},{"given":"Micka\u00ebl","family":"Coustaty","sequence":"additional","affiliation":[{"name":"L3i Lab, Universit\u00e9 of La Rochelle Av. Michel C\u00b4repeau, 17000 La Rochelle, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0854-768X","authenticated-orcid":false,"given":"Gyu","family":"Choi","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Engineering, Yeungnam University, Gyeongsan 712-749, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0880-5164","authenticated-orcid":false,"given":"Fahd","family":"Alotaibi","sequence":"additional","affiliation":[{"name":"Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21577, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2475-5590","authenticated-orcid":false,"given":"Nadeem","family":"Akhtar","sequence":"additional","affiliation":[{"name":"Department of Computer Science & IT, The Islamia University, Bahawalpur 63100, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5710-181X","authenticated-orcid":false,"given":"Muhammad","family":"Jhandir","sequence":"additional","affiliation":[{"name":"Department of Computer Science & IT, The Islamia University, Bahawalpur 63100, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7163-7453","authenticated-orcid":false,"given":"V.","family":"Prasath","sequence":"additional","affiliation":[{"name":"Department of Pediatrics Cincinnati, Children\u2019s Hospital Medical Center, Cincinnati, OH 45229 USA"}]},{"given":"Nadeem","family":"Salamat","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Khawaja Fareed University of Engineering and Technology, Rahim Yar Khan 64200, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9964-4716","authenticated-orcid":false,"given":"Mujtaba","family":"Husnain","sequence":"additional","affiliation":[{"name":"Department of Computer Science & IT, The Islamia University, Bahawalpur 63100, Pakistan"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Murphy, J., Link, M.W., Childs, J.H., Tesfaye, C.L., Dean, E., Stern, M., Pasek, J., Cohen, J., Callegaro, M., and Harwood, P. (2014). Social Media in Public Opinion Research: Report of the AAPOR Task Force on Emerging Technologies in Public Opinion Research, American Association of Public Opinion Research (AAPOR).","DOI":"10.1093\/poq\/nfu053"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1108\/14684520710764069","article-title":"Blog searching: The first general-purpose source of retrospective public opinion in the social sciences?","volume":"31","author":"Thelwall","year":"2007","journal-title":"Online Inf. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Verma, B., and Thakur, R.S. (2018). Sentiment Analysis Using Lexicon and Machine Learning-Based Approaches: A Survey. Proceedings of the International Conference on Recent Advancement on Computer and Communication, Springer.","DOI":"10.1007\/978-981-10-8198-9_46"},{"key":"ref_4","unstructured":"Kiyavitskaya, N., Zeni, N., Cordy, J.R., Mich, L., and Mylopoulos, J. (2005, January 14\u201316). Semi-Automatic Semantic Annotations for Web Documents. Proceedings of the SWAP 2005, 2nd Italian Semantic Web Workshop, Trento, Italy."},{"key":"ref_5","unstructured":"Oren, E., Muller, K., Scerri, S., Handschuh, S., and Sintek, M. (2019, March 01). What Are Semantic Annotations?. Available online: https:\/\/www.ontotext.com\/knowledgehub\/fundamentals\/semantic-annotation\/."},{"key":"ref_6","unstructured":"Hazarika, D., Poria, S., Gorantla, S., Cambria, E., Zimmermann, R., and Mihalcea, R. (arXiv, 2018). CASCADE: Contextual Sarcasm Detection in Online Discussion Forums, arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.datak.2018.04.001","article-title":"Sentiment analysis: An automatic contextual analysis and ensemble clustering approach and comparison","volume":"115","author":"Liu","year":"2018","journal-title":"Data Knowl. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1016\/j.procs.2018.05.111","article-title":"Real-Time Sentiment Analysis of Twitter Streaming data for Stock Prediction","volume":"132","author":"Das","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_9","unstructured":"Holzinger, A. (2014). Social Media Mining and Social Network Analysis: Emerging Research, Emerald Group Publishing Limited Howard House."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, S.M., and Hovy, E. (2006, January 22). Extracting opinions, opinion holders, and topics expressed in online news media text. Proceedings of the Workshop on Sentiment and Subjectivity in Text, Sydney, Australia.","DOI":"10.3115\/1654641.1654642"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1108\/OIR-05-2017-0139","article-title":"Gender bias in sentiment analysis","volume":"42","author":"Thelwall","year":"2018","journal-title":"Online Inf. Rev."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1108\/OIR-05-2017-0153","article-title":"Gender bias in machine learning for sentiment analysis","volume":"42","author":"Thelwall","year":"2018","journal-title":"Online Inf. Rev."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.knosys.2015.06.015","article-title":"A survey on opinion mining and sentiment analysis: Tasks, approaches and applications","volume":"89","author":"Ravi","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1108\/OIR-03-2015-0068","article-title":"Competitive intelligence in social media Twitter: iPhone 6 vs. Galaxy S5","volume":"40","author":"Kim","year":"2016","journal-title":"Online Inf. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1080\/13614568.2018.1448007","article-title":"SentiML++: An extension of the SentiML sentiment annotation scheme","volume":"24","author":"Coustaty","year":"2018","journal-title":"New Rev. Hypermedia Multimed."},{"key":"ref_16","first-page":"627","article-title":"Sentiment Analysis and Subjectivity","volume":"2","author":"Liu","year":"2010","journal-title":"Handb. Nat. Lang. Process."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Missen, M.M.S., Boughanem, M., and Cabanac, G. (2009, January 1\u20133). Challenges for Sentence Level Opinion Detection in Blogs. Proceedings of the Eighth IEEE\/ACIS International Conference on Computer and Information Sciencepp, Shanghai, China.","DOI":"10.1109\/ICIS.2009.190"},{"key":"ref_18","first-page":"79","article-title":"Thumbs Up?: Sentiment Classification Using Machine Learning Techniques","volume":"10","author":"Pang","year":"2002","journal-title":"Assoc. Comput. Linguist."},{"key":"ref_19","unstructured":"Maynard, D., Bontcheva, K., and Rout, D. (2012, January 21\u201327). Challenges in developing opinion mining tools for social media. Proceedings of the@ NLP Can U Tag# Usergeneratedcontent, Istanbul, Turkey."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1108\/OIR-04-2017-0114","article-title":"Sentiment annotations for reviews: An information quality perspective","volume":"42","author":"Yang","year":"2018","journal-title":"Online Inf. Rev."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1145\/944012.944013","article-title":"Measuring Praise and Criticism: Inference of Semantic Orientation from Association","volume":"21","author":"Turney","year":"2003","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhao, J., Dong, L., Wu, J., and Xu, K. (2012, January 12\u201316). Moodlens: An emoticon-based sentiment analysis system for chinese tweets. Proceedings of the 18th ACM SIGKDD International Conference On Knowledge Discovery and Data Mining, Beijing, China.","DOI":"10.1145\/2339530.2339772"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wilson, T., Wiebe, J., and Hoffmann, P. Recognizing contextual polarity in phrase-level sentiment analysis. Proceedings of the Conference on Human Language Technology And Empirical Methods in Natural Language Processing, Vancouver, BC, Canada, 6\u20138 October 2005.","DOI":"10.3115\/1220575.1220619"},{"key":"ref_24","unstructured":"Councill, I.G., McDonald, R., and Velikovich, L. (2010). What\u2019s Great and What\u2019s Not: Learning to Classify the Scope of Negation for Improved Sentiment Analysis. Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, Association for Computational Linguistics. NeSp-NLP \u201910."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1111\/j.1467-8640.2006.00277.x","article-title":"Sentiment Classification of Movie Reviews Using Contextual Valence Shifters","volume":"22","author":"Kennedy","year":"2006","journal-title":"Comput. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Krestel, R., and Siersdorfer, S. (2013, January 1\u20133). Generating Contextualized Sentiment Lexica Based on Latent Topics and User Ratings. Proceedings of the 24th ACM Conference on Hypertext and Social Media, Paris, France.","DOI":"10.1145\/2481492.2481506"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1007\/978-3-642-33412-2_46","article-title":"Mining and Estimating Users\u00e2 Opinion Strength in Forum Texts Regarding Governmental Decisions","volume":"Volume 382","author":"Iliadis","year":"2012","journal-title":"Artificial Intelligence Applications and Innovations"},{"key":"ref_28","unstructured":"Efron, M. (2004, January 21\u201324). Cultural orientation: Classifying subjective documents by cociation analysis. Proceedings of the AAAI Fall Symposium on Style and Meaning in Language, Art, and Music, Washington, DC, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Mohammad, A.S., Al-Ayyoub, M., Zhao, Y., Qin, B., and De Clercq, O. (2016, January 16\u201317). SemEval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), San Diego, CA, USA.","DOI":"10.18653\/v1\/S16-1002"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Di Bari, M., Sharoff, S., and Thomas, M. (2013, January 10). SentiML: Functional Annotation for Multilingual Sentiment Analysis. Proceedings of the 1st International Workshop on Collaborative Annotations in Shared Environment: Metadata, Vocabularies and Techniques in the Digital Humanities, Florence, Italy.","DOI":"10.1145\/2517978.2517994"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"454","DOI":"10.1016\/j.csi.2012.10.004","article-title":"OpinionMining-ML","volume":"35","author":"Robaldo","year":"2013","journal-title":"Comput. Standards Interfaces"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Schroder, M., Baggia, P., Burkhardt, F., Pelachaud, C., Peter, C., and Zovato, E. (2011). EmotionML\u2014An upcoming standard for representing emotions and related states. International Conference on Affective Computing and Intelligent Interaction, Springer.","DOI":"10.1007\/978-3-642-24600-5_35"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1109\/MIS.2016.31","article-title":"Affective computing and sentiment analysis","volume":"30","author":"Cambria","year":"2016","journal-title":"IEEE Intell. Syst."},{"key":"ref_34","unstructured":"Shankland, S. (2019, March 01). EmotionML: Will Computers Tap into Your Feelings?. CNET News, Available online: https:\/\/www.cnet.com\/news\/emotionml-will-computers-tap-into-your-feelings\/."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, B. (2012). Sentiment Analysis and Opinion Mining: Synthesis Lectures on Human Language Technologies, Morgan & Claypool Publishers.","DOI":"10.1007\/978-3-031-02145-9"},{"key":"ref_36","first-page":"1","article-title":"Opinion mining and sentiment analysis","volume":"2","author":"Pang","year":"2008","journal-title":"Inf. Retrieval"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1109\/MIS.2013.30","article-title":"New Avenues in Opinion Mining and Sentiment Analysis","volume":"28","author":"Cambria","year":"2013","journal-title":"IEEE Intell. Syst."},{"key":"ref_38","unstructured":"Swami, A., Mete, A., Bhosle, S., Nimbalkar, N., and Kale, S. (2017). Ferom: Feature Extraction and Refinement for Opinion Mining, Wiley Online Library."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/TAFFC.2014.2317187","article-title":"Are They Different? Affect, Feeling, Emotion, Sentiment, and Opinion Detection in Text","volume":"5","author":"Munezero","year":"2014","journal-title":"Affect. Comput. IEEE Trans."},{"key":"ref_40","unstructured":"Westerski, A., Iglesias, C.A., and Ric, F.T. (2012, January 23). Linked opinions: Describing sentiments on the structured web of data. Proceedings of the 4th International Workshop Social Data on the Web (SDoW2011), Bonn, Germany."},{"key":"ref_41","first-page":"71","article-title":"Onyx: Describing Emotions on the Web of Data","volume":"Volume 1096","author":"Iglesias","year":"2013","journal-title":"Proceedings of the First International Workshop on Emotion and Sentiment in Social and Expressive Media: Approaches and perspectives from AI (ESSEM 2013)"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1109\/MIS.2018.033001419","article-title":"OntoSenticNet: A commonsense ontology for sentiment analysis","volume":"33","author":"Dragoni","year":"2018","journal-title":"IEEE Intell. Syst."},{"key":"ref_43","unstructured":"Peroni, S. (2019, March 01). Graffoo: Graphical Framework for OWL Ontologies. Available online: https:\/\/opencitations.wordpress.com\/2011\/06\/29\/graffoo-a-graphical-framework-for-owl-ontologies\/."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Abecker, A., and van Elst, L. (2004). Ontologies for Knowledge Management, Handbook on Ontologies, Springer.","DOI":"10.1007\/978-3-540-24750-0_22"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/S0957-4174(02)00071-4","article-title":"Ontologies for information management: Balancing formality, stability, and sharing scope","volume":"23","author":"Abecker","year":"2002","journal-title":"Expert Syst. Appl."},{"key":"ref_46","unstructured":"Wilson, T.A. (2008). Fine-Grained Subjectivity and Sentiment Analysis: Recognizing The Intensity, Polarity, and Attitudes of Private States. [Ph.D. Thesis, University of Pittsburgh]."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1177\/001316447303300309","article-title":"The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability","volume":"33","author":"Fleiss","year":"1973","journal-title":"Educ. Psychol. Meas."},{"key":"ref_48","unstructured":"Baeza-Yates, R., and Ribeiro-Neto, B. (1999). Modern Information Retrieval, ACM Press."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1108\/OIR-07-2016-0201","article-title":"Application of social media analytics: A case of analyzing online hotel reviews","volume":"41","author":"He","year":"2017","journal-title":"Online Inf. Rev."},{"key":"ref_50","first-page":"28","article-title":"Timeml: Robust specification of event and temporal expressions in text","volume":"3","author":"Pustejovsky","year":"2003","journal-title":"New Direct. Quest. Answ."},{"key":"ref_51","unstructured":"Saur\u00ed, R., Littman, J., Knippen, B., Gaizauskas, R., Setzer, A., and Pustejovsky, J. (2019, March 01). TimeML Annotation Guidelines, Available online: https:\/\/www.researchgate.net\/profile\/James_Pustejovsky\/publication\/248737128_TimeML_Annotation_Guidelines_Version_121\/links\/55c9d67c08aeb97567483792.pdf."}],"updated-by":[{"DOI":"10.3390\/sym12020187","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2019,4,15]],"date-time":"2019-04-15T00:00:00Z","timestamp":1555286400000}}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/4\/545\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,4]],"date-time":"2025-08-04T06:40:25Z","timestamp":1754289625000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/4\/545"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,15]]},"references-count":51,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["sym11040545"],"URL":"https:\/\/doi.org\/10.3390\/sym11040545","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,15]]}}}