{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:40:55Z","timestamp":1760175655577,"version":"build-2065373602"},"reference-count":72,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,2,19]],"date-time":"2020-02-19T00:00:00Z","timestamp":1582070400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Data"],"abstract":"<jats:p>Through social media platforms, massive amounts of data are being produced. As a microblogging social media platform, Twitter enables its users to post short updates as \u201ctweets\u201d on an unprecedented scale. Once analyzed using machine learning (ML) techniques and in aggregate, Twitter data can be an invaluable resource for gaining insight into different domains of discussion and public opinion. However, when applied to real-time data streams, due to covariate shifts in the data (i.e., changes in the distributions of the inputs of ML algorithms), existing ML approaches result in different types of biases and provide uncertain outputs. In this paper, we describe VARTTA (Visual Analytics for Real-Time Twitter datA), a visual analytics system that combines data visualizations, human-data interaction, and ML algorithms to help users monitor, analyze, and make sense of the streams of tweets in a real-time manner. As a case study, we demonstrate the use of VARTTA in political discussions. VARTTA not only provides users with powerful analytical tools, but also enables them to diagnose and to heuristically suggest fixes for the errors in the outcome, resulting in a more detailed understanding of the tweets. Finally, we outline several issues to be considered while designing other similar visual analytics systems.<\/jats:p>","DOI":"10.3390\/data5010020","type":"journal-article","created":{"date-parts":[[2020,2,20]],"date-time":"2020-02-20T03:20:03Z","timestamp":1582168803000},"page":"20","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["VARTTA: A Visual Analytics System for Making Sense of Real-Time Twitter Data"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6383-1387","authenticated-orcid":false,"given":"Amir","family":"Haghighati","sequence":"first","affiliation":[{"name":"Insight Lab, Middlesex College, Western University, London, ON N6A 3K7, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kamran","family":"Sedig","sequence":"additional","affiliation":[{"name":"Insight Lab, Middlesex College, Western University, London, ON N6A 3K7, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhao, D., and Rosson, M.B. (2009, January 10\u201313). How and Why People Twitter: The Role That Micro-Blogging Plays in Informal Communication at Work. Proceedings of the ACM 2009 International Conference on Supporting Group Work, Sanibel Island, FL, USA.","DOI":"10.1145\/1531674.1531710"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kwak, H., Lee, C., Park, H., and Moon, S. (2010, January 26\u201330). What is Twitter, a Social Network or a News Media?. Proceedings of the 19th International Conference on World Wide Web, Raleigh, NC, USA.","DOI":"10.1145\/1772690.1772751"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"868","DOI":"10.1016\/j.chb.2016.06.022","article-title":"Branding with Social Media: User Gratifications, Usage Patterns, and Brand Message Content Strategies","volume":"63","author":"Gao","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.chb.2015.03.067","article-title":"Individual and Social Benefits of Online Discussion Forums","volume":"50","author":"Pendry","year":"2015","journal-title":"Comput. Hum. Behav."},{"key":"ref_5","unstructured":"(2019, December 06). Tweeting Made Easier. Available online: https:\/\/blog.twitter.com\/en_us\/topics\/product\/2017\/tweetingmadeeasier.html."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ninkov, A., and Sedig, K. (2019). VINCENT: A Visual Analytics System for Investigating the Online Vaccine debate. Online J. Public Health Inform., 11.","DOI":"10.5210\/ojphi.v11i2.10114"},{"key":"ref_7","unstructured":"(2019, December 10). Twitter\u2014Company. Available online: https:\/\/about.twitter.com\/en_us\/company.html."},{"key":"ref_8","unstructured":"(2019, December 06). Twitter: Number of Active Users 2010\u20132019. Available online: https:\/\/www.statista.com\/statistics\/282087\/number-of-monthly-active-twitter-users\/."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1145\/3162956","article-title":"Microblog Analysis as a Program of Work","volume":"1","author":"Tolmie","year":"2018","journal-title":"Trans. Soc. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1177\/1464884914550135","article-title":"What Journalists Retweet: Opinion, Humor, and Brand Development on Twitter","volume":"16","author":"Molyneux","year":"2015","journal-title":"Journalism"},{"key":"ref_11","unstructured":"Neuberger, C., vom Hofe, H.J., and Nuernbergk, C. (2014). The Use of Twitter by Professional Journalists: Results of a Newsroom Survey in Germany. Twitter and Society, Peter Lang."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1080\/19331681.2015.1132401","article-title":"Twitter Use in Election Campaigns: A Systematic Literature Review","volume":"13","author":"Jungherr","year":"2016","journal-title":"J. Inf. Technol. Polit."},{"key":"ref_13","unstructured":"Maireder, A., and Ausserhofer, J. (2014). Political Discourses on Twitter: Networking Topics, Objects, and People. Twitter and Society, Peter Lang."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1016\/j.giq.2017.11.001","article-title":"Analysis of Political Discourse on Twitter in the Context of the 2016 Us Presidential Elections","volume":"34","author":"Yaqub","year":"2017","journal-title":"Gov. Inf. Q."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.ijinfomgt.2018.07.008","article-title":"Twitter as a Tool for the Management and Analysis of Emergency Situations: A Systematic Literature Review","volume":"43","year":"2018","journal-title":"Int. J. Inf. Manag."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.2105\/AJPH.2016.303512","article-title":"Twitter as a Tool for Health Research: A Systematic Review","volume":"107","author":"Sinnenberg","year":"2017","journal-title":"Am. J. Public Health"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Charles-Smith, L.E., Reynolds, T.L., Cameron, M.A., Conway, M., Lau, E.H.Y., Olsen, J.M., Pavlin, J.A., Shigematsu, M., Streichert, L.C., and Suda, K.J. (2015). Using Social Media for Actionable Disease Surveillance and Outbreak Management: A Systematic Literature Review. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0139701"},{"key":"ref_18","first-page":"293","article-title":"Twitter, Brands, and User Engagement","volume":"89","author":"Nitins","year":"2014","journal-title":"Twitter and Society"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Weller, K., Bruns, A., Burgess, J., Mahrt, M., and Puschmann, C. (2013). Twitter and Society, Peter Lang. [1st ed.].","DOI":"10.3726\/978-1-4539-1170-9"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1007\/s11192-016-1910-9","article-title":"Grand Challenges in Altmetrics: Heterogeneity, Data Quality and Dependencies","volume":"108","author":"Haustein","year":"2016","journal-title":"Scientometrics"},{"key":"ref_21","unstructured":"(2019, December 18). Overview (Twitter API Documents). Available online: https:\/\/developer.twitter.com\/en\/docs\/tweets\/filter-realtime\/overview."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e48","DOI":"10.2196\/jmir.1249","article-title":"Social Media Use in The United States: Implications for Health Communication","volume":"11","author":"Chou","year":"2009","journal-title":"J. Med. Internet Res."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T., Sullivan, E., Drummond, R., and Herlocker, J. (2007, January 28\u201331). Toward Harnessing User Feedback for Machine Learning. Proceedings of the 12th International Conference on Intelligent User Interfaces, Honolulu, HI, USA.","DOI":"10.1145\/1216295.1216316"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pfahringer, B., Holmes, G., and Hoffmann, A. (2010). Sentiment Knowledge Discovery in Twitter Streaming Data. Discovery Science, Springer.","DOI":"10.1007\/978-3-642-16184-1"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.2105\/AJPH.2018.304497","article-title":"Toward Real-Time Infoveillance of Twitter Health Messages","volume":"108","author":"Colditz","year":"2018","journal-title":"Am. J. Public Health"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","article-title":"A Unifying View on Dataset Shift in Classification","volume":"45","author":"Raeder","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1145\/2523813","article-title":"A Survey on Concept Drift Adaptation","volume":"46","author":"Gama","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_28","first-page":"12","article-title":"Towards a Characterization of Interactivity in Visual Analytics","volume":"3","author":"Sedig","year":"2012","journal-title":"JMPT"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Parsons, P., Sedig, K., Didandeh, A., and Khosravi, A. (2015, January 5\u20138). Interactivity in Visual Analytics: Use of Conceptual Frameworks to Support Human-Centered Design of a Decision-Support Tool. Proceedings of the 2015 48th Hawaii International Conference on System Sciences, Kauai, HI, USA.","DOI":"10.1109\/HICSS.2015.138"},{"key":"ref_30","unstructured":"Yamamoto, S. (2016). Externalization of Data Analytics Models: Human Interface and the Management of Information: Information, Design and Interaction, Springer."},{"key":"ref_31","first-page":"1","article-title":"Design of Visualizations for Human-Information Interaction: A Pattern-Based Framework","volume":"4","author":"Sedig","year":"2016","journal-title":"Synth. Lect. Vis."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Laylavi, F., Rajabifard, A., and Kalantari, M. (2016). A Multi-Element Approach to Location Inference of Twitter: A Case for Emergency Response. ISPRS Int. J. Geo-Inf., 5.","DOI":"10.3390\/ijgi5050056"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1016\/j.ipm.2016.04.003","article-title":"Measuring User Influence on Twitter: A Survey","volume":"52","author":"Riquelme","year":"2016","journal-title":"Inf. Process. Manag."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4:1","DOI":"10.1145\/2700060","article-title":"Aggregate Characterization of User Behavior in Twitter and Analysis of the Retweet Graph","volume":"15","author":"Bild","year":"2015","journal-title":"ACM Trans. Internet Technol."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Desai, M., and Mehta, M.A. (2016, January 29\u201330). Techniques for Sentiment Analysis of Twitter Data: A Comprehensive Survey. Proceedings of the 2016 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India.","DOI":"10.1109\/CCAA.2016.7813707"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1007\/s002360050095","article-title":"A Survey of Stream Processing","volume":"34","author":"Stephens","year":"1997","journal-title":"Acta Inform."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1145\/1107499.1107504","article-title":"The 8 Requirements of Real-Time Stream Processing","volume":"34","author":"Stonebraker","year":"2005","journal-title":"SIGMOD Rec."},{"key":"ref_38","unstructured":"(2019, December 17). Spark Streaming|Apache Spark. Available online: https:\/\/spark.apache.org\/streaming\/."},{"key":"ref_39","unstructured":"(2020, January 26). Apama Community Edition. Available online: http:\/\/www.apamacommunity.com\/."},{"key":"ref_40","unstructured":"(2020, January 26). Dataflow: Stream & Batch Processing. Available online: https:\/\/cloud.google.com\/dataflow."},{"key":"ref_41","unstructured":"Thomas, J.J., and Cook, K.A. (2005). Illuminating the Path: The Research and Development Agenda for Visual Analytics, National Visualization and Analytics Ctr."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1007\/s10844-014-0304-9","article-title":"The Human Is the Loop: New Directions for Visual Analytics","volume":"43","author":"Endert","year":"2014","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Parsons, P., Sedig, K., Mercer, R.E., Khordad, M., Knoll, J., and Rogan, P. (2015, January 25). Visual Analytics for Supporting Evidence-based Interpretation of Molecular Cytogenomic Findings. Proceedings of the 2015 Workshop on Visual Analytics in Healthcare, Chicago, IL, USA.","DOI":"10.1145\/2836034.2836036"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1177\/1473871611415987","article-title":"Many Roads Lead to Rome: Mapping Users\u2019 Problem-Solving Strategies","volume":"10","author":"Mayr","year":"2011","journal-title":"Inf. Vis."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Huang, W. (2014). Distribution of Information Processing While Performing Complex Cognitive Activities with Visualization Tools. Handbook of Human Centric Visualization, Springer.","DOI":"10.1007\/978-1-4614-7485-2"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2135","DOI":"10.1109\/TMM.2016.2614220","article-title":"A Survey on Visual Analytics of Social Media Data","volume":"18","author":"Wu","year":"2016","journal-title":"IEEE Trans. Multimed."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1111\/cgf.13211","article-title":"Social Media Visual Analytics","volume":"36","author":"Chen","year":"2017","journal-title":"Comput. Graph. Forum"},{"key":"ref_48","unstructured":"Zhao, K., Ward, M., Rundensteiner, E., and Higgins, H. (2019, December 19). MaVis: Machine Learning Aided Multi-Model Framework for Time Series Visual Analytics. Available online: https:\/\/www.ingentaconnect.com\/content\/ist\/ei\/2016\/00002016\/00000001\/art00018."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"36","DOI":"10.3389\/fninf.2016.00036","article-title":"A Multi-Facetted Visual Analytics Tool for Exploratory Analysis of Human Brain and Function Datasets","volume":"10","author":"Angulo","year":"2016","journal-title":"Front. Neuroinform."},{"key":"ref_50","first-page":"1","article-title":"Bridging Information Visualization with Machine Learning (Dagstuhl Seminar 15101)","volume":"5","author":"Keim","year":"2015","journal-title":"Dagstuhl Rep."},{"key":"ref_51","first-page":"223","article-title":"The Challenge of Big Data in Public Health: An Opportunity for Visual Analytics","volume":"5","author":"Ola","year":"2014","journal-title":"Online J. Public Health Inform."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Torra, V., Narukawa, Y., Navarro-Arribas, G., and Ya\u00f1ez, C. (2016). Machine Learning Combining with Visualization for Intrusion Detection: A Survey. Modeling Decisions for Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-319-45656-0"},{"key":"ref_53","unstructured":"Fr\u00e9nay, B., and Dumas, B. (2016, January 27\u201329). Information Visualisation and Machine Learning: Characteristics, Convergence and Perspective. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2016), Bruges, Belgium."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Majchrzak, T.A., Bi\u00f8rn-Hansen, A., and Gr\u00f8nli, T.-M. (2018, January 2\u20136). Progressive Web Apps: The Definite Approach to Cross-Platform Development?. Proceedings of the 51st Hawaii International Conference on System Sciences (HICSS 2018), Hilton Waikoloa Village, Hi, USA.","DOI":"10.24251\/HICSS.2018.718"},{"key":"ref_55","unstructured":"(2018, July 26). Progressive Web Apps|Web|Google Developers. Available online: https:\/\/developers.google.com\/web\/progressive-web-apps\/."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/MIC.2012.64","article-title":"Communicating and Displaying Real-Time Data with WebSocket","volume":"16","author":"Pimentel","year":"2012","journal-title":"IEEE Internet Comput."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Malavolta, I., Procaccianti, G., Noorland, P., and Vukmirovic, P. (2017, January 22\u201323). Assessing the Impact of Service Workers on the Energy Efficiency of Progressive Web Apps. Proceedings of the 2017 IEEE\/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), Buenos Aires, Argentina.","DOI":"10.1109\/MOBILESoft.2017.7"},{"key":"ref_58","unstructured":"Hanchett, E., and Listwon, B. (2018). Vue.Js in Action, Manning Publications Co.. [1st ed.]."},{"key":"ref_59","unstructured":"(2019, December 17). The Progressive JavaScript Framework. Available online: https:\/\/vuejs.org\/."},{"key":"ref_60","unstructured":"(2019, December 20). socketio\/socket.io. Available online: https:\/\/github.com\/socketio\/socket.io."},{"key":"ref_61","unstructured":"(2020, January 09). What is Meta Framework|IGI Global. Available online: https:\/\/www.igi-global.com\/dictionary\/java-web-application-frameworks\/18372."},{"key":"ref_62","unstructured":"(2019, December 17). The Progressive Vue.js Framework. Available online: https:\/\/nuxtjs.org."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"2451","DOI":"10.1109\/TVCG.2015.2509990","article-title":"Online Visual Analytics of Text Streams","volume":"22","author":"Liu","year":"2016","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Kucher, K., and Kerren, A. (2015, January 14\u201317). Text visualization techniques: Taxonomy, visual survey, and community insights. Proceedings of the 2015 IEEE Pacific Visualization Symposium (PacificVis), Hangzhou, China.","DOI":"10.1109\/PACIFICVIS.2015.7156366"},{"key":"ref_65","unstructured":"(2019, December 17). NaturalNode\/natural. Available online: https:\/\/github.com\/NaturalNode\/natural."},{"key":"ref_66","unstructured":"Nielsen, F.\u00c5. (2011). A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs. arXiv."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"5984","DOI":"10.1016\/j.eswa.2014.04.005","article-title":"Building Layered, Multilingual Sentiment Lexicons at Synset and Lemma Levels","volume":"41","author":"Cruz","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_68","unstructured":"(2019, December 17). Clips\/Pattern. Available online: https:\/\/github.com\/clips\/pattern."},{"key":"ref_69","unstructured":"Sliwinski, A. (2019, December 17). Thisandagain\/Sentiment. Available online: https:\/\/github.com\/thisandagain\/sentiment."},{"key":"ref_70","unstructured":"(2019, December 17). Watson Natural Language Understanding\u2014Overview. Available online: https:\/\/www.ibm.com\/cloud\/watson-natural-language-understanding."},{"key":"ref_71","unstructured":"(2019, December 20). Scikit-Learn: Machine Learning in Python\u2014Scikit-Learn 0.22 Documentation. Available online: https:\/\/scikit-learn.org\/stable\/."},{"key":"ref_72","unstructured":"(2019, December 20). TensorFlow. Available online: https:\/\/www.tensorflow.org\/."}],"container-title":["Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5729\/5\/1\/20\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:59:01Z","timestamp":1760173141000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5729\/5\/1\/20"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,19]]},"references-count":72,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["data5010020"],"URL":"https:\/\/doi.org\/10.3390\/data5010020","relation":{},"ISSN":["2306-5729"],"issn-type":[{"type":"electronic","value":"2306-5729"}],"subject":[],"published":{"date-parts":[[2020,2,19]]}}}