{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T15:07:38Z","timestamp":1770995258867,"version":"3.50.1"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2020,4,19]],"date-time":"2020-04-19T00:00:00Z","timestamp":1587254400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Nvidia Corporation through their Academic GPU"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Trans. Soc. Comput."],"published-print":{"date-parts":[[2020,6,30]]},"abstract":"<jats:p>\n            In recent years, streaming platforms for video games have seen increasingly large interest, as so-called esports have developed into a lucrative branch of business. Like for other sports, watching esports has become a new kind of entertainment medium, which is possible due to platforms that allow gamers to live stream their gameplay, the most popular platform being Twitch.tv. On these platforms, users can comment on streams in real time and thereby express their opinion about the events in the stream. Due to the popularity of Twitch.tv, this can be a valuable source of feedback for streamers aiming to improve their reception in a gaming-oriented audience. In this work, we explore the possibility of deriving feedback for video streams on Twitch.tv by analyzing the sentiment of live text comments made by stream viewers in highly active channels. Automatic sentiment analysis on these comments is a challenging task, as one can compare the language used in Twitch.tv with that used by an audience in a stadium, shouting as loud as possible in sometimes nonorganized ways. This language is very different from common English, mixing Internet slang and gaming-related language with abbreviations, intentional and unintentional grammatical and orthographic mistakes, and emoji-like images called\n            <jats:italic>emotes<\/jats:italic>\n            . Classic lexicon-based sentiment analysis techniques therefore fail when applied to Twitch comments.\n          <\/jats:p>\n          <jats:p>To overcome the challenge posed by the nonstandard language, we propose two unsupervised lexicon-based approaches that make heavy use of the information encoded in emotes, as well as a weakly supervised neural network\u2013based classifier trained on the lexicon-based outputs, which is supposed to help generalization to unknown words by use of domain-specific word embeddings. To enable better understanding of Twitch.tv comments, we analyze a large dataset of comments, uncovering specific properties of their language, and provide a smaller set of comments labeled with sentiment information by crowdsourcing.<\/jats:p>\n          <jats:p>We present two case studies showing the effectiveness of our methods in generating sentiment trajectories for events live streamed on Twitch.tv that correlate well with specific topics in the given stream. This allows for a new kind of implicit real-time feedback gathering for Twitch streamers and companies producing games or streaming content on Twitch.<\/jats:p>\n          <jats:p>\n            We make our datasets and code publicly available for further research.\n            <jats:sup>1<\/jats:sup>\n          <\/jats:p>","DOI":"10.1145\/3365523","type":"journal-article","created":{"date-parts":[[2020,5,4]],"date-time":"2020-05-04T08:09:04Z","timestamp":1588579744000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Emote-Controlled"],"prefix":"10.1145","volume":"3","author":[{"given":"Konstantin","family":"Kobs","sequence":"first","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]},{"given":"Albin","family":"Zehe","sequence":"additional","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]},{"given":"Armin","family":"Bernstetter","sequence":"additional","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]},{"given":"Julian","family":"Chibane","sequence":"additional","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]},{"given":"Jan","family":"Pfister","sequence":"additional","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]},{"given":"Julian","family":"Tritscher","sequence":"additional","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]},{"given":"Andreas","family":"Hotho","sequence":"additional","affiliation":[{"name":"Julius-Maximilians-University W\u00fcrzburg, W\u00fcrzburg, Germany"}]}],"member":"320","published-online":{"date-parts":[[2020,4,19]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Modern Information Retrieval","author":"Baeza-Yates Ricardo","unstructured":"Ricardo Baeza-Yates and Berthier Ribeiro-Neto . 1999. Modern Information Retrieval . Vol. 463 . ACM Press , New York, NY . Ricardo Baeza-Yates and Berthier Ribeiro-Neto. 1999. Modern Information Retrieval. Vol. 463. ACM Press, New York, NY."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-4402"},{"key":"e_1_2_2_3_1","volume-title":"Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis. 100--107","author":"Basile Valerio","year":"2013","unstructured":"Valerio Basile and Malvina Nissim . 2013 . Sentiment analysis on Italian tweets . In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis. 100--107 . Valerio Basile and Malvina Nissim. 2013. Sentiment analysis on Italian tweets. In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis. 100--107."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1298306.1298309"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/IWQOS.2008.32"},{"key":"e_1_2_2_7_1","volume-title":"SocialNLP@EMNLP, L.-W. Ku, J. Y. J. Hsu, and C.-T. Li (Eds.)","author":"Eisner Ben","year":"2016","unstructured":"Ben Eisner , Tim Rockt\u00e4schel , Isabelle Augenstein , Matko Bosnjak , and Sebastian Riedel . 2016. emoji2vec: Learning emoji representations from their description . In SocialNLP@EMNLP, L.-W. Ku, J. Y. J. Hsu, and C.-T. Li (Eds.) . Association for Computational Linguistics , Stroudsburg, PA , 48--54. http:\/\/dblp.uni-trier.de\/db\/conf\/acl-socialnlp\/acl-socialnlp 2016 .html#EisnerRABR16. Ben Eisner, Tim Rockt\u00e4schel, Isabelle Augenstein, Matko Bosnjak, and Sebastian Riedel. 2016. emoji2vec: Learning emoji representations from their description. In SocialNLP@EMNLP, L.-W. Ku, J. Y. J. Hsu, and C.-T. Li (Eds.). Association for Computational Linguistics, Stroudsburg, PA, 48--54. http:\/\/dblp.uni-trier.de\/db\/conf\/acl-socialnlp\/acl-socialnlp2016.html#EisnerRABR16."},{"key":"e_1_2_2_8_1","volume-title":"Measuring nominal scale agreement among many raters.Psychological Bulletin 76, 5","author":"Fleiss Joseph L.","year":"1971","unstructured":"Joseph L. Fleiss . 1971. Measuring nominal scale agreement among many raters.Psychological Bulletin 76, 5 ( 1971 ), 378. Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters.Psychological Bulletin 76, 5 (1971), 378."},{"key":"e_1_2_2_9_1","volume-title":"Proceedings of the 8th International Conference on Weblogs and Social Media (ICWSM-14)","author":"Hutto C. J.","year":"2014","unstructured":"C. J. Hutto and Eric Gilbert . 2014 . VADER: A parsimonious rule-based model for sentiment analysis of social media text . In Proceedings of the 8th International Conference on Weblogs and Social Media (ICWSM-14) . http:\/\/comp.social.gatech.edu\/papers\/icwsm14.vader.hutto.pdf. C. J. Hutto and Eric Gilbert. 2014. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the 8th International Conference on Weblogs and Social Media (ICWSM-14). http:\/\/comp.social.gatech.edu\/papers\/icwsm14.vader.hutto.pdf."},{"key":"e_1_2_2_10_1","unstructured":"Alec Go Richa Bhayani and Lei Huang. 2009. Twitter Sentiment Classification Using Distant Supervision. CS224N Project Report. Stanford.  Alec Go Richa Bhayani and Lei Huang. 2009. Twitter Sentiment Classification Using Distant Supervision. CS224N Project Report. Stanford."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-4322"},{"key":"e_1_2_2_12_1","volume-title":"Proceedings of the 16th International Conference on World Wide Web. ACM","author":"Martin","unstructured":"Martin J. Halvey and Mark T. Keane. 2007. Exploring social dynamics in online media sharing . In Proceedings of the 16th International Conference on World Wide Web. ACM , New York, NY, 1273--1274. Martin J. Halvey and Mark T. Keane. 2007. Exploring social dynamics in online media sharing. In Proceedings of the 16th International Conference on World Wide Web. ACM, New York, NY, 1273--1274."},{"key":"e_1_2_2_13_1","volume-title":"Information Retrieval: Computational and Theoretical Aspects","author":"Heaps H. S.","year":"1978","unstructured":"H. S. Heaps . 1978 . Information Retrieval: Computational and Theoretical Aspects . Academic Press , San Diego, CA . H. S. Heaps. 1978. Information Retrieval: Computational and Theoretical Aspects. Academic Press, San Diego, CA."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187980.2188259"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","first-page":"e0144296","DOI":"10.1371\/journal.pone.0144296","article-title":"Sentiment of emojis","volume":"10","author":"Novak Petra Kralj","year":"2015","unstructured":"Petra Kralj Novak , Jasmina Smailovi\u0107 , Borut Sluban , and Igor Mozeti\u010d . 2015 . Sentiment of emojis . PLoS ONE 10 , 12 (2015), e0144296 . http:\/\/dx.doi.org\/10.1371\/journal.pone.0144296. 10.1371\/journal.pone.0144296 Petra Kralj Novak, Jasmina Smailovi\u0107, Borut Sluban, and Igor Mozeti\u010d. 2015. Sentiment of emojis. PLoS ONE 10, 12 (2015), e0144296. http:\/\/dx.doi.org\/10.1371\/journal.pone.0144296.","journal-title":"PLoS ONE"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-1618"},{"key":"e_1_2_2_18_1","unstructured":"Ruokuang Lin Qianli D. Y. Ma and Chunhua Bian. 2014. Scaling laws in human speech decreasing emergence of new words and a generalized model. arxiv:cs.CL\/1412.4846.  Ruokuang Lin Qianli D. Y. Ma and Chunhua Bian. 2014. Scaling laws in human speech decreasing emergence of new words and a generalized model. arxiv:cs.CL\/1412.4846."},{"key":"e_1_2_2_19_1","series-title":"Lecture Notes in Computer Science","volume-title":"INTERACT","author":"L\u00f6ffler Diana","year":"2017","unstructured":"Diana L\u00f6ffler , Lennart Giron , and J\u00f6rn Hurtienne . 2017. Night mode, dark thoughts: Background color influences the perceived sentiment of chat messages . In INTERACT . Lecture Notes in Computer Science , Vol. 10514 . Springer , 184--201. http:\/\/dblp.uni-trier.de\/db\/conf\/interact\/interact 2017 -2.html#LofflerGH17. Diana L\u00f6ffler, Lennart Giron, and J\u00f6rn Hurtienne. 2017. Night mode, dark thoughts: Background color influences the perceived sentiment of chat messages. In INTERACT. Lecture Notes in Computer Science, Vol. 10514. Springer, 184--201. http:\/\/dblp.uni-trier.de\/db\/conf\/interact\/interact2017-2.html#LofflerGH17."},{"key":"e_1_2_2_20_1","volume-title":"Creativity and Universality in Language","author":"Loreto Vittorio","unstructured":"Vittorio Loreto , Vito D. P. Servedio , Steven H. Strogatz , and Francesca Tria . 2016. Dynamics on expanding spaces: Modeling the emergence of novelties . In Creativity and Universality in Language . Springer , 59--83. Vittorio Loreto, Vito D. P. Servedio, Steven H. Strogatz, and Francesca Tria. 2016. Dynamics on expanding spaces: Modeling the emergence of novelties. In Creativity and Universality in Language. Springer, 59--83."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_2_2_22_1","unstructured":"Julian J. McAuley Rahul Pandey and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. arXiv:1506.08839. http:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1506.html#McAuleyPL15.  Julian J. McAuley Rahul Pandey and Jure Leskovec. 2015. Inferring networks of substitutable and complementary products. arXiv:1506.08839. http:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1506.html#McAuleyPL15."},{"key":"e_1_2_2_23_1","unstructured":"Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv:1301.3781.  Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv:1301.3781."},{"key":"e_1_2_2_24_1","volume-title":"Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL\u201913)","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov , Wen-Tau Yih , and Geoffrey Zweig . 2013 b. Linguistic regularities in continuous space word representations . In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL\u201913) . 746--751. Tomas Mikolov, Wen-Tau Yih, and Geoffrey Zweig. 2013b. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL\u201913). 746--751."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S16-1001"},{"key":"e_1_2_2_26_1","volume-title":"Proceedings of the 2nd Joint Conference on Lexical and Computational Semantis (*SEM)","volume":"320","author":"Nakov Preslav","year":"2013","unstructured":"Preslav Nakov , Sara Rosenthal , Zornitsa Kozareva , Veselin Stoyanov , Alan Ritter , and Theresa Wilson . 2013 . SemEval-2013 task 2: Sentiment analysis in Twitter . In Proceedings of the 2nd Joint Conference on Lexical and Computational Semantis (*SEM) , Volume 2: Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval\u201913). 312-- 320 . Preslav Nakov, Sara Rosenthal, Zornitsa Kozareva, Veselin Stoyanov, Alan Ritter, and Theresa Wilson. 2013. SemEval-2013 task 2: Sentiment analysis in Twitter. In Proceedings of the 2nd Joint Conference on Lexical and Computational Semantis (*SEM), Volume 2: Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval\u201913). 312--320."},{"key":"e_1_2_2_27_1","volume-title":"Proceedings of the Workshop on Knowledge Discovery, Data Mining, and Machine Learning (KDML at LWA\u201912)","author":"Narr Sascha","year":"2012","unstructured":"Sascha Narr , Michael Hulfenhaus , and Sahin Albayrak . 2012 . Language-independent Twitter sentiment analysis . In Proceedings of the Workshop on Knowledge Discovery, Data Mining, and Machine Learning (KDML at LWA\u201912) . 12--14. Sascha Narr, Michael Hulfenhaus, and Sahin Albayrak. 2012. Language-independent Twitter sentiment analysis. In Proceedings of the Workshop on Knowledge Discovery, Data Mining, and Machine Learning (KDML at LWA\u201912). 12--14."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/LAWeb.2014.9"},{"key":"e_1_2_2_29_1","volume-title":"Diablo: Immortal Broke the Unspoken Rules of Blizzard, and BlizzCon. Retrieved","year":"2018","unstructured":"Polygon. 2018 . Diablo: Immortal Broke the Unspoken Rules of Blizzard, and BlizzCon. Retrieved March 7, 2020 from https:\/\/www.polygon.com\/2018\/11\/5\/18064290\/blizzard-diablo-immortal-reaction-explainer-blizzcon. Polygon. 2018. Diablo: Immortal Broke the Unspoken Rules of Blizzard, and BlizzCon. Retrieved March 7, 2020 from https:\/\/www.polygon.com\/2018\/11\/5\/18064290\/blizzard-diablo-immortal-reaction-explainer-blizzcon."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S17-2088"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S15-2078"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/S14-2009"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/42.3-4.425"},{"key":"e_1_2_2_34_1","volume-title":"Wright","author":"Smith Thomas","year":"2013","unstructured":"Thomas Smith , Marianna Obrist , and Peter C . Wright . 2013 . Live-streaming changes the (video) game. In EuroITV, P. Paolini, P. Cremonesi, and G. Lekakos (Eds.). ACM, New York, NY , 131--138. http:\/\/dblp.uni-trier.de\/db\/conf\/euroitv\/euroitv2013.html#SmithOW13. Thomas Smith, Marianna Obrist, and Peter C. Wright. 2013. Live-streaming changes the (video) game. In EuroITV, P. Paolini, P. Cremonesi, and G. Lekakos (Eds.). ACM, New York, NY, 131--138. http:\/\/dblp.uni-trier.de\/db\/conf\/euroitv\/euroitv2013.html#SmithOW13."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1146"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.3115\/992730.992783"},{"key":"e_1_2_2_37_1","first-page":"402","article-title":"II. A mathematical theory of evolution, based on the conclusions of Dr. JC Willis","volume":"213","author":"Yule G. Udny","year":"1925","unstructured":"G. Udny Yule . 1925 . II. A mathematical theory of evolution, based on the conclusions of Dr. JC Willis , FR S. Philosophical Transactions of the Royal Society of London: Series B 213 , 402 -- 410 (1925), 21--87. G. Udny Yule. 1925. II. A mathematical theory of evolution, based on the conclusions of Dr. JC Willis, FR S. Philosophical Transactions of the Royal Society of London: Series B 213, 402--410 (1925), 21--87.","journal-title":"FR S. Philosophical Transactions of the Royal Society of London: Series B"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736084.2736091"},{"key":"e_1_2_2_39_1","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 253--263","author":"Zhang Ye","year":"2017","unstructured":"Ye Zhang and Byron Wallace . 2017 . A sensitivity analysis of (and practitioners\u2019 guide to) convolutional neural networks for sentence classification . In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 253--263 . Ye Zhang and Byron Wallace. 2017. A sensitivity analysis of (and practitioners\u2019 guide to) convolutional neural networks for sentence classification. In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 253--263."}],"container-title":["ACM Transactions on Social Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3365523","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3365523","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:20Z","timestamp":1750203860000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3365523"}},"subtitle":["Obtaining Implicit Viewer Feedback Through Emote-Based Sentiment Analysis on Comments of Popular Twitch.tv Channels"],"short-title":[],"issued":{"date-parts":[[2020,4,19]]},"references-count":39,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,6,30]]}},"alternative-id":["10.1145\/3365523"],"URL":"https:\/\/doi.org\/10.1145\/3365523","relation":{},"ISSN":["2469-7818","2469-7826"],"issn-type":[{"value":"2469-7818","type":"print"},{"value":"2469-7826","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,19]]},"assertion":[{"value":"2018-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}