{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T21:04:11Z","timestamp":1772831051893,"version":"3.50.1"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,9,30]],"date-time":"2017-09-30T00:00:00Z","timestamp":1506729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"framework grant \u201cThe Digitized Society\u2014Past, Present, and Future\u201d"},{"DOI":"10.13039\/501100004359","name":"Swedish Research Council","doi-asserted-by":"crossref","award":["2012-5659"],"award-info":[{"award-number":["2012-5659"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2017,9,30]]},"abstract":"<jats:p>The automatic detection and classification of stance (e.g., certainty or agreement) in text data using natural language processing and machine-learning methods creates an opportunity to gain insight into the speakers\u2019 attitudes toward their own and other people\u2019s utterances. However, identifying stance in text presents many challenges related to training data collection and classifier training. To facilitate the entire process of training a stance classifier, we propose a visual analytics approach, called ALVA, for text data annotation and visualization. ALVA\u2019s interplay with the stance classifier follows an active learning strategy to select suitable candidate utterances for manual annotaion. Our approach supports annotation process management and provides the annotators with a clean user interface for labeling utterances with multiple stance categories. ALVA also contains a visualization method to help analysts of the annotation and training process gain a better understanding of the categories used by the annotators. The visualization uses a novel visual representation, called CatCombos, which groups individual annotation items by the combination of stance categories. Additionally, our system makes a visualization of a vector space model available that is itself based on utterances. ALVA is already being used by our domain experts in linguistics and computational linguistics to improve the understanding of stance phenomena and to build a stance classifier for applications such as social media monitoring.<\/jats:p>","DOI":"10.1145\/3132169","type":"journal-article","created":{"date-parts":[[2017,10,4]],"date-time":"2017-10-04T12:17:29Z","timestamp":1507119449000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Active Learning and Visual Analytics for Stance Classification with ALVA"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1907-7820","authenticated-orcid":false,"given":"Kostiantyn","family":"Kucher","sequence":"first","affiliation":[{"name":"Linnaeus University, V\u00e4xj\u00f6, Sweden"}]},{"given":"Carita","family":"Paradis","sequence":"additional","affiliation":[{"name":"Lund University, Lund, Sweden"}]},{"given":"Magnus","family":"Sahlgren","sequence":"additional","affiliation":[{"name":"Swedish Institute of Computer Science and Gavagai AB, Stockholm, Sweden"}]},{"given":"Andreas","family":"Kerren","sequence":"additional","affiliation":[{"name":"Linnaeus University, V\u00e4xj\u00f6, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2017,10,4]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1515\/text-2013-0031"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-034-R2"},{"key":"e_1_2_2_3_1","volume-title":"Eurographics 2013\u2014State of the Art Reports","author":"Borgo Rita"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441813"},{"key":"e_1_2_2_5_1","series-title":"Pragmatics 8 Beyond New Series","volume-title":"Stancetaking in Discourse: Subjectivity, Evaluation, Interaction, Robert Englebretson (Ed.)","author":"Du Bois John W."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12919"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.70535"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1075\/pbns.164"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.277"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2012.6400492"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2449396.2449400"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/WSC.2012.6465208"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2014.04.001"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/PACIFICVIS.2015.7156366"},{"key":"e_1_2_2_15_1","volume-title":"Poster Abstracts of the EG\/VGTC Conference on Visualization (EuroVis\u201916)","author":"Kucher Kostiantyn","year":"2016"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13217"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1177\/1473871615575079"},{"key":"e_1_2_2_18_1","volume-title":"Proceedings of the 2nd IEEE Workshop on Interactive Visual Text Analytics \u201cTask-Driven Analysis of Social Media\u201d (TextVis\u201912)","author":"Kuksenok Katie","year":"2012"},{"key":"e_1_2_2_19_1","volume-title":"Visualizing NLP annotations for crowdsourcing. CoRR abs\/1508.06044","author":"Li Hanchuan","year":"2015"},{"key":"e_1_2_2_20_1","volume-title":"Proceedings of the Eurographics Conference on Visualization (EuroVis)\u2014STARs (EuroVis\u201915)","author":"Liu Shusen","year":"2015"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2014.7042495"},{"key":"e_1_2_2_22_1","first-page":"2579","article-title":"Visualizing data using t-SNE","author":"van der Maaten Laurens","year":"2008","journal-title":"J. Mach. Learn. Res. 9"},{"key":"e_1_2_2_23_1","volume-title":"Proceedings of the 26th International Conference on Computational Linguistics\u2014System Demonstrations (COLING\u201916)","author":"Magnini Bernardo","year":"2016"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.5220\/0004745301780186"},{"key":"e_1_2_2_25_1","volume-title":"Manning and Hinrich Sch\u00fctze","author":"Christopher","year":"1999"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-08-100508-8.00009-6"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S16-1003"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3003433"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073427.1073436"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the Fourth SIGdial Workshop on Discourse and Dialogue (SIGdial\u201903)","author":"M\u00fcller Christoph","year":"2003"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2317187"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.3115\/1225785.1225791"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000011"},{"key":"e_1_2_2_34_1","volume-title":"Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12 (Oct.","author":"Pedregosa Fabian","year":"2011"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2014.11.005"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1110"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346249"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.153"},{"key":"e_1_2_2_39_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201911)","author":"Settles Burr","year":"2011"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.2200\/S00429ED1V01Y201207AIM018"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1177\/1473871614550537"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.166"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-66429-3_70"},{"key":"e_1_2_2_44_1","volume-title":"Poster Abstracts of the Symposium on Methods and Linguistic Theories (MaLT\u201915)","author":"Skeppstedt Maria","year":"2015"},{"key":"e_1_2_2_45_1","volume-title":"Proceedings of the Workshop on Computational Modeling of People\u2019s Opinions, Personality, and Emotions in Social Media at COLING 2016 (PEOPLES\u201916)","author":"Skeppstedt Maria","year":"2016"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2818"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-2923"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-66429-3_70"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-011-0349-2"},{"key":"e_1_2_2_50_1","volume-title":"Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL\u201912)","author":"Stenetorp Pontus","year":"2012"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/1642059.1642061"},{"key":"e_1_2_2_52_1","first-page":"45","article-title":"Support vector machine active learning with applications to text classification","author":"Tong Simon","year":"2001","journal-title":"J. Mach. Learn. Res. 2"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2468356.2479507"},{"key":"e_1_2_2_54_1","unstructured":"Edward R. Tufte. 2006. Beautiful Evidence. Graphics Press.  Edward R. Tufte. 2006. Beautiful Evidence. Graphics Press."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.5555\/1861751.1861756"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1057\/PALGRAVE.IVS.9500025"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.2001.0499"},{"key":"e_1_2_2_58_1","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics\u2014Conference System Demonstrations (ACL\u201913)","author":"Yimam Seid Muhie","year":"2013"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3132169","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3132169","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:13:56Z","timestamp":1750212836000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3132169"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,9,30]]},"references-count":58,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,9,30]]}},"alternative-id":["10.1145\/3132169"],"URL":"https:\/\/doi.org\/10.1145\/3132169","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,9,30]]},"assertion":[{"value":"2016-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-10-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}