{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T10:53:19Z","timestamp":1790765599084,"version":"4.1.0"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2022,3,30]],"date-time":"2022-03-30T00:00:00Z","timestamp":1648598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,3,30]]},"abstract":"<jats:p>Detecting anomalies in time-varying multivariate data is crucial in various industries for the predictive maintenance of equipment. Numerous machine learning (ML) algorithms have been proposed to support automated anomaly identification. However, a significant amount of human knowledge is still required to interpret, analyze, and calibrate the results of automated analysis. This paper investigates current practices used to detect and investigate anomalies in time series data in industrial contexts and identifies corresponding needs. Through iterative design and working with nine experts from two industry domains (aerospace and energy), we characterize six design elements required for a successful visualization system that supports effective detection, investigation, and annotation of time series anomalies. We summarize an ideal human-AI collaboration workflow that streamlines the process and supports efficient and collaborative analysis. We introduce MTV (MultivariateTime SeriesVisualization), a visual analytics system to support such workflow. The system incorporates a set of novel visualization and interaction designs to support multi-faceted time series exploration, efficient in-situ anomaly annotation, and insight communication. Two user studies, one with 6 spacecraft experts (with routine anomaly analysis tasks) and one with 25 general end-users (without such tasks), are conducted to demonstrate the effectiveness and usefulness of MTV.<\/jats:p>","DOI":"10.1145\/3512950","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T12:54:59Z","timestamp":1649336099000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":23,"title":["MTV: Visual Analytics for Detecting, Investigating, and Annotating Anomalies in Multivariate Time Series"],"prefix":"10.1145","volume":"6","author":[{"given":"Dongyu","family":"Liu","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Alnegheimish","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandra","family":"Zytek","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kalyan","family":"Veeramachaneni","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.04.070"},{"key":"e_1_2_1_2_1","volume-title":"Visualization of time-oriented data","author":"Aigner Wolfgang","unstructured":"Wolfgang Aigner , Silvia Miksch , Heidrun Schumann , and Christian Tominski . 2011. Visualization of time-oriented data . Springer Science & Business Media . Wolfgang Aigner, Silvia Miksch, Heidrun Schumann, and Christian Tominski. 2011. Visualization of time-oriented data .Springer Science & Business Media."},{"key":"e_1_2_1_3_1","unstructured":"Bilal Alsallakh Markus B\u00f6gl Theresia Gschwandtner Silvia Miksch Bilal Esmael Arghad Arnaout Gerhard Thonhauser and Philipp Z\u00f6llner. 2014. A visual analytics approach to segmenting and labeling multivariate time series data. In EuroVA@ EuroVis .  Bilal Alsallakh Markus B\u00f6gl Theresia Gschwandtner Silvia Miksch Bilal Esmael Arghad Arnaout Gerhard Thonhauser and Philipp Z\u00f6llner. 2014. A visual analytics approach to segmenting and labeling multivariate time series data. In EuroVA@ EuroVis ."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357311"},{"key":"e_1_2_1_5_1","first-page":"1","article-title":"Variational autoencoder based anomaly detection using reconstruction probability","volume":"2","author":"An Jinwon","year":"2015","unstructured":"Jinwon An and Sungzoon Cho . 2015 . Variational autoencoder based anomaly detection using reconstruction probability . Special Lecture on IE , Vol. 2 , 1 (2015), 1 -- 18 . Jinwon An and Sungzoon Cho. 2015. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE, Vol. 2, 1 (2015), 1--18.","journal-title":"Special Lecture on IE"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45681-3_2"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-017-9298-5"},{"key":"e_1_2_1_8_1","first-page":"298","article-title":"Comparing visual-interactive labeling with active learning: An experimental study","volume":"24","author":"Bernard J\u00fcrgen","year":"2017","unstructured":"J\u00fcrgen Bernard , Marco Hutter , Matthias Zeppelzauer , Dieter Fellner , and Michael Sedlmair . 2017 a. Comparing visual-interactive labeling with active learning: An experimental study . IEEE TVCG , Vol. 24 , 1 (2017), 298 -- 308 . J\u00fcrgen Bernard, Marco Hutter, Matthias Zeppelzauer, Dieter Fellner, and Michael Sedlmair. 2017a. Comparing visual-interactive labeling with active learning: An experimental study. IEEE TVCG, Vol. 24, 1 (2017), 298--308.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_9_1","volume-title":"Visual-interactive similarity search for complex objects by example of soccer player analysis. arXiv preprint arXiv:1703.03385","author":"Bernard J\u00fcrgen","year":"2017","unstructured":"J\u00fcrgen Bernard , Christian Ritter , David Sessler , Matthias Zeppelzauer , J\u00f6rn Kohlhammer , and Dieter Fellner . 2017b. Visual-interactive similarity search for complex objects by example of soccer player analysis. arXiv preprint arXiv:1703.03385 ( 2017 ). J\u00fcrgen Bernard, Christian Ritter, David Sessler, Matthias Zeppelzauer, J\u00f6rn Kohlhammer, and Dieter Fellner. 2017b. Visual-interactive similarity search for complex objects by example of soccer player analysis. arXiv preprint arXiv:1703.03385 (2017)."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-018-1500-3"},{"key":"e_1_2_1_11_1","volume-title":"KDD workshop","volume":"10","author":"Berndt Donald J","year":"1994","unstructured":"Donald J Berndt and James Clifford . 1994 . Using dynamic time warping to find patterns in time series . In KDD workshop , Vol. 10 . Seattle, WA, USA:, 359--370. Donald J Berndt and James Clifford. 1994. Using dynamic time warping to find patterns in time series. In KDD workshop, Vol. 10. Seattle, WA, USA:, 359--370."},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Susan E Brennan Klaus Mueller Greg Zelinsky IV Ramakrishnan David S Warren and Arie Kaufman. 2006. Toward a multi-analyst collaborative framework for visual analytics. In 2006 IEEE VAST. 129--136.  Susan E Brennan Klaus Mueller Greg Zelinsky IV Ramakrishnan David S Warren and Arie Kaufman. 2006. Toward a multi-analyst collaborative framework for visual analytics. In 2006 IEEE VAST. 129--136.","DOI":"10.1109\/VAST.2006.261439"},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Markus M Breunig Hans-Peter Kriegel Raymond T Ng and J\u00f6rg Sander. 2000. LOF: identifying density-based local outliers. In 2000 ACM SIGMOD. 93--104.  Markus M Breunig Hans-Peter Kriegel Raymond T Ng and J\u00f6rg Sander. 2000. LOF: identifying density-based local outliers. In 2000 ACM SIGMOD. 93--104.","DOI":"10.1145\/335191.335388"},{"key":"e_1_2_1_14_1","first-page":"23","article-title":"Voila: Visual anomaly detection and monitoring with streaming spatiotemporal data","volume":"24","author":"Cao Nan","year":"2017","unstructured":"Nan Cao , Chaoguang Lin , Qiuhan Zhu , Yu-Ru Lin , Xian Teng , and Xidao Wen . 2017 . Voila: Visual anomaly detection and monitoring with streaming spatiotemporal data . IEEE TVCG , Vol. 24 , 1 (2017), 23 -- 33 . Nan Cao, Chaoguang Lin, Qiuhan Zhu, Yu-Ru Lin, Xian Teng, and Xidao Wen. 2017. Voila: Visual anomaly detection and monitoring with streaming spatiotemporal data. IEEE TVCG, Vol. 24, 1 (2017), 23--33.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_15_1","first-page":"280","article-title":"TargetVue: Visual analysis of anomalous user behaviors in online communication systems","volume":"22","author":"Cao Nan","year":"2015","unstructured":"Nan Cao , Conglei Shi , Sabrina Lin , Jie Lu , Yu-Ru Lin , and Ching-Yung Lin . 2015 . TargetVue: Visual analysis of anomalous user behaviors in online communication systems . IEEE TVCG , Vol. 22 , 1 (2015), 280 -- 289 . Nan Cao, Conglei Shi, Sabrina Lin, Jie Lu, Yu-Ru Lin, and Ching-Yung Lin. 2015. TargetVue: Visual analysis of anomalous user behaviors in online communication systems. IEEE TVCG, Vol. 22, 1 (2015), 280--289.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1541880.1541882"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114836"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-017-9286-9"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466443"},{"key":"e_1_2_1_20_1","first-page":"235","article-title":"The 3c cooperation model applied to the classical requirement analysis","volume":"9","author":"Gava Vagner Luiz","year":"2012","unstructured":"Vagner Luiz Gava , Mauro de Mesquita Spinola , Antonio Carlos Tonini , and Jos\u00e9 Cardenas Medina . 2012 . The 3c cooperation model applied to the classical requirement analysis . JISTEM-Journal of Information Systems and Technology Management , Vol. 9 , 2 (2012), 235 -- 264 . Vagner Luiz Gava, Mauro de Mesquita Spinola, Antonio Carlos Tonini, and Jos\u00e9 Cardenas Medina. 2012. The 3c cooperation model applied to the classical requirement analysis. JISTEM-Journal of Information Systems and Technology Management, Vol. 9, 2 (2012), 235--264.","journal-title":"JISTEM-Journal of Information Systems and Technology Management"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData50022.2020.9378139"},{"key":"e_1_2_1_22_1","first-page":"523","article-title":"Comparing similarity perception in time series visualizations","volume":"25","author":"Gogolou Anna","year":"2018","unstructured":"Anna Gogolou , Theophanis Tsandilas , Themis Palpanas , and Anastasia Bezerianos . 2018 . Comparing similarity perception in time series visualizations . IEEE TVCG , Vol. 25 , 1 (2018), 523 -- 533 . Anna Gogolou, Theophanis Tsandilas, Themis Palpanas, and Anastasia Bezerianos. 2018. Comparing similarity perception in time series visualizations. IEEE TVCG, Vol. 25, 1 (2018), 523--533.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0152173"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820071"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijinfomgt.2018.08.006"},{"key":"e_1_2_1_26_1","first-page":"2189","article-title":"Supporting awareness through collaborative brushing and linking of tabular data","volume":"19","author":"Hajizadeh Amir Hossein","year":"2013","unstructured":"Amir Hossein Hajizadeh , Melanie Tory , and Rock Leung . 2013 . Supporting awareness through collaborative brushing and linking of tabular data . IEEE TVCG , Vol. 19 , 12 (2013), 2189 -- 2197 . Amir Hossein Hajizadeh, Melanie Tory, and Rock Leung. 2013. Supporting awareness through collaborative brushing and linking of tabular data. IEEE TVCG, Vol. 19, 12 (2013), 2189--2197.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_27_1","volume-title":"A Framework for Generating Explanations from Temporal Personal Health Data. arXiv preprint arXiv:2003.09530","author":"Harris Jonathan J","year":"2020","unstructured":"Jonathan J Harris , Ching-Hua Chen , and Mohammed J Zaki . 2020. A Framework for Generating Explanations from Temporal Personal Health Data. arXiv preprint arXiv:2003.09530 ( 2020 ). Jonathan J Harris, Ching-Hua Chen, and Mohammed J Zaki. 2020. A Framework for Generating Explanations from Temporal Personal Health Data. arXiv preprint arXiv:2003.09530 (2020)."},{"key":"e_1_2_1_28_1","volume-title":"Design considerations for collaborative visual analytics. Information visualization","author":"Heer Jeffrey","year":"2008","unstructured":"Jeffrey Heer and Maneesh Agrawala . 2008. Design considerations for collaborative visual analytics. Information visualization , Vol. 7 , 1 ( 2008 ), 49--62. Jeffrey Heer and Maneesh Agrawala. 2008. Design considerations for collaborative visual analytics. Information visualization, Vol. 7, 1 (2008), 49--62."},{"key":"e_1_2_1_29_1","volume-title":"Sheelagh Carpendale, Chris Weaver, and Petra Isenberg.","author":"Heer Jeffrey","year":"2008","unstructured":"Jeffrey Heer , Frank Van Ham , Sheelagh Carpendale, Chris Weaver, and Petra Isenberg. 2008 . Creation and collaboration: Engaging new audiences for information visualization. In Information visualization. Springer , 92--133. Jeffrey Heer, Frank Van Ham, Sheelagh Carpendale, Chris Weaver, and Petra Isenberg. 2008. Creation and collaboration: Engaging new audiences for information visualization. In Information visualization. Springer, 92--133."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240781"},{"key":"e_1_2_1_31_1","first-page":"2839","article-title":"Visual classifier training for text document retrieval","volume":"18","author":"Heimerl Florian","year":"2012","unstructured":"Florian Heimerl , Steffen Koch , Harald Bosch , and Thomas Ertl . 2012 . Visual classifier training for text document retrieval . IEEE TVCG , Vol. 18 , 12 (2012), 2839 -- 2848 . Florian Heimerl, Steffen Koch, Harald Bosch, and Thomas Ertl. 2012. Visual classifier training for text document retrieval. IEEE TVCG, Vol. 18, 12 (2012), 2839--2848.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_32_1","volume-title":"A survey of outlier detection methodologies. Artificial intelligence review","author":"Hodge Victoria","year":"2004","unstructured":"Victoria Hodge and Jim Austin . 2004. A survey of outlier detection methodologies. Artificial intelligence review , Vol. 22 , 2 ( 2004 ), 85--126. Victoria Hodge and Jim Austin. 2004. A survey of outlier detection methodologies. Artificial intelligence review, Vol. 22, 2 (2004), 85--126."},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Benjamin H\u00f6ferlin Rudolf Netzel Markus H\u00f6ferlin Daniel Weiskopf and Gunther Heidemann. 2012. Inter-active learning of ad-hoc classifiers for video visual analytics. In 2012 IEEE VAST. 23--32.  Benjamin H\u00f6ferlin Rudolf Netzel Markus H\u00f6ferlin Daniel Weiskopf and Gunther Heidemann. 2012. Inter-active learning of ad-hoc classifiers for video visual analytics. In 2012 IEEE VAST. 23--32.","DOI":"10.1109\/VAST.2012.6400492"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173640"},{"key":"e_1_2_1_35_1","volume-title":"Irina Smoke, Sangwha Sien, Janet Ng, Mark Zachry, and Juho Kim.","author":"Hong Sungsoo","year":"2019","unstructured":"Sungsoo Hong , Minhyang Suh , Tae Soo Kim , Irina Smoke, Sangwha Sien, Janet Ng, Mark Zachry, and Juho Kim. 2019 . Design for Collaborative Information-Seeking: Understanding User Challenges and Deploying Collaborative Dynamic Queries. Proceedings of the ACM on Human-Computer Interaction, Vol. 3 , CSCW ( 2019), 1--24. Sungsoo Hong, Minhyang Suh, Tae Soo Kim, Irina Smoke, Sangwha Sien, Janet Ng, Mark Zachry, and Juho Kim. 2019. Design for Collaborative Information-Seeking: Understanding User Challenges and Deploying Collaborative Dynamic Queries. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--24."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219845"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.70568"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/1473871611412817"},{"key":"e_1_2_1_39_1","volume-title":"Computer Graphics Forum","author":"Isenberg Petra","unstructured":"Petra Isenberg and Danyel Fisher . 2009. Collaborative brushing and linking for co-located visual analytics of document collections . In Computer Graphics Forum , Vol. 28 . Wiley Online Library , 1031--1038. Petra Isenberg and Danyel Fisher. 2009. Collaborative brushing and linking for co-located visual analytics of document collections. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 1031--1038."},{"key":"e_1_2_1_40_1","first-page":"689","article-title":"Co-located collaborative visual analytics around a tabletop display","volume":"18","author":"Isenberg Petra","year":"2011","unstructured":"Petra Isenberg , Danyel Fisher , Sharoda A Paul , Meredith Ringel Morris , Kori Inkpen , and Mary Czerwinski . 2011 b. Co-located collaborative visual analytics around a tabletop display . IEEE TVCG , Vol. 18 , 5 (2011), 689 -- 702 . Petra Isenberg, Danyel Fisher, Sharoda A Paul, Meredith Ringel Morris, Kori Inkpen, and Mary Czerwinski. 2011b. Co-located collaborative visual analytics around a tabletop display. IEEE TVCG, Vol. 18, 5 (2011), 689--702.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASSP.1975.1162641"},{"key":"e_1_2_1_42_1","first-page":"927","article-title":"Graphical perception of multiple time series","volume":"16","author":"Javed Waqas","year":"2010","unstructured":"Waqas Javed , Bryan McDonnel , and Niklas Elmqvist . 2010 . Graphical perception of multiple time series . IEEE TVCG , Vol. 16 , 6 (2010), 927 -- 934 . Waqas Javed, Bryan McDonnel, and Niklas Elmqvist. 2010. Graphical perception of multiple time series. IEEE TVCG, Vol. 16, 6 (2010), 927--934.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1133265.1133348"},{"key":"e_1_2_1_44_1","volume-title":"Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220","author":"Li Jiwei","year":"2016","unstructured":"Jiwei Li , Will Monroe , and Dan Jurafsky . 2016. Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220 ( 2016 ). Jiwei Li, Will Monroe, and Dan Jurafsky. 2016. Understanding neural networks through representation erasure. arXiv preprint arXiv:1612.08220 (2016)."},{"key":"e_1_2_1_45_1","first-page":"1","article-title":"Deeptracker: Visualizing the training process of convolutional neural networks","volume":"10","author":"Liu Dongyu","year":"2018","unstructured":"Dongyu Liu , Weiwei Cui , Kai Jin , Yuxiao Guo , and Huamin Qu . 2018 a. Deeptracker: Visualizing the training process of convolutional neural networks . ACM Transactions on Intelligent Systems and Technology (TIST) , Vol. 10 , 1 (2018), 1 -- 25 . Dongyu Liu, Weiwei Cui, Kai Jin, Yuxiao Guo, and Huamin Qu. 2018a. Deeptracker: Visualizing the training process of convolutional neural networks. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 10, 1 (2018), 1--25.","journal-title":"ACM Transactions on Intelligent Systems and Technology (TIST)"},{"key":"e_1_2_1_46_1","volume-title":"Dow","author":"Liu Weichen","year":"2018","unstructured":"Weichen Liu , Sijia Xiao , Jacob T. Browne , Ming Yang , and Steven P . Dow . 2018 b. ConsensUs: Supporting Multi-Criteria Group Decisions by Visualizing Points of Disagreement., Vol. 1 , 1 (2018). https:\/\/doi.org\/10.1145\/3159649 10.1145\/3159649 Weichen Liu, Sijia Xiao, Jacob T. Browne, Ming Yang, and Steven P. Dow. 2018b. ConsensUs: Supporting Multi-Criteria Group Decisions by Visualizing Points of Disagreement., Vol. 1, 1 (2018). https:\/\/doi.org\/10.1145\/3159649"},{"key":"e_1_2_1_47_1","volume-title":"Collaborative visualization for supporting the analysis of mobile device data. In 2015 ECSCW","author":"Ludwig Thomas","unstructured":"Thomas Ludwig , Tino Hilbert , and Volkmar Pipek . 2015. Collaborative visualization for supporting the analysis of mobile device data. In 2015 ECSCW . Springer , 305--316. Thomas Ludwig, Tino Hilbert, and Volkmar Pipek. 2015. Collaborative visualization for supporting the analysis of mobile device data. In 2015 ECSCW. Springer, 305--316."},{"key":"e_1_2_1_48_1","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"van der Maaten Laurens","year":"2008","unstructured":"Laurens van der Maaten and Geoffrey Hinton . 2008 . Visualizing data using t-SNE . Journal of machine learning research , Vol. 9 , Nov (2008), 2579 -- 2605 . Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, Nov (2008), 2579--2605.","journal-title":"Journal of machine learning research"},{"key":"e_1_2_1_49_1","first-page":"1633","article-title":"Supporting communication and coordination in collaborative sensemaking","volume":"20","author":"Mahyar Narges","year":"2014","unstructured":"Narges Mahyar and Melanie Tory . 2014 . Supporting communication and coordination in collaborative sensemaking . IEEE TVCG , Vol. 20 , 12 (2014), 1633 -- 1642 . Narges Mahyar and Melanie Tory. 2014. Supporting communication and coordination in collaborative sensemaking. IEEE TVCG, Vol. 20, 12 (2014), 1633--1642.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357286"},{"key":"e_1_2_1_51_1","first-page":"1","article-title":"Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision","volume":"4","author":"Miceli Milagros","year":"2020","unstructured":"Milagros Miceli , Martin Schuessler , and Tianling Yang . 2020 . Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision . Computer Supported Cooperative Work (CSCW) , Vol. 4 , 2 (2020), 1 -- 25 . Milagros Miceli, Martin Schuessler, and Tianling Yang. 2020. Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision. Computer Supported Cooperative Work (CSCW), Vol. 4, 2 (2020), 1--25.","journal-title":"Computer Supported Cooperative Work (CSCW)"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441910"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1294211.1294215"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/SCCC.2013.18"},{"key":"e_1_2_1_55_1","first-page":"1408","article-title":"Visual analysis of the air pollution problem in Hong Kong","volume":"13","author":"Qu Huamin","year":"2007","unstructured":"Huamin Qu , Wing-Yi Chan , Anbang Xu , Kai-Lun Chung , Kai-Hon Lau , and Ping Guo . 2007 . Visual analysis of the air pollution problem in Hong Kong . IEEE TVCG , Vol. 13 , 6 (2007), 1408 -- 1415 . Huamin Qu, Wing-Yi Chan, Anbang Xu, Kai-Lun Chung, Kai-Hon Lau, and Ping Guo. 2007. Visual analysis of the air pollution problem in Hong Kong. IEEE TVCG, Vol. 13, 6 (2007), 1408--1415.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASSP.1978.1163055"},{"key":"e_1_2_1_57_1","volume-title":"2016 IEEE Symposium on Visual Languages and Human-Centric Computing. 78--86","author":"Sarkar Advait","unstructured":"Advait Sarkar , Martin Spott , Alan F Blackwell , and Mateja Jamnik . [n.d.]. Visual discovery and model-driven explanation of time series patterns . In 2016 IEEE Symposium on Visual Languages and Human-Centric Computing. 78--86 . Advait Sarkar, Martin Spott, Alan F Blackwell, and Mateja Jamnik. [n.d.]. Visual discovery and model-driven explanation of time series patterns. In 2016 IEEE Symposium on Visual Languages and Human-Centric Computing. 78--86."},{"key":"e_1_2_1_58_1","volume-title":"Towards a rigorous evaluation of XAI Methods on Time Series. arXiv preprint arXiv:1909.07082","author":"Schlegel Udo","year":"2019","unstructured":"Udo Schlegel , Hiba Arnout , Mennatallah El-Assady , Daniela Oelke , and Daniel A Keim . 2019. Towards a rigorous evaluation of XAI Methods on Time Series. arXiv preprint arXiv:1909.07082 ( 2019 ). Udo Schlegel, Hiba Arnout, Mennatallah El-Assady, Daniela Oelke, and Daniel A Keim. 2019. Towards a rigorous evaluation of XAI Methods on Time Series. arXiv preprint arXiv:1909.07082 (2019)."},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1467--1478","author":"Settles Burr","year":"2011","unstructured":"Burr Settles . 2011 . Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances . In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1467--1478 . Burr Settles. 2011. Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1467--1478."},{"key":"e_1_2_1_60_1","volume-title":"Collaborative information seeking: The art and science of making the whole greater than the sum of all","author":"Shah Chirag","unstructured":"Chirag Shah . 2012. Collaborative information seeking: The art and science of making the whole greater than the sum of all . Vol. 34 . Springer Science & Business Media . Chirag Shah. 2012. Collaborative information seeking: The art and science of making the whole greater than the sum of all. Vol. 34. Springer Science & Business Media."},{"key":"e_1_2_1_61_1","volume-title":"The craft of information visualization","author":"Shneiderman Ben","unstructured":"Ben Shneiderman . 2003. The eyes have it: A task by data type taxonomy for information visualizations . In The craft of information visualization . Elsevier , 364--371. Ben Shneiderman. 2003. The eyes have it: A task by data type taxonomy for information visualizations. In The craft of information visualization. Elsevier, 364--371."},{"key":"e_1_2_1_62_1","volume-title":"James Max Kanter, and Kalyan Veeramachaneni","author":"Smith Micah J","year":"2020","unstructured":"Micah J Smith , Carles Sala , James Max Kanter, and Kalyan Veeramachaneni . 2020 . The machine learning bazaar: Harnessing the ML ecosystem for effective system development. In 2020 ACM SIGMOD. 785--800. Micah J Smith, Carles Sala, James Max Kanter, and Kalyan Veeramachaneni. 2020. The machine learning bazaar: Harnessing the ML ecosystem for effective system development. In 2020 ACM SIGMOD. 785--800."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10831-5_9"},{"key":"e_1_2_1_64_1","unstructured":"Jarke J Van Wijk and Edward R Van Selow. 1999. Cluster and calendar based visualization of time series data. In 1999 InfoVis. 4--9.  Jarke J Van Wijk and Edward R Van Selow. 1999. Cluster and calendar based visualization of time series data. In 1999 InfoVis. 4--9."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1147\/sj.454.0801"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.5121\/mlaij.2016.3103"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300831"},{"key":"e_1_2_1_68_1","unstructured":"Colin Ware. 2019. Information visualization: perception for design .Morgan Kaufmann.  Colin Ware. 2019. Information visualization: perception for design .Morgan Kaufmann."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFVIS.2001.963273"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979407"},{"key":"e_1_2_1_71_1","volume-title":"Yingjie Victor Chen, and Zhenyu Cheryl Qian","author":"Xia Jing","year":"2014","unstructured":"Jing Xia , Jieqiong Zhao , Isaac Sheeley , Joseph Christopher , Qiaoying Wang , Chen Guo , Jiawei Zhang , David S Ebert , Yingjie Victor Chen, and Zhenyu Cheryl Qian . 2014 . AnnotatedTimeTree: Visualization and annotation of news text and other heterogeneous document collections. In 2014 IEEE VAST. 337--338. Jing Xia, Jieqiong Zhao, Isaac Sheeley, Joseph Christopher, Qiaoying Wang, Chen Guo, Jiawei Zhang, David S Ebert, Yingjie Victor Chen, and Zhenyu Cheryl Qian. 2014. AnnotatedTimeTree: Visualization and annotation of news text and other heterogeneous document collections. In 2014 IEEE VAST. 337--338."},{"key":"e_1_2_1_72_1","first-page":"215","article-title":"A visual analytics framework for the detection of anomalous call stack trees in high performance computing applications","volume":"25","author":"Xie Cong","year":"2018","unstructured":"Cong Xie , Wei Xu , and Klaus Mueller . 2018 . A visual analytics framework for the detection of anomalous call stack trees in high performance computing applications . IEEE TVCG , Vol. 25 , 1 (2018), 215 -- 224 . Cong Xie, Wei Xu, and Klaus Mueller. 2018. A visual analytics framework for the detection of anomalous call stack trees in high performance computing applications. IEEE TVCG, Vol. 25, 1 (2018), 215--224.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174237"},{"key":"e_1_2_1_74_1","first-page":"1107","article-title":"CloudDet: Interactive Visual Analysis of Anomalous Performances in Cloud Computing Systems","volume":"26","author":"Xu Ke","year":"2019","unstructured":"Ke Xu , Yun Wang , Leni Yang , Yifang Wang , Bo Qiao , Si Qin , Yong Xu , Haidong Zhang , and Huamin Qu . 2019 . CloudDet: Interactive Visual Analysis of Anomalous Performances in Cloud Computing Systems . IEEE TVCG , Vol. 26 , 1 (2019), 1107 -- 1117 . Ke Xu, Yun Wang, Leni Yang, Yifang Wang, Bo Qiao, Si Qin, Yong Xu, Haidong Zhang, and Huamin Qu. 2019. CloudDet: Interactive Visual Analysis of Anomalous Performances in Cloud Computing Systems. IEEE TVCG, Vol. 26, 1 (2019), 1107--1117.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_75_1","first-page":"340","article-title":"Supporting handoff in asynchronous collaborative sensemaking using knowledge-transfer graphs","volume":"24","author":"Zhao Jian","year":"2017","unstructured":"Jian Zhao , Michael Glueck , Petra Isenberg , Fanny Chevalier , and Azam Khan . 2017 . Supporting handoff in asynchronous collaborative sensemaking using knowledge-transfer graphs . IEEE TVCG , Vol. 24 , 1 (2017), 340 -- 350 . Jian Zhao, Michael Glueck, Petra Isenberg, Fanny Chevalier, and Azam Khan. 2017. Supporting handoff in asynchronous collaborative sensemaking using knowledge-transfer graphs. IEEE TVCG, Vol. 24, 1 (2017), 340--350.","journal-title":"IEEE TVCG"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.03.029"},{"key":"e_1_2_1_77_1","doi-asserted-by":"crossref","unstructured":"Bin Zhou Shenghua Liu Bryan Hooi Xueqi Cheng and Jing Ye. 2019. BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series. In IJCAI. 4433--4439.  Bin Zhou Shenghua Liu Bryan Hooi Xueqi Cheng and Jing Ye. 2019. BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series. In IJCAI. 4433--4439.","DOI":"10.24963\/ijcai.2019\/616"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114864"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512950","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3512950","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T15:31:19Z","timestamp":1750174279000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512950"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,30]]},"references-count":78,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2022,3,30]]}},"alternative-id":["10.1145\/3512950"],"URL":"https:\/\/doi.org\/10.1145\/3512950","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,30]]},"assertion":[{"value":"2022-04-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}