{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T03:19:40Z","timestamp":1777778380590,"version":"3.51.4"},"reference-count":100,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2024,8,5]],"date-time":"2024-08-05T00:00:00Z","timestamp":1722816000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Information Visualization"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:p>Exploratory data analysis of high-dimensional datasets is a crucial task for which visual analytics can be especially useful. However, the ad hoc nature of exploratory analysis can also lead users to draw incorrect causal inferences. Previous studies have demonstrated this risk and shown that integrating counterfactual concepts within visual analytics systems can improve users\u2019 understanding of visualized data. However, effectively leveraging counterfactual concepts can be challenging, with only bespoke implementations found in prior work. Moreover, it can require expertise in both counterfactual subset analysis and visualization to implement the functionalities practically. This paper aims to help address these challenges in two ways. First, we propose an operator-based conceptual model for the use of counterfactuals that is informed by prior work in visualization research. Second, we contribute the Co-op library, an open and extensible reference implementation of this model that can support the integration of counterfactual-based subset computation with visualization systems. To evaluate the effectiveness and generalizability of Co-op, the library was used to construct two different visual analytics systems each supporting a distinct user workflow. In addition, expert interviews were conducted with professional visual analytics researchers and engineers to gain more insights regarding how Co-op could be leveraged. Finally, informed in part by these evaluation results, we distil a set of key design implications for effectively leveraging counterfactuals in future visualization systems.<\/jats:p>","DOI":"10.1177\/14738716241265120","type":"journal-article","created":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T00:34:13Z","timestamp":1722904453000},"page":"24-41","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":4,"title":["A framework to improve causal inferences from visualizations using counterfactual operators"],"prefix":"10.1177","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7491-7570","authenticated-orcid":false,"given":"Arran Zeyu","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Computer Science, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0162-4080","authenticated-orcid":false,"given":"David","family":"Borland","sequence":"additional","affiliation":[{"name":"RENCI, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6424-7374","authenticated-orcid":false,"given":"David","family":"Gotz","sequence":"additional","affiliation":[{"name":"School of Information and Library Science, The University of North Carolina at Chapel Hil, Chapel Hill, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2024,8,5]]},"reference":[{"key":"bibr1-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2016.59"},{"key":"bibr2-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030465"},{"key":"bibr3-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14823"},{"key":"bibr4-14738716241265120","volume-title":"Exploratory data analysis","author":"Tukey J"},{"key":"bibr5-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-34738-2"},{"key":"bibr6-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114779"},{"key":"bibr7-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1145\/3588319"},{"key":"bibr8-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2467191"},{"key":"bibr9-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934538"},{"key":"bibr10-14738716241265120","volume-title":"2021 ACM CHI Conference","author":"Zhou Z"},{"key":"bibr11-14738716241265120","volume-title":"2024 ACM CHI Conference","author":"Quadri G"},{"key":"bibr12-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1177\/1473871618821747"},{"key":"bibr13-14738716241265120","volume-title":"2016 Annual IEEE Systems Conference","author":"Oghbaie M"},{"key":"bibr14-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934399"},{"key":"bibr15-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13680"},{"key":"bibr16-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511803161"},{"key":"bibr17-14738716241265120","volume-title":"2017 NeurIPS Conference","author":"Kusner M"},{"key":"bibr18-14738716241265120","volume-title":"2021 ACL Conference","author":"Wu T"},{"key":"bibr19-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114824"},{"key":"bibr20-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030342"},{"key":"bibr21-14738716241265120","volume-title":"2020 ACM IUI Conference","author":"Gomez O"},{"key":"bibr22-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1177\/14738716241229437"},{"key":"bibr23-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2023.3338788"},{"key":"bibr24-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1996.10476902"},{"key":"bibr25-14738716241265120","volume-title":"2016 ICML Conference","author":"Johansson F"},{"key":"bibr26-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1214\/09-STS313"},{"key":"bibr27-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-6419.2007.00527.x"},{"key":"bibr28-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1093\/pan\/mpr013"},{"key":"bibr29-14738716241265120","volume-title":"Causal inference in statistics: a primer","author":"Glymour M","year":"2016"},{"key":"bibr30-14738716241265120","volume-title":"2007 Smart Graphics Symposium","author":"Zuk T"},{"key":"bibr31-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2467931"},{"key":"bibr32-14738716241265120","volume-title":"2017 IEEE VAST Conference","author":"Wang J"},{"key":"bibr33-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3102051"},{"key":"bibr34-14738716241265120","volume-title":"2023 SIGCHI Conference on Human Factors in Computing Systems","author":"Guo G"},{"key":"bibr35-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3028957"},{"key":"bibr36-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114875"},{"key":"bibr37-14738716241265120","author":"Hotz I","journal-title":"Foundations of data visualization"},{"key":"bibr38-14738716241265120","volume-title":"A mathematical modeling approach from nonlinear dynamics to complex systems","author":"Rodrigues FA."},{"key":"bibr39-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1086\/229694"},{"key":"bibr40-14738716241265120","volume-title":"2005 IEEE Symposium on Information Visualization","author":"Wilkinson L"},{"key":"bibr41-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934796"},{"key":"bibr42-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1002\/wics.101"},{"issue":"11","key":"bibr43-14738716241265120","volume":"9","author":"Van der Maaten L","year":"2008","journal-title":"J Mach Learn Res"},{"key":"bibr44-14738716241265120","volume-title":"2019 ACM CHI Conference","author":"Kim Y"},{"key":"bibr45-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1111\/rssa.12378"},{"key":"bibr46-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.132"},{"key":"bibr47-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346325"},{"key":"bibr48-14738716241265120","volume-title":"2022 ACM CHI Conference","author":"Wu A"},{"key":"bibr49-14738716241265120","volume-title":"1994 ICML Conference","author":"John G"},{"key":"bibr50-14738716241265120","volume-title":"2021 AAAI Conference","author":"Killamsetty K"},{"key":"bibr51-14738716241265120","volume-title":"2019 IEEE WACV Conference","author":"Kaushal V"},{"key":"bibr52-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009726021843"},{"key":"bibr53-14738716241265120","volume-title":"1996 IEEE Symposium on Visual Languages","author":"Shneiderman B"},{"key":"bibr54-14738716241265120","volume-title":"2016 ACM IUI Conference","author":"Gotz D"},{"issue":"1","key":"bibr55-14738716241265120","first-page":"429","volume":"26","author":"Borland D","year":"2020","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr56-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2018.2874782"},{"key":"bibr57-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030455"},{"issue":"1","key":"bibr58-14738716241265120","first-page":"440","volume":"26","author":"Gotz D","year":"2020","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr59-14738716241265120","volume-title":"2019 ACM IUI Conference","author":"Lee D"},{"key":"bibr60-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1177\/1473871614526077"},{"key":"bibr61-14738716241265120","volume-title":"2017 ACM SIGMOD Conference","author":"Tang B"},{"issue":"3","key":"bibr62-14738716241265120","first-page":"3","author":"Hullman J","year":"2021","journal-title":"Harv Data Sci Rev"},{"issue":"1","key":"bibr63-14738716241265120","first-page":"56","volume":"26","author":"Wexler J","year":"2020","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"bibr64-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1177\/14738716231216030"},{"key":"bibr65-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-014-9522-9"},{"key":"bibr66-14738716241265120","volume-title":"2016 SIGMOD Conference","author":"Krishnan S"},{"key":"bibr67-14738716241265120","volume-title":"2016 SIGMOD Conference","author":"Chu X"},{"key":"bibr68-14738716241265120","volume-title":"2020 ACM RecSys Conference","author":"Dong Z"},{"key":"bibr69-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2865240"},{"key":"bibr70-14738716241265120","volume-title":"2009 ACM IUI Conference","author":"Gotz D"},{"key":"bibr71-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3209445"},{"key":"bibr72-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2017.3271463"},{"issue":"1","key":"bibr73-14738716241265120","first-page":"61","volume":"5","author":"Han D","year":"2021","journal-title":"Vis Inf"},{"key":"bibr74-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13178"},{"key":"bibr75-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1145\/2699696"},{"key":"bibr76-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btx398"},{"key":"bibr77-14738716241265120","volume-title":"In 2017 LearningSys Workshop at NeurIPS Conference","author":"Nishino R"},{"key":"bibr78-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1016\/0146-664X(80)90054-4"},{"issue":"3","key":"bibr79-14738716241265120","volume":"17","author":"Virtanen P","year":"2020","journal-title":"Nat Methods"},{"key":"bibr80-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1950.tb00463.x"},{"key":"bibr81-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7439(99)00047-7"},{"key":"bibr82-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"bibr83-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1214\/aop\/1176996454"},{"key":"bibr84-14738716241265120","volume-title":"2018 ICML Conference","author":"Wu H"},{"key":"bibr85-14738716241265120","volume-title":"1996 IEEE Symposium on Information Visualization","author":"Chuah M"},{"key":"bibr86-14738716241265120","volume-title":"1998 IEEE Symposium on Information Visualization","author":"Chi E"},{"key":"bibr87-14738716241265120","volume-title":"The theory of groups","author":"Hall M","year":"2018"},{"key":"bibr88-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2649-2"},{"issue":"9","key":"bibr89-14738716241265120","first-page":"1","volume-title":"PyHPC Workshop","volume":"14","author":"McKinney W","year":"2011"},{"key":"bibr90-14738716241265120","volume-title":"2017 ACM CHI Conference","author":"Wongsuphasawat K"},{"key":"bibr91-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346452"},{"key":"bibr92-14738716241265120","volume-title":"2023 IEEE VIS Poster Proceedings","author":"Wang A"},{"key":"bibr93-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(00)02442-9"},{"key":"bibr94-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1005510"},{"key":"bibr95-14738716241265120","first-page":"841","volume":"31","author":"Wachter S","year":"2017","journal-title":"Harv J Law Technol"},{"key":"bibr96-14738716241265120","volume-title":"2012 AAAI Conference","author":"Gibson R"},{"key":"bibr97-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1016\/j.eneco.2015.12.009"},{"key":"bibr98-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.70515"},{"key":"bibr99-14738716241265120","doi-asserted-by":"publisher","DOI":"10.1057\/ivs.2008.31"},{"key":"bibr100-14738716241265120","volume-title":"2006 Intelligence Analysis Workshop in ACM IUI Conference","author":"Gotz D"}],"container-title":["Information Visualization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14738716241265120","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/14738716241265120","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/14738716241265120","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T19:19:22Z","timestamp":1777490362000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/14738716241265120"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,5]]},"references-count":100,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10.1177\/14738716241265120"],"URL":"https:\/\/doi.org\/10.1177\/14738716241265120","relation":{},"ISSN":["1473-8716","1473-8724"],"issn-type":[{"value":"1473-8716","type":"print"},{"value":"1473-8724","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,5]]}}}