{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T05:55:08Z","timestamp":1782280508930,"version":"3.54.5"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2024,4,17]],"date-time":"2024-04-17T00:00:00Z","timestamp":1713312000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006374","name":"NSF","doi-asserted-by":"publisher","award":["IIS 1941613, IIS 1527200"],"award-info":[{"award-number":["IIS 1941613, IIS 1527200"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2024,4,17]]},"abstract":"<jats:p>Causal belief is a cognitive practice that humans apply everyday to reason about cause and effect relations between factors, phenomena, or events. Like optical illusions, humans are prone to drawing causal relations between events that are only coincidental (i.e., causal illusions). Researchers in domains such as cognitive psychology and healthcare often use logistically expensive experiments to understand causal beliefs and illusions. In this paper, we propose Belief Miner, a crowdsourcing method for evaluating people's causal beliefs and illusions. Our method uses the (dis)similarities between the causal relations collected from the crowds and experts to surface the causal beliefs and illusions. Through an iterative design process, we developed a web-based interface for collecting causal relations from a target population. We then conducted a crowdsourced experiment with 101 workers on Amazon Mechanical Turk and Prolific using this interface and analyzed the collected data with Belief Miner. We discovered a variety of causal beliefs and potential illusions, and we report the design implications for future research.<\/jats:p>","DOI":"10.1145\/3637298","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T10:05:31Z","timestamp":1714385131000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Belief Miner: A Methodology for Discovering Causal Beliefs and Causal Illusions from General Populations"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-9214-450X","authenticated-orcid":false,"given":"Shahreen","family":"Salim","sequence":"first","affiliation":[{"name":"Stony Brook University, Stony Brook, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0878-501X","authenticated-orcid":false,"given":"Md Naimul","family":"Hoque","sequence":"additional","affiliation":[{"name":"University of Maryland, College Park, MD, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0996-8590","authenticated-orcid":false,"given":"Klaus","family":"Mueller","sequence":"additional","affiliation":[{"name":"Stony Brook University, Stony Brook, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,26]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-021-01665-8"},{"key":"e_1_2_2_2_1","volume-title":"Proc. 20th int. conf. very large data bases, VLDB","volume":"1215","author":"Agrawal Rakesh","year":"1994","unstructured":"Rakesh Agrawal, Ramakrishnan Srikant, et al . 1994. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, Vol. 1215. Santiago, Chile, 487--499."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03334492"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341161.3342958"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274293"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866078"},{"key":"e_1_2_2_7_1","volume-title":"Human-computer interaction and collective intelligence. Handbook of collective intelligence 57","author":"Bigham Jeffrey P","year":"2015","unstructured":"Jeffrey P Bigham, Michael S Bernstein, and Eytan Adar. 2015. Human-computer interaction and collective intelligence. Handbook of collective intelligence 57 (2015)."},{"key":"e_1_2_2_8_1","volume-title":"Vadillo","author":"Blanco Fernando","year":"2012","unstructured":"Fernando Blanco, Helena Matute, and Miguel A. Vadillo. 2012. Mediating role of activity level in the depressive realism effect. (2012)."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13420-013-0108-8"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.3182\/20120611-3-IE-4029.00033"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2011.185"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.3115\/1699510.1699548"},{"key":"e_1_2_2_13_1","volume-title":"Alternative medicine & the death of Steve Jobs. Psychology Today. October 21","author":"Cartwright MM","year":"2011","unstructured":"MM Cartwright. 2011. Alternative medicine & the death of Steve Jobs. Psychology Today. October 21 (2011)."},{"key":"e_1_2_2_14_1","volume-title":"Proceedings of the Workshop Events and Stories in the News","author":"Caselli Tommaso","year":"2018","unstructured":"Tommaso Caselli and Oana Inel. 2018. Crowdsourcing StoryLines: Harnessing the Crowd for Causal Relation Annotation. In Proceedings of the Workshop Events and Stories in the News 2018. Association for Computational Linguistics, Santa Fe, New Mexico, U.S.A, 44--54. https:\/\/aclanthology.org\/W18--4306"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026044"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445477"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030358"},{"key":"e_1_2_2_18_1","unstructured":"Breanne Chryst Jennifer Marlon Xinran Wang Sander van der Linden Edward Maibach Connie Roser-Renouf and Anthony Leiserowitz. [n. d.]. Six Americas Super Short Survey (SASSY!). https:\/\/climatecommunication.yale.edu\/visualizations-data\/sassy\/. (Accessed on 07\/10\/2023)."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359164"},{"key":"e_1_2_2_20_1","unstructured":"Climate.gov. [n. d.]. Climate Change: Atmospheric Carbon Dioxide. https:\/\/www.climate.gov\/news-features\/understanding-climate\/climate-change-atmospheric-carbon-dioxide. (Accessed on 07\/10\/2022)."},{"key":"e_1_2_2_21_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.2016.2643"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145355"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753688"},{"key":"e_1_2_2_25_1","unstructured":"Drought.gov. 2022. Wildfire Management. https:\/\/www.drought.gov\/sectors\/wildfire-management (Accessed: 2023-07--10)."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45848-4_57"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1111\/risa.13587"},{"key":"e_1_2_2_28_1","unstructured":"American Forests. 2022. Forests as Carbon Sinks. https:\/\/www.americanforests.org\/article\/forests-as-carbon-sinks\/ (Accessed: 2022-02-04)."},{"key":"e_1_2_2_29_1","first-page":"454","article-title":"Death by homeopathy: issues for civil, criminal and coronial law and for health service policy","volume":"19","author":"Freckelton Ian","year":"2012","unstructured":"Ian Freckelton. 2012. Death by homeopathy: issues for civil, criminal and coronial law and for health service policy. Journal of Law and medicine 19, 3 (2012), 454--478.","journal-title":"Journal of Law and medicine"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702443"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3209484"},{"key":"e_1_2_2_32_1","unstructured":"NASA Global Climate Change. 2021. Is the Sun causing global warming? https:\/\/climate.nasa.gov\/faq\/14\/is-the-sun-causing-global-warming\/ Accessed: 2022-06-04."},{"key":"e_1_2_2_33_1","volume-title":"NHMRC Information Paper: Evidence on the effectiveness of homeopathy for treating health conditions","author":"National Health and Medical Research Council. 2015.","unstructured":"National Health and Medical Research Council. 2015. NHMRC Information Paper: Evidence on the effectiveness of homeopathy for treating health conditions. National Health and Medical Research Council."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3102051"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807342.1807376"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837906"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820016"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3291009"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173670"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357127"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145357"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441923"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858115"},{"key":"e_1_2_2_45_1","volume-title":"Counterfactual Fairness. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Kusner Matt J.","year":"2017","unstructured":"Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4--9, 2017, Long Beach, CA, USA. 4066--4076. https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/a486cd07e4ac3d270571622f4f316ec5-Abstract.html"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v8i1.14494"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702594"},{"key":"e_1_2_2_48_1","volume-title":"Climate change in the American mind","author":"Leiserowitz Anthony","year":"2019","unstructured":"Anthony Leiserowitz, Edward W Maibach, Seth Rosenthal, John Kotcher, Parrish Bergquist, Matthew Ballew, Matthew Goldberg, and Abel Gustafson. 2019. Climate change in the American mind: April 2019. Yale University and George Mason University. New Haven, CT: Yale Program on Climate Change Communication (2019)."},{"key":"e_1_2_2_49_1","unstructured":"World Wild Life. [n. d.]. Why are glaciers and sea ice melting? https:\/\/www.worldwildlife.org\/pages\/why-are-glaciers-and-sea-ice-melting. (Accessed on 07\/10\/2023)."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173961"},{"key":"e_1_2_2_51_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_2_2_52_1","volume-title":"Mediation analysis. Annual review of psychology 58","author":"MacKinnon David P","year":"2007","unstructured":"David P MacKinnon, Amanda J Fairchild, and Matthew S Fritz. 2007. Mediation analysis. Annual review of psychology 58 (2007), 593."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173769"},{"key":"e_1_2_2_54_1","volume-title":"Illusions of causality: how they bias our everyday thinking and how they could be reduced. Frontiers in psychology 6","author":"Matute Helena","year":"2015","unstructured":"Helena Matute, Fernando Blanco, Ion Yarritu, Marcos D\u00edaz-Lago, Miguel A Vadillo, and Itxaso Barberia. 2015. Illusions of causality: how they bias our everyday thinking and how they could be reduced. Frontiers in psychology 6 (2015), 888."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3542954.3542995"},{"key":"e_1_2_2_56_1","volume-title":"Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)."},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047198"},{"key":"e_1_2_2_58_1","unstructured":"Nps.gov. 2022. Wildfire Causes and Evaluations. https:\/\/www.nps.gov\/articles\/wildfire-causes-and-evaluation. (Accessed: 2023-07--10)."},{"key":"e_1_2_2_59_1","unstructured":"World Meteorological Organization. [n. d.]. Heatwaves. https:\/\/www.britannica.com\/science\/heat-wave-meteorology. (Accessed on 07\/10\/2022)."},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.7189\/jogh.09.020318"},{"key":"e_1_2_2_61_1","unstructured":"Judea Pearl and Dana Mackenzie. 2018. The book of why: the new science of cause and effect. Basic books."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376403"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512930"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702469"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380125"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300292"},{"key":"e_1_2_2_68_1","volume-title":"Belief","author":"Schwitzgebel Eric","unstructured":"Eric Schwitzgebel. 2011. Belief. In The Routledge Companion to Epistemology. Routledge, 40--50."},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448139.3448205"},{"key":"e_1_2_2_70_1","volume-title":"Trick or treatment: The undeniable facts about alternative medicine","author":"Singh Simon","unstructured":"Simon Singh and Edzard Ernst. 2008. Trick or treatment: The undeniable facts about alternative medicine. WW Norton & Company."},{"key":"e_1_2_2_71_1","unstructured":"Psychology Today. 2022. Why Don't People Believe in Climate Change? https:\/\/www.psychologytoday.com\/us\/blog\/psych-unseen\/202204\/why-dont-people-believe-in-climate-change. (Accessed on 07\/10\/2022)."},{"key":"e_1_2_2_72_1","unstructured":"usability.gov. [n. d.]. System Usability Scale (SUS). https:\/\/www.usability.gov\/how-to-and-tools\/methods\/system-usability-scale.html. (Accessed on 07\/10\/2022)."},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.11144\/Javeriana.upsy12-1.ficu"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-010-0032-2"},{"key":"e_1_2_2_75_1","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society","volume":"26","author":"Walsh Clare R","year":"2004","unstructured":"Clare R Walsh and Steven A Sloman. 2004. Revising causal beliefs. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 26."},{"key":"e_1_2_2_76_1","volume-title":"The visual causality analyst: An interactive interface for causal reasoning","author":"Wang Jun","year":"2015","unstructured":"Jun Wang and Klaus Mueller. 2015. The visual causality analyst: An interactive interface for causal reasoning. IEEE transactions on visualization and computer graphics 22, 1 (2015), 230--239."},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998234"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3373464.3373475"},{"key":"e_1_2_2_79_1","doi-asserted-by":"crossref","unstructured":"Adrienne Yapo and Joseph Weiss. 2018. Ethical implications of bias in machine learning. (2018).","DOI":"10.24251\/HICSS.2018.668"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1111\/bjop.12119"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1027\/1618-3169\/a000225"},{"key":"e_1_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479588"},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581021"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174004"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637298","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3637298","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T17:16:44Z","timestamp":1755883004000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637298"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,17]]},"references-count":84,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2024,4,17]]}},"alternative-id":["10.1145\/3637298"],"URL":"https:\/\/doi.org\/10.1145\/3637298","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,17]]},"assertion":[{"value":"2024-04-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}