{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T09:26:21Z","timestamp":1762507581882,"version":"3.41.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW","license":[{"start":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T00:00:00Z","timestamp":1541030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1447634"],"award-info":[{"award-number":["IIS-1447634"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>Accurately and efficiently crowdsourcing complex, open-ended tasks can be difficult, as crowd participants tend to favor short, repetitive \"microtasks\". We study the crowdsourcing of large networks where the crowd provides the network topology via microtasks. Crowds can explore many types of social and information networks, but we focus on the network of causal attributions, an important network that signifies cause-and-effect relationships. We conduct experiments on Amazon Mechanical Turk (AMT) testing how workers can propose and validate individual causal relationships and introduce a method for independent crowd workers to explore large networks. The core of the method, Iterative Pathway Refinement, is a theoretically-principled mechanism for efficient exploration via microtasks. We evaluate the method using synthetic networks and apply it on AMT to extract a large-scale causal attribution network. Worker interactions reveal important characteristics of causal perception and the generated network data can help improve our understanding of causality and causal inference.<\/jats:p>","DOI":"10.1145\/3274293","type":"journal-article","created":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T21:21:27Z","timestamp":1541107287000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Efficient Crowd Exploration of Large Networks"],"prefix":"10.1145","volume":"2","author":[{"given":"Daniel","family":"Berenberg","sequence":"first","affiliation":[{"name":"University of Vermont, Burlington, VT, USA"}]},{"given":"James P.","family":"Bagrow","sequence":"additional","affiliation":[{"name":"University of Vermont, Burlington, VT, USA"}]}],"member":"320","published-online":{"date-parts":[[2018,11]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Search in power-law networks. Physical review E","author":"Adamic Lada A","year":"2001","unstructured":"Lada A Adamic, Rajan M Lukose, Amit R Puniyani, and Bernardo A Huberman. 2001. Search in power-law networks. Physical review E , Vol. 64, 4 (2001), 046135."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979214"},{"key":"e_1_2_1_3_1","volume-title":"Crowd ideation of supervised learning problems. arXiv preprint arXiv:1802.05101","author":"Bagrow James P","year":"2018","unstructured":"James P Bagrow. 2018. Crowd ideation of supervised learning problems. arXiv preprint arXiv:1802.05101 (2018)."},{"key":"e_1_2_1_4_1","volume-title":"Emergence of scaling in random networks. science","author":"Barab\u00e1si Albert-L\u00e1szl\u00f3","year":"1999","unstructured":"Albert-L\u00e1szl\u00f3 Barab\u00e1si and R\u00e9ka Albert. 1999. Emergence of scaling in random networks. science , Vol. 286, 5439 (1999), 509--512."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2791285"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0087756"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1002\/ejsp.2420180404"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCA.2012.2195168"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1354856507084420"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.socnet.2007.11.001"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(83)90006-9"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2986459.2986710"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702146"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466265"},{"key":"e_1_2_1_15_1","volume-title":"Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics","author":"Dawid Alexander Philip","year":"1979","unstructured":"Alexander Philip Dawid and Allan M Skene. 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied statistics (1979), 20--28."},{"volume-title":"Scaling concepts in polymer physics","author":"De Gennes Pierre-Gilles","key":"e_1_2_1_16_1","unstructured":"Pierre-Gilles De Gennes. 1979. Scaling concepts in polymer physics .Cornell university press."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741685"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/371920.372165"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.5486\/PMD.1959.6.3-4.12"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/0165551512437638"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/646815.708596"},{"key":"e_1_2_1_22_1","volume-title":"Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica: Journal of the Econometric Society","author":"Granger Clive W J","year":"1969","unstructured":"Clive W J Granger. 1969. Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica: Journal of the Econometric Society (1969), 424--438."},{"key":"e_1_2_1_23_1","volume-title":"Respondent-driven sampling: a new approach to the study of hidden populations. Social problems","author":"Heckathorn Douglas D","year":"1997","unstructured":"Douglas D Heckathorn. 1997. Respondent-driven sampling: a new approach to the study of hidden populations. Social problems , Vol. 44, 2 (1997), 174--199."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.71.016103"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.3773"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.107.1.65"},{"key":"e_1_2_1_27_1","volume-title":"The rise of crowdsourcing. Wired magazine","author":"Howe Jeff","year":"2006","unstructured":"Jeff Howe. 2006. The rise of crowdsourcing. Wired magazine , Vol. 14, 6 (2006), 1--4."},{"volume-title":"A Treatise of Human Nature","author":"Hume David","key":"e_1_2_1_28_1","unstructured":"David Hume. 2012. A Treatise of Human Nature .Courier Corporation."},{"key":"e_1_2_1_29_1","volume-title":"First AAAI Conference on Human Computation and Crowdsourcing .","author":"Jacques Jason T","year":"2013","unstructured":"Jason T Jacques and Per Ola Kristensson. 2013. Crowdsourcing a HIT: measuring workers' pre-task interactions on microtask markets. In First AAAI Conference on Human Computation and Crowdsourcing ."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2044-8295.1971.tb02040.x"},{"volume-title":"Critique of Pure Reason","author":"Kant Immanuel","key":"e_1_2_1_31_1","unstructured":"Immanuel Kant and Paul Guyer. 1998. Critique of Pure Reason .Cambridge University Press."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","unstructured":"David R Karger Sewoong Oh and Devavrat Shah. 2011. Iterative learning for reliable crowdsourcing systems. In Advances in neural information processing systems. 1953--1961.","DOI":"10.5555\/2986459.2986677"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.57.102904.190100"},{"key":"e_1_2_1_34_1","volume-title":"Nebraska symposium on motivation","author":"Kelley Harold H","year":"1967","unstructured":"Harold H Kelley. 1967. Attribution Theory in Social Psychology.. In Nebraska symposium on motivation. University of Nebraska Press."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2505612"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1869086.1869096"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357127"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_1_39_1","volume-title":"Nature","volume":"406","author":"Kleinberg Jon M","year":"2000","unstructured":"Jon M Kleinberg. 2000. Navigation in a small world. Nature , Vol. 406, 6798 (2000), 845."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2615731.2617437"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02178552"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2835776.2835797"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866040"},{"key":"e_1_2_1_44_1","volume-title":"et almbox","author":"Marbach Daniel","year":"2012","unstructured":"Daniel Marbach, James C Costello, Robert K\u00fcffner, Nicole M Vega, Robert J Prill, Diogo M Camacho, Kyle R Allison, Andrej Aderhold, Richard Bonneau, Yukun Chen, et almbox. 2012. Wisdom of crowds for robust gene network inference. Nature methods , Vol. 9, 8 (2012), 796."},{"key":"e_1_2_1_45_1","volume-title":"Bagrow","author":"McAndrew Thomas C.","year":"2017","unstructured":"Thomas C. McAndrew, Elizaveta Guseva, and James P. Bagrow. 2017. Reply & Supply: Efficient crowdsourcing when workers do more than answer questions. PLOS ONE , Vol. 12, 8 (2017), e69829. . 2001. A guide to first-passage processes .Cambridge University Press."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2012.12.017"},{"key":"e_1_2_1_47_1","volume-title":"Modeling, Decisions. J. Amer. Statist. Assoc.","author":"Rubin Donald B","year":"2011","unstructured":"Donald B Rubin. 2011. Causal Inference Using Potential Outcomes: Design, Modeling, Decisions. J. Amer. Statist. Assoc. (2011)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0123483"},{"key":"e_1_2_1_49_1","volume-title":"Perceptual causality and animacy. Trends in cognitive sciences","author":"Scholl Brian J","year":"2000","unstructured":"Brian J Scholl and Patrice D Tremoulet. 2000. Perceptual causality and animacy. Trends in cognitive sciences , Vol. 4, 8 (2000), 299--309."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675239"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/1613715.1613751"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0077095"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858108"},{"key":"e_1_2_1_54_1","first-page":"61","article-title":"The small world problem","volume":"1","author":"Travers Jeffrey","year":"1967","unstructured":"Jeffrey Travers and Stanley Milgram. 1967. The small world problem. Phychology Today , Vol. 1, 1 (1967), 61--67.","journal-title":"Phychology Today"},{"key":"e_1_2_1_55_1","volume-title":"Crowdsourcing Predictors of Residential Electric Energy Usage","author":"Wagy M. D.","year":"2017","unstructured":"M. D. Wagy, J. C. Bongard, J. P. Bagrow, and P. D. H. Hines. 2017. Crowdsourcing Predictors of Residential Electric Energy Usage. IEEE Systems Journal , Vol. PP, 99 (2017), 1--10."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btl038"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.71.016107"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274293","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3274293","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3274293","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:06Z","timestamp":1750207446000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274293"}},"subtitle":["The Case of Causal Attribution"],"short-title":[],"issued":{"date-parts":[[2018,11]]},"references-count":57,"journal-issue":{"issue":"CSCW","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.1145\/3274293"],"URL":"https:\/\/doi.org\/10.1145\/3274293","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2018,11]]},"assertion":[{"value":"2018-11-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}