{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:29:12Z","timestamp":1792787352407,"version":"4.5.0"},"publisher-location":"New York, NY, USA","reference-count":54,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,4,21]],"date-time":"2019-04-21T00:00:00Z","timestamp":1555804800000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"DARPA","award":["16-43-D3M-FP-040"],"award-info":[{"award-number":["16-43-D3M-FP-040"]}],"id":[{"id":"https:\/\/ror.org\/02caytj08","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Oracle","id":[{"id":"https:\/\/ror.org\/006c77m33","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Intel Science and Technology Center for Big Data"},{"name":"VMware","id":[{"id":"https:\/\/ror.org\/01mwmh458","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Mellanox"},{"name":"Microsoft","id":[{"id":"https:\/\/ror.org\/00d0nc645","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Adobe"},{"name":"Google","id":[{"id":"https:\/\/ror.org\/00njsd438","id-type":"ROR","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["IIS-1514491, IIS-1562657"],"award-info":[{"award-number":["IIS-1514491, IIS-1562657"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,4,21]]},"DOI":"10.1145\/3173574.3174053","type":"proceedings-article","created":{"date-parts":[[2018,4,20]],"date-time":"2018-04-20T11:38:35Z","timestamp":1524224315000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":76,"title":["Investigating the Effect of the Multiple Comparisons Problem in Visual Analysis"],"prefix":"10.1145","author":[{"given":"Emanuel","family":"Zgraggen","sequence":"first","affiliation":[{"name":"Brown University, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zheguang","family":"Zhao","sequence":"additional","affiliation":[{"name":"Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Zeleznik","sequence":"additional","affiliation":[{"name":"Brown University, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim","family":"Kraska","sequence":"additional","affiliation":[{"name":"Brown University&Massachusetts Institute of Technology, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,4,21]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/2068462.2068633"},{"key":"e_1_3_2_2_2_1","volume-title":"Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health and quality of life outcomes 12, 1","author":"Anthoine Emmanuelle","year":"2014","unstructured":"Emmanuelle Anthoine, Le\u00efla Moret, Antoine Regnault, V\u00e9ronique S\u00e9bille, and Jean-Benoit Hardouin. 2014. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health and quality of life outcomes 12, 1 (2014), 2."},{"key":"e_1_3_2_2_3_1","volume-title":"The hot hand fallacy and the gambler's fallacy: Two faces of subjective randomness? Memory&cognition 32, 8","author":"Ayton Peter","year":"2004","unstructured":"Peter Ayton and Ilan Fischer. 2004. The hot hand fallacy and the gambler's fallacy: Two faces of subjective randomness? Memory&cognition 32, 8 (2004), 1369--1378."},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/0001-6918(80)90046-3"},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/bimj.200900299"},{"key":"e_1_3_2_2_6_1","series-title":"Series B (Methodological)","volume-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society","author":"Benjamini Yoav","year":"1995","unstructured":"Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological) (1995), 289--300."},{"key":"e_1_3_2_2_7_1","volume-title":"Tsitsiklis","author":"Bertsekas Dimitri P.","year":"2002","unstructured":"Dimitri P. Bertsekas and John N. Tsitsiklis. 2002. Introduction to probability. Vol. 1. Athena Scientific Belmont, MA."},{"key":"e_1_3_2_2_8_1","volume-title":"Eppler","author":"Bresciani Sabrina","year":"2009","unstructured":"Sabrina Bresciani and Martin J. Eppler. 2009. The risks of visualization. Identit\u00e4t und Vielfalt der Kommunikations-wissenschaft (2009), 165--178."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2009.0120"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","unstructured":"Stuart K. Card Jock D. Mackinlay and Ben Shneiderman. 1999. Readings in information visualization: using vision to think. Morgan Kaufmann.","DOI":"10.5555\/300679"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/1669282.1669285"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1984.10478080"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","unstructured":"Michael Correll and Jeffrey Heer. 2017. Regression by Eye: Estimating Trends in Bivariate Visualizations. In ACM Human Factors in Computing Systems (CHI). http:\/\/idl.cs.washington.edu\/papers\/regression-by-eye 10.1145\/3025453.3025922","DOI":"10.1145\/3025453.3025922"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824127"},{"key":"e_1_3_2_2_15_1","volume-title":"Phillip Good: Permutation, parametric, and bootstrap tests of hypotheses.","author":"Droge Bernd","year":"2006","unstructured":"Bernd Droge. 2006. Phillip Good: Permutation, parametric, and bootstrap tests of hypotheses. (2006)."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1961.10482090"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","unstructured":"Cynthia Dwork Vitaly Feldman Moritz Hardt Toniann Pitassi Omer Reingold and Aaron Leon Roth. 2015. Preserving statistical validity in adaptive data analysis. In STOC. ACM 117--126. 10.1145\/2746539.2746580","DOI":"10.1145\/2746539.2746580"},{"key":"e_1_3_2_2_18_1","volume-title":"Protocol analysis","author":"Ericsson Karl Anders","unstructured":"Karl Anders Ericsson and Herbert Alexander Simon. 1993. Protocol analysis. MIT press Cambridge, MA."},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2007.00643.x"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691614551642"},{"key":"e_1_3_2_2_21_1","unstructured":"Andrew Gelman and Eric Loken. 2013. The garden of forking paths: Why multiple comparisons can be a problem even when there is no \"fishing expedition\" or \"p-hacking\" and the research hypothesis was posited ahead of time. Department of Statistics Columbia University (2013)."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s001800000040"},{"key":"e_1_3_2_2_23_1","volume-title":"Visual Analytics Science&Technology (VAST), 2014 IEEE Conference on. IEEE, 63--72","author":"Gomez Steven R.","unstructured":"Steven R. Gomez, Hua Guo, Caroline Ziemkiewicz, and David H. Laidlaw. 2014. An insight-and task-based methodology for evaluating spatiotemporal visual analytics. In Visual Analytics Science&Technology (VAST), 2014 IEEE Conference on. IEEE, 63--72."},{"key":"e_1_3_2_2_24_1","volume-title":"Laidlaw","author":"Gramazio Connor C.","year":"2014","unstructured":"Connor C. Gramazio, Karen B. Schloss, and David H. Laidlaw. 2014. The relation between visualization size, grouping, and user performance. IEEE transactions on visualization and computer graphics 20, 12 (2014), 1953--1962."},{"key":"e_1_3_2_2_25_1","volume-title":"Proceedings of the 23rd annual conference of the cognitive science society","author":"Thomas","unstructured":"Thomas L. Griffiths and Joshua B. Tenenbaum. 2001. Randomness and coincidences: Reconciling intuition and probability theory. In Proceedings of the 23rd annual conference of the cognitive science society. University of Edinburgh Edinburgh, 370--375."},{"key":"e_1_3_2_2_26_1","volume-title":"Laidlaw","author":"Guo Hua","year":"2016","unstructured":"Hua Guo, Steven R. Gomez, Caroline Ziemkiewicz, and David H. Laidlaw. 2016. A case study using visualization interaction logs and insight metrics to understand how analysts arrive at insights. IEEE transactions on visualization and computer graphics 22, 1 (2016), 51--60."},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213902"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.1002106"},{"key":"e_1_3_2_2_29_1","volume-title":"Hubscher and August Strindberg","author":"Sandra","year":"2007","unstructured":"Sandra L. Hubscher and August Strindberg. 2007. Apophenia: Definition and analysis. Digital Bits Skeptic (2007)."},{"key":"e_1_3_2_2_30_1","unstructured":"UC Irvine. 2017. UC Irvine Machine Learning Repository. (2017). http:\/\/archive.ics.uci.edu\/ml\/."},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.224"},{"key":"e_1_3_2_2_32_1","volume-title":"The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges. Behavioral and brain sciences 19, 1","author":"Koehler Jonathan J.","year":"1996","unstructured":"Jonathan J. Koehler. 1996. The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges. Behavioral and brain sciences 19, 1 (1996), 1--17."},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1559-1816.1996.tb01793.x"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1057\/ivs.2008.13"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346452"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2013.808157"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2006.70"},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/989863.989880"},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01694.x"},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2005.53"},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3025111.3025126"},{"key":"e_1_3_2_2_42_1","unstructured":"Tarique Siddiqui John Lee Albert Kim Edward Xue Xiaofo Yu Sean Zou Lijin Guo Changfeng Liu Chaoran Wang Karrie Karahalios and others. 2017. Fast-Forwarding to Desired Visualizations with Zenvisage. In CIDR."},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S1930297500000309","article-title":"Biases in casino betting: The hot hand and the gambler's fallacy","volume":"1","author":"Sundali James","year":"2006","unstructured":"James Sundali and Rachel Croson. 2006. Biases in casino betting: The hot hand and the gambler's fallacy. Judgment and Decision Making 1, 1 (2006), 1.","journal-title":"Judgment and Decision Making"},{"key":"e_1_3_2_2_44_1","unstructured":"Tableau. 2017. Tableau Product Description. (2017). https:\/\/www.tableau.com\/products\/desktop."},{"key":"e_1_3_2_2_45_1","unstructured":"TIBCO. 2017. TIBCO Spotfire Product Description. (2017). http:\/\/spotfire.tibco.com\/data-discovery."},{"key":"e_1_3_2_2_46_1","unstructured":"Christian Tominski. 2006. Event based visualization for user centered visual analysis. Ph.D. Dissertation."},{"key":"e_1_3_2_2_47_1","volume-title":"Exploratory data analysis","author":"Tukey John W.","year":"1977","unstructured":"John W. Tukey. 1977. Exploratory data analysis. Addison-Wesley Series in Behavioral Science: Quantitative Methods, Reading, Mass.: Addison-Wesley, 1977 (1977)."},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.80"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.14778\/2831360.2831371"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2010.161"},{"key":"e_1_3_2_2_51_1","volume-title":"Moments calculation for the doubly truncated multivariate normal density. arXiv preprint arXiv:1206.5387","author":"Wilhelm Stefan","year":"2012","unstructured":"Stefan Wilhelm and others. 2012. Moments calculation for the doubly truncated multivariate normal density. arXiv preprint arXiv:1206.5387 (2012)."},{"key":"e_1_3_2_2_52_1","volume-title":"How Progressive Visualizations Affect Exploratory Analysis","author":"Zgraggen Emanuel","year":"2016","unstructured":"Emanuel Zgraggen, Alex Galakatos, Andrew Crotty, Jean-Daniel Fekete, and Tim Kraska. 2016. How Progressive Visualizations Affect Exploratory Analysis. IEEE Transactions on Visualization and Computer Graphics (2016)."},{"key":"e_1_3_2_2_53_1","volume-title":"Drucker","author":"Zgraggen Emanuel","year":"2014","unstructured":"Emanuel Zgraggen, Robert Zeleznik, and Steven M. Drucker. 2014. Panoramicdata: Data analysis through pen&touch. IEEE transactions on visualization and computer graphics 20, 12 (2014), 2112--2121."},{"key":"e_1_3_2_2_54_1","volume-title":"Emanuel Zgraggen, Carsten Binnig, Eli Upfal, and Tim Kraska.","author":"Zhao Zheguang","year":"2016","unstructured":"Zheguang Zhao, Lorenzo De Stefani, Emanuel Zgraggen, Carsten Binnig, Eli Upfal, and Tim Kraska. 2016. Controlling False Discoveries During Interactive Data Exploration. arXiv preprint arXiv:1612.01040 (2016)."}],"event":{"name":"CHI '18: CHI Conference on Human Factors in Computing Systems","location":"Montreal QC Canada","acronym":"CHI '18","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3173574.3174053","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3173574.3174053","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3173574.3174053","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T14:47:00Z","timestamp":1790002020000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3173574.3174053"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,21]]},"references-count":54,"alternative-id":["10.1145\/3173574.3174053","10.1145\/3173574"],"URL":"https:\/\/doi.org\/10.1145\/3173574.3174053","relation":{},"subject":[],"published":{"date-parts":[[2018,4,21]]},"assertion":[{"value":"2018-04-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}