{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T19:00:06Z","timestamp":1776106806825,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":72,"publisher":"ACM","license":[{"start":{"date-parts":[[2025,4,25]],"date-time":"2025-04-25T00:00:00Z","timestamp":1745539200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,4,26]]},"DOI":"10.1145\/3706599.3716230","type":"proceedings-article","created":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T21:02:04Z","timestamp":1745442124000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3445-3414","authenticated-orcid":false,"given":"Anamaria","family":"Crisan","sequence":"first","affiliation":[{"name":"Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8255-4258","authenticated-orcid":false,"given":"Andrew M","family":"McNutt","sequence":"additional","affiliation":[{"name":"Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,4,25]]},"reference":[{"key":"e_1_3_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300760"},{"key":"e_1_3_3_2_3_2","doi-asserted-by":"publisher","unstructured":"Mohammad Allahbakhsh Boualem Benatallah Aleksandar Ignjatovic Hamid\u00a0Reza Motahari-Nezhad Elisa Bertino and Schahram Dustdar. 2013. Quality Control in Crowdsourcing Systems: Issues and Directions. IEEE Internet Computing 17 2 (2013) 76\u201381. 10.1109\/MIC.2013.20","DOI":"10.1109\/MIC.2013.20"},{"key":"e_1_3_3_2_4_2","unstructured":"AntV. 2023. Advisor.lint. ava.antv.antgroup.com\/en\/api\/advice\/advisor-lint."},{"key":"e_1_3_3_2_5_2","doi-asserted-by":"publisher","unstructured":"Daniel\u00a0W Barowy Emery\u00a0D Berger and Benjamin Zorn. 2018. ExceLint: automatically finding spreadsheet formula errors. Proceedings of the ACM on Programming Languages 2 OOPSLA (2018) 1\u201326. 10.1145\/3276518","DOI":"10.1145\/3276518"},{"key":"e_1_3_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866078"},{"key":"e_1_3_3_2_7_2","doi-asserted-by":"publisher","unstructured":"R. Borgo L. Micallef B. Bach F. McGee and B. Lee. 2018. Information Visualization Evaluation Using Crowdsourcing. Computer Graphics Forum 37 3 (2018) 573\u2013595. 10.1111\/cgf.13444","DOI":"10.1111\/cgf.13444"},{"key":"e_1_3_3_2_8_2","doi-asserted-by":"publisher","unstructured":"David Borland and Russell M\u00a0Taylor Ii. 2007. Rainbow color map (still) considered harmful. IEEE computer graphics and applications 27 2 (2007) 14\u201317. 10.1109\/MCG.2007.46","DOI":"10.1109\/MCG.2007.46"},{"key":"e_1_3_3_2_9_2","unstructured":"John Carpenter. 1988. They Live. Universal Pictures."},{"key":"e_1_3_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026044"},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"publisher","unstructured":"Qing Chen Fuling Sun Xinyue Xu Zui Chen Jiazhe Wang and Nan Cao. 2021. Vizlinter: A linter and fixer framework for data visualization. Transactions on Visualization and Computer Graphics 28 1 (2021) 206\u2013216. 10.1109\/TVCG.2021.3114804","DOI":"10.1109\/TVCG.2021.3114804"},{"key":"e_1_3_3_2_12_2","volume-title":"Casual creators: Defining a genre of autotelic creativity support systems","author":"Compton Katherine","year":"2019","unstructured":"Katherine Compton. 2019. Casual creators: Defining a genre of autotelic creativity support systems. University of California, Santa Cruz. https:\/\/www.escholarship.org\/uc\/item\/4kg8g9gd"},{"key":"e_1_3_3_2_13_2","unstructured":"Michael Correll. 2021. The Clippy-ization of Human-Computer Design. Medium (2021). https:\/\/mcorrell.medium.com\/the-clippy-ization-of-human-computer-design-c66e8042de88"},{"key":"e_1_3_3_2_14_2","volume-title":"DECISIVe : Workshop on Dealing with Cognitive Biases in Visualisations","author":"Correll Michael","year":"2014","unstructured":"Michael Correll and Michael Gleicher. 2014. Bad for Data, Good for the Brain : Knowledge-First Axioms For Visualization Design. In DECISIVe : Workshop on Dealing with Cognitive Biases in Visualisations. http:\/\/graphics.cs.wisc.edu\/Papers\/2014\/CG14a"},{"key":"e_1_3_3_2_15_2","unstructured":"Alan Cruickshank. 2023. Rolling out SQLFluff with a new team. docs.sqlfluff.com\/en\/stable\/teamrollout.html."},{"key":"e_1_3_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.2312\/EUROVISSHORT.20181079"},{"key":"e_1_3_3_2_17_2","doi-asserted-by":"publisher","unstructured":"Chiara\u00a0Patricia Drolsbach Kirill Solovev and Nicolas Pr\u00f6llochs. 2024. Community notes increase trust in fact-checking on social media. PNAS Nexus (2024) pgae217. 10.1093\/pnasnexus\/pgae217","DOI":"10.1093\/pnasnexus\/pgae217"},{"key":"e_1_3_3_2_18_2","unstructured":"Eleventy. 2024. Inclusive Language Plugin. https:\/\/www.11ty.dev\/docs\/plugins\/inclusive-language\/."},{"key":"e_1_3_3_2_19_2","doi-asserted-by":"publisher","unstructured":"Hamza Elhamdadi Adam Stefkovics Johanna Beyer Eric Moerth Hanspeter Pfister Cindy\u00a0Xiong Bearfield and Carolina Nobre. 2023. Vistrust: a multidimensional framework and empirical study of trust in data visualizations. IEEE Transactions on Visualization and Computer Graphics (2023). 10.1109\/TVCG.2023.3326579","DOI":"10.1109\/TVCG.2023.3326579"},{"key":"e_1_3_3_2_20_2","unstructured":"Brian Ford. 2019. Write Good: Naive linter for English prose. github.com\/btford\/write-good."},{"key":"e_1_3_3_2_21_2","doi-asserted-by":"publisher","unstructured":"Robert Gorwa Reuben Binns and Christian Katzenbach. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society 7 1 (2020) 2053951719897945. 10.1177\/2053951719897945","DOI":"10.1177\/2053951719897945"},{"key":"e_1_3_3_2_22_2","doi-asserted-by":"publisher","unstructured":"Madeleine Grunde-McLaughlin Michelle\u00a0S Lam Ranjay Krishna Daniel\u00a0S Weld and Jeffrey Heer. 2023. Designing LLM chains by adapting techniques from crowdsourcing workflows. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2312.11681 (2023). 10.48550\/arXiv.2312.11681","DOI":"10.48550\/arXiv.2312.11681"},{"key":"e_1_3_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376804"},{"key":"e_1_3_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470744"},{"key":"e_1_3_3_2_25_2","doi-asserted-by":"publisher","unstructured":"Jeffrey Heer. 2019. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences 116 6 (2019) 1844\u20131850. 10.1073\/pnas.1807184115","DOI":"10.1073\/pnas.1807184115"},{"key":"e_1_3_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753357"},{"key":"e_1_3_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13975"},{"key":"e_1_3_3_2_28_2","doi-asserted-by":"publisher","unstructured":"Matthew Hull Vivian Pednekar Hannah Murray Nimisha Roy Emmanuel Tung Susanta Routray Connor Guerin Justin Chen Zijie\u00a0J. Wang Seongmin Lee Mahdi Roozbahani and Duen\u00a0Horng Chau. 2023. VISGRADER: Automatic Grading of D3 Visualizations. Transactions on Visualization and Computer Graphics (2023). 10.1109\/TVCG.2023.3327181","DOI":"10.1109\/TVCG.2023.3327181"},{"key":"e_1_3_3_2_29_2","unstructured":"Peter Hyams. 1994. TimeCop. Universal Pictures."},{"key":"e_1_3_3_2_30_2","volume-title":"NIPS MLSys Workshop","author":"Hynes Nick","year":"2017","unstructured":"Nick Hynes, D Sculley, and Michael Terry. 2017. Data linter: Lightweight, automated sanity checking for ml data sets. In NIPS MLSys Workshop."},{"key":"e_1_3_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702564"},{"key":"e_1_3_3_2_32_2","doi-asserted-by":"publisher","unstructured":"Alex Kale Ziyang Guo Xiao\u00a0Li Qiao Jeffrey Heer and Jessica Hullman. 2023. EVM: Incorporating model checking into exploratory visual analysis. Transactions on Visualization and Computer Graphics (2023). 10.1109\/TVCG.2023.3326516","DOI":"10.1109\/TVCG.2023.3326516"},{"key":"e_1_3_3_2_33_2","doi-asserted-by":"publisher","unstructured":"Uku Kangur Roshni Chakraborty and Rajesh Sharma. 2024. Who Checks the Checkers? Exploring Source Credibility in Twitter\u2019s Community Notes. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.12444 (2024). 10.48550\/arXiv.2406.12444","DOI":"10.48550\/arXiv.2406.12444"},{"key":"e_1_3_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587102.3588777"},{"key":"e_1_3_3_2_35_2","unstructured":"Know Your Meme. 2022. Alignment Charts - Salad Alignment Chart. https:\/\/knowyourmeme.com\/photos\/2382569-alignment-charts Online meme database entry."},{"key":"e_1_3_3_2_36_2","doi-asserted-by":"publisher","unstructured":"Robert Kosara. 2024. Business Data Visualization Beyond the Boring. IEEE Computer Graphics and Applications 44 05 (Sept. 2024) 153\u2013158. 10.1109\/MCG.2024.3407489","DOI":"10.1109\/MCG.2024.3407489"},{"key":"e_1_3_3_2_37_2","first-page":"411","volume-title":"Research Methods in Human Computer Interaction (Second Edition) (second edition ed.)","author":"Lazar Jonathan","year":"2017","unstructured":"Jonathan Lazar, Jinjuan\u00a0Heidi Feng, and Harry Hochheiser. 2017. Chapter 14 - Online and ubiquitous HCI research. In Research Methods in Human Computer Interaction (Second Edition) (second edition ed.). Morgan Kaufmann, Boston, 411\u2013453."},{"key":"e_1_3_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-805390-4.00014-5"},{"key":"e_1_3_3_2_39_2","doi-asserted-by":"publisher","unstructured":"Elsie Lee-Robbins and Andrew McNutt. 2023. Only YOU Can Make IEEE VIS Environmentally Sustainable. alt.vis (2023). 10.48550\/arXiv.2308.15429 https:\/\/arxiv.org\/abs\/2308.15429.","DOI":"10.48550\/arXiv.2308.15429"},{"key":"e_1_3_3_2_40_2","doi-asserted-by":"publisher","unstructured":"Fan Lei Arlen Fan Alan\u00a0M MacEachren and Ross Maciejewski. 2023. GeoLinter: A Linting Framework for Choropleth Maps. Transactions on Visualization and Computer Graphics (2023). 10.1109\/TVCG.2023.3322372","DOI":"10.1109\/TVCG.2023.3322372"},{"key":"e_1_3_3_2_41_2","doi-asserted-by":"crossref","unstructured":"Maxim Lisnic Zach Cutler Marina Kogan and Alexander Lex. 2025. Visualization Guardrails: Designing Interventions Against Cherry-Picking in Interactive Data Explorers. (2025). To Appear.","DOI":"10.31219\/osf.io\/4j9nr_v2"},{"key":"e_1_3_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675283"},{"key":"e_1_3_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3381814"},{"key":"e_1_3_3_2_44_2","first-page":"1","volume-title":"VisGuides (IEEEVIS Workshop)","author":"McNutt Andrew","year":"2018","unstructured":"Andrew McNutt and Gordon Kindlmann. 2018. Linting for visualization: Towards a practical automated visualization guidance system. In VisGuides (IEEEVIS Workshop). 1\u201314."},{"key":"e_1_3_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376420"},{"key":"e_1_3_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580683"},{"key":"e_1_3_3_2_47_2","doi-asserted-by":"publisher","unstructured":"Andrew McNutt Maureen\u00a0C Stone and Jeffrey Heer. 2024. Mixing Linters with GUIs: A Color Palette Design Probe. IEEE Transactions on Visualization and Computer Graphics (2024). 10.1109\/TVCG.2024.3456317","DOI":"10.1109\/TVCG.2024.3456317"},{"key":"e_1_3_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580940"},{"key":"e_1_3_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/2724660.2728672"},{"key":"e_1_3_3_2_50_2","doi-asserted-by":"publisher","unstructured":"Arpit Narechania Adam Coscia Emily Wall and Alex Endert. 2021. Lumos: Increasing awareness of analytic behavior during visual data analysis. IEEE Transactions on Visualization and Computer Graphics 28 1 (2021) 1009\u20131018. 10.1109\/TVCG.2021.3114827","DOI":"10.1109\/TVCG.2021.3114827"},{"key":"e_1_3_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/14666.003.0010"},{"key":"e_1_3_3_2_52_2","doi-asserted-by":"publisher","unstructured":"Gordon Pennycook Adam Bear Evan\u00a0T Collins and David\u00a0G Rand. 2020. The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings. Management science 66 11 (2020) 4944\u20134957. 10.1287\/mnsc.2019.3478","DOI":"10.1287\/mnsc.2019.3478"},{"key":"e_1_3_3_2_53_2","unstructured":"Polygon. 2019. Art Sqool took me back to real-life art school. https:\/\/www.polygon.com\/2019\/3\/1\/18246354\/art-sqool-impressions-pc-mac."},{"key":"e_1_3_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979148"},{"key":"e_1_3_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675246"},{"key":"e_1_3_3_2_56_2","first-page":"254","volume-title":"Empirical Methods in Natural Language Processing","author":"Snow Rion","year":"2008","unstructured":"Rion Snow, Brendan O\u2019Connor, Daniel Jurafsky, and Andrew Ng. 2008. Cheap and Fast \u2013 But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. In Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Honolulu, Hawaii, 254\u2013263. https:\/\/aclanthology.org\/D08-1027\/"},{"key":"e_1_3_3_2_57_2","doi-asserted-by":"publisher","unstructured":"Nicole Sultanum Dennis Bromley and Michael Correll. 2024. Data Guards: Challenges and Solutions for Fostering Trust in Data. IEEE Visualization Conference (2024). 10.1109\/VIS55277.2024.00019","DOI":"10.1109\/VIS55277.2024.00019"},{"key":"e_1_3_3_2_58_2","doi-asserted-by":"publisher","unstructured":"Twitter Notes [n. d.]. Note ranking algorithm. https:\/\/communitynotes.x.com\/guide\/en\/under-the-hood\/ranking-notes. 10.1007\/s10288-003-0021-7 Accessed 10\/29\/2024.","DOI":"10.1007\/s10288-003-0021-7"},{"key":"e_1_3_3_2_59_2","first-page":"1","volume-title":"SIGCHI Conference on Human Factors in Computing Systems","author":"Wang Huichen\u00a0Will","year":"2025","unstructured":"Huichen\u00a0Will Wang, Larry Birnbaum, and Vidya Setlur. 2025. Jupybara: Operationalizing a Design Space for Actionable Data Analysis and Storytelling with LLMs. In SIGCHI Conference on Human Factors in Computing Systems. ACM, 1\u201313."},{"key":"e_1_3_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642335"},{"key":"e_1_3_3_2_61_2","doi-asserted-by":"publisher","unstructured":"Colin Ware and Matthew\u00a0D Plumlee. 2013. Designing a better weather display. Information Visualization 12 3-4 (2013) 221\u2013239. 10.1177\/1473871612465214","DOI":"10.1177\/1473871612465214"},{"key":"e_1_3_3_2_62_2","doi-asserted-by":"publisher","unstructured":"Colin Ware Maureen Stone and Danielle\u00a0Albers Szafir. 2023. Rainbow colormaps are not all bad. IEEE Computer Graphics and Applications 43 3 (2023) 88\u201393. 10.1109\/MCG.2023.3246111","DOI":"10.1109\/MCG.2023.3246111"},{"key":"e_1_3_3_2_63_2","doi-asserted-by":"publisher","unstructured":"Brian Whitworth. 2005. Polite computing. Behaviour & Information Technology 24 5 (2005) 353\u2013363. 10.1080\/01449290512331333700","DOI":"10.1080\/01449290512331333700"},{"key":"e_1_3_3_2_64_2","unstructured":"Wikipedia. 2024. Colorless green ideas sleep furiously \u2014 Wikipedia The Free Encyclopedia. http:\/\/en.wikipedia.org\/w\/index.php?title=Colorless%20green%20ideas%20sleep%20furiously&oldid=1258719573. [Online; accessed 24-November-2024]."},{"key":"e_1_3_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207709"},{"key":"e_1_3_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/VL-HCC57772.2023.00043"},{"key":"e_1_3_3_2_67_2","doi-asserted-by":"publisher","unstructured":"Jo Wood Alexander Kachkaev and Jason Dykes. 2018. Design exposition with literate visualization. Transactions on Visualization and Computer Graphics 25 1 (2018) 759\u2013768. 10.1109\/TVCG.2018.2864836","DOI":"10.1109\/TVCG.2018.2864836"},{"key":"e_1_3_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445179"},{"key":"e_1_3_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517582"},{"key":"e_1_3_3_2_70_2","doi-asserted-by":"publisher","unstructured":"Tongshuang Wu Haiyi Zhu Maya Albayrak Alexis Axon Amanda Bertsch Wenxing Deng Ziqi Ding Bill Guo Sireesh Gururaja Tzu-Sheng Kuo et\u00a0al. 2023. Llms as workers in human-computational algorithms? replicating crowdsourcing pipelines with llms. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2307.10168 (2023). 10.48550\/arXiv.2307.10168","DOI":"10.48550\/arXiv.2307.10168"},{"key":"e_1_3_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531604"},{"key":"e_1_3_3_2_72_2","unstructured":"Nicholas\u00a0C. Zakas and Nicole Sullivan. 2011. CSS Lint. csslint.net."},{"key":"e_1_3_3_2_73_2","unstructured":"Wei Zheng and Youssef Sherif. 2019. Visualization Linter."}],"event":{"name":"CHI EA '25: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems","location":"Yokohama Japan","acronym":"CHI EA '25","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3706599.3716230","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3706599.3716230","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:11Z","timestamp":1750298231000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3706599.3716230"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,25]]},"references-count":72,"alternative-id":["10.1145\/3706599.3716230","10.1145\/3706599"],"URL":"https:\/\/doi.org\/10.1145\/3706599.3716230","relation":{},"subject":[],"published":{"date-parts":[[2025,4,25]]},"assertion":[{"value":"2025-04-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}