{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T17:37:55Z","timestamp":1776101875713,"version":"3.50.1"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T00:00:00Z","timestamp":1618272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100005801","name":"Facebook","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100005801","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004705","name":"Capital One Financial Corporation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100004705","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Microsoft"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,4,13]]},"abstract":"<jats:p>Surveys are a common instrument to gauge self-reported opinions from the crowd for scholars in the CSCW community, the social sciences, and many other research areas. Researchers often use surveys to prioritize a subset of given options when there are resource constraints. Over the past century, researchers have developed a wide range of surveying techniques, including one of the most popular instruments, the Likert ordinal scale, to elicit individual preferences. However, the challenge to elicit accurate and rich self-reported responses with surveys in a resource-constrained context still persists today. In this study, we examine Quadratic Voting (QV), a voting mechanism powered by the affordances of a modern computer and straddles ratings and rankings approaches, as an alternative online survey technique. We argue that QV could elicit more accurate self-reported responses compared to the Likert scale when the goal is to understand relative preferences under resource constraints. We conducted two randomized controlled experiments on Amazon Mechanical Turk, one in the context of public opinion polling and the other in a human-computer interaction user study. Based on our Bayesian analysis results, a QV survey with a sufficient amount of voice credits, aligned significantly closer to participants' incentive-compatible behaviors than a Likert scale survey, with a medium to high effect size. In addition, we extended QV's application scenario from typical public policy and education research to a problem setting familiar to the CSCW community: a prototypical HCI user study. Our experiment results, QV survey design, and QV interface serve as a stepping stone for CSCW researchers to further explore this surveying methodology in their studies and encourage decision-makers from other communities to consider QV as a promising alternative.<\/jats:p>","DOI":"10.1145\/3449281","type":"journal-article","created":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T17:51:09Z","timestamp":1619113869000},"page":"1-43","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["\"I can show what I really like.\""],"prefix":"10.1145","volume":"5","author":[{"given":"Ti-Chung","family":"Cheng","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Tiffany Wenting","family":"Li","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Yi-Hung","family":"Chou","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, Hong Kong"}]},{"given":"Karrie","family":"Karahalios","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Hari","family":"Sundaram","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,4,22]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Retrieved","year":"2020","unstructured":"2019. $120 million in requests and $40 million in the bank. how an obscure theory helped prioritize the colorado budget. Retrieved Oct. 15, 2020 from https:\/\/coloradosun.com\/2019\/05\/28\/quadratic-voting-colorado-house-budget\/."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1086\/268949"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1086\/261662"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/19312458.2017.1317337"},{"key":"e_1_2_2_5_1","first-page":"2","article-title":"Extreme response style: a meta-analysis","volume":"16","author":"Batchelor John H","year":"2016","unstructured":"John H Batchelor and Chao Miao. 2016. Extreme response style: a meta-analysis. Journal of Organizational Psychology, 16, 2.","journal-title":"Journal of Organizational Psychology"},{"key":"e_1_2_2_6_1","doi-asserted-by":"crossref","unstructured":"Gerdus Benade Swaprava Nath Ariel D Procaccia and Nisarg Shah. 2020. Preference elicitation for participatory budgeting. Management Science.","DOI":"10.1287\/mnsc.2020.3666"},{"key":"e_1_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Matthias Benz and Stephan Meier. 2008. Do people behave in experiments as in the field?-evidence from donations. Experimental economics 11 3 268--281.","DOI":"10.1007\/s10683-007-9192-y"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Adam J Berinsky Gregory A Huber and Gabriel S Lenz. 2012. Evaluating online labor markets for experimental research: amazon. com's mechanical turk. Political analysis 20 3 351--368.","DOI":"10.1093\/pan\/mpr057"},{"key":"e_1_2_2_9_1","doi-asserted-by":"crossref","unstructured":"Yves Cabannes. 2004. Participatory budgeting: a significant contribution to participatory democracy. Environment and urbanization 16 1 27--46.","DOI":"10.1177\/095624780401600104"},{"key":"e_1_2_2_10_1","unstructured":"Charlotte Cavaill\u00e9 Daniel L Chen and Karine Van Der Straeten. 2018. Towards a general theory of survey response: likert scales vs. quadratic voting for attitudinal research."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1006\/jeem.1997.0988"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998258"},{"key":"e_1_2_2_13_1","volume-title":"Karrie Karahalios, and Hari Sundaram.","author":"Cheng Ti-Chung","year":"2021","unstructured":"Ti-Chung Cheng, Tiffany Wenting Li, Karrie Karahalios, and Hari Sundaram. 2021. Dataset for '?I can show what I really like.\": Eliciting Preferences via Quadratic Voting'. (2021)."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998213"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2006.07.015"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","unstructured":"John Dawes. 2008. Do data characteristics change according to the number of scale points used? an experiment using 5-point 7-point and 10-point scales. International journal of market research 50 1 61--104.","DOI":"10.1177\/147078530805000106"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159661"},{"key":"e_1_2_2_18_1","volume-title":"Retrieved","year":"2020","unstructured":"2019. Educational attainment in the united states: 2018. Retrieved Oct. 13, 2020 from https:\/\/www.census.gov\/data\/ tables\/2018\/demo\/education-attainment\/cps-detailed-tables.html."},{"key":"e_1_2_2_19_1","doi-asserted-by":"crossref","unstructured":"Jon X Eguia Nicole Immorlica Katrina Ligett E Glen Weyl and Dimitrios Xefteris. 2019. Quadratic voting with multiple alternatives. Available at SSRN 3319508.","DOI":"10.2139\/ssrn.3319508"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.soc.33.040406.131812"},{"key":"e_1_2_2_21_1","first-page":"43","article-title":"Human motivation and social cooperation: experimental and analytical foundations. Annu","volume":"33","author":"Fehr Ernst","year":"2007","unstructured":"Ernst Fehr and Herbert Gintis. 2007. Human motivation and social cooperation: experimental and analytical foundations. Annu. Rev. Sociol., 33, 43--64.","journal-title":"Rev. Sociol."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/2835434.2835437"},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Adrian Furnham. 1986. Response bias social desirability and dissimulation. Personality and individual differences 7 3 385--400.","DOI":"10.1016\/0191-8869(86)90014-0"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2007.02.005"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.2501\/S147078531020148X"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/713676576"},{"key":"e_1_2_2_27_1","unstructured":"P Glaser. 2008. Response rates. encyclopedia of survey research methods. (2008)."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11135-007-9089-z"},{"key":"e_1_2_2_29_1","unstructured":"Ashish Goel Anilesh K Krishnaswamy and Sukolsak Sakshuwong. 2016. Budget aggregation via knapsack voting: welfare-maximization and strategy-proofness. Collective Intelligence 783--809."},{"key":"e_1_2_2_30_1","unstructured":"Ashish Goel Anilesh K Krishnaswamy Sukolsak Sakshuwong and Tanja Aitamurto. [n. d.] Knapsack voting."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.15346\/hc.v2i1.4"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/274946.274959"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1093\/swr\/27.1.45"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBC.2008.2001246"},{"key":"e_1_2_2_35_1","doi-asserted-by":"crossref","unstructured":"John PA Ioannidis. 2005. Why most published research findings are false. PLoS medicine 2 8 e124.","DOI":"10.1371\/journal.pmed.0020124"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.79.6.995"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146167211399776"},{"key":"e_1_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Ankur Joshi Saket Kale Satish Chandel and D Kumar Pal. 2015. Likert scale: explored and explained. Current Journal of Applied Science and Technology 396--403.","DOI":"10.9734\/BJAST\/2015\/14975"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858465"},{"key":"e_1_2_2_40_1","volume-title":"Proceedings of the 13th annual ACM international conference on Multimedia, 829--838","author":"Knoche Hendrik","year":"2005","unstructured":"Hendrik Knoche, John D McCarthy, and M Angela Sasse. 2005. Can small be beautiful? assessing image resolution requirements for mobile tv. In Proceedings of the 13th annual ACM international conference on Multimedia, 829--838."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-006-0076-5"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446502500404"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1002\/wcs.72"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.12.008"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1257\/pandp.20181002"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1257\/pandp.20181002"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173610"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v2i1.13150"},{"key":"e_1_2_2_49_1","unstructured":"Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of psychology."},{"key":"e_1_2_2_50_1","doi-asserted-by":"crossref","unstructured":"Leib Litman Jonathan Robinson and Tzvi Abberbock. 2017. Turkprime. com: a versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior research methods 49 2 433--442.","DOI":"10.3758\/s13428-016-0727-z"},{"key":"e_1_2_2_51_1","volume-title":"Retrieved","year":"2020","unstructured":"2019. Little public support for reductions in federal spending. Retrieved Oct. 15, 2020 from https:\/\/www.pewresearch. org\/politics\/2019\/04\/11\/little-public-support-for-reductions-in-federal-spending\/."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998256"},{"key":"e_1_2_2_53_1","unstructured":"Naresh K Malhotra and Mark Peterson. 2006. Basic marketing research: A decision-making approach. Prentice hall."},{"key":"e_1_2_2_54_1","volume-title":"Statistical rethinking: A Bayesian course with examples in R and Stan","author":"McElreath Richard","unstructured":"Richard McElreath. 2015. Statistical rethinking: A Bayesian course with examples in R and Stan. Chapman and Hall\/CRC."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1162\/JEEA.2007.5.6.1203"},{"key":"e_1_2_2_56_1","doi-asserted-by":"crossref","unstructured":"Gerhard Meisenberg and Amandy Williams. 2008. Are acquiescent and extreme response styles related to low intelligence and education? Personality and individual differences 44 7 1539--1550.","DOI":"10.1016\/j.paid.2008.01.010"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/BMSB.2013.6621787"},{"key":"e_1_2_2_58_1","volume-title":"John PTM Gelissen, and Jeroen K Vermunt","author":"Moors Guy","year":"2016","unstructured":"Guy Moors, Ingrid Vriens, John PTM Gelissen, and Jeroen K Vermunt. 2016. Two of a kind. similarities between ranking and rating data in measuring values. In Survey Research Methods number 1. Vol. 10, 15--33."},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.5204\/ssj.v8i2.377"},{"key":"e_1_2_2_60_1","unstructured":"Olga Nikolayeva Esteban Buz Linda Liu Andrew Watts and Tim Florian Jaeger. 2015. Web based tutorial on experimental design. https:\/\/www.hlp.rochester.edu\/resources\/BCS152-Tutorial\/."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/35.256878"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2399016.2399025"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1180639.1180831"},{"key":"e_1_2_2_64_1","volume-title":"Radical markets: Uprooting capitalism and democracy for a just society","author":"Posner Eric A","unstructured":"Eric A Posner and E Glen Weyl. 2018. Radical markets: Uprooting capitalism and democracy for a just society. Princeton University Press."},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11127-017-0416-1"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.3368\/le.86.2.363"},{"key":"e_1_2_2_67_1","first-page":"1","article-title":"Exploring slider vs. categorical response formats in web-based surveys","volume":"11","author":"Roster Catherine A","year":"2015","unstructured":"Catherine A Roster, Lorenzo Lucianetti, and Gerald Albaum. 2015. Exploring slider vs. categorical response formats in web-based surveys. Journal of Research Practice, 11, 1, 1.","journal-title":"Journal of Research Practice"},{"key":"e_1_2_2_68_1","doi-asserted-by":"crossref","unstructured":"Alvin E Roth. 1982. Incentive compatibility in a market with indivisible goods. Economics letters 9 2 127--132.","DOI":"10.1016\/0165-1765(82)90003-9"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/1785414.1785439"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.55"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797614563958"},{"key":"e_1_2_2_72_1","volume-title":"Retrieved","year":"2020","unstructured":"2020. Shelf solutions. Retrieved Oct. 12, 2020 from https:\/\/www.nielsen.com\/apac\/en\/solutions\/measurement\/shelf-solutions\/."},{"key":"e_1_2_2_73_1","first-page":"35","article-title":"Modern information retrieval: a brief overview","volume":"24","author":"Amit Singhal","year":"2001","unstructured":"Amit Singhal et al. 2001. Modern information retrieval: a brief overview. IEEE Data Eng. Bull., 24, 4, 35--43.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/49.481694"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1561\/100.00006005"},{"key":"e_1_2_2_76_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction, 3, CSCW, 1--28","author":"Xiao Ziang","year":"2019","unstructured":"Ziang Xiao, Po-Shiun Ho, Xinran Wang, Karrie Karahalios, and Hari Sundaram. 2019. Should we use an abstract comic form to persuade? experiments with online charitable donation. Proceedings of the ACM on Human-Computer Interaction, 3, CSCW, 1--28."},{"key":"e_1_2_2_77_1","unstructured":"Ewa Zawojska Miko?aj Czajkowski et al. 2015. Re-examining empirical evidence on contingent valuation--Importance of incentive compatibility. Tech. rep."},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998235"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449281","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3449281","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:22Z","timestamp":1750195702000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3449281"}},"subtitle":["Eliciting Preferences via Quadratic Voting"],"short-title":[],"issued":{"date-parts":[[2021,4,13]]},"references-count":78,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2021,4,13]]}},"alternative-id":["10.1145\/3449281"],"URL":"https:\/\/doi.org\/10.1145\/3449281","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,13]]},"assertion":[{"value":"2021-04-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}