{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:30:58Z","timestamp":1750221058617,"version":"3.41.0"},"reference-count":67,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW","license":[{"start":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T00:00:00Z","timestamp":1541030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["HCC 1314778"],"award-info":[{"award-number":["HCC 1314778"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>Inspired by policy deliberation methods and iterative writing in crowdsourcing, we developed and evaluated a task in which newcomers to an online policy discussion, before entering the discussion, generate prompts that encourage existing commenters to engage with each other. In an experiment with 453 Amazon Mechanical Turk (AMT) crowd workers, we found that newcomers can often craft acceptable prompts, especially when given guidance on prompt-writing and balanced opinions between the comments they synthesize. However, crafting these prompts had little effect on the quality of comments they posted to a simulated discussion forum following the prompt task, as measured by the reasoning and topic coherence of comments. Our results inform best practices and pose questions for the design of discussion systems, both in general and for online policy discussion in particular.<\/jats:p>","DOI":"10.1145\/3274390","type":"journal-article","created":{"date-parts":[[2018,11,1]],"date-time":"2018-11-01T21:21:27Z","timestamp":1541107287000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Crafting Policy Discussion Prompts as a Task for Newcomers"],"prefix":"10.1145","volume":"2","author":[{"given":"Brian","family":"McInnis","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}]},{"given":"Gilly","family":"Leshed","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}]},{"given":"Dan","family":"Cosley","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}]}],"member":"320","published-online":{"date-parts":[[2018,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvb.2007.02.001"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2791285"},{"key":"e_1_2_1_3_1","volume-title":"Methods for analyzing and measuring group deliberation . Sourcebook of political communication research: Methods, measures, and analytical techniques","author":"Black Laura W.","year":"2010","unstructured":"Laura W. Black, Stephanie Burkhalter, John Gastil, and Jennifer Stromer-Galley. 2010. Methods for analyzing and measuring group deliberation . Sourcebook of political communication research: Methods, measures, and analytical techniques (2010), 323--345."},{"key":"e_1_2_1_4_1","volume-title":"DeGroot","author":"Black Laura W.","year":"2011","unstructured":"Laura W. Black, Howard T. Welser, Dan Cosley, and Jocelyn M. DeGroot. 2011. Self-Governance Through Group Discussion in Wikipedia . Small Group Research , Vol. 42, 5 (10 2011), 595--634."},{"key":"e_1_2_1_5_1","volume-title":"Membership Claims and Requests: Conversation-Level Newcomer Socialization Strategies in Online Groups","author":"Burke Moira","year":"2010","unstructured":"Moira Burke, Robert E. Kraut, and Elisabeth Joyce. 2010. Membership Claims and Requests: Conversation-Level Newcomer Socialization Strategies in Online Groups . Small Group Research , Vol. 41, 1 (2 2010), 4--40."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518847"},{"key":"e_1_2_1_7_1","first-page":"398","article-title":"A conceptual definition and theoretical model of public deliberation in small face-to-face groups","volume":"12","author":"Burkhalter Stephanie","year":"2002","unstructured":"Stephanie Burkhalter, John Gastil, and Todd Kelshaw. 2002. A conceptual definition and theoretical model of public deliberation in small face-to-face groups . Communication Theory , Vol. 12, 4 (2002), 398--422.","journal-title":"Communication Theory"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718918.1718940"},{"key":"e_1_2_1_9_1","volume-title":"Old wine in a new bottle: Impact of membership change on group creativity . Organizational Behavior and human decision","author":"Choi Hoon-Seok","year":"2005","unstructured":"Hoon-Seok Choi and Leigh Thompson. 2005. Old wine in a new bottle: Impact of membership change on group creativity . Organizational Behavior and human decision , Vol. 98, 2 (2005), 121--132."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcom.12104"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/2898607.2898793"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488416"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/520697"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145355"},{"key":"e_1_2_1_15_1","volume-title":"Weld","author":"Drapeau Ryan","year":"2016","unstructured":"Ryan Drapeau, Lydia B. Chilton, Jonathan Bragg, and Daniel S. Weld. 2016. MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy . Human Computation and Crowdsourcing (2016)."},{"key":"e_1_2_1_16_1","article-title":"The Magic Sauce: Practices of Facilitation in Online Policy Deliberation","volume":"12","author":"Epstein Dmitry","year":"2016","unstructured":"Dmitry Epstein and Gilly Leshed. 2016. The Magic Sauce: Practices of Facilitation in Online Policy Deliberation . Journal of Public Deliberation , Vol. 12, 1 (2016).","journal-title":"Journal of Public Deliberation"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998210"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2010.06.011"},{"volume-title":"Becoming political: Comparative perspectives on citizenship education","author":"Hahn Carole","key":"e_1_2_1_19_1","unstructured":"Carole Hahn. 1998. Becoming political: Comparative perspectives on citizenship education. Suny Press."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858364"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441872"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2012.10.008"},{"volume-title":"Controversy in the classroom: The democratic power of discussion","author":"Hess Diana E.","key":"e_1_2_1_23_1","unstructured":"Diana E. Hess. 2009. Controversy in the classroom: The democratic power of discussion. Routledge."},{"key":"e_1_2_1_24_1","volume-title":"Hess and Paula McAvoy","author":"Diana","year":"2014","unstructured":"Diana E. Hess and Paula McAvoy. 2014. The political classroom: Evidence and ethics in democratic education. Routledge."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441870"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470742"},{"key":"e_1_2_1_27_1","volume-title":"Karpowitz and Chad Raphael","author":"Christopher","year":"2014","unstructured":"Christopher F. Karpowitz and Chad Raphael. 2014. Deliberation, democracy, and civic forums: Improving equality and publicity. Cambridge University Press."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820072"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998196"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441923"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_1_32_1","volume-title":"How to harvest collective wisdom on complex problems: An introduction to the mit deliberatorium","author":"Klein Mark","year":"2011","unstructured":"Mark Klein. 2011. How to harvest collective wisdom on complex problems: An introduction to the mit deliberatorium . Center for Collective Intelligence working paper (2011)."},{"key":"e_1_2_1_33_1","volume-title":"Dealing with Newcomers . Evidence-based Social Design: Mining the Social Sciences to Build Online Communities","author":"Kraut Robert E.","year":"2010","unstructured":"Robert E. Kraut, Moira Burke, John Riedl, and Paul Resnick. 2010. Dealing with Newcomers . Evidence-based Social Design: Mining the Social Sciences to Build Online Communities , Vol. 1, 1 (2010), 42."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208621"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505057"},{"key":"e_1_2_1_36_1","volume-title":"Chul Hyun Park, and Erik Johnston","author":"Lampe Cliff","year":"2014","unstructured":"Cliff Lampe, Paul Zube, Jusil Lee, Chul Hyun Park, and Erik Johnston. 2014. Crowdsourcing civility: A natural experiment examining the effects of distributed moderation in online forums . Government Information Quarterly , Vol. 31, 2 (4 2014), 317--326."},{"key":"e_1_2_1_37_1","volume-title":"Moreland","author":"Levine John M.","year":"2003","unstructured":"John M. Levine, Hoon-Seok Choi, and Richard L. Moreland. 2003. Newcomer innovation in work teams . Group creativity: Innovation (2003)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/14792779543000093"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866040"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcc4.12155"},{"key":"e_1_2_1_41_1","doi-asserted-by":"crossref","unstructured":"Jane Mansbridge Janette Hartz-Karp Matthew Amengual and John Gastil. 2006. Norms of deliberation: An inductive study . (2006).","DOI":"10.16997\/jdd.35"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531663"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/curi.12000"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3148330.3148348"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858539"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2968077"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820075"},{"key":"e_1_2_1_48_1","volume-title":"Online public deliberation in China: evolution of interaction patterns and network homophily in the Tianya discussion forum . Information","author":"Medaglia Rony","year":"2017","unstructured":"Rony Medaglia and Yang Yang. 2017. Online public deliberation in China: evolution of interaction patterns and network homophily in the Tianya discussion forum . Information, Communication & Society , Vol. 20, 5 (5 2017), 733--753."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1080\/01292986.2014.917116"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/19460171.2012.689735"},{"key":"e_1_2_1_51_1","volume-title":"Socialization in small groups: Temporal changes in individual-group relations . Advances in experimental social psychology","author":"Moreland Richard L","year":"1982","unstructured":"Richard L Moreland and John M Levine. 1982. Socialization in small groups: Temporal changes in individual-group relations . Advances in experimental social psychology , Vol. 15 (1982), 137--192."},{"key":"e_1_2_1_52_1","unstructured":"Richard L Moreland and John M Levine. 1988. Group dynamics over time: Development and socialization in small groups. (1988)."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753543"},{"key":"e_1_2_1_54_1","volume-title":"Leah Wing, Xiaoxi Xu, and Natasha Shrikant.","author":"Murray Tom","year":"2013","unstructured":"Tom Murray, Lynn Stephens, Beverly Park Woolf, Leah Wing, Xiaoxi Xu, and Natasha Shrikant. 2013. Supporting Social Deliberative Skills Online: The Effects of Reflective Scaffolding Tools .Springer, Berlin, Heidelberg, 313--322."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.3102\/0013189X035008011"},{"key":"e_1_2_1_56_1","volume-title":"The reader-to-leader framework: Motivating technology-mediated social participation . AIS Transactions on Human-Computer","author":"Preece Jennifer","year":"2009","unstructured":"Jennifer Preece and Ben Shneiderman. 2009. The reader-to-leader framework: Motivating technology-mediated social participation . AIS Transactions on Human-Computer (2009)."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1108\/TG-02-2013-0005"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1080\/08838151.2015.1093482"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1475-682X.1980.tb00021.x"},{"key":"e_1_2_1_60_1","article-title":"Who Moderates the Moderators? The Effect of Non-neutral Moderators in Deliberative Decision Making","volume":"9","author":"Spada Paolo","year":"2013","unstructured":"Paolo Spada and James Raymond Vreeland. 2013. Who Moderates the Moderators? The Effect of Non-neutral Moderators in Deliberative Decision Making . Journal of Public Deliberation , Vol. 9, 2 (2013). http:\/\/www.publicdeliberation.net\/jpd http:\/\/www.publicdeliberation.net\/jpd\/vol9\/iss2\/art3","journal-title":"Journal of Public Deliberation"},{"key":"e_1_2_1_61_1","volume-title":"Online Newspapers' Readers' Comments - Democratic Conversation Platforms or Virtual Soapboxes? Comunicacc a o e Sociedade","author":"Strandberg Kim","year":"2013","unstructured":"Kim Strandberg and Janne Berg. 2013. Online Newspapers' Readers' Comments - Democratic Conversation Platforms or Virtual Soapboxes? Comunicacc a o e Sociedade , Vol. 23, 1 (2013), 132."},{"key":"e_1_2_1_62_1","article-title":"Measuring deliberation's content: A coding scheme","volume":"3","author":"Stromer-Galley Jennifer","year":"2007","unstructured":"Jennifer Stromer-Galley. 2007. Measuring deliberation's content: A coding scheme . Journal of public deliberation , Vol. 3, 1 (2007).","journal-title":"Journal of public deliberation"},{"key":"e_1_2_1_63_1","volume-title":"Martinson","author":"Stromer-Galley Jennifer","year":"2009","unstructured":"Jennifer Stromer-Galley and Anna M. Martinson. 2009. Coherence in political computer-mediated communication: analyzing topic relevance and drift in chat . Discourse & Communication , Vol. 3, 2 (5 2009), 195--216."},{"volume-title":"Second AAAI Conference on Human Computation and Crowdsourcing .","author":"Verroios Vasilis","key":"e_1_2_1_64_1","unstructured":"Vasilis Verroios and Michael S. Bernstein. 2014. Context trees: Crowdsourcing global understanding from local views. In Second AAAI Conference on Human Computation and Crowdsourcing ."},{"key":"e_1_2_1_65_1","volume-title":"Legitimate peripheral participation in communities of practice . Supporting lifelong learning","author":"Wenger Etienne","year":"2002","unstructured":"Etienne Wenger and Jean Lave. 2002. Legitimate peripheral participation in communities of practice . Supporting lifelong learning (2002)."},{"key":"e_1_2_1_66_1","volume-title":"Malone","author":"Woolley Anita W.","year":"2010","unstructured":"Anita W. Woolley, Christopher F. Chabris, Alex Pentland, Nada Hashmi, and Thomas W. Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. Science (New York, N.Y.) , Vol. 330, 6004 (10 2010), 686--8."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998235"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274390","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3274390","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3274390","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:35Z","timestamp":1750207475000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3274390"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11]]},"references-count":67,"journal-issue":{"issue":"CSCW","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.1145\/3274390"],"URL":"https:\/\/doi.org\/10.1145\/3274390","relation":{},"ISSN":["2573-0142"],"issn-type":[{"type":"electronic","value":"2573-0142"}],"subject":[],"published":{"date-parts":[[2018,11]]},"assertion":[{"value":"2018-11-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}