{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T00:32:42Z","timestamp":1760661162994,"version":"build-2065373602"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T00:00:00Z","timestamp":1760572800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["21-06882 and 21-06865"],"award-info":[{"award-number":["21-06882 and 21-06865"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,10,18]]},"abstract":"<jats:p>We explore patterns of interaction with different learning resources (e.g., forums) to predict learning outcomes in an online citizen science project called Gravity Spy. To explore how volunteers engage with and benefit from these resources, we categorize them based on S\u00f8rensen's three forms of presence in learning environments: authority-subject, agent-centered, and communal presence. Methodologically, we apply sequence analysis to traces of volunteer interactions with the project to identify engagement patterns with these resources that predict learning. Our interpretation of these patterns is augmented by insights gleaned from interviews with volunteers about their work and use of learning resources. We find that early in the project, volunteers have only a simple task to learn, and completing that task is most predictive of their learning. At more advanced levels, when tasks become more complex, discussions with other volunteers become increasingly important, and interaction patterns become more varied. Viewing learning as a series of routines allows us to articulate precisely how and in what context learning occurs. We conclude by discussing the implications of these findings for designing citizen science projects that promote learning.<\/jats:p>","DOI":"10.1145\/3757489","type":"journal-article","created":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T17:32:00Z","timestamp":1760635920000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Leveling Up or Dropping Out: Searching for Learning Routines in Crowdsourced Environments"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0828-4506","authenticated-orcid":false,"given":"Corey","family":"Jackson","sequence":"first","affiliation":[{"name":"The Information School, University of Wisconsin, Madison, Madison, Wisconsin, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0612-1551","authenticated-orcid":false,"given":"Carsten","family":"\u00d8sterlund","sequence":"additional","affiliation":[{"name":"School of Information Studies, Syracuse University, Syracuse, NY, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4886-699X","authenticated-orcid":false,"given":"Anand","family":"Vamsi","sequence":"additional","affiliation":[{"name":"Syracuse University, Syracuse, NY, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3719-2232","authenticated-orcid":false,"given":"Alexander Owen","family":"Smith","sequence":"additional","affiliation":[{"name":"School of Information Studies, Syracuse University, Syracuse, NY, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1996-3600","authenticated-orcid":false,"given":"Kevin","family":"Crowston","sequence":"additional","affiliation":[{"name":"School of Information Studies, Syracuse University, Syracuse, NY, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,10,16]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.1080.0200"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998317"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366711"},{"key":"e_1_2_1_4_1","volume-title":"Special issue on action research in information systems: Making IS research relevant to practice: Foreword. MIS quarterly","author":"Baskerville Richard","year":"2004","unstructured":"Richard Baskerville and Michael D Myers. 2004. Special issue on action research in information systems: Making IS research relevant to practice: Foreword. MIS quarterly (2004), 329-335."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/icc\/dth026"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820029"},{"key":"e_1_2_1_7_1","volume-title":"Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education","author":"Bonney Rick","unstructured":"Rick Bonney, Heidi Ballard, Rebecca Jordan, Ellen McCallie, Tina Phillips, Jennifer Shirk, and Candie C Wilderman. 2009. Public Participation in Scientific Research: Defining the Field and Assessing Its Potential for Informal Science Education. A CAISE Inquiry Group Report., Technical Report."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1099203.1099205"},{"key":"e_1_2_1_9_1","volume-title":"Electronic Communications of the EASST","volume":"33","author":"Cerone Antonio","year":"2010","unstructured":"Antonio Cerone and Sulayman Sowe. 2010. Using free\/libre open source software projects as e-learning tools. Electronic Communications of the EASST, Vol. 33 (2010)."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1088\/0264-9381\/21\/20\/024"},{"key":"e_1_2_1_11_1","volume-title":"International Conference on Information. Springer, 563-580","author":"Crowston Kevin","year":"2023","unstructured":"Kevin Crowston, Corey Jackson, Isabella Corieri, and Carsten \u00d8sterlund. 2023. Design principles for background knowledge to enhance learning in citizen science. In International Conference on Information. Springer, 563-580."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1017\/S174413741000024X"},{"key":"e_1_2_1_13_1","volume-title":"Learning Networks and Connective Knowledge. Collective intelligence and elearning","author":"Downes Stephen","year":"2006","unstructured":"Stephen Downes. 2006. Learning Networks and Connective Knowledge. Collective intelligence and elearning, Vol. 20 (2006), 1-26."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-005-9000-1"},{"key":"e_1_2_1_15_1","volume-title":"Learning by Expanding","author":"Engestr\u00f6m Yrj\u00f6","unstructured":"Yrj\u00f6 Engestr\u00f6m. 2014. Learning by Expanding. Cambridge University Press, , Cambridge, MA, USA."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.1100.0612"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.2307\/3556620"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998210"},{"key":"e_1_2_1_19_1","volume-title":"understanding, and supporting open collaboration: Lessons from the literature. American behavioral scientist","author":"Forte Andrea","year":"2013","unstructured":"Andrea Forte and Cliff Lampe. 2013. Defining, understanding, and supporting open collaboration: Lessons from the literature. American behavioral scientist, Vol. 57, 5 (2013), 535-547."},{"key":"e_1_2_1_20_1","volume-title":"From Green Peas to STEVE: citizen science engagement in space science. Space Science and Public Engagement","author":"Fortson Lucy","year":"2021","unstructured":"Lucy Fortson. 2021. From Green Peas to STEVE: citizen science engagement in space science. Space Science and Public Engagement (2021), 185-219."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2011.455"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441872"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3311957.3361856"},{"key":"e_1_2_1_24_1","first-page":"265","volume-title":"CSCL, 2013 Conference Proceedings Volume, 2 textemdash Short Papers, , Panels, Posters, Demos, & Community Events","volume":"2","author":"DeVries Hassman Katie","year":"2013","unstructured":"Katie DeVries Hassman, Gabriel Mugar, Carsten \u00d8sterlund, and Corey Jackson. 2013. Learning at the Seafloor, Looking at the Sky: The, Relationship between Individual Tasks and Collaborative Engagement in Two Citizen Science Projects. In CSCL, 2013 Conference Proceedings Volume, 2 textemdash Short Papers, , Panels, Posters, Demos, & Community Events, , Vol. 2. International Society of the Learning Sciences, Madison, WI, , 265-266."},{"key":"e_1_2_1_25_1","unstructured":"Christine Hine. 2008. Virtual Ethnography. SAGE Publications Ltd."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-018-9327-z"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274346"},{"key":"e_1_2_1_28_1","first-page":"1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","volume":"4","author":"Jackson Corey Brian","year":"2020","unstructured":"Corey Brian Jackson, Carsten \u00d8sterlund, Kevin Crowston, Mahboobeh Harandi, and Laura Trouille. 2020. Shifting forms of engagement: volunteer learning in online citizen science. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW1 (2020), 1-19."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819962"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998196"},{"key":"e_1_2_1_31_1","volume-title":"Perspectives on Socially Shared Cognition, Lauren B","author":"Lave Jean","unstructured":"Jean Lave. 1991. Situating Learning in Communities of Practice. In Perspectives on Socially Shared Cognition, Lauren B. Resnick, John M. Levine, and Stephanie D. Teasley (Eds.). American Psychological Association, , Washington D.C., 63-82."},{"key":"e_1_2_1_32_1","first-page":"1","article-title":"Who does what: Collaboration patterns in the Wikipedia and their impact on article quality","volume":"2","author":"Liu Jun","year":"2011","unstructured":"Jun Liu and Sudha Ram. 2011. Who does what: Collaboration patterns in the Wikipedia and their impact on article quality. ACM Transactions on Management Information Systems (TMIS), Vol. 2, 2 (2011), 1-23.","journal-title":"ACM Transactions on Management Information Systems (TMIS)"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531721"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1080\/01972240590925294"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5465\/AMBPP.2014.16799abstract"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-020-09372-2"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272973.3272986"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infoandorg.2008.08.001"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.1110.0624"},{"key":"e_1_2_1_40_1","first-page":"255","volume-title":"Proceedings of the Conference, on Computer Support, for Collaborative Learning, : Foundations, for a CSCL Community, (CSCL, '02)","author":"Reiser Brian J.","year":"2002","unstructured":"Brian J. Reiser. 2002. Why Scaffolding Should Sometimes Make Tasks More Difficult for Learners. In Proceedings of the Conference, on Computer Support, for Collaborative Learning, : Foundations, for a CSCL Community, (CSCL, '02). International Society of the Learning Sciences, Boulder, Colorado, , 255-264."},{"key":"e_1_2_1_41_1","volume-title":"M\u00fcller","author":"Ritschard Gilbert","year":"2009","unstructured":"Gilbert Ritschard, Matthias Studer, Alexis Gabadinho, and Nicolas S. M\u00fcller. 2009. TraMineR: A Toolbox for Exploring and Rendering Sequence Data. Swiss Center for Life Course and Inequality Research. https:\/\/cran.r-project.org\/package=TraMineR R package version 2.4-0."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2567948.2579215"},{"key":"e_1_2_1_43_1","first-page":"B1","article-title":"SHAPING COLLECTIVE COGNITION AND BEHAVIOR THROUGH COLLECTIVE LEARNING.. In Academy of Management Proceedings, Vol. 2000. Academy of Management Briarcliff Manor","volume":"10510","author":"Sitkin Sim B","year":"2000","unstructured":"Sim B Sitkin. 2000. SHAPING COLLECTIVE COGNITION AND BEHAVIOR THROUGH COLLECTIVE LEARNING.. In Academy of Management Proceedings, Vol. 2000. Academy of Management Briarcliff Manor, NY 10510, B1-B6.","journal-title":"NY"},{"key":"e_1_2_1_44_1","volume-title":"The Materiality of Learning","author":"S\u00f8rensen Estrid","unstructured":"Estrid S\u00f8rensen. 2009. The Materiality of Learning. Cambridge University Press, , Cambridge, MA, USA."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.1120.0472"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.17705\/1jais.00590"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702420"},{"key":"e_1_2_1_48_1","volume-title":"joining, and specialization in open source software innovation: a case study. Research policy","author":"Krogh Georg Von","year":"2003","unstructured":"Georg Von Krogh, Sebastian Spaeth, and Karim R Lakhani. 2003. Community, joining, and specialization in open source software innovation: a case study. Research policy, Vol. 32, 7 (2003), 1217-1241."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6382\/aa5cea"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757489","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3757489","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T17:45:19Z","timestamp":1760636719000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3757489"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,16]]},"references-count":49,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,10,18]]}},"alternative-id":["10.1145\/3757489"],"URL":"https:\/\/doi.org\/10.1145\/3757489","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,16]]},"assertion":[{"value":"2025-10-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}