{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T05:55:18Z","timestamp":1782280518559,"version":"3.54.5"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"vor","delay-in-days":370,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP21H03552"],"award-info":[{"award-number":["JP21H03552"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["JPMJCR16E3"],"award-info":[{"award-number":["JPMJCR16E3"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,13]]},"abstract":"<jats:p>Workers are the most important resource in crowdsourcing. However, only investing in worker-centric needs, such as skill improvement, often conflicts with short-term platform-centric needs, such as task throughput. This paper studies learning strategies in task assignment in crowdsourcing and their impact on platform-centric needs. We formalize learning potential of individual tasks and collaborative tasks, and devise an iterative task assignment and completion approach that implements strategies grounded in learning theories. We conduct experiments to compare several learning strategies in terms of skill improvement, and in terms of task throughput and contribution quality. We discuss how our findings open new research directions in learning and collaboration.<\/jats:p>","DOI":"10.1145\/3479519","type":"journal-article","created":{"date-parts":[[2021,10,19]],"date-time":"2021-10-19T02:32:07Z","timestamp":1634610727000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Task Assignment Strategies for Crowd Worker Ability Improvement"],"prefix":"10.1145","volume":"5","author":[{"given":"Masaki","family":"Matsubara","sequence":"first","affiliation":[{"name":"University of Tsukuba, Tsukuba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ria Mae","family":"Borromeo","sequence":"additional","affiliation":[{"name":"University of the Philippines, Diliman, Philippines"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sihem","family":"Amer-Yahia","sequence":"additional","affiliation":[{"name":"CNRS, Universit\u00e9 Grenoble Alpes, Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Atsuyuki","family":"Morishima","sequence":"additional","affiliation":[{"name":"University of Tsukuba, Tsukuba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2755660"},{"key":"e_1_2_1_2_1","first-page":"576","article-title":"Crowdsourcing higher education: A design proposal for distributed learning","volume":"7","author":"Anderson Michael","year":"2011","unstructured":"Michael Anderson. 2011. Crowdsourcing higher education: A design proposal for distributed learning. MERLOT Journal of Online Learning and Teaching, Vol. 7, 4 (2011), 576--590.","journal-title":"MERLOT Journal of Online Learning and Teaching"},{"key":"e_1_2_1_3_1","volume-title":"Item response theory: Parameter estimation techniques","author":"Baker Frank B","unstructured":"Frank B Baker and Seock-Ho Kim. 2004. Item response theory: Parameter estimation techniques .CRC Press."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493394.2493404"},{"key":"e_1_2_1_5_1","volume-title":"Cognitive domain.","author":"Bloom Benjamin S","year":"1956","unstructured":"Benjamin S Bloom. 1956. Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York: McKay (1956), 20--24."},{"key":"e_1_2_1_6_1","unstructured":"Kenneth A Bruffee. 1999. Collaborative learning: Higher education interdependence and the authority of knowledge. ERIC."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2015.2493525"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858237"},{"key":"e_1_2_1_9_1","first-page":"1","article-title":"A polynomial time approximation scheme for the multiple knapsack problem","volume":"38","author":"Chekuri Chandra","year":"2006","unstructured":"Chandra Chekuri and Sanjeev Khanna. 2006. A polynomial time approximation scheme for the multiple knapsack problem. Society for Industrial and Applied Mathematics Journal on Computing, Vol. 38, 3 (2006), 1.","journal-title":"Society for Industrial and Applied Mathematics Journal on Computing"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274306"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675251"},{"key":"e_1_2_1_12_1","unstructured":"Mihaly Csikszentmihalyi. 1975. Beyond boredom and anxiety: The experience of play in work and games. Jossey-Bass."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675260"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2018.05.019"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.promfg.2017.07.208"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v2i1.13154"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883030"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557217"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5269"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145355"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v4i1.13270"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330945"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027385.3027402"},{"key":"e_1_2_1_24_1","volume-title":"Design for Teaching and Learning in a Networked World","author":"Gadiraju Ujwal","unstructured":"Ujwal Gadiraju, Besnik Fetahu, and Ricardo Kawase. 2015. Training workers for improving performance in crowdsourcing microtasks. In Design for Teaching and Learning in a Networked World. Springer, 100--114."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2757226.2757249"},{"key":"e_1_2_1_26_1","volume-title":"Vassilis Kostakos, and Jorge Goncalves.","author":"Hettiachchi Danula","year":"2020","unstructured":"Danula Hettiachchi, Niels Van Berkel, Vassilis Kostakos, and Jorge Goncalves. 2020. CrowdCog: A Cognitive skill based system for heterogeneous task assignment and recommendation in crowdsourcing. Proceedings of the ACM on Human-Computer Interaction, Vol. 4, CSCW2 (2020), 1--22."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025917"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the Seventeenth Americas Conference on Information Systems. 1--11","author":"Kaufmann Nicolas","year":"2011","unstructured":"Nicolas Kaufmann, Thimo Schulze, and Daniel Veit. 2011. More than fun and money. Worker motivation in crowdsourcing -- a study on mechanical turk.. In Proceedings of the Seventeenth Americas Conference on Information Systems. 1--11."},{"key":"e_1_2_1_29_1","unstructured":"Juho Kim. 2015. Learnersourcing: improving learning with collective learner activity. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998196"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v6i1.13324"},{"key":"e_1_2_1_32_1","volume-title":"Human Computation","volume":"8","author":"Kobayashi Masaki","year":"2021","unstructured":"Masaki Kobayashi, Hiromi Morita, Masaki Matsubara, Nobuyuki Shimizu, and Atsuyuki Morishima. 2021. Empirical Study on Effects of Self-Correction in Crowdsourced Microtasks. Human Computation, Vol. 8 (2021)."},{"key":"e_1_2_1_33_1","unstructured":"David A Kolb. 1984. Experiential learning: Experience as the source of learning and development .Englewood Cliffs."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1177\/1046878112459261"},{"key":"e_1_2_1_35_1","unstructured":"David R Krathwohl and Lorin W Anderson. 2009. A taxonomy for learning teaching and assessing: A revision of Bloom's taxonomy of educational objectives .Longman."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025883"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v6i1.13328"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/00220973.2012.727884"},{"key":"e_1_2_1_39_1","volume-title":"Apprenticeship training in the workplace: Computer-coached practice environment as a new form of apprenticeship. Machine-mediated learning","author":"Lajoie Susanne P","year":"1989","unstructured":"Susanne P Lajoie and Alan Lesgold. 1989. Apprenticeship training in the workplace: Computer-coached practice environment as a new form of apprenticeship. Machine-mediated learning, Vol. 3, 1 (1989), 7--28."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2015.7357194"},{"key":"e_1_2_1_41_1","volume-title":"Situated learning: Legitimate peripheral participation","author":"Lave Jean","unstructured":"Jean Lave and Etienne Wenger. 1991. Situated learning: Legitimate peripheral participation .Cambridge university press."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858560"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488466"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1086\/258725"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC3.2014.6897145"},{"key":"e_1_2_1_46_1","volume-title":"Shonan Future-of-Work Workshop. 2020","volume":"49","author":"Participants","year":"2020","unstructured":"Participants of Shonan Future-of-Work Workshop. 2020. Making ai machines work for humans in fow. ACM SIGMOD Record, Vol. 49, 2 (2020), 30--35."},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of the 20th International Conference on Extending Database Technology (EDBT). 246--257","author":"Pilourdault Julien","year":"2017","unstructured":"Julien Pilourdault, Sihem Amer-Yahia, Dongwon Lee, and Senjuti Roy. 2017. Motivation-aware task assignment in crowdsourcing. In Proceedings of the 20th International Conference on Extending Database Technology (EDBT). 246--257."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2018.00041"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v1i1.13127"},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the 33rd International Conference on International Conference on Machine Learning -","volume":"48","author":"Shah Nihar","year":"2016","unstructured":"Nihar Shah and Dengyong Zhou. 2016. No oops, you won't do it again: Mechanisms for self-correction in crowdsourcing. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. 1--10."},{"key":"e_1_2_1_51_1","volume-title":"Last accessed","year":"2020","unstructured":"SurveyMonkey. Last accessed, 2020\/09\/10. Calculating the Number of Respondents You Need. newlinehttps:\/\/help.surveymonkey.com\/articles\/en_US\/kb\/How-many-respondents-do-I-need."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858121"},{"key":"e_1_2_1_53_1","volume-title":"Christos Athanasiadis, and Penny Standen.","author":"Taheri Mohammad","year":"2017","unstructured":"Mohammad Taheri, Nasser Sherkat, Nick Shopland, Dorothea Tsatsou, Enrique Hortal Nicholas Vretos, Christos Athanasiadis, and Penny Standen. 2017. Adaptation and personalization principles based on MaThiSiS findings. In Public report on Managing Affective-learning THrough Intelligent atoms and Smart InteractionS project ."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE48307.2020.00022"},{"key":"e_1_2_1_55_1","volume-title":"Mind in society: The development of higher psychological processes","author":"Vygotsky Lev Semenovich","unstructured":"Lev Semenovich Vygotsky. 1980. Mind in society: The development of higher psychological processes. Harvard university press."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274447"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2876034.2876042"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.5555\/1658616.1658651"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819953"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531718"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479519","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479519","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479519","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:57:32Z","timestamp":1752469052000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479519"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":60,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2021,10,13]]}},"alternative-id":["10.1145\/3479519"],"URL":"https:\/\/doi.org\/10.1145\/3479519","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,13]]},"assertion":[{"value":"2021-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}