{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T02:44:11Z","timestamp":1768013051275,"version":"3.49.0"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,2,21]],"date-time":"2019-02-21T00:00:00Z","timestamp":1550707200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"EU's H2020 programme","award":["732328"],"award-info":[{"award-number":["732328"]}]},{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","award":["P00P2153023"],"award-info":[{"award-number":["P00P2153023"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100006785","name":"Google","doi-asserted-by":"crossref","award":["Research Award"],"award-info":[{"award-number":["Research Award"]}],"id":[{"id":"10.13039\/100006785","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Trans. Soc. Comput."],"published-print":{"date-parts":[[2019,3,31]]},"abstract":"<jats:p>\n            Crowdsourcing has become an integral part of many systems and services that deliver high-quality results for complex tasks such as data linkage, schema matching, and content annotation. A standard function of such\n            <jats:italic>crowd-powered<\/jats:italic>\n            systems is to publish a batch of tasks on a crowdsourcing platform automatically and to collect the results once the workers complete them. Currently, these systems provide limited guarantees over the execution time, which is problematic for many applications. Timely completion may even be impossible to guarantee due to factors specific to the crowdsourcing platform, such as the availability of workers and concurrent tasks. In our previous work, we presented the architecture of a crowd-powered system that reshapes the interaction mechanism with the crowd. Specifically, we studied a push-crowdsourcing model whereby the workers receive tasks instead of selecting them from a portal. Based on this interaction model, we employed scheduling techniques similar to those found in distributed computing infrastructures to automate the task assignment process. In this work, we first devise a generic scheduling strategy that supports both fairness and deadline-awareness. Second, to complement the proof-of-concept experiments previously performed with the crowd, we present an extensive set of simulations meant to analyze the properties of the proposed scheduling algorithms in an environment with thousands of workers and tasks. Our experimental results show that, by accounting for human factors, micro-task scheduling can achieve fairness for best-effort batches and boosts production batches.\n          <\/jats:p>","DOI":"10.1145\/3301003","type":"journal-article","created":{"date-parts":[[2019,2,22]],"date-time":"2019-02-22T17:01:44Z","timestamp":1550854904000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Deadline-Aware Fair Scheduling for Multi-Tenant Crowd-Powered Systems"],"prefix":"10.1145","volume":"2","author":[{"given":"Djellel","family":"Difallah","sequence":"first","affiliation":[{"name":"New York University, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alessandro","family":"Checco","sequence":"additional","affiliation":[{"name":"University of Sheffield, Sheffield, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gianluca","family":"Demartini","sequence":"additional","affiliation":[{"name":"University of Queensland, Brisbane, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philippe","family":"Cudr\u00e9-Mauroux","sequence":"additional","affiliation":[{"name":"University of Fribourg, Fribourg, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,2,21]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Baeza-Yates","author":"Alonso Omar","year":"2011"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871515"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187950"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054186"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047201"},{"key":"e_1_2_1_6_1","unstructured":"Michael S. Bernstein David R. Karger Robert C. Miller and Joel Brandt. 2012. Analytic methods for optimizing realtime crowdsourcing. arXiv preprint arXiv:1204.2995.  Michael S. Bernstein David R. Karger Robert C. Miller and Joel Brandt. 2012. Analytic methods for optimizing realtime crowdsourcing. arXiv preprint arXiv:1204.2995."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866080"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2452376.2452451"},{"key":"e_1_2_1_9_1","volume-title":"Retrieved","author":"Chen Edwin","year":"2013"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1989.559777"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837889"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0057410"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187900"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-013-0324-z"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings CIDR. www.cidrdb.org.","author":"Demartini Gianluca","year":"2013"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2525314.2525370"},{"key":"e_1_2_1_17_1","unstructured":"Ernesto Diaz-Aviles and Ricardo Kawase. 2012. Exploiting Twitter as a social channel for human computation. In CrowdSearch. 15--19.  Ernesto Diaz-Aviles and Ricardo Kawase. 2012. Exploiting Twitter as a social channel for human computation. In CrowdSearch. 15--19."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings 2nd AAAI Conference on Human Computation and Crowdsourcing. 50--58","author":"Difallah Djellel","year":"2014"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741685"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883030"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159661"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488421"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 11th AAAI Conference on Human Computation (AAAIWS\u201911-11)","author":"Faradani Siamak","year":"2011"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989331"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings NSDI\u201911","author":"Ghodsi Ali","year":"2011"},{"key":"e_1_2_1_26_1","volume-title":"Franklin","author":"Haas Daniel","year":"2017"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.14778\/2856318.2856331"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 5th AAAI Conference on Human Computation and Crowdsourcing (HCOMP","author":"Ting-Hao","year":"2017"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1869086.1869094"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/1622716.1622723"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25535-9_20"},{"key":"e_1_2_1_33_1","volume-title":"An Axiomatic Theory of Fairness in Network Resource Allocation. In 2010 Proceedings IEEE INFOCOM. 1--9.","author":"Lan Tian","year":"2010"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/2049960"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5244\/C.18.98"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1561\/1900000044"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883070"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/2735471.2735474"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-36071-8_29"},{"key":"e_1_2_1_41_1","volume-title":"ICML Workshop on \u201cMachine Learning Meets Crowdsourcing.\u201d PMLR, 1--9.","author":"Rajan Vaibhav","year":"2013"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2018","author":"Rao Akshay","year":"2018"},{"key":"e_1_2_1_43_1","volume-title":"Optimization in knowledge-intensive crowdsourcing. CoRR abs\/1401.1302","author":"Roy Senjuti Basu","year":"2014"},{"key":"e_1_2_1_44_1","volume-title":"HCOMP (Works in Progress \/ Demos) (AAAI Workshops)","volume":"18","author":"Rzeszotarski Jeffrey M.","year":"2013"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837905"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488490"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2037373.2037418"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985733"},{"key":"e_1_2_1_49_1","unstructured":"Jing Wang Siamak Faridani and P. Ipeirotis. 2011. Estimating the completion time of crowdsourced tasks using survival analysis models. Crowdsourcing for Search and Data Mining (CSDM 2011) 31 (2011) 31--38.  Jing Wang Siamak Faridani and P. Ipeirotis. 2011. Estimating the completion time of crowdsourced tasks using survival analysis models. Crowdsourcing for Search and Data Mining (CSDM 2011) 31 (2011) 31--38."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350263"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 2nd AAAI Conference on Human Computation (HCOMP). AAAI Press, 234--241","author":"Yin Ming","year":"2014"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1755913.1755940"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536360.2536374"}],"container-title":["ACM Transactions on Social Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3301003","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3301003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:23Z","timestamp":1750206323000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3301003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,21]]},"references-count":51,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,3,31]]}},"alternative-id":["10.1145\/3301003"],"URL":"https:\/\/doi.org\/10.1145\/3301003","relation":{},"ISSN":["2469-7818","2469-7826"],"issn-type":[{"value":"2469-7818","type":"print"},{"value":"2469-7826","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,21]]},"assertion":[{"value":"2017-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-02-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}