{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,5]],"date-time":"2025-10-05T04:36:00Z","timestamp":1759638960171,"version":"3.41.0"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2018,1,10]],"date-time":"2018-01-10T00:00:00Z","timestamp":1515542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001866","name":"Fonds National de la Recherche Luxembourg","doi-asserted-by":"publisher","award":["8734708"],"award-info":[{"award-number":["8734708"]}],"id":[{"id":"10.13039\/501100001866","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Trans. Soc. Comput."],"published-print":{"date-parts":[[2018,3,31]]},"abstract":"<jats:p>\n            We introduce the problem of Task Assignment and Sequencing, which models online optimization in expert crowdsourcing settings that involve non-decomposable macrotasks. Non-decomposition is a property of certain types of complex problems, like the formulation of an R&amp;D approach or the definition of a research methodology, which cannot be handled through the \"divide-and-conquer\" approach typically used in microtask crowdsourcing. In contrast to splitting the macrotask to multiple microtasks and allocating them to several workers in parallel, our model supports the sequential improvement of the macrotask one worker at a time, across distinct time slots of a given timeline, until a sufficient quality level is achieved. Our model assumes an online environment where expert workers are available only at specific time slots and worker\/task arrivals are not known\n            <jats:italic>a priori<\/jats:italic>\n            . With respect to this setting, we propose TAS-ONLINE, an online algorithm that aims to complete as many tasks as possible within budget, required quality, and a given timeline, without any future input information regarding job release dates or worker availabilities. Experimental results comparing TAS-ONLINE to five benchmarks show that it achieves more completed jobs, lower flow times, and higher job quality. This work bears practical implications for providing performance and quality guarantees to expert crowdsourcing platforms that wish to integrate non-decomposable macrotasks into their offered services.\n          <\/jats:p>","DOI":"10.1145\/3140459","type":"journal-article","created":{"date-parts":[[2018,4,5]],"date-time":"2018-04-05T20:10:25Z","timestamp":1522959025000},"page":"1-33","source":"Crossref","is-referenced-by-count":28,"title":["Online Sequencing of Non-Decomposable Macrotasks in Expert Crowdsourcing"],"prefix":"10.1145","volume":"1","author":[{"given":"Heinz","family":"Schmitz","sequence":"first","affiliation":[{"name":"Hochschule Trier, Trier, Germany"}]},{"given":"Ioanna","family":"Lykourentzou","sequence":"additional","affiliation":[{"name":"Luxembourg Institute of Science and Technology, Esch-sur-Alzette, Luxembourg"}]}],"member":"320","published-online":{"date-parts":[[2018,1,10]]},"reference":[{"volume-title":"The Innovation Algorithm: TRIZ, Systematic Innovation and Technical Creativity","author":"Altshuller Genrich","key":"e_1_2_1_1_1","unstructured":"Genrich Altshuller . 2005. The Innovation Algorithm: TRIZ, Systematic Innovation and Technical Creativity . Worchester, MA : Technical Innovation Center . Genrich Altshuller. 2005. The Innovation Algorithm: TRIZ, Systematic Innovation and Technical Creativity. Worchester, MA: Technical Innovation Center."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145382"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jom.2009.02.002"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the VLDB Workshop on Databases and Crowdsourcing (DBCrowd\u201913)","author":"Roy Senjuti Basu","year":"2013","unstructured":"Senjuti Basu Roy , Ioanna Lykourentzou , Saravanan Thirumuruganathan , Sihem Amer-Yahia , and Gautam Das . 2013 . Crowds, not drones: Modeling human factors in interactive crowdsourcing . In Proceedings of the VLDB Workshop on Databases and Crowdsourcing (DBCrowd\u201913) , Reynold Cheng, Anish Das Sarma, Silviu Maniu, and Pierre Senellart (Eds.). 39--42. Senjuti Basu Roy, Ioanna Lykourentzou, Saravanan Thirumuruganathan, Sihem Amer-Yahia, and Gautam Das. 2013. Crowds, not drones: Modeling human factors in interactive crowdsourcing. In Proceedings of the VLDB Workshop on Databases and Crowdsourcing (DBCrowd\u201913), Reynold Cheng, Anish Das Sarma, Silviu Maniu, and Pierre Senellart (Eds.). 39--42."},{"key":"e_1_2_1_5_1","unstructured":"Senjuti Basu Roy Ioanna Lykourentzou Saravanan Thirumuruganathan Sihem Amer-Yahia and Gautam Das. 2014. Optimization in knowledge-intensive crowdsourcing. ArXiv e-prints. http:\/\/adsabs.harvard.edu\/abs\/2014arXiv1401.1302B.  Senjuti Basu Roy Ioanna Lykourentzou Saravanan Thirumuruganathan Sihem Amer-Yahia and Gautam Das. 2014. Optimization in knowledge-intensive crowdsourcing. ArXiv e-prints. http:\/\/adsabs.harvard.edu\/abs\/2014arXiv1401.1302B."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-015-0385-2"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047201"},{"key":"e_1_2_1_8_1","volume-title":"Analytic methods for optimizing realtime crowdsourcing. CoRR abs\/1204.2995","author":"Bernstein Michael S.","year":"2012","unstructured":"Michael S. Bernstein , David R. Karger , Robert C. Miller , and Joel Brandt . 2012. Analytic methods for optimizing realtime crowdsourcing. CoRR abs\/1204.2995 ( 2012 ). http:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1204.html#abs-1204-2995. Michael S. Bernstein, David R. Karger, Robert C. Miller, and Joel Brandt. 2012. Analytic methods for optimizing realtime crowdsourcing. CoRR abs\/1204.2995 (2012). http:\/\/dblp.uni-trier.de\/db\/journals\/corr\/corr1204.html#abs-1204-2995."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866078"},{"volume-title":"Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201915)","author":"Biswas Arpita","key":"e_1_2_1_10_1","unstructured":"Arpita Biswas , Shweta Jain , Debmalya Mandal , and Y. Narahari . 2015. A truthful budget feasible multi-armed bandit mechanism for crowdsourcing time critical tasks . In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201915) . International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1101--1109. Arpita Biswas, Shweta Jain, Debmalya Mandal, and Y. Narahari. 2015. A truthful budget feasible multi-armed bandit mechanism for crowdsourcing time critical tasks. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201915). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1101--1109."},{"volume-title":"Online Computation and Competitive Analysis","author":"Borodin Allan","key":"e_1_2_1_11_1","unstructured":"Allan Borodin and Ran El-Yaniv . 2005. Online Computation and Competitive Analysis . Cambridge University Press . Allan Borodin and Ran El-Yaniv. 2005. Online Computation and Competitive Analysis. Cambridge University Press."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2014.9"},{"volume-title":"Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing.","author":"Bragg Jonathan","key":"e_1_2_1_13_1","unstructured":"Jonathan Bragg , Andrey Kolobov , Mausam, and Daniel S. Weld . 2014. Parallel task routing for crowdsourcing . In Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing. Jonathan Bragg, Andrey Kolobov, Mausam, and Daniel S. Weld. 2014. Parallel task routing for crowdsourcing. In Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820023"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702146"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466265"},{"volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI10)","author":"Dai Peng","key":"e_1_2_1_17_1","unstructured":"Peng Dai , Mausam Daniel , and S. Weld . 2010. Decision-theoretic control of crowd-sourced workflows . In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI10) . Peng Dai, Mausam Daniel, and S. Weld. 2010. Decision-theoretic control of crowd-sourced workflows. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI10)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063504.2063507"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883030"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145355"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753688"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/SFCS.1975.21"},{"key":"e_1_2_1_23_1","first-page":"1","article-title":"CrowdOp: Query optimization for declarative crowdsourcing systems","volume":"99","author":"Fan Ju","year":"2015","unstructured":"Ju Fan , Meihui Zhang , Stanley Kok , Meiyu Lu , and Beng Chin Ooi . 2015 . CrowdOp: Query optimization for declarative crowdsourcing systems . IEEE Trans. Knowl. Data Eng. 99 (2015), 1 -- 1 . Ju Fan, Meihui Zhang, Stanley Kok, Meiyu Lu, and Beng Chin Ooi. 2015. CrowdOp: Query optimization for declarative crowdsourcing systems. IEEE Trans. Knowl. Data Eng. 99 (2015), 1--1.","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"e_1_2_1_24_1","volume-title":"Human Computation (AAAI Workshops)","volume":"11","author":"Faradani Siamak","unstructured":"Siamak Faradani , Bjoern Hartmann , and Panagiotis G. Ipeirotis . 2011. What\u2019s the right price? Pricing tasks for finishing on time . In Human Computation (AAAI Workshops) , Vol. WS-11- 11 . AAAI. Siamak Faradani, Bjoern Hartmann, and Panagiotis G. Ipeirotis. 2011. What\u2019s the right price? Pricing tasks for finishing on time. In Human Computation (AAAI Workshops), Vol. WS-11-11. AAAI."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.respol.2013.09.006"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2567948.2577311"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201916)","author":"Goto Shinsuke","year":"2016","unstructured":"Shinsuke Goto , Toru Ishida , and Donghui Lin . 2016 . Understanding crowdsourcingworkflow: Modeling and optimizing iterative and parallel processes . In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201916) . Shinsuke Goto, Toru Ishida, and Donghui Lin. 2016. Understanding crowdsourcingworkflow: Modeling and optimizing iterative and parallel processes. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201916)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824062"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the 26th AAAI Conference on Artificial Intelligence.","author":"Ho Chien-Ju","year":"2012","unstructured":"Chien-Ju Ho and Jennifer Wortman Vaughan . 2012 . Online task assignment in crowdsourcing markets . In Proceedings of the 26th AAAI Conference on Artificial Intelligence. Chien-Ju Ho and Jennifer Wortman Vaughan. 2012. Online task assignment in crowdsourcing markets. In Proceedings of the 26th AAAI Conference on Artificial Intelligence."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807342.1807376"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1080\/13571516.2016.1156247"},{"key":"e_1_2_1_32_1","unstructured":"Muhammad Imran Ioanna Lykourentzou Yannick Naudet and Carlos Castillo. 2013. Engineering crowdsourced stream processing systems. ArXiv e-prints. http:\/\/adsabs.harvard.edu\/abs\/2013arXiv1310.5463I.  Muhammad Imran Ioanna Lykourentzou Yannick Naudet and Carlos Castillo. 2013. Engineering crowdsourced stream processing systems. ArXiv e-prints. http:\/\/adsabs.harvard.edu\/abs\/2013arXiv1310.5463I."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2567988"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/0142-694X(91)90003-F"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2005.05.019"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201914)","author":"Karanam Saraschandra","year":"2014","unstructured":"Saraschandra Karanam , Deepthi Chander , L. Elisa Celis , Koustuv Dasgupta , and Vaibhav Rajan . 2014 . Adaptive performance optimization over crowd labor channels . In Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201914) . Saraschandra Karanam, Deepthi Chander, L. Elisa Celis, Koustuv Dasgupta, and Vaibhav Rajan. 2014. Adaptive performance optimization over crowd labor channels. In Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201914)."},{"key":"e_1_2_1_37_1","volume-title":"Budget-optimal task allocation for reliable crowdsourcing systems. CoRR abs\/1110.3564","author":"Karger David R.","year":"2011","unstructured":"David R. Karger , Sewoong Oh , and Devavrat Shah . 2011. Budget-optimal task allocation for reliable crowdsourcing systems. CoRR abs\/1110.3564 ( 2011 ). David R. Karger, Sewoong Oh, and Devavrat Shah. 2011. Budget-optimal task allocation for reliable crowdsourcing systems. CoRR abs\/1110.3564 (2011)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4684-2001-2_9"},{"volume-title":"Knapsack Problems","author":"Kellerer Hans","key":"e_1_2_1_39_1","unstructured":"Hans Kellerer , Ulrich Pferschy , and David Pisinger . 2013. Knapsack Problems . Springer Science 8 Business Media. Hans Kellerer, Ulrich Pferschy, and David Pisinger. 2013. Knapsack Problems. Springer Science 8 Business Media."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.4108\/icst.collaboratecom.2012.250434"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531638"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998196"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357127"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-218X(99)00202-4"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145354"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380122"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837907"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2009.08.003"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201915)","author":"Lykourentzou Ioanna","year":"2015","unstructured":"Ioanna Lykourentzou and Heinz Schmitz . 2015 . An Online Approach to Task Assignment and Sequencing in Expert Crowdsourcing . In Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201915) . Ioanna Lykourentzou and Heinz Schmitz. 2015. An Online Approach to Task Assignment and Sequencing in Expert Crowdsourcing. In Proceedings of the 3rd AAAI Conference on Human Computation and Crowdsourcing (HCOMP\u201915)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsis.2013.07.004"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01585758"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 2nd Workshop on Social Computing and User Generated Content. ACM Conference on Electronic Commerce (ACM-EC","author":"Minder Patrick","year":"2012","unstructured":"Patrick Minder , Sven Seuken , Abraham Bernstein , and Mengia Zollinger . 2012 . Crowdmanager-combinatorial allocation and pricing of crowdsourcing tasks with time constraints . In Proceedings of the 2nd Workshop on Social Computing and User Generated Content. ACM Conference on Electronic Commerce (ACM-EC 2012). Patrick Minder, Sven Seuken, Abraham Bernstein, and Mengia Zollinger. 2012. Crowdmanager-combinatorial allocation and pricing of crowdsourcing tasks with time constraints. In Proceedings of the 2nd Workshop on Social Computing and User Generated Content. ACM Conference on Electronic Commerce (ACM-EC 2012)."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2505755"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470745"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35311-6_16"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1287\/orsc.1040.0093"},{"key":"e_1_2_1_59_1","volume-title":"Akash Das Sarma, and Vipul Venkataraman","author":"Parameswaran Aditya","year":"2016","unstructured":"Aditya Parameswaran , Akash Das Sarma, and Vipul Venkataraman . 2016 . Optimizing Open-Ended Crowdsourcing: The Next Frontier in Crowdsourced Data Management. ArXiv e-prints. http:\/\/adsabs.harvard.edu\/abs\/2016arXiv161005377P. Aditya Parameswaran, Akash Das Sarma, and Vipul Venkataraman. 2016. Optimizing Open-Ended Crowdsourcing: The Next Frontier in Crowdsourced Data Management. ArXiv e-prints. http:\/\/adsabs.harvard.edu\/abs\/2016arXiv161005377P."},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the ACM Conference on Electronic Commerce (ACM-EC\u201912)","author":"Rajan Vaibhav","year":"2013","unstructured":"Vaibhav Rajan , Sakyajit Bhattacharya , Elisa L. Celis , Deepthi Chander , Koustuv Dasgupta , and Sarachandra Karanam . 2013 . CrowdControl: An online learning approach for optimal task scheduling in a dynamic crowd platform . In Proceedings of the ACM Conference on Electronic Commerce (ACM-EC\u201912) . Vaibhav Rajan, Sakyajit Bhattacharya, Elisa L. Celis, Deepthi Chander, Koustuv Dasgupta, and Sarachandra Karanam. 2013. CrowdControl: An online learning approach for optimal task scheduling in a dynamic crowd platform. In Proceedings of the ACM Conference on Electronic Commerce (ACM-EC\u201912)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647409"},{"key":"e_1_2_1_62_1","volume-title":"Proceedings of the 5th International Conference on Weblogs and Social Media. http:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM11\/paper\/view\/2778","author":"Rogstadius Jakob","year":"2011","unstructured":"Jakob Rogstadius , Vassilis Kostakos , Aniket Kittur , Boris Smus , Jim Laredo , and Maja Vukovic . 2011 . An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets . In Proceedings of the 5th International Conference on Weblogs and Social Media. http:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM11\/paper\/view\/2778 . Jakob Rogstadius, Vassilis Kostakos, Aniket Kittur, Boris Smus, Jim Laredo, and Maja Vukovic. 2011. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. In Proceedings of the 5th International Conference on Weblogs and Social Media. http:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM11\/paper\/view\/2778."},{"key":"e_1_2_1_63_1","volume-title":"Managerial challenges in open innovation: A study of innovation intermediation in the chemical industry. R8D Manage. 40, 3","author":"Sieg Jan Henrik","year":"2010","unstructured":"Jan Henrik Sieg , Martin W. Wallin , and Georg Von Krogh . 2010. Managerial challenges in open innovation: A study of innovation intermediation in the chemical industry. R8D Manage. 40, 3 ( 2010 ), 281--291. Jan Henrik Sieg, Martin W. Wallin, and Georg Von Krogh. 2010. Managerial challenges in open innovation: A study of innovation intermediation in the chemical industry. R8D Manage. 40, 3 (2010), 281--291."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2851581.2856480"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2559206.2581181"},{"volume-title":"Proceedings of the 2013 9th International Conference on Information, Communications and Signal Processing (ICICS\u201913)","author":"Yu Han","key":"e_1_2_1_66_1","unstructured":"Han Yu , Zhiqi Shen , and C. Leung . 2013. Bringing reputation-awareness into crowdsourcing . In Proceedings of the 2013 9th International Conference on Information, Communications and Signal Processing (ICICS\u201913) . 1--5. Han Yu, Zhiqi Shen, and C. Leung. 2013. Bringing reputation-awareness into crowdsourcing. In Proceedings of the 2013 9th International Conference on Information, Communications and Signal Processing (ICICS\u201913). 1--5."},{"volume-title":"Crowdsourcing, Wei Li, Michael N. Huhns, Wei-Tek Tsai","author":"Yue Tao","key":"e_1_2_1_67_1","unstructured":"Tao Yue , Shaukat Ali , and Shuai Wang . 2015. An evolutionary and automated virtual team making approach for crowdsourcing platforms . In Crowdsourcing, Wei Li, Michael N. Huhns, Wei-Tek Tsai , and Wenjun Wu (Eds.). Springer , Berlin , 113--130. Tao Yue, Shaukat Ali, and Shuai Wang. 2015. An evolutionary and automated virtual team making approach for crowdsourcing platforms. In Crowdsourcing, Wei Li, Michael N. Huhns, Wei-Tek Tsai, and Wenjun Wu (Eds.). Springer, Berlin, 113--130."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/iThings\/CPSCom.2011.128"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2442657.2442661"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-34481-7_63"},{"key":"e_1_2_1_71_1","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","volume":"1","author":"Omar","unstructured":"Omar F. Zaidan and Chris Callison-Burch. 2011. Crowdsourcing translation: Professional quality from non-professionals . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (HLT\u201911). Association for Computational Linguistics, Stroudsburg, PA, 1220--1229. Omar F. Zaidan and Chris Callison-Burch. 2011. Crowdsourcing translation: Professional quality from non-professionals. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (HLT\u201911). Association for Computational Linguistics, Stroudsburg, PA, 1220--1229."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207708"}],"container-title":["ACM Transactions on Social Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3140459","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3140459","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:11:20Z","timestamp":1750212680000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3140459"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,10]]},"references-count":72,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,3,31]]}},"alternative-id":["10.1145\/3140459"],"URL":"https:\/\/doi.org\/10.1145\/3140459","relation":{},"ISSN":["2469-7818","2469-7826"],"issn-type":[{"type":"print","value":"2469-7818"},{"type":"electronic","value":"2469-7826"}],"subject":[],"published":{"date-parts":[[2018,1,10]]}}}