{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,15]],"date-time":"2025-07-15T03:14:45Z","timestamp":1752549285209,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":37,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T00:00:00Z","timestamp":1597881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,8,23]]},"DOI":"10.1145\/3394486.3403369","type":"proceedings-article","created":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T23:03:59Z","timestamp":1597964639000},"page":"3172-3182","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Prediction of Hourly Earnings and Completion Time on a Crowdsourcing Platform"],"prefix":"10.1145","author":[{"given":"Anna","family":"Lioznova","sequence":"first","affiliation":[{"name":"Yandex, Moscow, Russian Fed."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexey","family":"Drutsa","sequence":"additional","affiliation":[{"name":"Yandex, Moscow, Russian Fed."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vladimir","family":"Kukushkin","sequence":"additional","affiliation":[{"name":"Huawei, Saint Petersburg, Russian Fed."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anastasia","family":"Bezzubtseva","sequence":"additional","affiliation":[{"name":"Yandex, Moscow, Russian Fed."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,8,20]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"R. Budylin A. Drutsa G. Gusev P. Serdyukov and I. Yashkov. 2018. Online evaluation for effective web service development. Tutorial at KDD'2018.  R. Budylin A. Drutsa G. Gusev P. Serdyukov and I. Yashkov. 2018. Online evaluation for effective web service development. Tutorial at KDD'2018."},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"crossref","unstructured":"Ricardo Buettner. 2015. A systematic literature review of crowdsourcing research from a human resource management perspective. In HICSS.  Ricardo Buettner. 2015. A systematic literature review of crowdsourcing research from a human resource management perspective. In HICSS.","DOI":"10.1109\/HICSS.2015.549"},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"crossref","unstructured":"Chris Callison-Burch. 2014. Crowd-Workers: Aggregating Information Across Turkers to Help Them Find Higher Paying Work. In HCOMP.  Chris Callison-Burch. 2014. Crowd-Workers: Aggregating Information Across Turkers to Help Them Find Higher Paying Work. In HCOMP.","DOI":"10.1609\/hcomp.v2i1.13198"},{"key":"e_1_3_2_2_4_1","volume-title":"Bernstein","author":"Cheng Justin","year":"2015","unstructured":"Justin Cheng , Jaime Teevan , and Michael S . Bernstein . 2015 . Measuring Crowdsourcing Effort with Error-Time Curves. In CHI. Justin Cheng, Jaime Teevan, and Michael S. Bernstein. 2015. Measuring Crowdsourcing Effort with Error-Time Curves. In CHI."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"crossref","unstructured":"Djellel Eddine Difallah Michele Catasta Gianluca Demartini Panagiotis G Ipeirotis and Philippe Cudr\u00e9-Mauroux. 2015. The dynamics of micro-task crowdsourcing: The case of amazon mturk. In WWW. 238--247.  Djellel Eddine Difallah Michele Catasta Gianluca Demartini Panagiotis G Ipeirotis and Philippe Cudr\u00e9-Mauroux. 2015. The dynamics of micro-task crowdsourcing: The case of amazon mturk. In WWW. 238--247.","DOI":"10.1145\/2740908.2744109"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"crossref","unstructured":"A. Drutsa V. Farafonova V. Fedorova O. Megorskaya E. Zerminova and O. Zhilinskaya. 2019. Practice of Efficient Data Collection via Crowdsourcing at Large-Scale. Tutorial at KDD'2019.  A. Drutsa V. Farafonova V. Fedorova O. Megorskaya E. Zerminova and O. Zhilinskaya. 2019. Practice of Efficient Data Collection via Crowdsourcing at Large-Scale. Tutorial at KDD'2019.","DOI":"10.1145\/3336191.3371875"},{"volume-title":"WSDM'2020","author":"Drutsa A.","key":"e_1_3_2_2_7_1","unstructured":"A. Drutsa , V. Fedorova , D. Ustalov , O. Megorskaya , E. Zerminova , and D. Baidakova . 2020 a. Practice of Efficient Data Collection via Crowdsourcing: Aggregation, Incremental Relabelling, and Pricing . In WSDM'2020 . 873--876. A. Drutsa, V. Fedorova, D. Ustalov, O. Megorskaya, E. Zerminova, and D. Baidakova. 2020 a. Practice of Efficient Data Collection via Crowdsourcing: Aggregation, Incremental Relabelling, and Pricing. In WSDM'2020. 873--876."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052664"},{"volume-title":"CVPR'2020","author":"Drutsa A.","key":"e_1_3_2_2_9_1","unstructured":"A. Drutsa , D. Rogachevsky , O. Megorskaya , A. Slesarev , E. Zerminova , D. Baidakova , A. Rykov , and A. Golomedov . 2020 b. Efficient Data Annotation for Self-Driving Cars via Crowdsourcing on a Large-Scale . In CVPR'2020 . A. Drutsa, D. Rogachevsky, O. Megorskaya, A. Slesarev, E. Zerminova, D. Baidakova, A. Rykov, and A. Golomedov. 2020 b. Efficient Data Annotation for Self-Driving Cars via Crowdsourcing on a Large-Scale. In CVPR'2020."},{"key":"e_1_3_2_2_10_1","volume-title":"Practical Aspects of Sensitivity in Online Experimentation with User Engagement Metrics. In CIKM'2015","author":"Drutsa Alexey","year":"2015","unstructured":"Alexey Drutsa , Anna Ufliand , and Gleb Gusev . 2015 . Practical Aspects of Sensitivity in Online Experimentation with User Engagement Metrics. In CIKM'2015 . Alexey Drutsa, Anna Ufliand, and Gleb Gusev. 2015. Practical Aspects of Sensitivity in Online Experimentation with User Engagement Metrics. In CIKM'2015."},{"volume-title":"SIGMOD'2020","author":"Drutsa A.","key":"e_1_3_2_2_11_1","unstructured":"A. Drutsa , D. Ustalov , E. Zerminova , V. Fedorova , O. Megorskaya , and D. Baidakova . 2020 c. Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing . In SIGMOD'2020 . 2623--2627. A. Drutsa, D. Ustalov, E. Zerminova, V. Fedorova, O. Megorskaya, and D. Baidakova. 2020 c. Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing. In SIGMOD'2020. 2623--2627."},{"volume-title":"Matchmakers: the new economics of multisided platforms","author":"Evans David S","key":"e_1_3_2_2_12_1","unstructured":"David S Evans and Richard Schmalensee . 2016. Matchmakers: the new economics of multisided platforms . Harvard Business Review Press . David S Evans and Richard Schmalensee. 2016. Matchmakers: the new economics of multisided platforms. Harvard Business Review Press."},{"key":"e_1_3_2_2_13_1","volume-title":"What's the Right Price? Pricing Tasks for Finishing on Time. Human comp","author":"Faradani Siamak","year":"2011","unstructured":"Siamak Faradani , Bj\u00f6rn Hartmann , and Panagiotis G Ipeirotis . 2011. What's the Right Price? Pricing Tasks for Finishing on Time. Human comp ., Vol. 11 ( 2011 ), 11. Siamak Faradani, Bj\u00f6rn Hartmann, and Panagiotis G Ipeirotis. 2011. What's the Right Price? Pricing Tasks for Finishing on Time. Human comp., Vol. 11 (2011), 11."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Tanya Goyal Tyler McDonnell Mucahid Kutlu Tamer Elsayed and Matthew Lease. 2018. Your Behavior Signals Your Reliability: Modeling Crowd Behavioral Traces to Ensure Quality Relevance Annotations. In HCOMP.  Tanya Goyal Tyler McDonnell Mucahid Kutlu Tamer Elsayed and Matthew Lease. 2018. Your Behavior Signals Your Reliability: Modeling Crowd Behavioral Traces to Ensure Quality Relevance Annotations. In HCOMP.","DOI":"10.1609\/hcomp.v6i1.13331"},{"key":"e_1_3_2_2_15_1","first-page":"3","article-title":"Investigating the Amazon Mechanical Turk Market Through Tool Design","volume":"27","author":"Hanrahan B.V","year":"2018","unstructured":"B.V Hanrahan , D. Martin , J. Willamowski , and J.M Carroll . 2018 . Investigating the Amazon Mechanical Turk Market Through Tool Design . CSCW , Vol. 27 , 3 -- 6 (2018). B.V Hanrahan, D. Martin, J. Willamowski, and J.M Carroll. 2018. Investigating the Amazon Mechanical Turk Market Through Tool Design. CSCW, Vol. 27, 3--6 (2018).","journal-title":"CSCW"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"crossref","unstructured":"Benjamin V Hanrahan Jutta K Willamowski Saiganesh Swaminathan and David B Martin. 2015. TurkBench: Rendering the market for Turkers. In HFCS.  Benjamin V Hanrahan Jutta K Willamowski Saiganesh Swaminathan and David B Martin. 2015. TurkBench: Rendering the market for Turkers. In HFCS.","DOI":"10.1145\/2702123.2702279"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Kotaro Hara Abigail Adams Kristy Milland Saiph Savage Chris Callison-Burch and Jeffrey P Bigham. 2018. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In CHI.  Kotaro Hara Abigail Adams Kristy Milland Saiph Savage Chris Callison-Burch and Jeffrey P Bigham. 2018. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In CHI.","DOI":"10.1145\/3173574.3174023"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Simo Hosio Jorge Goncalves Vili Lehdonvirta Denzil Ferreira and Vassilis Kostakos. 2014. Situated crowdsourcing using a market model. In UIST. ACM.  Simo Hosio Jorge Goncalves Vili Lehdonvirta Denzil Ferreira and Vassilis Kostakos. 2014. Situated crowdsourcing using a market model. In UIST. ACM.","DOI":"10.1145\/2642918.2647362"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1869086.1869094"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470742"},{"key":"e_1_3_2_2_21_1","volume-title":"HCOMP'2018","author":"Kaplan T.","year":"2018","unstructured":"T. Kaplan , S. Saito , K. Hara , and J. P Bigham . 2018 . Striving to earn more: a survey of work strategies and tool use among crowd workers . In HCOMP'2018 . T. Kaplan, S. Saito, K. Hara, and J. P Bigham. 2018. Striving to earn more: a survey of work strategies and tool use among crowd workers. In HCOMP'2018."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-008-0114-1"},{"key":"e_1_3_2_2_23_1","unstructured":"Pavel Kucherbaev Florian Daniel Stefano Tranquillini and Maurizio Marchese. 2016. ReLauncher: crowdsourcing micro-tasks runtime controller. In CSCWSC.  Pavel Kucherbaev Florian Daniel Stefano Tranquillini and Maurizio Marchese. 2016. ReLauncher: crowdsourcing micro-tasks runtime controller. In CSCWSC."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1561\/1900000044"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"crossref","unstructured":"David Martin Benjamin V Hanrahan Jacki O'Neill and Neha Gupta. 2014. Being a turker. In CSCWSC.  David Martin Benjamin V Hanrahan Jacki O'Neill and Neha Gupta. 2014. Being a turker. In CSCWSC.","DOI":"10.1145\/2531602.2531663"},{"volume-title":"Workshop on Social Computing and User Generated Content in ACM EC.","author":"Minder P.","key":"e_1_3_2_2_26_1","unstructured":"P. Minder , S. Seuken , A. Bernstein , and M. Zollinger . 2012. Crowdmanager-combinatorial allocation and pricing of crowdsourcing tasks with time constraints . In Workshop on Social Computing and User Generated Content in ACM EC. P. Minder, S. Seuken, A. Bernstein, and M. Zollinger. 2012. Crowdmanager-combinatorial allocation and pricing of crowdsourcing tasks with time constraints. In Workshop on Social Computing and User Generated Content in ACM EC."},{"key":"e_1_3_2_2_27_1","volume-title":"Anna Veronika Dorogush, and Andrey Gulin","author":"Prokhorenkova Liudmila","year":"2018","unstructured":"Liudmila Prokhorenkova , Gleb Gusev , Aleksandr Vorobev , Anna Veronika Dorogush, and Andrey Gulin . 2018 . CatBoost: unbiased boosting with categorical features. In Advances in Neural Information Processing Systems . 6638--6648. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. In Advances in Neural Information Processing Systems. 6638--6648."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/poi3.102"},{"key":"e_1_3_2_2_29_1","first-page":"2","article-title":"Efficient Learning-Based Recommendation Algorithms for Top-N Tasks and Top-N Workers in Large-Scale Crowdsourcing Systems","volume":"37","author":"Safran Mejdl","year":"2018","unstructured":"Mejdl Safran and Dunren Che . 2018 . Efficient Learning-Based Recommendation Algorithms for Top-N Tasks and Top-N Workers in Large-Scale Crowdsourcing Systems . ACM Transactions on Information Systems (TOIS) , Vol. 37 , 1 (2018), 2 . Mejdl Safran and Dunren Che. 2018. Efficient Learning-Based Recommendation Algorithms for Top-N Tasks and Top-N Workers in Large-Scale Crowdsourcing Systems. ACM Transactions on Information Systems (TOIS), Vol. 37, 1 (2018), 2.","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"e_1_3_2_2_30_1","volume-title":"TurkScanner: Predicting the Hourly Wage of Microtasks. In The World Wide Web Conference. ACM, 3187--3193","author":"Saito Susumu","year":"2019","unstructured":"Susumu Saito , Chun-Wei Chiang , Saiph Savage , Teppei Nakano , Tetsunori Kobayashi , and Jeffrey P Bigham . 2019 . TurkScanner: Predicting the Hourly Wage of Microtasks. In The World Wide Web Conference. ACM, 3187--3193 . Susumu Saito, Chun-Wei Chiang, Saiph Savage, Teppei Nakano, Tetsunori Kobayashi, and Jeffrey P Bigham. 2019. TurkScanner: Predicting the Hourly Wage of Microtasks. In The World Wide Web Conference. ACM, 3187--3193."},{"key":"e_1_3_2_2_31_1","unstructured":"Thimo Schulze Simone Krug and Martin Schader. 2012. Workers' task choice in crowdsourcing and human computation markets. (2012).  Thimo Schulze Simone Krug and Martin Schader. 2012. Workers' task choice in crowdsourcing and human computation markets. (2012)."},{"key":"e_1_3_2_2_32_1","volume-title":"Estimating the completion time of crowdsourced tasks using survival analysis models. Crowdsourcing for search and data mining (CSDM","author":"Wang Jing","year":"2011","unstructured":"Jing Wang , Siamak Faridani , and Panagiotis Ipeirotis . 2011. Estimating the completion time of crowdsourced tasks using survival analysis models. Crowdsourcing for search and data mining (CSDM 2011 ), Vol. 31 (2011). Jing Wang, Siamak Faridani, and Panagiotis Ipeirotis. 2011. Estimating the completion time of crowdsourced tasks using survival analysis models. Crowdsourcing for search and data mining (CSDM 2011), Vol. 31 (2011)."},{"key":"e_1_3_2_2_33_1","unstructured":"Jing Wang and Panagiotis Ipeirotis. 2013. A framework for quality assurance in crowdsourcing. (2013).  Jing Wang and Panagiotis Ipeirotis. 2013. A framework for quality assurance in crowdsourcing. (2013)."},{"key":"e_1_3_2_2_34_1","unstructured":"Chaolun Xia and Shan Muthukrishnan. 2017. Revenue-Maximizing Stable Pricing in Online Labor Markets. (2017).  Chaolun Xia and Shan Muthukrishnan. 2017. Revenue-Maximizing Stable Pricing in Online Labor Markets. (2017)."},{"key":"e_1_3_2_2_35_1","unstructured":"Yandex. Since 2014. Yandex.Toloka. http:\/\/toloka.ai  Yandex. Since 2014. Yandex.Toloka. http:\/\/toloka.ai"},{"key":"e_1_3_2_2_36_1","unstructured":"Yandex.Toloka. 2020. Dataset \"Toloka Users & Tasks\". https:\/\/toloka.ai\/datasets  Yandex.Toloka. 2020. Dataset \"Toloka Users & Tasks\". https:\/\/toloka.ai\/datasets"},{"key":"e_1_3_2_2_37_1","volume-title":"Tsinghua Science and Technology","volume":"23","author":"Zhang Y.","year":"2018","unstructured":"Y. Zhang , H. Qin , B. Li , J. Wang , S. Lee , and Zh. Huang . 2018 . Truthful mechanism for crowdsourcing task assignment . Tsinghua Science and Technology , Vol. 23 , 6 (2018). Y. Zhang, H. Qin, B. Li, J. Wang, S. Lee, and Zh. Huang. 2018. Truthful mechanism for crowdsourcing task assignment. Tsinghua Science and Technology, Vol. 23, 6 (2018)."}],"event":{"name":"KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"],"location":"Virtual Event CA USA","acronym":"KDD '20"},"container-title":["Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403369","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394486.3403369","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:29Z","timestamp":1750195889000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403369"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,20]]},"references-count":37,"alternative-id":["10.1145\/3394486.3403369","10.1145\/3394486"],"URL":"https:\/\/doi.org\/10.1145\/3394486.3403369","relation":{},"subject":[],"published":{"date-parts":[[2020,8,20]]},"assertion":[{"value":"2020-08-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}