{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,12]],"date-time":"2025-07-12T01:20:44Z","timestamp":1752283244534},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"9","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:p>\n            Crowdsourcing for Human Intelligence Tasks (HIT) has been widely used to crowdsource human knowledge, such as image annotation for machine learning. We use a public blockchain to play the role of traditional centralized HIT systems, such that the blockchain deals with cryptocurrency payments and acts as a trustworthy judge to resolve disputes between a worker and a requester in a decentralized setting, preventing false-reporting and free-riding. Our approach neither uses expensive cryptographic tools, such as zero-knowledge proofs, nor sends the worker's answers to the blockchain. Compared with prior works, our approach significantly reduces on-chain cost: it only requires O(1) on-chain storage and O(log\n            <jats:italic>N<\/jats:italic>\n            ) smart contract computation, where\n            <jats:italic>N<\/jats:italic>\n            is the question number of a HIT. Additionally, our approach uses known answers or gold standards to determine the worker's answer quality. To motivate the requester to use honest known answers, the requester cannot learn the worker's answers if the answer quality does not meet the requirement. We further provide formal security definitions for our decentralized HIT and prove security of our construction.\n          <\/jats:p>","DOI":"10.14778\/3538598.3538609","type":"journal-article","created":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T17:12:31Z","timestamp":1658941951000},"page":"1875-1888","source":"Crossref","is-referenced-by-count":9,"title":["Decentralized crowdsourcing for human intelligence tasks with efficient on-chain cost"],"prefix":"10.14778","volume":"15","author":[{"given":"Yihuai","family":"Liang","sequence":"first","affiliation":[{"name":"Inha University, Incheon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Li","sequence":"additional","affiliation":[{"name":"Inha University, Incheon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Byeong-Seok","family":"Shin","sequence":"additional","affiliation":[{"name":"Inha University, Incheon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,7,27]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"AMT. Retrieved","author":"AMT","year":"2017","unstructured":"AMT 2017 . Tutorial: How to label thousands of images using the crowd . AMT. Retrieved May 02, 2022 from https:\/\/blog.mturk.com\/tutorial-how-to-label-thousands-of-images-using-the-crowd-bea164ccbefc AMT 2017. Tutorial: How to label thousands of images using the crowd. AMT. Retrieved May 02, 2022 from https:\/\/blog.mturk.com\/tutorial-how-to-label-thousands-of-images-using-the-crowd-bea164ccbefc"},{"key":"e_1_2_1_2_1","volume-title":"AMT. Retrieved","author":"AMT","year":"2017","unstructured":"AMT 2017 . Tutorial: How to verify crowdsourced training data using a Known Answer Review Policy . AMT. Retrieved May 02, 2022 from https:\/\/blog.mturk.com\/tutorial-how-to-verify-crowdsourced-training-data-using-a-known-answer-review-policy-85596fb55ed AMT 2017. Tutorial: How to verify crowdsourced training data using a Known Answer Review Policy. AMT. Retrieved May 02, 2022 from https:\/\/blog.mturk.com\/tutorial-how-to-verify-crowdsourced-training-data-using-a-known-answer-review-policy-85596fb55ed"},{"key":"e_1_2_1_3_1","volume-title":"EOSeoul. Retrieved","year":"2018","unstructured":"EOSeoul 2018 . 2nd result of EOSIO TPS test by EOSeoul --- Verification of BlockOne test guide and JIT test . EOSeoul. Retrieved October 15, 2021 from https:\/\/medium.com\/eoseoul\/2nd-result-of-eosio-tps-test-by-eoseoul-verification-of-blockone-test-guide-and-jit-test-a1f4157c2aa9 EOSeoul 2018. 2nd result of EOSIO TPS test by EOSeoul --- Verification of BlockOne test guide and JIT test. EOSeoul. Retrieved October 15, 2021 from https:\/\/medium.com\/eoseoul\/2nd-result-of-eosio-tps-test-by-eoseoul-verification-of-blockone-test-guide-and-jit-test-a1f4157c2aa9"},{"key":"e_1_2_1_4_1","unstructured":"AMT 2022. Amazon Mechanical Turk. AMT. Retrieved May 02 2022 from https:\/\/www.mturk.com  AMT 2022. Amazon Mechanical Turk. AMT. Retrieved May 02 2022 from https:\/\/www.mturk.com"},{"key":"e_1_2_1_5_1","volume-title":"AMT. Retrieved","author":"AMT","year":"2022","unstructured":"AMT 2022 . Amazon's MTurk pricing . AMT. Retrieved May 02, 2022 from https:\/\/www.mturk.com\/pricing AMT 2022. Amazon's MTurk pricing. AMT. Retrieved May 02, 2022 from https:\/\/www.mturk.com\/pricing"},{"key":"e_1_2_1_6_1","volume-title":"CoinMarketCap. Retrieved","year":"2022","unstructured":"CoinMarketCap 2022 . Cryptocurrency Market Capitalization . CoinMarketCap. Retrieved October 08, 2021 from https:\/\/coinmarketcap.com CoinMarketCap 2022. Cryptocurrency Market Capitalization. CoinMarketCap. Retrieved October 08, 2021 from https:\/\/coinmarketcap.com"},{"key":"e_1_2_1_7_1","volume-title":"Etherscan. Retrieved","author":"Etherscan","year":"2022","unstructured":"Etherscan 2022 . Ether Daily Price (USD) Chart . Etherscan. Retrieved May 02, 2022 from https:\/\/etherscan.io\/chart\/etherprice Etherscan 2022. Ether Daily Price (USD) Chart. Etherscan. Retrieved May 02, 2022 from https:\/\/etherscan.io\/chart\/etherprice"},{"key":"e_1_2_1_8_1","volume-title":"Ethereum. Retrieved","author":"Ethereum","year":"2022","unstructured":"Ethereum 2022 . Understanding the Eth2 vision . Ethereum. Retrieved May 02, 2022 from https:\/\/ethereum.org\/en\/eth2\/vision Ethereum 2022. Understanding the Eth2 vision. Ethereum. Retrieved May 02, 2022 from https:\/\/ethereum.org\/en\/eth2\/vision"},{"key":"e_1_2_1_9_1","volume-title":"Healthcare","author":"Agbo Cornelius C","unstructured":"Cornelius C Agbo , Qusay H Mahmoud , and J Mikael Eklund . 2019. Blockchain technology in healthcare: a systematic review . In Healthcare , Vol. 7 . Multidisciplinary Digital Publishing Institute , 56. Cornelius C Agbo, Qusay H Mahmoud, and J Mikael Eklund. 2019. Blockchain technology in healthcare: a systematic review. In Healthcare, Vol. 7. Multidisciplinary Digital Publishing Institute, 56."},{"key":"e_1_2_1_10_1","first-page":"244","article-title":"Blockchain for Internet of Things (IoT) research issues challenges & future directions: A review","volume":"19","author":"Alamri Malak","year":"2019","unstructured":"Malak Alamri , NZ Jhanjhi , and Mamoona Humayun . 2019 . Blockchain for Internet of Things (IoT) research issues challenges & future directions: A review . Int. J. Comput. Sci. Netw. Secur 19 (2019), 244 -- 258 . Malak Alamri, NZ Jhanjhi, and Mamoona Humayun. 2019. Blockchain for Internet of Things (IoT) research issues challenges & future directions: A review. Int. J. Comput. Sci. Netw. Secur 19 (2019), 244--258.","journal-title":"Int. J. Comput. Sci. Netw. Secur"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3319535.3363213"},{"key":"e_1_2_1_12_1","unstructured":"Vitalik Buterin et al. 2014. Ethereum white paper: a next generation smart contract & decentralized application platform. First version 53 (2014).  Vitalik Buterin et al. 2014. Ethereum white paper: a next generation smart contract & decentralized application platform. First version 53 (2014)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-78381-9_11"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-70936-7_4"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134010"},{"key":"e_1_2_1_16_1","volume-title":"Conducting clinical research using crowdsourced convenience samples. Annual review of clinical psychology 12","author":"Chandler Jesse","year":"2016","unstructured":"Jesse Chandler and Danielle Shapiro . 2016. Conducting clinical research using crowdsourced convenience samples. Annual review of clinical psychology 12 ( 2016 ), 53--81. Jesse Chandler and Danielle Shapiro. 2016. Conducting clinical research using crowdsourced convenience samples. Annual review of clinical psychology 12 (2016), 53--81."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2920987"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319889"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557011"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243857"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2019.00020"},{"key":"e_1_2_1_22_1","volume-title":"Foundations of cryptography","author":"Goldreich Oded","unstructured":"Oded Goldreich . 2009. Foundations of cryptography : volume 2 , basic applications. Cambridge university press . Oded Goldreich. 2009. Foundations of cryptography: volume 2, basic applications. Cambridge university press."},{"key":"e_1_2_1_23_1","volume-title":"Retrieved","author":"Gregg Aaron","year":"2021","unstructured":"Aaron Gregg and Drew Harwell . 2021 . Amazon Web Services' third outage in a month exposes a weak point in the Internet's backbone . Retrieved May 02, 2022 from https:\/\/www.washingtonpost.com\/business\/2021\/12\/22\/amazon-web-services-experiences-another-big-outage Aaron Gregg and Drew Harwell. 2021. Amazon Web Services' third outage in a month exposes a weak point in the Internet's backbone. Retrieved May 02, 2022 from https:\/\/www.washingtonpost.com\/business\/2021\/12\/22\/amazon-web-services-experiences-another-big-outage"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320238"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2019.2956629"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.3390\/s18113894"},{"key":"e_1_2_1_27_1","volume-title":"Retrieved","author":"Kelly Heather","year":"2012","unstructured":"Heather Kelly . 2012 . Apple account hack raises concern about cloud storage . Retrieved May 02, 2022 from https:\/\/edition.cnn.com\/2012\/08\/06\/tech\/mobile\/icloud-security-hack Heather Kelly. 2012. Apple account hack raises concern about cloud storage. Retrieved May 02, 2022 from https:\/\/edition.cnn.com\/2012\/08\/06\/tech\/mobile\/icloud-security-hack"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63688-7_12"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.55"},{"key":"e_1_2_1_30_1","volume-title":"29th USENIX Security Symposium (USENIX Security 20)","author":"Kosba Ahmed","year":"2020","unstructured":"Ahmed Kosba , Dimitrios Papadopoulos , Charalampos Papamanthou , and Dawn Song . 2020 . MIRAGE: Succinct Arguments for Randomized Algorithms with Applications to Universal zk-SNARKs . In 29th USENIX Security Symposium (USENIX Security 20) . USENIX Association, 2129--2146. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/kosba Ahmed Kosba, Dimitrios Papadopoulos, Charalampos Papamanthou, and Dawn Song. 2020. MIRAGE: Succinct Arguments for Randomized Algorithms with Applications to Universal zk-SNARKs. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 2129--2146. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/kosba"},{"key":"e_1_2_1_31_1","volume-title":"Retrieved","author":"Li Feifei","year":"2020","unstructured":"Feifei Li . 2020 . ImageNet . Retrieved May 02, 2022 from https:\/\/www.image-net.org\/about.php Feifei Li. 2020. ImageNet. Retrieved May 02, 2022 from https:\/\/www.image-net.org\/about.php"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2018.2881735"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.3390\/s20113172"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.32604\/cmc.2021.016682"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2018.00087"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS47774.2020.00084"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813659"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.57"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnca.2019.02.027"},{"key":"e_1_2_1_40_1","volume-title":"Conference on the theory and application of cryptographic techniques. Springer, 369--378","author":"Merkle Ralph C","year":"1987","unstructured":"Ralph C Merkle . 1987 . A digital signature based on a conventional encryption function . In Conference on the theory and application of cryptographic techniques. Springer, 369--378 . Ralph C Merkle. 1987. A digital signature based on a conventional encryption function. In Conference on the theory and application of cryptographic techniques. Springer, 369--378."},{"key":"e_1_2_1_41_1","volume-title":"Bitcoin: A peer-to-peer electronic cash system. Decentralized Business Review","author":"Nakamoto Satoshi","year":"2008","unstructured":"Satoshi Nakamoto . 2008 . Bitcoin: A peer-to-peer electronic cash system. Decentralized Business Review (2008), 21260. Satoshi Nakamoto. 2008. Bitcoin: A peer-to-peer electronic cash system. Decentralized Business Review (2008), 21260."},{"key":"e_1_2_1_42_1","volume-title":"Annual international cryptology conference","author":"Pedersen Torben Pryds","unstructured":"Torben Pryds Pedersen . 1991. Non-interactive and information-theoretic secure verifiable secret sharing . In Annual international cryptology conference . Springer , 129--140. Torben Pryds Pedersen. 1991. Non-interactive and information-theoretic secure verifiable secret sharing. In Annual international cryptology conference. Springer, 129--140."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3380594"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2018.2870658"},{"key":"e_1_2_1_45_1","volume-title":"European Conference on Computer Vision. Springer, 1--14","author":"Russakovsky Olga","year":"2010","unstructured":"Olga Russakovsky and Li Fei-Fei . 2010 . Attribute learning in large-scale datasets . In European Conference on Computer Vision. Springer, 1--14 . Olga Russakovsky and Li Fei-Fei. 2010. Attribute learning in large-scale datasets. In European Conference on Computer Vision. Springer, 1--14."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1080\/00207543.2018.1533261"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.36"},{"key":"e_1_2_1_48_1","volume-title":"International conference on machine learning. PMLR, 10--19","author":"Shah Nihar","year":"2015","unstructured":"Nihar Shah , Dengyong Zhou , and Yuval Peres . 2015 . Approval voting and incentives in crowdsourcing . In International conference on machine learning. PMLR, 10--19 . Nihar Shah, Dengyong Zhou, and Yuval Peres. 2015. Approval voting and incentives in crowdsourcing. In International conference on machine learning. PMLR, 10--19."},{"key":"e_1_2_1_49_1","volume-title":"Double or nothing: Multiplicative incentive mechanisms for crowdsourcing. Advances in neural information processing systems 28","author":"Shah Nihar Bhadresh","year":"2015","unstructured":"Nihar Bhadresh Shah and Dengyong Zhou . 2015. Double or nothing: Multiplicative incentive mechanisms for crowdsourcing. Advances in neural information processing systems 28 ( 2015 ), 1--9. Nihar Bhadresh Shah and Dengyong Zhou. 2015. Double or nothing: Multiplicative incentive mechanisms for crowdsourcing. Advances in neural information processing systems 28 (2015), 1--9."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319883"},{"key":"e_1_2_1_51_1","volume-title":"Enigma: Decentralized Computation Platform with Guaranteed Privacy. 425--454.","author":"Shrobe Howard","year":"2018","unstructured":"Howard Shrobe , David L. Shrier , and Alex Pentland . 2018 . Enigma: Decentralized Computation Platform with Guaranteed Privacy. 425--454. Howard Shrobe, David L. Shrier, and Alex Pentland. 2018. Enigma: Decentralized Computation Platform with Guaranteed Privacy. 425--454."},{"key":"e_1_2_1_52_1","volume-title":"Proceedings of the 2008 conference on empirical methods in natural language processing. 254--263","author":"Snow Rion","year":"2008","unstructured":"Rion Snow , Brendan O'connor , Dan Jurafsky , and Andrew Y Ng . 2008 . Cheap and fast-but is it good? evaluating non-expert annotations for natural language tasks . In Proceedings of the 2008 conference on empirical methods in natural language processing. 254--263 . Rion Snow, Brendan O'connor, Dan Jurafsky, and Andrew Y Ng. 2008. Cheap and fast-but is it good? evaluating non-expert annotations for natural language tasks. In Proceedings of the 2008 conference on empirical methods in natural language processing. 254--263."},{"key":"e_1_2_1_53_1","volume-title":"Formalizing and securing relationships on public networks. First monday","author":"Szabo Nick","year":"1997","unstructured":"Nick Szabo . 1997. Formalizing and securing relationships on public networks. First monday ( 1997 ). Nick Szabo. 1997. Formalizing and securing relationships on public networks. First monday (1997)."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2948863"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-012-0564-1"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2805837"},{"key":"e_1_2_1_57_1","volume-title":"Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151","author":"Gavin Wood","year":"2014","unstructured":"Gavin Wood et al. 2014 . Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151 , 2014 (2014), 1--32. Gavin Wood et al. 2014. Ethereum: A secure decentralised generalised transaction ledger. Ethereum project yellow paper 151, 2014 (2014), 1--32."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2015.2441031"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2019.2941735"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2018.2823705"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2018.2823705"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.5555\/2867541.2867781"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3538598.3538609","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:28:11Z","timestamp":1672219691000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3538598.3538609"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5]]},"references-count":62,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2022,5]]}},"alternative-id":["10.14778\/3538598.3538609"],"URL":"https:\/\/doi.org\/10.14778\/3538598.3538609","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2022,5]]}}}