{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,28]],"date-time":"2025-09-28T04:14:58Z","timestamp":1759032898809,"version":"3.41.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2015,5,13]],"date-time":"2015-05-13T00:00:00Z","timestamp":1431475200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"BPM4People project of the EU FP7 SME Capacities program"},{"name":"Evaluation and Enhancement of Social, Economic, and Emotional Wellbeing of Older Adults project under agreement 14.Z50.310029 of Tomsk Polytechnic University, Russia"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Web"],"published-print":{"date-parts":[[2015,5,26]]},"abstract":"<jats:p>\n            Crowdsourcing (CS) is the outsourcing of a unit of work to a crowd of people via an open call for contributions. Thanks to the availability of online CS platforms, such as Amazon Mechanical Turk or CrowdFlower, the practice has experienced a tremendous growth over the past few years and demonstrated its viability in a variety of fields, such as data collection and analysis or human computation. Yet it is also increasingly struggling with the inherent limitations of these platforms: each platform has its own\n            <jats:italic>logic<\/jats:italic>\n            of how to crowdsource work (e.g., marketplace or contest), there is only very little support for\n            <jats:italic>structured<\/jats:italic>\n            work (work that requires the coordination of multiple tasks), and it is hard to\n            <jats:italic>integrate<\/jats:italic>\n            crowdsourced tasks into state-of-the-art business process management (BPM) or information systems.\n          <\/jats:p>\n          <jats:p>\n            We attack these three shortcomings by (1) developing a flexible CS platform (we call it\n            <jats:italic>Crowd Computer<\/jats:italic>\n            , or CC) that allows one to program custom CS logics for individual and structured tasks, (2) devising a BPMN--based modeling language that allows one to program CC intuitively, (3) equipping the language with a dedicated visual editor, and (4) implementing CC on top of standard BPM technology that can easily be integrated into existing software and processes. We demonstrate the effectiveness of the approach with a case study on the crowd-based mining of mashup model patterns.\n          <\/jats:p>","DOI":"10.1145\/2746353","type":"journal-article","created":{"date-parts":[[2015,5,15]],"date-time":"2015-05-15T16:05:34Z","timestamp":1431705934000},"page":"1-43","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Modeling, Enacting, and Integrating Custom Crowdsourcing Processes"],"prefix":"10.1145","volume":"9","author":[{"given":"Stefano","family":"Tranquillini","sequence":"first","affiliation":[{"name":"University of Trento, Italy"}]},{"given":"Florian","family":"Daniel","sequence":"additional","affiliation":[{"name":"University of Trento, Italy"}]},{"given":"Pavel","family":"Kucherbaev","sequence":"additional","affiliation":[{"name":"University of Trento, Italy"}]},{"given":"Fabio","family":"Casati","sequence":"additional","affiliation":[{"name":"University of Trento, Italy"}]}],"member":"320","published-online":{"date-parts":[[2015,5,13]]},"reference":[{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047203"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2013.20"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2398857.2384663"},{"key":"e_1_2_1_5_1","volume-title":"Cloud Computing: Web-Based Dynamic IT Services","author":"Baun Christian","year":"2011","unstructured":"Christian Baun , Marcel Kunze , Jens Nimis , and Stefan Tai . 2011 . Cloud Computing: Web-Based Dynamic IT Services . Springer . Christian Baun, Marcel Kunze, Jens Nimis, and Stefan Tai. 2011. Cloud Computing: Web-Based Dynamic IT Services. Springer."},{"key":"e_1_2_1_6_1","series-title":"Lecture Notes in Computer Science","volume-title":"Web Engineering","author":"Bozzon Alessandro","unstructured":"Alessandro Bozzon , Marco Brambilla , Stefano Ceri , Andrea Mauri , and Riccardo Volonterio . 2014. Pattern-based specification of crowdsourcing applications . In Web Engineering . Lecture Notes in Computer Science , Vol. 8541 . Springer , 218--235. Alessandro Bozzon, Marco Brambilla, Stefano Ceri, Andrea Mauri, and Riccardo Volonterio. 2014. Pattern-based specification of crowdsourcing applications. In Web Engineering. Lecture Notes in Computer Science, Vol. 8541. Springer, 218--235."},{"key":"e_1_2_1_7_1","series-title":"Lecture Notes in Business Information Processing","volume-title":"Business Process Model and Notation","author":"Brambilla Marco","unstructured":"Marco Brambilla , Piero Fraternali , and Carmen Vaca . 2011. A notation for supporting social business process modeling . In Business Process Model and Notation . Lecture Notes in Business Information Processing , Vol. 95 . Springer , 88--102. Marco Brambilla, Piero Fraternali, and Carmen Vaca. 2011. A notation for supporting social business process modeling. In Business Process Model and Notation. Lecture Notes in Business Information Processing, Vol. 95. Springer, 88--102."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187980.2188014"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/2343776.2343793"},{"key":"e_1_2_1_10_1","volume-title":"Retrieved","author":"Week Crowdsourcing","year":"2014","unstructured":"Crowdsourcing Week . 2014 . The 2014 Global Crowdsourcing Pulsecheck: 1st Annual Survey Topline Results . Retrieved April 19, 2015, from http:\/\/www.slideshare.net\/crowdsourcingweek\/2014-global-crowdsourcing-pulsecheck-1st-annual-survey-topline-results. Crowdsourcing Week. 2014. The 2014 Global Crowdsourcing Pulsecheck: 1st Annual Survey Topline Results. Retrieved April 19, 2015, from http:\/\/www.slideshare.net\/crowdsourcingweek\/2014-global-crowdsourcing-pulsecheck-1st-annual-survey-topline-results."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/INMW.2009.5195939"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-55049-2","volume-title":"Mashups: Concepts, Models and Architectures","author":"Daniel Florian","year":"2014","unstructured":"Florian Daniel and Maristella Matera . 2014 . Mashups: Concepts, Models and Architectures . Springer . Florian Daniel and Maristella Matera. 2014. Mashups: Concepts, Models and Architectures. Springer."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-20511-8_37"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145355"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.v22:6\/7"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989331"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.mcm.2012.01.006"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP\u201914)","author":"Ho\u00dffeld Tobias","year":"2014","unstructured":"Tobias Ho\u00dffeld , Matthias Hirth , Pavel Korshunov , Philippe Hanhart , Bruno Gardlo , Christian Keimel , and Christian Timmerer . 2014 a. Survey of Web-based crowdsourcing frameworks for subjective quality assessment . In Proceedings of the 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP\u201914) . 1--6. Tobias Ho\u00dffeld, Matthias Hirth, Pavel Korshunov, Philippe Hanhart, Bruno Gardlo, Christian Keimel, and Christian Timmerer. 2014a. Survey of Web-based crowdsourcing frameworks for subjective quality assessment. In Proceedings of the 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP\u201914). 1--6."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 2011 23rd International Teletraffic Congress (ITC\u201911)","author":"Ho\u00dffeld Tobias","year":"2011","unstructured":"Tobias Ho\u00dffeld , Matthias Hirth , and Phuoc Tran-Gia . 2011 . Modeling of crowdsourcing platforms and granularity of work organization in Future Internet . In Proceedings of the 2011 23rd International Teletraffic Congress (ITC\u201911) . 142--149. Tobias Ho\u00dffeld, Matthias Hirth, and Phuoc Tran-Gia. 2011. Modeling of crowdsourcing platforms and granularity of work organization in Future Internet. In Proceedings of the 2011 23rd International Teletraffic Congress (ITC\u201911). 142--149."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2013.2291663"},{"key":"e_1_2_1_22_1","volume-title":"Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business","author":"Howe Jeff","year":"2008","unstructured":"Jeff Howe . 2008 . Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business . Crown Publishing Group , New York, NY . Jeff Howe. 2008. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of Business. Crown Publishing Group, New York, NY."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1869086.1869094"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837906"},{"key":"e_1_2_1_25_1","series-title":"Lecture Notes in Business Information Processing","volume-title":"Business Process Management Workshops","author":"Johannesson Paul","unstructured":"Paul Johannesson , Birger Andersson , and Petia Wohed . 2009. Business process management with social software systems\u2014a new paradigm for work organisation . In Business Process Management Workshops . Lecture Notes in Business Information Processing . Springer , 659--665. Paul Johannesson, Birger Andersson, and Petia Wohed. 2009. Business process management with social software systems\u2014a new paradigm for work organisation. In Business Process Management Workshops. Lecture Notes in Business Information Processing. Springer, 659--665."},{"key":"e_1_2_1_26_1","unstructured":"Diane Jordan and John Evdemon. 2007. Web Services Business Process Execution Language Version 2.0. OASIS. http:\/\/docs.oasis-open.org\/wsbpel\/2.0\/OS\/wsbpel-v2.0-OS.html.  Diane Jordan and John Evdemon. 2007. Web Services Business Process Execution Language Version 2.0. OASIS. http:\/\/docs.oasis-open.org\/wsbpel\/2.0\/OS\/wsbpel-v2.0-OS.html."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.4108\/icst.collaboratecom.2012.250434"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145357"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441923"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1057\/jit.2009.21"},{"key":"e_1_2_1_32_1","series-title":"Lecture Notes in Business Information Processing","volume-title":"Business Process Management Workshops","author":"Kucherbaev Pavel","unstructured":"Pavel Kucherbaev , Stefano Tranquillini , Florian Daniel , Fabio Casati , Maurizio Marchese , Marco Brambilla , and Piero Fraternali . 2013. Business processes for the Crowd Computer . In Business Process Management Workshops . Lecture Notes in Business Information Processing , Vol. 132 . Springer , 256--267. Pavel Kucherbaev, Stefano Tranquillini, Florian Daniel, Fabio Casati, Maurizio Marchese, Marco Brambilla, and Piero Fraternali. 2013. Business processes for the Crowd Computer. In Business Process Management Workshops. Lecture Notes in Business Information Processing, Vol. 132. Springer, 256--267."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145354"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.5555\/1927229.1927276"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837907"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866040"},{"key":"e_1_2_1_37_1","unstructured":"Massolution. 2013. The Crowd in the Cloud: Exploring the Future of Outsourcing. White Paper. Massolution.  Massolution. 2013. The Crowd in the Cloud: Exploring the Future of Outsourcing. White Paper. Massolution."},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence.","author":"Minder Patrick","year":"2011","unstructured":"Patrick Minder and Abraham Bernstein . 2011 . CrowdLang\u2014first steps towards programmable human computers for general computation . In Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence. Patrick Minder and Abraham Bernstein. 2011. CrowdLang\u2014first steps towards programmable human computers for general computation. In Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35386-4_10"},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence.","author":"Oleson David","year":"2011","unstructured":"David Oleson , Alexander Sorokin , Greg P. Laughlin , Vaughn Hester , John Le , and Lukas Biewald . 2011 . Programmatic gold: Targeted and scalable quality assurance in crowdsourcing . In Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence. David Oleson, Alexander Sorokin, Greg P. Laughlin, Vaughn Hester, John Le, and Lukas Biewald. 2011. Programmatic gold: Targeted and scalable quality assurance in crowdsourcing. In Proceedings of the Workshops at the 25th AAAI Conference on Artificial Intelligence."},{"key":"e_1_2_1_41_1","unstructured":"Object Management Group. 2011. Business Process Model and Notation (BPMN) Version 2.0. Available at http:\/\/www.omg.org.  Object Management Group. 2011. Business Process Model and Notation (BPMN) Version 2.0. Available at http:\/\/www.omg.org."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979148"},{"key":"e_1_2_1_43_1","series-title":"Lecture Notes in Computer Science","volume-title":"Business Process Management","author":"Rodriguez Carlos","unstructured":"Carlos Rodriguez , Florian Daniel , and Fabio Casati . 2014. Crowd-based mining of reusable process model patterns . In Business Process Management . Lecture Notes in Computer Science , Vol. 8659 . Springer , 51--66. Carlos Rodriguez, Florian Daniel, and Fabio Casati. 2014. Crowd-based mining of reusable process model patterns. In Business Process Management. Lecture Notes in Computer Science, Vol. 8659. Springer, 51--66."},{"key":"e_1_2_1_44_1","unstructured":"Carlos Rodriguez Eros Zaupa Florian Daniel and Fabio Casati. 2013. Crowd-Based Pattern Mining: On the Crowdsourcing of Reusable Knowledge Identification from Mashup Models. UNITN.  Carlos Rodriguez Eros Zaupa Florian Daniel and Fabio Casati. 2013. Crowd-Based Pattern Mining: On the Crowdsourcing of Reusable Knowledge Identification from Mashup Models. UNITN."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25535-9_25"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187980.2188061"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2012.09.003"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-012-0180-6"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2008.66"},{"key":"e_1_2_1_50_1","first-page":"108","article-title":"Towards social crowd environments using service-oriented architectures","volume":"53","author":"Skopik Florian","year":"2011","unstructured":"Florian Skopik , Daniel Schall , Harald Psaier , Martin Treiber , and Schahram Dustdar . 2011 . Towards social crowd environments using service-oriented architectures . Information Technology 53 , 3, 108 -- 116 . Florian Skopik, Daniel Schall, Harald Psaier, Martin Treiber, and Schahram Dustdar. 2011. Towards social crowd environments using service-oriented architectures. Information Technology 53, 3, 108--116.","journal-title":"Information Technology"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the ACM SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation (CSE\u201910)","author":"Soleymani Mohammad","year":"2010","unstructured":"Mohammad Soleymani and Martha Larson . 2010 . Crowdsourcing for affective annotation of video: Development of a viewer-reported boredom corpus . In Proceedings of the ACM SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation (CSE\u201910) . 4--8. Mohammad Soleymani and Martha Larson. 2010. Crowdsourcing for affective annotation of video: Development of a viewer-reported boredom corpus. In Proceedings of the ACM SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation (CSE\u201910). 4--8."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/SERVICES-I.2009.56"},{"volume-title":"Languages, Architectures","author":"Weske Mathias","key":"e_1_2_1_53_1","unstructured":"Mathias Weske . 2007. Business Process Management: Concepts , Languages, Architectures . Springer . Mathias Weske. 2007. Business Process Management: Concepts, Languages, Architectures. Springer."}],"container-title":["ACM Transactions on the Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2746353","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2746353","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:16:58Z","timestamp":1750227418000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2746353"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,13]]},"references-count":52,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2015,5,26]]}},"alternative-id":["10.1145\/2746353"],"URL":"https:\/\/doi.org\/10.1145\/2746353","relation":{},"ISSN":["1559-1131","1559-114X"],"issn-type":[{"type":"print","value":"1559-1131"},{"type":"electronic","value":"1559-114X"}],"subject":[],"published":{"date-parts":[[2015,5,13]]},"assertion":[{"value":"2014-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-05-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}