{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T13:34:07Z","timestamp":1780925647110,"version":"3.54.1"},"reference-count":112,"publisher":"Tsinghua University Press","issue":"2","license":[{"start":{"date-parts":[[2019,8,16]],"date-time":"2019-08-16T00:00:00Z","timestamp":1565913600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJCS"],"published-print":{"date-parts":[[2019,9,24]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title> <jats:p>Human or machine, which one is more intelligent and powerful for performing computing and processing tasks? Over the years, researchers and scientists have spent significant amounts of money and effort to answer this question. Nonetheless, despite some outstanding achievements, replacing humans in the intellectual tasks is not yet a reality. Instead, to compensate for the weakness of machines in some (mostly cognitive) tasks, the idea of putting human in the loop has been introduced and widely accepted. In this paper, the notion of collective hybrid intelligence as a new computing framework and comprehensive.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title> <jats:p>According to the extensive acceptance and efficiency of crowdsourcing, hybrid intelligence and distributed computing concepts, the authors have come up with the (complementary) idea of collective hybrid intelligence. In this regard, besides providing a brief review of the efforts made in the related contexts, conceptual foundations and building blocks of the proposed framework are delineated. Moreover, some discussion on architectural and realization issues are presented.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title> <jats:p>The paper describes the conceptual architecture, workflow and schematic representation of a new hybrid computing concept. Moreover, by introducing three sample scenarios, its benefits, requirements, practical roadmap and architectural notes are explained.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title> <jats:p>The major contribution of this work is introducing the conceptual foundations to combine and integrate collective intelligence of humans and machines to achieve higher efficiency and (computing) performance. To the best of the authors\u2019 knowledge, this the first study in which such a blessing integration is considered. Therefore, it is believed that the proposed computing concept could inspire researchers toward realizing such unprecedented possibilities in practical and theoretical contexts.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/ijcs-03-2019-0012","type":"journal-article","created":{"date-parts":[[2020,2,21]],"date-time":"2020-02-21T16:24:58Z","timestamp":1582302298000},"page":"198-220","source":"Crossref","is-referenced-by-count":34,"title":["Collective hybrid intelligence: towards a conceptual framework"],"prefix":"10.26599","volume":"3","author":[{"given":"Morteza","family":"Moradi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammad","family":"Moradi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farhad","family":"Bayat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adel","family":"Nadjaran Toosi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"11138","published-online":{"date-parts":[[2019,8,16]]},"reference":[{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2016.05.003"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1038\/srep16557"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1145\/602382.602400"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1515\/fcds-2017-0001"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-64816-3_2"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1109\/ICCCN.2017.8038465"},{"key":"ref37","author":"gifford","year":"2009","journal-title":"Collective machine learning team learning and classification in multi-agent systems"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1109\/SysEng.2016.7753126"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1109168108"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2017.2765699"},{"key":"ref28","first-page":"26","article-title":"Mechanical cheat: spamming schemes and adversarial techniques on crowdsourcing platforms","author":"difallah","year":"0","journal-title":"Proceedings of the First International Workshop on Crowdsourcing Web Search"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1016\/j.camwa.2011.11.028"},{"key":"ref29","article-title":"A Definition of Artificial Intelligence","author":"dobrev","year":"2012","journal-title":"ArXiv Preprint"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1016\/S0743-7315(03)00006-6"},{"key":"ref22","article-title":"The Modern History of Computing","author":"copeland","year":"2000","journal-title":"The Stanford Encyclopedia of Philosophy (Winter 2017 Edition)"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2014.05.010"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1145\/3148148"},{"key":"ref23","article-title":"Artificial intelligence for artificial intelligence","author":"dai","year":"0","journal-title":"Proceedings of Twenty-Fifth AAAI Conference on Artificial Intelligence"},{"key":"ref101","doi-asserted-by":"publisher","DOI":"10.1145\/1868914.1868976"},{"key":"ref26","article-title":"18 Artificial intelligence researchers reveal the profound changes coming to our lives","author":"del prado","year":"2015","journal-title":"Business Insider"},{"key":"ref100","first-page":"76","article-title":"Is there something beyond AI? Frequently emerging, but seldom answered questions about artificial Super-Intelligence","author":"wiedermann","year":"0","journal-title":"Proceedings of the International Conference Beyond AI"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-59702-2_7"},{"key":"ref50","author":"karimi","year":"2004","journal-title":"Telegeoinformatics Location-Based Computing and Services"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1007\/0-387-27705-6_6"},{"key":"ref59","doi-asserted-by":"publisher","DOI":"10.1007\/s12599-010-0114-8"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-007-9079-x"},{"key":"ref57","article-title":"Folding@ Home and Genome@ Home: Using distributed computing to tackle previously intractable problems in computational biology","author":"larson","year":"2009","journal-title":"ArXiv Preprint"},{"key":"ref56","first-page":"460","article-title":"Collective robotic intelligence","author":"kube","year":"0","journal-title":"Proceedings of the Second International Conference on Simulation of Adaptive Behavior"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.1109\/GRID.2008.4662809"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.1007\/s00170-019-03785-0"},{"key":"ref53","doi-asserted-by":"publisher","DOI":"10.3390\/s18051426"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2003.1160055"},{"key":"ref40","article-title":"Measurements of collective machine intelligence","author":"halmes","year":"2013","journal-title":"ArXiv Preprint"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1142\/9789814675017_0002"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1007\/s10588-012-9121-2"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2018.01.050"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.07.039"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1016\/S0165-0114(01)00149-X"},{"key":"ref49","first-page":"4070","article-title":"Directions in hybrid intelligence: complementing AI systems with human intelligence","author":"kamar","year":"0","journal-title":"Proceedings of IJCAI"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2009.5160922"},{"key":"ref9","article-title":"Decisions 2.0: the power of collective intelligence","volume":"50","author":"bonabeau","year":"2009","journal-title":"MIT Sloan Management Review"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.3103\/S0146411617060049"},{"key":"ref45","article-title":"Towards hybrid intelligence for robotics","author":"huang","year":"0","journal-title":"Proceedings of The 5th Edition of the Collective Intelligence Conference"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-014-0352-3"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2017.2717883"},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.1145\/377025.377033"},{"key":"ref41","first-page":"146","article-title":"A formal definition of intelligence based on an intensional variant of algorithmic complexity","author":"hern\u00e1ndez-orallo","year":"0","journal-title":"Proceedings of International Symposium of Engineering of Intelligent Systems (EIS98)"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.2013.1879"},{"key":"ref43","first-page":"1","article-title":"The rise of crowdsourcing","volume":"14","author":"howe","year":"2006","journal-title":"Wired Magazine"},{"key":"ref73","doi-asserted-by":"publisher","DOI":"10.1145\/1743384.1743478"},{"key":"ref72","first-page":"68","article-title":"Human-level artificial intelligence? Be serious!","volume":"26","author":"nilsson","year":"2005","journal-title":"AI Magazine"},{"key":"ref71","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-26485-1_33"},{"key":"ref70","first-page":"6","article-title":"The third age of artificial intelligence. Field actions science reports","volume":"17","author":"miailhe","year":"2017","journal-title":"The Journal of Field Actions (Special Issue)"},{"key":"ref76","doi-asserted-by":"publisher","DOI":"10.1016\/J.ENG.2016.04.018"},{"key":"ref77","article-title":"Scalable Global Grid catalogue for LHC Run3 and beyond","author":"pedreira","year":"2017","journal-title":"ArXiv Preprint"},{"key":"ref74","article-title":"Towards accountable AI: hybrid human-machine analyses for characterizing system failure","author":"nushi","year":"0","journal-title":"Proceedings of the Sixth AAAI Conference on Human Computation and Crowdsourcing"},{"key":"ref75","doi-asserted-by":"publisher","DOI":"10.1089\/big.2014.0064"},{"key":"ref78","doi-asserted-by":"publisher","DOI":"10.1016\/j.jesp.2017.01.006"},{"key":"ref79","first-page":"73","article-title":"Does money matter? Motivational factors for participation in paid- and non-profit-crowdsourcing communities","volume":"37","author":"pilz","year":"2013","journal-title":"Wirtschaftsinformatik"},{"key":"ref60","doi-asserted-by":"publisher","DOI":"10.1108\/IJCS-04-2018-0007"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1007\/s10723-015-9329-8"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-016-0727-z"},{"key":"ref63","doi-asserted-by":"publisher","DOI":"10.1007\/s11036-017-0932-8"},{"key":"ref64","doi-asserted-by":"publisher","DOI":"10.1007\/s11518-009-5095-0"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1109\/MAHC.1988.10011"},{"key":"ref66","doi-asserted-by":"publisher","DOI":"10.1016\/j.futures.2017.03.006"},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2014.07.036"},{"key":"ref68","article-title":"Volunteering versus work for pay: incentives and tradeoffs in crowdsourcing","author":"mao","year":"0","journal-title":"Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2012.07.010"},{"key":"ref69","doi-asserted-by":"publisher","DOI":"10.2307\/2010552"},{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1145\/581571.581573"},{"key":"ref109","doi-asserted-by":"publisher","DOI":"10.1287\/inte.33.6.72.25177"},{"key":"ref95","first-page":"4402e","article-title":"Intelligent vehicle navigation system with assistance and alerting capabilities","author":"vochin","year":"2018","journal-title":"Concurrency and Computation Practice and Experience"},{"key":"ref108","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2008.926583"},{"key":"ref94","doi-asserted-by":"publisher","DOI":"10.1016\/j.apacoust.2018.03.029"},{"key":"ref107","doi-asserted-by":"publisher","DOI":"10.1109\/WI-IAT.2012.104"},{"key":"ref93","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-018-0830-z"},{"key":"ref106","doi-asserted-by":"publisher","DOI":"10.1108\/IJCS-08-2017-0013"},{"key":"ref92","first-page":"1","article-title":"Making better use of the crowd: how crowdsourcing can advance machine learning research","volume":"18","author":"vaughan","year":"2017","journal-title":"Journal of Machine Learning Research"},{"key":"ref105","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186033"},{"key":"ref91","first-page":"4439","article-title":"Avoiding imposters and delinquents: adversarial crowdsourcing and peer prediction","author":"steinhardt","year":"2016","journal-title":"Advances in neural information processing systems"},{"key":"ref104","doi-asserted-by":"publisher","DOI":"10.1504\/IJBIC.2011.043624"},{"key":"ref90","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2010.118"},{"key":"ref103","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-21365-1_40"},{"key":"ref102","doi-asserted-by":"publisher","DOI":"10.1111\/nph.14535"},{"key":"ref111","first-page":"1061","article-title":"Active learning from crowds with unsure option","author":"zhong","year":"0","journal-title":"Proceedings of International Joint Conferences on Artificial Intelligence"},{"key":"ref112","doi-asserted-by":"publisher","DOI":"10.1007\/s13748-017-0140-6"},{"key":"ref110","doi-asserted-by":"publisher","DOI":"10.1117\/12.2514905"},{"key":"ref98","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-34103-8_42"},{"key":"ref99","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2014.09.004"},{"key":"ref96","first-page":"1","article-title":"Human computation","author":"von ahn","year":"0","journal-title":"Proceedings of the 2008 IEEE 24th International Conference on Data Engineering"},{"key":"ref97","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2007.41"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1145\/2342509.2342513"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1145\/2971648.2971741"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1177\/1354856507084420"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.5898\/JHRI.2.1.Breazeal"},{"key":"ref14","first-page":"32","article-title":"A brief history of decision making","volume":"84","author":"buchanan","year":"2006","journal-title":"Harvard Business Review"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1145\/2950042"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.2307\/3790872"},{"key":"ref82","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647409"},{"key":"ref17","first-page":"45","article-title":"A cognitive substrate for achieving human-level intelligence","volume":"27","author":"cassimatis","year":"2006","journal-title":"AI Magazine"},{"key":"ref81","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2016.2524510"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026044"},{"key":"ref84","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSE.2011.6028683"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-6462-4_14"},{"key":"ref83","doi-asserted-by":"publisher","DOI":"10.1038\/548520a"},{"key":"ref80","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979148"},{"key":"ref89","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-44781-0_45"},{"key":"ref85","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2018.00261"},{"key":"ref86","doi-asserted-by":"publisher","DOI":"10.1016\/j.respol.2016.02.003"},{"key":"ref87","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2016.2579198"},{"key":"ref88","doi-asserted-by":"publisher","DOI":"10.12694\/scpe.v20i2.1538"}],"container-title":["International Journal of Crowd Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJCS-03-2019-0012\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJCS-03-2019-0012\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,12]],"date-time":"2022-07-12T14:08:57Z","timestamp":1657634937000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJCS-03-2019-0012\/full\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,16]]},"references-count":112,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,8,16]]},"published-print":{"date-parts":[[2019,9,24]]}},"alternative-id":["10.1108\/IJCS-03-2019-0012"],"URL":"https:\/\/doi.org\/10.1108\/ijcs-03-2019-0012","relation":{},"ISSN":["2398-7294","2398-7294"],"issn-type":[{"value":"2398-7294","type":"print"},{"value":"2398-7294","type":"print"}],"subject":[],"published":{"date-parts":[[2019,8,16]]}}}