{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,23]],"date-time":"2025-10-23T05:39:50Z","timestamp":1761197990671,"version":"3.41.2"},"reference-count":69,"publisher":"Emerald","issue":"2","license":[{"start":{"date-parts":[[2021,10,14]],"date-time":"2021-10-14T00:00:00Z","timestamp":1634169600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJICC"],"published-print":{"date-parts":[[2022,4,26]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title><jats:p>As the number of joined service providers (SPs) in knowledge-intensive crowdsourcing (KI-C) continues to rise, there is an information overload problem for KI-C platforms and consumers to identify qualified SPs to complete tasks. To this end, this paper aims to propose a quality of service (QoS) evaluation framework for SPs in KI-C to effectively and comprehensively characterize the QoS of SPs, which can aid the efficient selection of qualified SPs.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title><jats:p>By literature summary and discussion with the expert team, a QoS evaluation indicator system for SPs in KI-C based on the SERVQUAL model is constructed. In addition, the Decision Making Trial and Evaluation Laboratory (DEMATEL) method is used to obtain evaluation indicators' weights. The SPs are evaluated and graded by the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) and rank\u2013sum ratio (RSR), respectively.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Findings<\/jats:title><jats:p>A QoS evaluation indicator system for SPs in KI-C incorporating 13 indicators based on SERVQUAL has been constructed, and a hybrid methodology combining DEMATEL, TOPSIS and RSR is applied to quantify and visualize the QoS of SPs.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title><jats:p>The QoS evaluation framework for SPs in KI-C proposed in this paper can quantify and visualize the QoS of SPs, which can help the crowdsourcing platform to realize differentiated management for SPs and assist SPs to improve their shortcomings in a targeted manner. And this is the first paper to evaluate SPs in KI-C from the prospect of QoS.<\/jats:p><\/jats:sec>","DOI":"10.1108\/ijicc-06-2021-0113","type":"journal-article","created":{"date-parts":[[2021,10,14]],"date-time":"2021-10-14T18:06:48Z","timestamp":1634234808000},"page":"198-223","source":"Crossref","is-referenced-by-count":5,"title":["Evaluating and visualizing QoS of service providers in knowledge-intensive crowdsourcing: a combined MCDM approach"],"prefix":"10.1108","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8048-4329","authenticated-orcid":false,"given":"Shixin","family":"Xie","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xu","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4782-0247","authenticated-orcid":false,"given":"Biyu","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7195-3883","authenticated-orcid":false,"given":"Longxiao","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0946-5483","authenticated-orcid":false,"given":"Jinfeng","family":"Yu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2021,10,14]]},"reference":[{"key":"key2024071714075048600_ref001","doi-asserted-by":"publisher","DOI":"10.1016\/j.enpol.2019.111174","article-title":"Improving renewable energy policy planning and decision-making through a hybrid MCDM method","volume":"137","year":"2020","journal-title":"Energy Policy"},{"issue":"9-10","key":"key2024071714075048600_ref002","first-page":"978","article-title":"Monitoring patient dissatisfaction: a methodology based on SERVQUAL scale and statistical process control charts","volume":"31","year":"2018","journal-title":"Total Quality Management and Business Excellence"},{"key":"key2024071714075048600_ref003","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.114564","article-title":"International location selection for production fragmentation","volume":"171","year":"2021","journal-title":"Expert Systems with Applications"},{"issue":"3","key":"key2024071714075048600_ref004","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1108\/IJICC-03-2021-0040","article-title":"A comparison study of fuzzy-based multiple-criteria decision-making methods to evaluating green concept alternatives in a new product development environment","volume":"14","year":"2021","journal-title":"International Journal of Intelligent Computing and Cybernetics"},{"key":"key2024071714075048600_ref005","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/j.eswa.2016.04.010","article-title":"Participation recommendation system for crowdsourcing contests","volume":"58","year":"2016","journal-title":"Expert Systems with Applications"},{"issue":"4","key":"key2024071714075048600_ref006","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1007\/s00778-015-0385-2","article-title":"Task assignment optimization in knowledge-intensive crowdsourcing","volume":"24","year":"2015","journal-title":"The VLDB Journal"},{"issue":"3","key":"key2024071714075048600_ref007","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1016\/j.eswa.2012.05.046","article-title":"Integrating fuzzy DEMATEL and fuzzy hierarchical TOPSIS methods for truck selection","volume":"40","year":"2013","journal-title":"Expert Systems with Applications"},{"issue":"2","key":"key2024071714075048600_ref008","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.jbusvent.2006.02.001","article-title":"Early teams: the impact of team demography on VC financing and going public","volume":"22","year":"2007","journal-title":"Journal of Business Venturing"},{"issue":"1","key":"key2024071714075048600_ref009","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1177\/1354856507084420","article-title":"Crowdsourcing as a model for problem solving","volume":"14","year":"2008","journal-title":"Convergence: The International Journal of Research Into New Media Technologies"},{"issue":"3","key":"key2024071714075048600_ref010","doi-asserted-by":"crossref","first-page":"2341","DOI":"10.1016\/j.eswa.2011.08.061","article-title":"A combined fuzzy AHP and fuzzy TOPSIS based strategic analysis of electronic service quality in healthcare industry","volume":"39","year":"2012","journal-title":"Expert Systems with Applications"},{"key":"key2024071714075048600_ref011","first-page":"28","article-title":"Customer service: an essential component for a successful Web site","volume":"20","year":"2000","journal-title":"Marketing Health Services"},{"issue":"06","key":"key2024071714075048600_ref012","doi-asserted-by":"crossref","first-page":"1229","DOI":"10.1142\/S0219622014500758","article-title":"Integrating TOPSIS and DEMATEL methods to rank the risk of failure of FMEA","volume":"13","year":"2014","journal-title":"International Journal of Information Technology and Decision Making"},{"key":"key2024071714075048600_ref013","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.ssci.2015.05.006","article-title":"Road safety risk evaluation by means of improved entropy TOPSIS\u2013RSR","volume":"79","year":"2015","journal-title":"Safety Science"},{"key":"key2024071714075048600_ref014","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.ijinfomgt.2018.03.005","article-title":"A mixed method investigation of sharing economy driven car-hailing services: online and offline perspectives","volume":"41","year":"2018","journal-title":"International Journal of Information Management"},{"issue":"5","key":"key2024071714075048600_ref015","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1108\/TQM-11-2012-0103","article-title":"Service quality and relative performance of public universities in East Africa","volume":"25","year":"2013","journal-title":"The TQM Journal"},{"issue":"2","key":"key2024071714075048600_ref016","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.technovation.2005.06.014","article-title":"Success in Israeli high-tech start-ups; Critical factors and process","volume":"26","year":"2006","journal-title":"Technovation"},{"key":"key2024071714075048600_ref017","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1016\/j.egyr.2021.01.048","article-title":"Two-dimensional evaluation model of electrical equipment based on combined weighting and rating algorithm","volume":"7","year":"2021","journal-title":"Energy Reports"},{"year":"2018","key":"key2024071714075048600_ref018","article-title":"Financial sector-based analysis of the G20 economies using the integrated decision-making approach with DEMATEL and TOPSIS"},{"issue":"3","key":"key2024071714075048600_ref019","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1509\/jmr.13.0165","article-title":"Direct and indirect effects of buyers and sellers on search advertising revenues in business-to-business electronic platforms","volume":"52","year":"2015","journal-title":"Journal of Marketing Research"},{"issue":"4","key":"key2024071714075048600_ref020","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.bushor.2015.03.003","article-title":"Crowdsourcing: a new way of employing non-employees?","volume":"58","year":"2015","journal-title":"Business Horizons"},{"volume-title":"Perceptions of the World Problematique: Communication Procedure, Communicating with Those Bearing Collective Responsibility","year":"1973","key":"key2024071714075048600_ref021"},{"key":"key2024071714075048600_ref022","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.dss.2014.05.007","article-title":"Personalized task recommendation in crowdsourcing information systems \u2014 current state of the art","volume":"65","year":"2014","journal-title":"Decision Support Systems"},{"issue":"2","key":"key2024071714075048600_ref023","first-page":"301","article-title":"Estimating participants for knowledge-intensive tasks in a network of crowdsourcing marketplaces","volume":"19","year":"2016","journal-title":"Information Systems Frontiers"},{"key":"key2024071714075048600_ref024","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.jairtraman.2017.06.001","article-title":"Evaluating service quality of airline industry using hybrid best worst method and VIKOR","volume":"68","year":"2018","journal-title":"Journal of Air Transport Management"},{"key":"key2024071714075048600_ref025","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.trd.2017.07.001","article-title":"Sustainable decision making for joint distribution center location choice","volume":"55","year":"2017","journal-title":"Transportation Research Part D: Transport and Environment"},{"issue":"9","key":"key2024071714075048600_ref026","doi-asserted-by":"crossref","first-page":"1969","DOI":"10.1108\/IMDS-10-2020-0598","article-title":"Blockchain performance in supply chain management: application in blockchain integration companies","volume":"121","year":"2021","journal-title":"Industrial Management and Data Systems"},{"issue":"6","key":"key2024071714075048600_ref027","first-page":"176","article-title":"The rise of crowdsourcing","volume":"14","year":"2006","journal-title":"Wired Magazine"},{"volume-title":"Multiple Attribute Decision Making: Methods and Applications","year":"1981","key":"key2024071714075048600_ref028"},{"issue":"1","key":"key2024071714075048600_ref029","first-page":"85","article-title":"Understanding the service quality and customer satisfaction of mobile banking in Bangladesh: using a structural equation model","volume":"22","year":"2018","journal-title":"Global Business Review"},{"issue":"2","key":"key2024071714075048600_ref030","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.bushor.2016.10.001","article-title":"Crowdsourcing: a revised definition and introduction to new research","volume":"60","year":"2017","journal-title":"Business Horizons"},{"volume-title":"The E-Commerce Book: Building the E-Empire","year":"2001","key":"key2024071714075048600_ref031"},{"issue":"5","key":"key2024071714075048600_ref032","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1108\/02652328910134590","article-title":"Quality in the service sector: a review","volume":"7","year":"1989","journal-title":"International Journal of Bank Marketing"},{"key":"key2024071714075048600_ref033","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.jairtraman.2017.01.006","article-title":"A hybrid approach based on fuzzy AHP and 2-tuple fuzzy linguistic method for evaluation in-flight service quality","volume":"60","year":"2017","journal-title":"Journal of Air Transport Management"},{"key":"key2024071714075048600_ref034","first-page":"1","article-title":"A bayesian best-worst method-based multicriteria competence analysis of crowdsourcing delivery personnel","volume":"2020","year":"2020","journal-title":"Complexity"},{"issue":"53","key":"key2024071714075048600_ref035","doi-asserted-by":"crossref","first-page":"29630","DOI":"10.1016\/j.ijhydene.2020.07.243","article-title":"Risk assessment of hydrogen generation unit considering dependencies using integrated DEMATEL and TOPSIS approach","volume":"45","year":"2020","journal-title":"International Journal of Hydrogen Energy"},{"issue":"2","key":"key2024071714075048600_ref036","doi-asserted-by":"crossref","first-page":"1379","DOI":"10.32604\/cmc.2020.011995","article-title":"A crowdsourcing recommendation that considers the influence of workers","volume":"66","year":"2021","journal-title":"Computers, Materials and Continua"},{"issue":"12","key":"key2024071714075048600_ref037","doi-asserted-by":"publisher","DOI":"10.3390\/sym12121962","article-title":"A hybrid MCDM model to evaluate and classify outsourcing providers in manufacturing","volume":"12","year":"2020","journal-title":"Symmetry"},{"first-page":"61","article-title":"The study of quality evaluation model for the real-time interactive online teaching","year":"2021","key":"key2024071714075048600_ref038"},{"issue":"2","key":"key2024071714075048600_ref039","first-page":"187","article-title":"A survey of task-oriented crowdsourcing","volume":"44","year":"2014","journal-title":"Artificial Intelligence Review"},{"key":"key2024071714075048600_ref040","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhm.2020.102853","article-title":"Listening to the voice of the guest: a framework to improve decision-making processes with text data","volume":"94","year":"2021","journal-title":"International Journal of Hospitality Management"},{"first-page":"985","article-title":"Task allocation in dependency-aware spatial crowdsourcing","year":"2020","key":"key2024071714075048600_ref041"},{"key":"key2024071714075048600_ref042","doi-asserted-by":"publisher","DOI":"10.1016\/j.seps.2020.100816","article-title":"A fuzzy service quality based approach for performance evaluation of educational units","volume":"73","year":"2021","journal-title":"Socio-Economic Planning Sciences"},{"issue":"1","key":"key2024071714075048600_ref043","first-page":"1","article-title":"Assessing service quality of online bill payment system using extended SERVQUAL model (SERVQUAL-Butterfly model): a case study of Dhaka electric supply company limited (DESCO), Bangladesh","volume":"4","year":"2017","journal-title":"Cogent Business and Management"},{"issue":"9","key":"key2024071714075048600_ref044","doi-asserted-by":"crossref","first-page":"1939","DOI":"10.1108\/IMDS-02-2020-0066","article-title":"Unlocking causal relations of barriers to big data analytics in manufacturing firms","volume":"121","year":"2021","journal-title":"Industrial Management and Data Systems"},{"key":"key2024071714075048600_ref045","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.tourman.2017.11.009","article-title":"Quality assessment of airline baggage handling systems using SERVQUAL and BWM","volume":"66","year":"2018","journal-title":"Tourism Management"},{"key":"key2024071714075048600_ref046","first-page":"3551","article-title":"Workers' task choice in crowdsourcing and human computation markets","volume":"4","year":"2012","journal-title":"AISeL"},{"issue":"3","key":"key2024071714075048600_ref047","first-page":"303","article-title":"Critical success factors for e-commerce entrepreneurship: an empirical study of Thailand","volume":"32","year":"2008","journal-title":"Small Business Economics"},{"key":"key2024071714075048600_ref048","first-page":"1","article-title":"Organizational performance influenced by academic service quality: an investigation in public universities in Malaysia","volume":"2021","year":"2021","journal-title":"Education Research International"},{"issue":"3","key":"key2024071714075048600_ref049","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.knosys.2010.01.013","article-title":"A DEMATEL method in identifying key success factors of hospital service quality","volume":"23","year":"2010","journal-title":"Knowledge-Based Systems"},{"issue":"3","key":"key2024071714075048600_ref050","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1177\/002224299505900306","article-title":"Market orientation and the learning organization","volume":"59","year":"1995","journal-title":"Journal of Marketing"},{"key":"key2024071714075048600_ref051","first-page":"448","article-title":"Competence at work : models for superior performance","volume":"2","year":"1993","journal-title":"Competence at Work: Models for Superior Performance"},{"issue":"3","key":"key2024071714075048600_ref052","doi-asserted-by":"crossref","first-page":"1272","DOI":"10.1016\/j.chb.2011.01.009","article-title":"Using SERVQUAL to assess the quality of e-learning experience","volume":"27","year":"2011","journal-title":"Computers in Human Behavior"},{"issue":"1","key":"key2024071714075048600_ref053","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.elerap.2011.07.010","article-title":"Consumers rule: how consumer reviews influence perceived trustworthiness of online stores","volume":"11","year":"2012","journal-title":"Electronic Commerce Research and Applications"},{"issue":"4","key":"key2024071714075048600_ref054","first-page":"437","article-title":"E-service quality expectations: a case study","volume":"14","year":"2010","journal-title":"Total Quality Management and Business Excellence"},{"key":"key2024071714075048600_ref055","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.ins.2016.01.076","article-title":"A novel hybrid MCDM model combining the SAW, TOPSIS and GRA methods based on experimental design","volume":"345","year":"2016","journal-title":"Information Sciences"},{"key":"key2024071714075048600_ref056","doi-asserted-by":"crossref","first-page":"4019","DOI":"10.1016\/j.jclepro.2016.10.049","article-title":"Assessing industrial ecosystem vulnerability in the coal mining area under economic fluctuations","volume":"142","year":"2017","journal-title":"Journal of Cleaner Production"},{"key":"key2024071714075048600_ref057","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1016\/j.enpol.2017.12.017","article-title":"Coal miners' livelihood vulnerability to economic shock: multi-criteria assessment and policy implications","volume":"114","year":"2018","journal-title":"Energy Policy"},{"issue":"7","key":"key2024071714075048600_ref058","doi-asserted-by":"crossref","first-page":"1637","DOI":"10.1109\/TMC.2017.2771259","article-title":"Multi-objective optimization based allocation of heterogeneous spatial crowdsourcing tasks","volume":"17","year":"2018","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"key2024071714075048600_ref059","doi-asserted-by":"publisher","DOI":"10.1007\/s10660-021-09469-w","article-title":"The influence of community engagement on seller opportunistic behaviors in e-commerce platform","year":"2021","journal-title":"Electronic Commerce Research"},{"issue":"4","key":"key2024071714075048600_ref060","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1109\/TKDE.2019.2893240","article-title":"SRA: secure reverse auction for task assignment in spatial crowdsourcing","volume":"32","year":"2020","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"first-page":"84","article-title":"Service providers' competence identification in knowledge-intensive crowdsourcing context","year":"2020","key":"key2024071714075048600_ref061"},{"key":"key2024071714075048600_ref062","first-page":"1","article-title":"Understanding service providers' competency in knowledge-intensive crowdsourcing platforms: an LDA approach","volume":"2021","year":"2021","journal-title":"Complexity"},{"key":"key2024071714075048600_ref063","doi-asserted-by":"publisher","DOI":"10.1108\/K-02-2020-0067","article-title":"Blockchain crowdfunding projects evaluation using GRA-TOPSIS","year":"2020","journal-title":"Kybernetes"},{"issue":"3","key":"key2024071714075048600_ref064","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1108\/09604520210429187","article-title":"Service excellence in electronic channels","volume":"12","year":"2002","journal-title":"Managing Service Quality: An International Journal"},{"issue":"8","key":"key2024071714075048600_ref065","doi-asserted-by":"crossref","first-page":"1623","DOI":"10.1108\/K-12-2017-0468","article-title":"An approach to task recommendation in crowdsourcing based on 2-tuple fuzzy linguistic method","volume":"47","year":"2018","journal-title":"Kybernetes"},{"key":"key2024071714075048600_ref066","doi-asserted-by":"publisher","DOI":"10.1016\/j.cie.2019.106085","article-title":"A combined fuzzy DEMATEL and TOPSIS approach for estimating participants in knowledge-intensive crowdsourcing","volume":"137","year":"2019","journal-title":"Computers & Industrial Engineering"},{"key":"key2024071714075048600_ref067","doi-asserted-by":"publisher","DOI":"10.1007\/s10660-020-09408-1","article-title":"Investigating participants' attributes for participant estimation in knowledge-intensive crowdsourcing: a fuzzy DEMATEL based approach","year":"2020","journal-title":"Electronic Commerce Research"},{"issue":"1","key":"key2024071714075048600_ref068","article-title":"An integrated model for evaluation of maternal health care in China","volume":"16","year":"2021","journal-title":"PLoS One"},{"key":"key2024071714075048600_ref069","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.jclepro.2018.05.247","article-title":"Sustainable recycling partner selection using fuzzy DEMATEL-AEW-FVIKOR: a case study in small-and-medium enterprises (SMEs)","volume":"196","year":"2018","journal-title":"Journal of Cleaner Production"}],"container-title":["International Journal of Intelligent Computing and Cybernetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJICC-06-2021-0113\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJICC-06-2021-0113\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:54:25Z","timestamp":1753397665000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ijicc\/article\/15\/2\/198-223\/120050"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,14]]},"references-count":69,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,10,14]]},"published-print":{"date-parts":[[2022,4,26]]}},"alternative-id":["10.1108\/IJICC-06-2021-0113"],"URL":"https:\/\/doi.org\/10.1108\/ijicc-06-2021-0113","relation":{},"ISSN":["1756-378X"],"issn-type":[{"type":"print","value":"1756-378X"}],"subject":[],"published":{"date-parts":[[2021,10,14]]}}}