{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:14:45Z","timestamp":1760192085354,"version":"build-2065373602"},"reference-count":59,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2019,10,7]],"date-time":"2019-10-07T00:00:00Z","timestamp":1570406400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Improving the quality of research information systems is an important goal in the process of improving the performance of research management in Chinese universities. Since the evaluation of information system (IS) quality is a multicriteria decision problem, it is critical to identify the interrelationships among the dimensions and criteria, and decide on the important criteria for proposed improvement strategies. This paper suggests a hybrid multicriteria decision-making (MCDM) model for improving the quality of a research information system. First, a rough method combined with the decision-making trial and evaluation laboratory and analytical network process (rough DANP) model is used to improve the objectivity of expert judgements. Additionally, the rough DANP can be used to construct an influential network relationship map (INRM) between research information system components to derive the criterion weights. The complex proportional assessment of alternatives with rough numbers (COPRAS-R) is applied to evaluate the performance of the research information system. A Chinese university research information system is chosen to illustrate the usefulness of the proposed model. The results show that efficiency, effectiveness, and user frequency have the highest priorities for improvement. Selected management implications based on the actual case study are supplied.<\/jats:p>","DOI":"10.3390\/sym11101248","type":"journal-article","created":{"date-parts":[[2019,10,7]],"date-time":"2019-10-07T10:05:12Z","timestamp":1570442712000},"page":"1248","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A Rough Hybrid Multicriteria Decision-Making Model for Improving the Quality of a Research Information System"],"prefix":"10.3390","volume":"11","author":[{"given":"Qi-Gan","family":"Shao","sequence":"first","affiliation":[{"name":"School of Economics &amp; Management, Xiamen University of Technology, Xiamen 361005, China"},{"name":"Department of Information and Finance Management, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6918-6048","authenticated-orcid":false,"given":"James J. H.","family":"Liou","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering and Management, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sung-Shun","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Information and Finance Management, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1136-8727","authenticated-orcid":false,"given":"Yen-Ching","family":"Chuang","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering and Management, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1016\/j.joi.2010.04.003","article-title":"Growth and structure of Slovenia\u2019s scientific collaboration network","volume":"4","author":"Perc","year":"2010","journal-title":"J. Informetr."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1016\/j.joi.2010.03.001","article-title":"Zip\u2019s law and log-normal distribution in measures of sicentific output across fields and institutions: 40 years of years of Slovenia\u2019s research as an example","volume":"4","author":"Perc","year":"2010","journal-title":"J. Informetr."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lu\u017ear, B., Levnaji\u0107, Z., Povh, J., and Perc, M. (2014). Community structure and the evolution of interdisciplinarity in Slovenia\u2019s scientific collaboration network. PLoS ONE, 4.","DOI":"10.1371\/journal.pone.0094429"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1007\/s10955-014-1024-9","article-title":"Saving Human Lives: What Complexity Science and Information Systems can Contribute","volume":"158","author":"Helbing","year":"2015","journal-title":"J. Stat. Phys."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1287\/isre.3.1.60","article-title":"Information systems success: The quest for the dependent variable","volume":"3","author":"DeLone","year":"1992","journal-title":"Inf. Syst. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1080\/07421222.2003.11045748","article-title":"The DeLone and McLean Model of Information Systems Success: A Ten-Year Update","volume":"19","author":"DeLone","year":"2003","journal-title":"J. Manag. Inf. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"173","DOI":"10.2307\/249687","article-title":"Service quality: A measure of information systems effectiveness","volume":"19","author":"Pitt","year":"1995","journal-title":"MIS Q."},{"key":"ref_8","first-page":"2966","article-title":"Dimensions of information systems success","volume":"2","author":"Seddon","year":"1999","journal-title":"Commun. Assoc. Inf. Syst."},{"key":"ref_9","first-page":"1","article-title":"Information System Success Disparity between Developer and Users","volume":"9","author":"Yeo","year":"2016","journal-title":"Indian J. Sci. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.dss.2012.11.008","article-title":"The impacts of information quality and system quality on users\u2019 continuance intention in information-exchange virtual communities: An empirical investigation","volume":"56","author":"Zheng","year":"2013","journal-title":"Decis. Support Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.ijinfomgt.2018.02.007","article-title":"Analyzing data quality issues in research information systems via data profiling","volume":"41","author":"Azeroual","year":"2018","journal-title":"Int. J. Inf. Manag."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s10796-014-9500-y","article-title":"Research on information systems failures and successes: Status update and future directions","volume":"17","author":"Dwivedi","year":"2014","journal-title":"Inf. Syst. Front."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1016\/j.ijinfomgt.2016.03.011","article-title":"Information quality in design process docunment of quality management systems","volume":"36","author":"Hamrol","year":"2016","journal-title":"Int. J. Inf. Manag."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1016\/j.compedu.2009.09.017","article-title":"An application of fuzzy AHP for evaluating course website quality","volume":"54","author":"Lin","year":"2010","journal-title":"Comput. Educ."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1080\/0144929X.2012.695395","article-title":"Exploring the Critical Factors Influencing the Quality of Blog Interfaces Using the Decision-Making Trial and Evaluation Laboratory DEMATEL Method","volume":"33","author":"Hsu","year":"2014","journal-title":"Behav. Inf. Technol."},{"key":"ref_16","first-page":"15499","article-title":"An effectiveness evaluation model for the web-based marketing of airline industry","volume":"38","author":"Tsai","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref_17","first-page":"812","article-title":"Cloud e-learning service strategies for improving e-learning innovation performance in a fuzzy environment by using a new hybrid fuzzy multiple attribute decision-making model","volume":"24","author":"Su","year":"2015","journal-title":"Interact. Learn. Environ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"78","DOI":"10.3844\/jcssp.2017.78.90","article-title":"Scrutiny of Relationship between E-Banking Information Quality Dimensions and Customer Satisfaction","volume":"13","author":"Ayyash","year":"2017","journal-title":"J. Comput. Sci."},{"key":"ref_19","first-page":"131","article-title":"E-commerce systems success: An attempt to extend and respecify the Delone and McLean model of IS Success","volume":"2","author":"Molla","year":"2001","journal-title":"J. Electron. Commer. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/j.im.2007.11.004","article-title":"IT governance for enterprise resource planning supported by the DeLone-McLean model of information systems success","volume":"45","author":"Bernroider","year":"2008","journal-title":"Inf. Manag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/s10916-008-9193-1","article-title":"Assessing the quality of a Web-based Learning System for Nurses","volume":"33","author":"Chen","year":"2009","journal-title":"J. Med. Syst."},{"key":"ref_22","first-page":"940","article-title":"Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: Approach, results, and success factors","volume":"82","author":"Bossen","year":"2013","journal-title":"J. Med. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"43","DOI":"10.4258\/hir.2015.21.1.43","article-title":"Performance Evaluation of Public Hospital Information Systems by the Information System Success Model","volume":"21","author":"Kyoung","year":"2015","journal-title":"Health Inf. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1108\/LHT-12-2015-0113","article-title":"Implementation and evaluation of mobile e-books in a cloud bookcase using the information system sucess model","volume":"34","author":"Chiu","year":"2016","journal-title":"Libr. Hi Tech"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1007\/s11423-017-9513-6","article-title":"Understanding the quality factors that influence the continuance intention of students toward articipation in MOOCs","volume":"65","author":"Yang","year":"2017","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/0735633116671324","article-title":"The Measurement and Dimensionality of Mobile Learning Systems Success: Two-Stage Development and Validation","volume":"55","author":"Lin","year":"2017","journal-title":"J. Educ. Comput. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1108\/IntR-11-2013-0232","article-title":"Understanding the relationships of critical factors to Facebook educational usage intention","volume":"25","author":"Wu","year":"2015","journal-title":"Internet Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1111\/j.1365-2575.2007.00268.x","article-title":"Assessing e-commerce systems success: A respecification and validation of the DeLone and McLean model of IS success","volume":"18","author":"Wang","year":"2008","journal-title":"Inf. Syst. J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"225","DOI":"10.3233\/IP-140341","article-title":"Evaluating e-government systems in Turkey: The case of the \u2018e-movable system\u2019","volume":"19","author":"Sebetci","year":"2014","journal-title":"Inf. Polity"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1108\/EL-11-2012-0148","article-title":"Effectiveness of information technology in reducing corruption in China: A validation of the DeLone and McLean information systems success model","volume":"33","author":"Hu","year":"2015","journal-title":"Electron. Libr."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1028","DOI":"10.1016\/j.eswa.2006.02.004","article-title":"Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL","volume":"32","author":"Tzeng","year":"2007","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1444","DOI":"10.1016\/j.eswa.2007.11.058","article-title":"Selecting management systems for sustainable development in SMEs: A novel hybrid model based on DEMATEL, ANP, and ZOGP","volume":"36","author":"Tsai","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.jairtraman.2007.10.002","article-title":"Building an effective safety management system for airlines","volume":"14","author":"Liou","year":"2008","journal-title":"J. Air Transp. Manag."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1981","DOI":"10.1016\/j.eswa.2009.06.079","article-title":"Using a novel conjunctive MCDM approach based on DEMATEL, fuzzy ANP, and TOPSIS as an innovation support system for Taiwanese higher education","volume":"37","author":"Chen","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1080\/18756891.2015.1001945","article-title":"Identifying critical success factors in EDA industry using DEMATEL method","volume":"8","author":"Sun","year":"2015","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1016\/j.eswa.2012.10.040","article-title":"Integrating DANP and modified grey relation theory for the selection of an outsourcing provider","volume":"40","author":"Hsu","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.knosys.2013.10.010","article-title":"A method for exploring the interdependencies and importance of critical infrastructures","volume":"55","author":"Huang","year":"2014","journal-title":"Knowl. Based. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.jclepro.2014.10.053","article-title":"Building an effective system for carbon reduction management","volume":"103","author":"Liou","year":"2015","journal-title":"J. Clean. Prod."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Shao, Q.G., Liou, J.J.H., Weng, S.S., and Chuang, Y.C. (2018). Improving the Green Building Evaluation System in China Based on the DANP Method. Sustainability, 10.","DOI":"10.3390\/su10041173"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1016\/j.ijpe.2013.07.011","article-title":"A grey-based DEMATEL model for evaluating business process management critical success factors","volume":"146","author":"Bai","year":"2013","journal-title":"Int. J. Prod. Econ."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.jclepro.2011.06.010","article-title":"Using fuzzy DEMATEL to evaluate the green supply chain management practices","volume":"40","author":"Lin","year":"2013","journal-title":"J. Clean. Prod."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"7202","DOI":"10.1016\/j.eswa.2015.04.030","article-title":"Intuitionistic fuzzy based DEMATEL method for developing green practices and performances in a green supply chain","volume":"42","author":"Govindan","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.cie.2017.06.020","article-title":"A rough DEMATEL-based approach for evaluating interaction between requirements of product-service system","volume":"110","author":"Song","year":"2017","journal-title":"Comput. Ind. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.eswa.2017.06.037","article-title":"Novel approach to group multi-criteria decision making based on interval rough numbers: Hybrid DEMATEL-ANP-MAIRCA model","volume":"88","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/BF01001956","article-title":"Rough sets","volume":"11","author":"Pawlak","year":"1982","journal-title":"Int. J. Comput. Inf. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1007\/s00170-007-0989-9","article-title":"A rough set enhanced fuzzy approach to quality function deployment","volume":"37","author":"Zhai","year":"2008","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1016\/j.ejor.2009.03.031","article-title":"Using FSBT technique with Rough Set Theory for personal investment portfolio analysis","volume":"201","author":"Shyng","year":"2010","journal-title":"Eur. J. Oper. Res."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1267","DOI":"10.3233\/IDA-150242","article-title":"An assessment method for the impact of missing data in the rough set-based decision fusion","volume":"20","author":"Han","year":"2016","journal-title":"Intell. Data Anal."},{"key":"ref_49","first-page":"1231","article-title":"Improving airline service quality based on rough set theory and flow graphs","volume":"33","author":"Liou","year":"2016","journal-title":"J. Ind. Prod. Eng."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.knosys.2012.06.017","article-title":"A new hybrid MCDM model combining DANP with VIKOR to improve e-store business","volume":"37","author":"Chiu","year":"2013","journal-title":"Knowl. Based. Syst."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.jclepro.2016.12.145","article-title":"Identifying critical risk factors of sustainable supply chain management: A rough strength-relation analysis method","volume":"143","author":"Song","year":"2017","journal-title":"J. Clean. Prod."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.knosys.2013.07.023","article-title":"Combining VIKOR-DANP model for glamor stock selection and stock performance improvement","volume":"58","author":"Shen","year":"2014","journal-title":"Knowl. Based. Syst."},{"key":"ref_53","first-page":"131","article-title":"The new method of multicriteria complex proportional assesment projects","volume":"1","author":"Zavadskas","year":"1994","journal-title":"Technol. Econ. Dev. Econ."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"462","DOI":"10.3846\/1392-8619.2008.14.462-477","article-title":"Multicriteria Selection of Project Managers by Applying Grey Criteria","volume":"14","author":"Zavadskas","year":"2008","journal-title":"Technol. Econ. Dev. Econ."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"134","DOI":"10.5755\/j01.ee.22.2.310","article-title":"The comparative analysis of MCDA methods SAW and COPRAS","volume":"22","author":"Podvezko","year":"2011","journal-title":"Eng. Econ."},{"key":"ref_56","first-page":"6","article-title":"Froset roads location based on AHP and COPRAS-G method: An emprical study based on Iran","volume":"14","author":"Zolfani","year":"2011","journal-title":"E M Ekon. Manag."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1080\/00207543.2015.1010747","article-title":"New hybrid COPRAS-G MADM Model for improving and selecting suppliers in green supply chain management","volume":"54","author":"Liou","year":"2016","journal-title":"Int. J. Prod. Res."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.jclepro.2018.02.186","article-title":"Evaluating the performance of suppliers based on using the R\u2019AMATEL-MAIRCA method for green supply chain implementation in electronics industry","volume":"184","author":"Chatterjee","year":"2018","journal-title":"J. Clean. Prod."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Xie, M., Wu, L.F., Li, B., and Li, Z.C. (2019). A novel hybrid multivariate nonlinear grey model for forecasting the traffic-related emissions. Appl. Math. Model.","DOI":"10.1016\/j.apm.2019.09.013"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/10\/1248\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:28:04Z","timestamp":1760189284000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/10\/1248"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,7]]},"references-count":59,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["sym11101248"],"URL":"https:\/\/doi.org\/10.3390\/sym11101248","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2019,10,7]]}}}