{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T19:55:12Z","timestamp":1786046112689,"version":"3.56.0"},"reference-count":144,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2020,9,20]],"date-time":"2020-09-20T00:00:00Z","timestamp":1600560000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Multi-Criteria Decision-Analysis (MCDA) methods are successfully applied in different fields and disciplines. However, in many studies, the problem of selecting the proper methods and parameters for the decision problems is raised. The paper undertakes an attempt to benchmark selected Multi-Criteria Decision Analysis (MCDA) methods. To achieve that, a set of feasible MCDA methods was identified. Based on reference literature guidelines, a simulation experiment was planned. The formal foundations of the authors\u2019 approach provide a reference set of MCDA methods ( Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR), Complex Proportional Assessment (COPRAS), and PROMETHEE II: Preference Ranking Organization Method for Enrichment of Evaluations) along with their similarity coefficients (Spearman correlation coefficients and WS coefficient). This allowed the generation of a set of models differentiated by the number of attributes and decision variants, as well as similarity research for the obtained rankings sets. As the authors aim to build a complex benchmarking model, additional dimensions were taken into account during the simulation experiments. The aspects of the performed analysis and benchmarking methods include various weighing methods (results obtained using entropy and standard deviation methods) and varied techniques of normalization of MCDA model input data. Comparative analyses showed the detailed influence of values of particular parameters on the final form and a similarity of the final rankings obtained by different MCDA methods.<\/jats:p>","DOI":"10.3390\/sym12091549","type":"journal-article","created":{"date-parts":[[2020,9,20]],"date-time":"2020-09-20T21:20:28Z","timestamp":1600636828000},"page":"1549","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":433,"title":["Are MCDA Methods Benchmarkable? A Comparative Study of TOPSIS, VIKOR, COPRAS, and PROMETHEE II Methods"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7076-2519","authenticated-orcid":false,"given":"Wojciech","family":"Sa\u0142abun","sequence":"first","affiliation":[{"name":"Research Team on Intelligent Decision Support Systems, Department of Artificial Intelligence Methods and Applied Mathematics, Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Szczecin ul. \u017bo\u0142nierska 49, 71-210 Szczecin, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4415-9414","authenticated-orcid":false,"given":"Jaros\u0142aw","family":"W\u0105tr\u00f3bski","sequence":"additional","affiliation":[{"name":"Department of Information Systems Engineering, Faculty of Economics and Management, University of Szczecin, Mickiewicza 64, 71-101 Szczecin, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0834-2019","authenticated-orcid":false,"given":"Andrii","family":"Shekhovtsov","sequence":"additional","affiliation":[{"name":"Research Team on Intelligent Decision Support Systems, Department of Artificial Intelligence Methods and Applied Mathematics, Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Szczecin ul. \u017bo\u0142nierska 49, 71-210 Szczecin, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Greco, S., Figueira, J., and Ehrgott, M. (2016). Multiple Criteria Decision Analysis, Springer.","DOI":"10.1007\/978-1-4939-3094-4"},{"key":"ref_2","unstructured":"Roy, B. (2013). Multicriteria Methodology for Decision Aiding, Springer Science & Business Media."},{"key":"ref_3","unstructured":"Kodikara, P.N. (2008). Multi-Objective Optimal Operation of Urban Water Supply Systems. [Ph.D. Thesis, Victoria University]."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.omega.2015.05.013","article-title":"Comparative analysis of MCDM methods for the assessment of sustainable housing affordability","volume":"59","author":"Mulliner","year":"2016","journal-title":"Omega"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1028","DOI":"10.1016\/j.eswa.2006.02.004","article-title":"Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL","volume":"32","author":"Tzeng","year":"2007","journal-title":"Expert Syst. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"110916","DOI":"10.1016\/j.jenvman.2020.110916","article-title":"A study on offshore wind farm siting criteria using a novel interval-valued fuzzy-rough based Delphi method","volume":"270","author":"Deveci","year":"2020","journal-title":"J. Environ. Manag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.jclepro.2018.02.186","article-title":"Evaluating the performance of suppliers based on using the R\u2019AMATEL-MAIRCA method for green supply chain implementation in electronics industry","volume":"184","author":"Chatterjee","year":"2018","journal-title":"J. Clean. Prod."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/S0377-2217(01)00243-0","article-title":"Multicriteria classification and sorting methods: A literature review","volume":"138","author":"Zopounidis","year":"2002","journal-title":"Eur. J. Oper. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"13051","DOI":"10.1016\/j.eswa.2012.05.056","article-title":"A state-of the-art survey of TOPSIS applications","volume":"39","author":"Behzadian","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2294","DOI":"10.1016\/j.procs.2019.09.404","article-title":"The fuzzy TOPSIS applications in the last decade","volume":"159","author":"Palczewski","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_11","first-page":"40","article-title":"The mean error estimation of TOPSIS method using a fuzzy reference models","volume":"7","year":"2013","journal-title":"J. Theor. Appl. Comput. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"514","DOI":"10.1016\/j.ejor.2006.01.020","article-title":"Extended VIKOR method in comparison with outranking methods","volume":"178","author":"Opricovic","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Saaty, T. (1980). The Analytic Hierarchy Process, Mcgraw Hill.","DOI":"10.21236\/ADA214804"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/0022-2496(77)90033-5","article-title":"A scaling method for priorities in hierarchical structures","volume":"15","author":"Saaty","year":"1977","journal-title":"J. Math. Psychol."},{"key":"ref_15","first-page":"195","article-title":"Multi-attribute assessment of road design solutions by using the COPRAS method","volume":"2","author":"Zavadskas","year":"2007","journal-title":"Balt. J. Road Bridge Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ejor.2015.07.019","article-title":"ELECTRE: A comprehensive literature review on methodologies and applications","volume":"250","author":"Govindan","year":"2016","journal-title":"Eur. J. Oper. Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1554","DOI":"10.1016\/j.apm.2015.08.011","article-title":"Multicriteria group decision making with ELECTRE III method based on interval-valued intuitionistic fuzzy information","volume":"40","author":"Hashemi","year":"2016","journal-title":"Appl. Math. Model."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/0377-2217(86)90044-5","article-title":"How to select and how to rank projects: The PROMETHEE method","volume":"24","author":"Brans","year":"1986","journal-title":"Eur. J. Oper. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/s00267-015-0503-3","article-title":"Hybrid MCDA methods to integrate multiple ecosystem services in forest management planning: A critical review","volume":"56","author":"Uhde","year":"2015","journal-title":"Environ. Manag."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1016\/j.dib.2018.12.015","article-title":"Generalised framework for multi-criteria method selection: Rule set database and exemplary decision support system implementation blueprints","volume":"22","author":"Jankowski","year":"2019","journal-title":"Data Brief"},{"key":"ref_21","unstructured":"Wi\u0119ckowski, J., Kizielewicz, B., and Ko\u0142odziejczyk, J. Application of Hill Climbing Algorithm in Determining the Characteristic Objects Preferences Based on the Reference Set of Alternatives. Proceedings of the International Conference on Intelligent Decision Technologies."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.omega.2018.07.004","article-title":"Generalised framework for multi-criteria method selection","volume":"86","author":"Jankowski","year":"2019","journal-title":"Omega"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1016\/S0377-2217(98)00073-3","article-title":"Tentative guidelines to help choosing an appropriate MCDA method","volume":"109","author":"Guitouni","year":"1998","journal-title":"Eur. J. Oper. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1016\/S0377-2217(97)00147-1","article-title":"Multi-attribute decision making: A simulation comparison of select methods","volume":"107","author":"Zanakis","year":"1998","journal-title":"Eur. J. Oper. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1016\/0377-2217(84)90214-5","article-title":"The role of weights and scales in the application of multiobjective decision making","volume":"15","author":"Gershon","year":"1984","journal-title":"Eur. J. Oper. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"102261","DOI":"10.1016\/j.omega.2020.102261","article-title":"How to Support the Application of Multiple Criteria Decision Analysis? Let Us Start with a Comprehensive Taxonomy","volume":"96","author":"Cinelli","year":"2020","journal-title":"Omega"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hanne, T. (1999). Meta decision problems in multiple criteria decision making. Multicriteria Decision Making, Springer.","DOI":"10.1007\/978-1-4615-5025-9_6"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.omega.2005.12.003","article-title":"Ranking irregularities when evaluating alternatives by using some ELECTRE methods","volume":"36","author":"Wang","year":"2008","journal-title":"Omega"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2179","DOI":"10.1016\/j.asoc.2012.12.009","article-title":"A new method selection approach for fuzzy group multicriteria decision making","volume":"13","author":"Chang","year":"2013","journal-title":"Appl. Soft Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/j.ejor.2006.10.045","article-title":"A comparison of multiple criteria analysis techniques for water resource management","volume":"184","author":"Hajkowicz","year":"2008","journal-title":"Eur. J. Oper. Res."},{"key":"ref_31","unstructured":"Zak, J. (2005, January 13\u201316). The comparison of multiobjective ranking methods applied to solve the mass transit systems\u2019 decision problems. Proceedings of the 10th Jubilee Meeting of the EURO Working Group on Transportation, Poznan, Poland."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pamu\u010dar, D., Stevi\u0107, \u017d., and Sremac, S. (2018). A new model for determining weight coefficients of criteria in mcdm models: Full consistency method (fucom). Symmetry, 10.","DOI":"10.3390\/sym10090393"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zardari, N.H., Ahmed, K., Shirazi, S.M., and Yusop, Z.B. (2015). Weighting Methods and Their Effects on Multi-Criteria Decision Making Model Outcomes in Water Resources Management, Springer.","DOI":"10.1007\/978-3-319-12586-2"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"W\u0105tr\u00f3bski, J., Ziemba, E., Karczmarczyk, A., and Jankowski, J. (2018). An index to measure the sustainable information society: The Polish households case. Sustainability, 10.","DOI":"10.3390\/su10093223"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sa\u0142abun, W., Palczewski, K., and W\u0105tr\u00f3bski, J. (2019). Multicriteria approach to sustainable transport evaluation under incomplete knowledge: Electric bikes case study. Sustainability, 11.","DOI":"10.3390\/su11123314"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"W\u0105tr\u00f3bski, J., and Sa\u0142abun, W. (2016). Green supplier selection framework based on multi-criteria decision-analysis approach. Proceedings of the International Conference on Sustainable Design and Manufacturing, Springer.","DOI":"10.1007\/978-3-319-32098-4_31"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"533","DOI":"10.3846\/20294913.2013.814606","article-title":"A novel hybrid SWARA and VIKOR methodology for supplier selection in an agile environment","volume":"19","author":"Alimardani","year":"2013","journal-title":"Technol. Econ. Dev. Econ."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1007\/s001700200227","article-title":"Selecting plant location via a fuzzy TOPSIS approach","volume":"20","author":"Chu","year":"2002","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_39","unstructured":"Madi\u0107, M., Markovi\u0107, D., Petrovi\u0107, G., and Radovanovi\u0107, M. (2014). Application of COPRAS method for supplier selection. The Fifth International Conference Transport and Logistics-TIL 2014, Proceedings."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"412","DOI":"10.3846\/16484142.2014.983966","article-title":"Logistics freight center locations decision by using Fuzzy-PROMETHEE","volume":"29","author":"Elevli","year":"2014","journal-title":"Transport"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"106231","DOI":"10.1016\/j.cie.2019.106231","article-title":"Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement of alternatives and ranking according to COmpromise solution (MARCOS)","volume":"140","author":"Chatterjee","year":"2020","journal-title":"Comput. Ind. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1142\/S0218539310003779","article-title":"Selection of maintenance strategy for aircraft systems using multi-criteria decision making methodologies","volume":"17","author":"Ahmadi","year":"2010","journal-title":"Int. J. Reliab. Qual. Saf. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1975","DOI":"10.1080\/00207540601011519","article-title":"Evaluating flexible manufacturing systems using a combined multiple attribute decision making method","volume":"46","year":"2008","journal-title":"Int. J. Prod. Res."},{"key":"ref_44","first-page":"5","article-title":"Decision making in machine tool selection: An integrated approach with SWARA and COPRAS-G methods","volume":"24","author":"Aghdaie","year":"2013","journal-title":"Eng. Econ."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2029","DOI":"10.1007\/s11269-013-0271-x","article-title":"A compromise ratio method with an application to water resources management: An intuitionistic fuzzy set","volume":"27","author":"Hashemi","year":"2013","journal-title":"Water Resour. Manag."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1007\/s00267-005-0151-0","article-title":"Catchment-wide wetland assessment and prioritization using the multi-criteria decision-making method TOPSIS","volume":"38","author":"Liu","year":"2006","journal-title":"Environ. Manag."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"138499","DOI":"10.1016\/j.scitotenv.2020.138499","article-title":"Inter-basin water transfer planning with grey COPRAS and fuzzy COPRAS techniques: A case study in Iranian Central Plateau","volume":"726","author":"Roozbahani","year":"2020","journal-title":"Sci. Total Environ."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1690","DOI":"10.1016\/j.wasman.2006.07.009","article-title":"A multiple criteria analysis for household solid waste management in the urban community of Dakar","volume":"27","author":"Kapepula","year":"2007","journal-title":"Waste Manag."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Carnero, M.C. (2020). Waste Segregation FMEA Model Integrating Intuitionistic Fuzzy Set and the PAPRIKA Method. Mathematics, 8.","DOI":"10.3390\/math8081375"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1080\/15567240903047483","article-title":"The evaluation of renewable energy technologies for electricity generation in Turkey using intuitionistic fuzzy TOPSIS","volume":"7","author":"Boran","year":"2012","journal-title":"Energy Sources Part B Econ. Plan. Policy"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2517","DOI":"10.1016\/j.energy.2010.02.051","article-title":"Multicriteria renewable energy planning using an integrated fuzzy VIKOR & AHP methodology: The case of Istanbul","volume":"35","author":"Kaya","year":"2010","journal-title":"Energy"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Krishankumar, R., Ravichandran, K., Kar, S., Cavallaro, F., Zavadskas, E.K., and Mardani, A. (2019). Scientific decision framework for evaluation of renewable energy sources under q-rung orthopair fuzzy set with partially known weight information. Sustainability, 11.","DOI":"10.3390\/su11154202"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Rehman, A.U., Abidi, M.H., Umer, U., and Usmani, Y.S. (2019). Multi-Criteria Decision-Making Approach for Selecting Wind Energy Power Plant Locations. Sustainability, 11.","DOI":"10.3390\/su11216112"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"W\u0105tr\u00f3bski, J., Ziemba, P., and Wolski, W. (2015, January 13\u201316). Methodological aspects of decision support system for the location of renewable energy sources. Proceedings of the 2015 Federated Conference on Computer Science and Information Systems (FedCSIS), Lodz, Poland.","DOI":"10.15439\/2015F294"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Riaz, M., Sa\u0142abun, W., Farid, H.M.A., Ali, N., and W\u0105tr\u00f3bski, J. (2020). A Robust q-Rung Orthopair Fuzzy Information Aggregation Using Einstein Operations with Application to Sustainable Energy Planning Decision Management. Energies, 13.","DOI":"10.3390\/en13092155"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.1007\/s00170-005-0284-6","article-title":"Optimization of multi-response processes using the VIKOR method","volume":"31","author":"Tong","year":"2007","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s00170-004-2157-9","article-title":"Optimization of multiple responses using principal component analysis and technique for order preference by similarity to ideal solution","volume":"27","author":"Tong","year":"2005","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Mlela, M.K., Xu, H., Sun, F., Wang, H., and Madenge, G.D. (2020). Material Analysis and Molecular Dynamics Simulation for Cavitation Erosion and Corrosion Suppression in Water Hydraulic Valves. Materials, 13.","DOI":"10.3390\/ma13020453"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"56","DOI":"10.4018\/ijsds.2014040105","article-title":"VIKOR and its applications: A state-of-the-art survey","volume":"5","author":"Yazdani","year":"2014","journal-title":"Int. J. Strateg. Decis. Sci. (IJSDS)"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.ejor.2009.01.021","article-title":"PROMETHEE: A comprehensive literature review on methodologies and applications","volume":"200","author":"Behzadian","year":"2010","journal-title":"Eur. J. Oper. Res."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"397","DOI":"10.3846\/20294913.2011.593291","article-title":"Multiple criteria decision making (MCDM) methods in economics: An overview","volume":"17","author":"Zavadskas","year":"2011","journal-title":"Technol. Econ. Dev. Econ."},{"key":"ref_62","unstructured":"Guitouni, A., Martel, J.M., Vincke, P., and North, P. (1998). A Framework to Choose a Discrete Multicriterion Aggregation Procedure, Defence Research Establishment Valcatier (DREV). Available online: https:\/\/pdfs.semanticscholar.org\/27d5\/9c846657268bc840c4df8df98e85de66c562.pdf."},{"key":"ref_63","unstructured":"Spronk, J., Steuer, R.E., and Zopounidis, C. (2005). Multicriteria decision aid\/analysis in finance. Multiple Criteria Decision Analysis: State of the Art Surveys, Springer."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Roy, B. (1990). The outranking approach and the foundations of ELECTRE methods. Readings in Multiple Criteria Decision Aid, Springer.","DOI":"10.1007\/978-3-642-75935-2_8"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Ishizaka, A., and Nemery, P. (2013). Multi-Criteria Decision Analysis: Methods and Software, John Wiley & Sons.","DOI":"10.1002\/9781118644898"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Stevi\u0107, \u017d., Pamu\u010dar, D., Suboti\u0107, M., Antuchevi\u010diene, J., and Zavadskas, E.K. (2018). The location selection for roundabout construction using Rough BWM-Rough WASPAS approach based on a new Rough Hamy aggregator. Sustainability, 10.","DOI":"10.3390\/su10082817"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Fortemps, P., Greco, S., and S\u0142owi\u0144ski, R. (2004). Multicriteria choice and ranking using decision rules induced from rough approximation of graded preference relations. Proceedings of the International Conference on Rough Sets and Current Trends in Computing, Springer.","DOI":"10.1007\/978-3-540-25929-9_62"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"e Costa, C.A.B., and Vincke, P. (1990). Multiple criteria decision aid: An overview. Readings in Multiple Criteria Decision Aid, Springer.","DOI":"10.1007\/978-3-642-75935-2_1"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"3691","DOI":"10.1016\/j.cor.2006.01.017","article-title":"Using a hybrid multi-criteria decision aid method for information systems outsourcing","volume":"34","author":"Wang","year":"2007","journal-title":"Comput. Oper. Res."},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Figueira, J., Mousseau, V., and Roy, B. (2005). ELECTRE methods. Multiple Criteria Decision Analysis: State of the Art Surveys, Springer.","DOI":"10.1007\/b100605"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1023\/A:1016626919680","article-title":"Multi-criteria methodology contribution to the software quality evaluation","volume":"9","author":"Blin","year":"2001","journal-title":"Softw. Qual. J."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Edwards, W., Newman, J.R., Snapper, K., and Seaver, D. (1982). Multiattribute Evaluation, SAGE Publications. Number 26; Chronicle Books.","DOI":"10.4135\/9781412985123"},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/0377-2217(82)90155-2","article-title":"Assessing a set of additive utility functions for multicriteria decision-making, the UTA method","volume":"10","author":"Siskos","year":"1982","journal-title":"Eur. J. Oper. Res."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1111\/j.1475-3995.1994.00325.x","article-title":"MACBETH\u2014An interactive path towards the construction of cardinal value functions","volume":"1","author":"Vansnick","year":"1994","journal-title":"Int. Trans. Oper. Res."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1002\/(SICI)1099-1360(199703)6:2<107::AID-MCDA147>3.0.CO;2-1","article-title":"Applications of the MACBETH approach in the framework of an additive aggregation model","volume":"6","author":"Vansnick","year":"1997","journal-title":"J. Multi-Criteria Decis. Anal."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/j.ijpe.2005.03.009","article-title":"A fuzzy approach for supplier evaluation and selection in supply chain management","volume":"102","author":"Chen","year":"2006","journal-title":"Int. J. Prod. Econ."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"4257","DOI":"10.1016\/j.apm.2011.02.040","article-title":"Group decision making based on novel fuzzy modified TOPSIS method","volume":"35","author":"Vahdani","year":"2011","journal-title":"Appl. Math. Model."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1016\/j.asoc.2014.04.002","article-title":"Robot selection by using generalized interval-valued fuzzy numbers with TOPSIS","volume":"21","author":"Rashid","year":"2014","journal-title":"Appl. Soft Comput."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"4190","DOI":"10.1016\/j.eswa.2010.09.081","article-title":"Fuzzy TOPSIS for group decision making: A case study for accidents with oil spill in the sea","volume":"38","author":"Krohling","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref_80","first-page":"57","article-title":"Classement et choix en pr\u00e9sence de points de vue multiples","volume":"2","author":"Roy","year":"1968","journal-title":"Rev. Fran\u00e7aise Inform. Rech. Oper."},{"key":"ref_81","unstructured":"Roy, B., and Skalka, J.M. (1987). ELECTRE IS: Aspects M\u00e9thodologiques et Guide D\u2019utilisation, Universit\u00e9 de Paris Dauphine. LAMSADE, Unit\u00e9 Associ\u00e9e au CNRS no 825."},{"key":"ref_82","unstructured":"Roy, B., Bertier, P., and La m\u00e9thode ELECTRE, I. (1973). Une Application au Media Planning, North Holland."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1016\/0191-2607(82)90057-7","article-title":"Ranking of suburban line extension projects on the Paris metro system by a multicriteria method","volume":"16","author":"Roy","year":"1982","journal-title":"Transp. Res. Part A Gen."},{"key":"ref_84","unstructured":"Roy, B., and Bouyssou, D. (1993). Aide Multicrit\u00e8re \u00e0 la D\u00e9cision: M\u00e9thodes et Cas, Economica Paris."},{"key":"ref_85","unstructured":"Mareschal, B., Brans, J.P., and Vincke, P. (1984). PROMETHEE: A New Family of Outranking Methods in Multicriteria Analysis, ULB\u2014Universite Libre de. Technical Report."},{"key":"ref_86","doi-asserted-by":"crossref","unstructured":"Janssens, G.K., and Pangilinan, J.M. (2010, January 6\u20137). Multiple criteria performance analysis of non-dominated sets obtained by multi-objective evolutionary algorithms for optimisation. Proceedings of the IFIP International Conference on Artificial Intelligence Applications and Innovations, Larnaca, Cyprus.","DOI":"10.1007\/978-3-642-16239-8_15"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1007\/s00500-013-1109-4","article-title":"A PROMETHEE-based outranking method for multiple criteria decision analysis with interval type-2 fuzzy sets","volume":"18","author":"Chen","year":"2014","journal-title":"Soft Comput."},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/S0921-8009(97)00120-1","article-title":"Weak comparability of values as a foundation for ecological economics","volume":"26","author":"Munda","year":"1998","journal-title":"Ecol. Econ."},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/0921-8009(96)00048-1","article-title":"Cost-benefit analysis in integrated environmental assessment: Some methodological issues","volume":"19","author":"Munda","year":"1996","journal-title":"Ecol. Econ."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1002\/mcda.443","article-title":"Quantifying uncertainty using robustness analysis in the application of ORESTE to sewer rehabilitation projects prioritization\u2014Brussels case study","volume":"16","author":"Ana","year":"2009","journal-title":"J. Multi-Criteria Decis. Anal."},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0377-2217(82)90131-X","article-title":"Preference relations on actions and criteria in multicriteria decision making","volume":"10","author":"Roubens","year":"1982","journal-title":"Eur. J. Oper. Res."},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/BF00221383","article-title":"Qualitative multiple criteria choice analysis","volume":"24","author":"Hinloopen","year":"1990","journal-title":"Qual. Quant."},{"key":"ref_93","unstructured":"Martel, J.M., and Matarazzo, B. (2005). Other outranking approaches. Multiple Criteria Decision Analysis: State of the Art Surveys, Springer."},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.ejor.2006.03.015","article-title":"An axiomatic characterization of different majority concepts","volume":"179","author":"Marchant","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"ref_95","first-page":"4","article-title":"An Automated Explanation Approach for a Decision Support System based on MCDA","volume":"5","author":"Martel","year":"2005","journal-title":"ExaCt"},{"key":"ref_96","unstructured":"Guitouni, A., Martel, J., B\u00e9langer, M., and Hunter, C. (1999). Managing a Decision-Making Situation in the Context of the Canadian Airspace Protection, Facult\u00e9 des Sciences de L\u2019administration de L\u2019Universit\u00e9 Laval, Direction de la Recherche."},{"key":"ref_97","unstructured":"Nijkamp, P., Rietveld, P., and Voogd, H. (2013). Multicriteria Evaluation in Physical Planning, Elsevier."},{"key":"ref_98","doi-asserted-by":"crossref","unstructured":"Vincke, P. (1999). Outranking approach. Multicriteria Decision Making, Springer.","DOI":"10.1007\/978-1-4615-5025-9_11"},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1111\/j.1435-5597.1976.tb00958.x","article-title":"Qualitative multiple criteria analysis, environmental protection and multiregional development","volume":"Volume 36","author":"Paelinck","year":"1976","journal-title":"Papers of the Regional Science Association"},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/0377-2217(91)90322-M","article-title":"MAPPAC as a compromise between outranking methods and MAUT","volume":"54","author":"Matarazzo","year":"1991","journal-title":"Eur. J. Oper. Res."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1016\/S0377-2217(96)00022-7","article-title":"A new pcca method: Idra","volume":"98","author":"Greco","year":"1997","journal-title":"Eur. J. Oper. Res."},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1002\/mcda.1525","article-title":"The Characteristic Objects Method: A New Distance-based Approach to Multicriteria Decision-making Problems","volume":"22","year":"2015","journal-title":"J. Multi-Criteria Decis. Anal."},{"key":"ref_103","doi-asserted-by":"crossref","first-page":"25","DOI":"10.2298\/JMMB150806023K","article-title":"Assessment of ductile iron casting process with the use of the DRSA method","volume":"52","author":"Rogalewicz","year":"2016","journal-title":"J. Min. Metall. Sect. B Metall."},{"key":"ref_104","doi-asserted-by":"crossref","unstructured":"Wi\u0119ckowski, J., Kizielewicz, B., and Ko\u0142odziejczyk, J. (2020, January 17\u201319). Finding an Approximate Global Optimum of Characteristic Objects Preferences by Using Simulated Annealing. Proceedings of the International Conference on Intelligent Decision Technologies, Split, Croatia.","DOI":"10.1007\/978-981-15-5925-9_31"},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Sa\u0142abun, W., W\u0105tr\u00f3bski, J., and Piegat, A. (2016, January 12\u201316). Identification of a multi-criteria model of location assessment for renewable energy sources. Proceedings of the International Conference on Artificial Intelligence and Soft Computing, Zakopane, Poland.","DOI":"10.1007\/978-3-319-39378-0_28"},{"key":"ref_106","doi-asserted-by":"crossref","unstructured":"Sa\u0142abun, W., Karczmarczyk, A., W\u0105tr\u00f3bski, J., and Jankowski, J. (2018, January 18\u201321). Handling data uncertainty in decision making with COMET. Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, India.","DOI":"10.1109\/SSCI.2018.8628934"},{"key":"ref_107","doi-asserted-by":"crossref","unstructured":"Wi\u0119ckowski, J., Kizielewicz, B., and Ko\u0142odziejczyk, J. (2020, January 17\u201319). The Search of the Optimal Preference Values of the Characteristic Objects by Using Particle Swarm Optimization in the Uncertain Environment. Proceedings of the International Conference on Intelligent Decision Technologies, Split, Croatia.","DOI":"10.1007\/978-981-15-5925-9_30"},{"key":"ref_108","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1016\/j.ijar.2009.02.003","article-title":"Variable-precision dominance-based rough set approach and attribute reduction","volume":"50","author":"Inuiguchi","year":"2009","journal-title":"Int. J. Approx. Reason."},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1002\/int.10014","article-title":"Rough approximation by dominance relations","volume":"17","author":"Greco","year":"2002","journal-title":"Int. J. Intell. Syst."},{"key":"ref_110","doi-asserted-by":"crossref","unstructured":"S\u0142owi\u0144ski, R., Greco, S., and Matarazzo, B. (2014). Rough-set-based decision support. Search Methodologies, Springer.","DOI":"10.1007\/978-1-4614-6940-7_19"},{"key":"ref_111","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1016\/S0377-2217(96)00277-9","article-title":"Model choice in multicriteria decision aid","volume":"97","author":"Pearman","year":"1997","journal-title":"Eur. J. Oper. Res."},{"key":"ref_112","doi-asserted-by":"crossref","unstructured":"Kornyshova, E., and Salinesi, C. (2007, January 1\u20135). MCDM techniques selection approaches: State of the art. Proceedings of the 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making, Honolulu, HI, USA.","DOI":"10.1109\/MCDM.2007.369412"},{"key":"ref_113","doi-asserted-by":"crossref","unstructured":"Gershon, M., and Duckstein, L. (1983). An algorithm for choosing of a multiobjective technique. Essays and Surveys on Multiple Criteria Decision Making, Springer.","DOI":"10.1007\/978-3-642-46473-7_6"},{"key":"ref_114","doi-asserted-by":"crossref","unstructured":"Celik, M., Cicek, K., and Cebi, S. (2009, January 6\u20139). Establishing an international MBA program for shipping executives: Managing OR\/MS foundation towards a unique curriculum design. Proceedings of the 2009 International Conference on Computers & Industrial Engineering, Troyes, France.","DOI":"10.1109\/ICCIE.2009.5223818"},{"key":"ref_115","first-page":"211","article-title":"Identifying operational requirements to select suitable decision models for a public sector e-procurement decision support system","volume":"11","author":"Adil","year":"2014","journal-title":"JISTEM J. Inf. Syst. Technol. Manag."},{"key":"ref_116","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1007\/BF03326246","article-title":"An appropriate multiple criteria decision making method for solving electricity planning problems, addressing sustainability issue","volume":"8","author":"Moghaddam","year":"2011","journal-title":"Int. J. Environ. Sci. Technol."},{"key":"ref_117","unstructured":"Salinesi, C., and Kornyshova, E. (2020, July 10). Choosing a Prioritization Method-Case of IS Security Improvement. CAiSE Forum, Available online: https:\/\/pdfs.semanticscholar.org\/ef7b\/3d0658d0176aee128562b8a9a4c0bb278354.pdf."},{"key":"ref_118","first-page":"99","article-title":"Assessing the quality of different MCDA methods","volume":"4","author":"Omann","year":"2004","journal-title":"Altern. Environ. Valuat."},{"key":"ref_119","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.ecolind.2014.06.011","article-title":"Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment","volume":"46","author":"Cinelli","year":"2014","journal-title":"Ecol. Indic."},{"key":"ref_120","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1016\/j.trpro.2016.11.051","article-title":"Outline of multicriteria decision-making in green logistics","volume":"16","year":"2016","journal-title":"Transp. Res. Procedia"},{"key":"ref_121","doi-asserted-by":"crossref","unstructured":"W\u0105tr\u00f3bski, J., and Jankowski, J. (2015, January 13\u201316). Knowledge management in MCDA domain. Proceedings of the 2015 Federated Conference on Computer Science and Information Systems (FedCSIS), Lodz, Poland.","DOI":"10.15439\/2015F295"},{"key":"ref_122","doi-asserted-by":"crossref","unstructured":"W\u0105tr\u00f3bski, J., and Jankowski, J. (2016). Guideline for MCDA method selection in production management area. New Frontiers in Information and Production Systems Modelling and Analysis, Springer.","DOI":"10.1007\/978-3-319-23338-3_6"},{"key":"ref_123","doi-asserted-by":"crossref","unstructured":"Ulengin, F., Topcu, Y.I., and Sahin, S.O. (2001). An artificial neural network approach to multicriteria model selection. Multiple Criteria Decision Making in the New Millennium, Springer.","DOI":"10.1007\/978-3-642-56680-6_9"},{"key":"ref_124","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1111\/j.1366-9516.2005.00202.x","article-title":"Incorporating multiple criteria into the design of conservation area networks: A minireview with recommendations","volume":"12","author":"Moffett","year":"2006","journal-title":"Divers. Distrib."},{"key":"ref_125","doi-asserted-by":"crossref","unstructured":"Hwang, C.L., and Yoon, K. (1981). Methods for multiple attribute decision making. Multiple Attribute Decision Making, Springer.","DOI":"10.1007\/978-3-642-48318-9"},{"key":"ref_126","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1080\/03088830903346038","article-title":"Analytical modelling of shipping business processes based on MCDM methods","volume":"36","author":"Celik","year":"2009","journal-title":"Marit. Policy Manag."},{"key":"ref_127","doi-asserted-by":"crossref","first-page":"4398","DOI":"10.1016\/j.matdes.2010.04.044","article-title":"An integrated decision aid extension to material selection problem","volume":"31","author":"Cicek","year":"2010","journal-title":"Mater. Des."},{"key":"ref_128","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1016\/j.rser.2019.06.018","article-title":"Testing the reliability of deterministic multi-criteria decision-making methods using building performance simulation","volume":"112","author":"Kokaraki","year":"2019","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_129","doi-asserted-by":"crossref","unstructured":"Kolios, A., Mytilinou, V., Lozano-Minguez, E., and Salonitis, K. (2016). A comparative study of multiple-criteria decision-making methods under stochastic inputs. Energies, 9.","DOI":"10.3390\/en9070566"},{"key":"ref_130","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/s13748-016-0093-1","article-title":"A comparative analysis of multi-criteria decision-making methods","volume":"5","author":"Ceballos","year":"2016","journal-title":"Prog. Artif. Intell."},{"key":"ref_131","doi-asserted-by":"crossref","first-page":"113399","DOI":"10.1016\/j.eswa.2020.113399","article-title":"Integration and comparison of multi-criteria decision making methods in safe route planner","volume":"154","author":"Sarraf","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_132","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1111\/risa.13391","article-title":"Evaluation of Multicriteria Decision Analysis Algorithms in Food Safety: A Case Study on Emerging Zoonoses Prioritization","volume":"40","author":"Garre","year":"2020","journal-title":"Risk Anal."},{"key":"ref_133","doi-asserted-by":"crossref","unstructured":"Chakraborty, S., and Yeh, C.H. (2009, January 6\u20139). A simulation comparison of normalization procedures for TOPSIS. Proceedings of the 2009 International Conference on Computers & Industrial Engineering, Troyes, France.","DOI":"10.1109\/ICCIE.2009.5223811"},{"key":"ref_134","doi-asserted-by":"crossref","unstructured":"Papathanasiou, J., and Ploskas, N. (2018). Multiple Criteria Decision Aid, Springer.","DOI":"10.1007\/978-3-319-91648-4"},{"key":"ref_135","first-page":"5","article-title":"Multicriteria optimization of civil engineering systems","volume":"2","author":"Opricovic","year":"1998","journal-title":"Fac. Civ. Eng. Belgrade"},{"key":"ref_136","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1016\/S0377-2217(03)00020-1","article-title":"Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS","volume":"156","author":"Opricovic","year":"2004","journal-title":"Eur. J. Oper. Res."},{"key":"ref_137","first-page":"94","article-title":"Determination of an efficient contractor by using the new method of multicriteria assessment","volume":"Volume 2","author":"Zavadskas","year":"1996","journal-title":"Proceedings of the International Symposium for \u201cThe Organization and Management of Construction\u201d. Shaping Theory and Practice"},{"key":"ref_138","first-page":"131","article-title":"The new method of multicriteria complex proportional assessment of projects","volume":"1","author":"Zavadskas","year":"1994","journal-title":"Technol. Econ. Dev. Econ."},{"key":"ref_139","first-page":"171","article-title":"Normalized weighted geometric bonferroni mean operator of interval rough numbers\u2014Application in interval rough dematel-copras model","volume":"16","author":"Lukovac","year":"2018","journal-title":"Facta Univ. Ser. Mech. Eng."},{"key":"ref_140","first-page":"199","article-title":"Selection of drill for drilling with high pressure coolant using entropy and copras MCDM method","volume":"79","year":"2017","journal-title":"UPB Sci. Bull. Ser. D Mech. Eng."},{"key":"ref_141","doi-asserted-by":"crossref","unstructured":"Brans, J.P., and De Smet, Y. (2016). PROMETHEE methods. Multiple Criteria Decision Analysis, Springer.","DOI":"10.1007\/978-1-4939-3094-4_6"},{"key":"ref_142","doi-asserted-by":"crossref","unstructured":"Ranjan, R., Chatterjee, P., Panchal, D., and Pamucar, D. (2019). Performance Evaluation of Sustainable Smart Cities in India: An Adaptation of Cartography in PROMETHEE-GIS Approach. Advanced Multi-Criteria Decision Making for Addressing Complex Sustainability Issues, IGI Global.","DOI":"10.4018\/978-1-5225-8579-4.ch002"},{"key":"ref_143","first-page":"51","article-title":"Distortions introduced by normalisation of values of criteria in multiple criteria methods of evaluation","volume":"55","author":"Podviezko","year":"2014","journal-title":"LMD Darb"},{"key":"ref_144","doi-asserted-by":"crossref","unstructured":"Sa\u0142abun, W., and Urbaniak, K. (2020, January 3\u20135). A new coefficient of rankings similarity in decision-making problems. Proceedings of the International Conference on Computational Science, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-030-50417-5_47"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/9\/1549\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:11:43Z","timestamp":1760177503000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/9\/1549"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,20]]},"references-count":144,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["sym12091549"],"URL":"https:\/\/doi.org\/10.3390\/sym12091549","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,20]]}}}