{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T19:54:33Z","timestamp":1781898873267,"version":"3.54.5"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T00:00:00Z","timestamp":1649289600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T00:00:00Z","timestamp":1649289600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ann Oper Res"],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Pairwise comparisons have been a long-standing technique for comparing alternatives\/criteria and their role has been pivotal in the development of modern decision-making methods. The evaluation is very often done linguistically. Several scales have been proposed to translate the linguistic evaluation into a quantitative evaluation. In this paper, we perform an experiment to investigate, under our methodological choices, which type of scale provides the best matching of the decision-maker\u2019s verbal representation. The experiment aims to evaluate the suitability of eight evaluation scales for problems of different sizes. We find that the inverse linear scale provides the best matching verbal representation whenever the objective data are measured by means of pairwise comparisons matrices and a suitable distance between matrices is applied for computing the matching error.<\/jats:p>","DOI":"10.1007\/s10479-022-04682-8","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T10:21:36Z","timestamp":1649326896000},"page":"951-965","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Evaluating scales for pairwise comparisons"],"prefix":"10.1007","volume":"325","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5102-3232","authenticated-orcid":false,"given":"Bice","family":"Cavallo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alessio","family":"Ishizaka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,4,7]]},"reference":[{"issue":"3","key":"4682_CR1","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1002\/(SICI)1099-1360(199805)7:3<123::AID-MCDA181>3.0.CO;2-8","volume":"7","author":"J Barzilai","year":"1998","unstructured":"Barzilai, J. (1998). Consistency measures for pairwise comparison matrices. Journal of Multi-Criteria Decision Analysis, 7(3), 123\u2013132.","journal-title":"Journal of Multi-Criteria Decision Analysis"},{"issue":"1","key":"4682_CR2","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1007\/s10479-013-1328-1","volume":"211","author":"S Boz\u00f3ki","year":"2013","unstructured":"Boz\u00f3ki, S., Dezs\u00f6, L., Poesz, A., et al. (2013). Analysis of pairwise comparison matrices: An empirical research. Annals of Operations Research, 211(1), 511\u2013528.","journal-title":"Annals of Operations Research"},{"key":"4682_CR3","doi-asserted-by":"crossref","first-page":"104808","DOI":"10.1016\/j.knosys.2019.06.016","volume":"187","author":"M Brunelli","year":"2020","unstructured":"Brunelli, M., & Cavallo, B. (2020). Distance-based measures of incoherence for pairwise comparisons. Knowledge-Based Systems, 187, 104808.","journal-title":"Knowledge-Based Systems"},{"issue":"2","key":"4682_CR4","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1007\/s10203-020-00291-x","volume":"43","author":"M Brunelli","year":"2020","unstructured":"Brunelli, M., & Cavallo, B. (2020). Incoherence measures and relations between coherence conditions for pairwise comparisons. Decisions in Economics and Finance, 43(2), 613\u2013635.","journal-title":"Decisions in Economics and Finance"},{"issue":"3","key":"4682_CR5","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1016\/0749-5978(85)90007-X","volume":"36","author":"DV Budescu","year":"1985","unstructured":"Budescu, D. V., & Wallsten, T. S. (1985). Consistency in interpretation of probabilistic phrases. Organizational Behavior and Human Decision Processes, 36(3), 391\u2013405.","journal-title":"Organizational Behavior and Human Decision Processes"},{"issue":"1","key":"4682_CR6","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/s10700-018-9286-3","volume":"18","author":"B Cavallo","year":"2019","unstructured":"Cavallo, B. (2019). $${\\cal{G}}$$-distance and $${\\cal{G}}$$-decomposition for improving $${\\cal{G}}$$-consistency of a pairwise comparison matrix. Fuzzy Optimization and Decision Making, 18(1), 57\u201383.","journal-title":"Fuzzy Optimization and Decision Making"},{"issue":"4","key":"4682_CR7","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1002\/int.20329","volume":"24","author":"B Cavallo","year":"2009","unstructured":"Cavallo, B., & D\u2019Apuzzo, L. (2009). A general unified framework for pairwise comparison matrices in multicriterial methods. International Journal of Intelligent Systems, 24(4), 377\u2013398.","journal-title":"International Journal of Intelligent Systems"},{"issue":"2","key":"4682_CR8","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1007\/s10203-020-00289-5","volume":"43","author":"B Cavallo","year":"2020","unstructured":"Cavallo, B., & D\u2019Apuzzo, L. (2020). Relations between coherence conditions and row orders in pairwise comparison matrices. Decisions in Economics and Finance, 43(2), 637\u2013656.","journal-title":"Decisions in Economics and Finance"},{"issue":"5","key":"4682_CR9","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1080\/01605682.2018.1464427","volume":"70","author":"B Cavallo","year":"2019","unstructured":"Cavallo, B., Ishizaka, A., Olivieri, M. G., et al. (2019). Comparing inconsistency of pairwise comparison matrices depending on entries. Journal of the Operational Research Society, 70(5), 842\u2013850.","journal-title":"Journal of the Operational Research Society"},{"issue":"1\u20132","key":"4682_CR10","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/s10479-017-2440-4","volume":"254","author":"L Csat\u00f3","year":"2017","unstructured":"Csat\u00f3, L. (2017). On the ranking of a Swiss system chess team tournament. Annals of Operations Research, 254(1\u20132), 17\u201336.","journal-title":"Annals of Operations Research"},{"issue":"1\u20132","key":"4682_CR11","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/s10479-017-2627-8","volume":"261","author":"L Csat\u00f3","year":"2018","unstructured":"Csat\u00f3, L. (2018). Characterization of an inconsistency ranking for pairwise comparison matrices. Annals of Operations Research, 261(1\u20132), 155\u2013165.","journal-title":"Annals of Operations Research"},{"issue":"1\u20132","key":"4682_CR12","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s10479-019-03312-0","volume":"280","author":"L Csat\u00f3","year":"2019","unstructured":"Csat\u00f3, L. (2019). Axiomatizations of inconsistency indices for triads. Annals of Operations Research, 280(1\u20132), 99\u2013110.","journal-title":"Annals of Operations Research"},{"issue":"3","key":"4682_CR13","first-page":"295","volume":"41","author":"HA Donegan","year":"1992","unstructured":"Donegan, H. A., Dodd, F. J., & McMaster, T. B. M. (1992). A new approach to AHP decision-making. Journal of the Royal Statistical Society: Series D (The Statistician), 41(3), 295\u2013302.","journal-title":"Journal of the Royal Statistical Society: Series D (The Statistician)"},{"issue":"3","key":"4682_CR14","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1016\/j.ejor.2013.03.019","volume":"229","author":"Y Dong","year":"2013","unstructured":"Dong, Y., Hong, W. C., Xu, Y., et al. (2013). Numerical scales generated individually for analytic hierarchy process. European Journal of Operational Research, 229(3), 654\u2013662.","journal-title":"European Journal of Operational Research"},{"key":"4682_CR15","doi-asserted-by":"crossref","DOI":"10.1002\/9781119138341","volume-title":"The handbook of behavioral operations","author":"K Donohue","year":"2018","unstructured":"Donohue, K., Katok, E., & Leider, S. (2018). The handbook of behavioral operations. Hoboken: Wiley."},{"issue":"7","key":"4682_CR16","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1016\/j.ress.2010.02.013","volume":"95","author":"MA Elliott","year":"2010","unstructured":"Elliott, M. A. (2010). Selecting numerical scales for pairwise comparisons. Reliability Engineering & System Safety, 95(7), 750\u2013763.","journal-title":"Reliability Engineering & System Safety"},{"issue":"11","key":"4682_CR17","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1287\/mnsc.33.11.1383","volume":"33","author":"PT Harker","year":"1987","unstructured":"Harker, P. T., & Vargas, L. G. (1987). The theory of ratio scale estimation: Saaty\u2019s analytic hierarchy process. Management Science, 33(11), 1383\u20131403.","journal-title":"Management Science"},{"issue":"3","key":"4682_CR18","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1006\/obhd.1997.2708","volume":"70","author":"EK Huizingh","year":"1997","unstructured":"Huizingh, E. K., & Vrolijk, H. C. (1997). A comparison of verbal and numerical judgments in the Analytic Hierarchy Process. Organizational Behavior and Human Decision Processes, 70(3), 237\u2013247.","journal-title":"Organizational Behavior and Human Decision Processes"},{"issue":"4","key":"4682_CR19","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1057\/ori.2009.10","volume":"22","author":"A Ishizaka","year":"2009","unstructured":"Ishizaka, A., & Labib, A. (2009). Analytic hierarchy process and expert choice: Benefits and limitations. OR Insight, 22(4), 201\u2013220.","journal-title":"OR Insight"},{"issue":"9","key":"4682_CR20","doi-asserted-by":"crossref","first-page":"3775","DOI":"10.1016\/j.eswa.2012.12.089","volume":"40","author":"A Ishizaka","year":"2013","unstructured":"Ishizaka, A., & Nguyen, N. H. (2013). Calibrated fuzzy AHP for current bank account selection. Expert Systems with Applications, 40(9), 3775\u20133783.","journal-title":"Expert Systems with Applications"},{"issue":"2","key":"4682_CR21","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1016\/j.ejor.2017.05.041","volume":"264","author":"A Ishizaka","year":"2018","unstructured":"Ishizaka, A., & Siraj, S. (2018). Are multi-criteria decision-making tools useful? An experimental comparative study of three methods. European Journal of Operational Research, 264(2), 462\u2013471.","journal-title":"European Journal of Operational Research"},{"issue":"2","key":"4682_CR22","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1007\/s10203-020-00309-4","volume":"43","author":"A Ishizaka","year":"2020","unstructured":"Ishizaka, A., & Siraj, S. (2020). Interactive consistency correction in the analytic hierarchy process to preserve ranks. Decisions in Economics and Finance, 43(2), 443\u2013464.","journal-title":"Decisions in Economics and Finance"},{"issue":"4","key":"4682_CR23","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1057\/jors.2010.23","volume":"62","author":"A Ishizaka","year":"2011","unstructured":"Ishizaka, A., Balkenborg, D., & Kaplan, T. (2011). Influence of aggregation and measurement scale on ranking a compromise alternative in AHP. Journal of the Operational Research Society, 62(4), 700\u2013710.","journal-title":"Journal of the Operational Research Society"},{"key":"4682_CR24","volume-title":"The handbook of experimental economics","author":"J Kagel","year":"2017","unstructured":"Kagel, J., & Roth, A. (2017). The handbook of experimental economics (Vol. 2). Princeton: Princeton University Press."},{"issue":"9","key":"4682_CR25","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1287\/mnsc.36.9.1011","volume":"36","author":"RL Keeney","year":"1990","unstructured":"Keeney, R. L., von Winterfeldt, D., & Eppel, T. (1990). Eliciting public values for complex policy decisions. Management Science, 36(9), 1011\u20131030.","journal-title":"Management Science"},{"issue":"1","key":"4682_CR26","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.seps.2005.10.006","volume":"42","author":"L Liang","year":"2008","unstructured":"Liang, L., Wang, G., Hua, Z., et al. (2008). Mapping verbal responses to numerical scales in the analytic hierarchy process. Socio-Economic Planning Sciences, 42(1), 46\u201355.","journal-title":"Socio-Economic Planning Sciences"},{"issue":"2","key":"4682_CR27","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1016\/j.ejor.2007.11.030","volume":"193","author":"P Linares","year":"2009","unstructured":"Linares, P. (2009). Are inconsistent decisions better? An experiment with pairwise comparisons. European Journal of Operational Research, 193(2), 492\u2013498.","journal-title":"European Journal of Operational Research"},{"issue":"1","key":"4682_CR28","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/0377-2217(89)90278-6","volume":"40","author":"F Lootsma","year":"1989","unstructured":"Lootsma, F. (1989). Conflict resolution via pairwise comparison of concessions. European Journal of Operational Research, 40(1), 109\u2013116.","journal-title":"European Journal of Operational Research"},{"key":"4682_CR29","doi-asserted-by":"crossref","unstructured":"Ma D, Zheng X (1991) 9\/9-9\/1 scale method of AHP. In: 2nd Int. Symposium on AHP, pp 197\u2013202.","DOI":"10.13033\/isahp.y1991.001"},{"key":"4682_CR30","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.asoc.2016.12.040","volume":"53","author":"BR Meesariganda","year":"2017","unstructured":"Meesariganda, B. R., & Ishizaka, A. (2017). Mapping verbal AHP scale to numerical scale for cloud computing strategy selection. Applied Soft Computing, 53, 111\u2013118.","journal-title":"Applied Soft Computing"},{"issue":"1","key":"4682_CR31","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1002\/(SICI)1099-1360(199701)6:1<41::AID-MCDA122>3.0.CO;2-D","volume":"6","author":"I Millet","year":"1997","unstructured":"Millet, I. (1997). The effectiveness of alternative preference elicitation methods in the analytic hierarchy process. Journal of Multi-Criteria Decision Analysis, 6(1), 41\u201351.","journal-title":"Journal of Multi-Criteria Decision Analysis"},{"issue":"2","key":"4682_CR32","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1002\/bdm.1929","volume":"30","author":"HH Por","year":"2017","unstructured":"Por, H. H., & Budescu, D. V. (2017). Eliciting subjective probabilities through pair-wise comparisons. Journal of Behavioral Decision Making, 30(2), 181\u2013196.","journal-title":"Journal of Behavioral Decision Making"},{"issue":"2","key":"4682_CR33","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1007\/s10700-014-9199-8","volume":"14","author":"J Ram\u00edk","year":"2015","unstructured":"Ram\u00edk, J. (2015). Isomorphisms between fuzzy pairwise comparison matrices. Fuzzy Optimization and Decision Making, 14(2), 199\u2013209.","journal-title":"Fuzzy Optimization and Decision Making"},{"key":"4682_CR34","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.omega.2014.11.009","volume":"53","author":"J Rezaei","year":"2015","unstructured":"Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49\u201357.","journal-title":"Omega"},{"key":"4682_CR35","doi-asserted-by":"crossref","first-page":"1369","DOI":"10.1007\/s10726-013-9374-2","volume":"23","author":"E Rokou","year":"2014","unstructured":"Rokou, E., & Kirytopoulos, K. (2014). Calibrated group decision process. Group Decision and Negotiation, 23, 1369\u20131384.","journal-title":"Group Decision and Negotiation"},{"issue":"3","key":"4682_CR36","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1016\/0022-2496(77)90033-5","volume":"15","author":"TL Saaty","year":"1977","unstructured":"Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234\u2013281.","journal-title":"Journal of Mathematical Psychology"},{"issue":"6","key":"4682_CR37","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1002\/(SICI)1099-1360(199711)6:6<309::AID-MCDA163>3.0.CO;2-2","volume":"6","author":"AA Salo","year":"1997","unstructured":"Salo, A. A., & H\u00e4m\u00e4l\u00e4inen, R. P. (1997). On the measurement of preferences in the analytic hierarchy process. Journal of Multi-Criteria Decision Analysis, 6(6), 309\u2013319.","journal-title":"Journal of Multi-Criteria Decision Analysis"},{"issue":"2","key":"4682_CR38","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/0165-0114(84)90032-0","volume":"12","author":"T Tanino","year":"1984","unstructured":"Tanino, T. (1984). Fuzzy preference orderings in group decision making. Fuzzy Sets and Systems, 12(2), 117\u2013131.","journal-title":"Fuzzy Sets and Systems"},{"issue":"4","key":"4682_CR39","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1037\/h0070288","volume":"34","author":"LL Thurstone","year":"1927","unstructured":"Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273\u2013286.","journal-title":"Psychological Review"},{"issue":"7","key":"4682_CR40","doi-asserted-by":"crossref","first-page":"840","DOI":"10.1016\/j.mcm.2007.03.018","volume":"46","author":"R Whitaker","year":"2007","unstructured":"Whitaker, R. (2007). Validation examples of the analytic hierarchy process and analytic network process. Mathematical and Computer Modelling, 46(7), 840\u2013859.","journal-title":"Mathematical and Computer Modelling"},{"issue":"4","key":"4682_CR41","first-page":"343","volume":"2","author":"PD Windschitl","year":"1996","unstructured":"Windschitl, P. D., & Wells, G. L. (1996). Measuring psychological uncertainty: Verbal versus numeric methods. Journal of Experimental Psychology: Applied, 2(4), 343\u2013364.","journal-title":"Journal of Experimental Psychology: Applied"},{"key":"4682_CR42","doi-asserted-by":"crossref","DOI":"10.1002\/9781119170174","volume-title":"Stevens\u2019 handbook of experimental psychology and cognitive neuroscience","author":"J Wixted","year":"2018","unstructured":"Wixted, J. (2018). Stevens\u2019 handbook of experimental psychology and cognitive neuroscience. Hoboken: Wiley."},{"issue":"3","key":"4682_CR43","doi-asserted-by":"crossref","first-page":"357","DOI":"10.2307\/1414000","volume":"32","author":"M Yokoyama","year":"1921","unstructured":"Yokoyama, M. (1921). The nature of the affective judgment in the method of paired comparisons. The American Journal of Psychology, 32(3), 357\u2013369.","journal-title":"The American Journal of Psychology"}],"container-title":["Annals of Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10479-022-04682-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10479-022-04682-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10479-022-04682-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,1]],"date-time":"2023-06-01T17:25:27Z","timestamp":1685640327000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10479-022-04682-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,7]]},"references-count":43,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["4682"],"URL":"https:\/\/doi.org\/10.1007\/s10479-022-04682-8","relation":{},"ISSN":["0254-5330","1572-9338"],"issn-type":[{"value":"0254-5330","type":"print"},{"value":"1572-9338","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,7]]},"assertion":[{"value":"18 March 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 April 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 July 2022","order":3,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":4,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Missing Open Access funding information has been added in the Funding Note.","order":5,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}}]}}