{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T20:39:18Z","timestamp":1776371958337,"version":"3.51.2"},"reference-count":24,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2019,1,10]],"date-time":"2019-01-10T00:00:00Z","timestamp":1547078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005416","name":"Research Council of Norway","doi-asserted-by":"publisher","award":["POLARPROG"],"award-info":[{"award-number":["POLARPROG"]}],"id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This paper develops an automated algorithm to process input data for segmented string relative rankings (SSRRs). The purpose of the SSRR methodology is to create rankings of countries, companies, or any other units based on surveys of expert opinion. This is done without the use of grading systems, which can distort the results due to varying degrees of strictness among experts. However, the original SSRR approach relies on manual application, which is highly laborious and also carries a risk of human error. This paper seeks to solve this problem by further developing the SSRR approach by employing link analysis, which is based on network theory and is similar to the PageRank algorithm used by the Google search engine. The ranking data are treated as part of a linear, hierarchical network and each unit receives a score according to how many units are positioned below it in the network. This approach makes it possible to efficiently resolve contradictions among experts providing input for a ranking. A hypertext preprocessor (PHP) script for the algorithm is included in the article\u2019s appendix. The proposed methodology is suitable for use across a range of social science disciplines, especially economics, sociology, and political science.<\/jats:p>","DOI":"10.3390\/a12010019","type":"journal-article","created":{"date-parts":[[2019,1,11]],"date-time":"2019-01-11T04:10:16Z","timestamp":1547179816000},"page":"19","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Algorithm for Producing Rankings Based on Expert Surveys"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5955-4759","authenticated-orcid":false,"given":"Indra","family":"Overland","sequence":"first","affiliation":[{"name":"Norwegian Institute of International Affairs (NUPI), 0130 Oslo, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Javlon","family":"Juraev","sequence":"additional","affiliation":[{"name":"Norwegian Institute of International Affairs (NUPI), 0130 Oslo, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1038\/463294a","article-title":"A route to more tractable expert advice","volume":"463","author":"Aspinall","year":"2010","journal-title":"Nature"},{"key":"ref_2","unstructured":"Beddow, R. (2015). Fighting Corruption, Demanding Justice: Impact Report, Transparency International."},{"key":"ref_3","unstructured":"Lambsdorff, J.G. (2018, November 09). The methodology of the TI Corruption Perceptions Index. Transparency International and University of Passau. Available online: https:\/\/www.transparency.org\/files\/content\/tool\/2006_CPI_LongMethodology_EN.pdf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1146\/annurev.polisci.10.081205.095418","article-title":"What have we learned about the causes of corruption from ten years of cross-national empirical research?","volume":"10","author":"Treisman","year":"2007","journal-title":"Annu. Rev. Political Sci."},{"key":"ref_5","unstructured":"Langfeldt, L. (2002). Decision-Making in Expert Panels Evaluating Research: Constraints, Processes and Bias, NIFU."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1111\/j.1475-6765.2006.00694.x","article-title":"Evaluating expert judgments","volume":"46","author":"Steenbergen","year":"2007","journal-title":"Eur. J. Political Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1002\/asi.23985","article-title":"A critical evaluation of expert survey-based journal rankings: The role of personal research interests","volume":"69","author":"Serenko","year":"2018","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1023\/A:1013882225402","article-title":"International validation of the Corruption Perceptions Index: Implications for business ethics and entrepreneurship education","volume":"35","author":"Wilhelm","year":"2002","journal-title":"J. Bus. Ethics"},{"key":"ref_9","unstructured":"Atkenson, L.R., and Alvarez, R.M. (2016). Expert surveys as a measurement tool: Challenges and new frontiers. The Oxford Handbook of Polling and Survey Methods, Oxford University Press."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1017\/pan.2018.28","article-title":"IRT models for expert-coded panel data","volume":"26","author":"Marquardt","year":"2018","journal-title":"Political Anal."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1287\/ited.2013.0124","article-title":"Add or multiply? A tutorial on ranking and choosing with multiple criteria","volume":"14","author":"Tofallis","year":"2014","journal-title":"INFORMS Trans. Educ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"51","DOI":"10.31181\/dmame1802050m","article-title":"A sensitivity analysis in MCDM problems: A statistical approach","volume":"1","author":"Mukhametzyanov","year":"2018","journal-title":"Decis. Mak. Appl. Manag. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sremac, S., Stevi\u0107, \u017d., Pamu\u010dar, D., Arsi\u0107, M., and Mati\u0107, B. (2018). Evaluation of a Third-Party Logistics (3PL) Provider Using a Rough SWARA\u2013WASPAS Model Based on a New Rough Dombi Aggregator. Symmetry, 10.","DOI":"10.3390\/sym10080305"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.jclepro.2018.02.186","article-title":"Evaluating the performance of suppliers based on using the R\u2019AMATEL-MAIRCA method for green supply chain implementation in electronics industry","volume":"184","author":"Chatterjee","year":"2018","journal-title":"J. Clean. Prod."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"121","DOI":"10.31181\/dmame1802128l","article-title":"A multicriteria model for the selection of the transport service provider: A single valued neutrosophic DEMATEL multicriteria model","volume":"1","author":"Liu","year":"2018","journal-title":"Decis. Mak. Appl. Manag. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Overland, I. (2018). Lonely minds: Natural resource governance without input from society. Public Brainpower: Civil Society and Natural Resource Management, Palgrave. Available online: https:\/\/www.academia.edu\/35377095.","DOI":"10.1007\/978-3-319-60627-9_21"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Overland, I. (2018). Introduction: Civil society, public debate and natural resource management. Public Brainpower: Civil Society and Natural Resource Management, Palgrave. Available online: https:\/\/www.researchgate.net\/publication\/320656629.","DOI":"10.1007\/978-3-319-60627-9_1"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1145\/1052934.1052942","article-title":"Link analysis ranking: Algorithms, theory, and experiments","volume":"5","author":"Borodin","year":"2005","journal-title":"ACM Trans. Internet Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1002\/asmb.656","article-title":"Damping factor in Google page ranking","volume":"22","author":"Fu","year":"2006","journal-title":"Appl. Stoch. Models Bus. Ind."},{"key":"ref_20","first-page":"19","article-title":"Discussion on damping factor value in PageRank computation","volume":"9","author":"Srivastava","year":"2017","journal-title":"Int. J. Intell. Syst. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1137\/S003614450342480","article-title":"The structure and function of complex networks","volume":"45","author":"Newman","year":"2003","journal-title":"SIAM Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1504\/EJIM.2013.052090","article-title":"Challenges in international survey research: A review with illustrations and suggested solutions for best practice","volume":"7","author":"Harzing","year":"2013","journal-title":"Eur. J. Int. Manag."},{"key":"ref_23","first-page":"16","article-title":"Problems and prospects in survey research","volume":"93","author":"Moy","year":"2016","journal-title":"J. Mass Commun. Q."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/BF02558589","article-title":"Survey research: Some problems","volume":"4","author":"Thomas","year":"1974","journal-title":"Res. Sci. Educ."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/1\/19\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:24:53Z","timestamp":1760185493000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/1\/19"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,10]]},"references-count":24,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,1]]}},"alternative-id":["a12010019"],"URL":"https:\/\/doi.org\/10.3390\/a12010019","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,10]]}}}