{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T16:20:01Z","timestamp":1781108401865,"version":"3.54.1"},"reference-count":74,"publisher":"IGI Global Scientific Publishing","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,7,1]]},"abstract":"<p>This article describes how rank aggregation focuses on synthesizing a single ranked list based on rankings supplied by multiple judges. Such aggregations are widely applied in the areas of information retrieval, web search, and data mining. The problem of rank aggregation has been shown to be NP-hard and this article presents a heuristic approach to create an aggregated ranking score for all items on the lists. The proposed heuristic is scalable and performs. A computational study, as well as a real-life study involving the ranking of 147 engineering colleges in the US is presented to elucidate the performance. The authors' key finding is that the quality of the solution is sensitive to (a) the number of judges available to rank, (b) how the items are assigned to judges, and (c) how consistent\/inconsistent the judges are. All these factors are generally considered exogenous in most of the rank aggregation algorithms in extant literature.<\/p>","DOI":"10.4018\/ijissc.2018070101","type":"journal-article","created":{"date-parts":[[2018,5,3]],"date-time":"2018-05-03T12:11:05Z","timestamp":1525349465000},"page":"1-22","source":"Crossref","is-referenced-by-count":1,"title":["A Heuristic Approach for Ranking Items Based on Inputs from Multiple Experts"],"prefix":"10.4018","volume":"9","author":[{"given":"Dong","family":"Xu","sequence":"first","affiliation":[{"name":"Department of Industrial Engineering, University of Miami, Coral Gables, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nazrul I.","family":"Shaikh","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, University of Miami, Coral Gables, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2432","reference":[{"key":"IJISSC.2018070101-0","doi-asserted-by":"publisher","DOI":"10.1145\/1411509.1411513"},{"key":"IJISSC.2018070101-1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2010.09.001"},{"key":"IJISSC.2018070101-2","doi-asserted-by":"crossref","unstructured":"Aslam, J. A., & Montague, M. (2001). Models for metasearch. Paper presented at the24th annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans, LA.","DOI":"10.1145\/383952.384007"},{"key":"IJISSC.2018070101-3","first-page":"276","article-title":"Models for metasearch.","author":"J. A.Aslam","year":"2001","journal-title":"Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval"},{"key":"IJISSC.2018070101-4","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864733"},{"key":"IJISSC.2018070101-5","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376626"},{"key":"IJISSC.2018070101-6","unstructured":"Borda, J. D. (1784). M\u00e9moire sur les \u00e9lections au scrutin. Histoire de l'Academie Royale des Sciences pour."},{"key":"IJISSC.2018070101-7","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbp034"},{"key":"IJISSC.2018070101-8","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2010.09.003"},{"key":"IJISSC.2018070101-9","doi-asserted-by":"publisher","DOI":"10.1111\/j.1911-3838.2010.00011.x"},{"key":"IJISSC.2018070101-10","volume":"Vol. 12","author":"H. A.David","year":"1963","journal-title":"The method of paired comparisons"},{"key":"IJISSC.2018070101-11","article-title":"Combining Results of Microarray Experiments: A Rank Aggregation Approach.","volume":"5","author":"P.DeConde Robert","year":"2006","journal-title":"Statistical Applications in Genetics and Molecular Biology"},{"key":"IJISSC.2018070101-12","unstructured":"Ding, W., Ishwar, P., & Saligrama, V. (2014, October). A topic modeling approach to rank aggregation. In Advances in on Neural Information Processing Systems, workshop on Analysis of Rank data."},{"key":"IJISSC.2018070101-13","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1145\/371920.372165","article-title":"Rank aggregation methods for the web.","author":"C.Dwork","year":"2001","journal-title":"Proceedings of the 10th international conference on World Wide Web"},{"key":"IJISSC.2018070101-14","unstructured":"Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation revisited."},{"key":"IJISSC.2018070101-15","doi-asserted-by":"publisher","DOI":"10.1002\/mcda.313"},{"key":"IJISSC.2018070101-16","doi-asserted-by":"publisher","DOI":"10.1145\/1055558.1055568"},{"key":"IJISSC.2018070101-17","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1145\/775152.775204","article-title":"Searching the workplace web.","author":"R.Fagin","year":"2003","journal-title":"Proceedings of the 12th international conference on World Wide Web"},{"key":"IJISSC.2018070101-18","first-page":"247","article-title":"Incorporating user preferences in multimedia queries.","author":"R.Fagin","year":"1997","journal-title":"International Conference on Database Theory"},{"key":"IJISSC.2018070101-19","doi-asserted-by":"crossref","unstructured":"Fern\u00e1ndez, M., Vallet, D., & Castells, P. (2006). Probabilistic score normalization for rank aggregation. In Advances in Information Retrieval (pp. 553-556).","DOI":"10.1007\/11735106_63"},{"key":"IJISSC.2018070101-20","unstructured":"Fox, J. A., & Shaw, E. (1994). Combination of multiple sources: The trec-2 interactive track matrix experiment. In ACM SIGIR-94."},{"issue":"Nov","key":"IJISSC.2018070101-21","first-page":"933","article-title":"An efficient boosting algorithm for combining preferences.","volume":"4","author":"Y.Freund","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"IJISSC.2018070101-22","article-title":"Combination of molecular similarity measures using data fusion","author":"C. M.Ginn","year":"2000","journal-title":"Virtual Screening: An Alternative or Complement to High Throughput Screening?"},{"key":"IJISSC.2018070101-23","doi-asserted-by":"publisher","DOI":"10.2307\/2527957"},{"key":"IJISSC.2018070101-24","article-title":"Semi-supervised ensemble ranking.","author":"S. C.Hoi","year":"2008","journal-title":"Proceedings of the 23rd national conference on Artificial intelligence-Volume 2"},{"key":"IJISSC.2018070101-25","unstructured":"Iyer, R. D. (1999). An efficient boosting algorithm for combining preferences [Doctoral dissertation]. Massachusetts Institute of Technology."},{"key":"IJISSC.2018070101-26","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-012-0300-3"},{"key":"IJISSC.2018070101-27","doi-asserted-by":"publisher","DOI":"10.4018\/jissc.2012010104"},{"key":"IJISSC.2018070101-28","doi-asserted-by":"publisher","DOI":"10.4018\/IJISSC.2017040102"},{"key":"IJISSC.2018070101-29","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-74958-5_60"},{"key":"IJISSC.2018070101-30","first-page":"472","article-title":"Unsupervised rank aggregation with distance-based models.","author":"A.Klementiev","year":"2008","journal-title":"Proceedings of the 25th international conference on Machine learning"},{"key":"IJISSC.2018070101-31","doi-asserted-by":"crossref","unstructured":"Klementiev, A., Roth, D., Small, K., & Titov, I. (2009). Unsupervised rank aggregation with domain-specific expertise. Paper presented at the21st international jont conference on Artifical intelligence, Pasadena, CA.","DOI":"10.1145\/1390156.1390216"},{"key":"IJISSC.2018070101-32","article-title":"Unsupervised rank aggregation with domain-specific expertise.","volume":"51","author":"A.Klementiev","year":"2009","journal-title":"Urbana (Caracas, Venezuela)"},{"key":"IJISSC.2018070101-33","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btr709"},{"key":"IJISSC.2018070101-34","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871764"},{"key":"IJISSC.2018070101-35","unstructured":"Lebanon, G., & Lafferty, J. (2002, July). Cranking: Combining rankings using conditional probability models on permutations. In ICML (Vol. 2, pp. 363-370)."},{"key":"IJISSC.2018070101-36","doi-asserted-by":"publisher","DOI":"10.1145\/2806416.2806489"},{"key":"IJISSC.2018070101-37","doi-asserted-by":"publisher","DOI":"10.1145\/258525.258587"},{"key":"IJISSC.2018070101-38","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0096431"},{"key":"IJISSC.2018070101-39","doi-asserted-by":"publisher","DOI":"10.1002\/wics.111"},{"key":"IJISSC.2018070101-40","doi-asserted-by":"publisher","DOI":"10.1111\/j.1541-0420.2008.01044.x"},{"key":"IJISSC.2018070101-41","unstructured":"Lin, W. H., & Hauptmann, A. (2004, June). Merging rank lists from multiple sources in video classification. In Proceedings of the 2004 IEEE International Conference on Multimedia and Expo ICME \u201904 (Vol. 3, pp. 1535-1538). IEEE."},{"key":"IJISSC.2018070101-42","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242638"},{"key":"IJISSC.2018070101-43","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071"},{"key":"IJISSC.2018070101-44","doi-asserted-by":"publisher","DOI":"10.1007\/BF02294299"},{"key":"IJISSC.2018070101-45","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.10.3.285"},{"key":"IJISSC.2018070101-46","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1145\/502585.502657","article-title":"Relevance score normalization for metasearch.","author":"M.Montague","year":"2001","journal-title":"Proceedings of the tenth international conference on Information and knowledge management"},{"key":"IJISSC.2018070101-47","doi-asserted-by":"publisher","DOI":"10.1080\/0740817X.2015.1109737"},{"key":"IJISSC.2018070101-48","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-44956-2_8"},{"key":"IJISSC.2018070101-49","unstructured":"Niu, S., Lan, Y., Guo, J., & Cheng, X. (2013). Stochastic rank aggregation. arXiv:1309.6852"},{"key":"IJISSC.2018070101-50","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2005.03.023"},{"key":"IJISSC.2018070101-51","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btm158"},{"key":"IJISSC.2018070101-52","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2012.6252467"},{"key":"IJISSC.2018070101-53","unstructured":"Qin, T., Geng, X., & Liu, T. Y. (2010). A new probabilistic model for rank aggregation. In Advances in neural information processing systems (pp. 1948-1956)."},{"key":"IJISSC.2018070101-54","doi-asserted-by":"publisher","DOI":"10.1145\/952532.952698"},{"key":"IJISSC.2018070101-55","first-page":"83","article-title":"The mathematics of voting: Democratic symmetry.","author":"D. G.Saari","year":"2000","journal-title":"Economist"},{"key":"IJISSC.2018070101-56","author":"G.Salton","year":"1986","journal-title":"Introduction to Modern Information Retrieval"},{"key":"IJISSC.2018070101-57","unstructured":"Savoy, J., Le Calv\u00e9, A., & Vrajitoru, D. (1997). Report on the TREC-5 Experiment: Data Fusion and Collection Fusion. NIST."},{"key":"IJISSC.2018070101-58","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611972771.66"},{"key":"IJISSC.2018070101-59","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935930"},{"key":"IJISSC.2018070101-60","doi-asserted-by":"publisher","DOI":"10.4018\/ijissc.2015010102"},{"key":"IJISSC.2018070101-61","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1990.10476235"},{"key":"IJISSC.2018070101-62","doi-asserted-by":"publisher","DOI":"10.1109\/PASSAT\/SocialCom.2011.167"},{"key":"IJISSC.2018070101-63","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(90)90097-L"},{"key":"IJISSC.2018070101-64","doi-asserted-by":"publisher","DOI":"10.1037\/h0070288"},{"key":"IJISSC.2018070101-65","doi-asserted-by":"publisher","DOI":"10.1037\/h0070025"},{"key":"IJISSC.2018070101-66","author":"L. L.Thurstone","year":"1959","journal-title":"The measurement of values"},{"key":"IJISSC.2018070101-67","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1957.10501401"},{"issue":"3","key":"IJISSC.2018070101-68","first-page":"337","article-title":"Competence-conscious associative rank aggregation.","volume":"2","author":"A.Veloso","year":"2011","journal-title":"Journal of Information and Data Management"},{"key":"IJISSC.2018070101-69","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009980820262"},{"issue":"6","key":"IJISSC.2018070101-70","doi-asserted-by":"crossref","first-page":"1232","DOI":"10.1002\/asi.21296","article-title":"iRANK: A rank-learn-combine framework for unsupervised ensemble ranking.","volume":"61","author":"F.Wei","year":"2010","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"IJISSC.2018070101-71","doi-asserted-by":"publisher","DOI":"10.1021\/ci300547g"},{"issue":"13","key":"IJISSC.2018070101-72","first-page":"1177","article-title":"On the fusion of documents from multiple collection information retrieval systems.","volume":"49","author":"R. R.Yager","year":"1998","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"IJISSC.2018070101-73","doi-asserted-by":"publisher","DOI":"10.1137\/0135023"}],"container-title":["International Journal of Information Systems and Social Change"],"original-title":[],"language":"ng","link":[{"URL":"https:\/\/www.igi-global.com\/viewtitle.aspx?TitleId=205489","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,6]],"date-time":"2024-07-06T09:25:12Z","timestamp":1720257912000},"score":1,"resource":{"primary":{"URL":"https:\/\/services.igi-global.com\/resolvedoi\/resolve.aspx?doi=10.4018\/IJISSC.2018070101"}},"subtitle":[""],"short-title":[],"issued":{"date-parts":[[2018,7,1]]},"references-count":74,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2018,7]]}},"URL":"https:\/\/doi.org\/10.4018\/ijissc.2018070101","relation":{},"ISSN":["1941-868X","1941-8698"],"issn-type":[{"value":"1941-868X","type":"print"},{"value":"1941-8698","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,1]]}}}