{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:15:36Z","timestamp":1758845736110,"version":"3.44.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T00:00:00Z","timestamp":1738540800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T00:00:00Z","timestamp":1738540800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["IRTG-2379"],"award-info":[{"award-number":["IRTG-2379"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Data Sci Anal"],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>We consider the problem of ranking a set of objects based on their performance when the measurement of said performance is subject to noise. In this scenario, the performance is measured repeatedly, resulting in a range of measurements for each object. If the ranges of two objects do not overlap, then we consider one object as \u2018better\u2019 than the other, and we expect it to receive a higher rank; if, however, the ranges overlap, then the objects are <jats:italic>incomparable<\/jats:italic>, and we wish them to be assigned the same rank. Unfortunately, the incomparability relation of ranges is in general not transitive; as a consequence, in general the two requirements cannot be satisfied simultaneously, i.e., it is not possible to guarantee both distinct ranks for objects with separated ranges, and same rank for objects with overlapping ranges. This conflict leads to more than one reasonable way to rank a set of objects. Although the problem of ranking with ties has been widely studied, there remains a lack of clarity regarding what constitutes a set of reasonable rankings. In this paper, we explore the ambiguities that arise when ranking with ties, and define a set of reasonable rankings, which we call <jats:italic>partial rankings<\/jats:italic>. We develop and analyze three different methodologies to compute a partial ranking. Finally, we show how performance differences among objects can be investigated with the help of partial ranking.<\/jats:p>","DOI":"10.1007\/s41060-025-00722-1","type":"journal-article","created":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T00:36:45Z","timestamp":1738543005000},"page":"4363-4384","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Ranking with ties based on noisy performance data"],"prefix":"10.1007","volume":"20","author":[{"given":"Aravind","family":"Sankaran","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lars","family":"Karlsson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paolo","family":"Bientinesi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,2,3]]},"reference":[{"issue":"2","key":"722_CR1","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1007\/s00453-008-9211-1","volume":"57","author":"N Ailon","year":"2010","unstructured":"Ailon, N.: Aggregation of partial rankings, p-ratings and top-m lists. Algorithmica 57(2), 284\u2013300 (2010). https:\/\/doi.org\/10.1007\/s00453-008-9211-1","journal-title":"Algorithmica"},{"key":"722_CR2","doi-asserted-by":"publisher","first-page":"101064","DOI":"10.1016\/j.seps.2021.101064","volume":"82","author":"LS Alaimo","year":"2022","unstructured":"Alaimo, L.S., Fiore, M., Galati, A.: Measuring consumers\u2019 level of satisfaction for online food shopping during COVID-19 in Italy using POSETs. Socio-Econ. Plann. Sci. 82, 101064 (2022). https:\/\/doi.org\/10.1016\/j.seps.2021.101064","journal-title":"Socio-Econ. Plann. Sci."},{"issue":"3","key":"722_CR3","doi-asserted-by":"publisher","first-page":"22:1","DOI":"10.1145\/3446632","volume":"47","author":"H Barthels","year":"2021","unstructured":"Barthels, H., Psarras, C., Bientinesi, P.: Linnea: automatic generation of efficient linear algebra programs. ACM Trans. Math. Softw. 47(3), 22:1-22:26 (2021). https:\/\/doi.org\/10.1145\/3446632","journal-title":"ACM Trans. Math. Softw."},{"issue":"1","key":"722_CR4","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1137\/141000671","volume":"59","author":"J Bezanson","year":"2017","unstructured":"Bezanson, J., Edelman, A., Karpinski, S., et al.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65\u201398 (2017). https:\/\/doi.org\/10.1137\/141000671","journal-title":"SIAM Rev."},{"doi-asserted-by":"crossref","unstructured":"Br\u00fcggemann, R., Patil, GP.: Ranking and prioritization for multi-indicator systems: introduction to partial order applications. No. v. 5 in Environmental and ecological statistics, Springer, New York (2011a)","key":"722_CR5","DOI":"10.1007\/978-1-4419-8477-7"},{"doi-asserted-by":"publisher","unstructured":"Br\u00fcggemann, R., Patil, GP.: Structures of partial orders. In: Ranking and prioritization for multi-indicator systems. Springer, New York, pp. 57\u201374 (2011b). https:\/\/doi.org\/10.1007\/978-1-4419-8477-7","key":"722_CR6","DOI":"10.1007\/978-1-4419-8477-7"},{"doi-asserted-by":"publisher","unstructured":"Charles, J., Jassi, P., Ananth, NS.: et\u00a0al Evaluation of the Intel\u00ae Core\u2122 i7 Turbo Boost feature. In: 2009 IEEE International Symposium on Workload Characterization (IISWC), pp. 188\u2013197 (2009). https:\/\/doi.org\/10.1109\/IISWC.2009.5306782","key":"722_CR7","DOI":"10.1109\/IISWC.2009.5306782"},{"doi-asserted-by":"publisher","unstructured":"Chen, T., Guo, Q., Temam, O., et al.: Statistical performance comparisons of computers. IEEE Trans. Comput. 64(5), 1442\u20131455 (2015). https:\/\/doi.org\/10.1109\/TC.2014.2315614, conference Name: IEEE Transactions on Computers","key":"722_CR8","DOI":"10.1109\/TC.2014.2315614"},{"key":"722_CR9","doi-asserted-by":"publisher","first-page":"119209","DOI":"10.1016\/j.apenergy.2022.119209","volume":"318","author":"PB Dao","year":"2022","unstructured":"Dao, P.B.: On Wilcoxon rank sum test for condition monitoring and fault detection of wind turbines. Appl. Energy 318, 119209 (2022). https:\/\/doi.org\/10.1016\/j.apenergy.2022.119209","journal-title":"Appl. Energy"},{"doi-asserted-by":"publisher","unstructured":"Fagin, R., Kumar, R., Mahdian, et\u00a0al.: Comparing Partial Rankings. SIAM J. Discrete Math. 20(3), 628\u2013648 (2006). https:\/\/doi.org\/10.1137\/05063088X, publisher: Society for Industrial and Applied Mathematics","key":"722_CR10","DOI":"10.1137\/05063088X"},{"doi-asserted-by":"publisher","unstructured":"Hoefler, T., Belli, R.: Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Association for Computing Machinery, New York, NY, USA, SC \u201915, pp. 1\u201312 (2015). https:\/\/doi.org\/10.1145\/2807591.2807644,","key":"722_CR11","DOI":"10.1145\/2807591.2807644"},{"issue":"4","key":"722_CR12","doi-asserted-by":"publisher","first-page":"11:1","DOI":"10.1145\/1391729.1391730","volume":"40","author":"IF Ilyas","year":"2008","unstructured":"Ilyas, I.F., Beskales, G., Soliman, M.A.: A survey of top-k query processing techniques in relational database systems. ACM Comput. Surv. 40(4), 11:1-11:58 (2008). https:\/\/doi.org\/10.1145\/1391729.1391730","journal-title":"ACM Comput. Surv."},{"issue":"1\u20132","key":"722_CR13","doi-asserted-by":"publisher","first-page":"638","DOI":"10.14778\/1920841.1920923","volume":"3","author":"J Li","year":"2010","unstructured":"Li, J., Deshpande, A.: Ranking continuous probabilistic datasets. Proc. VLDB Endow. 3(1\u20132), 638\u2013649 (2010). https:\/\/doi.org\/10.14778\/1920841.1920923","journal-title":"Proc. VLDB Endow."},{"doi-asserted-by":"publisher","unstructured":"Liu, Y., Forrest, J., Xie, N.: Ranking grey numbers based on dominance grey degrees. Journal of Systems Engineering and Electronics 25(4), 618\u2013626 (2014). https:\/\/doi.org\/10.1109\/JSEE.2014.00072, conference Name: Journal of Systems Engineering and Electronics","key":"722_CR14","DOI":"10.1109\/JSEE.2014.00072"},{"key":"722_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/9780470479216.corpsy0524","volume-title":"The Corsini Encyclopedia of Psychology","author":"PE McKnight","year":"2010","unstructured":"McKnight, P.E., Najab, J.: Mann-Whitney test. In: Weiner, I.B., Craighead, W.E. (eds.) The Corsini Encyclopedia of Psychology, 1st edn., pp. 1\u20131. Wiley, New York (2010). https:\/\/doi.org\/10.1002\/9780470479216.corpsy0524","edition":"1"},{"doi-asserted-by":"publisher","unstructured":"Myers, W., Patil, G.: Chapter 7 semi-subordination sequences in multi-measure prioritization problems. In: Data Handling in Science and Technology, vol\u00a027. Elsevier, pp. 159\u2013168. https:\/\/doi.org\/10.1016\/S0922-3487(08)10007-7 (2008). https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0922348708100077","key":"722_CR16","DOI":"10.1016\/S0922-3487(08)10007-7"},{"issue":"7","key":"722_CR17","doi-asserted-by":"publisher","first-page":"1560","DOI":"10.1134\/S1995080221070192","volume":"42","author":"DA Nikitenko","year":"2021","unstructured":"Nikitenko, D.A., Wolf, F., Mohr, B., et al.: Influence of noisy environments on behavior of HPC applications. Lobachevskii J. Math. 42(7), 1560\u20131570 (2021). https:\/\/doi.org\/10.1134\/S1995080221070192","journal-title":"Lobachevskii J. Math."},{"issue":"2","key":"722_CR18","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1023\/B:EEST.0000027209.93218.d9","volume":"11","author":"G Patil","year":"2004","unstructured":"Patil, G., Taillie, C.: Multiple indicators, partially ordered sets, and linear extensions: multi-criterion ranking and prioritization. Environ. Ecol. Stat. 11(2), 199\u2013228 (2004). https:\/\/doi.org\/10.1023\/B:EEST.0000027209.93218.d9","journal-title":"Environ. Ecol. Stat."},{"doi-asserted-by":"publisher","unstructured":"Pavan, M., Todeschini, R.: New indices for analysing partial ranking diagrams. Anal. Chim. Acta 515(1), 167\u2013181 (2004). https:\/\/doi.org\/10.1016\/j.aca.2003.11.019. (https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0003267003014879)","key":"722_CR19","DOI":"10.1016\/j.aca.2003.11.019"},{"key":"722_CR20","volume-title":"Scientific Data Ranking Methods: Theory and Applications. Data Handling in Science and Technology","author":"M Pavan","year":"2008","unstructured":"Pavan, M., Todeschini, R.: Scientific Data Ranking Methods: Theory and Applications. Data Handling in Science and Technology, vol. 27, 1st edn. Elsevier, Oxford (2008)","edition":"1"},{"doi-asserted-by":"publisher","unstructured":"Sankaran, A, Bientinesi, P.: Performance comparison for scientific computations on the edge via relative performance. In: 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 887\u2013895 (2021). https:\/\/doi.org\/10.1109\/IPDPSW52791.2021.00132","key":"722_CR21","DOI":"10.1109\/IPDPSW52791.2021.00132"},{"doi-asserted-by":"publisher","unstructured":"Sankaran, A., Bientinesi, P.: A test for FLOPs as a discriminant for linear algebra algorithms. In: 2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), pp. 221\u2013230 (2022). https:\/\/doi.org\/10.1109\/SBAC-PAD55451.2022.00033, ISSN: 2643-3001","key":"722_CR22","DOI":"10.1109\/SBAC-PAD55451.2022.00033"},{"doi-asserted-by":"publisher","unstructured":"Sankaran, A., Zhukov, I., Frings, W., et\u00a0al.: Inspection of I\/O operations from system call traces using directly-follows-graph. In: SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, pp. 1562\u20131575 (2024). https:\/\/doi.org\/10.1109\/SCW63240.2024.00196","key":"722_CR23","DOI":"10.1109\/SCW63240.2024.00196"},{"issue":"6","key":"722_CR24","doi-asserted-by":"publisher","first-page":"1862","DOI":"10.1213\/ANE.0000000000005101","volume":"131","author":"P Schober","year":"2020","unstructured":"Schober, P., Vetter, T.R.: Nonparametric statistical methods in medical research. Anesth. Analg. 131(6), 1862\u20131863 (2020). https:\/\/doi.org\/10.1213\/ANE.0000000000005101","journal-title":"Anesth. Analg."},{"key":"722_CR25","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-2528-7","volume-title":"Stochastic Ordering and Dependence in Applied Probability, Lecture Notes in Statistics","author":"R Szekli","year":"1995","unstructured":"Szekli, R.: Stochastic Ordering and Dependence in Applied Probability, Lecture Notes in Statistics, vol. 97. Springer, New York (1995). https:\/\/doi.org\/10.1007\/978-1-4612-2528-7"},{"doi-asserted-by":"publisher","unstructured":"Van Der Aalst, W.M.P., Carmona, J. (eds.): Process Mining Handbook, Lecture Notes in Business Information Processing, vol 448. Springer, Cham (2022). https:\/\/doi.org\/10.1007\/978-3-031-08848-3","key":"722_CR26","DOI":"10.1007\/978-3-031-08848-3"},{"key":"722_CR27","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1007\/978-3-662-49851-4_11","volume-title":"Process Mining: Data Science in Action","author":"W van der Aalst","year":"2016","unstructured":"van der Aalst, W.: Process mining software. In: van der Aalst, W. (ed.) Process Mining: Data Science in Action, pp. 325\u2013352. Springer, Berlin (2016). https:\/\/doi.org\/10.1007\/978-3-662-49851-4_11"},{"key":"722_CR28","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.knosys.2015.07.025","volume":"91","author":"HY Zhang","year":"2016","unstructured":"Zhang, H.Y., Yang, S.Y., Ma, J.M.: Ranking interval sets based on inclusion measures and applications to three-way decisions. Knowl.-Based Syst. 91, 62\u201370 (2016). https:\/\/doi.org\/10.1016\/j.knosys.2015.07.025","journal-title":"Knowl.-Based Syst."},{"issue":"8","key":"722_CR29","doi-asserted-by":"publisher","first-page":"3675","DOI":"10.1109\/TNNLS.2021.3054063","volume":"33","author":"Q Zhang","year":"2022","unstructured":"Zhang, Q., Cheng, Y., Zhao, F., et al.: Optimal scale combination selection integrating three-way decision with Hasse diagram. IEEE Trans. Neural Netw. Learn. Syst. 33(8), 3675\u20133689 (2022). https:\/\/doi.org\/10.1109\/TNNLS.2021.3054063","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."}],"container-title":["International Journal of Data Science and Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-025-00722-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41060-025-00722-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-025-00722-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T10:47:59Z","timestamp":1758797279000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41060-025-00722-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,3]]},"references-count":29,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,10]]}},"alternative-id":["722"],"URL":"https:\/\/doi.org\/10.1007\/s41060-025-00722-1","relation":{},"ISSN":["2364-415X","2364-4168"],"issn-type":[{"type":"print","value":"2364-415X"},{"type":"electronic","value":"2364-4168"}],"subject":[],"published":{"date-parts":[[2025,2,3]]},"assertion":[{"value":"26 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 February 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}}]}}