{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T08:00:18Z","timestamp":1761897618036,"version":"3.41.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T00:00:00Z","timestamp":1634083200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["146371743-TRR 89"],"award-info":[{"award-number":["146371743-TRR 89"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Evol. Learn. Optim."],"published-print":{"date-parts":[[2021,12,31]]},"abstract":"<jats:p>Real-world problems typically require the simultaneous optimization of multiple, often conflicting objectives. Many of these<jats:italic>multi-objective optimization problems<\/jats:italic>are characterized by wide ranges of uncertainties in their decision variables or objective functions. To cope with such uncertainties,<jats:italic>stochastic<\/jats:italic>and<jats:italic>robust optimization<\/jats:italic>techniques are widely studied aiming to distinguish candidate solutions with uncertain objectives specified by confidence intervals, probability distributions, sampled data, or uncertainty sets. In this scope, this article first introduces a novel empirical approach for the comparison of candidate solutions with uncertain objectives that can follow arbitrary distributions. The comparison is performed through accurate and efficient calculations of the probability that one solution dominates the other in terms of each uncertain objective. Second, such an operator can be flexibly used and combined with many existing multi-objective optimization frameworks and techniques by just substituting their standard comparison operator, thus easily enabling the Pareto front optimization of problems with multiple uncertain objectives. Third, a new benchmark for evaluating uncertainty-aware optimization techniques is introduced by incorporating different types of uncertainties into a well-known benchmark for multi-objective optimization problems. Fourth, the new comparison operator and benchmark suite are integrated into an existing multi-objective optimization framework that features a selection of multi-objective optimization problems and algorithms. Fifth, the efficiency in terms of performance and execution time of the proposed comparison operator is evaluated on the introduced uncertainty benchmark. Finally, statistical tests are applied giving evidence of the superiority of the new comparison operator in terms of<jats:inline-formula content-type=\"math\/tex\"><jats:tex-math notation=\"TeX\" version=\"MathJax\">\\epsilon<\/jats:tex-math><\/jats:inline-formula>-dominance and attainment surfaces in comparison to previously proposed approaches.<\/jats:p>","DOI":"10.1145\/3469801","type":"journal-article","created":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T22:05:35Z","timestamp":1634162735000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Efficient Computation of Probabilistic Dominance in Multi-objective Optimization"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9032-4751","authenticated-orcid":false,"given":"Faramarz","family":"Khosravi","sequence":"first","affiliation":[{"name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU), Erlangen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1274-6398","authenticated-orcid":false,"given":"Alexander","family":"Rass","sequence":"additional","affiliation":[{"name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU), Erlangen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6285-5862","authenticated-orcid":false,"given":"J\u00fcrgen","family":"Teich","sequence":"additional","affiliation":[{"name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU), Erlangen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,13]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables","author":"Abramowitz Milton","year":"1964","unstructured":"Milton Abramowitz and Irene A. Stegun. 1964. Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables. Dover Publications."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2007.03.003"},{"key":"e_1_3_2_4_2","volume-title":"Introduction to Stochastic Programming","author":"Birge John R.","year":"1997","unstructured":"John R. Birge and Fran\u00e7ois Louveaux. 1997. Introduction to Stochastic Programming. Springer-Verlag, New York, NY."},{"key":"e_1_3_2_5_2","volume-title":"Retrieved from http:\/\/wrap.warwick.ac.uk\/55724","author":"Branke J\u00fcrgen","year":"2013","unstructured":"J\u00fcrgen Branke, Gideon Avigad, and Amiram Moshaiov. 2013. Multi-objective worst case optimization by means of evolutionary algorithms. Retrieved from http:\/\/wrap.warwick.ac.uk\/55724."},{"issue":"2","key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s40747-019-0113-4","article-title":"Evolutionary multiobjective optimization: open research areas and some challenges lying ahead","volume":"6","author":"Coello Carlos A. Coello","year":"2020","unstructured":"Carlos A. Coello Coello et\u00a0al. 2020. Evolutionary multiobjective optimization: open research areas and some challenges lying ahead. Complex Intell. Syst. 6, 2 (2020), 221\u2013236.","journal-title":"Complex Intell. Syst."},{"key":"e_1_3_2_7_2","first-page":"849","volume-title":"Proceedings of the Parallel Problem Solving from Nature","author":"Deb Kalyanmoy","year":"2000","unstructured":"Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and Tamt Meyarivan. 2000. A Fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II. In Proceedings of the Parallel Problem Solving from Nature (PPSN\u201900). 849\u2013858."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1162\/evco.2006.14.4.463"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2002.1007032"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/1869916"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177728174"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2014.03.013"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35813-5_9"},{"key":"e_1_3_2_14_2","first-page":"243","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation","volume":"1","author":"Fieldsend Jonathan E.","year":"2005","unstructured":"Jonathan E. Fieldsend and Richard M. Everson. 2005. Multi-objective optimisation in the presence of uncertainty. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC\u201905), Vol. 1. 243\u2013250."},{"issue":"1","key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/TEVC.2016.2634625","article-title":"A set-based genetic algorithm for interval many-objective optimization problems","volume":"22","author":"Gong Dunwei","year":"2016","unstructured":"Dunwei Gong, Jing Sun, and Zhuang Miao. 2016. A set-based genetic algorithm for interval many-objective optimization problems. IEEE Trans. Evolution. Comput. 22, 1 (2016), 47\u201360.","journal-title":"IEEE Trans. Evolution. Comput."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10479-013-1369-5"},{"issue":"2","key":"e_1_3_2_17_2","first-page":"316","article-title":"Robust multiobjective optimization via evolutionary algorithms","volume":"23","author":"He Zhenan","year":"2018","unstructured":"Zhenan He, Gary G. Yen, and Zhang Yi. 2018. Robust multiobjective optimization via evolutionary algorithms. IEEE Trans. Evolution. Comput. 23, 2 (2018), 316\u2013330.","journal-title":"IEEE Trans. Evolution. Comput."},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1007\/3-540-44719-9_23","volume-title":"Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization","author":"Hughes Evan J.","year":"2001","unstructured":"Evan J. Hughes. 2001. Evolutionary multi-objective ranking with uncertainty and noise. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization (EMO\u201901). 329\u2013343."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2005.846356"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2007.07.035"},{"key":"e_1_3_2_21_2","first-page":"1","volume-title":"Proceedings of the IEEE Congress on Evolutionary Computation","author":"Khosravi Faramarz","year":"2018","unstructured":"Faramarz Khosravi, Michael Borst, and J\u00fcrgen Teich. 2018. Probabilistic Dominance in Robust Multi-Objective Optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC\u201918). 1\u20136."},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"97","DOI":"10.7873\/DATE.2015.0319","volume-title":"Proceedings of the Design, Automation and Test in Europe","author":"Khosravi Faramarz","year":"2015","unstructured":"Faramarz Khosravi, Malte M\u00fcller, Michael Glass, and J\u00fcrgen Teich. 2015. Uncertainty-Aware Reliability Analysis and Optimization. In Proceedings of the Design, Automation and Test in Europe (DATE\u201915). 97\u2013102."},{"key":"e_1_3_2_23_2","volume-title":"TIK-Report No. 214, Computer Engineering and Networks Laboratory, ETH Zurich","author":"Knowles Joshua D.","year":"2006","unstructured":"Joshua D. Knowles, Lothar Thiele, and Eckart Zitzler. 2006. A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. TIK-Report No. 214, Computer Engineering and Networks Laboratory, ETH Zurich."},{"issue":"2","key":"e_1_3_2_24_2","first-page":"305","article-title":"On robust multiobjective optimization","volume":"40","author":"Kuroiwa Daishi","year":"2012","unstructured":"Daishi Kuroiwa and Gue Myung Lee. 2012. On robust multiobjective optimization. Vietnam J. Math. 40, 2&3 (2012), 305\u2013317.","journal-title":"Vietnam J. Math."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2014.2310651"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1007\/978-3-540-31880-4_29","volume-title":"Proceedings of the Evolutionary Multi-Criterion Optimization","author":"Limbourg Philipp","year":"2005","unstructured":"Philipp Limbourg. 2005. Multi-objective Optimization of Problems with Epistemic Uncertainty. In Proceedings of the Evolutionary Multi-Criterion Optimization (EMO\u201905). 413\u2013427."},{"key":"e_1_3_2_27_2","first-page":"1723","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Lukasiewycz Martin","year":"2011","unstructured":"Martin Lukasiewycz, Michael Gla\u00df, Felix Reimann, and J\u00fcrgen Teich. 2011. Opt4J - A modular framework for meta-heuristic optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO\u201911). 1723\u20131730."},{"issue":"1","key":"e_1_3_2_28_2","first-page":"77","article-title":"Portfolio selection","volume":"7","author":"Markowitz Harry","year":"1952","unstructured":"Harry Markowitz. 1952. Portfolio selection. J. Finance 7, 1 (1952), 77\u201391.","journal-title":"J. Finance"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.04.056"},{"key":"e_1_3_2_30_2","article-title":"Comparing solutions under uncertainty in multiobjective optimization","volume":"2014","author":"Mlakar Miha","year":"2014","unstructured":"Miha Mlakar, Tea Tu\u0161ar, and Bogdan Filipi\u010d. 2014. Comparing solutions under uncertainty in multiobjective optimization. Math. Problems Eng. 2014 (2014).","journal-title":"Math. Problems Eng."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2004.03.002"},{"issue":"3","key":"e_1_3_2_32_2","first-page":"287","article-title":"Multi-objective particle swarm optimizers: A survey of the state-of-the-art","volume":"2","author":"Reyes-Sierra Margarita","year":"2006","unstructured":"Margarita Reyes-Sierra and Carlos A. Coello. 2006. Multi-objective particle swarm optimizers: A survey of the state-of-the-art. Int. J. Comput. Intell. Res. 2, 3 (2006), 287\u2013308.","journal-title":"Int. J. Comput. Intell. Res."},{"key":"e_1_3_2_33_2","first-page":"317","volume-title":"Proceedings of the IEEE International Conference on Data Engineering","author":"Soliman Mohamed A.","year":"2009","unstructured":"Mohamed A. Soliman and Ihab F. Ilyas. 2009. Ranking with uncertain scores. In Proceedings of the IEEE International Conference on Data Engineering. 317\u2013328."},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1007\/3-540-44719-9_22","volume-title":"Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization","author":"Teich J\u00fcrgen","year":"2001","unstructured":"J\u00fcrgen Teich. 2001. Pareto-front exploration with uncertain objectives. In Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization (EMO\u201901). 314\u2013328."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2011.2161029"},{"issue":"9","key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"1997","DOI":"10.1109\/TCYB.2015.2459137","article-title":"Regularity model for noisy multiobjective optimization","volume":"46","author":"Wang Handing","year":"2015","unstructured":"Handing Wang, Qingfu Zhang, Licheng Jiao, and Xin Yao. 2015. Regularity model for noisy multiobjective optimization. IEEE Trans. Cybernet. 46, 9 (2015), 1997\u20132009.","journal-title":"IEEE Trans. Cybernet."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2007.892759"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1162\/106365600568202"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2003.810758"}],"container-title":["ACM Transactions on Evolutionary Learning and Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469801","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3469801","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:15Z","timestamp":1750188615000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469801"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,12,31]]}},"alternative-id":["10.1145\/3469801"],"URL":"https:\/\/doi.org\/10.1145\/3469801","relation":{},"ISSN":["2688-299X","2688-3007"],"issn-type":[{"type":"print","value":"2688-299X"},{"type":"electronic","value":"2688-3007"}],"subject":[],"published":{"date-parts":[[2021,10,13]]},"assertion":[{"value":"2021-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}