{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:12:53Z","timestamp":1760148773134,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T00:00:00Z","timestamp":1685491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In spite of being introduced over twenty-five years ago, Fonseca and Fleming\u2019s attainment surfaces have not been widely used. This article investigates some of the shortcomings that may have led to the lack of adoption of this performance measure. The quantitative measure based on attainment surfaces, introduced by Knowles and Corne, is analyzed. The analysis shows that the results obtained by the Knowles and Corne approach are influenced (biased) by the shape of the attainment surface. Improvements to the Knowles and Corne approach for bi-objective Pareto-optimal front (POF) comparisons are proposed. Furthermore, assuming M objective functions, an M-dimensional attainment-surface-based quantitative measure, named the porcupine measure, is proposed for comparing the performance of multi-objective optimization algorithms. A computationally optimized version of the porcupine measure is presented and empirically analyzed.<\/jats:p>","DOI":"10.3390\/a16060283","type":"journal-article","created":{"date-parts":[[2023,6,1]],"date-time":"2023-06-01T02:12:45Z","timestamp":1685585565000},"page":"283","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["The Porcupine Measure for Comparing the Performance of Multi-Objective Optimization Algorithms"],"prefix":"10.3390","volume":"16","author":[{"given":"Christiaan","family":"Scheepers","sequence":"first","affiliation":[{"name":"Independent Researcher, Pretoria 0001, South Africa"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0242-3539","authenticated-orcid":false,"given":"Andries","family":"Engelbrecht","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, Computer Science Division, Stellenbosh University, Stellenbosch 7600, South Africa"},{"name":"Center for Applied Mathematics and Bioinformatics, Gulf Unversity of Science and Technology, Hawally 32093, Kuwait"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1007\/3-540-61723-X_1022","article-title":"On the Performance Assessment and Comparison of Stochastic Multiobjective Optimisers","volume":"Volume 1141","author":"Fonseca","year":"1995","journal-title":"Parallel Problem Solving from Nature\u2014PPSN IV"},{"key":"ref_2","unstructured":"Zitzler, E., and Thiele, L. (1998). Parallel Problem Solving from Nature\u2014PPSN V, Springer."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Van Veldhuizen, D.A. (1999). Multiobjective Evolutionary Algorithms: Classifications, Analyses and New Innovations. [Ph.D. Thesis, Air Force Institute of Technology].","DOI":"10.1145\/298151.298382"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Coello Coello, C.A., and Reyes-Sierra, M. (2004, January 26\u201330). A Study of the Parallelization of a Coevolutionary Multi-Objective Evolutionary Algorithm. Proceedings of the Third Mexican International Conference on Artificial Intelligence, Mexico City, Mexico.","DOI":"10.1007\/978-3-540-24694-7_71"},{"key":"ref_5","unstructured":"Reyes-Sierra, M., and Coello Coello, C.A. (2004). A New Multi-Objective Particle Swarm Optimizer with Improved Selection and Diversity Mechanisms, Evolutionary Computation Group at CINVESTAV-IPN. Technical report."},{"key":"ref_6","unstructured":"Ishibuchi, H., Masuda, H., Tanigaki, Y., and Nojima, Y. (April, January 29). An Analysis of Quality Indicators Using Approximated Optimal Distributions in a Three-dimensional Objective Space. Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization, Guimaraes, Portugal."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/4235.996017","article-title":"A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II","volume":"6","author":"Deb","year":"2002","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1162\/106365600568167","article-title":"Approximating the nondominated front using the Pareto Archived Evolution Strategy","volume":"8","author":"Knowles","year":"2000","journal-title":"Evol. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Gibbons, J.D., and Chakraborti, S. (2010). Nonparametric Statistical Inference, CRC Press. [5th ed.].","DOI":"10.1201\/9781439896129"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Knowles, J.D. (2005, January 8\u201310). A summary-attainment-surface plotting method for visualizing the performance of stochastic multiobjective optimizers. Proceedings of the 5th International Conference on Intelligent Systems Design and Applications 2005, ISDA \u201905, Warsaw, Poland.","DOI":"10.1109\/ISDA.2005.15"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Smith, K.I., Everson, R.M., and Fieldsend, J.E. (2004, January 19\u201323). Dominance measures for multi-objective simulated annealing. Proceedings of the IEEE Congress on Evolutionary Computation, Portland, OR, USA.","DOI":"10.1109\/CEC.2004.1330833"},{"key":"ref_12","unstructured":"Da Fonseca, V.G., Fonseca, C.M., and Hall, A.O. (2001, January 7\u20139). Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function. Proceedings of the Evolutionary Multi-Criterion Optimization, Zurich, Switzerland."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bartz-Beielstein, T., Chiarandini, M., Paquete, L., and Preuss, M. (2010). Experimental Methods for the Analysis of Optimization Algorithms, Springer.","DOI":"10.1007\/978-3-642-02538-9"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fonseca, C.M., Da Fonseca, V.G., and Paquete, L. (2005, January 9\u201311). Exploring the Performance of Stochastic Multiobjective Optimisers with the Second-Order Attainment Function. Proceedings of the Evolutionary Multi-Criterion Optimization, Guanajuato, Mexico.","DOI":"10.1007\/978-3-540-31880-4_18"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Fonseca, C.M., Guerreiro, A.P., L\u00f3pez-Ib\u00e1\u00f1ez, M., and Paquete, L. (2011, January 5\u20138). On the Computation of the Empirical Attainment Function. Proceedings of the Evolutionary Multi-Criterion Optimization: 6th International Conference, EMO 2011, Ouro Preto, Brazil.","DOI":"10.1007\/978-3-642-19893-9_8"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Tu\u0161ar, T., and Filipi\u010d, B. (2014). Visualizing Exact and Approximated 3D Empirical Attainment Functions. Math. Probl. Eng., 2014.","DOI":"10.1155\/2014\/569346"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Parsopoulos, K.E., and Vrahatis, M.N. (2002, January 10\u201314). Particle swarm optimization method in multiobjective problems. Proceedings of the ACM Symposium on Applied Computing, Madrid, Spain.","DOI":"10.1145\/508791.508907"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Reyes-Sierra, M., and Coello Coello, C.A. (2005, January 9\u201311). Improving PSO-Based Multi-objective Optimization Using Crowding, Mutation and \u03f5-Dominance. Proceedings of the Third International Conference on Evolutionary Multi-Criterion Optimization, Guanajuato, Mexico.","DOI":"10.1007\/978-3-540-31880-4_35"},{"key":"ref_19","unstructured":"Nebro, A.J., Durillo, J.J., Garc\u00eda-Nieto, J., Coello Coello, C.A., Luna, F., and Alba, E. (April, January 30). SMPSO: A New PSO-based Metaheuristic for Multi-objective Optimization. Proceedings of the IEEE Symposium on Multi-Criteria Decision-Making, Nashville, TN, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1162\/106365600568202","article-title":"Comparison of Multiobjective Evolutionary Algorithms: Empirical Results","volume":"8","author":"Zitzler","year":"2000","journal-title":"Evol. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Huband, S., Barone, L., While, L., and Hingston, P. (2005, January 9\u201311). A Scalable Multi-objective Test Problem Toolkit. Proceedings of the Third International Conference on Evolutionary Multi-Criterion Optimization, Guanajuato, Mexico.","DOI":"10.1007\/978-3-540-31880-4_20"},{"key":"ref_22","unstructured":"Scheepers, C. (2018). Multi-guided Particle Swarm Optimization: A Multi-Objective Particle Swarm Optimizer. [Ph.D. Thesis, University of Pretoria]."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/6\/283\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:46:40Z","timestamp":1760125600000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/6\/283"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,31]]},"references-count":22,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["a16060283"],"URL":"https:\/\/doi.org\/10.3390\/a16060283","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2023,5,31]]}}}