{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T06:20:08Z","timestamp":1778566808156,"version":"3.51.4"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T00:00:00Z","timestamp":1670371200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T00:00:00Z","timestamp":1670371200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003051","name":"New Energy and Industrial Technology Development Organization","doi-asserted-by":"publisher","award":["JPNP18002"],"award-info":[{"award-number":["JPNP18002"]}],"id":[{"id":"10.13039\/501100003051","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Optim Lett"],"published-print":{"date-parts":[[2023,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Black-box optimization (BBO) algorithms are widely employed by practitioners to address computationally expensive real-world problems such as automatic tuning of machine learning models and evacuation route planning. The Nelder\u2013Mead (NM) method is a well-known local search heuristic for BBO that has been applied to solve many real-world problems from way back because of its promising performance. However, this method has a strong dependence on initialization due to its local search tendency. Nevertheless, a discussion on the proper initialization of the NM method is limited to the recent study by Wessing (Optim Lett 13(4):847\u2013856, 2019), which is solely based on an analysis using the simple sphere function. In this study, we take a further step to improve Wessing\u2019s result by massively investigating how the initialization affects the search performance in views of the initial simplex size and shape and a constraint handling method that is employed on 24 BBO benchmarking problems. Based on the numerical results, we present the empirical best practice for the initialization of the NM method for cases involving a limited evaluation budget.<\/jats:p>","DOI":"10.1007\/s11590-022-01953-y","type":"journal-article","created":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T08:02:45Z","timestamp":1670400165000},"page":"283-297","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Practical initialization of the Nelder\u2013Mead method for computationally expensive optimization problems"],"prefix":"10.1007","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6824-7603","authenticated-orcid":false,"given":"Shintaro","family":"Takenaga","sequence":"first","affiliation":[]},{"given":"Yoshihiko","family":"Ozaki","sequence":"additional","affiliation":[]},{"given":"Masaki","family":"Onishi","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,12,7]]},"reference":[{"key":"1953_CR1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68913-5","volume-title":"Derivative-Free and Blackbox Optimization","author":"C Audet","year":"2017","unstructured":"Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer (2017)"},{"key":"1953_CR2","unstructured":"Cohen, G., Ruch, P., Hilario, M.: Model selection for support vector classifiers via direct simplex search. In: FLAIRS Conference, pp. 431\u2013435 (2005)"},{"key":"1953_CR3","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718768","volume-title":"Introduction to Derivative-Free Optimization","author":"AR Conn","year":"2009","unstructured":"Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. SIAM (2009)"},{"key":"1953_CR4","unstructured":"Digabel, S.L., Wild, S.M.: A taxonomy of constraints in simulation-based optimization (2015). arXiv preprint arXiv:1505.07881"},{"key":"1953_CR5","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611970319","volume-title":"The Jackknife, the Bootstrap and Other Resampling Plans","author":"B Efron","year":"1982","unstructured":"Efron, B.: The Jackknife, the Bootstrap and Other Resampling Plans. SIAM (1982)"},{"issue":"397","key":"1953_CR6","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1080\/01621459.1987.10478410","volume":"82","author":"B Efron","year":"1987","unstructured":"Efron, B.: Better bootstrap confidence intervals. J. Am. Stat. Assoc. 82(397), 171\u2013185 (1987)","journal-title":"J. Am. Stat. Assoc."},{"key":"1953_CR7","unstructured":"Fan, E.: Global optimization of the Lennard\u2013Jones atomic cluster. Master\u2019s thesis, McMaster University (2002)"},{"key":"1953_CR8","unstructured":"Finck, S., Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2009. Presentation of the noiseless functions. Technical report. Research Center PPE (2009)"},{"issue":"1","key":"1953_CR9","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1007\/s10589-010-9329-3","volume":"51","author":"F Gao","year":"2012","unstructured":"Gao, F., Han, L.: Implementing the Nelder\u2013Mead simplex algorithm with adaptive parameters. Comput. Optim. Appl. 51(1), 259\u2013277 (2012)","journal-title":"Comput. Optim. Appl."},{"issue":"7","key":"1953_CR10","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1177\/0021998307088592","volume":"42","author":"H Ghiasi","year":"2008","unstructured":"Ghiasi, H., Pasini, D., Lessard, L.: Constrained globalized Nelder\u2013Mead method for simultaneous structural and manufacturing optimization of a composite bracket. J. Compos. Mater. 42(7), 717\u2013736 (2008)","journal-title":"J. Compos. Mater."},{"key":"1953_CR11","doi-asserted-by":"crossref","unstructured":"Hansen, N.: Benchmarking the Nelder-Mead downhill simplex algorithm with many local restarts. In: Proceedings of 11th Annual Conference on Companion Genetic Evolution Computation Conference, Late Breaking Papers, pp. 2403\u20132408 (2009)","DOI":"10.1145\/1570256.1570335"},{"issue":"1","key":"1953_CR12","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1109\/TEVC.2008.924423","volume":"13","author":"N Hansen","year":"2008","unstructured":"Hansen, N., Niederberger, A.S., Guzzella, L., Koumoutsakos, P.: A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Trans. Evol. Comput. 13(1), 180\u2013197 (2008)","journal-title":"IEEE Trans. Evol. Comput."},{"issue":"5","key":"1953_CR13","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1109\/TEVC.2006.886448","volume":"11","author":"WB Langdon","year":"2007","unstructured":"Langdon, W.B., Poli, R.: Evolving problems to learn about particle swarm optimizers and other search algorithms. IEEE Trans. Evol. Comput. 11(5), 561\u2013578 (2007)","journal-title":"IEEE Trans. Evol. Comput."},{"issue":"4","key":"1953_CR14","doi-asserted-by":"publisher","first-page":"1082","DOI":"10.1137\/S1052623496300507","volume":"9","author":"RM Lewis","year":"1999","unstructured":"Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Optim. 9(4), 1082\u20131099 (1999)","journal-title":"SIAM J. Optim."},{"key":"1953_CR15","doi-asserted-by":"crossref","unstructured":"Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 50\u201360 (1947)","DOI":"10.1214\/aoms\/1177730491"},{"key":"1953_CR16","volume-title":"Compact Numerical Methods for Computers: Linear Algebra and Function Minimisation","author":"JC Nash","year":"1990","unstructured":"Nash, J.C.: Compact Numerical Methods for Computers: Linear Algebra and Function Minimisation. CRC Press (1990)"},{"issue":"4","key":"1953_CR17","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1093\/comjnl\/7.4.308","volume":"7","author":"JA Nelder","year":"1965","unstructured":"Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7(4), 308\u2013313 (1965)","journal-title":"Comput. J."},{"issue":"1","key":"1953_CR18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s41074-016-0012-1","volume":"9","author":"Y Ozaki","year":"2017","unstructured":"Ozaki, Y., Yano, M., Onishi, M.: Effective hyperparameter optimization using Nelder\u2013Mead method in deep learning. IPSJ Trans. Comput. Vis. Appl. 9(1), 1\u201312 (2017)","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"issue":"8","key":"1953_CR19","doi-asserted-by":"publisher","first-page":"756","DOI":"10.1080\/00029890.2002.11919910","volume":"109","author":"HR Parks","year":"2002","unstructured":"Parks, H.R., Wills, D.C.: An elementary calculation of the dihedral angle of the regular n-simplex. Am. Math. Mon. 109(8), 756\u2013758 (2002)","journal-title":"Am. Math. Mon."},{"key":"1953_CR20","unstructured":"Purchla, M., Malanowski, M., Terlecki, P., Arabas, J.: Experimental comparison of repair methods for box constraints. In: Proceedings of 7th National Conference on Evolution is a Global Optimization, pp. 135\u2013142. Warsaw Univ. of Technology Publishing House, Warsaw (2004)"},{"key":"1953_CR21","unstructured":"Shigenaka, S., Takami, S., Ozaki, Y., Onishi, M., Yamashita, T., Noda, I.: Evaluation of optimization for pedestrian route guidance in real-world crowded scene. In: Proceedings of 18th International Conference on Autonomous Agents and Multi-Agent Systems, pp. 2192\u20132194 (2019)"},{"issue":"4","key":"1953_CR22","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1080\/00401706.1962.10490033","volume":"4","author":"W Spendley","year":"1962","unstructured":"Spendley, W., Hext, G.R., Himsworth, F.R.: Sequential application of simplex designs in optimisation and evolutionary operation. Technometrics 4(4), 441\u2013461 (1962)","journal-title":"Technometrics"},{"key":"1953_CR23","unstructured":"Varadhan, R., Borchers, H.: Package dfoptim: derivative-free optimization (2016)"},{"key":"1953_CR24","unstructured":"Wessing, S.: dfoalgos:derivative-free optimization algorithms (2018). https:\/\/pypi.org\/project\/dfoalgos\/"},{"key":"1953_CR25","doi-asserted-by":"crossref","unstructured":"Wessing, S.: Repair methods for box constraints revisited. In: EvoApplications, pp. 469\u2013478. Springer, Berlin (2013)","DOI":"10.1007\/978-3-642-37192-9_47"},{"issue":"4","key":"1953_CR26","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1007\/s11590-018-1284-4","volume":"13","author":"S Wessing","year":"2019","unstructured":"Wessing, S.: Proper initialization is crucial for the Nelder\u2013Mead simplex search. Optim. Lett. 13(4), 847\u2013856 (2019)","journal-title":"Optim. Lett."}],"container-title":["Optimization Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11590-022-01953-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11590-022-01953-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11590-022-01953-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T23:07:02Z","timestamp":1675984022000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11590-022-01953-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,7]]},"references-count":26,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["1953"],"URL":"https:\/\/doi.org\/10.1007\/s11590-022-01953-y","relation":{},"ISSN":["1862-4472","1862-4480"],"issn-type":[{"value":"1862-4472","type":"print"},{"value":"1862-4480","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,7]]},"assertion":[{"value":"27 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 November 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 December 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}