{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T15:12:33Z","timestamp":1793805153185,"version":"4.6.1"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2017,11,10]],"date-time":"2017-11-10T00:00:00Z","timestamp":1510272000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2017,11,10]],"date-time":"2017-11-10T00:00:00Z","timestamp":1510272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001863","name":"New Energy and Industrial Technology Development Organization","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001863","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["IPSJ T Comput Vis Appl"],"published-print":{"date-parts":[[2017,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In deep learning, deep neural network (DNN) hyperparameters can severely affect network performance. Currently, such hyperparameters are frequently optimized by several methods, such as Bayesian optimization and the covariance matrix adaptation evolution strategy. However, it is difficult for non-experts to employ these methods. In this paper, we adapted the simpler coordinate-search and Nelder-Mead methods to optimize hyperparameters. Several hyperparameter optimization methods were compared by configuring DNNs for character recognition and age\/gender classification. Numerical results demonstrated that the Nelder-Mead method outperforms the other methods and achieves state-of-the-art accuracy for age\/gender classification.<\/jats:p>","DOI":"10.1186\/s41074-017-0030-7","type":"journal-article","created":{"date-parts":[[2017,11,10]],"date-time":"2017-11-10T12:05:14Z","timestamp":1510315514000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":56,"title":["Effective hyperparameter optimization using Nelder-Mead method in deep learning"],"prefix":"10.1186","volume":"9","author":[{"given":"Yoshihiko","family":"Ozaki","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Masaki","family":"Yano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Masaki","family":"Onishi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2017,11,10]]},"reference":[{"issue":"11","key":"30_CR1","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun Y, Bottou L, Bengio Y, Patrick H (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278\u20132324.","journal-title":"Proc IEEE"},{"key":"30_CR2","first-page":"1097","volume":"25","author":"A Krizhevsky","year":"2012","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst (NIPS) 25:1097\u20131105.","journal-title":"Adv Neural Inf Process Syst (NIPS)"},{"key":"30_CR3","doi-asserted-by":"crossref","unstructured":"Christian S, Wei L, Yangqing J, Pierre S, Scott R, Dragomir A, Dumitru E, Vincent V, Andrew R (2015) Going deeper with convolutions. Comput Vis Pattern Recognit (CVPR):1\u20139. http:\/\/ieeexplore.ieee.org\/document\/7298594\/.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"30_CR4","first-page":"281","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281\u2013305.","journal-title":"J Mach Learn Res"},{"key":"30_CR5","doi-asserted-by":"crossref","unstructured":"Mockus J (1974) On Bayesian Methods for Seeking the Extremum In: Optimization Techniques IFIP Technical Conference, 400\u2013404.","DOI":"10.1007\/978-3-662-38527-2_55"},{"key":"30_CR6","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1162\/106365601750190398","volume":"9","author":"N Hansen","year":"2001","unstructured":"Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9:159\u2013195.","journal-title":"Evol Comput"},{"key":"30_CR7","first-page":"2546","volume":"24","author":"J Bergstra","year":"2011","unstructured":"Bergstra J, Bardenet R, Bengio Y, K\u00e9gl B (2011) Algorithms for hyper-parameter optimization. Adv Neural Inf Process Syst (NIPS) 24:2546\u20132554.","journal-title":"Adv Neural Inf Process Syst (NIPS)"},{"key":"30_CR8","first-page":"2951","volume":"25","author":"J Snoek","year":"2012","unstructured":"Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. Adv Neural Inf Process Syst (NIPS) 25:2951\u20132959.","journal-title":"Adv Neural Inf Process Syst (NIPS)"},{"key":"30_CR9","doi-asserted-by":"crossref","unstructured":"Watanabe S, Le Roux J (2014) A black box optimization for automatic speech recognition In: International Conference on Acoustics, Speech, and Signal Processing, 3256\u20133260.. (ICASSP).","DOI":"10.1109\/ICASSP.2014.6854202"},{"key":"30_CR10","unstructured":"Loshchilov I, Hutter F (2016) CMA-ES for hyperparameter optimization of deep neural networks. https:\/\/arxiv.org\/abs\/1604.07269. Accessed 20 Sept 2017."},{"key":"30_CR11","unstructured":"Brochu E, Cora VM, De Freitas N (2010) A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. https:\/\/arxiv.org\/abs\/1012.2599. Accessed 20 Sept 2017."},{"key":"30_CR12","unstructured":"Snoek J, Rippel O, Swersky K, Kiros R, Satish N, Sundaram N, Patwary M, Prabhat M, Adams R (2015) Scalable Bayesian optimization using deep neural networks In: International Conference on Machine Learning, 2171\u20132180.. (ICML)."},{"key":"30_CR13","doi-asserted-by":"crossref","unstructured":"Hansen N, Auger A, Ros R, Finck S, Po\u0161\u00edk P (2010) Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009 In: Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, 1689\u20131696.","DOI":"10.1145\/1830761.1830790"},{"key":"30_CR14","doi-asserted-by":"crossref","unstructured":"Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. MPS-SIAM Ser Optim. http:\/\/epubs.siam.org\/doi\/book\/10.1137\/1.9780898718768.","DOI":"10.1137\/1.9780898718768"},{"key":"30_CR15","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1093\/comjnl\/7.4.308","volume":"7","author":"JA Nelder","year":"1965","unstructured":"Nelder JA, Mead RA (1965) Simplex method for function minimization. Comput J 7:308\u2013313.","journal-title":"Comput J"},{"key":"30_CR16","unstructured":"Gilles C, Patrick R, M\u00e9lanie H (2005) Model selection for support vector classifiers via direct simplex search. The Florida Artificial Intelligence Research Society (FLAIRS) Conference:431\u2013435."},{"key":"30_CR17","unstructured":"Domhan T, Springenberg JT, Hutter F (2015) Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves In: Proceedings of the 24th International Joint Conference on Artificial Intelligence, 3460\u20133468.. (IJCAI)."},{"key":"30_CR18","unstructured":"Klein A, Falkner S, Springenberg JT, Hutter F (2016) Bayesian neural networks for predicting learning curves. Workshop on Bayesian Deep Learning, NIPS. http:\/\/bayesiandeeplearning.org\/2016\/papers\/BDL_38.pdf."},{"key":"30_CR19","unstructured":"Klein A, Falkner S, Springenberg JT, Hutter F (2017) Learning curve prediction with bayesian neural networks. International Conference on Learning Representations (ICLR). http:\/\/www.iclr.cc\/doku.php?id=iclr2017:conference_posters#tuesday_morning."},{"key":"30_CR20","unstructured":"De Rainville FM, Fortin FA, Gardner MA, Parizeau M, Gagn\u00e9 C (2012) \u201cDEAP: A Python Framework for Evolutionary Algorithms\u201d In: EvoSoft Workshop, Companion proc. of the Genetic and Evolutionary Computation Conference (GECCO). https:\/\/dl.acm.org\/citation.cfm?id=2330799."},{"key":"30_CR21","unstructured":"LeCun Y, Cortes C (2010) MNIST handwritten digit database. http:\/\/yann.lecun.com\/exdb\/mnist\/. Accessed 20 Sept 2017."},{"key":"30_CR22","unstructured":"Yangqing J, Evan S, Jeff D, Sergey K, Jonathan L, Ross G, Sergio G, Trevor D (2014) Caffe: Convolutional architecture for fast feature embedding. https:\/\/arxiv.org\/abs\/1408.5093. Accessed 20 Sept 2017."},{"key":"30_CR23","unstructured":"Evan S (2014) Training LeNet on MNIST with Caffe. http:\/\/caffe.berkeleyvision.org\/gathered\/examples\/mnist.html. Accessed 20 Sept 2017."},{"key":"30_CR24","unstructured":"Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. International Conference on Machine Learning. https:\/\/dl.acm.org\/citation.cfm?id=3104425."},{"key":"30_CR25","unstructured":"Chang JR, Chen YS (2015) Batch-Normalized Maxout Network in Network In: Proceedings of the 33rd International Conference on Machine Learning. https:\/\/arxiv.org\/abs\/1511.02583. Accessed 20 Sept 2017."},{"key":"30_CR26","unstructured":"Gil L, Tal H (2015) Age and gender classification using convolutional neural networks. Computer Vision and Pattern Recognition Workshops (CVPRW). http:\/\/ieeexplore.ieee.org\/document\/7301352\/."},{"key":"30_CR27","unstructured":"Gil L, Tal H (2015) Age and gender classification using convolutional neural networks. http:\/\/www.openu.ac.il\/home\/hassner\/projects\/cnn_agegender. Accessed 20 Sept 2017."},{"issue":"12","key":"30_CR28","doi-asserted-by":"publisher","first-page":"2170","DOI":"10.1109\/TIFS.2014.2359646","volume":"9","author":"E Eran","year":"2014","unstructured":"Eran E, Roee E, Tal E (2014) Age and gender estimation of unfiltered faces. IEEE Trans Inf Forensic Secur 9(12):2170\u20132179.","journal-title":"IEEE Trans Inf Forensic Secur"},{"key":"30_CR29","unstructured":"Yangjin J (2013) The learning rate decay policy. https:\/\/github.com\/BVLC\/caffe\/blob\/master\/src\/caffe\/proto\/caffe.proto. Accessed 20 Sept 2017."}],"container-title":["IPSJ Transactions on Computer Vision and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41074-017-0030-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s41074-017-0030-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41074-017-0030-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T23:38:10Z","timestamp":1750981090000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s41074-017-0030-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,10]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,12]]}},"alternative-id":["30"],"URL":"https:\/\/doi.org\/10.1186\/s41074-017-0030-7","relation":{},"ISSN":["1882-6695"],"issn-type":[{"value":"1882-6695","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,10]]},"assertion":[{"value":"9 January 2017","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 November 2017","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"20"}}