{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T09:10:34Z","timestamp":1778231434057,"version":"3.51.4"},"reference-count":68,"publisher":"Institute for Operations Research and the Management Sciences (INFORMS)","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Mathematics of OR"],"published-print":{"date-parts":[[2026,5]]},"abstract":"<jats:p>We introduce a derivative-free optimization algorithm that efficiently computes minima for various classes of one-dimensional functions, including nonconvex and nonsmooth functions. This algorithm numerically approximates the gradient flow of a relaxed functional, integrating strategies such as Monte Carlo methods, rejection sampling, and adaptive techniques. These strategies enhance performance in solving a diverse range of optimization problems while significantly reducing the number of required function evaluations compared with established methods. We present a proof of the convergence of the algorithm for locally convex functions and illustrate its numerical performance by comprehensive benchmarking with test functions, showcasing different properties and characteristics. The proposed algorithm offers a substantial potential for real-world models. It is particularly advantageous in situations requiring computationally intensive objective function evaluations, such as hyperparameter tuning in machine learning or line search in large-scale optimization problems involving the discretization of partial differential equations.<\/jats:p>\n                  <jats:p>Funding: The research reported in this publication was supported by funding from King Abdullah University of Science and Technology (KAUST). D. A. Gomes was supported by KAUST [baseline funds and Grant KAUST OSR-CRG2021-4674].<\/jats:p>\n                  <jats:p>Supplemental Material: The online appendix is available at https:\/\/doi.org\/10.1287\/moor.2023.0340 .<\/jats:p>","DOI":"10.1287\/moor.2023.0340","type":"journal-article","created":{"date-parts":[[2025,5,6]],"date-time":"2025-05-06T10:44:46Z","timestamp":1746528286000},"page":"1007-1036","source":"Crossref","is-referenced-by-count":0,"title":["A Derivative-Free Algorithm for Minimization in One Dimension: Relaxation, Monte Carlo, and Sampling"],"prefix":"10.1287","volume":"51","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7533-6033","authenticated-orcid":false,"given":"Alexandra A.","family":"Gomes","sequence":"first","affiliation":[{"name":"CEMSE Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3129-3956","authenticated-orcid":false,"given":"Diogo A.","family":"Gomes","sequence":"additional","affiliation":[{"name":"CEMSE Division, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"109","reference":[{"key":"B1","doi-asserted-by":"crossref","unstructured":"Akimoto Y, Nagata Y, Ono I, Kobayashi S (2010) Bidirectional relation between CMA evolution strategies and natural evolution strategies. Schaefer R, Cotta C, Ko\u0142odziej J, Rudolph G, eds.\n                      Parallel Problem Solving Nature, PPSN XI\n                      , Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, Heidelberg), 154\u2013163.","DOI":"10.1007\/978-3-642-15844-5_16"},{"key":"B2","doi-asserted-by":"crossref","unstructured":"Amari SI, Douglas SC (1998) Why natural gradient?\n                      Proc. 1998 IEEE Internat. Conf. Acoustics Speech Signal Processing, ICASSP98 (Cat. No. 98CH36181)\n                      , vol. 2 (IEEE, Piscataway, NJ), 1213\u20131216.","DOI":"10.1109\/ICASSP.1998.675489"},{"key":"B3","first-page":"601","volume":"9","author":"Barricelli NA","year":"1957","journal-title":"Methodos"},{"issue":"10","key":"B4","first-page":"281","volume":"13","author":"Bergstra J","year":"2012","journal-title":"J. Machine Learn. Res."},{"key":"B5","doi-asserted-by":"crossref","unstructured":"Berny A (2000) Selection and reinforcement learning for combinatorial optimization. Schoenauer M, Deb K, Rudolph G, Yao X, Lutton E, Merelo JJ, Schwefel HP, eds.\n                      Parallel Problem Solving Nature, PPSN VI\n                      , Lecture Notes in Computer Science, vol. 1917 (Springer, Berlin, Heidelberg), 601\u2013610.","DOI":"10.1007\/3-540-45356-3_59"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1023\/A:1015059928466"},{"key":"B7","doi-asserted-by":"crossref","unstructured":"Bosman PA, Thierens D (2000) Expanding from discrete to continuous estimation of distribution algorithms. Schoenauer M, Deb K, Rudolph G, Yao X, Lutton E, Merelo JJ, Schwefel HP, eds.\n                      Parallel Problem Solving Nature, PPSN VI\n                      , Lecture Notes in Computer Science, vol. 1917 (Springer, Berlin, Heidelberg), 767\u2013776.","DOI":"10.1007\/3-540-45356-3_75"},{"key":"B8","volume-title":"Numerical Analysis","author":"Burden RL","year":"2011"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1137\/0724076"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1007\/s40687-018-0148-y"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718768"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2016.01.004"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1016\/j.swevo.2019.04.008"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-014-0793-x"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-015-9747-3"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623400374495"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2015.2409256"},{"key":"B18","volume-title":"Bayesian Optimization","author":"Garnett R","year":"2022"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1137\/120880811"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2011.2160625"},{"key":"B21","doi-asserted-by":"crossref","unstructured":"Glasmachers T, Schaul T, Yi S, Wierstra D, Schmidhuber J (2010) Exponential natural evolution strategies.\n                      GECCO \u201810: Proc. 12th Annual Conf. Genetic Evolutionary Comput.\n                      (Association for Computing Machinery, New York), 393\u2013400.","DOI":"10.1145\/1830483.1830557"},{"key":"B22","volume-title":"Monte Carlo Methods in Financial Engineering","volume":"53","author":"Glasserman P","year":"2004"},{"key":"B23","volume-title":"Genetic Algorithm in Search Optimization and Machine Learning","author":"Goldberg DE","year":"1989"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-32494-1_4"},{"key":"B25","doi-asserted-by":"crossref","unstructured":"Hansen N, Ostermeier A (1996) Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation.\n                      Proc. IEEE Internat. Conf. Evolutionary Comput.\n                      (IEEE, Piscataway, NJ), 312\u2013317.","DOI":"10.1109\/ICEC.1996.542381"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1162\/106365601750190398"},{"key":"B27","volume-title":"Adaptation in Natural and Artificial Systems","author":"Holland JH","year":"1975"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1145\/321062.321069"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008382309369"},{"key":"B30","doi-asserted-by":"crossref","unstructured":"Kennedy J, Eberhart R (1995) Particle swarm optimization.\n                      Proc. ICNN\u201995 Internat. Conf. Neural Networks\n                      , vol. 4 (IEEE, Piscataway, NJ), 1942\u20131948.","DOI":"10.1109\/ICNN.1995.488968"},{"issue":"5","key":"B31","first-page":"519","volume":"24","author":"Khachaturyan A","year":"1979","journal-title":"Soviet Phys. Crystallography"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1126\/science.220.4598.671"},{"key":"B33","doi-asserted-by":"crossref","unstructured":"Lagarias JC, Reeds JA, Wright MH, Wright PE (1998) Convergence properties of the nelder-mead simplex method in low dimensions.\n                      SIAM J. Optim.\n                      9(1):112\u2013147.","DOI":"10.1137\/S1052623496303470"},{"key":"B34","volume-title":"Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation","volume":"2","author":"Larra\u00f1aga P","year":"2001"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492919000060"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611972672"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejco.2021.100012"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1007\/s11590-014-0816-9"},{"key":"B39","doi-asserted-by":"crossref","unstructured":"Lu J, Tadmor E, Zengino\u011flu A (2024) Swarm-based gradient descent method for non-convex optimization.\n                      Comm. Amer. Mathematical Soc.\n                      4:787\u2013822.","DOI":"10.1090\/cams\/42"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1002\/aic.690190413"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2018.2868770"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.107849"},{"issue":"2","key":"B43","first-page":"246","volume":"26","author":"Matyas J","year":"1965","journal-title":"Automation Remote Control"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623496303482"},{"key":"B45","first-page":"166","volume-title":"Towards Global Optimization","author":"Mockus J","year":"1975"},{"key":"B46","first-page":"117","volume-title":"Towards Global Optimization","volume":"2","author":"Mockus J","year":"1978"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1137\/0904038"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/7.4.308"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-015-9296-2"},{"key":"B50","doi-asserted-by":"crossref","unstructured":"Osher S, Heaton H, Wu Fung S (2023) A Hamilton-Jacobi-based proximal operator.\n                      Proc. Natl. Acad. Sci.\n                      120(14):e2220469120.","DOI":"10.1073\/pnas.2220469120"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013500812258"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1287\/opre.18.6.1225"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1007\/BF01584660"},{"key":"B54","volume-title":"Differential Evolution: A Practical Approach to Global Optimization","author":"Price K","year":"2006"},{"key":"B55","first-page":"1337","volume":"24","author":"Rastrigin L","year":"1963","journal-title":"Automation Remote Control"},{"key":"B56","volume-title":"Evolutionsstrategie. Optimierung Technischer Systeme Nach Prinzipien Derbiologischen Evolution","author":"Rechenberg I","year":"1973"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-012-9951-y"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-4321-0"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2020.105165"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-0348-5927-1_5"},{"key":"B61","doi-asserted-by":"crossref","unstructured":"Shi Y, Eberhart R (1998) A modified particle swarm optimizer.\n                      1998 IEEE Internat. Conf. Evolutionary Comput. Proc. IEEE World Congress Comput. Intelligence (Cat. No. 98TH8360)\n                      , (IEEE, Piscataway, NJ), 69\u201373.","DOI":"10.1109\/ICEC.1998.699146"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1137\/0722003"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008202821328"},{"key":"B64","doi-asserted-by":"crossref","unstructured":"Sun Y, Wierstra D, Schaul T, Schmidhuber J (2009) Efficient natural evolution strategies.\n                      GECCO \u201809: Proc. 11th Annual Conf. Genetic Evolutionary Comput.\n                      (Association for Computing Machinery, New York), 539\u2013546.","DOI":"10.1145\/1569901.1569976"},{"key":"B65","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623493250780"},{"key":"B66","unstructured":"Vardhan H, Stich SU (2022) Tackling benign nonconvexity with smoothing and stochastic gradients. Preprint, submitted February 18, https:\/\/arxiv.org\/abs\/2202.09052."},{"issue":"1","key":"B67","first-page":"949","volume":"15","author":"Wierstra D","year":"2014","journal-title":"J. Machine Learn. Res."},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.07.061"}],"container-title":["Mathematics of Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubsonline.informs.org\/doi\/pdf\/10.1287\/moor.2023.0340","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T08:25:44Z","timestamp":1778228744000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubsonline.informs.org\/doi\/10.1287\/moor.2023.0340"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5]]},"references-count":68,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,5]]}},"alternative-id":["10.1287\/moor.2023.0340"],"URL":"https:\/\/doi.org\/10.1287\/moor.2023.0340","relation":{},"ISSN":["0364-765X","1526-5471"],"issn-type":[{"value":"0364-765X","type":"print"},{"value":"1526-5471","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5]]}}}