{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:50:46Z","timestamp":1760147446156,"version":"build-2065373602"},"reference-count":66,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,3]],"date-time":"2023-02-03T00:00:00Z","timestamp":1675382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"the Natural Sciences and Engineering Research Council of Canada (NSERC)","doi-asserted-by":"publisher","award":["2018-03865"],"award-info":[{"award-number":["2018-03865"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This paper examines a calculus-based approach to building model functions in a derivative-free algorithm. This calculus-based approach can be used when the objective function considered is defined via more than one blackbox. Two versions of a derivative-free trust-region method are implemented. The first version builds model functions by using a calculus-based approach, and the second version builds model functions by directly considering the objective function. The numerical experiments demonstrate that the calculus-based approach provides better results in most situations and significantly better results in specific situations.<\/jats:p>","DOI":"10.3390\/a16020084","type":"journal-article","created":{"date-parts":[[2023,2,3]],"date-time":"2023-02-03T03:36:57Z","timestamp":1675395417000},"page":"84","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["About the Performance of a Calculus-Based Approach to Building Model Functions in a Derivative-Free Trust-Region Algorithm"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4240-3903","authenticated-orcid":false,"given":"Warren","family":"Hare","sequence":"first","affiliation":[{"name":"Department of Mathematics, University of British Columbia, Okanagan Campus, Kelowna, BC V1V 1V7, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1827-8508","authenticated-orcid":false,"given":"Gabriel","family":"Jarry-Bolduc","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of British Columbia, Okanagan Campus, Kelowna, BC V1V 1V7, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Conn, A., Gould, N., and Toint, P. (2000). Trust Region Methods, SIAM.","DOI":"10.1137\/1.9780898719857"},{"key":"ref_2","unstructured":"Nocedal, J., and Wright, S. (2006). Numerical Optimization, Springer Science & Business Media."},{"key":"ref_3","first-page":"164","article-title":"A Method for the Solution of Certain Problems in Least Squares","volume":"2","author":"Levenberg","year":"1944","journal-title":"Appl. Math."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1007\/BF00932218","article-title":"Two new unconstrained optimization algorithms which use function and gradient values","volume":"28","author":"Dennis","year":"1979","journal-title":"J. Optim. Theory Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Dennis, J., and Schnabel, R. (1996). Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM.","DOI":"10.1137\/1.9781611971200"},{"key":"ref_6","unstructured":"Fletcher, R. (1980). Practical Methods of Optimization: Unconstrained Optimization, Wiley."},{"key":"ref_7","first-page":"541","article-title":"Maximization by quadratic hill-climbing","volume":"34","author":"Goldfeld","year":"1966","journal-title":"Econom. J. Econom. Soc."},{"key":"ref_8","unstructured":"Hebden, M. (1973). An Algorithm for Minimization Using Exact Second Derivatives, Citeseer."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1093\/imamat\/16.3.321","article-title":"An algorithm for minimax solution of overdetermined systems of non-linear equations","volume":"16","author":"Madsen","year":"1975","journal-title":"IMA J. Appl. Math."},{"key":"ref_10","unstructured":"Mor\u00e9, J. (1978). Numerical Analysis, Springer."},{"key":"ref_11","unstructured":"Mor\u00e9, J. (1983). Mathematical Programming: The State of the Art, Springer."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1137\/0904038","article-title":"Computing a trust region step","volume":"4","author":"Sorensen","year":"1983","journal-title":"SIAM J. Sci. Stat. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1017\/S033427000000120X","article-title":"Nonlinear least squares\u2014the Levenberg algorithm revisited","volume":"19","author":"Osborne","year":"1976","journal-title":"J. Aust. Math. Soc."},{"key":"ref_14","unstructured":"Powell, M. (1970). A Fortran Subroutine for Solving Systems of Nonlinear Algebraic Equations, Atomic Energy Research Establishment. Technical Report."},{"key":"ref_15","unstructured":"Powell, M. (1970). Numerical Methods for Nonlinear Algebraic Equations, Gordon and Breach."},{"key":"ref_16","unstructured":"Powell, M. (1970). Nonlinear Programming, Elsevier."},{"key":"ref_17","unstructured":"Mangasarian, O.L., Meyer, R.R., and Robinson, S.M. (1975). Nonlinear Programming 2, Elsevier."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/BF02591998","article-title":"On the global convergence of trust region algorithms for unconstrained minimization","volume":"29","author":"Powell","year":"1984","journal-title":"Math. Program."},{"key":"ref_19","unstructured":"Sorensen, D. (1981). Trust-Region Methods for Unconstrained Minimization, U.S. Department of Energy Office of Scientific and Technical Information."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1137\/0719026","article-title":"Newton\u2019s method with a model trust region modification","volume":"19","author":"Sorensen","year":"1982","journal-title":"SIAM J. Numer. Anal."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1137\/0720042","article-title":"The conjugate gradient method and trust regions in large scale optimization","volume":"20","author":"Steihaug","year":"1983","journal-title":"SIAM J. Numer. Anal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1090\/S0025-5718-1978-0483452-7","article-title":"Some numerical results using a sparse matrix updating formula in unconstrained optimization","volume":"32","author":"Toint","year":"1978","journal-title":"Math. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1137\/0716076","article-title":"On the superlinear convergence of an algorithm for solving a sparse minimization problem","volume":"16","author":"Toint","year":"1979","journal-title":"SIAM J. Numer. Anal."},{"key":"ref_24","unstructured":"Toint, P. (1981). Convergence Properties of a Class of Minimization Algorithms That Use a Possibly Unbounded Sequence of Quadratic Approximations, Elsevier."},{"key":"ref_25","unstructured":"Toint, P. (1981). Sparse Matrices and Their Uses, Academic Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1093\/imamat\/12.3.339","article-title":"Function minimization by interpolation in a data table","volume":"12","author":"Winfield","year":"1973","journal-title":"IMA J. Appl. Math."},{"key":"ref_27","unstructured":"Winfield, D. (1969). Function and Functional Optimization by Interpolation in Data Tables. [Ph.D. Thesis, Harvard University]."},{"key":"ref_28","first-page":"19","article-title":"A brief introduction to quasi-Newton methods","volume":"22","author":"Dennis","year":"1978","journal-title":"Numer. Anal."},{"key":"ref_29","unstructured":"Conn, A., Scheinberg, K., and Toint, P. (1997). Approximation Theory and Optimization: Tributes to M.J.D. Powell, Cambridge University Press."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1007\/s101070100264","article-title":"Wedge trust region methods for derivative free optimization","volume":"91","author":"Marazzi","year":"2002","journal-title":"Math. Program."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1007\/s101070100290","article-title":"UOBYQA: Unconstrained optimization by quadratic approximation","volume":"92","author":"Powell","year":"2002","journal-title":"Math. Program."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1007\/s10107-003-0430-6","article-title":"On trust region methods for unconstrained minimization without derivatives","volume":"97","author":"Powell","year":"2003","journal-title":"Math. Program."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1080\/10556780500140227","article-title":"Optimizing partially separable functions without derivatives","volume":"20","author":"Colson","year":"2005","journal-title":"Optim. Methods Softw."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1137\/060673424","article-title":"Global convergence of general derivative-free trust-region algorithms to first-and second-order critical points","volume":"20","author":"Conn","year":"2009","journal-title":"SIAM J. Optim."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Conn, A., Scheinberg, K., and Vicente, L. (2009). Introduction to Derivative-Free Optimization, SIAM.","DOI":"10.1137\/1.9780898718768"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Audet, C., and Hare, W. (2017). Derivative-free and Blackbox Optimization, Springer.","DOI":"10.1007\/978-3-319-68913-5"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3012","DOI":"10.1137\/19M125772X","article-title":"Trust-region methods for the derivative-free optimization of nonsmooth black-box functions","volume":"29","author":"Liuzzi","year":"2019","journal-title":"SIAM J. Optim."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1080\/02286203.2007.11442407","article-title":"A new derivative-free algorithm for the medical image registration problem","volume":"27","author":"Oeuvray","year":"2007","journal-title":"Int. J. Model. Simul."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1137\/100814688","article-title":"Derivative-Free Optimization of Expensive Functions with Computational Error Using Weighted Regression","volume":"23","author":"Billups","year":"2013","journal-title":"SIAM J. Optim."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Conn, A., Scheinberg, K., and Toint, P. (1998, January 2\u20134). A derivative free optimization algorithm in practice. Proceedings of the 7th AIAA\/USAF\/NASA\/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, USA.","DOI":"10.2514\/6.1998-4718"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1080\/10556788.2010.549231","article-title":"An active-set trust-region method for derivative-free nonlinear bound-constrained optimization","volume":"26","author":"Gratton","year":"2011","journal-title":"Optim. Methods Softw."},{"key":"ref_42","unstructured":"Powell, M. (2009). The BOBYQA Algorithm for Bound Constrained Optimization without Derivatives, Cambridge NA Report NA2009\/06; University of Cambridge."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3197","DOI":"10.1137\/070691814","article-title":"ORBIT: Optimization by radial basis function interpolation in trust-regions","volume":"30","author":"Wild","year":"2008","journal-title":"SIAM J. Sci. Comput."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1137\/09074927X","article-title":"Global convergence of radial basis function trust region derivative-free algorithms","volume":"21","author":"Wild","year":"2011","journal-title":"SIAM J. Optim."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1377612.1377613","article-title":"SNOBFIT\u2014Stable noisy optimization by branch and fit","volume":"35","author":"Huyer","year":"2008","journal-title":"ACM Trans. Math. Softw. (TOMS)"},{"key":"ref_46","first-page":"171","article-title":"Active-set strategy in Powell\u2019s method for optimization without derivatives","volume":"30","author":"Echebest","year":"2011","journal-title":"Comput. Appl. Math."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.cam.2004.11.029","article-title":"CONDOR, a new parallel, constrained extension of Powell\u2019s UOBYQA algorithm: Experimental results and comparison with the DFO algorithm","volume":"181","author":"Berghen","year":"2005","journal-title":"J. Comput. Appl. Math."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1080\/10556788.2016.1226305","article-title":"CONORBIT: Constrained optimization by radial basis function interpolation in trust regions","volume":"32","author":"Regis","year":"2017","journal-title":"Optim. Methods Softw."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1007\/s10589-014-9715-3","article-title":"A derivative-free trust-funnel method for equality-constrained nonlinear optimization","volume":"61","author":"Sampaio","year":"2015","journal-title":"Comput. Optim. Appl."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1007\/s10589-013-9607-y","article-title":"A derivative-free algorithm for linearly constrained optimization problems","volume":"57","author":"Gumma","year":"2014","journal-title":"Comput. Optim. Appl."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s12532-015-0084-4","article-title":"On fast trust region methods for quadratic models with linear constraints","volume":"7","author":"Powell","year":"2015","journal-title":"Math. Program. Comput."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1080\/10556788.2015.1026968","article-title":"A trust-region derivative-free algorithm for constrained optimization","volume":"30","author":"Conejo","year":"2015","journal-title":"Optim. Methods Softw."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"3001","DOI":"10.1137\/17M114741X","article-title":"Manifold sampling for optimization of nonconvex functions that are piecewise linear compositions of smooth components","volume":"28","author":"Khan","year":"2018","journal-title":"SIAM J. Optim."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Larson, J., Menickelly, M., and Zhou, B. (2020). Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions. arXiv.","DOI":"10.1137\/20M1378089"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1137\/18M1215864","article-title":"Calculus identities for generalized simplex gradients: Rules and applications","volume":"30","author":"Hare","year":"2020","journal-title":"SIAM J. Optim."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1007\/s11228-020-00556-y","article-title":"A Discussion on Variational Analysis in Derivative-Free Optimization","volume":"28","author":"Hare","year":"2020","journal-title":"Set-Valued Var. Anal."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1007\/s11590-014-0815-x","article-title":"The calculus of simplex gradients","volume":"9","author":"Regis","year":"2015","journal-title":"Optim. Lett."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Hare, W., Jarry-Bolduc, G., and Planiden, C. (2020). Error bounds for overdetermined and underdetermined generalized centred simplex gradients. arXiv.","DOI":"10.1093\/imanum\/draa089"},{"key":"ref_59","unstructured":"Hare, W., Jarry-Bolduc, G., and Planiden, C. (2020). Hessian approximations. arXiv."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Chen, Y., Jarry-Bolduc, G., and Hare, W. (2021). Error Analysis of Surrogate Models Constructed through Operations on Sub-models. arXiv.","DOI":"10.1287\/moor.2022.1344"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Hough, M., and Roberts, L. (2021). Model-Based Derivative-Free Methods for Convex-Constrained Optimization. arXiv.","DOI":"10.1137\/21M1460971"},{"key":"ref_62","unstructured":"Audet, C., and Hare, W. (2020). Numerical Nonsmooth Optimization, Springer."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1137\/080724083","article-title":"Benchmarking derivative-free optimization algorithms","volume":"20","author":"Wild","year":"2009","journal-title":"SIAM J. Optim."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Shi, H.J.M., Xuan, M.Q., Oztoprak, F., and Nocedal, J. (2022, December 01). On the Numerical Performance of Finite-Difference-Based Methods for Derivative-Free Optimization. Available online: https:\/\/doi.org\/10.1080\/10556788.2022.2121832.","DOI":"10.1080\/10556788.2022.2121832"},{"key":"ref_65","unstructured":"Audet, C., Le Digabel, S., Rochon Montplaisir, V., and Tribes, C. (2021). NOMAD version 4: Nonlinear optimization with the MADS algorithm. arXiv."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Cartis, C., and Roberts, L. (2022, December 01). Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization. Available online: https:\/\/doi.org\/10.1007\/s10107-022-01836-1.","DOI":"10.1007\/s10107-022-01836-1"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/2\/84\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:22:55Z","timestamp":1760120575000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/2\/84"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,3]]},"references-count":66,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["a16020084"],"URL":"https:\/\/doi.org\/10.3390\/a16020084","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2023,2,3]]}}}