{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T21:55:51Z","timestamp":1768514151316,"version":"3.49.0"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2021,4,6]],"date-time":"2021-04-06T00:00:00Z","timestamp":1617667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"FCT\/Portugal","award":["PD\/BI\/128105\/2016"],"award-info":[{"award-number":["PD\/BI\/128105\/2016"]}]},{"name":"FCT\/Portugal","award":["UID\/MAT\/00324\/2019"],"award-info":[{"award-number":["UID\/MAT\/00324\/2019"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,13]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>In this paper we suggest two ways of calculating interpolation models for unconstrained smooth nonlinear optimization when Hessian-vector products are available. The main idea is to interpolate the objective function using a quadratic on a set of points around the current one, and concurrently using the curvature information from products of the Hessian times appropriate vectors, possibly defined by the interpolating points. These enriched interpolating conditions then form an affine space of model Hessians or model Newton directions, from which a particular one can be computed once an equilibrium or least secant principle is defined. A first approach consists of recovering the Hessian matrix satisfying the enriched interpolating conditions, from which then a Newton direction model can be computed. In a second approach we pose the recovery problem directly in the Newton direction. These techniques can lead to a significant reduction in the overall number of Hessian-vector products when compared to the inexact or truncated Newton method, although simple implementations may pay a cost in the number of function evaluations and the dense linear algebra involved poses a scalability challenge.<\/jats:p>","DOI":"10.1093\/imanum\/drab022","type":"journal-article","created":{"date-parts":[[2021,3,31]],"date-time":"2021-03-31T19:14:42Z","timestamp":1617218082000},"page":"1766-1788","source":"Crossref","is-referenced-by-count":5,"title":["Modeling Hessian-vector products in nonlinear optimization: new Hessian-free methods"],"prefix":"10.1093","volume":"42","author":[{"given":"L","family":"Song","sequence":"first","affiliation":[{"name":"CMUC, Department of Mathematics, University of Coimbra, 3001-501 Coimbra, Portugal"}]},{"given":"L N","family":"Vicente","sequence":"additional","affiliation":[{"name":"Department of Industrial and Systems Engineering, Lehigh University, 200 West Packer Avenue, Bethlehem, PA 18015-1582, USA and Centre for Mathematics of the University of Coimbra (CMUC), 3001-501 Coimbra, Portugal"}]}],"member":"286","published-online":{"date-parts":[[2021,4,6]]},"reference":[{"key":"2022042011472942100_ref1","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1137\/1.9780898718133.ch16","article-title":"Parallel algorithms for PDE-constrained optimization","volume-title":"Parallel Processing for Scientific Computing","author":"Ak\u00e7elik","year":"2006"},{"key":"2022042011472942100_ref2","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/s10107-012-0578-z","article-title":"Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization","volume":"134","author":"Bandeira","year":"2012","journal-title":"Math. Program."},{"key":"2022042011472942100_ref3","first-page":"5595","article-title":"Automatic differentiation in machine learning: a survey","volume":"18","author":"Baydin","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"2022042011472942100_ref4","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1137\/18M1177718","article-title":"Derivative-free optimization of noisy functions via quasi-Newton methods","volume":"29","author":"Berahas","year":"2019","journal-title":"SIAM J. Optim."},{"key":"2022042011472942100_ref5","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1137\/S106482750241565X","article-title":"Parallel Lagrange\u2013Newton\u2013Krylov\u2013Schur methods for PDE-constrained optimization. Part I: the Krylov\u2013Schur solver","volume":"27","author":"Biros","year":"2005","journal-title":"SIAM J. Sci. Comput."},{"key":"2022042011472942100_ref6","article-title":"Lecture Notes in Computational Science and Engineering","volume-title":"Automatic Differentiation: Applications, Theory, and Implementations","author":"B\u00fccker","year":"2005"},{"key":"2022042011472942100_ref7","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1080\/10556780500140227","article-title":"Optimizing partially separable functions without derivatives","volume":"20","author":"Colson","year":"2005","journal-title":"Optim. Methods Softw."},{"key":"2022042011472942100_ref8","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/s10107-006-0073-5","article-title":"Geometry of interpolation sets in derivative free optimization","volume":"111","author":"Conn","year":"2008","journal-title":"Math. Program."},{"key":"2022042011472942100_ref9","article-title":"MPS-SIAM Series on Optimization","volume-title":"Introduction to Derivative-Free Optimization","author":"Conn","year":"2009"},{"key":"2022042011472942100_ref10","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1093\/imamat\/13.1.117","article-title":"On the estimation of sparse Jacobian matrices","volume":"13","author":"Curtis","year":"1974","journal-title":"J. Inst. Math. Appl."},{"key":"2022042011472942100_ref11","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1137\/0719025","article-title":"Inexact Newton methods","volume":"19","author":"Dembo","year":"1982","journal-title":"SIAM J. Numer. Anal."},{"key":"2022042011472942100_ref12","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1137\/1019005","article-title":"Quasi-Newton methods, motivation and theory","volume":"19","author":"Dennis","year":"1977","journal-title":"SIAM Rev."},{"key":"2022042011472942100_ref13","doi-asserted-by":"crossref","volume-title":"Numerical Methods for Unconstrained Optimization and Nonlinear Equations","author":"Dennis","DOI":"10.1137\/1.9781611971200"},{"key":"2022042011472942100_ref14","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1137\/1021091","article-title":"Least change secant updates for quasi-Newton methods","volume":"21","author":"Dennis","year":"1996","journal-title":"SIAM Rev."},{"key":"2022042011472942100_ref15","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/s101070100263","article-title":"Benchmarking optimization software with performance profiles","volume":"91","author":"Dolan","year":"2002","journal-title":"Math. Program."},{"key":"2022042011472942100_ref16","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1137\/15M1020575","article-title":"JuMP: a modeling language for mathematical optimization","volume":"59","author":"Dunning","year":"2017","journal-title":"SIAM Rev."},{"key":"2022042011472942100_ref17","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1137\/0804022","article-title":"Globally convergent inexact Newton methods","volume":"4","author":"Eisenstat","year":"1994","journal-title":"SIAM J. Optim."},{"key":"2022042011472942100_ref18","volume-title":"Practical Methods of Optimization","author":"Fletcher","year":"1987"},{"key":"2022042011472942100_ref19","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1137\/0805010","article-title":"An optimal positive definite update for sparse Hessian matrices","volume":"5","author":"Fletcher","year":"1995","journal-title":"SIAM J. Optim."},{"key":"2022042011472942100_ref20","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/978-1-4612-1960-6_3","article-title":"Computing sparse Hessian and Jacobian approximations with optimal hereditary properties","volume-title":"Large-Scale Optimization with Applications","author":"Fletcher","year":"1997"},{"key":"2022042011472942100_ref21","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1137\/0904025","article-title":"Computing forward-difference intervals for numerical optimization","volume":"4","author":"Gill","year":"1983","journal-title":"SIAM J. Sci. Statist. Comput.,"},{"key":"2022042011472942100_ref22","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1137\/0805015","article-title":"An implicit filtering algorithm for optimization of functions with many local minima","volume":"5","author":"Gilmore","year":"1995","journal-title":"SIAM J. Optim."},{"key":"2022042011472942100_ref23","article-title":"Personal communication","volume-title":"June","author":"Gould","year":"2018"},{"key":"2022042011472942100_ref24","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1137\/S1052623497322735","article-title":"Solving the trust-region subproblem using the Lanczos method","volume":"9","author":"Gould","year":"1999","journal-title":"SIAM J. Optim."},{"key":"2022042011472942100_ref25","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1007\/s10589-014-9687-3","article-title":"CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization","volume":"60","author":"Gould","year":"2015","journal-title":"Comput., Optim. Appl."},{"key":"2022042011472942100_ref26","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s10107-018-1328-7","article-title":"A decoupled first\/second-order steps technique for nonconvex nonlinear unconstrained optimization with improved complexity bounds","volume":"179","author":"Gratton","year":"2020","journal-title":"Math. Program."},{"key":"2022042011472942100_ref27","volume-title":"Evaluating Derivatives: Principles and Techniques of Automatic differentiation","author":"Griewank","year":"2000"},{"key":"2022042011472942100_ref28","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1137\/0723046","article-title":"A nonmonotone line search technique for Newton\u2019s method","volume":"34","author":"Grippo","year":"1986","journal-title":"SIAM J. Numer. Anal."},{"key":"2022042011472942100_ref29","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1007\/s11081-014-9258-6","article-title":"Inexact Hessian-vector products in reduced-space differential-equation constrained optimization","volume":"15","author":"Hicken","year":"2014","journal-title":"Optim. Eng."},{"key":"2022042011472942100_ref30","volume-title":"Optimization with PDE Constraints","author":"Hinze","year":"2008"},{"key":"2022042011472942100_ref31","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2012-3","article-title":"Scalable minimum Bayes risk training of deep neural network acoustic models using distributed Hessian-free optimization","volume-title":"Interspeech","author":"Kingsbury","year":"2012"},{"key":"2022042011472942100_ref32","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1007\/BF01589116","article-title":"On the limited memory BFGS method for large scale optimization","volume":"45","author":"Liu","year":"1989","journal-title":"Math. Program."},{"key":"2022042011472942100_ref33","article-title":"Learning your optimisation problem: making the most of sparsity","author":"Mackinder","year":"2016"},{"key":"2022042011472942100_ref34","first-page":"735","article-title":"Deep learning via Hessian-free optimization","volume":"27","author":"Martens","year":"2010","journal-title":"ICML"},{"key":"2022042011472942100_ref35","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1137\/0721052","article-title":"Newton-type minimization via the Lanczos method","volume":"21","author":"Nash","year":"1984","journal-title":"SIAM J. Numer. Anal."},{"key":"2022042011472942100_ref36","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1090\/S0025-5718-1980-0572855-7","article-title":"Updating quasi-Newton matrices with limited storage","volume":"35","author":"Nocedal","year":"1980","journal-title":"Math. Comp."},{"key":"2022042011472942100_ref37","volume-title":"Numerical Optimization","author":"Nocedal","year":"2006"},{"key":"2022042011472942100_ref38","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/s10107-003-0490-7","article-title":"Least Frobenius norm updating of quadratic models that satisfy interpolation conditions","volume":"100","author":"Powell","year":"2004","journal-title":"Math. Program."},{"key":"2022042011472942100_ref39","doi-asserted-by":"crossref","first-page":"1060","DOI":"10.1137\/0716078","article-title":"On the estimation of sparse Hessian matrices","volume":"16","author":"Powell","year":"1979","journal-title":"SIAM J. Numer. Anal."},{"key":"2022042011472942100_ref40","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1137\/0720042","article-title":"The conjugate gradient method and trust regions in large scale optimization","volume":"20","author":"Steihaug","year":"1983","journal-title":"SIAM J. Numer. Anal."},{"key":"2022042011472942100_ref41","volume-title":"Optimization Theory and Methods: Nonlinear Programming","author":"Sun","year":"2006"},{"key":"2022042011472942100_ref42","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1090\/S0025-5718-1977-0455338-4","article-title":"On sparse and symmetric matrix updating subject to a linear equation","volume":"31","author":"Toint","year":"1977","journal-title":"Math. Comp."},{"key":"2022042011472942100_ref43","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1090\/S0025-5718-1978-0483452-7","article-title":"Some numerical results using a sparse matrix updating formula in unconstrained optimization","volume":"32","author":"Toint","year":"1978","journal-title":"Math. Comp."}],"container-title":["IMA Journal of Numerical Analysis"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/imajna\/article-pdf\/42\/2\/1766\/43373810\/drab022.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/imajna\/article-pdf\/42\/2\/1766\/43373810\/drab022.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,20]],"date-time":"2022-04-20T11:49:59Z","timestamp":1650455399000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/imajna\/article\/42\/2\/1766\/6199349"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,6]]},"references-count":43,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,4,6]]},"published-print":{"date-parts":[[2022,4,13]]}},"URL":"https:\/\/doi.org\/10.1093\/imanum\/drab022","relation":{},"ISSN":["0272-4979","1464-3642"],"issn-type":[{"value":"0272-4979","type":"print"},{"value":"1464-3642","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,4]]},"published":{"date-parts":[[2021,4,6]]}}}